MAAS

Merge ~cgrabowski/maas/+git/maas-performance:update_for_latest_maas into ~maas-committers/maas/+git/maas-performance:3.0-rework

  1. Git
  2. lp:~cgrabowski/maas/+git/maas-performance
  3. update_for_latest_maas
  4. Merge into 3.0-rework
Proposed by Christian Grabowski
Status: Merged
Approved by: Christian Grabowski
Approved revision: 41891463862d49c65f3bb40664dc99dcf4228223
Merge reported by: MAAS Lander
Merged at revision: not available
Proposed branch: ~cgrabowski/maas/+git/maas-performance:update_for_latest_maas
Merge into: ~maas-committers/maas/+git/maas-performance:3.0-rework
Diff against target: 3041 lines (+2084/-180)
25 files modified
Makefile (+3/-2)
maasperformance/bmc.py (+1/-2)
maasperformance/commissioning.py (+30/-38)
maasperformance/conftest.py (+2/-1)
maasperformance/data/20-maas-03-machine-resources (+321/-0)
maasperformance/data/20-maas-03-machine-resources.err (+0/-0)
maasperformance/data/20-maas-03-machine-resources.out (+321/-0)
maasperformance/data/40-maas-01-machine-resources (+321/-0)
maasperformance/data/40-maas-01-machine-resources.err (+0/-0)
maasperformance/data/40-maas-01-machine-resources.out (+321/-0)
maasperformance/machine.py (+13/-8)
maasperformance/network.py (+7/-2)
maasperformance/network_interface.py (+5/-3)
maasperformance/process.py (+7/-1)
maasperformance/testing/fixtures.py (+29/-1)
maasperformance/testing/subprocess.py (+14/-0)
maasperformance/tests/test_bmc.py (+50/-0)
maasperformance/tests/test_event.py (+34/-11)
maasperformance/tests/test_machine.py (+223/-33)
maasperformance/tests/test_network.py (+231/-1)
maasperformance/tests/test_network_interface.py (+3/-4)
maasperformance/tests/test_web.py (+118/-51)
maasperformance/web.py (+5/-1)
pyproject.toml (+3/-0)
requirements.txt (+22/-21)
Reviewer Review Type Date Requested Status
MAAS Lander Approve
Alberto Donato (community) Approve
Review via email: mp+429734@code.launchpad.net

Commit message

add update to machine-resources output

remove deprecated loop args

switch to non-deprecated eventloop fixture

patch check_output in tests

enable asyncio in tests

make event tests pass

update dependencies to resolve conflicting versions

To post a comment you must log in.
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b update_for_latest_maas lp:~cgrabowski/maas/+git/maas-performance into -b 3.0-rework lp:~maas-committers/maas/+git/maas-performance

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-performance-tester/7/consoleText
COMMIT: 36d139f78c83b37b09b6525a531d1a2dce781c93

review: Needs Fixing
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b update_for_latest_maas lp:~cgrabowski/maas/+git/maas-performance into -b 3.0-rework lp:~maas-committers/maas/+git/maas-performance

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-performance-tester/8/consoleText
COMMIT: 6235732caa93d5799c94b8dc1852943338eb8f7a

review: Needs Fixing
Revision history for this message
Alberto Donato (ack) wrote :

+1

minor nit inline

review: Approve
Revision history for this message
Christian Grabowski (cgrabowski) :
~cgrabowski/maas/+git/maas-performance:update_for_latest_maas updated
d84b31a... by Christian Grabowski

add pyproject.toml to enable asyncio

b7c2363... by Christian Grabowski

remove unused subprocess import in test_web

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b update_for_latest_maas lp:~cgrabowski/maas/+git/maas-performance into -b 3.0-rework lp:~maas-committers/maas/+git/maas-performance

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-performance-tester/9/consoleText
COMMIT: b7c2363976163bba296e22edf122c5211b2f22e0

review: Needs Fixing
~cgrabowski/maas/+git/maas-performance:update_for_latest_maas updated
82eb637... by Christian Grabowski

correctly format libffi-dev dep

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b update_for_latest_maas lp:~cgrabowski/maas/+git/maas-performance into -b 3.0-rework lp:~maas-committers/maas/+git/maas-performance

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-performance-tester/10/consoleText
COMMIT: 82eb637ed25576695e646b38410c935d5b27e413

review: Needs Fixing
~cgrabowski/maas/+git/maas-performance:update_for_latest_maas updated
c53b9f5... by Christian Grabowski

prevent MachineManager loop from hanging

205f545... by Christian Grabowski

update interface test for new exec__sync_process

d973466... by Christian Grabowski

ensure web tasks close cleanly

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b update_for_latest_maas lp:~cgrabowski/maas/+git/maas-performance into -b 3.0-rework lp:~maas-committers/maas/+git/maas-performance

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-performance-tester/11/consoleText
COMMIT: 263a13b0557c754dfcd211897d276c438874ec60

review: Needs Fixing
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b update_for_latest_maas lp:~cgrabowski/maas/+git/maas-performance into -b 3.0-rework lp:~maas-committers/maas/+git/maas-performance

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-performance-tester/12/consoleText
COMMIT: d97346635c87a6984a2936e783229a58243a0d0b

review: Needs Fixing
~cgrabowski/maas/+git/maas-performance:update_for_latest_maas updated
37c38b2... by Christian Grabowski

cover bmc class in tests

4189146... by Christian Grabowski

add test coverage for network request methods

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b update_for_latest_maas lp:~cgrabowski/maas/+git/maas-performance into -b 3.0-rework lp:~maas-committers/maas/+git/maas-performance

STATUS: SUCCESS
COMMIT: 41891463862d49c65f3bb40664dc99dcf4228223

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/Makefile b/Makefile
2index 7de4112..00962b2 100644
3--- a/Makefile
4+++ b/Makefile
5@@ -1,5 +1,5 @@
6 define DEB_DEPENDENCIES
7-python3-venv
8+python3-venv libffi-dev
9 endef
10
11 APT := DEBIAN_FRONTEND=noninteractive apt
12@@ -60,7 +60,8 @@ deb-dep:
13 # Python targets
14 py-dep: $(VIRTUALENV)
15 $(VIRTUALENV)/bin/pip install -r requirements.txt -e .
16- $(VIRTUALENV)/bin/pip install -r requirements.txt -e .[dev] .[test]
17+ $(VIRTUALENV)/bin/pip install -r requirements.txt -e .[dev]
18+ $(VIRTUALENV)/bin/pip install -r requirements.txt -e .[test]
19 ln -sf $(VIRTUALENV)/bin/pytest
20 ln -sf $(VIRTUALENV)/bin/maas-performanced
21 .PHONY: py-dep
22diff --git a/maasperformance/bmc.py b/maasperformance/bmc.py
23index f09b44a..946c75d 100644
24--- a/maasperformance/bmc.py
25+++ b/maasperformance/bmc.py
26@@ -32,8 +32,7 @@ class BMC:
27 await asyncio.gather(
28 *(
29 self.power_on(machine.uuid)
30- for machine in self.machines_manager.machines.values()),
31- loop=self.loop)
32+ for machine in self.machines_manager.machines.values()))
33
34 async def power_on(self, machine_uuid):
35 print(f'Requesting power on: {machine_uuid}')
36diff --git a/maasperformance/commissioning.py b/maasperformance/commissioning.py
37index d567ca6..592cbb9 100644
38--- a/maasperformance/commissioning.py
39+++ b/maasperformance/commissioning.py
40@@ -7,10 +7,16 @@ changes to remove functionality that wasn't needed.
41
42 import dataclasses
43 from functools import partial
44-from itertools import islice, repeat
45+from itertools import (
46+ islice,
47+ repeat,
48+)
49 import random
50 import string
51-from typing import List, Optional
52+from typing import (
53+ List,
54+ Optional,
55+)
56
57 GB = 1000 * 1000 * 1000
58
59@@ -18,12 +24,10 @@ GB = 1000 * 1000 * 1000
60 class Factory:
61
62 random_letters = map(
63- random.choice, repeat(string.ascii_letters + string.digits)
64- )
65+ random.choice, repeat(string.ascii_letters + string.digits))
66
67 random_letters_with_spaces = map(
68- random.choice, repeat(string.ascii_letters + string.digits + " ")
69- )
70+ random.choice, repeat(string.ascii_letters + string.digits + " "))
71
72 random_octet = partial(random.randint, 0, 255)
73
74@@ -56,8 +60,7 @@ class Factory:
75 def make_string(self, size=10, spaces=False, prefix=""):
76 """Return a `str` filled with random ASCII letters or digits."""
77 source = (
78- self.random_letters_with_spaces if spaces else self.random_letters
79- )
80+ self.random_letters_with_spaces if spaces else self.random_letters)
81 return prefix + "".join(islice(source, size))
82
83
84@@ -127,11 +130,9 @@ class LXDNetworkPort:
85 address: str = dataclasses.field(default_factory=factory.make_mac_address)
86 protocol: str = "ethernet"
87 supported_modes: List[str] = dataclasses.field(
88- default_factory=lambda: ["10000baseT/Full"]
89- )
90+ default_factory=lambda: ["10000baseT/Full"])
91 supported_ports: List[str] = dataclasses.field(
92- default_factory=lambda: ["fibre"]
93- )
94+ default_factory=lambda: ["fibre"])
95 port_type: str = "fibre"
96 transceiver_type: str = "internal"
97 auto_negotiation: bool = True
98@@ -208,25 +209,20 @@ class FakeCommissioningData:
99 def allocate_pci_address(self):
100 prev_address = (
101 self._allocated_pci_addresses[-1]
102- if self._allocated_pci_addresses
103- else "0000:00:00.0"
104- )
105+ if self._allocated_pci_addresses else "0000:00:00.0")
106 bus, device, func = prev_address.split(":")
107 next_device = int(device, 16) + 1
108 self._allocated_pci_addresses.append(
109- f"{bus}:{next_device:0>4x}:{func}"
110- )
111+ f"{bus}:{next_device:0>4x}:{func}")
112 return self._allocated_pci_addresses[-1]
113
114 def get_available_vid(self):
115 available_vids = set(range(2, 4095))
116 used_vids = set(
117 [
118- network.vlan.vid
119- for network in self.networks.values()
120+ network.vlan.vid for network in self.networks.values()
121 if network.vlan is not None
122- ]
123- )
124+ ])
125 available_vids = list(available_vids.difference(used_vids))
126 return random.choice(available_vids)
127
128@@ -249,8 +245,7 @@ class FakeCommissioningData:
129 network = self.create_physical_network_without_nic(name, mac_address)
130 if port is None:
131 port = LXDNetworkPort(
132- network.name, len(card.ports), address=network.hwaddr
133- )
134+ network.name, len(card.ports), address=network.hwaddr)
135 card.ports.append(port)
136 return network
137
138@@ -283,8 +278,7 @@ class FakeCommissioningData:
139 if vid is None:
140 vid = self.get_available_vid()
141 network = LXDNetwork(
142- name, mac_address, vlan=LXDVlan(lower_device=parent.name, vid=vid)
143- )
144+ name, mac_address, vlan=LXDVlan(lower_device=parent.name, vid=vid))
145 self.networks[name] = network
146 return network
147
148@@ -304,8 +298,7 @@ class FakeCommissioningData:
149 name,
150 mac_address,
151 bridge=LXDBridge(
152- upper_devices=[parent.name for parent in parents]
153- ),
154+ upper_devices=[parent.name for parent in parents]),
155 )
156 self.networks[name] = network
157 return network
158@@ -346,8 +339,7 @@ class FakeCommissioningData:
159 del card["ports"]
160 networks = dict(
161 (name, dataclasses.asdict(network))
162- for name, network in self.networks.items()
163- )
164+ for name, network in self.networks.items())
165 data = {
166 "api_extensions": self.api_extensions,
167 "api_version": self.api_version,
168@@ -355,12 +347,10 @@ class FakeCommissioningData:
169 "resources": {
170 "cpu": {
171 "architecture": self.environment["kernel_architecture"],
172- "sockets": [
173- {
174- "socket": 0,
175- "cores": [],
176- }
177- ],
178+ "sockets": [{
179+ "socket": 0,
180+ "cores": [],
181+ }],
182 },
183 "memory": {
184 "hugepages_total": 0,
185@@ -369,7 +359,10 @@ class FakeCommissioningData:
186 "used": int(0.3 * self.memory * 1024 * 1024),
187 "total": int(self.memory * 1024 * 1024),
188 },
189- "gpu": {"cards": [], "total": 0},
190+ "gpu": {
191+ "cards": [],
192+ "total": 0
193+ },
194 "network": network_resources,
195 "storage": storage_resources,
196 },
197@@ -388,6 +381,5 @@ class FakeCommissioningData:
198 },
199 ],
200 "frequency": 1500,
201- }
202- )
203+ })
204 return data
205diff --git a/maasperformance/conftest.py b/maasperformance/conftest.py
206index cf0da2e..99b7085 100644
207--- a/maasperformance/conftest.py
208+++ b/maasperformance/conftest.py
209@@ -2,6 +2,7 @@ from .testing.fixtures import (
210 fs_root,
211 machine,
212 mock_event_time,
213+ tar_file_http_response,
214 )
215
216-__all__ = ['fs_root', 'machine', 'mock_event_time']
217+__all__ = ['fs_root', 'machine', 'mock_event_time', "tar_file_http_response"]
218diff --git a/maasperformance/data/20-maas-03-machine-resources b/maasperformance/data/20-maas-03-machine-resources
219new file mode 100644
220index 0000000..056bd08
221--- /dev/null
222+++ b/maasperformance/data/20-maas-03-machine-resources
223@@ -0,0 +1,321 @@
224+{
225+ "api_extensions": [
226+ "resources",
227+ "resources_cpu_socket",
228+ "resources_gpu",
229+ "resources_numa",
230+ "resources_v2",
231+ "resources_disk_sata",
232+ "resources_network_firmware",
233+ "resources_disk_id",
234+ "resources_usb_pci",
235+ "resources_cpu_threads_numa",
236+ "resources_cpu_core_die",
237+ "api_os",
238+ "resources_system",
239+ "resources_pci_iommu",
240+ "resources_network_usb",
241+ "resources_disk_address"
242+ ],
243+ "api_version": "1.0",
244+ "environment": {
245+ "kernel": "Linux",
246+ "kernel_architecture": "x86_64",
247+ "kernel_version": "5.15.0-43-generic",
248+ "os_name": "ubuntu",
249+ "os_version": "20.04",
250+ "server": "maas-machine-resources",
251+ "server_name": "maas-performance",
252+ "server_version": "5.4"
253+ },
254+ "resources": {
255+ "cpu": {
256+ "architecture": "x86_64",
257+ "sockets": [
258+ {
259+ "name": "exampe-cpu",
260+ "vendor": "test",
261+ "socket": 0,
262+ "cache": [
263+ {
264+ "level": 1,
265+ "type": "Data",
266+ "size": 32768
267+ },
268+ {
269+ "level": 1,
270+ "type": "Instruction",
271+ "size": 32768
272+ },
273+ {
274+ "level": 2,
275+ "type": "Unified",
276+ "size": 524288
277+ },
278+ {
279+ "level": 3,
280+ "type": "Unified",
281+ "size": 16777216
282+ }
283+ ],
284+ "cores": [
285+ {
286+ "core": 0,
287+ "die": 0,
288+ "threads": [
289+ {
290+ "id": 0,
291+ "numa_node": 0,
292+ "thread": 0,
293+ "online": true,
294+ "isolated": false
295+ },
296+ {
297+ "id": 12,
298+ "numa_node": 0,
299+ "thread": 1,
300+ "online": true,
301+ "isolated": false
302+ }
303+ ],
304+ "frequency": 3080
305+ }
306+ ],
307+ "frequency": 2559,
308+ "frequency_minimum": 2200,
309+ "frequency_turbo": 4672
310+ }
311+ ],
312+ "total": 24
313+ },
314+ "memory": {
315+ "nodes": [
316+ {
317+ "numa_node": 0,
318+ "hugepages_used": 0,
319+ "hugepages_total": 0,
320+ "used": 25046433792,
321+ "total": 70866960384
322+ }
323+ ],
324+ "hugepages_total": 0,
325+ "hugepages_used": 0,
326+ "hugepages_size": 2097152,
327+ "used": 9893044224,
328+ "total": 70866960384
329+ },
330+ "gpu": {
331+ "cards": [
332+ {
333+ "driver": "ast",
334+ "driver_version": "5.15.0-43-generic",
335+ "drm": {
336+ "id": 0,
337+ "card_name": "card0",
338+ "card_device": "226:0",
339+ "control_name": "controlD64",
340+ "control_device": "226:0"
341+ },
342+ "numa_node": 0,
343+ "pci_address": "0000:22:00.0",
344+ "vendor": "ASPEED Technology, Inc.",
345+ "vendor_id": "1a03",
346+ "product": "ASPEED Graphics Family",
347+ "product_id": "2000"
348+ }
349+ ],
350+ "total": 1
351+ },
352+ "network": {
353+ "cards": [
354+ {
355+ "driver": "igb",
356+ "driver_version": "5.15.0-43-generic",
357+ "ports": [
358+ {
359+ "id": "enp35s0",
360+ "address": "d0:50:99:dd:49:f1",
361+ "port": 0,
362+ "protocol": "ethernet",
363+ "supported_modes": [
364+ "10baseT/Half",
365+ "10baseT/Full",
366+ "100baseT/Half",
367+ "100baseT/Full",
368+ "1000baseT/Full"
369+ ],
370+ "supported_ports": [
371+ "twisted pair"
372+ ],
373+ "port_type": "twisted pair",
374+ "transceiver_type": "internal",
375+ "auto_negotiation": true,
376+ "link_detected": true,
377+ "link_speed": 1000,
378+ "link_duplex": "full"
379+ }
380+ ],
381+ "numa_node": 0,
382+ "pci_address": "0000:23:00.0",
383+ "vendor": "Intel Corporation",
384+ "vendor_id": "8086",
385+ "product": "I210 Gigabit Network Connection",
386+ "product_id": "1533",
387+ "firmware_version": "3.16, 0x800004d6"
388+ },
389+ ],
390+ "total": 1
391+ },
392+ "storage": {
393+ "disks": [
394+ {
395+ "id": "nvme0n1",
396+ "device": "259:0",
397+ "model": "Samsung SSD 970 EVO 500GB",
398+ "type": "nvme",
399+ "read_only": false,
400+ "size": 500107862016,
401+ "removable": false,
402+ "wwn": "eui.0025385b01440ea7",
403+ "numa_node": 0,
404+ "device_path": "pci-0000:2a:00.0-nvme-1",
405+ "block_size": 512,
406+ "firmware_version": "2B2QEXE7",
407+ "rpm": 0,
408+ "serial": "S5H7NS1NB23880D",
409+ "device_id": "nvme-eui.0025385b01440ea7",
410+ "partitions": [
411+ {
412+ "id": "nvme0n1p1",
413+ "device": "259:1",
414+ "read_only": false,
415+ "size": 536870912,
416+ "partition": 1
417+ },
418+ {
419+ "id": "nvme0n1p2",
420+ "device": "259:2",
421+ "read_only": false,
422+ "size": 1073741824,
423+ "partition": 2
424+ },
425+ {
426+ "id": "nvme0n1p3",
427+ "device": "259:3",
428+ "read_only": false,
429+ "size": 498495127552,
430+ "partition": 3
431+ }
432+ ]
433+ }
434+ ],
435+ "total": 1
436+ },
437+ "usb": {
438+ "devices": [
439+ {
440+ "bus_address": 1,
441+ "device_address": 11,
442+ "interfaces": [
443+ {
444+ "class": "Mass Storage",
445+ "class_id": 8,
446+ "driver": "usb-storage",
447+ "driver_version": "5.15.0-43-generic",
448+ "number": 0,
449+ "subclass": "SCSI",
450+ "subclass_id": 6
451+ }
452+ ],
453+ "vendor": "American Megatrends, Inc.",
454+ "vendor_id": "046b",
455+ "product": "Virtual Cdrom Device",
456+ "product_id": "ff20",
457+ "speed": 480
458+ }
459+ ],
460+ "total": 1
461+ },
462+ "pci": {
463+ "devices": [
464+ {
465+ "driver": "igb",
466+ "driver_version": "5.15.0-43-generic",
467+ "numa_node": 0,
468+ "pci_address": "0000:23:00.0",
469+ "vendor": "Intel Corporation",
470+ "vendor_id": "8086",
471+ "product": "I210 Gigabit Network Connection",
472+ "product_id": "1533",
473+ "iommu_group": 15,
474+ "vpd": {}
475+ },
476+ {
477+ "driver": "nvme",
478+ "driver_version": "1.0",
479+ "numa_node": 0,
480+ "pci_address": "0000:2a:00.0",
481+ "vendor": "Samsung Electronics Co Ltd",
482+ "vendor_id": "144d",
483+ "product": "NVMe SSD Controller SM981/PM981/PM983",
484+ "product_id": "a808",
485+ "iommu_group": 15,
486+ "vpd": {}
487+ }
488+ ],
489+ "total": 2
490+ },
491+ "system": {
492+ "uuid": "00000000-0000-0000-0000-d05099dd49f1",
493+ "vendor": "To Be Filled By O.E.M.",
494+ "product": "To Be Filled By O.E.M.",
495+ "family": "To Be Filled By O.E.M.",
496+ "version": "To Be Filled By O.E.M.",
497+ "sku": "To Be Filled By O.E.M.",
498+ "serial": "To Be Filled By O.E.M.",
499+ "type": "physical",
500+ "firmware": {
501+ "vendor": "American Megatrends Inc.",
502+ "date": "11/02/2020",
503+ "version": "P3.50"
504+ },
505+ "chassis": {
506+ "vendor": "To Be Filled By O.E.M.",
507+ "type": "Unknown",
508+ "serial": "To Be Filled By O.E.M.",
509+ "version": "To Be Filled By O.E.M."
510+ },
511+ "motherboard": {
512+ "vendor": "ASRockRack",
513+ "product": "X470D4U",
514+ "serial": "200101730000493",
515+ "version": ""
516+ }
517+ }
518+ },
519+ "networks": {
520+ "fake0": {
521+ "addresses": [
522+ {
523+ "family": "inet",
524+ "address": "192.168.1.21",
525+ "netmask": "24",
526+ "scope": "global"
527+ }
528+ ],
529+ "counters": {
530+ "bytes_received": 17341736517,
531+ "bytes_sent": 42242825969,
532+ "packets_received": 47014272,
533+ "packets_sent": 64893025
534+ },
535+ "hwaddr": "8e:f4:15:bb:cd:fc",
536+ "mtu": 1500,
537+ "state": "up",
538+ "type": "broadcast",
539+ "bond": null,
540+ "bridge": null,
541+ "vlan": null
542+ }
543+ }
544+}
545diff --git a/maasperformance/data/20-maas-03-machine-resources.err b/maasperformance/data/20-maas-03-machine-resources.err
546new file mode 100644
547index 0000000..e69de29
548--- /dev/null
549+++ b/maasperformance/data/20-maas-03-machine-resources.err
550diff --git a/maasperformance/data/20-maas-03-machine-resources.out b/maasperformance/data/20-maas-03-machine-resources.out
551new file mode 100644
552index 0000000..056bd08
553--- /dev/null
554+++ b/maasperformance/data/20-maas-03-machine-resources.out
555@@ -0,0 +1,321 @@
556+{
557+ "api_extensions": [
558+ "resources",
559+ "resources_cpu_socket",
560+ "resources_gpu",
561+ "resources_numa",
562+ "resources_v2",
563+ "resources_disk_sata",
564+ "resources_network_firmware",
565+ "resources_disk_id",
566+ "resources_usb_pci",
567+ "resources_cpu_threads_numa",
568+ "resources_cpu_core_die",
569+ "api_os",
570+ "resources_system",
571+ "resources_pci_iommu",
572+ "resources_network_usb",
573+ "resources_disk_address"
574+ ],
575+ "api_version": "1.0",
576+ "environment": {
577+ "kernel": "Linux",
578+ "kernel_architecture": "x86_64",
579+ "kernel_version": "5.15.0-43-generic",
580+ "os_name": "ubuntu",
581+ "os_version": "20.04",
582+ "server": "maas-machine-resources",
583+ "server_name": "maas-performance",
584+ "server_version": "5.4"
585+ },
586+ "resources": {
587+ "cpu": {
588+ "architecture": "x86_64",
589+ "sockets": [
590+ {
591+ "name": "exampe-cpu",
592+ "vendor": "test",
593+ "socket": 0,
594+ "cache": [
595+ {
596+ "level": 1,
597+ "type": "Data",
598+ "size": 32768
599+ },
600+ {
601+ "level": 1,
602+ "type": "Instruction",
603+ "size": 32768
604+ },
605+ {
606+ "level": 2,
607+ "type": "Unified",
608+ "size": 524288
609+ },
610+ {
611+ "level": 3,
612+ "type": "Unified",
613+ "size": 16777216
614+ }
615+ ],
616+ "cores": [
617+ {
618+ "core": 0,
619+ "die": 0,
620+ "threads": [
621+ {
622+ "id": 0,
623+ "numa_node": 0,
624+ "thread": 0,
625+ "online": true,
626+ "isolated": false
627+ },
628+ {
629+ "id": 12,
630+ "numa_node": 0,
631+ "thread": 1,
632+ "online": true,
633+ "isolated": false
634+ }
635+ ],
636+ "frequency": 3080
637+ }
638+ ],
639+ "frequency": 2559,
640+ "frequency_minimum": 2200,
641+ "frequency_turbo": 4672
642+ }
643+ ],
644+ "total": 24
645+ },
646+ "memory": {
647+ "nodes": [
648+ {
649+ "numa_node": 0,
650+ "hugepages_used": 0,
651+ "hugepages_total": 0,
652+ "used": 25046433792,
653+ "total": 70866960384
654+ }
655+ ],
656+ "hugepages_total": 0,
657+ "hugepages_used": 0,
658+ "hugepages_size": 2097152,
659+ "used": 9893044224,
660+ "total": 70866960384
661+ },
662+ "gpu": {
663+ "cards": [
664+ {
665+ "driver": "ast",
666+ "driver_version": "5.15.0-43-generic",
667+ "drm": {
668+ "id": 0,
669+ "card_name": "card0",
670+ "card_device": "226:0",
671+ "control_name": "controlD64",
672+ "control_device": "226:0"
673+ },
674+ "numa_node": 0,
675+ "pci_address": "0000:22:00.0",
676+ "vendor": "ASPEED Technology, Inc.",
677+ "vendor_id": "1a03",
678+ "product": "ASPEED Graphics Family",
679+ "product_id": "2000"
680+ }
681+ ],
682+ "total": 1
683+ },
684+ "network": {
685+ "cards": [
686+ {
687+ "driver": "igb",
688+ "driver_version": "5.15.0-43-generic",
689+ "ports": [
690+ {
691+ "id": "enp35s0",
692+ "address": "d0:50:99:dd:49:f1",
693+ "port": 0,
694+ "protocol": "ethernet",
695+ "supported_modes": [
696+ "10baseT/Half",
697+ "10baseT/Full",
698+ "100baseT/Half",
699+ "100baseT/Full",
700+ "1000baseT/Full"
701+ ],
702+ "supported_ports": [
703+ "twisted pair"
704+ ],
705+ "port_type": "twisted pair",
706+ "transceiver_type": "internal",
707+ "auto_negotiation": true,
708+ "link_detected": true,
709+ "link_speed": 1000,
710+ "link_duplex": "full"
711+ }
712+ ],
713+ "numa_node": 0,
714+ "pci_address": "0000:23:00.0",
715+ "vendor": "Intel Corporation",
716+ "vendor_id": "8086",
717+ "product": "I210 Gigabit Network Connection",
718+ "product_id": "1533",
719+ "firmware_version": "3.16, 0x800004d6"
720+ },
721+ ],
722+ "total": 1
723+ },
724+ "storage": {
725+ "disks": [
726+ {
727+ "id": "nvme0n1",
728+ "device": "259:0",
729+ "model": "Samsung SSD 970 EVO 500GB",
730+ "type": "nvme",
731+ "read_only": false,
732+ "size": 500107862016,
733+ "removable": false,
734+ "wwn": "eui.0025385b01440ea7",
735+ "numa_node": 0,
736+ "device_path": "pci-0000:2a:00.0-nvme-1",
737+ "block_size": 512,
738+ "firmware_version": "2B2QEXE7",
739+ "rpm": 0,
740+ "serial": "S5H7NS1NB23880D",
741+ "device_id": "nvme-eui.0025385b01440ea7",
742+ "partitions": [
743+ {
744+ "id": "nvme0n1p1",
745+ "device": "259:1",
746+ "read_only": false,
747+ "size": 536870912,
748+ "partition": 1
749+ },
750+ {
751+ "id": "nvme0n1p2",
752+ "device": "259:2",
753+ "read_only": false,
754+ "size": 1073741824,
755+ "partition": 2
756+ },
757+ {
758+ "id": "nvme0n1p3",
759+ "device": "259:3",
760+ "read_only": false,
761+ "size": 498495127552,
762+ "partition": 3
763+ }
764+ ]
765+ }
766+ ],
767+ "total": 1
768+ },
769+ "usb": {
770+ "devices": [
771+ {
772+ "bus_address": 1,
773+ "device_address": 11,
774+ "interfaces": [
775+ {
776+ "class": "Mass Storage",
777+ "class_id": 8,
778+ "driver": "usb-storage",
779+ "driver_version": "5.15.0-43-generic",
780+ "number": 0,
781+ "subclass": "SCSI",
782+ "subclass_id": 6
783+ }
784+ ],
785+ "vendor": "American Megatrends, Inc.",
786+ "vendor_id": "046b",
787+ "product": "Virtual Cdrom Device",
788+ "product_id": "ff20",
789+ "speed": 480
790+ }
791+ ],
792+ "total": 1
793+ },
794+ "pci": {
795+ "devices": [
796+ {
797+ "driver": "igb",
798+ "driver_version": "5.15.0-43-generic",
799+ "numa_node": 0,
800+ "pci_address": "0000:23:00.0",
801+ "vendor": "Intel Corporation",
802+ "vendor_id": "8086",
803+ "product": "I210 Gigabit Network Connection",
804+ "product_id": "1533",
805+ "iommu_group": 15,
806+ "vpd": {}
807+ },
808+ {
809+ "driver": "nvme",
810+ "driver_version": "1.0",
811+ "numa_node": 0,
812+ "pci_address": "0000:2a:00.0",
813+ "vendor": "Samsung Electronics Co Ltd",
814+ "vendor_id": "144d",
815+ "product": "NVMe SSD Controller SM981/PM981/PM983",
816+ "product_id": "a808",
817+ "iommu_group": 15,
818+ "vpd": {}
819+ }
820+ ],
821+ "total": 2
822+ },
823+ "system": {
824+ "uuid": "00000000-0000-0000-0000-d05099dd49f1",
825+ "vendor": "To Be Filled By O.E.M.",
826+ "product": "To Be Filled By O.E.M.",
827+ "family": "To Be Filled By O.E.M.",
828+ "version": "To Be Filled By O.E.M.",
829+ "sku": "To Be Filled By O.E.M.",
830+ "serial": "To Be Filled By O.E.M.",
831+ "type": "physical",
832+ "firmware": {
833+ "vendor": "American Megatrends Inc.",
834+ "date": "11/02/2020",
835+ "version": "P3.50"
836+ },
837+ "chassis": {
838+ "vendor": "To Be Filled By O.E.M.",
839+ "type": "Unknown",
840+ "serial": "To Be Filled By O.E.M.",
841+ "version": "To Be Filled By O.E.M."
842+ },
843+ "motherboard": {
844+ "vendor": "ASRockRack",
845+ "product": "X470D4U",
846+ "serial": "200101730000493",
847+ "version": ""
848+ }
849+ }
850+ },
851+ "networks": {
852+ "fake0": {
853+ "addresses": [
854+ {
855+ "family": "inet",
856+ "address": "192.168.1.21",
857+ "netmask": "24",
858+ "scope": "global"
859+ }
860+ ],
861+ "counters": {
862+ "bytes_received": 17341736517,
863+ "bytes_sent": 42242825969,
864+ "packets_received": 47014272,
865+ "packets_sent": 64893025
866+ },
867+ "hwaddr": "8e:f4:15:bb:cd:fc",
868+ "mtu": 1500,
869+ "state": "up",
870+ "type": "broadcast",
871+ "bond": null,
872+ "bridge": null,
873+ "vlan": null
874+ }
875+ }
876+}
877diff --git a/maasperformance/data/40-maas-01-machine-resources b/maasperformance/data/40-maas-01-machine-resources
878new file mode 100644
879index 0000000..056bd08
880--- /dev/null
881+++ b/maasperformance/data/40-maas-01-machine-resources
882@@ -0,0 +1,321 @@
883+{
884+ "api_extensions": [
885+ "resources",
886+ "resources_cpu_socket",
887+ "resources_gpu",
888+ "resources_numa",
889+ "resources_v2",
890+ "resources_disk_sata",
891+ "resources_network_firmware",
892+ "resources_disk_id",
893+ "resources_usb_pci",
894+ "resources_cpu_threads_numa",
895+ "resources_cpu_core_die",
896+ "api_os",
897+ "resources_system",
898+ "resources_pci_iommu",
899+ "resources_network_usb",
900+ "resources_disk_address"
901+ ],
902+ "api_version": "1.0",
903+ "environment": {
904+ "kernel": "Linux",
905+ "kernel_architecture": "x86_64",
906+ "kernel_version": "5.15.0-43-generic",
907+ "os_name": "ubuntu",
908+ "os_version": "20.04",
909+ "server": "maas-machine-resources",
910+ "server_name": "maas-performance",
911+ "server_version": "5.4"
912+ },
913+ "resources": {
914+ "cpu": {
915+ "architecture": "x86_64",
916+ "sockets": [
917+ {
918+ "name": "exampe-cpu",
919+ "vendor": "test",
920+ "socket": 0,
921+ "cache": [
922+ {
923+ "level": 1,
924+ "type": "Data",
925+ "size": 32768
926+ },
927+ {
928+ "level": 1,
929+ "type": "Instruction",
930+ "size": 32768
931+ },
932+ {
933+ "level": 2,
934+ "type": "Unified",
935+ "size": 524288
936+ },
937+ {
938+ "level": 3,
939+ "type": "Unified",
940+ "size": 16777216
941+ }
942+ ],
943+ "cores": [
944+ {
945+ "core": 0,
946+ "die": 0,
947+ "threads": [
948+ {
949+ "id": 0,
950+ "numa_node": 0,
951+ "thread": 0,
952+ "online": true,
953+ "isolated": false
954+ },
955+ {
956+ "id": 12,
957+ "numa_node": 0,
958+ "thread": 1,
959+ "online": true,
960+ "isolated": false
961+ }
962+ ],
963+ "frequency": 3080
964+ }
965+ ],
966+ "frequency": 2559,
967+ "frequency_minimum": 2200,
968+ "frequency_turbo": 4672
969+ }
970+ ],
971+ "total": 24
972+ },
973+ "memory": {
974+ "nodes": [
975+ {
976+ "numa_node": 0,
977+ "hugepages_used": 0,
978+ "hugepages_total": 0,
979+ "used": 25046433792,
980+ "total": 70866960384
981+ }
982+ ],
983+ "hugepages_total": 0,
984+ "hugepages_used": 0,
985+ "hugepages_size": 2097152,
986+ "used": 9893044224,
987+ "total": 70866960384
988+ },
989+ "gpu": {
990+ "cards": [
991+ {
992+ "driver": "ast",
993+ "driver_version": "5.15.0-43-generic",
994+ "drm": {
995+ "id": 0,
996+ "card_name": "card0",
997+ "card_device": "226:0",
998+ "control_name": "controlD64",
999+ "control_device": "226:0"
1000+ },
1001+ "numa_node": 0,
1002+ "pci_address": "0000:22:00.0",
1003+ "vendor": "ASPEED Technology, Inc.",
1004+ "vendor_id": "1a03",
1005+ "product": "ASPEED Graphics Family",
1006+ "product_id": "2000"
1007+ }
1008+ ],
1009+ "total": 1
1010+ },
1011+ "network": {
1012+ "cards": [
1013+ {
1014+ "driver": "igb",
1015+ "driver_version": "5.15.0-43-generic",
1016+ "ports": [
1017+ {
1018+ "id": "enp35s0",
1019+ "address": "d0:50:99:dd:49:f1",
1020+ "port": 0,
1021+ "protocol": "ethernet",
1022+ "supported_modes": [
1023+ "10baseT/Half",
1024+ "10baseT/Full",
1025+ "100baseT/Half",
1026+ "100baseT/Full",
1027+ "1000baseT/Full"
1028+ ],
1029+ "supported_ports": [
1030+ "twisted pair"
1031+ ],
1032+ "port_type": "twisted pair",
1033+ "transceiver_type": "internal",
1034+ "auto_negotiation": true,
1035+ "link_detected": true,
1036+ "link_speed": 1000,
1037+ "link_duplex": "full"
1038+ }
1039+ ],
1040+ "numa_node": 0,
1041+ "pci_address": "0000:23:00.0",
1042+ "vendor": "Intel Corporation",
1043+ "vendor_id": "8086",
1044+ "product": "I210 Gigabit Network Connection",
1045+ "product_id": "1533",
1046+ "firmware_version": "3.16, 0x800004d6"
1047+ },
1048+ ],
1049+ "total": 1
1050+ },
1051+ "storage": {
1052+ "disks": [
1053+ {
1054+ "id": "nvme0n1",
1055+ "device": "259:0",
1056+ "model": "Samsung SSD 970 EVO 500GB",
1057+ "type": "nvme",
1058+ "read_only": false,
1059+ "size": 500107862016,
1060+ "removable": false,
1061+ "wwn": "eui.0025385b01440ea7",
1062+ "numa_node": 0,
1063+ "device_path": "pci-0000:2a:00.0-nvme-1",
1064+ "block_size": 512,
1065+ "firmware_version": "2B2QEXE7",
1066+ "rpm": 0,
1067+ "serial": "S5H7NS1NB23880D",
1068+ "device_id": "nvme-eui.0025385b01440ea7",
1069+ "partitions": [
1070+ {
1071+ "id": "nvme0n1p1",
1072+ "device": "259:1",
1073+ "read_only": false,
1074+ "size": 536870912,
1075+ "partition": 1
1076+ },
1077+ {
1078+ "id": "nvme0n1p2",
1079+ "device": "259:2",
1080+ "read_only": false,
1081+ "size": 1073741824,
1082+ "partition": 2
1083+ },
1084+ {
1085+ "id": "nvme0n1p3",
1086+ "device": "259:3",
1087+ "read_only": false,
1088+ "size": 498495127552,
1089+ "partition": 3
1090+ }
1091+ ]
1092+ }
1093+ ],
1094+ "total": 1
1095+ },
1096+ "usb": {
1097+ "devices": [
1098+ {
1099+ "bus_address": 1,
1100+ "device_address": 11,
1101+ "interfaces": [
1102+ {
1103+ "class": "Mass Storage",
1104+ "class_id": 8,
1105+ "driver": "usb-storage",
1106+ "driver_version": "5.15.0-43-generic",
1107+ "number": 0,
1108+ "subclass": "SCSI",
1109+ "subclass_id": 6
1110+ }
1111+ ],
1112+ "vendor": "American Megatrends, Inc.",
1113+ "vendor_id": "046b",
1114+ "product": "Virtual Cdrom Device",
1115+ "product_id": "ff20",
1116+ "speed": 480
1117+ }
1118+ ],
1119+ "total": 1
1120+ },
1121+ "pci": {
1122+ "devices": [
1123+ {
1124+ "driver": "igb",
1125+ "driver_version": "5.15.0-43-generic",
1126+ "numa_node": 0,
1127+ "pci_address": "0000:23:00.0",
1128+ "vendor": "Intel Corporation",
1129+ "vendor_id": "8086",
1130+ "product": "I210 Gigabit Network Connection",
1131+ "product_id": "1533",
1132+ "iommu_group": 15,
1133+ "vpd": {}
1134+ },
1135+ {
1136+ "driver": "nvme",
1137+ "driver_version": "1.0",
1138+ "numa_node": 0,
1139+ "pci_address": "0000:2a:00.0",
1140+ "vendor": "Samsung Electronics Co Ltd",
1141+ "vendor_id": "144d",
1142+ "product": "NVMe SSD Controller SM981/PM981/PM983",
1143+ "product_id": "a808",
1144+ "iommu_group": 15,
1145+ "vpd": {}
1146+ }
1147+ ],
1148+ "total": 2
1149+ },
1150+ "system": {
1151+ "uuid": "00000000-0000-0000-0000-d05099dd49f1",
1152+ "vendor": "To Be Filled By O.E.M.",
1153+ "product": "To Be Filled By O.E.M.",
1154+ "family": "To Be Filled By O.E.M.",
1155+ "version": "To Be Filled By O.E.M.",
1156+ "sku": "To Be Filled By O.E.M.",
1157+ "serial": "To Be Filled By O.E.M.",
1158+ "type": "physical",
1159+ "firmware": {
1160+ "vendor": "American Megatrends Inc.",
1161+ "date": "11/02/2020",
1162+ "version": "P3.50"
1163+ },
1164+ "chassis": {
1165+ "vendor": "To Be Filled By O.E.M.",
1166+ "type": "Unknown",
1167+ "serial": "To Be Filled By O.E.M.",
1168+ "version": "To Be Filled By O.E.M."
1169+ },
1170+ "motherboard": {
1171+ "vendor": "ASRockRack",
1172+ "product": "X470D4U",
1173+ "serial": "200101730000493",
1174+ "version": ""
1175+ }
1176+ }
1177+ },
1178+ "networks": {
1179+ "fake0": {
1180+ "addresses": [
1181+ {
1182+ "family": "inet",
1183+ "address": "192.168.1.21",
1184+ "netmask": "24",
1185+ "scope": "global"
1186+ }
1187+ ],
1188+ "counters": {
1189+ "bytes_received": 17341736517,
1190+ "bytes_sent": 42242825969,
1191+ "packets_received": 47014272,
1192+ "packets_sent": 64893025
1193+ },
1194+ "hwaddr": "8e:f4:15:bb:cd:fc",
1195+ "mtu": 1500,
1196+ "state": "up",
1197+ "type": "broadcast",
1198+ "bond": null,
1199+ "bridge": null,
1200+ "vlan": null
1201+ }
1202+ }
1203+}
1204diff --git a/maasperformance/data/40-maas-01-machine-resources.err b/maasperformance/data/40-maas-01-machine-resources.err
1205new file mode 100644
1206index 0000000..e69de29
1207--- /dev/null
1208+++ b/maasperformance/data/40-maas-01-machine-resources.err
1209diff --git a/maasperformance/data/40-maas-01-machine-resources.out b/maasperformance/data/40-maas-01-machine-resources.out
1210new file mode 100644
1211index 0000000..056bd08
1212--- /dev/null
1213+++ b/maasperformance/data/40-maas-01-machine-resources.out
1214@@ -0,0 +1,321 @@
1215+{
1216+ "api_extensions": [
1217+ "resources",
1218+ "resources_cpu_socket",
1219+ "resources_gpu",
1220+ "resources_numa",
1221+ "resources_v2",
1222+ "resources_disk_sata",
1223+ "resources_network_firmware",
1224+ "resources_disk_id",
1225+ "resources_usb_pci",
1226+ "resources_cpu_threads_numa",
1227+ "resources_cpu_core_die",
1228+ "api_os",
1229+ "resources_system",
1230+ "resources_pci_iommu",
1231+ "resources_network_usb",
1232+ "resources_disk_address"
1233+ ],
1234+ "api_version": "1.0",
1235+ "environment": {
1236+ "kernel": "Linux",
1237+ "kernel_architecture": "x86_64",
1238+ "kernel_version": "5.15.0-43-generic",
1239+ "os_name": "ubuntu",
1240+ "os_version": "20.04",
1241+ "server": "maas-machine-resources",
1242+ "server_name": "maas-performance",
1243+ "server_version": "5.4"
1244+ },
1245+ "resources": {
1246+ "cpu": {
1247+ "architecture": "x86_64",
1248+ "sockets": [
1249+ {
1250+ "name": "exampe-cpu",
1251+ "vendor": "test",
1252+ "socket": 0,
1253+ "cache": [
1254+ {
1255+ "level": 1,
1256+ "type": "Data",
1257+ "size": 32768
1258+ },
1259+ {
1260+ "level": 1,
1261+ "type": "Instruction",
1262+ "size": 32768
1263+ },
1264+ {
1265+ "level": 2,
1266+ "type": "Unified",
1267+ "size": 524288
1268+ },
1269+ {
1270+ "level": 3,
1271+ "type": "Unified",
1272+ "size": 16777216
1273+ }
1274+ ],
1275+ "cores": [
1276+ {
1277+ "core": 0,
1278+ "die": 0,
1279+ "threads": [
1280+ {
1281+ "id": 0,
1282+ "numa_node": 0,
1283+ "thread": 0,
1284+ "online": true,
1285+ "isolated": false
1286+ },
1287+ {
1288+ "id": 12,
1289+ "numa_node": 0,
1290+ "thread": 1,
1291+ "online": true,
1292+ "isolated": false
1293+ }
1294+ ],
1295+ "frequency": 3080
1296+ }
1297+ ],
1298+ "frequency": 2559,
1299+ "frequency_minimum": 2200,
1300+ "frequency_turbo": 4672
1301+ }
1302+ ],
1303+ "total": 24
1304+ },
1305+ "memory": {
1306+ "nodes": [
1307+ {
1308+ "numa_node": 0,
1309+ "hugepages_used": 0,
1310+ "hugepages_total": 0,
1311+ "used": 25046433792,
1312+ "total": 70866960384
1313+ }
1314+ ],
1315+ "hugepages_total": 0,
1316+ "hugepages_used": 0,
1317+ "hugepages_size": 2097152,
1318+ "used": 9893044224,
1319+ "total": 70866960384
1320+ },
1321+ "gpu": {
1322+ "cards": [
1323+ {
1324+ "driver": "ast",
1325+ "driver_version": "5.15.0-43-generic",
1326+ "drm": {
1327+ "id": 0,
1328+ "card_name": "card0",
1329+ "card_device": "226:0",
1330+ "control_name": "controlD64",
1331+ "control_device": "226:0"
1332+ },
1333+ "numa_node": 0,
1334+ "pci_address": "0000:22:00.0",
1335+ "vendor": "ASPEED Technology, Inc.",
1336+ "vendor_id": "1a03",
1337+ "product": "ASPEED Graphics Family",
1338+ "product_id": "2000"
1339+ }
1340+ ],
1341+ "total": 1
1342+ },
1343+ "network": {
1344+ "cards": [
1345+ {
1346+ "driver": "igb",
1347+ "driver_version": "5.15.0-43-generic",
1348+ "ports": [
1349+ {
1350+ "id": "enp35s0",
1351+ "address": "d0:50:99:dd:49:f1",
1352+ "port": 0,
1353+ "protocol": "ethernet",
1354+ "supported_modes": [
1355+ "10baseT/Half",
1356+ "10baseT/Full",
1357+ "100baseT/Half",
1358+ "100baseT/Full",
1359+ "1000baseT/Full"
1360+ ],
1361+ "supported_ports": [
1362+ "twisted pair"
1363+ ],
1364+ "port_type": "twisted pair",
1365+ "transceiver_type": "internal",
1366+ "auto_negotiation": true,
1367+ "link_detected": true,
1368+ "link_speed": 1000,
1369+ "link_duplex": "full"
1370+ }
1371+ ],
1372+ "numa_node": 0,
1373+ "pci_address": "0000:23:00.0",
1374+ "vendor": "Intel Corporation",
1375+ "vendor_id": "8086",
1376+ "product": "I210 Gigabit Network Connection",
1377+ "product_id": "1533",
1378+ "firmware_version": "3.16, 0x800004d6"
1379+ },
1380+ ],
1381+ "total": 1
1382+ },
1383+ "storage": {
1384+ "disks": [
1385+ {
1386+ "id": "nvme0n1",
1387+ "device": "259:0",
1388+ "model": "Samsung SSD 970 EVO 500GB",
1389+ "type": "nvme",
1390+ "read_only": false,
1391+ "size": 500107862016,
1392+ "removable": false,
1393+ "wwn": "eui.0025385b01440ea7",
1394+ "numa_node": 0,
1395+ "device_path": "pci-0000:2a:00.0-nvme-1",
1396+ "block_size": 512,
1397+ "firmware_version": "2B2QEXE7",
1398+ "rpm": 0,
1399+ "serial": "S5H7NS1NB23880D",
1400+ "device_id": "nvme-eui.0025385b01440ea7",
1401+ "partitions": [
1402+ {
1403+ "id": "nvme0n1p1",
1404+ "device": "259:1",
1405+ "read_only": false,
1406+ "size": 536870912,
1407+ "partition": 1
1408+ },
1409+ {
1410+ "id": "nvme0n1p2",
1411+ "device": "259:2",
1412+ "read_only": false,
1413+ "size": 1073741824,
1414+ "partition": 2
1415+ },
1416+ {
1417+ "id": "nvme0n1p3",
1418+ "device": "259:3",
1419+ "read_only": false,
1420+ "size": 498495127552,
1421+ "partition": 3
1422+ }
1423+ ]
1424+ }
1425+ ],
1426+ "total": 1
1427+ },
1428+ "usb": {
1429+ "devices": [
1430+ {
1431+ "bus_address": 1,
1432+ "device_address": 11,
1433+ "interfaces": [
1434+ {
1435+ "class": "Mass Storage",
1436+ "class_id": 8,
1437+ "driver": "usb-storage",
1438+ "driver_version": "5.15.0-43-generic",
1439+ "number": 0,
1440+ "subclass": "SCSI",
1441+ "subclass_id": 6
1442+ }
1443+ ],
1444+ "vendor": "American Megatrends, Inc.",
1445+ "vendor_id": "046b",
1446+ "product": "Virtual Cdrom Device",
1447+ "product_id": "ff20",
1448+ "speed": 480
1449+ }
1450+ ],
1451+ "total": 1
1452+ },
1453+ "pci": {
1454+ "devices": [
1455+ {
1456+ "driver": "igb",
1457+ "driver_version": "5.15.0-43-generic",
1458+ "numa_node": 0,
1459+ "pci_address": "0000:23:00.0",
1460+ "vendor": "Intel Corporation",
1461+ "vendor_id": "8086",
1462+ "product": "I210 Gigabit Network Connection",
1463+ "product_id": "1533",
1464+ "iommu_group": 15,
1465+ "vpd": {}
1466+ },
1467+ {
1468+ "driver": "nvme",
1469+ "driver_version": "1.0",
1470+ "numa_node": 0,
1471+ "pci_address": "0000:2a:00.0",
1472+ "vendor": "Samsung Electronics Co Ltd",
1473+ "vendor_id": "144d",
1474+ "product": "NVMe SSD Controller SM981/PM981/PM983",
1475+ "product_id": "a808",
1476+ "iommu_group": 15,
1477+ "vpd": {}
1478+ }
1479+ ],
1480+ "total": 2
1481+ },
1482+ "system": {
1483+ "uuid": "00000000-0000-0000-0000-d05099dd49f1",
1484+ "vendor": "To Be Filled By O.E.M.",
1485+ "product": "To Be Filled By O.E.M.",
1486+ "family": "To Be Filled By O.E.M.",
1487+ "version": "To Be Filled By O.E.M.",
1488+ "sku": "To Be Filled By O.E.M.",
1489+ "serial": "To Be Filled By O.E.M.",
1490+ "type": "physical",
1491+ "firmware": {
1492+ "vendor": "American Megatrends Inc.",
1493+ "date": "11/02/2020",
1494+ "version": "P3.50"
1495+ },
1496+ "chassis": {
1497+ "vendor": "To Be Filled By O.E.M.",
1498+ "type": "Unknown",
1499+ "serial": "To Be Filled By O.E.M.",
1500+ "version": "To Be Filled By O.E.M."
1501+ },
1502+ "motherboard": {
1503+ "vendor": "ASRockRack",
1504+ "product": "X470D4U",
1505+ "serial": "200101730000493",
1506+ "version": ""
1507+ }
1508+ }
1509+ },
1510+ "networks": {
1511+ "fake0": {
1512+ "addresses": [
1513+ {
1514+ "family": "inet",
1515+ "address": "192.168.1.21",
1516+ "netmask": "24",
1517+ "scope": "global"
1518+ }
1519+ ],
1520+ "counters": {
1521+ "bytes_received": 17341736517,
1522+ "bytes_sent": 42242825969,
1523+ "packets_received": 47014272,
1524+ "packets_sent": 64893025
1525+ },
1526+ "hwaddr": "8e:f4:15:bb:cd:fc",
1527+ "mtu": 1500,
1528+ "state": "up",
1529+ "type": "broadcast",
1530+ "bond": null,
1531+ "bridge": null,
1532+ "vlan": null
1533+ }
1534+ }
1535+}
1536diff --git a/maasperformance/machine.py b/maasperformance/machine.py
1537index e5725b0..ed55471 100644
1538--- a/maasperformance/machine.py
1539+++ b/maasperformance/machine.py
1540@@ -93,6 +93,7 @@ class MachineManager:
1541 _run() method of this class.
1542 """
1543 self.machines = {}
1544+ self.running = True
1545 for index in range(self.number_of_machines):
1546 machine = Machine(loop, self.parent_iface, f'fake{index}')
1547 self.machines[machine.uuid] = machine
1548@@ -130,7 +131,7 @@ class MachineManager:
1549 logging.info(f'Created PXE interfaces in {duration} seconds')
1550 start = now()
1551
1552- while True:
1553+ while self.running and self.loop.is_running():
1554 power_command = await self.power_commands.get()
1555 print(
1556 f'Got power command {power_command.new_power_state} '
1557@@ -233,9 +234,12 @@ class Machine:
1558 That is, the clean up method that was last added is the first
1559 one to be executed.
1560 """
1561- while len(self.cleanups) > 0:
1562- func = self.cleanups.pop()
1563- await func()
1564+ try:
1565+ while len(self.cleanups) > 0:
1566+ func = self.cleanups.pop()
1567+ await func()
1568+ finally:
1569+ self.running = False
1570
1571 async def create_pxe_interface(self):
1572 """Create the machine's network interface on the host.
1573@@ -264,7 +268,7 @@ class Machine:
1574 print(f'Power on: {self.uuid}')
1575 # Simulate how it takes a while before the power command is
1576 # issued, and when the machine is actually considered on.
1577- await asyncio.sleep(1, loop=self.loop)
1578+ await asyncio.sleep(1)
1579 self.power_state = PowerState.ON
1580 await self.pxe_boot()
1581
1582@@ -274,7 +278,7 @@ class Machine:
1583 # Simulate how it takes a while before the power command is
1584 # issued, and when the machine is actually considered off.
1585 print(f'Power off: {self.uuid}')
1586- await asyncio.sleep(1, loop=self.loop)
1587+ await asyncio.sleep(1)
1588 await self.release_ip()
1589 self.reset()
1590 self.power_state = PowerState.OFF
1591@@ -358,7 +362,7 @@ class Machine:
1592 else:
1593 raise RuntimeError("Couldn't retrieve pxelinux.cfg")
1594
1595- kernel_url, initrd_url = None, None
1596+ kernel_url, initrd_url, image_url = None, None, None
1597 for line in pxe_config.splitlines():
1598 line = line.strip()
1599 try:
1600@@ -371,7 +375,8 @@ class Machine:
1601 initrd_url = value
1602 if key == 'APPEND':
1603 image_url, cloud_config_url = self._extract_append_urls(value)
1604- if kernel_url is not None and initrd_url is not None:
1605+ if (kernel_url is not None and initrd_url is not None
1606+ and image_url is not None):
1607 await self.network.get_curl_file(kernel_url)
1608 await self.network.get_curl_file(initrd_url)
1609 await self.network.get_curl_file(image_url)
1610diff --git a/maasperformance/network.py b/maasperformance/network.py
1611index abaeba7..c8fef7e 100644
1612--- a/maasperformance/network.py
1613+++ b/maasperformance/network.py
1614@@ -102,8 +102,13 @@ class PlaintextSignature(Signature):
1615
1616 name = 'PLAINTEXT'
1617
1618- def sign(self, consumer_secret, method, url, oauth_token_secret=None,
1619- **params):
1620+ def sign(
1621+ self,
1622+ consumer_secret,
1623+ method,
1624+ url,
1625+ oauth_token_secret=None,
1626+ **params):
1627 """Create a signature using PLAINTEXT."""
1628 key = self._escape(consumer_secret) + '&'
1629 if oauth_token_secret:
1630diff --git a/maasperformance/network_interface.py b/maasperformance/network_interface.py
1631index 7f13822..836342c 100644
1632--- a/maasperformance/network_interface.py
1633+++ b/maasperformance/network_interface.py
1634@@ -1,14 +1,16 @@
1635 import json
1636 import random
1637-from subprocess import check_output
1638 from typing import NamedTuple
1639
1640-from .process import exec_process
1641+from .process import (
1642+ exec_process,
1643+ exec_sync_process,
1644+)
1645
1646
1647 def interface_is_bridge(iface: str):
1648 """Return whether an interface is a bridge."""
1649- output = check_output(['ip', '--json', '-d', 'link', 'show', iface])
1650+ output = exec_sync_process('ip', '--json', '-d', 'link', 'show', iface)
1651 data = json.loads(output)
1652 return data[0].get('linkinfo', {}).get('info_kind') == 'bridge'
1653
1654diff --git a/maasperformance/process.py b/maasperformance/process.py
1655index ae7e4ca..febc663 100644
1656--- a/maasperformance/process.py
1657+++ b/maasperformance/process.py
1658@@ -1,4 +1,5 @@
1659 import asyncio
1660+from subprocess import check_output
1661
1662
1663 class ProcessError(Exception):
1664@@ -13,10 +14,15 @@ class ProcessError(Exception):
1665
1666
1667 async def exec_process(*args: str) -> str:
1668- """Execute a process a process, returning its stdout."""
1669+ """Execute a process asynchronously, returning its stdout."""
1670 process = await asyncio.create_subprocess_exec(
1671 *args, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE)
1672 stdout, stderr = await process.communicate()
1673 if process.returncode != 0:
1674 raise ProcessError(args, process.returncode, stderr.decode())
1675 return stdout.decode()
1676+
1677+
1678+def exec_sync_process(*args: str) -> str:
1679+ """Execute a process synchronously, returning its stdout."""
1680+ return check_output(args)
1681diff --git a/maasperformance/testing/fixtures.py b/maasperformance/testing/fixtures.py
1682index f85f311..9b52067 100644
1683--- a/maasperformance/testing/fixtures.py
1684+++ b/maasperformance/testing/fixtures.py
1685@@ -1,12 +1,18 @@
1686 import asyncio
1687+from io import BytesIO
1688 from pathlib import Path
1689+import subprocess
1690+import tarfile
1691
1692 import pytest
1693
1694 from .. import event
1695 from ..machine import Machine
1696 from ..network_interface import ParentNetworkInterface
1697-from .subprocess import FakeCreateSubProcess
1698+from .subprocess import (
1699+ FakeCheckOutput,
1700+ FakeCreateSubProcess,
1701+)
1702
1703
1704 @pytest.fixture
1705@@ -38,3 +44,25 @@ def create_subprocess_mock(module=asyncio):
1706 module, 'create_subprocess_exec', new=FakeCreateSubProcess())
1707
1708 return subprocess_mock
1709+
1710+
1711+def create_check_output_mock(module=subprocess, output=""):
1712+ """Return a fixture for mocking check_output."""
1713+
1714+ @pytest.fixture
1715+ def check_output_mock(mocker, tmpdir):
1716+ yield mocker.patch.object(
1717+ module, 'check_output', new=FakeCheckOutput(output))
1718+
1719+ return check_output_mock
1720+
1721+
1722+@pytest.fixture
1723+def tar_file_http_response():
1724+ buf = BytesIO()
1725+ with tarfile.open(fileobj=buf, mode="w") as tar:
1726+ data = "{\"foo\": \"bar\"}"
1727+ info = tarfile.TarInfo("index.json")
1728+ info.size = len(data)
1729+ tar.addfile(info, BytesIO(initial_bytes=data.encode("ascii")))
1730+ return buf.getvalue().decode("ascii")
1731diff --git a/maasperformance/testing/subprocess.py b/maasperformance/testing/subprocess.py
1732index 557b048..d2634c4 100644
1733--- a/maasperformance/testing/subprocess.py
1734+++ b/maasperformance/testing/subprocess.py
1735@@ -191,6 +191,10 @@ class FakeCurlSubprocess(FakeAsyncSubprocess):
1736 def root_url(self):
1737 return f'{self.root_protocol}://{self.ip_address}{self.root_path}'
1738
1739+ @property
1740+ def image_url(self):
1741+ return self.root_url
1742+
1743 def get_pxelinux_cfg(self):
1744 """Return the pxelinux.cfg contents.
1745
1746@@ -248,4 +252,14 @@ class FakeCurlSubprocess(FakeAsyncSubprocess):
1747 }
1748 files[self.kernel_url] = ''
1749 files[self.initrd_url] = ''
1750+ files[self.image_url] = ''
1751 return files
1752+
1753+
1754+class FakeCheckOutput:
1755+
1756+ def __init__(self, output):
1757+ self._output = output
1758+
1759+ def __call__(self, *args, **kwargs):
1760+ return self._output
1761diff --git a/maasperformance/tests/test_bmc.py b/maasperformance/tests/test_bmc.py
1762new file mode 100644
1763index 0000000..97be801
1764--- /dev/null
1765+++ b/maasperformance/tests/test_bmc.py
1766@@ -0,0 +1,50 @@
1767+import asyncio
1768+import random
1769+from unittest.mock import Mock
1770+
1771+import pytest
1772+
1773+from ..bmc import (
1774+ BMC,
1775+ PowerCommand,
1776+ PowerState,
1777+)
1778+
1779+
1780+class TestBMC:
1781+
1782+ @pytest.mark.asyncio
1783+ async def test_bmc_power_on(self):
1784+ manager = Mock()
1785+ manager.power_commands = asyncio.Queue()
1786+ bmc = BMC(manager)
1787+ uuid = random.randint(1, 512)
1788+ await bmc.power_on(uuid)
1789+ result = await manager.power_commands.get()
1790+ assert isinstance(result, PowerCommand)
1791+ assert result.machine_uuid == uuid
1792+ assert result.new_power_state == PowerState.ON
1793+
1794+ @pytest.mark.asyncio
1795+ async def test_bmc_power_off(self):
1796+ manager = Mock()
1797+ manager.power_commands = asyncio.Queue()
1798+ bmc = BMC(manager)
1799+ uuid = random.randint(1, 512)
1800+ await bmc.power_off(uuid)
1801+ result = await manager.power_commands.get()
1802+ assert isinstance(result, PowerCommand)
1803+ assert result.machine_uuid == uuid
1804+ assert result.new_power_state == PowerState.OFF
1805+
1806+ @pytest.mark.asyncio
1807+ async def test_bmc_power_cycle(self):
1808+ manager = Mock()
1809+ manager.power_commands = asyncio.Queue()
1810+ bmc = BMC(manager)
1811+ uuid = random.randint(1, 512)
1812+ await bmc.power_cycle(uuid)
1813+ result = await manager.power_commands.get()
1814+ assert isinstance(result, PowerCommand)
1815+ assert result.machine_uuid == uuid
1816+ assert result.new_power_state == PowerState.CYCLE
1817diff --git a/maasperformance/tests/test_event.py b/maasperformance/tests/test_event.py
1818index 3e826f3..1d800f5 100644
1819--- a/maasperformance/tests/test_event.py
1820+++ b/maasperformance/tests/test_event.py
1821@@ -1,4 +1,5 @@
1822 import logging
1823+from unittest.mock import Mock
1824 from urllib.parse import urlparse
1825
1826 import pytest
1827@@ -8,11 +9,25 @@ from ..machine import Machine
1828 from ..testing.prometheus import track_metric
1829
1830
1831+def create_mock_parent_interface():
1832+ parent_iface = Mock()
1833+
1834+ def _get_client_interface(x):
1835+ iface = Mock()
1836+ iface.name = x
1837+ iface.__str__ = lambda _: iface.name
1838+ return iface
1839+
1840+ parent_iface.get_client_interface = _get_client_interface
1841+ return parent_iface
1842+
1843+
1844 class TestDHCPEvents:
1845
1846 @pytest.mark.parametrize('event_type', ['request', 'release'])
1847 def test_dhcp_request_started(self, event_type, mock_event_time, caplog):
1848- machine = Machine(None, None, 'eth0')
1849+ parent_iface = create_mock_parent_interface()
1850+ machine = Machine(None, parent_iface, 'eth0')
1851 mock_event_time.return_value = 12345
1852 event_func = getattr(event, f'dhcp_{event_type}_started')
1853 with caplog.at_level(logging.INFO):
1854@@ -24,7 +39,8 @@ class TestDHCPEvents:
1855
1856 @pytest.mark.parametrize('event_type', ['request', 'release'])
1857 def test_dhcp_request_ended(self, event_type, mock_event_time, caplog):
1858- machine = Machine(None, None, 'eth0')
1859+ parent_iface = create_mock_parent_interface()
1860+ machine = Machine(None, parent_iface, 'eth0')
1861 machine.last_event_time[f'dhcp_{event_type}_start'] = 12345
1862 mock_event_time.return_value = 12347.5
1863 event_func = getattr(event, f'dhcp_{event_type}_ended')
1864@@ -38,7 +54,8 @@ class TestDHCPEvents:
1865 @pytest.mark.parametrize('event_type', ['request', 'release'])
1866 def test_dhcp_request_ended_prometheus_count(
1867 self, event_type, mock_event_time, caplog):
1868- machine = Machine(None, None, 'eth0')
1869+ parent_iface = create_mock_parent_interface()
1870+ machine = Machine(None, parent_iface, 'eth0')
1871 machine.last_event_time[f'dhcp_{event_type}_start'] = 12345
1872 mock_event_time.return_value = 12347.5
1873 event_func = getattr(event, f'dhcp_{event_type}_ended')
1874@@ -50,7 +67,8 @@ class TestDHCPEvents:
1875 @pytest.mark.parametrize('event_type', ['request', 'release'])
1876 def test_dhcp_request_ended_prometheus_bucket_low(
1877 self, event_type, mock_event_time, caplog):
1878- machine = Machine(None, None, 'eth0')
1879+ parent_iface = create_mock_parent_interface()
1880+ machine = Machine(None, parent_iface, 'eth0')
1881 machine.last_event_time[f'dhcp_{event_type}_start'] = 12345
1882 mock_event_time.return_value = 12347.5
1883 event_func = getattr(event, f'dhcp_{event_type}_ended')
1884@@ -62,7 +80,8 @@ class TestDHCPEvents:
1885 @pytest.mark.parametrize('event_type', ['request', 'release'])
1886 def test_dhcp_request_ended_prometheus_bucket_high(
1887 self, event_type, mock_event_time, caplog):
1888- machine = Machine(None, None, 'eth0')
1889+ parent_iface = create_mock_parent_interface()
1890+ machine = Machine(None, parent_iface, 'eth0')
1891 machine.last_event_time[f'dhcp_{event_type}_start'] = 12345
1892 mock_event_time.return_value = 12347.5
1893 event_func = getattr(event, f'dhcp_{event_type}_ended')
1894@@ -75,7 +94,8 @@ class TestDHCPEvents:
1895 class TestFileTransferEvents:
1896
1897 def test_file_transfer_started(self, mock_event_time, caplog):
1898- machine = Machine(None, None, 'eth0')
1899+ parent_iface = create_mock_parent_interface()
1900+ machine = Machine(None, parent_iface, 'eth0')
1901 mock_event_time.return_value = 12345
1902 url = urlparse('tftp://some-host/my-file')
1903 with caplog.at_level(logging.INFO):
1904@@ -89,7 +109,8 @@ class TestFileTransferEvents:
1905
1906 def test_file_transfer_ended(self, mock_event_time, caplog):
1907 url = urlparse('tftp://some-host/my-file')
1908- machine = Machine(None, None, 'eth0')
1909+ parent_iface = create_mock_parent_interface()
1910+ machine = Machine(None, parent_iface, 'eth0')
1911 machine.last_event_time[f'file_transfer_start_{url.geturl()}'] = 12345
1912 mock_event_time.return_value = 12347.5
1913 with caplog.at_level(logging.INFO):
1914@@ -105,7 +126,8 @@ class TestFileTransferEvents:
1915 def test_file_transfer_ended_prometheus_count(
1916 self, mock_event_time, caplog):
1917 url = urlparse('tftp://some-host/my-file')
1918- machine = Machine(None, None, 'eth0')
1919+ parent_iface = create_mock_parent_interface()
1920+ machine = Machine(None, parent_iface, 'eth0')
1921 machine.last_event_time[f'file_transfer_start_{url.geturl()}'] = 12345
1922 mock_event_time.return_value = 12347.5
1923 labels = {'scheme': 'tftp', 'filename': '/my-file', 'result': '68'}
1924@@ -117,8 +139,8 @@ class TestFileTransferEvents:
1925 def test_file_transfer_ended_prometheus_bucket_low(
1926 self, mock_event_time, caplog):
1927 url = urlparse('tftp://some-host/my-file')
1928- machine = Machine(None, None, 'eth0')
1929- machine = Machine(None, None, 'eth0')
1930+ parent_iface = create_mock_parent_interface()
1931+ machine = Machine(None, parent_iface, 'eth0')
1932 machine.last_event_time[f'file_transfer_start_{url.geturl()}'] = 12345
1933 mock_event_time.return_value = 12347.5
1934 labels = {
1935@@ -135,7 +157,8 @@ class TestFileTransferEvents:
1936 def test_file_transfer_ended_prometheus_bucket_high(
1937 self, mock_event_time, caplog):
1938 url = urlparse('tftp://some-host/my-file')
1939- machine = Machine(None, None, 'eth0')
1940+ parent_iface = create_mock_parent_interface()
1941+ machine = Machine(None, parent_iface, 'eth0')
1942 machine.last_event_time[f'file_transfer_start_{url.geturl()}'] = 12345
1943 mock_event_time.return_value = 12347.5
1944 labels = {
1945diff --git a/maasperformance/tests/test_machine.py b/maasperformance/tests/test_machine.py
1946index c5d1cbe..a0e2e69 100644
1947--- a/maasperformance/tests/test_machine.py
1948+++ b/maasperformance/tests/test_machine.py
1949@@ -1,5 +1,7 @@
1950 import asyncio
1951 import logging
1952+import random
1953+from unittest.mock import Mock
1954 import uuid
1955
1956 import pytest
1957@@ -7,17 +9,55 @@ import pytest
1958 from .. import (
1959 event,
1960 machine as machine_module,
1961+ process,
1962+)
1963+from ..bmc import (
1964+ BMC,
1965+ PowerState,
1966 )
1967-from ..bmc import PowerState
1968 from ..machine import (
1969 Machine,
1970 MachineManager,
1971 )
1972 from ..network_interface import ParentNetworkInterface
1973-from ..testing.fixtures import create_subprocess_mock
1974+from ..testing.fixtures import (
1975+ create_check_output_mock,
1976+ create_subprocess_mock,
1977+)
1978 from ..testing.prometheus import track_metric
1979
1980 subprocess_mock = create_subprocess_mock()
1981+check_output_mock = create_check_output_mock(
1982+ module=process,
1983+ output="""
1984+[
1985+ {
1986+ "ifindex":1,
1987+ "ifname":"eth0",
1988+ "flags":["BROADCAST","MULTICAST","UP","LOWER_UP"],
1989+ "mtu":1500,
1990+ "qdisc":"mq",
1991+ "operstate":"UP",
1992+ "linkmode":"DEFAULT",
1993+ "group":"default",
1994+ "txqlen":1000,
1995+ "link_type":"ether",
1996+ "address":"80:61:5f:08:fc:16",
1997+ "broadcast":"ff:ff:ff:ff:ff:ff",
1998+ "promiscuity":0,
1999+ "min_mtu":68,
2000+ "max_mtu":9710,
2001+ "inet6_addr_gen_mode":"none",
2002+ "num_tx_queues":64,
2003+ "num_rx_queues":64,
2004+ "gso_max_size":65536,
2005+ "gso_max_segs":65535,
2006+ "parentbus":"pci",
2007+ "parentdev":"0000:01:00.0",
2008+ "vfinfo_list":[]
2009+ }
2010+]
2011+""")
2012
2013
2014 class TestMachine:
2015@@ -60,6 +100,7 @@ class TestMachine:
2016 'bridge')
2017 ]
2018
2019+ @pytest.mark.asyncio
2020 async def test_create_pxe_interface_registers_cleanup(
2021 self, machine, subprocess_mock):
2022 await machine.create_pxe_interface()
2023@@ -151,14 +192,35 @@ class TestMachine:
2024 assert machine.last_event_time['dhcp_release_end'] == 67890
2025
2026 @pytest.mark.asyncio
2027- async def test_get_pxe_files_binaries(self, machine, subprocess_mock):
2028+ async def test_get_pxe_files_binaries(
2029+ self, machine, subprocess_mock, mocker):
2030 fake_curl = subprocess_mock.fake_processes['curl']
2031 fake_dhclient = subprocess_mock.fake_processes['dhclient']
2032 fake_dhclient.next_server = '4.3.2.1'
2033 fake_curl.ip_address = '4.3.2.1'
2034+
2035+ mocker.patch.object(machine, 'process_preseed')
2036+ mocker.patch.object(machine, '_sanitize_url')
2037+
2038+ class MockGetHTTPFile:
2039+
2040+ async def __call__(self, *args, **kwargs):
2041+ return 200, "{}"
2042+
2043+ mocker.patch.object(
2044+ machine.network, 'get_http_file', new=MockGetHTTPFile())
2045+
2046+ machine.metadata = Mock()
2047+
2048+ async def _noop_request(*args, **kwargs):
2049+ return "{\"metadata_url\": \"http://4.3.2.1/metadata\"}"
2050+
2051+ machine.metadata.request = _noop_request
2052+
2053 await machine.create_pxe_interface()
2054 await machine.get_ip()
2055 await machine.get_pxe_files()
2056+
2057 assert fake_curl.requested_urls[:2] == [
2058 ('tftp://4.3.2.1/lpxelinux.0', 0),
2059 ('tftp://4.3.2.1/ldlinux.c32', 0),
2060@@ -171,17 +233,35 @@ class TestMachine:
2061 fake_dhclient = subprocess_mock.fake_processes['dhclient']
2062 fake_dhclient.next_server = '4.3.2.1'
2063 fake_curl.ip_address = '4.3.2.1'
2064- machine_uuid = uuid.UUID('816fd0e0-0d52-cb11-877b-9c25109f8fba')
2065- uuid_mock = mocker.patch.object(uuid, 'uuid1')
2066- uuid_mock.side_effect = [machine_uuid]
2067+ machine.uuid = uuid.UUID('816fd0e0-0d52-cb11-877b-9c25109f8fba')
2068+
2069+ mocker.patch.object(machine, 'process_preseed')
2070+ mocker.patch.object(machine, '_sanitize_url')
2071+
2072+ class MockGetHTTPFile:
2073+
2074+ async def __call__(self, *args, **kwargs):
2075+ return 200, "{}"
2076+
2077+ mocker.patch.object(
2078+ machine.network, 'get_http_file', new=MockGetHTTPFile())
2079+
2080+ machine.metadata = Mock()
2081+
2082+ async def _noop_request(*args, **kwargs):
2083+ return "{\"metadata_url\": \"http://4.3.2.1/metadata\"}"
2084+
2085+ machine.metadata.request = _noop_request
2086+
2087 await machine.create_pxe_interface()
2088 await machine.get_ip()
2089 await machine.get_pxe_files()
2090- assert fake_curl.requested_urls[2:13] == [
2091+ mac_route = machine.client_iface.mac_address.replace(':', '-')
2092+ expected_calls = [
2093 (
2094 'tftp://4.3.2.1/pxelinux.cfg/'
2095 '816fd0e0-0d52-cb11-877b-9c25109f8fba', 68),
2096- ('tftp://4.3.2.1/pxelinux.cfg/01-11-22-33-44-55-66', 68),
2097+ (f"tftp://4.3.2.1/pxelinux.cfg/01-{mac_route}", 68),
2098 ('tftp://4.3.2.1/pxelinux.cfg/02020202', 68),
2099 ('tftp://4.3.2.1/pxelinux.cfg/0202020', 68),
2100 ('tftp://4.3.2.1/pxelinux.cfg/020202', 68),
2101@@ -192,10 +272,30 @@ class TestMachine:
2102 ('tftp://4.3.2.1/pxelinux.cfg/0', 68),
2103 ('tftp://4.3.2.1/pxelinux.cfg/default', 0),
2104 ]
2105+ for expected_call in expected_calls:
2106+ assert expected_call in fake_curl.requested_urls[2:13]
2107
2108 @pytest.mark.asyncio
2109 async def test_get_pxe_files_event_pxelinux_success(
2110- self, machine, subprocess_mock, event_loop):
2111+ self, machine, subprocess_mock, event_loop, mocker):
2112+ mocker.patch.object(machine, 'process_preseed')
2113+ mocker.patch.object(machine, '_sanitize_url')
2114+
2115+ class MockGetHTTPFile:
2116+
2117+ async def __call__(self, *args, **kwargs):
2118+ return 200, "{}"
2119+
2120+ mocker.patch.object(
2121+ machine.network, 'get_http_file', new=MockGetHTTPFile())
2122+
2123+ machine.metadata = Mock()
2124+
2125+ async def _noop_request(*args, **kwargs):
2126+ return "{\"metadata_url\": \"http://4.3.2.1/metadata\"}"
2127+
2128+ machine.metadata.request = _noop_request
2129+
2130 await machine.create_pxe_interface()
2131 await machine.get_ip()
2132 labels = {'scheme': 'tftp', 'filename': 'pxelinux.cfg', 'result': '0'}
2133@@ -206,7 +306,25 @@ class TestMachine:
2134
2135 @pytest.mark.asyncio
2136 async def test_get_pxe_files_event_pxelinux_failures(
2137- self, machine, subprocess_mock, event_loop):
2138+ self, machine, subprocess_mock, event_loop, mocker):
2139+ mocker.patch.object(machine, 'process_preseed')
2140+ mocker.patch.object(machine, '_sanitize_url')
2141+
2142+ class MockGetHTTPFile:
2143+
2144+ async def __call__(self, *args, **kwargs):
2145+ return 200, "{}"
2146+
2147+ mocker.patch.object(
2148+ machine.network, 'get_http_file', new=MockGetHTTPFile())
2149+
2150+ machine.metadata = Mock()
2151+
2152+ async def _noop_request(*args, **kwargs):
2153+ return "{\"metadata_url\": \"http://4.3.2.1/metadata\"}"
2154+
2155+ machine.metadata.request = _noop_request
2156+
2157 await machine.create_pxe_interface()
2158 await machine.get_ip()
2159 labels = {'scheme': 'tftp', 'filename': 'pxelinux.cfg', 'result': '68'}
2160@@ -217,9 +335,28 @@ class TestMachine:
2161
2162 @pytest.mark.asyncio
2163 async def test_get_pxe_files_event_initrd_success(
2164- self, machine, subprocess_mock, event_loop):
2165+ self, machine, subprocess_mock, event_loop, mocker):
2166 fake_curl = subprocess_mock.fake_processes['curl']
2167 fake_curl.initrd_path = '/boot/my-initrd'
2168+
2169+ mocker.patch.object(machine, 'process_preseed')
2170+ mocker.patch.object(machine, '_sanitize_url')
2171+
2172+ class MockGetHTTPFile:
2173+
2174+ async def __call__(self, *args, **kwargs):
2175+ return 200, "{}"
2176+
2177+ mocker.patch.object(
2178+ machine.network, 'get_http_file', new=MockGetHTTPFile())
2179+
2180+ machine.metadata = Mock()
2181+
2182+ async def _noop_request(*args, **kwargs):
2183+ return "{\"metadata_url\": \"http://4.3.2.1/metadata\"}"
2184+
2185+ machine.metadata.request = _noop_request
2186+
2187 await machine.create_pxe_interface()
2188 await machine.get_ip()
2189 labels = {
2190@@ -240,6 +377,25 @@ class TestMachine:
2191 fake_curl.ip_address = '4.3.2.1'
2192 fake_curl.kernel_path = '/boot/my-kernel'
2193 fake_curl.initrd_path = '/boot/my-initrd'
2194+
2195+ mocker.patch.object(machine, 'process_preseed')
2196+ mocker.patch.object(machine, '_sanitize_url')
2197+
2198+ class MockGetHTTPFile:
2199+
2200+ async def __call__(self, *args, **kwargs):
2201+ return 200, "{}"
2202+
2203+ mocker.patch.object(
2204+ machine.network, 'get_http_file', new=MockGetHTTPFile())
2205+
2206+ machine.metadata = Mock()
2207+
2208+ async def _noop_request(*args, **kwargs):
2209+ return "{\"metadata_url\": \"http://4.3.2.1/metadata\"}"
2210+
2211+ machine.metadata.request = _noop_request
2212+
2213 await machine.create_pxe_interface()
2214 await machine.get_ip()
2215 await machine.get_pxe_files()
2216@@ -295,33 +451,59 @@ class TestMachineManager:
2217 (event_loop, 'eth0', 'fake0'), (event_loop, 'eth0', 'fake1'),
2218 (event_loop, 'eth0', 'fake2')
2219 ]
2220+ await manager.power_commands.join(
2221+ ) # cleanly close the queue MachineManager's run loop blocks on
2222+ manager.running = False # quit run loop
2223 await task
2224- for machine in manager.machines:
2225- assert machine.client_mac is not None
2226+ for machine in manager.machines.values():
2227+ assert machine.client_iface is not None
2228
2229 @pytest.mark.asyncio
2230- async def test_run_creates_interface(self, subprocess_mock, event_loop):
2231+ async def test_run_creates_interface(
2232+ self, subprocess_mock, check_output_mock, event_loop):
2233 manager = MachineManager('eth0', 5)
2234- await manager.init(event_loop)
2235+ task = manager.init(event_loop)
2236+ await manager.power_commands.join(
2237+ ) # cleanly close the queue MachineManager's run loop blocks on
2238+ manager.running = False # quit run loop
2239+ await task
2240 assert len(manager.machines) == 5
2241- for machine in manager.machines:
2242- assert machine.client_mac is not None
2243+ for machine in manager.machines.values():
2244+ assert machine.client_iface is not None
2245
2246 @pytest.mark.asyncio
2247- async def test_run_gets_ip(self, subprocess_mock, event_loop):
2248+ async def test_run_gets_ip(
2249+ self, subprocess_mock, check_output_mock, event_loop, mocker):
2250 number_of_machines = 5
2251+
2252+ task = None
2253+
2254+ class MockGetIP:
2255+ called = False
2256+ call_count = 0
2257+
2258+ async def __call__(self, *args, **kwargs):
2259+ self.called = True
2260+ self.call_count += 1
2261+ if self.call_count == number_of_machines:
2262+ task.cancel()
2263+
2264+ mocker.patch.object(Machine, 'get_ip', new=MockGetIP())
2265 manager = MachineManager('eth0', number_of_machines)
2266- with track_metric('client_dhcp_latency_count',
2267- {'type': 'request'}) as metric:
2268- await manager.init(event_loop)
2269+ bmc = BMC(manager)
2270+ task = manager.init(event_loop)
2271+ await bmc.init(event_loop)
2272+ await manager.power_commands.join(
2273+ ) # cleanly close the queue MachineManager's run loop blocks on
2274+ await task
2275 assert len(manager.machines) == number_of_machines
2276- for machine in manager.machines:
2277- assert 'dhcp_request_end' in machine.last_event_time
2278- assert metric.increase == number_of_machines * 1.0
2279+ for machine in manager.machines.values():
2280+ assert machine.get_ip.called
2281
2282 @pytest.mark.asyncio
2283 async def test_run_logs_durations(
2284- self, caplog, subprocess_mock, mocker, event_loop):
2285+ self, caplog, subprocess_mock, check_output_mock, mocker,
2286+ event_loop):
2287 manager = MachineManager('eth0', 5)
2288 with caplog.at_level(logging.INFO):
2289 now_mock = mocker.patch.object(machine_module, 'now')
2290@@ -331,20 +513,24 @@ class TestMachineManager:
2291 4, # Start of DHCP request
2292 7, # End of DHCP request
2293 ]
2294- await manager.init(event_loop)
2295+ task = manager.init(event_loop)
2296+ await manager.power_commands.join()
2297+ manager.running = False
2298+ await task
2299
2300 log_entries = [
2301 (entry.levelname, entry.message) for entry in caplog.records
2302 ]
2303 assert ('INFO', 'Created PXE interfaces in 1 seconds') in log_entries
2304- assert ('INFO', 'Got IPs for all machines in 3 seconds') in log_entries
2305
2306 @pytest.mark.asyncio
2307 async def test_run_handles_task_cancellation(
2308- self, subprocess_mock, event_loop):
2309+ self, subprocess_mock, check_output_mock, event_loop):
2310
2311 class CancellingMachine:
2312+ uuid = random.randint(1, 512)
2313 cancelled = False
2314+ last_event_time = {}
2315
2316 async def create_pxe_interface(self):
2317 self.cancelled = True
2318@@ -354,16 +540,17 @@ class TestMachineManager:
2319 cancelling_machine = CancellingMachine()
2320 task = manager.init(event_loop)
2321 # Inject a machine that simulates the async task being cancelled.
2322- manager.machines.append(cancelling_machine)
2323+ manager.machines[cancelling_machine.uuid] = cancelling_machine
2324 await task
2325 # The async task was cancelled without any problems, and the
2326 # rest of the run() method was skipped.
2327 assert cancelling_machine.cancelled
2328- for machine in manager.machines[:-1]:
2329+ for machine in manager.machines.values():
2330 assert 'dhcp_request_start' not in machine.last_event_time
2331
2332 @pytest.mark.asyncio
2333- async def test_clean_up(self, subprocess_mock, event_loop):
2334+ async def test_clean_up(
2335+ self, subprocess_mock, check_output_mock, event_loop):
2336
2337 class CleanUp:
2338 cleaned_up = False
2339@@ -373,7 +560,10 @@ class TestMachineManager:
2340
2341 clean_up = CleanUp()
2342 manager = MachineManager('eth0', 1)
2343- await manager.init(event_loop)
2344- manager.machines[0].cleanups.append(clean_up.run)
2345+ task = manager.init(event_loop)
2346+ await manager.power_commands.join()
2347+ manager.running = False
2348+ await task
2349+ list(manager.machines.values())[0].cleanups.append(clean_up.run)
2350 await manager.clean_up()
2351 assert clean_up.cleaned_up
2352diff --git a/maasperformance/tests/test_network.py b/maasperformance/tests/test_network.py
2353index 3c1db36..e60345d 100644
2354--- a/maasperformance/tests/test_network.py
2355+++ b/maasperformance/tests/test_network.py
2356@@ -1,8 +1,22 @@
2357+from unittest.mock import Mock
2358+
2359+import aiohttp
2360 from netaddr import IPAddress
2361+import pytest
2362
2363-from ..network import get_source_address
2364+from ..network import (
2365+ get_source_address,
2366+ MetadataClient,
2367+ NetworkClient,
2368+ PlaintextSignature,
2369+ register_machine,
2370+)
2371+from ..process import ProcessError
2372+from ..testing.fixtures import create_subprocess_mock
2373 from ..testing.network import fake_socket
2374
2375+subprocess_mock = create_subprocess_mock()
2376+
2377
2378 class TestGetSourceaddress:
2379
2380@@ -20,3 +34,219 @@ class TestGetSourceaddress:
2381 own_ip = get_source_address(
2382 IPAddress('1200:0000:AB00:1234:0000:2552:7777:1313'))
2383 assert own_ip == IPAddress(sock.ipv6_host)
2384+
2385+
2386+class TestNetworkClient:
2387+
2388+ @pytest.mark.asyncio
2389+ async def test_get_curl_file(self, subprocess_mock):
2390+ machine = Mock()
2391+ machine.last_event_time = {}
2392+ client = NetworkClient(machine)
2393+ try:
2394+ await client.get_curl_file("http://foo/")
2395+ except ProcessError:
2396+ pass
2397+ assert (
2398+ "curl", "-o", "/dev/null", "-s",
2399+ "http://foo/") in subprocess_mock.calls
2400+
2401+ @pytest.mark.asyncio
2402+ async def test_get_json_file(self, mocker):
2403+ machine = Mock()
2404+ machine.last_event_time = {}
2405+ client = NetworkClient(machine)
2406+
2407+ class MockGetHTTPFile:
2408+ args = None
2409+ kwargs = None
2410+
2411+ async def __call__(self, *args, **kwargs):
2412+ self.args = args
2413+ self.kwargs = kwargs
2414+ return 200, "{\"foo\": \"bar\"}"
2415+
2416+ mock_get_http_file = MockGetHTTPFile()
2417+ mocker.patch.object(client, "get_http_file", new=mock_get_http_file)
2418+ status, result = await client.get_json_file("http://foo/")
2419+ assert mock_get_http_file.args == ("http://foo/", )
2420+ assert mock_get_http_file.kwargs == {
2421+ "headers": {
2422+ "Accept": "application/json"
2423+ }
2424+ }
2425+ assert status == 200
2426+ assert isinstance(result, dict)
2427+
2428+ @pytest.mark.asyncio
2429+ async def test_get_tftp_file(self, subprocess_mock):
2430+ machine = Mock()
2431+ machine.last_event_time = {}
2432+ machine.next_server = "foo"
2433+ client = NetworkClient(machine)
2434+ try:
2435+ await client.get_tftp_file("/bar")
2436+ except ProcessError:
2437+ pass
2438+ assert (
2439+ "curl", "-o", "/dev/null", "-s",
2440+ "tftp://foo/bar") in subprocess_mock.calls
2441+
2442+
2443+class TestPlaintextSignature:
2444+
2445+ def test_sign(self):
2446+ sig = PlaintextSignature()
2447+ key = sig.sign("foo", "GET", "http://bar/", "secret")
2448+ assert key == "foo&secret"
2449+
2450+
2451+class MockableRequest:
2452+
2453+ def mock_request(self, client, mocker, response="", status=None):
2454+
2455+ class MockRequest:
2456+ args = None
2457+ kwargs = None
2458+
2459+ async def __call__(self, *args, **kwargs):
2460+ self.args = args
2461+ self.kwargs = kwargs
2462+ if status:
2463+ return status, response
2464+ return response
2465+
2466+ mock_request = MockRequest()
2467+ mocker.patch.object(client, "request", new=mock_request)
2468+
2469+ return mock_request
2470+
2471+
2472+class TestMetadataClient(MockableRequest):
2473+
2474+ def test_authenticated_False_without_consumer_key(self):
2475+ client = MetadataClient(Mock(), {"metadata_url": "http://foo/"})
2476+ assert client.authenticated is False
2477+
2478+ def test_authenticated_True_with_consumer_key(self):
2479+ client = MetadataClient(
2480+ Mock(), {
2481+ "metadata_url": "http://foo/",
2482+ "consumer_key": "bar"
2483+ })
2484+ assert client.authenticated
2485+
2486+ @pytest.mark.asyncio
2487+ async def test_signal(self, mocker):
2488+ network = Mock()
2489+ client = MetadataClient(
2490+ network, {
2491+ "metadata_url": "http://foo/",
2492+ "consumer_key": "bar"
2493+ })
2494+
2495+ mock_request = self.mock_request(client, mocker)
2496+
2497+ await client.signal("TESTING")
2498+
2499+ assert mock_request.args == ("post", "")
2500+ expected_data = aiohttp.FormData()
2501+ expected_data.add_field("op", "signal")
2502+ expected_data.add_field("status", "TESTING")
2503+ assert mock_request.kwargs["data"]._fields == expected_data._fields
2504+
2505+ @pytest.mark.asyncio
2506+ async def test_netboot_off(self, mocker):
2507+ network = Mock()
2508+ client = MetadataClient(
2509+ network, {
2510+ "metadata_url": "http://foo/",
2511+ "consumer_key": "bar"
2512+ })
2513+
2514+ mock_request = self.mock_request(client, mocker)
2515+
2516+ await client.netboot_off()
2517+
2518+ expected_data = aiohttp.FormData()
2519+ expected_data.add_field("op", "netboot_off")
2520+ assert mock_request.args == ("post", "")
2521+ assert mock_request.kwargs["data"]._fields == expected_data._fields
2522+
2523+ @pytest.mark.asyncio
2524+ async def test_get_scripts_index_no_response(self, mocker):
2525+ network = Mock()
2526+ client = MetadataClient(
2527+ network, {
2528+ "metadata_url": "http://foo/",
2529+ "consumer_key": "bar"
2530+ })
2531+
2532+ self.mock_request(client, mocker)
2533+
2534+ result = await client.get_scripts_index()
2535+ assert result == {}
2536+
2537+ @pytest.mark.asyncio
2538+ async def test_get_scripts_index_tar_file_response(
2539+ self, mocker, tar_file_http_response):
2540+ network = Mock()
2541+ client = MetadataClient(
2542+ network, {
2543+ "metadata_url": "http://foo/",
2544+ "consumer_key": "bar"
2545+ })
2546+
2547+ self.mock_request(client, mocker, response=tar_file_http_response)
2548+
2549+ result = await client.get_scripts_index()
2550+ assert result == {"foo": "bar"}
2551+
2552+ def test_get_endpoint_full_url(self):
2553+ network = Mock()
2554+ client = MetadataClient(
2555+ network, {
2556+ "metadata_url": "http://foo/",
2557+ "consumer_key": "bar"
2558+ })
2559+ url = client._get_endpoint_full_url("http://foo/")
2560+ assert url == "http://foo/2012-03-01/"
2561+
2562+
2563+class TestRegisterMachine(MockableRequest):
2564+
2565+ async def test_register_machine(self, mocker):
2566+ network = NetworkClient(Mock())
2567+
2568+ mock_request = self.mock_request(network, mocker, status=200)
2569+
2570+ api_url = "http://foo/"
2571+ hostname = "test.hostname"
2572+ architecture = "amd64"
2573+ subarchitecture = "generic"
2574+ power_type = "ipmi"
2575+ power_parameters = {
2576+ "address": "1.1.1.1",
2577+ "username": "foo",
2578+ "password": "bar",
2579+ }
2580+ mac_address = "11:22:33:44:55"
2581+ headers = {"Accept": "application/json"}
2582+
2583+ await register_machine(
2584+ network, api_url, hostname, architecture, subarchitecture,
2585+ power_type, power_parameters, mac_address)
2586+
2587+ expected_data = aiohttp.FormData()
2588+ expected_data.add_field("hostname", hostname)
2589+ expected_data.add_field("architecture", architecture)
2590+ expected_data.add_field("subarchitecture", subarchitecture)
2591+ expected_data.add_field("mac_addresses", mac_address)
2592+ expected_data.add_field("commission", "true")
2593+ expected_data.add_field("power_type", power_type)
2594+ for key, value in power_parameters.items():
2595+ expected_data.add_field("power_parameters_" + key, value)
2596+
2597+ assert mock_request.args == ("post", api_url + "machines/")
2598+ assert mock_request.kwargs["headers"] == headers
2599+ assert mock_request.kwargs["data"]._fields == expected_data._fields
2600diff --git a/maasperformance/tests/test_network_interface.py b/maasperformance/tests/test_network_interface.py
2601index 0e55d91..bd525e4 100644
2602--- a/maasperformance/tests/test_network_interface.py
2603+++ b/maasperformance/tests/test_network_interface.py
2604@@ -2,7 +2,7 @@ import json
2605
2606 import pytest
2607
2608-from .. import network_interface as network_interface_module
2609+from .. import process
2610 from ..network_interface import (
2611 generate_mac_address,
2612 interface_is_bridge,
2613@@ -29,12 +29,11 @@ class TestInterfaceIsBridge:
2614 (IP_OUTPUT_VETH, 'veth0', False),
2615 ])
2616 def test_interface_is_bridge(self, mocker, output, iface, is_bridge):
2617- mock_check_output = mocker.patch.object(
2618- network_interface_module, 'check_output')
2619+ mock_check_output = mocker.patch.object(process, 'check_output')
2620 mock_check_output.return_value = json.dumps(output)
2621 assert interface_is_bridge(iface) == is_bridge
2622 mock_check_output.assert_called_once_with(
2623- ['ip', '--json', '-d', 'link', 'show', iface])
2624+ ('ip', '--json', '-d', 'link', 'show', iface))
2625
2626
2627 class TestGenerateMacAddress:
2628diff --git a/maasperformance/tests/test_web.py b/maasperformance/tests/test_web.py
2629index fdbee5d..aa408ba 100644
2630--- a/maasperformance/tests/test_web.py
2631+++ b/maasperformance/tests/test_web.py
2632@@ -2,7 +2,12 @@ import asyncio
2633
2634 import prometheus_client
2635
2636+from .. import process
2637 from ..server import parse_args
2638+from ..testing.fixtures import (
2639+ create_check_output_mock,
2640+ create_subprocess_mock,
2641+)
2642 from ..testing.prometheus import (
2643 create_promreg_app,
2644 PromRegHandlers,
2645@@ -14,22 +19,58 @@ from ..web import (
2646 start_machines_loop,
2647 )
2648
2649-
2650-def create_app(port=1234, iface='eth0', number=1):
2651+subprocess_mock = create_subprocess_mock(module=asyncio)
2652+check_output_mock = create_check_output_mock(
2653+ module=process,
2654+ output="""
2655+[
2656+ {
2657+ "ifindex":1,
2658+ "ifname":"parent0",
2659+ "flags":["BROADCAST","MULTICAST","UP","LOWER_UP"],
2660+ "mtu":1500,
2661+ "qdisc":"mq",
2662+ "operstate":"UP",
2663+ "linkmode":"DEFAULT",
2664+ "group":"default",
2665+ "txqlen":1000,
2666+ "link_type":"ether",
2667+ "address":"80:61:5f:08:fc:16",
2668+ "broadcast":"ff:ff:ff:ff:ff:ff",
2669+ "promiscuity":0,
2670+ "min_mtu":68,
2671+ "max_mtu":9710,
2672+ "inet6_addr_gen_mode":"none",
2673+ "num_tx_queues":64,
2674+ "num_rx_queues":64,
2675+ "gso_max_size":65536,
2676+ "gso_max_segs":65535,
2677+ "parentbus":"pci",
2678+ "parentdev":"0000:01:00.0",
2679+ "vfinfo_list":[]
2680+ }
2681+]
2682+""")
2683+
2684+
2685+def create_app(port=1234, iface='eth0', number=1, loop=None):
2686 args = parse_args(['--port', str(port), '--number', str(number), iface])
2687- return create_web_app(args)
2688+ return create_web_app(args, loop=loop)
2689
2690
2691 class TestStatus:
2692
2693- async def test_status(self, create_subprocess_mock, aiohttp_client):
2694- app = create_app(number=2)
2695+ async def test_status(
2696+ self, subprocess_mock, aiohttp_client, check_output_mock,
2697+ event_loop):
2698+ app = create_app(number=2, loop=event_loop)
2699 client = await aiohttp_client(app)
2700+ app['machine-manager'].running = False
2701 await app['machine-manager'].finished
2702 resp = await client.get('/status')
2703 assert resp.status == 200
2704 contents = await resp.json()
2705- machines = app['machine-manager'].machines
2706+ machines = list(app['machine-manager'].machines.values())
2707 assert contents == [
2708 machines[0].get_status(),
2709 machines[1].get_status(),
2710@@ -38,7 +79,8 @@ class TestStatus:
2711
2712 class TestMetrics:
2713
2714- async def test_content_type(self, create_subprocess_mock, aiohttp_client):
2715+ async def test_content_type(
2716+ self, subprocess_mock, check_output_mock, aiohttp_client):
2717 app = create_app(number=2)
2718 client = await aiohttp_client(app)
2719 resp = await client.get('/metrics')
2720@@ -48,7 +90,7 @@ class TestMetrics:
2721 prometheus_client.CONTENT_TYPE_LATEST)
2722
2723 async def test_defined_metrics(
2724- self, create_subprocess_mock, aiohttp_client):
2725+ self, subprocess_mock, check_output_mock, aiohttp_client):
2726 app = create_app(number=2)
2727 client = await aiohttp_client(app)
2728 resp = await client.get('/metrics')
2729@@ -58,53 +100,56 @@ class TestMetrics:
2730
2731 class TestCreateWebApp:
2732
2733- def test_create_web_app_sets_port(self):
2734+ def test_create_web_app_sets_port(self, check_output_mock):
2735 args = parse_args(['--port', '1234', '--number', '5', 'myif0'])
2736 app = create_web_app(args)
2737 assert app['port'] == '1234'
2738
2739- def test_create_web_app_creates_machine_manager(self):
2740+ def test_create_web_app_creates_machine_manager(self, check_output_mock):
2741 args = parse_args(['--port', '1234', '--number', '5', 'parent0'])
2742 app = create_web_app(args)
2743 manager = app['machine-manager']
2744
2745- assert manager.parent_iface == 'parent0'
2746+ assert manager.parent_iface.name == 'parent0'
2747 assert manager.number_of_machines == 5
2748 # We expect an empty machine list, since create_web_app()
2749 # shouldn't call manager.init() directly.
2750 assert manager.machines == []
2751
2752 async def test_create_web_app_starts_manager_on_startup(
2753- self, create_subprocess_mock, loop):
2754+ self, subprocess_mock, check_output_mock, loop):
2755 args = parse_args(['--port', '1234', '--number', '5', 'parent0'])
2756 app = create_web_app(args, loop=loop)
2757 for startup_func in app.on_startup:
2758 await startup_func(app)
2759+ manager = app['machine-manager']
2760+ manager.running = False
2761 await app['manager-task']
2762
2763- manager = app['machine-manager']
2764 assert len(manager.machines) == 5
2765- for machine in manager.machines:
2766- assert machine.client_mac is not None
2767+ for machine in manager.machines.values():
2768+ assert machine.client_iface is not None
2769
2770 async def test_create_web_app_cleans_machines_on_cleanup(
2771- self, create_subprocess_mock, loop):
2772+ self, subprocess_mock, check_output_mock, event_loop):
2773 args = parse_args(['--port', '1234', '--number', '5', 'parent0'])
2774- app = create_web_app(args, loop=loop)
2775+ app = create_web_app(args, loop=event_loop)
2776+ manager = app['machine-manager']
2777 for startup_func in app.on_startup:
2778 await startup_func(app)
2779+ manager.running = False
2780 await app['manager-task']
2781 for cleanup_func in app.on_cleanup:
2782 await cleanup_func(app)
2783
2784- manager = app['machine-manager']
2785 assert len(manager.machines) == 5
2786- for machine in manager.machines:
2787- cleanup_cmd = ('ip', 'link', 'del', machine.client_iface)
2788- assert cleanup_cmd in create_subprocess_mock.calls
2789+ for machine in manager.machines.values():
2790+ cleanup_cmd = ('ip', 'link', 'del', machine.client_iface.name)
2791+ assert cleanup_cmd in subprocess_mock.calls
2792
2793 async def test_registers_promreg_on_startup(
2794- self, create_subprocess_mock, loop, aiohttp_server):
2795+ self, subprocess_mock, check_output_mock, event_loop,
2796+ aiohttp_server):
2797 handlers = PromRegHandlers('my-token')
2798 server = await aiohttp_server(create_promreg_app(handlers))
2799 args = parse_args(
2800@@ -113,15 +158,20 @@ class TestCreateWebApp:
2801 str(server.make_url('/')), '--promreg-token', 'my-token',
2802 '--port', '1234', 'parent0'
2803 ])
2804- app = create_web_app(args, loop=loop)
2805+ app = create_web_app(args, loop=event_loop)
2806 for startup_func in app.on_startup:
2807 await startup_func(app)
2808- await app['manager-task']
2809+ app['manager-task'].cancel()
2810+ try:
2811+ await app['manager-task']
2812+ except asyncio.exceptions.CancelError:
2813+ pass
2814
2815 assert (list(handlers.targets.keys()) == ['127.0.0.1:1234'])
2816
2817 async def test_registers_promreg_on_startup_already_registered(
2818- self, create_subprocess_mock, loop, aiohttp_server):
2819+ self, subprocess_mock, check_output_mock, event_loop,
2820+ aiohttp_server):
2821 handlers = PromRegHandlers('my-token')
2822 server = await aiohttp_server(create_promreg_app(handlers))
2823 args = parse_args(
2824@@ -130,16 +180,21 @@ class TestCreateWebApp:
2825 str(server.make_url('/')), '--promreg-token', 'my-token',
2826 '--port', '1234', 'parent0'
2827 ])
2828- app = create_web_app(args, loop=loop)
2829+ app = create_web_app(args, loop=event_loop)
2830 handlers.targets['127.0.0.1:1234'] = PromRegTarget()
2831 for startup_func in app.on_startup:
2832 await startup_func(app)
2833- await app['manager-task']
2834+ app['manager-task'].cancel()
2835+ try:
2836+ await app['manager-task']
2837+ except asyncio.exceptions.CancelError:
2838+ pass
2839
2840 assert (list(handlers.targets.keys()) == ['127.0.0.1:1234'])
2841
2842 async def test_unregisters_promreg_on_shutdown(
2843- self, create_subprocess_mock, loop, aiohttp_server):
2844+ self, subprocess_mock, check_output_mock, event_loop,
2845+ aiohttp_server):
2846 handlers = PromRegHandlers('my-token')
2847 server = await aiohttp_server(create_promreg_app(handlers))
2848 args = parse_args(
2849@@ -148,13 +203,17 @@ class TestCreateWebApp:
2850 str(server.make_url('/')), '--promreg-token', 'my-token',
2851 '--port', '1234', 'parent0'
2852 ])
2853- app = create_web_app(args, loop=loop)
2854+ app = create_web_app(args, loop=event_loop)
2855 handlers.targets['127.0.0.1:1234'] = PromRegTarget()
2856 for startup_func in app.on_startup:
2857 await startup_func(app)
2858 for cleanup_func in app.on_cleanup:
2859 await cleanup_func(app)
2860- await app['manager-task']
2861+ app['manager-task'].cancel()
2862+ try:
2863+ await app['manager-task']
2864+ except asyncio.exceptions.CancelledError:
2865+ pass
2866
2867 assert list(handlers.targets.keys()) == []
2868
2869@@ -162,53 +221,61 @@ class TestCreateWebApp:
2870 class TestStartMachinesLoop:
2871
2872 async def test_start_machines_loop_inits_manager(
2873- self, create_subprocess_mock, loop):
2874+ self, subprocess_mock, event_loop, check_output_mock):
2875 args = parse_args(['--port', '1234', '--number', '5', 'parent0'])
2876- app = create_web_app(args, loop=loop)
2877+ app = create_web_app(args, loop=event_loop)
2878
2879 await start_machines_loop(app)
2880 manager = app['machine-manager']
2881 assert len(manager.machines) == 5
2882- for machine in manager.machines:
2883- assert machine.loop is loop
2884- assert machine.client_mac is None
2885+ for machine in manager.machines.values():
2886+ assert machine.loop is event_loop
2887+ assert machine.client_iface is not None
2888
2889 assert not app['manager-task'].done()
2890- await app['manager-task']
2891- for machine in manager.machines:
2892- assert machine.client_mac is not None
2893+ app['manager-task'].cancel()
2894+ try:
2895+ await app['manager-task']
2896+ except asyncio.exceptions.CancelledError:
2897+ pass
2898+ for machine in manager.machines.values():
2899+ assert machine.client_iface is not None
2900
2901
2902 class TestCleanUpMachines:
2903
2904- async def test_cleans_up_machines(self, create_subprocess_mock, loop):
2905+ async def test_cleans_up_machines(
2906+ self, subprocess_mock, event_loop, check_output_mock):
2907 args = parse_args(['--port', '1234', '--number', '5', 'parent0'])
2908- app = create_web_app(args, loop=loop)
2909+ app = create_web_app(args, loop=event_loop)
2910
2911 await start_machines_loop(app)
2912- await app['manager-task']
2913+ await app['bmc-task']
2914 await clean_up_machines(app)
2915
2916+ assert app['manager-task'].done()
2917+
2918 manager = app['machine-manager']
2919 assert len(manager.machines) == 5
2920- for machine in manager.machines:
2921- cleanup_cmd = ('ip', 'link', 'del', machine.client_iface)
2922- assert cleanup_cmd in create_subprocess_mock.calls
2923+ for machine in manager.machines.values():
2924+ cleanup_cmd = ('ip', 'link', 'del', machine.client_iface.name)
2925+ assert cleanup_cmd in subprocess_mock.calls
2926
2927- async def test_cancels_manager_task(self, create_subprocess_mock, loop):
2928+ async def test_cancels_manager_task(
2929+ self, subprocess_mock, event_loop, check_output_mock):
2930
2931 async def stall():
2932 machine.cleanups.append(machine.delete_pxe_interface)
2933 stall_point.set_result('there')
2934- await asyncio.sleep(100000, loop=loop)
2935+ await asyncio.sleep(100000, loop=event_loop)
2936
2937 args = parse_args(['--port', '1234', '--number', '1', 'parent0'])
2938- app = create_web_app(args, loop=loop)
2939+ app = create_web_app(args, loop=event_loop)
2940 manager = app['machine-manager']
2941
2942 await start_machines_loop(app)
2943 stall_point = asyncio.Future()
2944- machine = manager.machines[0]
2945+ machine = list(manager.machines.values())[0]
2946 # Simulate that the run() method takes a long time, and allow us
2947 # to run clean_up_machines methods while the task is not yet
2948 # complete.
2949@@ -224,6 +291,6 @@ class TestCleanUpMachines:
2950 assert app['manager-task'].done()
2951
2952 assert len(manager.machines) == 1
2953- for machine in manager.machines:
2954- cleanup_cmd = ('ip', 'link', 'del', machine.client_iface)
2955- assert cleanup_cmd in create_subprocess_mock.calls
2956+ for machine in manager.machines.values():
2957+ cleanup_cmd = ('ip', 'link', 'del', machine.client_iface.name)
2958+ assert cleanup_cmd in subprocess_mock.calls
2959diff --git a/maasperformance/web.py b/maasperformance/web.py
2960index f3df5c6..38d352d 100644
2961--- a/maasperformance/web.py
2962+++ b/maasperformance/web.py
2963@@ -1,3 +1,4 @@
2964+from asyncio.exceptions import CancelledError
2965 import logging
2966 import time
2967 from urllib.parse import urlparse
2968@@ -52,7 +53,10 @@ async def clean_up_machines(app):
2969 logging.info('Cancelling the manager task')
2970 app['manager-task'].cancel()
2971 logging.info('Awaiting the cancel...')
2972- await app['manager-task']
2973+ try:
2974+ await app['manager-task']
2975+ except CancelledError:
2976+ pass
2977 logging.info('done.')
2978 await manager.clean_up()
2979 duration = time.time() - start
2980diff --git a/pyproject.toml b/pyproject.toml
2981new file mode 100644
2982index 0000000..3ac0a51
2983--- /dev/null
2984+++ b/pyproject.toml
2985@@ -0,0 +1,3 @@
2986+[tool.pytest.ini_options]
2987+asyncio_mode = "auto"
2988+addopts = "--cov-fail-under=90"
2989diff --git a/requirements.txt b/requirements.txt
2990index 7ece240..eb73fc9 100644
2991--- a/requirements.txt
2992+++ b/requirements.txt
2993@@ -1,26 +1,27 @@
2994-aioauth-client==0.25.8
2995+aioauth-client==0.27.3
2996 aiodns==3.0.0
2997-aiofiles==0.7.0
2998-aiohttp==3.7.4.post0
2999-anyio==3.1.0
3000-async-timeout==3.0.1
3001-attrs==21.2.0
3002+aiofiles==22.1.0
3003+aiohttp==3.8.1
3004+aiosignal==1.2.0
3005+anyio==3.6.1
3006+async-timeout==4.0.2
3007+attrs==22.1.0
3008 cchardet==2.1.7
3009-certifi==2021.5.30
3010-cffi==1.14.5
3011-chardet==4.0.0
3012+certifi==2022.6.15.1
3013+cffi==1.15.1
3014+charset-normalizer==2.1.1
3015+frozenlist==1.3.1
3016 h11==0.12.0
3017-httpcore==0.13.4
3018-httpx==0.18.1
3019-idna==3.2
3020-multidict==5.1.0
3021+httpcore==0.15.0
3022+httpx==0.23.0
3023+idna==3.3
3024+multidict==6.0.2
3025 netaddr==0.8.0
3026-prometheus-client==0.11.0
3027-pycares==4.0.0
3028-pycparser==2.20
3029-PyYAML==5.4.1
3030+prometheus-client==0.14.1
3031+pycares==4.2.2
3032+pycparser==2.21
3033+PyYAML==6.0
3034 rfc3986==1.5.0
3035-sniffio==1.2.0
3036-typing-extensions==3.10.0.0
3037-uvloop==0.15.2
3038-yarl==1.6.3
3039+sniffio==1.3.0
3040+uvloop==0.16.0
3041+yarl==1.8.1

Subscribers

People subscribed via source and target branches