Merge lp:~ltrager/maas/virt_detect into lp:maas/trunk

Proposed by Lee Trager on 2016-06-10
Status: Merged
Approved by: Lee Trager on 2016-06-11
Approved revision: 5103
Merged at revision: 5105
Proposed branch: lp:~ltrager/maas/virt_detect
Merge into: lp:maas/trunk
Diff against target: 166 lines (+29/-30)
5 files modified
src/metadataserver/models/commissioningscript.py (+8/-8)
src/metadataserver/models/tests/test_commissioningscript.py (+8/-10)
src/metadataserver/tests/test_api.py (+4/-4)
src/metadataserver/tests/test_api_status.py (+1/-1)
src/provisioningserver/refresh/node_info_scripts.py (+8/-7)
To merge this branch: bzr merge lp:~ltrager/maas/virt_detect
Reviewer Review Type Date Requested Status
Mike Pontillo (community) 2016-06-10 Approve on 2016-06-10
Review via email: mp+297020@code.launchpad.net

Commit message

If available use systemd-detect-virt to determine if the commissioning node is virtualized

Description of the change

This modifies the virtuality script to use systemd-detect-virt if available. When systemd-detect-virt is not available(i.e commissioning with trusty) it falls back on the old method which only detects qemu. systemd-detect-virt outputs the detected virtualization which is now stored so users can review it and possibly create virt type tags later on.

To post a comment you must log in.
Mike Pontillo (mpontillo) wrote :

The python code looks okay to me, but the shell script where you detect the binary looks odd to me. I think I caught a small issue below, though I'm not certain what the behavior would be.

review: Needs Fixing
Lee Trager (ltrager) wrote :

Thanks for the review, I've updated up the shell script as you suggested.

Mike Pontillo (mpontillo) wrote :

Looks better now.

You could still probably lose the shell redirection on the fallback grep and just use "grep -q". But I won't block you on that.

review: Approve
MAAS Lander (maas-lander) wrote :
Download full text (31.3 KiB)

The attempt to merge lp:~ltrager/maas/virt_detect into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [94.5 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Fetched 189 kB in 0s (443 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu3).
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
build-essential is already the newest version (12.1ubuntu2).
bzr is already the newest version (2.7.0-2ubuntu1).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
isc-dhcp-common is already the newest version (4.3.3-5ubuntu12).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubun...

MAAS Lander (maas-lander) wrote :
Download full text (1.3 MiB)

The attempt to merge lp:~ltrager/maas/virt_detect into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [94.5 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Fetched 189 kB in 0s (433 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu3).
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
build-essential is already the newest version (12.1ubuntu2).
bzr is already the newest version (2.7.0-2ubuntu1).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
isc-dhcp-common is already the newest version (4.3.3-5ubuntu12).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubun...

MAAS Lander (maas-lander) wrote :
Download full text (1.3 MiB)

The attempt to merge lp:~ltrager/maas/virt_detect into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [94.5 kB]
Hit:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Get:4 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]
Get:5 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [212 kB]
Get:6 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [93.3 kB]
Fetched 494 kB in 0s (1,166 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu3).
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
build-essential is already the newest version (12.1ubuntu2).
bzr is already the newest version (2.7.0-2ubuntu1).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
isc-dhcp-common is already the newest version (4.3.3-5ubuntu12).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu...

lp:~ltrager/maas/virt_detect updated on 2016-06-11
5103. By Lee Trager on 2016-06-11

Fix test

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'src/metadataserver/models/commissioningscript.py'
2--- src/metadataserver/models/commissioningscript.py 2016-06-02 12:06:46 +0000
3+++ src/metadataserver/models/commissioningscript.py 2016-06-11 23:59:05 +0000
4@@ -231,7 +231,7 @@
5 """Process the results of `VIRTUALITY_SCRIPT`.
6
7 This adds or removes the *virtual* tag from the node, depending on
8- the presence of the terms "notvirtual" or "virtual" in `output`.
9+ whether a virtualization type is listed.
10
11 If `exit_status` is non-zero, this function returns without doing
12 anything.
13@@ -239,16 +239,16 @@
14 assert isinstance(output, bytes)
15 if exit_status != 0:
16 return
17+ decoded_output = output.decode('ascii').strip()
18 tag, _ = Tag.objects.get_or_create(name='virtual')
19- if b'notvirtual' in output:
20+ if 'none' in decoded_output:
21 node.tags.remove(tag)
22- elif b'virtual' in output:
23+ elif decoded_output == '':
24+ logger.warn(
25+ "No virtual type reported in VIRTUALITY_SCRIPT output for node "
26+ "%s", node.system_id)
27+ else:
28 node.tags.add(tag)
29- else:
30- logger.warn(
31- "Neither 'virtual' nor 'notvirtual' appeared in the "
32- "captured VIRTUALITY_SCRIPT output for node %s.",
33- node.system_id)
34
35
36 _xpath_routers = "/lldp//id[@type='mac']/text()"
37
38=== modified file 'src/metadataserver/models/tests/test_commissioningscript.py'
39--- src/metadataserver/models/tests/test_commissioningscript.py 2016-06-02 12:06:46 +0000
40+++ src/metadataserver/models/tests/test_commissioningscript.py 2016-06-11 23:59:05 +0000
41@@ -266,38 +266,36 @@
42 def test_sets_virtual_tag(self):
43 node = factory.make_Node()
44 self.assertTagsEqual(node, [])
45- set_virtual_tag(node, b"virtual", 0)
46+ set_virtual_tag(node, b"qemu", 0)
47 self.assertTagsEqual(node, ["virtual"])
48
49 def test_removes_virtual_tag(self):
50 node = factory.make_Node()
51 node.tags.add(self.getVirtualTag())
52 self.assertTagsEqual(node, ["virtual"])
53- set_virtual_tag(node, b"notvirtual", 0)
54+ set_virtual_tag(node, b"none", 0)
55 self.assertTagsEqual(node, [])
56
57 def test_output_not_containing_virtual_does_not_set_tag(self):
58 logger = self.useFixture(FakeLogger())
59 node = factory.make_Node()
60 self.assertTagsEqual(node, [])
61- set_virtual_tag(node, b"wibble", 0)
62+ set_virtual_tag(node, b"", 0)
63 self.assertTagsEqual(node, [])
64 self.assertIn(
65- "Neither 'virtual' nor 'notvirtual' appeared in the captured "
66- "VIRTUALITY_SCRIPT output for node %s.\n" % node.system_id,
67- logger.output)
68+ "No virtual type reported in VIRTUALITY_SCRIPT output for node "
69+ "%s" % node.system_id, logger.output)
70
71 def test_output_not_containing_virtual_does_not_remove_tag(self):
72 logger = self.useFixture(FakeLogger())
73 node = factory.make_Node()
74 node.tags.add(self.getVirtualTag())
75 self.assertTagsEqual(node, ["virtual"])
76- set_virtual_tag(node, b"wibble", 0)
77+ set_virtual_tag(node, b"", 0)
78 self.assertTagsEqual(node, ["virtual"])
79 self.assertIn(
80- "Neither 'virtual' nor 'notvirtual' appeared in the captured "
81- "VIRTUALITY_SCRIPT output for node %s.\n" % node.system_id,
82- logger.output)
83+ "No virtual type reported in VIRTUALITY_SCRIPT output for node "
84+ "%s" % node.system_id, logger.output)
85
86
87 class TestUpdateHardwareDetails(MAASServerTestCase):
88
89=== modified file 'src/metadataserver/tests/test_api.py'
90--- src/metadataserver/tests/test_api.py 2016-05-24 14:53:56 +0000
91+++ src/metadataserver/tests/test_api.py 2016-06-11 23:59:05 +0000
92@@ -934,21 +934,21 @@
93 def test_signal_stores_virtual_tag_on_node_if_virtual(self):
94 node = factory.make_Node(status=NODE_STATUS.COMMISSIONING)
95 client = make_node_client(node=node)
96- content = 'virtual'.encode('utf-8')
97+ content = 'qemu'.encode('utf-8')
98 response = call_signal(
99 client, script_result=0,
100 files={'00-maas-02-virtuality.out': content})
101 self.assertEqual(http.client.OK, response.status_code)
102 node = reload_object(node)
103 self.assertEqual(
104- ["virtual"], [each_tag.name for each_tag in node.tags.all()])
105+ ['virtual'], [each_tag.name for each_tag in node.tags.all()])
106
107 def test_signal_removes_virtual_tag_on_node_if_not_virtual(self):
108 node = factory.make_Node(status=NODE_STATUS.COMMISSIONING)
109 tag, _ = Tag.objects.get_or_create(name='virtual')
110 node.tags.add(tag)
111 client = make_node_client(node=node)
112- content = 'notvirtual'.encode('utf-8')
113+ content = 'none'.encode('utf-8')
114 response = call_signal(
115 client, script_result=0,
116 files={'00-maas-02-virtuality.out': content})
117@@ -960,7 +960,7 @@
118 def test_signal_leaves_untagged_physical_node_unaltered(self):
119 node = factory.make_Node(status=NODE_STATUS.COMMISSIONING)
120 client = make_node_client(node=node)
121- content = 'notvirtual'.encode('utf-8')
122+ content = 'none'.encode('utf-8')
123 response = call_signal(
124 client, script_result=0,
125 files={'00-maas-02-virtuality.out': content})
126
127=== modified file 'src/metadataserver/tests/test_api_status.py'
128--- src/metadataserver/tests/test_api_status.py 2016-05-24 14:53:56 +0000
129+++ src/metadataserver/tests/test_api_status.py 2016-06-11 23:59:05 +0000
130@@ -616,7 +616,7 @@
131 tag, _ = Tag.objects.get_or_create(name='virtual')
132 node.tags.add(tag)
133 client = make_node_client(node=node)
134- content = 'notvirtual'.encode('utf-8')
135+ content = 'none'.encode('utf-8')
136 payload = {
137 'event_type': 'finish',
138 'result': 'SUCCESS',
139
140=== modified file 'src/provisioningserver/refresh/node_info_scripts.py'
141--- src/provisioningserver/refresh/node_info_scripts.py 2016-06-01 19:30:03 +0000
142+++ src/provisioningserver/refresh/node_info_scripts.py 2016-06-11 23:59:05 +0000
143@@ -70,16 +70,17 @@
144 ip addr
145 """)
146
147-# Built-in script to detect virtual instances. It will only detect QEMU
148-# for now and may need expanding/generalising at some point.
149-# XXX ltrager 2016-01-14 - Replace with virt-what
150+# Built-in script to detect virtual instances.
151 VIRTUALITY_SCRIPT = dedent("""\
152 #!/bin/sh
153- grep '^model name.*QEMU.*' /proc/cpuinfo >/dev/null 2>&1
154- if [ $? -eq 0 ]; then
155- echo "virtual"
156+ if type systemd-detect-virt 2>&1 > /dev/null; then
157+ systemd-detect-virt
158+ elif grep -q '^model name.*QEMU.*' /proc/cpuinfo; then
159+ # Fall back if systemd-detect-virt isn't available. This method only
160+ # detects QEMU virtualization not including KVM.
161+ echo "qemu"
162 else
163- echo "notvirtual"
164+ echo "none"
165 fi
166 """)
167