Merge lp:~ltrager/maas/virt_detect into lp:~maas-committers/maas/trunk

Proposed by Lee Trager
Status: Merged
Approved by: Lee Trager
Approved revision: no longer in the source branch.
Merged at revision: 5105
Proposed branch: lp:~ltrager/maas/virt_detect
Merge into: lp:~maas-committers/maas/trunk
Diff against target: 166 lines (+29/-30)
5 files modified
src/metadataserver/models/commissioningscript.py (+8/-8)
src/metadataserver/models/tests/test_commissioningscript.py (+8/-10)
src/metadataserver/tests/test_api.py (+4/-4)
src/metadataserver/tests/test_api_status.py (+1/-1)
src/provisioningserver/refresh/node_info_scripts.py (+8/-7)
To merge this branch: bzr merge lp:~ltrager/maas/virt_detect
Reviewer Review Type Date Requested Status
Mike Pontillo (community) Approve
Review via email: mp+297020@code.launchpad.net

Commit message

If available use systemd-detect-virt to determine if the commissioning node is virtualized

Description of the change

This modifies the virtuality script to use systemd-detect-virt if available. When systemd-detect-virt is not available(i.e commissioning with trusty) it falls back on the old method which only detects qemu. systemd-detect-virt outputs the detected virtualization which is now stored so users can review it and possibly create virt type tags later on.

To post a comment you must log in.
Revision history for this message
Mike Pontillo (mpontillo) wrote :

The python code looks okay to me, but the shell script where you detect the binary looks odd to me. I think I caught a small issue below, though I'm not certain what the behavior would be.

review: Needs Fixing
Revision history for this message
Lee Trager (ltrager) wrote :

Thanks for the review, I've updated up the shell script as you suggested.

Revision history for this message
Mike Pontillo (mpontillo) wrote :

Looks better now.

You could still probably lose the shell redirection on the fallback grep and just use "grep -q". But I won't block you on that.

review: Approve
Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (31.3 KiB)

The attempt to merge lp:~ltrager/maas/virt_detect into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [94.5 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Fetched 189 kB in 0s (443 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu3).
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
build-essential is already the newest version (12.1ubuntu2).
bzr is already the newest version (2.7.0-2ubuntu1).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
isc-dhcp-common is already the newest version (4.3.3-5ubuntu12).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubun...

Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (1.3 MiB)

The attempt to merge lp:~ltrager/maas/virt_detect into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [94.5 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Fetched 189 kB in 0s (433 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu3).
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
build-essential is already the newest version (12.1ubuntu2).
bzr is already the newest version (2.7.0-2ubuntu1).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
isc-dhcp-common is already the newest version (4.3.3-5ubuntu12).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubun...

Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (1.3 MiB)

The attempt to merge lp:~ltrager/maas/virt_detect into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [94.5 kB]
Hit:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Get:4 http://security.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]
Get:5 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [212 kB]
Get:6 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [93.3 kB]
Fetched 494 kB in 0s (1,166 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu3).
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
build-essential is already the newest version (12.1ubuntu2).
bzr is already the newest version (2.7.0-2ubuntu1).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
isc-dhcp-common is already the newest version (4.3.3-5ubuntu12).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'src/metadataserver/models/commissioningscript.py'
--- src/metadataserver/models/commissioningscript.py 2016-06-02 12:06:46 +0000
+++ src/metadataserver/models/commissioningscript.py 2016-06-11 23:59:05 +0000
@@ -231,7 +231,7 @@
231 """Process the results of `VIRTUALITY_SCRIPT`.231 """Process the results of `VIRTUALITY_SCRIPT`.
232232
233 This adds or removes the *virtual* tag from the node, depending on233 This adds or removes the *virtual* tag from the node, depending on
234 the presence of the terms "notvirtual" or "virtual" in `output`.234 whether a virtualization type is listed.
235235
236 If `exit_status` is non-zero, this function returns without doing236 If `exit_status` is non-zero, this function returns without doing
237 anything.237 anything.
@@ -239,16 +239,16 @@
239 assert isinstance(output, bytes)239 assert isinstance(output, bytes)
240 if exit_status != 0:240 if exit_status != 0:
241 return241 return
242 decoded_output = output.decode('ascii').strip()
242 tag, _ = Tag.objects.get_or_create(name='virtual')243 tag, _ = Tag.objects.get_or_create(name='virtual')
243 if b'notvirtual' in output:244 if 'none' in decoded_output:
244 node.tags.remove(tag)245 node.tags.remove(tag)
245 elif b'virtual' in output:246 elif decoded_output == '':
247 logger.warn(
248 "No virtual type reported in VIRTUALITY_SCRIPT output for node "
249 "%s", node.system_id)
250 else:
246 node.tags.add(tag)251 node.tags.add(tag)
247 else:
248 logger.warn(
249 "Neither 'virtual' nor 'notvirtual' appeared in the "
250 "captured VIRTUALITY_SCRIPT output for node %s.",
251 node.system_id)
252252
253253
254_xpath_routers = "/lldp//id[@type='mac']/text()"254_xpath_routers = "/lldp//id[@type='mac']/text()"
255255
=== modified file 'src/metadataserver/models/tests/test_commissioningscript.py'
--- src/metadataserver/models/tests/test_commissioningscript.py 2016-06-02 12:06:46 +0000
+++ src/metadataserver/models/tests/test_commissioningscript.py 2016-06-11 23:59:05 +0000
@@ -266,38 +266,36 @@
266 def test_sets_virtual_tag(self):266 def test_sets_virtual_tag(self):
267 node = factory.make_Node()267 node = factory.make_Node()
268 self.assertTagsEqual(node, [])268 self.assertTagsEqual(node, [])
269 set_virtual_tag(node, b"virtual", 0)269 set_virtual_tag(node, b"qemu", 0)
270 self.assertTagsEqual(node, ["virtual"])270 self.assertTagsEqual(node, ["virtual"])
271271
272 def test_removes_virtual_tag(self):272 def test_removes_virtual_tag(self):
273 node = factory.make_Node()273 node = factory.make_Node()
274 node.tags.add(self.getVirtualTag())274 node.tags.add(self.getVirtualTag())
275 self.assertTagsEqual(node, ["virtual"])275 self.assertTagsEqual(node, ["virtual"])
276 set_virtual_tag(node, b"notvirtual", 0)276 set_virtual_tag(node, b"none", 0)
277 self.assertTagsEqual(node, [])277 self.assertTagsEqual(node, [])
278278
279 def test_output_not_containing_virtual_does_not_set_tag(self):279 def test_output_not_containing_virtual_does_not_set_tag(self):
280 logger = self.useFixture(FakeLogger())280 logger = self.useFixture(FakeLogger())
281 node = factory.make_Node()281 node = factory.make_Node()
282 self.assertTagsEqual(node, [])282 self.assertTagsEqual(node, [])
283 set_virtual_tag(node, b"wibble", 0)283 set_virtual_tag(node, b"", 0)
284 self.assertTagsEqual(node, [])284 self.assertTagsEqual(node, [])
285 self.assertIn(285 self.assertIn(
286 "Neither 'virtual' nor 'notvirtual' appeared in the captured "286 "No virtual type reported in VIRTUALITY_SCRIPT output for node "
287 "VIRTUALITY_SCRIPT output for node %s.\n" % node.system_id,287 "%s" % node.system_id, logger.output)
288 logger.output)
289288
290 def test_output_not_containing_virtual_does_not_remove_tag(self):289 def test_output_not_containing_virtual_does_not_remove_tag(self):
291 logger = self.useFixture(FakeLogger())290 logger = self.useFixture(FakeLogger())
292 node = factory.make_Node()291 node = factory.make_Node()
293 node.tags.add(self.getVirtualTag())292 node.tags.add(self.getVirtualTag())
294 self.assertTagsEqual(node, ["virtual"])293 self.assertTagsEqual(node, ["virtual"])
295 set_virtual_tag(node, b"wibble", 0)294 set_virtual_tag(node, b"", 0)
296 self.assertTagsEqual(node, ["virtual"])295 self.assertTagsEqual(node, ["virtual"])
297 self.assertIn(296 self.assertIn(
298 "Neither 'virtual' nor 'notvirtual' appeared in the captured "297 "No virtual type reported in VIRTUALITY_SCRIPT output for node "
299 "VIRTUALITY_SCRIPT output for node %s.\n" % node.system_id,298 "%s" % node.system_id, logger.output)
300 logger.output)
301299
302300
303class TestUpdateHardwareDetails(MAASServerTestCase):301class TestUpdateHardwareDetails(MAASServerTestCase):
304302
=== modified file 'src/metadataserver/tests/test_api.py'
--- src/metadataserver/tests/test_api.py 2016-05-24 14:53:56 +0000
+++ src/metadataserver/tests/test_api.py 2016-06-11 23:59:05 +0000
@@ -934,21 +934,21 @@
934 def test_signal_stores_virtual_tag_on_node_if_virtual(self):934 def test_signal_stores_virtual_tag_on_node_if_virtual(self):
935 node = factory.make_Node(status=NODE_STATUS.COMMISSIONING)935 node = factory.make_Node(status=NODE_STATUS.COMMISSIONING)
936 client = make_node_client(node=node)936 client = make_node_client(node=node)
937 content = 'virtual'.encode('utf-8')937 content = 'qemu'.encode('utf-8')
938 response = call_signal(938 response = call_signal(
939 client, script_result=0,939 client, script_result=0,
940 files={'00-maas-02-virtuality.out': content})940 files={'00-maas-02-virtuality.out': content})
941 self.assertEqual(http.client.OK, response.status_code)941 self.assertEqual(http.client.OK, response.status_code)
942 node = reload_object(node)942 node = reload_object(node)
943 self.assertEqual(943 self.assertEqual(
944 ["virtual"], [each_tag.name for each_tag in node.tags.all()])944 ['virtual'], [each_tag.name for each_tag in node.tags.all()])
945945
946 def test_signal_removes_virtual_tag_on_node_if_not_virtual(self):946 def test_signal_removes_virtual_tag_on_node_if_not_virtual(self):
947 node = factory.make_Node(status=NODE_STATUS.COMMISSIONING)947 node = factory.make_Node(status=NODE_STATUS.COMMISSIONING)
948 tag, _ = Tag.objects.get_or_create(name='virtual')948 tag, _ = Tag.objects.get_or_create(name='virtual')
949 node.tags.add(tag)949 node.tags.add(tag)
950 client = make_node_client(node=node)950 client = make_node_client(node=node)
951 content = 'notvirtual'.encode('utf-8')951 content = 'none'.encode('utf-8')
952 response = call_signal(952 response = call_signal(
953 client, script_result=0,953 client, script_result=0,
954 files={'00-maas-02-virtuality.out': content})954 files={'00-maas-02-virtuality.out': content})
@@ -960,7 +960,7 @@
960 def test_signal_leaves_untagged_physical_node_unaltered(self):960 def test_signal_leaves_untagged_physical_node_unaltered(self):
961 node = factory.make_Node(status=NODE_STATUS.COMMISSIONING)961 node = factory.make_Node(status=NODE_STATUS.COMMISSIONING)
962 client = make_node_client(node=node)962 client = make_node_client(node=node)
963 content = 'notvirtual'.encode('utf-8')963 content = 'none'.encode('utf-8')
964 response = call_signal(964 response = call_signal(
965 client, script_result=0,965 client, script_result=0,
966 files={'00-maas-02-virtuality.out': content})966 files={'00-maas-02-virtuality.out': content})
967967
=== modified file 'src/metadataserver/tests/test_api_status.py'
--- src/metadataserver/tests/test_api_status.py 2016-05-24 14:53:56 +0000
+++ src/metadataserver/tests/test_api_status.py 2016-06-11 23:59:05 +0000
@@ -616,7 +616,7 @@
616 tag, _ = Tag.objects.get_or_create(name='virtual')616 tag, _ = Tag.objects.get_or_create(name='virtual')
617 node.tags.add(tag)617 node.tags.add(tag)
618 client = make_node_client(node=node)618 client = make_node_client(node=node)
619 content = 'notvirtual'.encode('utf-8')619 content = 'none'.encode('utf-8')
620 payload = {620 payload = {
621 'event_type': 'finish',621 'event_type': 'finish',
622 'result': 'SUCCESS',622 'result': 'SUCCESS',
623623
=== modified file 'src/provisioningserver/refresh/node_info_scripts.py'
--- src/provisioningserver/refresh/node_info_scripts.py 2016-06-01 19:30:03 +0000
+++ src/provisioningserver/refresh/node_info_scripts.py 2016-06-11 23:59:05 +0000
@@ -70,16 +70,17 @@
70 ip addr70 ip addr
71 """)71 """)
7272
73# Built-in script to detect virtual instances. It will only detect QEMU73# Built-in script to detect virtual instances.
74# for now and may need expanding/generalising at some point.
75# XXX ltrager 2016-01-14 - Replace with virt-what
76VIRTUALITY_SCRIPT = dedent("""\74VIRTUALITY_SCRIPT = dedent("""\
77 #!/bin/sh75 #!/bin/sh
78 grep '^model name.*QEMU.*' /proc/cpuinfo >/dev/null 2>&176 if type systemd-detect-virt 2>&1 > /dev/null; then
79 if [ $? -eq 0 ]; then77 systemd-detect-virt
80 echo "virtual"78 elif grep -q '^model name.*QEMU.*' /proc/cpuinfo; then
79 # Fall back if systemd-detect-virt isn't available. This method only
80 # detects QEMU virtualization not including KVM.
81 echo "qemu"
81 else82 else
82 echo "notvirtual"83 echo "none"
83 fi84 fi
84 """)85 """)
8586