Merge lp:~corey.bryant/nova/2014.1.2 into lp:~ubuntu-server-dev/nova/icehouse

[Corey Bryant]

[Chuck Short (zulcss)]

Can you resync the branch with what we have in the archive?

review: Needs Fixing

[Corey Bryant (corey.bryant)]

> Can you resync the branch with what we have in the archive?

Ok, new branch has been pushed after resyncing. The resync resulted in a new patch that hasn't been applied yet upstream.

[Corey Bryant (corey.bryant)]

> > Can you resync the branch with what we have in the archive?
>
> Ok, new branch has been pushed after resyncing. The resync resulted in a new
> patch that hasn't been applied yet upstream.

The new patch is libvirt-convert-cpu-features-attribute-from-list-to-.patch

[Corey Bryant (corey.bryant)]

> Can you resync the branch with what we have in the archive?

Chuck, thanks for reviewing. I've resync'd and pushed a new version that is ready for review.

[lp:~corey.bryant/nova/2014.1.2]

693. By James Page on 2014-08-11

* Resynchronize with stable/icehouse (c545075) (LP: #1354159):
  - [1408081] Mask block_device_info auth_password in virt driver debug logs
  - [8d7de91] VMware: Add check for datacenter with no datastore
  - [0d9419f] Fixes hyper-v volume attach when host is AD member
  - [1469c8e] Prevent clean-up of migrating instances on compute init
  - [0617601] VMware: use default values in get_info() when properties are missing
  - [c8df2ac] Set python hash seed to 0 in tox.ini
  - [7de0132] Do not pass instances without host to compute API
  - [811cab7] Do not process events for instances without host
  - [0e1580e] Attach/detach interface to paused instance with affect live flag
  - [397de7b] Turn periodic tasks off in all unit tests
  - [9f59ca7] Avoid possible timing attack in metadata api
  - [72cc37d] Handle service creation race by service workers
  - [af7ba1c] Fixes rbd backend image size
  - [a8b52c0] VMware: Fix race in spawn() when resizing cached image
  - [132d5a2] Avoid re-adding iptables rules for instances that have disappeared
  - [0142324] VMware: Fix fake raising the wrong exception in _remove_file
  - [b08f62a] Fix security group list when not defined for an instance
  - [f598864] libvirt: Make nwfilter driver use right filterref
  - [da777f1] Make sure domain exists before referencing it
  - [8d260a8] Scheduler: enable scheduler hint to pass the group name
  - [4c551d9] Instance groups: add method get_by_hint
  - [9f5d2a6] Avoid referencing stale instance/network_info dicts in firewall
  - [e43c43e] Neutronv2 api does not support neutron without port quota
  - [120ecf3] Fix the wrong dest of 'vlan' option and add new 'vlan_start' option
  - [f864329] Use default rpc_response_timeout in unit tests
  - [4aeefa7] Network: ensure that ports are 'unset' when instance is deleted
  - [a365897] Save connection info in libvirt after volume connect
  - [d1c5736] Cleanup allocating networks when InstanceNotFound is raised
  - [2f40191] Failure during termination should always leave state as error()
  - [485f25d] Use no_timer_check with soft-qemu
  - [0994254] Use correct project/user for quotas
  - [4c6b0f7] Revert "Remove broken quota-classes API"
  - [96212b1] libvirt: Refresh volume connection_info after volume snapshot
  - [c104f2f] Set the volume access mode during volume attach
  - [c754305] VCDriver - Ignore host in Maintenance mode in stats update
  - [4820dbb] VMware: Fix memory leaks caused by caches
  - [0252c87] Include pending task in log message on skip sync_power_state
  - [9095bcf] Fixes Hyper-V iSCSI target login method
  - [17e71f7] VMware: prevent image snapshot if no root disk defined
  - [c545075] versions API: ignore request with a body
* d/p/fix-requirements.patch: Refreshed.
* d/watch: Point to tarballs.openstack.org for release artifacts.
* Cherry picked two fixes from upstream.
  - [2cebfd2] libvirt: convert cpu features attribute from list to a set (LP: #1267191)
  - [b86a0e5] Fixes rdb backend image size (LP: #1219658)

=== modified file 'debian/changelog'
--- debian/changelog	2014-06-24 15:47:40 +0000
+++ debian/changelog	2014-08-08 18:21:34 +0000
@@ -1,3 +1,52 @@
+nova (1:2014.1.2-0ubuntu1) trusty; urgency=medium
+
+  * Resynchronize with stable/icehouse (c545075) (LP: #1354159):
+    - [1408081] Mask block_device_info auth_password in virt driver debug logs
+    - [8d7de91] VMware: Add check for datacenter with no datastore
+    - [0d9419f] Fixes hyper-v volume attach when host is AD member
+    - [1469c8e] Prevent clean-up of migrating instances on compute init
+    - [0617601] VMware: use default values in get_info() when properties are missing
+    - [c8df2ac] Set python hash seed to 0 in tox.ini
+    - [7de0132] Do not pass instances without host to compute API
+    - [811cab7] Do not process events for instances without host
+    - [0e1580e] Attach/detach interface to paused instance with affect live flag
+    - [397de7b] Turn periodic tasks off in all unit tests
+    - [9f59ca7] Avoid possible timing attack in metadata api
+    - [72cc37d] Handle service creation race by service workers
+    - [af7ba1c] Fixes rbd backend image size
+    - [a8b52c0] VMware: Fix race in spawn() when resizing cached image
+    - [132d5a2] Avoid re-adding iptables rules for instances that have disappeared
+    - [0142324] VMware: Fix fake raising the wrong exception in _remove_file
+    - [b08f62a] Fix security group list when not defined for an instance
+    - [f598864] libvirt: Make nwfilter driver use right filterref
+    - [da777f1] Make sure domain exists before referencing it
+    - [8d260a8] Scheduler: enable scheduler hint to pass the group name
+    - [4c551d9] Instance groups: add method get_by_hint
+    - [9f5d2a6] Avoid referencing stale instance/network_info dicts in firewall
+    - [e43c43e] Neutronv2 api does not support neutron without port quota
+    - [120ecf3] Fix the wrong dest of 'vlan' option and add new 'vlan_start' option
+    - [f864329] Use default rpc_response_timeout in unit tests
+    - [4aeefa7] Network: ensure that ports are 'unset' when instance is deleted
+    - [a365897] Save connection info in libvirt after volume connect
+    - [d1c5736] Cleanup allocating networks when InstanceNotFound is raised
+    - [2f40191] Failure during termination should always leave state as error()
+    - [485f25d] Use no_timer_check with soft-qemu
+    - [0994254] Use correct project/user for quotas
+    - [4c6b0f7] Revert "Remove broken quota-classes API"
+    - [96212b1] libvirt: Refresh volume connection_info after volume snapshot
+    - [c104f2f] Set the volume access mode during volume attach
+    - [c754305] VCDriver - Ignore host in Maintenance mode in stats update
+    - [4820dbb] VMware: Fix memory leaks caused by caches
+    - [0252c87] Include pending task in log message on skip sync_power_state
+    - [9095bcf] Fixes Hyper-V iSCSI target login method
+    - [17e71f7] VMware: prevent image snapshot if no root disk defined
+    - [c545075] versions API: ignore request with a body
+  * d/p/fix-requirements.patch: Refreshed.
+  * d/p/libvirt-convert-cpu-features-attribute-from-list-to-.patch: convert cpu
+        features attribute from list to a set (LP: #1267191)
+
+ -- Corey Bryant <[e-mail]>  Thu, 07 Aug 2014 17:16:50 -0400
+
 nova (1:2014.1.1-0ubuntu1) trusty; urgency=medium
 
   * Resynchronize with stable/icehouse (867341f) (LP: #1328134):

=== added file 'debian/patches/fix-requirements.patch'
--- debian/patches/fix-requirements.patch	1970-01-01 00:00:00 +0000
+++ debian/patches/fix-requirements.patch	2014-08-08 18:21:34 +0000
@@ -0,0 +1,16 @@
+Description: Disable websockify and set min version of six.
+Author: Corey Bryant <[e-mail]>
+Forwarded: Not needed.
+--- a/requirements.txt
++++ b/requirements.txt
+@@ -25,9 +25,8 @@
+ python-neutronclient>=2.3.4,<3
+ python-glanceclient>=0.9.0
+ python-keystoneclient>=0.7.0
+-six>=1.6.0
++six>=1.5.2
+ stevedore>=0.14
+-websockify>=0.5.1,<0.6
+ wsgiref>=0.1.2
+ oslo.config>=1.2.0
+ oslo.rootwrap

=== removed file 'debian/patches/fix-requirements.patch'
--- debian/patches/fix-requirements.patch	2014-03-30 17:14:41 +0000
+++ debian/patches/fix-requirements.patch	1970-01-01 00:00:00 +0000
@@ -1,13 +0,0 @@
-diff --git a/requirements.txt b/requirements.txt
-index dd1c1af..93f27b3 100644
---- a/requirements.txt
-+++ b/requirements.txt
-@@ -27,7 +27,6 @@ python-glanceclient>=0.9.0
- python-keystoneclient>=0.7.0
- six>=1.5.2
- stevedore>=0.14
--websockify>=0.5.1,<0.6
- wsgiref>=0.1.2
- oslo.config>=1.2.0
- oslo.rootwrap
-

=== added file 'debian/patches/libvirt-convert-cpu-features-attribute-from-list-to-.patch'
--- debian/patches/libvirt-convert-cpu-features-attribute-from-list-to-.patch	1970-01-01 00:00:00 +0000
+++ debian/patches/libvirt-convert-cpu-features-attribute-from-list-to-.patch	2014-08-08 18:21:34 +0000
@@ -0,0 +1,160 @@
+Description: libvirt: convert cpu features attribute from list to a set
+
+Currently, the cpu features list which is being sent to libvirt,
+when creating a domain or calling compareCPU, must contain only
+unique entries. Multiple issues arise when we are updating the
+features attribute in LibvirtConfigCPU class (for example during
+migration).
+
+This change will change the features attribute from being a list
+to a set. This make the LibvirtConfigCPU class keep only unique
+features.
+Adjusting the LibvirtConfigCPUFeature class to support set
+operations by overriding the __eq__, __ne__ and __hash__
+methods.
+
+Author: Vladik Romanovsky <[e-mail]>
+Origin: upstream, Change-Id: I6350fe0e827c860aea77cc4fe56f18f5c1483580
+Bug-Ubuntu: #1303536
+Last-Update: 2014-06-25
+
+---
+ nova/tests/virt/libvirt/test_libvirt.py        |  8 ++++----
+ nova/tests/virt/libvirt/test_libvirt_config.py | 24 +++++++++++++++++++++++-
+ nova/virt/libvirt/config.py                    | 16 +++++++++++++---
+ nova/virt/libvirt/driver.py                    |  2 +-
+ 4 files changed, 41 insertions(+), 9 deletions(-)
+
+diff --git a/nova/tests/virt/libvirt/test_libvirt.py b/nova/tests/virt/libvirt/test_libvirt.py
+index 1f744b7..9139ab9 100644
+--- a/nova/tests/virt/libvirt/test_libvirt.py
++++ b/nova/tests/virt/libvirt/test_libvirt.py
+@@ -2206,8 +2206,8 @@ class LibvirtConnTestCase(test.TestCase):
+             cpu.model = "Opteron_G4"
+             cpu.vendor = "AMD"
+ 
+-            cpu.features.append(vconfig.LibvirtConfigGuestCPUFeature("tm2"))
+-            cpu.features.append(vconfig.LibvirtConfigGuestCPUFeature("ht"))
++            cpu.add_feature(vconfig.LibvirtConfigGuestCPUFeature("tm2"))
++            cpu.add_feature(vconfig.LibvirtConfigGuestCPUFeature("ht"))
+ 
+             caps = vconfig.LibvirtConfigCaps()
+             caps.host = vconfig.LibvirtConfigCapsHost()
+@@ -2235,8 +2235,8 @@ class LibvirtConnTestCase(test.TestCase):
+         self.assertEqual(conf.cpu.model, "Opteron_G4")
+         self.assertEqual(conf.cpu.vendor, "AMD")
+         self.assertEqual(len(conf.cpu.features), 2)
+-        self.assertEqual(conf.cpu.features[0].name, "tm2")
+-        self.assertEqual(conf.cpu.features[1].name, "ht")
++        self.assertEqual(conf.cpu.features.pop().name, "tm2")
++        self.assertEqual(conf.cpu.features.pop().name, "ht")
+ 
+     def test_get_guest_cpu_config_custom_old(self):
+         def get_lib_version_stub():
+diff --git a/nova/tests/virt/libvirt/test_libvirt_config.py b/nova/tests/virt/libvirt/test_libvirt_config.py
+index 2251bf8..fa65ebc 100644
+--- a/nova/tests/virt/libvirt/test_libvirt_config.py
++++ b/nova/tests/virt/libvirt/test_libvirt_config.py
+@@ -235,8 +235,30 @@ class LibvirtConfigCPUTest(LibvirtConfigBaseTest):
+               <arch>x86_64</arch>
+               <model>Penryn</model>
+               <vendor>Intel</vendor>
++              <feature name="apic"/>
+               <feature name="mtrr"/>
++            </cpu>
++        """)
++
++    def test_only_uniq_cpu_featues(self):
++        obj = config.LibvirtConfigCPU()
++        obj.model = "Penryn"
++        obj.vendor = "Intel"
++        obj.arch = "x86_64"
++
++        obj.add_feature(config.LibvirtConfigCPUFeature("mtrr"))
++        obj.add_feature(config.LibvirtConfigCPUFeature("apic"))
++        obj.add_feature(config.LibvirtConfigCPUFeature("apic"))
++        obj.add_feature(config.LibvirtConfigCPUFeature("mtrr"))
++
++        xml = obj.to_xml()
++        self.assertXmlEqual(xml, """
++            <cpu>
++              <arch>x86_64</arch>
++              <model>Penryn</model>
++              <vendor>Intel</vendor>
+               <feature name="apic"/>
++              <feature name="mtrr"/>
+             </cpu>
+         """)
+ 
+@@ -285,8 +307,8 @@ class LibvirtConfigGuestCPUTest(LibvirtConfigBaseTest):
+               <arch>x86_64</arch>
+               <model>Penryn</model>
+               <vendor>Intel</vendor>
+-              <feature name="mtrr" policy="require"/>
+               <feature name="apic" policy="require"/>
++              <feature name="mtrr" policy="require"/>
+             </cpu>
+         """)
+ 
+diff --git a/nova/virt/libvirt/config.py b/nova/virt/libvirt/config.py
+index a931b57..f32e178 100644
+--- a/nova/virt/libvirt/config.py
++++ b/nova/virt/libvirt/config.py
+@@ -250,6 +250,15 @@ class LibvirtConfigCPUFeature(LibvirtConfigObject):
+ 
+         return ft
+ 
++    def __eq__(self, obj):
++        return obj.name == self.name
++
++    def __ne__(self, obj):
++        return obj.name != self.name
++
++    def __hash__(self):
++        return hash(self.name)
++
+ 
+ class LibvirtConfigCPU(LibvirtConfigObject):
+ 
+@@ -265,7 +274,7 @@ class LibvirtConfigCPU(LibvirtConfigObject):
+         self.cores = None
+         self.threads = None
+ 
+-        self.features = []
++        self.features = set()
+ 
+     def parse_dom(self, xmldoc):
+         super(LibvirtConfigCPU, self).parse_dom(xmldoc)
+@@ -305,13 +314,14 @@ class LibvirtConfigCPU(LibvirtConfigObject):
+             top.set("threads", str(self.threads))
+             cpu.append(top)
+ 
+-        for f in self.features:
++        # sorting the features to allow more predictable tests
++        for f in sorted(self.features, key=lambda x: x.name):
+             cpu.append(f.format_dom())
+ 
+         return cpu
+ 
+     def add_feature(self, feat):
+-        self.features.append(feat)
++        self.features.add(feat)
+ 
+ 
+ class LibvirtConfigGuestCPUFeature(LibvirtConfigCPUFeature):
+diff --git a/nova/virt/libvirt/driver.py b/nova/virt/libvirt/driver.py
+index df367dd..7402363 100644
+--- a/nova/virt/libvirt/driver.py
++++ b/nova/virt/libvirt/driver.py
+@@ -2888,7 +2888,7 @@ class LibvirtDriver(driver.ComputeDriver):
+         for hostfeat in hostcpu.features:
+             guestfeat = vconfig.LibvirtConfigGuestCPUFeature(hostfeat.name)
+             guestfeat.policy = "require"
+-            guestcpu.features.append(guestfeat)
++            guestcpu.add_feature(guestfeat)
+ 
+         return guestcpu
+ 
+-- 
+1.9.1
+

=== modified file 'debian/patches/series'
--- debian/patches/series	2014-06-24 15:47:40 +0000
+++ debian/patches/series	2014-08-08 18:21:34 +0000
@@ -1,4 +1,5 @@
 # Ubuntu specific patches below here. Note these can be dropped eventually.
+libvirt-convert-cpu-features-attribute-from-list-to-.patch
 fix-requirements.patch
 skip_ipv6_test.patch
 arm-console-patch.patch