Merge lp:~ntt-pf-lab/nova/ipv6-support into lp:~hudson-openstack/nova/trunk

Proposed by Nachi Ueno
Status: Superseded
Proposed branch: lp:~ntt-pf-lab/nova/ipv6-support
Merge into: lp:~hudson-openstack/nova/trunk
Diff against target: 1537 lines (+809/-53)
22 files modified
bin/nova-manage (+7/-3)
contrib/boto_v6/__init__.py (+37/-0)
contrib/boto_v6/ec2/connection.py (+41/-0)
contrib/boto_v6/ec2/instance.py (+37/-0)
contrib/nova.sh (+6/-0)
nova/api/ec2/cloud.py (+13/-0)
nova/db/api.py (+12/-0)
nova/db/sqlalchemy/api.py (+27/-0)
nova/db/sqlalchemy/models.py (+4/-0)
nova/network/linux_net.py (+88/-0)
nova/network/manager.py (+29/-3)
nova/test.py (+2/-1)
nova/tests/test_api.py (+67/-0)
nova/tests/test_network.py (+22/-0)
nova/utils.py (+49/-0)
nova/virt/libvirt.xml.template (+1/-0)
nova/virt/libvirt_conn.py (+137/-34)
smoketests/admin_smoketests.py (+1/-2)
smoketests/base.py (+15/-2)
smoketests/flags.py (+2/-1)
smoketests/public_network_smoketests.py (+180/-0)
smoketests/user_smoketests.py (+32/-7)
To merge this branch: bzr merge lp:~ntt-pf-lab/nova/ipv6-support
Reviewer Review Type Date Requested Status
Soren Hansen (community) Needs Fixing
Devin Carlen Pending
Review via email: mp+45986@code.launchpad.net

This proposal supersedes a proposal from 2011-01-05.

This proposal has been superseded by a proposal from 2011-01-13.

Description of the change

OpenStack Compute (Nova) IPv4/IPv6 dual stack support
http://wiki.openstack.org/BexarIpv6supportReadme

Tested with
 unit test
 smoke test

No conflict with current branch r 551.

Fixed comment by Soren

To post a comment you must log in.
Revision history for this message
Soren Hansen (soren) wrote : Posted in a previous version of this proposal
Download full text (9.6 KiB)

Please find my comments inline.

Please note that ipv6 is still a bit of a mystery to me, so I'll review
structure and style moreso than technical correctness.

2011/1/5 Nachi Ueno <email address hidden>:
> Nachi Ueno has proposed merging lp:~ntt-pf-lab/nova/ipv6-support into lp:nova.
>
> Requested reviews:
>  Nova Core (nova-core)
>
> For more details, see:
> https://code.launchpad.net/~ntt-pf-lab/nova/ipv6-support/+merge/45228
>
> OpenStack Compute (Nova) IPv4/IPv6 dual stack support
> http://wiki.openstack.org/BexarIpv6supportReadme
>
> Tested with
>  unit test
>  smoke test
>
> No conflict with current branch r 515.
> --
> https://code.launchpad.net/~ntt-pf-lab/nova/ipv6-support/+merge/45228
> You are subscribed to branch lp:nova.
>
> === modified file 'bin/nova-manage'
> --- bin/nova-manage     2010-12-28 20:11:41 +0000
> +++ bin/nova-manage     2011-01-05 11:56:16 +0000
> @@ -89,7 +89,7 @@
>  flags.DECLARE('network_size', 'nova.network.manager')
>  flags.DECLARE('vlan_start', 'nova.network.manager')
>  flags.DECLARE('vpn_start', 'nova.network.manager')
> -
> +flags.DECLARE('fixed_range_v6', 'nova.network.manager')
>
>  class VpnCommands(object):
>     """Class for managing VPNs."""
> @@ -432,11 +432,11 @@
>     """Class for managing networks."""
>
>     def create(self, fixed_range=None, num_networks=None,
> -               network_size=None, vlan_start=None, vpn_start=None):
> +               network_size=None, vlan_start=None, vpn_start=None,fixed_range_v6=None):

This line is too long. Nova is pep8 clean. You can install the pep8
utility and check for yourself.

>         """Creates fixed ips for host by range
>         arguments: [fixed_range=FLAG], [num_networks=FLAG],
>                    [network_size=FLAG], [vlan_start=FLAG],
> -                   [vpn_start=FLAG]"""
> +                   [vpn_start=FLAG],[fixed_range_v6=FLAG]"""

Missing a space (for consistency).

>         if not fixed_range:
>             fixed_range = FLAGS.fixed_range
>         if not num_networks:
> @@ -447,11 +447,15 @@
>             vlan_start = FLAGS.vlan_start
>         if not vpn_start:
>             vpn_start = FLAGS.vpn_start
> +        if not fixed_range_v6:
> +            fixed_range_v6 = FLAGS.fixed_range_v6
>         net_manager = utils.import_object(FLAGS.network_manager)
>         net_manager.create_networks(context.get_admin_context(),
>                                     fixed_range, int(num_networks),
>                                     int(network_size), int(vlan_start),
> -                                    int(vpn_start))
> +                                    int(vpn_start),fixed_range_v6)
> +
> +

Same here. Please add a space (pep8 will tell you this, too).

> === modified file 'nova/api/ec2/cloud.py'
> --- nova/api/ec2/cloud.py       2011-01-03 19:29:39 +0000
> +++ nova/api/ec2/cloud.py       2011-01-05 11:56:16 +0000
> @@ -30,7 +30,7 @@
>
>  from nova import context
>  import IPy
> -
> +import urllib
>  from nova import crypto
>  from nova import db
>  from nova import exception
> @@ -347,6 +347,7 @@
>             values['group_id'] = source_security_group['id']
>         elif cidr_ip:
>             # If this fails, it throws an...

Read more...

Revision history for this message
Soren Hansen (soren) : Posted in a previous version of this proposal
review: Needs Fixing
Revision history for this message
Devin Carlen (devcamcar) wrote : Posted in a previous version of this proposal

It looks like we are essentially providing our own IPV6 version of boto with this patch, which is a bit scary. If we aren't able to use the official libraries (boto, etc.) for IPV6, then maybe we shouldn't be trying to do this with the EC2 API? Forgive me if this has already been discussed by the community, but I think it's a fair question.

review: Needs Information
Revision history for this message
Nachi Ueno (nati-ueno) wrote : Posted in a previous version of this proposal

>Devin Carlen
Thank you for your comment.
We extended DescribeInstance. However we will revert DescribeInstance,and We will add DescribeInstanceV6 today.
Users,who don't need to see v6 address in a DescribeInstance response,can use standard boto libraries with no side effect in current version.
(Users can use EC2 API and DescribeInstanceV6 API)

We continue to support IPv6 version of boto.
As you can see, we created boto_v6 by extending boto classes.
So the code is really simple :).

Revision history for this message
Soren Hansen (soren) wrote :
Download full text (5.8 KiB)

I'm getting a number of failures when I run the test suite.

======================================================================
ERROR: test_associate_disassociate_address (nova.tests.test_cloud.CloudTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/soren/src/openstack/nova/nova/nova/tests/test_cloud.py", line 112, in test_associate_disassociate_address
    public_ip=address)
  File "/home/soren/src/openstack/nova/nova/nova/api/ec2/cloud.py", line 758, in associate_address
    self.compute_api.associate_floating_ip(context, instance_id, public_ip)
  File "/home/soren/src/openstack/nova/nova/nova/compute/api.py", line 501, in associate_floating_ip
    instance['fixed_ip'])
  File "/home/soren/src/openstack/nova/nova/nova/network/api.py", line 70, in associate_floating_ip
    host = fixed_ip['network']['host']
  File "/home/soren/src/openstack/nova/nova/nova/db/sqlalchemy/models.py", line 74, in __getitem__
    return getattr(self, key)
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/orm/attributes.py", line 163, in __get__
    instance_dict(instance))
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/orm/attributes.py", line 382, in get
    value = callable_(passive=passive)
  File "/usr/lib/python2.6/dist-packages/sqlalchemy/orm/strategies.py", line 578, in __call__
    (mapperutil.state_str(state), self.key)
DetachedInstanceError: Parent instance <FixedIp at 0x5a13e90> is not bound to a Session; lazy load operation of attribute 'network' cannot proceed

======================================================================
ERROR: test_private_ipv6 (nova.tests.test_network.NetworkTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/soren/src/openstack/nova/nova/nova/tests/test_network.py", line 108, in test_private_ipv6
    instance_ref['id'])
  File "/home/soren/src/openstack/nova/nova/nova/db/api.py", line 361, in instance_get_fixed_address_v6
    return IMPL.instance_get_fixed_address_v6(context, instance_id)
  File "/home/soren/src/openstack/nova/nova/nova/db/sqlalchemy/api.py", line 107, in wrapper
    return f(*args, **kwargs)
  File "/home/soren/src/openstack/nova/nova/nova/db/sqlalchemy/api.py", line 810, in instance_get_fixed_address_v6
    network_ref = network_get_by_instance(context, instance_id)
  File "/home/soren/src/openstack/nova/nova/nova/db/sqlalchemy/api.py", line 95, in wrapper
    raise exception.NotAuthorized()
NotAuthorized: None

======================================================================
ERROR: test_static_filters (nova.tests.test_virt.IptablesFirewallTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/soren/src/openstack/nova/nova/nova/tests/test_virt.py", line 306, in test_static_filters
    out_rules = self.fw.modify_rules(self.in_rules)
TypeError: modify_rules() takes exactly 3 arguments (2 given)

======================================================================
FAIL: test_describe_instances (nova.tests.test_cloud.CloudTestCase)
---------------------------...

Read more...

review: Needs Fixing
lp:~ntt-pf-lab/nova/ipv6-support updated
487. By Nachi Ueno

fixed method signature of modify_rules
fixed unit_test for ipv6

488. By Nachi Ueno

merged with r555

Revision history for this message
Nachi Ueno (nati-ueno) wrote :

Thank you.
We fixed some bug.

export TEST=1
nova/contrib/nova.sh run

~~~~
----------------------------------------------------------------------
Ran 277 tests in 501.898s

OK

lp:~ntt-pf-lab/nova/ipv6-support updated
489. By Koji Iida

Fixed missing _().
Fixed to follow logging to LOG changes.
Fixed merge miss (get_fixed_ip was moved away).
Update some missing comments.

490. By Nachi Ueno

Merged with 557

491. By Hisaharu Ishii

Fixed Authors

492. By Nachi Ueno

Added netaddr for pip-requires

493. By Nachi Ueno

Moved commands which needs sudo to nova.sh

494. By Nachi Ueno

Merged with r561

495. By Hisaharu Ishii

Merged with r562

496. By Hisaharu Ishii

sort Authors

497. By Koji Iida

Merged to rev.563

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'bin/nova-manage'
2--- bin/nova-manage 2011-01-12 20:12:08 +0000
3+++ bin/nova-manage 2011-01-13 00:56:22 +0000
4@@ -91,6 +91,7 @@
5 flags.DECLARE('network_size', 'nova.network.manager')
6 flags.DECLARE('vlan_start', 'nova.network.manager')
7 flags.DECLARE('vpn_start', 'nova.network.manager')
8+flags.DECLARE('fixed_range_v6', 'nova.network.manager')
9
10
11 class VpnCommands(object):
12@@ -439,11 +440,12 @@
13 """Class for managing networks."""
14
15 def create(self, fixed_range=None, num_networks=None,
16- network_size=None, vlan_start=None, vpn_start=None):
17+ network_size=None, vlan_start=None, vpn_start=None,
18+ fixed_range_v6=None):
19 """Creates fixed ips for host by range
20 arguments: [fixed_range=FLAG], [num_networks=FLAG],
21 [network_size=FLAG], [vlan_start=FLAG],
22- [vpn_start=FLAG]"""
23+ [vpn_start=FLAG], [fixed_range_v6=FLAG]"""
24 if not fixed_range:
25 fixed_range = FLAGS.fixed_range
26 if not num_networks:
27@@ -454,11 +456,13 @@
28 vlan_start = FLAGS.vlan_start
29 if not vpn_start:
30 vpn_start = FLAGS.vpn_start
31+ if not fixed_range_v6:
32+ fixed_range_v6 = FLAGS.fixed_range_v6
33 net_manager = utils.import_object(FLAGS.network_manager)
34 net_manager.create_networks(context.get_admin_context(),
35 fixed_range, int(num_networks),
36 int(network_size), int(vlan_start),
37- int(vpn_start))
38+ int(vpn_start), fixed_range_v6)
39
40
41 class ServiceCommands(object):
42
43=== added directory 'contrib/boto_v6'
44=== added file 'contrib/boto_v6/__init__.py'
45--- contrib/boto_v6/__init__.py 1970-01-01 00:00:00 +0000
46+++ contrib/boto_v6/__init__.py 2011-01-13 00:56:22 +0000
47@@ -0,0 +1,37 @@
48+# Copyright (c) 2006-2010 Mitch Garnaat http://garnaat.org/
49+# Copyright (c) 2010, Eucalyptus Systems, Inc.
50+# All rights reserved.
51+#
52+# Permission is hereby granted, free of charge, to any person obtaining a
53+# copy of this software and associated documentation files (the
54+# "Software"), to deal in the Software without restriction, including
55+# without limitation the rights to use, copy, modify, merge, publish, dis-
56+# tribute, sublicense, and/or sell copies of the Software, and to permit
57+# persons to whom the Software is furnished to do so, subject to the fol-
58+# lowing conditions:
59+#
60+# The above copyright notice and this permission notice shall be included
61+# in all copies or substantial portions of the Software.
62+#
63+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
64+# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABIL-
65+# ITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
66+# SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
67+# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
68+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
69+# IN THE SOFTWARE.
70+
71+
72+def connect_ec2(aws_access_key_id=None, aws_secret_access_key=None, **kwargs):
73+ """
74+ :type aws_access_key_id: string
75+ :param aws_access_key_id: Your AWS Access Key ID
76+
77+ :type aws_secret_access_key: string
78+ :param aws_secret_access_key: Your AWS Secret Access Key
79+
80+ :rtype: :class:`boto.ec2.connection.EC2Connection`
81+ :return: A connection to Amazon's EC2
82+ """
83+ from boto_v6.ec2.connection import EC2ConnectionV6
84+ return EC2ConnectionV6(aws_access_key_id, aws_secret_access_key, **kwargs)
85
86=== added directory 'contrib/boto_v6/ec2'
87=== added file 'contrib/boto_v6/ec2/__init__.py'
88=== added file 'contrib/boto_v6/ec2/connection.py'
89--- contrib/boto_v6/ec2/connection.py 1970-01-01 00:00:00 +0000
90+++ contrib/boto_v6/ec2/connection.py 2011-01-13 00:56:22 +0000
91@@ -0,0 +1,41 @@
92+'''
93+Created on 2010/12/20
94+
95+@author: Nachi Ueno <ueno.nachi@lab.ntt.co.jp>
96+'''
97+import boto
98+import boto.ec2
99+from boto_v6.ec2.instance import ReservationV6
100+
101+
102+class EC2ConnectionV6(boto.ec2.EC2Connection):
103+ '''
104+ EC2Connection for OpenStack IPV6 mode
105+ '''
106+ def get_all_instances(self, instance_ids=None, filters=None):
107+ """
108+ Retrieve all the instances associated with your account.
109+
110+ :type instance_ids: list
111+ :param instance_ids: A list of strings of instance IDs
112+
113+ :type filters: dict
114+ :param filters: Optional filters that can be used to limit
115+ the results returned. Filters are provided
116+ in the form of a dictionary consisting of
117+ filter names as the key and filter values
118+ as the value. The set of allowable filter
119+ names/values is dependent on the request
120+ being performed. Check the EC2 API guide
121+ for details.
122+
123+ :rtype: list
124+ :return: A list of :class:`boto.ec2.instance.Reservation`
125+ """
126+ params = {}
127+ if instance_ids:
128+ self.build_list_params(params, instance_ids, 'InstanceId')
129+ if filters:
130+ self.build_filter_params(params, filters)
131+ return self.get_list('DescribeInstancesV6', params,
132+ [('item', ReservationV6)])
133
134=== added file 'contrib/boto_v6/ec2/instance.py'
135--- contrib/boto_v6/ec2/instance.py 1970-01-01 00:00:00 +0000
136+++ contrib/boto_v6/ec2/instance.py 2011-01-13 00:56:22 +0000
137@@ -0,0 +1,37 @@
138+'''
139+Created on 2010/12/20
140+
141+@author: Nachi Ueno <ueno.nachi@lab.ntt.co.jp>
142+'''
143+import boto
144+from boto.resultset import ResultSet
145+from boto.ec2.instance import Reservation
146+from boto.ec2.instance import Group
147+from boto.ec2.instance import Instance
148+
149+
150+class ReservationV6(Reservation):
151+ def startElement(self, name, attrs, connection):
152+ if name == 'instancesSet':
153+ self.instances = ResultSet([('item', InstanceV6)])
154+ return self.instances
155+ elif name == 'groupSet':
156+ self.groups = ResultSet([('item', Group)])
157+ return self.groups
158+ else:
159+ return None
160+
161+
162+class InstanceV6(Instance):
163+ def __init__(self, connection=None):
164+ Instance.__init__(self, connection)
165+ self.dns_name_v6 = None
166+
167+ def endElement(self, name, value, connection):
168+ Instance.endElement(self, name, value, connection)
169+ if name == 'dnsNameV6':
170+ self.dns_name_v6 = value
171+
172+ def _update(self, updated):
173+ self.__dict__.update(updated.__dict__)
174+ self.dns_name_v6 = updated.dns_name_v6
175
176=== modified file 'contrib/nova.sh'
177--- contrib/nova.sh 2010-12-30 00:07:41 +0000
178+++ contrib/nova.sh 2011-01-13 00:56:22 +0000
179@@ -86,6 +86,10 @@
180 sudo apt-get install -y python-twisted python-sqlalchemy python-mox python-greenlet python-carrot
181 sudo apt-get install -y python-daemon python-eventlet python-gflags python-tornado python-ipy
182 sudo apt-get install -y python-libvirt python-libxml2 python-routes
183+#For IPV6
184+ sudo apt-get install -y python-netaddr
185+ sudo apt-get install -y radvd
186+
187 if [ "$USE_MYSQL" == 1 ]; then
188 cat <<MYSQL_PRESEED | debconf-set-selections
189 mysql-server-5.1 mysql-server/root_password password $MYSQL_PASS
190@@ -107,6 +111,8 @@
191
192 if [ "$CMD" == "run" ]; then
193 killall dnsmasq
194+ #For IPv6
195+ killall radvd
196 screen -d -m -S nova -t nova
197 sleep 1
198 if [ "$USE_MYSQL" == 1 ]; then
199
200=== modified file 'nova/api/ec2/cloud.py'
201--- nova/api/ec2/cloud.py 2011-01-12 11:34:16 +0000
202+++ nova/api/ec2/cloud.py 2011-01-13 00:56:22 +0000
203@@ -26,9 +26,11 @@
204 import datetime
205 import IPy
206 import os
207+import urllib
208
209 from nova import compute
210 from nova import context
211+
212 from nova import crypto
213 from nova import db
214 from nova import exception
215@@ -378,6 +380,7 @@
216 values['group_id'] = source_security_group['id']
217 elif cidr_ip:
218 # If this fails, it throws an exception. This is what we want.
219+ cidr_ip = urllib.unquote(cidr_ip).decode()
220 IPy.IP(cidr_ip)
221 values['cidr'] = cidr_ip
222 else:
223@@ -639,6 +642,10 @@
224 def describe_instances(self, context, **kwargs):
225 return self._format_describe_instances(context, **kwargs)
226
227+ def describe_instances_v6(self, context, **kwargs):
228+ kwargs['use_v6'] = True
229+ return self._format_describe_instances(context, **kwargs)
230+
231 def _format_describe_instances(self, context, **kwargs):
232 return {'reservationSet': self._format_instances(context, **kwargs)}
233
234@@ -674,10 +681,16 @@
235 if instance['fixed_ip']['floating_ips']:
236 fixed = instance['fixed_ip']
237 floating_addr = fixed['floating_ips'][0]['address']
238+ if instance['fixed_ip']['network'] and 'use_v6' in kwargs:
239+ i['dnsNameV6'] = utils.to_global_ipv6(
240+ instance['fixed_ip']['network']['cidr_v6'],
241+ instance['mac_address'])
242+
243 i['privateDnsName'] = fixed_addr
244 i['publicDnsName'] = floating_addr
245 i['dnsName'] = i['publicDnsName'] or i['privateDnsName']
246 i['keyName'] = instance['key_name']
247+
248 if context.user.is_admin():
249 i['keyName'] = '%s (%s, %s)' % (i['keyName'],
250 instance['project_id'],
251
252=== modified file 'nova/db/api.py'
253--- nova/db/api.py 2011-01-12 11:34:16 +0000
254+++ nova/db/api.py 2011-01-13 00:56:22 +0000
255@@ -295,6 +295,10 @@
256 return IMPL.fixed_ip_get_instance(context, address)
257
258
259+def fixed_ip_get_instance_v6(context, address):
260+ return IMPL.fixed_ip_get_instance_v6(context, address)
261+
262+
263 def fixed_ip_get_network(context, address):
264 """Get a network for a fixed ip by address."""
265 return IMPL.fixed_ip_get_network(context, address)
266@@ -353,6 +357,10 @@
267 return IMPL.instance_get_fixed_address(context, instance_id)
268
269
270+def instance_get_fixed_address_v6(context, instance_id):
271+ return IMPL.instance_get_fixed_address_v6(context, instance_id)
272+
273+
274 def instance_get_floating_address(context, instance_id):
275 """Get the first floating ip address of an instance."""
276 return IMPL.instance_get_floating_address(context, instance_id)
277@@ -548,6 +556,10 @@
278 return IMPL.project_get_network(context, project_id)
279
280
281+def project_get_network_v6(context, project_id):
282+ return IMPL.project_get_network_v6(context, project_id)
283+
284+
285 ###################
286
287
288
289=== modified file 'nova/db/sqlalchemy/api.py'
290--- nova/db/sqlalchemy/api.py 2011-01-12 22:24:16 +0000
291+++ nova/db/sqlalchemy/api.py 2011-01-13 00:56:22 +0000
292@@ -606,6 +606,17 @@
293 return fixed_ip_ref.instance
294
295
296+@require_context
297+def fixed_ip_get_instance_v6(context, address):
298+ session = get_session()
299+ mac = utils.to_mac(address)
300+
301+ result = session.query(models.Instance
302+ ).filter_by(mac_address=mac
303+ ).first()
304+ return result
305+
306+
307 @require_admin_context
308 def fixed_ip_get_network(context, address):
309 fixed_ip_ref = fixed_ip_get_by_address(context, address)
310@@ -794,6 +805,17 @@
311
312
313 @require_context
314+def instance_get_fixed_address_v6(context, instance_id):
315+ session = get_session()
316+ with session.begin():
317+ instance_ref = instance_get(context, instance_id, session=session)
318+ network_ref = network_get_by_instance(context, instance_id)
319+ prefix = network_ref.cidr_v6
320+ mac = instance_ref.mac_address
321+ return utils.to_global_ipv6(prefix, mac)
322+
323+
324+@require_context
325 def instance_get_floating_address(context, instance_id):
326 session = get_session()
327 with session.begin():
328@@ -1130,6 +1152,11 @@
329 return result
330
331
332+@require_context
333+def project_get_network_v6(context, project_id):
334+ return project_get_network(context, project_id)
335+
336+
337 ###################
338
339
340
341=== modified file 'nova/db/sqlalchemy/models.py'
342--- nova/db/sqlalchemy/models.py 2011-01-12 11:34:16 +0000
343+++ nova/db/sqlalchemy/models.py 2011-01-13 00:56:22 +0000
344@@ -411,6 +411,10 @@
345
346 injected = Column(Boolean, default=False)
347 cidr = Column(String(255), unique=True)
348+ cidr_v6 = Column(String(255), unique=True)
349+
350+ ra_server = Column(String(255))
351+
352 netmask = Column(String(255))
353 bridge = Column(String(255))
354 gateway = Column(String(255))
355
356=== modified file 'nova/network/linux_net.py'
357--- nova/network/linux_net.py 2011-01-11 05:49:46 +0000
358+++ nova/network/linux_net.py 2011-01-13 00:56:22 +0000
359@@ -50,6 +50,7 @@
360 'Public IP of network host')
361 flags.DEFINE_bool('use_nova_chains', False,
362 'use the nova_ routing chains instead of default')
363+
364 flags.DEFINE_string('dns_server', None,
365 'if set, uses specific dns server for dnsmasq')
366 flags.DEFINE_string('dmz_cidr', '10.128.0.0/24',
367@@ -123,6 +124,11 @@
368 (FLAGS.fixed_range, FLAGS.dmz_cidr))
369 _confirm_rule("POSTROUTING", "-t nat -s %(range)s -d %(range)s -j ACCEPT" %
370 {'range': FLAGS.fixed_range})
371+ if(FLAGS.use_ipv6):
372+ _execute('sudo bash -c ' +
373+ '"echo 1 > /proc/sys/net/ipv6/conf/all/forwarding"')
374+ _execute('sudo bash -c ' +
375+ '"echo 0 > /proc/sys/net/ipv6/conf/all/accept_ra"')
376
377
378 def bind_floating_ip(floating_ip, check_exit_code=True):
379@@ -196,6 +202,10 @@
380 net_attrs['gateway'],
381 net_attrs['broadcast'],
382 net_attrs['netmask']))
383+ if(FLAGS.use_ipv6):
384+ _execute("sudo ifconfig %s add %s up" % \
385+ (bridge,
386+ net_attrs['cidr_v6']))
387 else:
388 _execute("sudo ifconfig %s up" % bridge)
389 if FLAGS.use_nova_chains:
390@@ -262,6 +272,50 @@
391 _execute(command, addl_env=env)
392
393
394+def update_ra(context, network_id):
395+ network_ref = db.network_get(context, network_id)
396+
397+ conffile = _ra_file(network_ref['bridge'], 'conf')
398+ with open(conffile, 'w') as f:
399+ conf_str = """
400+interface %s
401+{
402+ AdvSendAdvert on;
403+ MinRtrAdvInterval 3;
404+ MaxRtrAdvInterval 10;
405+ prefix %s
406+ {
407+ AdvOnLink on;
408+ AdvAutonomous on;
409+ };
410+};
411+""" % (network_ref['bridge'], network_ref['cidr_v6'])
412+ f.write(conf_str)
413+
414+ # Make sure dnsmasq can actually read it (it setuid()s to "nobody")
415+ os.chmod(conffile, 0644)
416+
417+ pid = _ra_pid_for(network_ref['bridge'])
418+
419+ # if dnsmasq is already running, then tell it to reload
420+ if pid:
421+ out, _err = _execute('cat /proc/%d/cmdline'
422+ % pid, check_exit_code=False)
423+ if conffile in out:
424+ try:
425+ _execute('sudo kill -HUP %d' % pid)
426+ return
427+ except Exception as exc: # pylint: disable-msg=W0703
428+ logging.debug("Hupping radvd threw %s", exc)
429+ else:
430+ logging.debug("Pid %d is stale, relaunching radvd", pid)
431+ command = _ra_cmd(network_ref)
432+ _execute(command)
433+ db.network_update(context, network_id,
434+ {"ra_server":
435+ utils.get_my_linklocal(network_ref['bridge'])})
436+
437+
438 def _host_dhcp(fixed_ip_ref):
439 """Return a host string for an address"""
440 instance_ref = fixed_ip_ref['instance']
441@@ -323,6 +377,15 @@
442 return ''.join(cmd)
443
444
445+def _ra_cmd(net):
446+ """Builds dnsmasq command"""
447+ cmd = ['sudo -E radvd',
448+# ' -u nobody',
449+ ' -C %s' % _ra_file(net['bridge'], 'conf'),
450+ ' -p %s' % _ra_file(net['bridge'], 'pid')]
451+ return ''.join(cmd)
452+
453+
454 def _stop_dnsmasq(network):
455 """Stops the dnsmasq instance for a given network"""
456 pid = _dnsmasq_pid_for(network)
457@@ -344,6 +407,16 @@
458 kind))
459
460
461+def _ra_file(bridge, kind):
462+ """Return path to a pid, leases or conf file for a bridge"""
463+
464+ if not os.path.exists(FLAGS.networks_path):
465+ os.makedirs(FLAGS.networks_path)
466+ return os.path.abspath("%s/nova-ra-%s.%s" % (FLAGS.networks_path,
467+ bridge,
468+ kind))
469+
470+
471 def _dnsmasq_pid_for(bridge):
472 """Returns the pid for prior dnsmasq instance for a bridge
473
474@@ -357,3 +430,18 @@
475 if os.path.exists(pid_file):
476 with open(pid_file, 'r') as f:
477 return int(f.read())
478+
479+
480+def _ra_pid_for(bridge):
481+ """Returns the pid for prior dnsmasq instance for a bridge
482+
483+ Returns None if no pid file exists
484+
485+ If machine has rebooted pid might be incorrect (caller should check)
486+ """
487+
488+ pid_file = _ra_file(bridge, 'pid')
489+
490+ if os.path.exists(pid_file):
491+ with open(pid_file, 'r') as f:
492+ return int(f.read())
493
494=== modified file 'nova/network/manager.py'
495--- nova/network/manager.py 2011-01-10 02:08:54 +0000
496+++ nova/network/manager.py 2011-01-13 00:56:22 +0000
497@@ -82,6 +82,7 @@
498 flags.DEFINE_string('floating_range', '4.4.4.0/24',
499 'Floating IP address block')
500 flags.DEFINE_string('fixed_range', '10.0.0.0/8', 'Fixed IP address block')
501+flags.DEFINE_string('fixed_range_v6', 'fd00::/48', 'Fixed IPv6 address block')
502 flags.DEFINE_integer('cnt_vpn_clients', 5,
503 'Number of addresses reserved for vpn clients')
504 flags.DEFINE_string('network_driver', 'nova.network.linux_net',
505@@ -90,6 +91,9 @@
506 'Whether to update dhcp when fixed_ip is disassociated')
507 flags.DEFINE_integer('fixed_ip_disassociate_timeout', 600,
508 'Seconds after which a deallocated ip is disassociated')
509+
510+flags.DEFINE_bool('use_ipv6', True,
511+ 'use the ipv6')
512 flags.DEFINE_string('network_host', socket.gethostname(),
513 'Network host to use for ip allocation in flat modes')
514 flags.DEFINE_bool('fake_call', False,
515@@ -235,7 +239,7 @@
516 """Get the network host for the current context."""
517 raise NotImplementedError()
518
519- def create_networks(self, context, num_networks, network_size,
520+ def create_networks(self, context, num_networks, network_size, cidr_v6,
521 *args, **kwargs):
522 """Create networks based on parameters."""
523 raise NotImplementedError()
524@@ -321,9 +325,11 @@
525 pass
526
527 def create_networks(self, context, cidr, num_networks, network_size,
528- *args, **kwargs):
529+ cidr_v6, *args, **kwargs):
530 """Create networks based on parameters."""
531 fixed_net = IPy.IP(cidr)
532+ fixed_net_v6 = IPy.IP(cidr_v6)
533+ significant_bits_v6 = 64
534 for index in range(num_networks):
535 start = index * network_size
536 significant_bits = 32 - int(math.log(network_size, 2))
537@@ -336,7 +342,13 @@
538 net['gateway'] = str(project_net[1])
539 net['broadcast'] = str(project_net.broadcast())
540 net['dhcp_start'] = str(project_net[2])
541+
542+ if(FLAGS.use_ipv6):
543+ cidr_v6 = "%s/%s" % (fixed_net_v6[0], significant_bits_v6)
544+ net['cidr_v6'] = cidr_v6
545+
546 network_ref = self.db.network_create_safe(context, net)
547+
548 if network_ref:
549 self._create_fixed_ips(context, network_ref['id'])
550
551@@ -482,12 +494,16 @@
552 network_ref['bridge'])
553
554 def create_networks(self, context, cidr, num_networks, network_size,
555- vlan_start, vpn_start):
556+ vlan_start, vpn_start, cidr_v6):
557 """Create networks based on parameters."""
558 fixed_net = IPy.IP(cidr)
559+ fixed_net_v6 = IPy.IP(cidr_v6)
560+ network_size_v6 = 1 << 64
561+ significant_bits_v6 = 64
562 for index in range(num_networks):
563 vlan = vlan_start + index
564 start = index * network_size
565+ start_v6 = index * network_size_v6
566 significant_bits = 32 - int(math.log(network_size, 2))
567 cidr = "%s/%s" % (fixed_net[start], significant_bits)
568 project_net = IPy.IP(cidr)
569@@ -500,6 +516,13 @@
570 net['dhcp_start'] = str(project_net[3])
571 net['vlan'] = vlan
572 net['bridge'] = 'br%s' % vlan
573+ if(FLAGS.use_ipv6):
574+ cidr_v6 = "%s/%s" % (
575+ fixed_net_v6[start_v6],
576+ significant_bits_v6
577+ )
578+ net['cidr_v6'] = cidr_v6
579+
580 # NOTE(vish): This makes ports unique accross the cloud, a more
581 # robust solution would be to make them unique per ip
582 net['vpn_public_port'] = vpn_start + index
583@@ -538,6 +561,7 @@
584 self.driver.ensure_vlan_bridge(network_ref['vlan'],
585 network_ref['bridge'],
586 network_ref)
587+
588 # NOTE(vish): only ensure this forward if the address hasn't been set
589 # manually.
590 if address == FLAGS.vpn_ip:
591@@ -546,6 +570,8 @@
592 network_ref['vpn_private_address'])
593 if not FLAGS.fake_network:
594 self.driver.update_dhcp(context, network_id)
595+ if(FLAGS.use_ipv6):
596+ self.driver.update_ra(context, network_id)
597
598 @property
599 def _bottom_reserved_ips(self):
600
601=== modified file 'nova/test.py'
602--- nova/test.py 2010-12-17 01:05:54 +0000
603+++ nova/test.py 2011-01-13 00:56:22 +0000
604@@ -156,7 +156,8 @@
605 FLAGS.fixed_range,
606 5, 16,
607 FLAGS.vlan_start,
608- FLAGS.vpn_start)
609+ FLAGS.vpn_start,
610+ FLAGS.fixed_range_v6)
611
612 # emulate some of the mox stuff, we can't use the metaclass
613 # because it screws with our generators
614
615=== modified file 'nova/tests/test_api.py'
616--- nova/tests/test_api.py 2010-12-17 01:05:54 +0000
617+++ nova/tests/test_api.py 2011-01-13 00:56:22 +0000
618@@ -24,6 +24,7 @@
619 import random
620 import StringIO
621 import webob
622+import logging
623
624 from nova import context
625 from nova import flags
626@@ -265,6 +266,72 @@
627
628 return
629
630+ def test_authorize_revoke_security_group_cidr_v6(self):
631+ """
632+ Test that we can add and remove CIDR based rules
633+ to a security group for IPv6
634+ """
635+ self.expect_http()
636+ self.mox.ReplayAll()
637+ user = self.manager.create_user('fake', 'fake', 'fake')
638+ project = self.manager.create_project('fake', 'fake', 'fake')
639+
640+ # At the moment, you need both of these to actually be netadmin
641+ self.manager.add_role('fake', 'netadmin')
642+ project.add_role('fake', 'netadmin')
643+
644+ security_group_name = "".join(random.choice("sdiuisudfsdcnpaqwertasd")
645+ for x in range(random.randint(4, 8)))
646+
647+ group = self.ec2.create_security_group(security_group_name,
648+ 'test group')
649+
650+ self.expect_http()
651+ self.mox.ReplayAll()
652+ group.connection = self.ec2
653+
654+ group.authorize('tcp', 80, 81, '::/0')
655+
656+ self.expect_http()
657+ self.mox.ReplayAll()
658+
659+ rv = self.ec2.get_all_security_groups()
660+ # I don't bother checkng that we actually find it here,
661+ # because the create/delete unit test further up should
662+ # be good enough for that.
663+ for group in rv:
664+ if group.name == security_group_name:
665+ self.assertEquals(len(group.rules), 1)
666+ self.assertEquals(int(group.rules[0].from_port), 80)
667+ self.assertEquals(int(group.rules[0].to_port), 81)
668+ self.assertEquals(len(group.rules[0].grants), 1)
669+ self.assertEquals(str(group.rules[0].grants[0]), '::/0')
670+
671+ self.expect_http()
672+ self.mox.ReplayAll()
673+ group.connection = self.ec2
674+
675+ group.revoke('tcp', 80, 81, '::/0')
676+
677+ self.expect_http()
678+ self.mox.ReplayAll()
679+
680+ self.ec2.delete_security_group(security_group_name)
681+
682+ self.expect_http()
683+ self.mox.ReplayAll()
684+ group.connection = self.ec2
685+
686+ rv = self.ec2.get_all_security_groups()
687+
688+ self.assertEqual(len(rv), 1)
689+ self.assertEqual(rv[0].name, 'default')
690+
691+ self.manager.delete_project(project)
692+ self.manager.delete_user(user)
693+
694+ return
695+
696 def test_authorize_revoke_security_group_foreign_group(self):
697 """
698 Test that we can grant and revoke another security group access
699
700=== modified file 'nova/tests/test_network.py'
701--- nova/tests/test_network.py 2011-01-04 05:23:35 +0000
702+++ nova/tests/test_network.py 2011-01-13 00:56:22 +0000
703@@ -96,6 +96,28 @@
704 self.context.project_id = self.projects[project_num].id
705 self.network.deallocate_fixed_ip(self.context, address)
706
707+ def test_private_ipv6(self):
708+ """Make sure ipv6 is OK"""
709+ if FLAGS.use_ipv6:
710+ instance_ref = self._create_instance(0)
711+ address = self._create_address(0, instance_ref['id'])
712+ network_ref = db.project_get_network(
713+ context.get_admin_context(),
714+ self.context.project_id)
715+ address_v6 = db.instance_get_fixed_address_v6(
716+ context.get_admin_context(),
717+ instance_ref['id'])
718+ self.assertEqual(instance_ref['mac_address'],
719+ utils.to_mac(address_v6))
720+ instance_ref2 = db.fixed_ip_get_instance_v6(
721+ context.get_admin_context(),
722+ address_v6)
723+ self.assertEqual(instance_ref['id'], instance_ref2['id'])
724+ self.assertEqual(address_v6,
725+ utils.to_global_ipv6(
726+ network_ref['cidr_v6'],
727+ instance_ref['mac_address']))
728+
729 def test_public_network_association(self):
730 """Makes sure that we can allocaate a public ip"""
731 # TODO(vish): better way of adding floating ips
732
733=== modified file 'nova/utils.py'
734--- nova/utils.py 2011-01-12 01:32:12 +0000
735+++ nova/utils.py 2011-01-13 00:56:22 +0000
736@@ -30,6 +30,8 @@
737 import sys
738 import time
739 from xml.sax import saxutils
740+import re
741+import netaddr
742
743 from eventlet import event
744 from eventlet import greenthread
745@@ -200,6 +202,53 @@
746 return int(address.split(".")[-1])
747
748
749+def get_my_ip():
750+ """Returns the actual ip of the local machine."""
751+ try:
752+ csock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
753+ csock.connect(('8.8.8.8', 80))
754+ (addr, port) = csock.getsockname()
755+ csock.close()
756+ return addr
757+ except socket.gaierror as ex:
758+ logging.warn(_("Couldn't get IP, using 127.0.0.1 %s"), ex)
759+ return "127.0.0.1"
760+
761+
762+def get_my_linklocal(interface):
763+ try:
764+ if_str = execute("ip -f inet6 -o addr show %s" % interface)
765+ condition = "\s+inet6\s+([0-9a-f:]+/\d+)\s+scope\s+link"
766+ links = [re.search(condition, x) for x in if_str[0].split('\n')]
767+ address = [w.group(1) for w in links if w is not None]
768+ if address[0] is not None:
769+ return address[0]
770+ else:
771+ return 'fe00::'
772+ except IndexError as ex:
773+ logging.warn(_("Couldn't get Link Local IP of %s :%s"), interface, ex)
774+ except ProcessExecutionError as ex:
775+ logging.warn(_("Couldn't get Link Local IP of %s :%s"), interface, ex)
776+ except:
777+ return 'fe00::'
778+
779+
780+def to_global_ipv6(prefix, mac):
781+ mac64 = netaddr.EUI(mac).eui64().words
782+ int_addr = int(''.join(['%02x' % i for i in mac64]), 16)
783+ mac64_addr = netaddr.IPAddress(int_addr)
784+ maskIP = netaddr.IPNetwork(prefix).ip
785+ return (mac64_addr ^ netaddr.IPAddress('::0200:0:0:0') | maskIP).format()
786+
787+
788+def to_mac(ipv6_address):
789+ address = netaddr.IPAddress(ipv6_address)
790+ mask1 = netaddr.IPAddress("::ffff:ffff:ffff:ffff")
791+ mask2 = netaddr.IPAddress("::0200:0:0:0")
792+ mac64 = netaddr.EUI(int(address & mask1 ^ mask2)).words
793+ return ":".join(["%02x" % i for i in mac64[0:3] + mac64[5:8]])
794+
795+
796 def utcnow():
797 """Overridable version of datetime.datetime.utcnow."""
798 if utcnow.override_time:
799
800=== modified file 'nova/virt/libvirt.xml.template'
801--- nova/virt/libvirt.xml.template 2011-01-05 00:22:47 +0000
802+++ nova/virt/libvirt.xml.template 2011-01-13 00:56:22 +0000
803@@ -66,6 +66,7 @@
804 <filterref filter="nova-instance-${name}">
805 <parameter name="IP" value="${ip_address}" />
806 <parameter name="DHCPSERVER" value="${dhcp_server}" />
807+ <parameter name="RASERVER" value="${ra_server}" />
808 #if $getVar('extra_params', False)
809 ${extra_params}
810 #end if
811
812=== modified file 'nova/virt/libvirt_conn.py'
813--- nova/virt/libvirt_conn.py 2011-01-12 19:39:25 +0000
814+++ nova/virt/libvirt_conn.py 2011-01-13 00:56:22 +0000
815@@ -127,6 +127,16 @@
816 return str(net.net()), str(net.netmask())
817
818
819+def _get_net_and_prefixlen(cidr):
820+ net = IPy.IP(cidr)
821+ return str(net.net()), str(net.prefixlen())
822+
823+
824+def _get_ip_version(cidr):
825+ net = IPy.IP(cidr)
826+ return int(net.version())
827+
828+
829 class LibvirtConnection(object):
830
831 def __init__(self, read_only):
832@@ -383,7 +393,6 @@
833 instance['id'],
834 power_state.NOSTATE,
835 'launching')
836-
837 self.nwfilter.setup_basic_filtering(instance)
838 self.firewall_driver.prepare_instance_filter(instance)
839 self._create_image(instance, xml)
840@@ -552,12 +561,16 @@
841 if network_ref['injected']:
842 admin_context = context.get_admin_context()
843 address = db.instance_get_fixed_address(admin_context, inst['id'])
844+ ra_server = network_ref['ra_server']
845+ if not ra_server:
846+ ra_server = "fd00::"
847 with open(FLAGS.injected_network_template) as f:
848 net = f.read() % {'address': address,
849 'netmask': network_ref['netmask'],
850 'gateway': network_ref['gateway'],
851 'broadcast': network_ref['broadcast'],
852- 'dns': network_ref['dns']}
853+ 'dns': network_ref['dns'],
854+ 'ra_server': ra_server}
855 if key or net:
856 if key:
857 LOG.info(_('instance %s: injecting key into image %s'),
858@@ -612,13 +625,30 @@
859 instance['id'])
860 # Assume that the gateway also acts as the dhcp server.
861 dhcp_server = network['gateway']
862-
863+ ra_server = network['ra_server']
864+ if not ra_server:
865+ ra_server = 'fd00::'
866 if FLAGS.allow_project_net_traffic:
867- net, mask = _get_net_and_mask(network['cidr'])
868- extra_params = ("<parameter name=\"PROJNET\" "
869- "value=\"%s\" />\n"
870- "<parameter name=\"PROJMASK\" "
871- "value=\"%s\" />\n") % (net, mask)
872+ if FLAGS.use_ipv6:
873+ net, mask = _get_net_and_mask(network['cidr'])
874+ net_v6, prefixlen_v6 = _get_net_and_prefixlen(
875+ network['cidr_v6'])
876+ extra_params = ("<parameter name=\"PROJNET\" "
877+ "value=\"%s\" />\n"
878+ "<parameter name=\"PROJMASK\" "
879+ "value=\"%s\" />\n"
880+ "<parameter name=\"PROJNETV6\" "
881+ "value=\"%s\" />\n"
882+ "<parameter name=\"PROJMASKV6\" "
883+ "value=\"%s\" />\n") % \
884+ (net, mask, net_v6, prefixlen_v6)
885+ else:
886+ net, mask = _get_net_and_mask(network['cidr'])
887+ extra_params = ("<parameter name=\"PROJNET\" "
888+ "value=\"%s\" />\n"
889+ "<parameter name=\"PROJMASK\" "
890+ "value=\"%s\" />\n") % \
891+ (net, mask)
892 else:
893 extra_params = "\n"
894
895@@ -632,6 +662,7 @@
896 'mac_address': instance['mac_address'],
897 'ip_address': ip_address,
898 'dhcp_server': dhcp_server,
899+ 'ra_server': ra_server,
900 'extra_params': extra_params,
901 'rescue': rescue}
902 if not rescue:
903@@ -889,6 +920,15 @@
904 </rule>
905 </filter>'''
906
907+ def nova_ra_filter(self):
908+ return '''<filter name='nova-allow-ra-server' chain='root'>
909+ <uuid>d707fa71-4fb5-4b27-9ab7-ba5ca19c8804</uuid>
910+ <rule action='accept' direction='inout'
911+ priority='100'>
912+ <icmpv6 srcipaddr='$RASERVER'/>
913+ </rule>
914+ </filter>'''
915+
916 def setup_basic_filtering(self, instance):
917 """Set up basic filtering (MAC, IP, and ARP spoofing protection)"""
918 logging.info('called setup_basic_filtering in nwfilter')
919@@ -913,13 +953,17 @@
920 ['no-mac-spoofing',
921 'no-ip-spoofing',
922 'no-arp-spoofing',
923- 'allow-dhcp-server']))
924+ 'allow-dhcp-server'
925+ ]))
926 self._define_filter(self.nova_base_ipv4_filter)
927 self._define_filter(self.nova_base_ipv6_filter)
928 self._define_filter(self.nova_dhcp_filter)
929+ self._define_filter(self.nova_ra_filter)
930 self._define_filter(self.nova_vpn_filter)
931 if FLAGS.allow_project_net_traffic:
932 self._define_filter(self.nova_project_filter)
933+ if FLAGS.use_ipv6:
934+ self._define_filter(self.nova_project_filter_v6)
935
936 self.static_filters_configured = True
937
938@@ -951,13 +995,13 @@
939
940 def nova_base_ipv6_filter(self):
941 retval = "<filter name='nova-base-ipv6' chain='ipv6'>"
942- for protocol in ['tcp', 'udp', 'icmp']:
943+ for protocol in ['tcp-ipv6', 'udp-ipv6', 'icmpv6']:
944 for direction, action, priority in [('out', 'accept', 399),
945 ('in', 'drop', 400)]:
946 retval += """<rule action='%s' direction='%s' priority='%d'>
947- <%s-ipv6 />
948+ <%s />
949 </rule>""" % (action, direction,
950- priority, protocol)
951+ priority, protocol)
952 retval += '</filter>'
953 return retval
954
955@@ -970,10 +1014,20 @@
956 retval += '</filter>'
957 return retval
958
959+ def nova_project_filter_v6(self):
960+ retval = "<filter name='nova-project-v6' chain='ipv6'>"
961+ for protocol in ['tcp-ipv6', 'udp-ipv6', 'icmpv6']:
962+ retval += """<rule action='accept' direction='inout'
963+ priority='200'>
964+ <%s srcipaddr='$PROJNETV6'
965+ srcipmask='$PROJMASKV6' />
966+ </rule>""" % (protocol)
967+ retval += '</filter>'
968+ return retval
969+
970 def _define_filter(self, xml):
971 if callable(xml):
972 xml = xml()
973-
974 # execute in a native thread and block current greenthread until done
975 tpool.execute(self._conn.nwfilterDefineXML, xml)
976
977@@ -983,7 +1037,6 @@
978 it makes sure the filters for the security groups as well as
979 the base filter are all in place.
980 """
981-
982 if instance['image_id'] == FLAGS.vpn_image_id:
983 base_filter = 'nova-vpn'
984 else:
985@@ -995,11 +1048,15 @@
986 instance_secgroup_filter_children = ['nova-base-ipv4',
987 'nova-base-ipv6',
988 'nova-allow-dhcp-server']
989+ if FLAGS.use_ipv6:
990+ instance_secgroup_filter_children += ['nova-allow-ra-server']
991
992 ctxt = context.get_admin_context()
993
994 if FLAGS.allow_project_net_traffic:
995 instance_filter_children += ['nova-project']
996+ if FLAGS.use_ipv6:
997+ instance_filter_children += ['nova-project-v6']
998
999 for security_group in db.security_group_get_by_instance(ctxt,
1000 instance['id']):
1001@@ -1027,12 +1084,19 @@
1002 security_group = db.security_group_get(context.get_admin_context(),
1003 security_group_id)
1004 rule_xml = ""
1005+ v6protocol = {'tcp': 'tcp-ipv6', 'udp': 'udp-ipv6', 'icmp': 'icmpv6'}
1006 for rule in security_group.rules:
1007 rule_xml += "<rule action='accept' direction='in' priority='300'>"
1008 if rule.cidr:
1009- net, mask = _get_net_and_mask(rule.cidr)
1010- rule_xml += "<%s srcipaddr='%s' srcipmask='%s' " % \
1011- (rule.protocol, net, mask)
1012+ version = _get_ip_version(rule.cidr)
1013+ if(FLAGS.use_ipv6 and version == 6):
1014+ net, prefixlen = _get_net_and_prefixlen(rule.cidr)
1015+ rule_xml += "<%s srcipaddr='%s' srcipmask='%s' " % \
1016+ (v6protocol[rule.protocol], net, prefixlen)
1017+ else:
1018+ net, mask = _get_net_and_mask(rule.cidr)
1019+ rule_xml += "<%s srcipaddr='%s' srcipmask='%s' " % \
1020+ (rule.protocol, net, mask)
1021 if rule.protocol in ['tcp', 'udp']:
1022 rule_xml += "dstportstart='%s' dstportend='%s' " % \
1023 (rule.from_port, rule.to_port)
1024@@ -1047,8 +1111,11 @@
1025
1026 rule_xml += '/>\n'
1027 rule_xml += "</rule>\n"
1028- xml = "<filter name='nova-secgroup-%s' chain='ipv4'>%s</filter>" % \
1029- (security_group_id, rule_xml,)
1030+ xml = "<filter name='nova-secgroup-%s' " % security_group_id
1031+ if(FLAGS.use_ipv6):
1032+ xml += "chain='root'>%s</filter>" % rule_xml
1033+ else:
1034+ xml += "chain='ipv4'>%s</filter>" % rule_xml
1035 return xml
1036
1037 def _instance_filter_name(self, instance):
1038@@ -1077,11 +1144,17 @@
1039 def apply_ruleset(self):
1040 current_filter, _ = self.execute('sudo iptables-save -t filter')
1041 current_lines = current_filter.split('\n')
1042- new_filter = self.modify_rules(current_lines)
1043+ new_filter = self.modify_rules(current_lines, 4)
1044 self.execute('sudo iptables-restore',
1045 process_input='\n'.join(new_filter))
1046+ if(FLAGS.use_ipv6):
1047+ current_filter, _ = self.execute('sudo ip6tables-save -t filter')
1048+ current_lines = current_filter.split('\n')
1049+ new_filter = self.modify_rules(current_lines, 6)
1050+ self.execute('sudo ip6tables-restore',
1051+ process_input='\n'.join(new_filter))
1052
1053- def modify_rules(self, current_lines):
1054+ def modify_rules(self, current_lines, ip_version=4):
1055 ctxt = context.get_admin_context()
1056 # Remove any trace of nova rules.
1057 new_filter = filter(lambda l: 'nova-' not in l, current_lines)
1058@@ -1095,8 +1168,8 @@
1059 if not new_filter[rules_index].startswith(':'):
1060 break
1061
1062- our_chains = [':nova-ipv4-fallback - [0:0]']
1063- our_rules = ['-A nova-ipv4-fallback -j DROP']
1064+ our_chains = [':nova-fallback - [0:0]']
1065+ our_rules = ['-A nova-fallback -j DROP']
1066
1067 our_chains += [':nova-local - [0:0]']
1068 our_rules += ['-A FORWARD -j nova-local']
1069@@ -1106,7 +1179,10 @@
1070 # First, we add instance chains and rules
1071 for instance in self.instances:
1072 chain_name = self._instance_chain_name(instance)
1073- ip_address = self._ip_for_instance(instance)
1074+ if(ip_version == 4):
1075+ ip_address = self._ip_for_instance(instance)
1076+ elif(ip_version == 6):
1077+ ip_address = self._ip_for_instance_v6(instance)
1078
1079 our_chains += [':%s - [0:0]' % chain_name]
1080
1081@@ -1132,13 +1208,19 @@
1082
1083 our_rules += ['-A %s -j %s' % (chain_name, sg_chain_name)]
1084
1085- # Allow DHCP responses
1086- dhcp_server = self._dhcp_server_for_instance(instance)
1087- our_rules += ['-A %s -s %s -p udp --sport 67 --dport 68' %
1088- (chain_name, dhcp_server)]
1089+ if(ip_version == 4):
1090+ # Allow DHCP responses
1091+ dhcp_server = self._dhcp_server_for_instance(instance)
1092+ our_rules += ['-A %s -s %s -p udp --sport 67 --dport 68' %
1093+ (chain_name, dhcp_server)]
1094+ elif(ip_version == 6):
1095+ # Allow RA responses
1096+ ra_server = self._ra_server_for_instance(instance)
1097+ our_rules += ['-A %s -s %s -p icmpv6' %
1098+ (chain_name, ra_server)]
1099
1100 # If nothing matches, jump to the fallback chain
1101- our_rules += ['-A %s -j nova-ipv4-fallback' % (chain_name,)]
1102+ our_rules += ['-A %s -j nova-fallback' % (chain_name,)]
1103
1104 # then, security group chains and rules
1105 for security_group in security_groups:
1106@@ -1151,15 +1233,22 @@
1107
1108 for rule in rules:
1109 logging.info('%r', rule)
1110- args = ['-A', chain_name, '-p', rule.protocol]
1111
1112- if rule.cidr:
1113- args += ['-s', rule.cidr]
1114- else:
1115+ if not rule.cidr:
1116 # Eventually, a mechanism to grant access for security
1117 # groups will turn up here. It'll use ipsets.
1118 continue
1119
1120+ version = _get_ip_version(rule.cidr)
1121+ if version != ip_version:
1122+ continue
1123+
1124+ protocol = rule.protocol
1125+ if version == 6 and rule.protocol == 'icmp':
1126+ protocol = 'icmpv6'
1127+
1128+ args = ['-A', chain_name, '-p', protocol, '-s', rule.cidr]
1129+
1130 if rule.protocol in ['udp', 'tcp']:
1131 if rule.from_port == rule.to_port:
1132 args += ['--dport', '%s' % (rule.from_port,)]
1133@@ -1179,7 +1268,12 @@
1134 icmp_type_arg += '/%s' % icmp_code
1135
1136 if icmp_type_arg:
1137- args += ['-m', 'icmp', '--icmp-type', icmp_type_arg]
1138+ if(ip_version == 4):
1139+ args += ['-m', 'icmp', '--icmp-type',
1140+ icmp_type_arg]
1141+ elif(ip_version == 6):
1142+ args += ['-m', 'icmp6', '--icmpv6-type',
1143+ icmp_type_arg]
1144
1145 args += ['-j ACCEPT']
1146 our_rules += [' '.join(args)]
1147@@ -1205,7 +1299,16 @@
1148 return db.instance_get_fixed_address(context.get_admin_context(),
1149 instance['id'])
1150
1151+ def _ip_for_instance_v6(self, instance):
1152+ return db.instance_get_fixed_address_v6(context.get_admin_context(),
1153+ instance['id'])
1154+
1155 def _dhcp_server_for_instance(self, instance):
1156 network = db.project_get_network(context.get_admin_context(),
1157 instance['project_id'])
1158 return network['gateway']
1159+
1160+ def _ra_server_for_instance(self, instance):
1161+ network = db.project_get_network(context.get_admin_context(),
1162+ instance['project_id'])
1163+ return network['ra_server']
1164
1165=== modified file 'smoketests/admin_smoketests.py'
1166--- smoketests/admin_smoketests.py 2011-01-06 01:51:05 +0000
1167+++ smoketests/admin_smoketests.py 2011-01-13 00:56:22 +0000
1168@@ -43,7 +43,7 @@
1169 # TODO(devamcar): Use random tempfile
1170 ZIP_FILENAME = '/tmp/nova-me-x509.zip'
1171
1172-TEST_PREFIX = 'test%s' % int(random.random()*1000000)
1173+TEST_PREFIX = 'test%s' % int(random.random() * 1000000)
1174 TEST_USERNAME = '%suser' % TEST_PREFIX
1175 TEST_PROJECTNAME = '%sproject' % TEST_PREFIX
1176
1177@@ -96,4 +96,3 @@
1178 if __name__ == "__main__":
1179 suites = {'user': unittest.makeSuite(UserTests)}
1180 sys.exit(base.run_tests(suites))
1181-
1182
1183=== modified file 'smoketests/base.py'
1184--- smoketests/base.py 2010-11-04 22:50:23 +0000
1185+++ smoketests/base.py 2011-01-13 00:56:22 +0000
1186@@ -17,6 +17,7 @@
1187 # under the License.
1188
1189 import boto
1190+import boto_v6
1191 import commands
1192 import httplib
1193 import os
1194@@ -69,6 +70,17 @@
1195 'test.')
1196
1197 parts = self.split_clc_url(clc_url)
1198+ if FLAGS.use_ipv6:
1199+ return boto_v6.connect_ec2(aws_access_key_id=access_key,
1200+ aws_secret_access_key=secret_key,
1201+ is_secure=parts['is_secure'],
1202+ region=RegionInfo(None,
1203+ 'nova',
1204+ parts['ip']),
1205+ port=parts['port'],
1206+ path='/services/Cloud',
1207+ **kwargs)
1208+
1209 return boto.connect_ec2(aws_access_key_id=access_key,
1210 aws_secret_access_key=secret_key,
1211 is_secure=parts['is_secure'],
1212@@ -115,7 +127,8 @@
1213 return True
1214
1215 def upload_image(self, bucket_name, image):
1216- cmd = 'euca-upload-bundle -b %s -m /tmp/%s.manifest.xml' % (bucket_name, image)
1217+ cmd = 'euca-upload-bundle -b '
1218+ cmd += '%s -m /tmp/%s.manifest.xml' % (bucket_name, image)
1219 status, output = commands.getstatusoutput(cmd)
1220 if status != 0:
1221 print '%s -> \n %s' % (cmd, output)
1222@@ -130,6 +143,7 @@
1223 raise Exception(output)
1224 return True
1225
1226+
1227 def run_tests(suites):
1228 argv = FLAGS(sys.argv)
1229
1230@@ -151,4 +165,3 @@
1231 else:
1232 for suite in suites.itervalues():
1233 unittest.TextTestRunner(verbosity=2).run(suite)
1234-
1235
1236=== modified file 'smoketests/flags.py'
1237--- smoketests/flags.py 2010-11-04 22:50:23 +0000
1238+++ smoketests/flags.py 2011-01-13 00:56:22 +0000
1239@@ -33,6 +33,7 @@
1240 # __GLOBAL FLAGS ONLY__
1241 # Define any app-specific flags in their own files, docs at:
1242 # http://code.google.com/p/python-gflags/source/browse/trunk/gflags.py#39
1243+
1244 DEFINE_string('region', 'nova', 'Region to use')
1245 DEFINE_string('test_image', 'ami-tiny', 'Image to use for launch tests')
1246-
1247+DEFINE_string('use_ipv6', True, 'use the ipv6 or not')
1248
1249=== added file 'smoketests/public_network_smoketests.py'
1250--- smoketests/public_network_smoketests.py 1970-01-01 00:00:00 +0000
1251+++ smoketests/public_network_smoketests.py 2011-01-13 00:56:22 +0000
1252@@ -0,0 +1,180 @@
1253+# vim: tabstop=4 shiftwidth=4 softtabstop=4
1254+
1255+# Copyright 2010 United States Government as represented by the
1256+# Administrator of the National Aeronautics and Space Administration.
1257+# All Rights Reserved.
1258+#
1259+# Licensed under the Apache License, Version 2.0 (the "License"); you may
1260+# not use this file except in compliance with the License. You may obtain
1261+# a copy of the License at
1262+#
1263+# http://www.apache.org/licenses/LICENSE-2.0
1264+#
1265+# Unless required by applicable law or agreed to in writing, software
1266+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
1267+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
1268+# License for the specific language governing permissions and limitations
1269+# under the License.
1270+
1271+import commands
1272+import os
1273+import random
1274+import socket
1275+import sys
1276+import time
1277+import unittest
1278+
1279+from smoketests import flags
1280+from smoketests import base
1281+from smoketests import user_smoketests
1282+
1283+#Note that this test should run from
1284+#public network (outside of private network segments)
1285+#Please set EC2_URL correctly
1286+#You should use admin account in this test
1287+
1288+FLAGS = flags.FLAGS
1289+
1290+TEST_PREFIX = 'test%s' % int(random.random() * 1000000)
1291+TEST_BUCKET = '%s_bucket' % TEST_PREFIX
1292+TEST_KEY = '%s_key' % TEST_PREFIX
1293+TEST_KEY2 = '%s_key2' % TEST_PREFIX
1294+TEST_DATA = {}
1295+
1296+
1297+class InstanceTestsFromPublic(user_smoketests.UserSmokeTestCase):
1298+ def test_001_can_create_keypair(self):
1299+ key = self.create_key_pair(self.conn, TEST_KEY)
1300+ self.assertEqual(key.name, TEST_KEY)
1301+
1302+ def test_002_security_group(self):
1303+ security_group_name = "".join(random.choice("sdiuisudfsdcnpaqwertasd")
1304+ for x in range(random.randint(4, 8)))
1305+ group = self.conn.create_security_group(security_group_name,
1306+ 'test group')
1307+ group.connection = self.conn
1308+ group.authorize('tcp', 22, 22, '0.0.0.0/0')
1309+ if FLAGS.use_ipv6:
1310+ group.authorize('tcp', 22, 22, '::/0')
1311+
1312+ reservation = self.conn.run_instances(FLAGS.test_image,
1313+ key_name=TEST_KEY,
1314+ security_groups=[security_group_name],
1315+ instance_type='m1.tiny')
1316+ self.data['security_group_name'] = security_group_name
1317+ self.data['group'] = group
1318+ self.data['instance_id'] = reservation.instances[0].id
1319+
1320+ def test_003_instance_with_group_runs_within_60_seconds(self):
1321+ reservations = self.conn.get_all_instances([self.data['instance_id']])
1322+ instance = reservations[0].instances[0]
1323+ # allow 60 seconds to exit pending with IP
1324+ for x in xrange(60):
1325+ instance.update()
1326+ if instance.state == u'running':
1327+ break
1328+ time.sleep(1)
1329+ else:
1330+ self.fail('instance failed to start')
1331+ ip = reservations[0].instances[0].private_dns_name
1332+ self.failIf(ip == '0.0.0.0')
1333+ self.data['private_ip'] = ip
1334+ if FLAGS.use_ipv6:
1335+ ipv6 = reservations[0].instances[0].dns_name_v6
1336+ self.failIf(ipv6 is None)
1337+ self.data['ip_v6'] = ipv6
1338+
1339+ def test_004_can_ssh_to_ipv6(self):
1340+ if FLAGS.use_ipv6:
1341+ for x in xrange(20):
1342+ try:
1343+ conn = self.connect_ssh(
1344+ self.data['ip_v6'], TEST_KEY)
1345+ conn.close()
1346+ except Exception as ex:
1347+ print ex
1348+ time.sleep(1)
1349+ else:
1350+ break
1351+ else:
1352+ self.fail('could not ssh to instance')
1353+
1354+ def test_012_can_create_instance_with_keypair(self):
1355+ if 'instance_id' in self.data:
1356+ self.conn.terminate_instances([self.data['instance_id']])
1357+ reservation = self.conn.run_instances(FLAGS.test_image,
1358+ key_name=TEST_KEY,
1359+ instance_type='m1.tiny')
1360+ self.assertEqual(len(reservation.instances), 1)
1361+ self.data['instance_id'] = reservation.instances[0].id
1362+
1363+ def test_013_instance_runs_within_60_seconds(self):
1364+ reservations = self.conn.get_all_instances([self.data['instance_id']])
1365+ instance = reservations[0].instances[0]
1366+ # allow 60 seconds to exit pending with IP
1367+ for x in xrange(60):
1368+ instance.update()
1369+ if instance.state == u'running':
1370+ break
1371+ time.sleep(1)
1372+ else:
1373+ self.fail('instance failed to start')
1374+ ip = reservations[0].instances[0].private_dns_name
1375+ self.failIf(ip == '0.0.0.0')
1376+ self.data['private_ip'] = ip
1377+ if FLAGS.use_ipv6:
1378+ ipv6 = reservations[0].instances[0].dns_name_v6
1379+ self.failIf(ipv6 is None)
1380+ self.data['ip_v6'] = ipv6
1381+
1382+ def test_014_can_not_ping_private_ip(self):
1383+ for x in xrange(4):
1384+ # ping waits for 1 second
1385+ status, output = commands.getstatusoutput(
1386+ 'ping -c1 %s' % self.data['private_ip'])
1387+ if status == 0:
1388+ self.fail('can ping private ip from public network')
1389+ if FLAGS.use_ipv6:
1390+ status, output = commands.getstatusoutput(
1391+ 'ping6 -c1 %s' % self.data['ip_v6'])
1392+ if status == 0:
1393+ self.fail('can ping ipv6 from public network')
1394+ else:
1395+ pass
1396+
1397+ def test_015_can_not_ssh_to_private_ip(self):
1398+ for x in xrange(1):
1399+ try:
1400+ conn = self.connect_ssh(self.data['private_ip'], TEST_KEY)
1401+ conn.close()
1402+ except Exception:
1403+ time.sleep(1)
1404+ else:
1405+ self.fail('can ssh for ipv4 address from public network')
1406+
1407+ if FLAGS.use_ipv6:
1408+ for x in xrange(1):
1409+ try:
1410+ conn = self.connect_ssh(
1411+ self.data['ip_v6'], TEST_KEY)
1412+ conn.close()
1413+ except Exception:
1414+ time.sleep(1)
1415+ else:
1416+ self.fail('can ssh for ipv6 address from public network')
1417+
1418+ def test_999_tearDown(self):
1419+ self.delete_key_pair(self.conn, TEST_KEY)
1420+ security_group_name = self.data['security_group_name']
1421+ group = self.data['group']
1422+ if group:
1423+ group.revoke('tcp', 22, 22, '0.0.0.0/0')
1424+ if FLAGS.use_ipv6:
1425+ group.revoke('tcp', 22, 22, '::/0')
1426+ self.conn.delete_security_group(security_group_name)
1427+ if 'instance_id' in self.data:
1428+ self.conn.terminate_instances([self.data['instance_id']])
1429+
1430+if __name__ == "__main__":
1431+ suites = {'instance': unittest.makeSuite(InstanceTestsFromPublic)}
1432+ sys.exit(base.run_tests(suites))
1433
1434=== modified file 'smoketests/user_smoketests.py'
1435--- smoketests/user_smoketests.py 2011-01-06 01:51:05 +0000
1436+++ smoketests/user_smoketests.py 2011-01-13 00:56:22 +0000
1437@@ -45,7 +45,7 @@
1438 flags.DEFINE_string('bundle_image', 'openwrt-x86-ext2.image',
1439 'Local image file to use for bundling tests')
1440
1441-TEST_PREFIX = 'test%s' % int (random.random()*1000000)
1442+TEST_PREFIX = 'test%s' % int(random.random() * 1000000)
1443 TEST_BUCKET = '%s_bucket' % TEST_PREFIX
1444 TEST_KEY = '%s_key' % TEST_PREFIX
1445 TEST_GROUP = '%s_group' % TEST_PREFIX
1446@@ -80,7 +80,7 @@
1447
1448 def test_006_can_register_kernel(self):
1449 kernel_id = self.conn.register_image('%s/%s.manifest.xml' %
1450- (TEST_BUCKET, FLAGS.bundle_kernel))
1451+ (TEST_BUCKET, FLAGS.bundle_kernel))
1452 self.assert_(kernel_id is not None)
1453 self.data['kernel_id'] = kernel_id
1454
1455@@ -92,7 +92,7 @@
1456 time.sleep(1)
1457 else:
1458 print image.state
1459- self.assert_(False) # wasn't available within 10 seconds
1460+ self.assert_(False) # wasn't available within 10 seconds
1461 self.assert_(image.type == 'machine')
1462
1463 for i in xrange(10):
1464@@ -101,7 +101,7 @@
1465 break
1466 time.sleep(1)
1467 else:
1468- self.assert_(False) # wasn't available within 10 seconds
1469+ self.assert_(False) # wasn't available within 10 seconds
1470 self.assert_(kernel.type == 'kernel')
1471
1472 def test_008_can_describe_image_attribute(self):
1473@@ -152,14 +152,17 @@
1474 for x in xrange(60):
1475 instance.update()
1476 if instance.state == u'running':
1477- break
1478+ break
1479 time.sleep(1)
1480 else:
1481 self.fail('instance failed to start')
1482 ip = reservations[0].instances[0].private_dns_name
1483 self.failIf(ip == '0.0.0.0')
1484 self.data['private_ip'] = ip
1485- print self.data['private_ip']
1486+ if FLAGS.use_ipv6:
1487+ ipv6 = reservations[0].instances[0].dns_name_v6
1488+ self.failIf(ipv6 is None)
1489+ self.data['ip_v6'] = ipv6
1490
1491 def test_004_can_ping_private_ip(self):
1492 for x in xrange(120):
1493@@ -171,6 +174,16 @@
1494 else:
1495 self.fail('could not ping instance')
1496
1497+ if FLAGS.use_ipv6:
1498+ for x in xrange(120):
1499+ # ping waits for 1 second
1500+ status, output = commands.getstatusoutput(
1501+ 'ping6 -c1 %s' % self.data['ip_v6'])
1502+ if status == 0:
1503+ break
1504+ else:
1505+ self.fail('could not ping instance')
1506+
1507 def test_005_can_ssh_to_private_ip(self):
1508 for x in xrange(30):
1509 try:
1510@@ -183,6 +196,19 @@
1511 else:
1512 self.fail('could not ssh to instance')
1513
1514+ if FLAGS.use_ipv6:
1515+ for x in xrange(30):
1516+ try:
1517+ conn = self.connect_ssh(
1518+ self.data['ip_v6'], TEST_KEY)
1519+ conn.close()
1520+ except Exception:
1521+ time.sleep(1)
1522+ else:
1523+ break
1524+ else:
1525+ self.fail('could not ssh to instance v6')
1526+
1527 def test_006_can_allocate_elastic_ip(self):
1528 result = self.conn.allocate_address()
1529 self.assertTrue(hasattr(result, 'public_ip'))
1530@@ -388,7 +414,6 @@
1531 raise Exception("Timeout")
1532 time.sleep(1)
1533
1534-
1535 def test_999_tearDown(self):
1536 self.conn.delete_key_pair(TEST_KEY)
1537 self.conn.delete_security_group(TEST_GROUP)