OpenStack Compute (nova)

Merge lp:~openstack-gd/nova/libvirt-multinic-nova into lp:~hudson-openstack/nova/trunk

  1. libvirt-multinic-nova
  2. Merge into trunk
Proposed by Ilya Alekseyev
Status: Merged
Approved by: Paul Voccio
Approved revision: 826
Merged at revision: 876
Proposed branch: lp:~openstack-gd/nova/libvirt-multinic-nova
Merge into: lp:~hudson-openstack/nova/trunk
Diff against target: 692 lines (+260/-164)
6 files modified
nova/tests/test_virt.py (+4/-2)
nova/utils.py (+9/-5)
nova/virt/driver.py (+1/-1)
nova/virt/interfaces.template (+13/-12)
nova/virt/libvirt.xml.template (+12/-9)
nova/virt/libvirt_conn.py (+221/-135)
To merge this branch: bzr merge lp:~openstack-gd/nova/libvirt-multinic-nova
Reviewer Review Type Date Requested Status
Trey Morris (community) Needs Fixing
Paul Voccio (community) Approve
Matt Dietz (community) Approve
Nachi Ueno (community) Needs Fixing
Vish Ishaya Pending
Joshua McKenty Pending
Sandy Walsh Pending
Review via email: mp+54589@code.launchpad.net

This proposal supersedes a proposal from 2011-03-17.

Description of the change

libvirt driver multi_nic support. In this phase libvirt can work with and without multi_nic support, as in multi_nic support for xenapi: https://code.launchpad.net/~tr3buchet/nova/xs_multi_nic/+merge/53458

To post a comment you must log in.
Revision history for this message
Sandy Walsh (sandy-walsh) wrote : Posted in a previous version of this proposal

149 if not network_info:

180 if _m is unused, you can just use _

182 if network_ref['injected']:
if 'injected' in network_ref: might be safer (no KeyError if missing)

or better
if not 'injected' in network_ref:
    continue

and pull the indent in the for rest

183 do we need to make the context inside the loop? can this be done outside?

186-188 can be replaced with ra_server = network_ref.get('ra_server', "fd00::"

The 'eth##' will be incremented each time ... even if injected, is that what you wanted?

253 if not ...

263 commented out code?

264 seems to overwrite 262?

351 353 commented out code?

354 if ra_servers:

398 should be like 416

450, 459, 482 use append()?

review: Needs Fixing
Revision history for this message
Ilya Alekseyev (ilyaalekseyev) wrote : Posted in a previous version of this proposal

Sandy, thank you very much for good review. Please look at my only comment below.

> The 'eth##' will be incremented each time ... even if injected, is that what
> you wanted?
>
yes

lp:~openstack-gd/nova/libvirt-multinic-nova updated
821. By Eldar Nugaev

migration gateway_v6 to network_info

822. By Eldar Nugaev

small fix

Revision history for this message
Nachi Ueno (nati-ueno) wrote :
Download full text (136.0 KiB)

Unit test fails, would you please check?

Failure
    runTest ERROR

======================================================================
ERROR: test_403 (nova.tests.api.openstack.test_limits.WsgiLimiterProxyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nova/libvirt-multinic-nova/nova/tests/api/openstack/test_limits.py", line 578, in test_403
    delay, error = self.proxy.check_for_delay("GET", "/delayed")
  File "/home/nova/libvirt-multinic-nova/nova/api/openstack/limits.py", line 351, in check_for_delay
    conn.request("POST", "/", body, headers)
  File "/home/nova/libvirt-multinic-nova/nova/tests/api/openstack/test_limits.py", line 510, in request
    resp = str(req.get_response(self.app))
  File "/usr/lib/pymodules/python2.6/webob/request.py", line 918, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/pymodules/python2.6/webob/request.py", line 886, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/pymodules/python2.6/webob/dec.py", line 159, in __call__
    return resp(environ, start_response)
  File "/usr/lib/pymodules/python2.6/webob/exc.py", line 248, in __call__
    return self.generate_response(environ, start_response)
  File "/usr/lib/pymodules/python2.6/webob/exc.py", line 239, in generate_response
    content_type=content_type
  File "/usr/lib/pymodules/python2.6/webob/response.py", line 94, in __init__
    "You cannot set the body to a unicode value without a charset")
TypeError: You cannot set the body to a unicode value without a charset

======================================================================
ERROR: test_response_to_delays (nova.tests.api.openstack.test_limits.WsgiLimiterTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/nova/libvirt-multinic-nova/nova/tests/api/openstack/test_limits.py", line 455, in test_response_to_delays
    delay = self._request("GET", "/delayed")
  File "/home/nova/libvirt-multinic-nova/nova/tests/api/openstack/test_limits.py", line 426, in _request
    response = request.get_response(self.app)
  File "/usr/lib/pymodules/python2.6/webob/request.py", line 918, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/pymodules/python2.6/webob/request.py", line 886, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/pymodules/python2.6/webob/dec.py", line 159, in __call__
    return resp(environ, start_response)
  File "/usr/lib/pymodules/python2.6/webob/exc.py", line 248, in __call__
    return self.generate_response(environ, start_response)
  File "/usr/lib/pymodules/python2.6/webob/exc.py", line 239, in generate_response
    content_type=content_type
  File "/usr/lib/pymodules/python2.6/webob/response.py", line 94, in __init__
    "You cannot set the body to a unicode value without a charset")
TypeError: You cannot set the body to a unicode value without a charset

======================================================================
ERROR: test_response_to_d...

Revision history for this message
Nachi Ueno (nati-ueno) wrote :

I found the versions of some libraries causes the unit test fails.
Please ignore above comment.

Revision history for this message
Andrey Brindeyev (abrindeyev) wrote :

> I found the versions of some libraries causes the unit test fails.

Nachi,

Thanks for letting us know!

Could you please document here which libraries (and their versions) caused problems on your side?

Thanks in advance, Andrey.

Revision history for this message
Nachi Ueno (nati-ueno) wrote :
Download full text (11.2 KiB)

Hi Andrey,

We need following package when we use nova.sh.
sudo apt-get install python-webob
sudo apt-get install python-novaclient
# Sorry again,this is not by your branch.

Unit test was OK.
But I can not run instances.

When I run instance, I got a error message.
2011-03-24 07:01:07,634 ERROR nova.exception [-] Uncaught exception
(nova.exception): TRACE: Traceback (most recent call last):
(nova.exception): TRACE: File "/home/nova/libvirt-multinic-nova/nova/exception.py", line 120, in _wrap
(nova.exception): TRACE: return f(*args, **kw)
(nova.exception): TRACE: File "/home/nova/libvirt-multinic-nova/nova/virt/libvirt_conn.py", line 466, in spawn
(nova.exception): TRACE: xml = self.to_xml(instance, network_info)
(nova.exception): TRACE: File "/home/nova/libvirt-multinic-nova/nova/virt/libvirt_conn.py", line 811, in to_xml
(nova.exception): TRACE: network_info = _get_network_info(instance)
(nova.exception): TRACE: File "/home/nova/libvirt-multinic-nova/nova/virt/libvirt_conn.py", line 193, in _get_network_info
(nova.exception): TRACE: 'ip6s': [ip6_dict(ip) for ip in network_ips]}
(nova.exception): TRACE: File "/home/nova/libvirt-multinic-nova/nova/virt/libvirt_conn.py", line 178, in ip6_dict
(nova.exception): TRACE: "ip": utils.to_global_ipv6(prefix, mac),
(nova.exception): TRACE: File "/home/nova/libvirt-multinic-nova/nova/utils.py", line 316, in to_global_ipv6
(nova.exception): TRACE: maskIP = netaddr.IPNetwork(prefix).ip
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/netaddr/ip/__init__.py", line 688, in __init__
(nova.exception): TRACE: prefix, suffix = addr.split('/')
(nova.exception): TRACE: AttributeError: 'NoneType' object has no attribute 'split'
(nova.exception): TRACE:
2011-03-24 07:01:07,645 ERROR nova.compute.manager [MAD68M-0SV70DSREX1ZM admin admin] Instance '1' failed to spawn. Is virtualization enabled in the BIOS?
(nova.compute.manager): TRACE: Traceback (most recent call last):
(nova.compute.manager): TRACE: File "/home/nova/libvirt-multinic-nova/nova/compute/manager.py", line 225, in run_instance
(nova.compute.manager): TRACE: self.driver.spawn(instance_ref)
(nova.compute.manager): TRACE: File "/home/nova/libvirt-multinic-nova/nova/exception.py", line 126, in _wrap
(nova.compute.manager): TRACE: raise Error(str(e))
(nova.compute.manager): TRACE: Error: 'NoneType' object has no attribute 'split'
(nova.compute.manager): TRACE:
libvir: QEMU error : Domain not found: no domain with matching name 'instance-00000001'

I found that _get_network_info should check FLAGS.use_ipv6.

156 def _get_network_info(instance):
 157 #TODO(ilyaalekseyev) If we will keep this function
 158 # we should cache network_info
 159 admin_context = context.get_admin_context()
 160 ip_addresses = db.fixed_ip_get_all_by_instance(admin_context,
 161 instance['id'])
 162
 163 networks = db.network_get_all_by_instance(admin_context,
 164 instance['id'])
 165 network_info = []
 166
 167 def i...

review: Needs Fixing
Revision history for this message
Ilya Alekseyev (ilyaalekseyev) wrote :

Nachi, thank you much for review.
Fixing.

lp:~openstack-gd/nova/libvirt-multinic-nova updated
823. By Ilya Alekseyev

couple of bugs fixed

824. By Eldar Nugaev

pep8 clearing

825. By Ilya Alekseyev

trunk merged. conflicts resolved

Revision history for this message
Matt Dietz (cerberus) wrote :

Nachi: could you use http://paste.openstack.org/ in the future for large tracebacks or test failure lists? It would make things a lot easier to read overall

Revision history for this message
Ilya Alekseyev (ilyaalekseyev) wrote :

Hi Nachi!

We checked that VMs is starting and available via ipv4.
Thank you very much for you help.

Revision history for this message
Matt Dietz (cerberus) wrote :

I'd like to see a lot more unit tests covering this functionality. The only addition I see is you updated one test to check for a mac address. Considering the substantial change this represents, going forward untested is pretty scary. Do you think you could throw a few tests in there?

Nit-picks / Style junk:

42 + return (mac64_addr ^ netaddr.IPAddress('::0200:0:0:0') | maskIP).\
43 + format()

I'd like to see format() indented at least 8 spaces for readability's sake.

146 + #TODO(ilyaalekseyev) If we will keep this function
147 + # we should cache network_info

You should be consistent with your spacing. I'd move TODO over one space

review: Needs Fixing
lp:~openstack-gd/nova/libvirt-multinic-nova updated
826. By Ilya Alekseyev

style and spacing fixed

Revision history for this message
Eldar Nugaev (reldan) wrote :

Hi Matt.
Thank you too much for your review.

We're synchronizing our changes with Trey Morris, he's working on add multi-nic support for xen https://code.launchpad.net/~tr3buchet/nova/xs_multi_nic/+merge/53458. So we did our changes by the same way and at this moment it is not changing functionality. Existed unittest already covered our changes. But we agree with you. Let us merge our changes to the trunk before FF and create additional tests as a bug fix.

Next blueprint https://blueprints.launchpad.net/nova/+spec/nova-multi-nic contains changes in db model.

> I'd like to see a lot more unit tests covering this functionality. The only
> addition I see is you updated one test to check for a mac address. Considering
> the substantial change this represents, going forward untested is pretty
> scary. Do you think you could throw a few tests in there?
>
> Nit-picks / Style junk:
>
> 42 + return (mac64_addr ^ netaddr.IPAddress('::0200:0:0:0') |
> maskIP).\
> 43 + format()
>
> I'd like to see format() indented at least 8 spaces for readability's sake.
>
> 146 + #TODO(ilyaalekseyev) If we will keep this function
> 147 + # we should cache network_info
>
> You should be consistent with your spacing. I'd move TODO over one space

Revision history for this message
Eldar Nugaev (reldan) wrote :

Bug for additional tests - https://bugs.launchpad.net/nova/+bug/742053

Revision history for this message
Matt Dietz (cerberus) wrote :

Alright Eldar, that works for me.

review: Approve
Revision history for this message
Paul Voccio (pvo) wrote :

ok

review: Approve
Revision history for this message
Trey Morris (tr3buchet) wrote :

looks good to me. You'll want to add 'rxtx_cap' and 'broadcast' to network info in _get_network_info() as in vmops.

review: Needs Fixing
Revision history for this message
Trey Morris (tr3buchet) wrote :

or it's already merged.. holy hell.

Revision history for this message
Ilya Alekseyev (ilyaalekseyev) wrote :

Thank you Trey.
I've created new bug, we will fix it.
https://bugs.launchpad.net/nova/+bug/742128

Revision history for this message
Salvatore Orlando (salvatore-orlando) wrote :

Hi Ilya,

I have updated xenapi code for doing network injection, using interfaces.template.
I've seen that you populate the template in the following way:

net = str(Template(ifc_template, searchList=[{'interfaces': nets}]))

I tried to do the same thing, but Cheetah was complaining that it could not find 'use_ipv6'. I looked at the template and found out that use_ipv6 is a first-level variable:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

#for $ifc in $interfaces
auto ${ifc.name}
iface ${ifc.name} inet static
        address ${ifc.address}
        netmask ${ifc.netmask}
        broadcast ${ifc.broadcast}
        gateway ${ifc.gateway}
        dns-nameservers ${ifc.dns}

--> #if $use_ipv6 <--
iface ${ifc.name} inet6 static
    address ${ifc.address_v6}
    netmask ${ifc.netmask_v6}
    gateway ${ifc.gateway_v6}
#end if

#end for

I therefore updated my code in the following way (and it worked):

        net = str(template(template_data,
                        searchList=[{'interfaces': interfaces_info,
                                     'use_ipv6': FLAGS.use_ipv6}]))

What am I missing? Do you have any hint about why I neeeded to pass use_ipv6 in this way?

Thanks,
Salvatore

Revision history for this message
Ilya Alekseyev (ilyaalekseyev) wrote :

Hi Salvatore!

I think you did it in right way.
I kept use_ipv6 as first level variable bacause it depends on global flag.

I'll fix my code, asap. Thank you.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'nova/tests/test_virt.py'
--- nova/tests/test_virt.py 2011-03-24 09:52:41 +0000
+++ nova/tests/test_virt.py 2011-03-24 20:02:31 +0000
@@ -796,7 +796,8 @@
796796
797 instance_ref = db.instance_create(self.context,797 instance_ref = db.instance_create(self.context,
798 {'user_id': 'fake',798 {'user_id': 'fake',
799 'project_id': 'fake'})799 'project_id': 'fake',
800 'mac_address': '00:A0:C9:14:C8:29'})
800 inst_id = instance_ref['id']801 inst_id = instance_ref['id']
801802
802 ip = '10.11.12.13'803 ip = '10.11.12.13'
@@ -813,7 +814,8 @@
813 'instance_id': instance_ref['id']})814 'instance_id': instance_ref['id']})
814815
815 def _ensure_all_called():816 def _ensure_all_called():
816 instance_filter = 'nova-instance-%s' % instance_ref['name']817 instance_filter = 'nova-instance-%s-%s' % (instance_ref['name'],
818 '00A0C914C829')
817 secgroup_filter = 'nova-secgroup-%s' % self.security_group['id']819 secgroup_filter = 'nova-secgroup-%s' % self.security_group['id']
818 for required in [secgroup_filter, 'allow-dhcp-server',820 for required in [secgroup_filter, 'allow-dhcp-server',
819 'no-arp-spoofing', 'no-ip-spoofing',821 'no-arp-spoofing', 'no-ip-spoofing',
820822
=== modified file 'nova/utils.py'
--- nova/utils.py 2011-03-24 18:51:38 +0000
+++ nova/utils.py 2011-03-24 20:02:31 +0000
@@ -311,11 +311,15 @@
311311
312312
313def to_global_ipv6(prefix, mac):313def to_global_ipv6(prefix, mac):
314 mac64 = netaddr.EUI(mac).eui64().words314 try:
315 int_addr = int(''.join(['%02x' % i for i in mac64]), 16)315 mac64 = netaddr.EUI(mac).eui64().words
316 mac64_addr = netaddr.IPAddress(int_addr)316 int_addr = int(''.join(['%02x' % i for i in mac64]), 16)
317 maskIP = netaddr.IPNetwork(prefix).ip317 mac64_addr = netaddr.IPAddress(int_addr)
318 return (mac64_addr ^ netaddr.IPAddress('::0200:0:0:0') | maskIP).format()318 maskIP = netaddr.IPNetwork(prefix).ip
319 return (mac64_addr ^ netaddr.IPAddress('::0200:0:0:0') | maskIP).\
320 format()
321 except TypeError:
322 raise TypeError(_("Bad mac for to_global_ipv6: %s") % mac)
319323
320324
321def to_mac(ipv6_address):325def to_mac(ipv6_address):
322326
=== modified file 'nova/virt/driver.py'
--- nova/virt/driver.py 2011-03-24 02:15:41 +0000
+++ nova/virt/driver.py 2011-03-24 20:02:31 +0000
@@ -61,7 +61,7 @@
61 """Return a list of InstanceInfo for all registered VMs"""61 """Return a list of InstanceInfo for all registered VMs"""
62 raise NotImplementedError()62 raise NotImplementedError()
6363
64 def spawn(self, instance):64 def spawn(self, instance, network_info=None):
65 """Launch a VM for the specified instance"""65 """Launch a VM for the specified instance"""
66 raise NotImplementedError()66 raise NotImplementedError()
6767
6868
=== modified file 'nova/virt/interfaces.template'
--- nova/virt/interfaces.template 2011-03-02 01:12:47 +0000
+++ nova/virt/interfaces.template 2011-03-24 20:02:31 +0000
@@ -5,19 +5,20 @@
5auto lo5auto lo
6iface lo inet loopback6iface lo inet loopback
77
8# The primary network interface8#for $ifc in $interfaces
9auto eth09auto ${ifc.name}
10iface eth0 inet static10iface ${ifc.name} inet static
11 address ${address}11 address ${ifc.address}
12 netmask ${netmask}12 netmask ${ifc.netmask}
13 broadcast ${broadcast}13 broadcast ${ifc.broadcast}
14 gateway ${gateway}14 gateway ${ifc.gateway}
15 dns-nameservers ${dns}15 dns-nameservers ${ifc.dns}
1616
17#if $use_ipv617#if $use_ipv6
18iface eth0 inet6 static18iface ${ifc.name} inet6 static
19 address ${address_v6}19 address ${ifc.address_v6}
20 netmask ${netmask_v6}20 netmask ${ifc.netmask_v6}
21 gateway ${gateway_v6}21 gateway ${ifc.gateway_v6}
22#end if22#end if
2323
24#end for
2425
=== modified file 'nova/virt/libvirt.xml.template'
--- nova/virt/libvirt.xml.template 2011-03-02 01:12:47 +0000
+++ nova/virt/libvirt.xml.template 2011-03-24 20:02:31 +0000
@@ -69,21 +69,24 @@
69 </disk>69 </disk>
70 #end if70 #end if
71#end if71#end if
72
73#for $nic in $nics
72 <interface type='bridge'>74 <interface type='bridge'>
73 <source bridge='${bridge_name}'/>75 <source bridge='${nic.bridge_name}'/>
74 <mac address='${mac_address}'/>76 <mac address='${nic.mac_address}'/>
75 <!-- <model type='virtio'/> CANT RUN virtio network right now -->77 <!-- <model type='virtio'/> CANT RUN virtio network right now -->
76 <filterref filter="nova-instance-${name}">78 <filterref filter="nova-instance-${name}-${nic.id}">
77 <parameter name="IP" value="${ip_address}" />79 <parameter name="IP" value="${nic.ip_address}" />
78 <parameter name="DHCPSERVER" value="${dhcp_server}" /> 80 <parameter name="DHCPSERVER" value="${nic.dhcp_server}" />
79#if $getVar('extra_params', False)81#if $getVar('nic.extra_params', False)
80 ${extra_params}82 ${nic.extra_params}
81#end if83#end if
82#if $getVar('gateway_v6', False)84#if $getVar('nic.gateway_v6', False)
83 <parameter name="RASERVER" value="${gateway_v6}" />85 <parameter name="RASERVER" value="${nic.gateway_v6}" />
84#end if86#end if
85 </filterref>87 </filterref>
86 </interface>88 </interface>
89#end for
8790
88 <!-- The order is significant here. File must be defined first -->91 <!-- The order is significant here. File must be defined first -->
89 <serial type="file">92 <serial type="file">
9093
=== modified file 'nova/virt/libvirt_conn.py'
--- nova/virt/libvirt_conn.py 2011-03-24 17:53:54 +0000
+++ nova/virt/libvirt_conn.py 2011-03-24 20:02:31 +0000
@@ -153,6 +153,51 @@
153 return int(net.version())153 return int(net.version())
154154
155155
156def _get_network_info(instance):
157 # TODO(adiantum) If we will keep this function
158 # we should cache network_info
159 admin_context = context.get_admin_context()
160
161 ip_addresses = db.fixed_ip_get_all_by_instance(admin_context,
162 instance['id'])
163
164 networks = db.network_get_all_by_instance(admin_context,
165 instance['id'])
166 network_info = []
167
168 def ip_dict(ip):
169 return {
170 "ip": ip.address,
171 "netmask": network["netmask"],
172 "enabled": "1"}
173
174 def ip6_dict(ip6):
175 prefix = ip6.network.cidr_v6
176 mac = instance.mac_address
177 return {
178 "ip": utils.to_global_ipv6(prefix, mac),
179 "netmask": ip6.network.netmask_v6,
180 "gateway": ip6.network.gateway_v6,
181 "enabled": "1"}
182
183 for network in networks:
184 network_ips = [ip for ip in ip_addresses
185 if ip.network_id == network.id]
186
187 mapping = {
188 'label': network['label'],
189 'gateway': network['gateway'],
190 'mac': instance.mac_address,
191 'dns': [network['dns']],
192 'ips': [ip_dict(ip) for ip in network_ips]}
193
194 if FLAGS.use_ipv6:
195 mapping['ip6s'] = [ip6_dict(ip) for ip in network_ips]
196
197 network_info.append((network, mapping))
198 return network_info
199
200
156class LibvirtConnection(driver.ComputeDriver):201class LibvirtConnection(driver.ComputeDriver):
157202
158 def __init__(self, read_only):203 def __init__(self, read_only):
@@ -444,16 +489,18 @@
444 def poll_rescued_instances(self, timeout):489 def poll_rescued_instances(self, timeout):
445 pass490 pass
446491
492 # NOTE(ilyaalekseyev): Implementation like in multinics
493 # for xenapi(tr3buchet)
447 @exception.wrap_exception494 @exception.wrap_exception
448 def spawn(self, instance):495 def spawn(self, instance, network_info=None):
449 xml = self.to_xml(instance)496 xml = self.to_xml(instance, network_info)
450 db.instance_set_state(context.get_admin_context(),497 db.instance_set_state(context.get_admin_context(),
451 instance['id'],498 instance['id'],
452 power_state.NOSTATE,499 power_state.NOSTATE,
453 'launching')500 'launching')
454 self.firewall_driver.setup_basic_filtering(instance)501 self.firewall_driver.setup_basic_filtering(instance, network_info)
455 self.firewall_driver.prepare_instance_filter(instance)502 self.firewall_driver.prepare_instance_filter(instance, network_info)
456 self._create_image(instance, xml)503 self._create_image(instance, xml, network_info)
457 self._conn.createXML(xml, 0)504 self._conn.createXML(xml, 0)
458 LOG.debug(_("instance %s: is running"), instance['name'])505 LOG.debug(_("instance %s: is running"), instance['name'])
459 self.firewall_driver.apply_instance_filter(instance)506 self.firewall_driver.apply_instance_filter(instance)
@@ -609,7 +656,14 @@
609 utils.execute('truncate', target, '-s', "%dG" % local_gb)656 utils.execute('truncate', target, '-s', "%dG" % local_gb)
610 # TODO(vish): should we format disk by default?657 # TODO(vish): should we format disk by default?
611658
612 def _create_image(self, inst, libvirt_xml, suffix='', disk_images=None):659 def _create_image(self, inst, libvirt_xml, suffix='', disk_images=None,
660 network_info=None):
661 if not network_info:
662 network_info = _get_network_info(inst)
663
664 if not suffix:
665 suffix = ''
666
613 # syntactic nicety667 # syntactic nicety
614 def basepath(fname='', suffix=suffix):668 def basepath(fname='', suffix=suffix):
615 return os.path.join(FLAGS.instances_path,669 return os.path.join(FLAGS.instances_path,
@@ -685,28 +739,35 @@
685739
686 key = str(inst['key_data'])740 key = str(inst['key_data'])
687 net = None741 net = None
688 network_ref = db.network_get_by_instance(context.get_admin_context(),742
689 inst['id'])743 nets = []
690 if network_ref['injected']:744 ifc_template = open(FLAGS.injected_network_template).read()
691 admin_context = context.get_admin_context()745 ifc_num = -1
692 address = db.instance_get_fixed_address(admin_context, inst['id'])746 admin_context = context.get_admin_context()
747 for (network_ref, mapping) in network_info:
748 ifc_num += 1
749
750 if not 'injected' in network_ref:
751 continue
752
753 address = mapping['ips'][0]['ip']
693 address_v6 = None754 address_v6 = None
694 if FLAGS.use_ipv6:755 if FLAGS.use_ipv6:
695 address_v6 = db.instance_get_fixed_address_v6(admin_context,756 address_v6 = mapping['ip6s'][0]['ip']
696 inst['id'])757 net_info = {'name': 'eth%d' % ifc_num,
697758 'address': address,
698 interfaces_info = {'address': address,759 'netmask': network_ref['netmask'],
699 'netmask': network_ref['netmask'],760 'gateway': network_ref['gateway'],
700 'gateway': network_ref['gateway'],761 'broadcast': network_ref['broadcast'],
701 'broadcast': network_ref['broadcast'],762 'dns': network_ref['dns'],
702 'dns': network_ref['dns'],763 'address_v6': address_v6,
703 'address_v6': address_v6,764 'gateway_v6': network_ref['gateway_v6'],
704 'gateway_v6': network_ref['gateway_v6'],765 'netmask_v6': network_ref['netmask_v6'],
705 'netmask_v6': network_ref['netmask_v6'],766 'use_ipv6': FLAGS.use_ipv6}
706 'use_ipv6': FLAGS.use_ipv6}767 nets.append(net_info)
707768
708 net = str(Template(self.interfaces_xml,769 net = str(Template(ifc_template, searchList=[{'interfaces': nets}]))
709 searchList=[interfaces_info]))770
710 if key or net:771 if key or net:
711 inst_name = inst['name']772 inst_name = inst['name']
712 img_id = inst.image_id773 img_id = inst.image_id
@@ -728,20 +789,11 @@
728 if FLAGS.libvirt_type == 'uml':789 if FLAGS.libvirt_type == 'uml':
729 utils.execute('sudo', 'chown', 'root', basepath('disk'))790 utils.execute('sudo', 'chown', 'root', basepath('disk'))
730791
731 def to_xml(self, instance, rescue=False):792 def _get_nic_for_xml(self, network, mapping):
732 # TODO(termie): cache?
733 LOG.debug(_('instance %s: starting toXML method'), instance['name'])
734 network = db.network_get_by_instance(context.get_admin_context(),
735 instance['id'])
736 # FIXME(vish): stick this in db
737 instance_type = instance['instance_type']
738 # instance_type = test.INSTANCE_TYPES[instance_type]
739 instance_type = instance_types.get_instance_type(instance_type)
740 ip_address = db.instance_get_fixed_address(context.get_admin_context(),
741 instance['id'])
742 # Assume that the gateway also acts as the dhcp server.793 # Assume that the gateway also acts as the dhcp server.
743 dhcp_server = network['gateway']794 dhcp_server = network['gateway']
744 gateway_v6 = network['gateway_v6']795 gateway_v6 = network['gateway_v6']
796 mac_id = mapping['mac'].replace(':', '')
745797
746 if FLAGS.allow_project_net_traffic:798 if FLAGS.allow_project_net_traffic:
747 if FLAGS.use_ipv6:799 if FLAGS.use_ipv6:
@@ -766,6 +818,38 @@
766 (net, mask)818 (net, mask)
767 else:819 else:
768 extra_params = "\n"820 extra_params = "\n"
821
822 result = {
823 'id': mac_id,
824 'bridge_name': network['bridge'],
825 'mac_address': mapping['mac'],
826 'ip_address': mapping['ips'][0]['ip'],
827 'dhcp_server': dhcp_server,
828 'extra_params': extra_params,
829 }
830
831 if gateway_v6:
832 result['gateway_v6'] = gateway_v6 + "/128"
833
834 return result
835
836 def to_xml(self, instance, rescue=False, network_info=None):
837 # TODO(termie): cache?
838 LOG.debug(_('instance %s: starting toXML method'), instance['name'])
839
840 # TODO(adiantum) remove network_info creation code
841 # when multinics will be completed
842 if not network_info:
843 network_info = _get_network_info(instance)
844
845 nics = []
846 for (network, mapping) in network_info:
847 nics.append(self._get_nic_for_xml(network,
848 mapping))
849 # FIXME(vish): stick this in db
850 instance_type_name = instance['instance_type']
851 instance_type = instance_types.get_instance_type(instance_type_name)
852
769 if FLAGS.use_cow_images:853 if FLAGS.use_cow_images:
770 driver_type = 'qcow2'854 driver_type = 'qcow2'
771 else:855 else:
@@ -777,17 +861,11 @@
777 instance['name']),861 instance['name']),
778 'memory_kb': instance_type['memory_mb'] * 1024,862 'memory_kb': instance_type['memory_mb'] * 1024,
779 'vcpus': instance_type['vcpus'],863 'vcpus': instance_type['vcpus'],
780 'bridge_name': network['bridge'],
781 'mac_address': instance['mac_address'],
782 'ip_address': ip_address,
783 'dhcp_server': dhcp_server,
784 'extra_params': extra_params,
785 'rescue': rescue,864 'rescue': rescue,
786 'local': instance_type['local_gb'],865 'local': instance_type['local_gb'],
787 'driver_type': driver_type}866 'driver_type': driver_type,
867 'nics': nics}
788868
789 if gateway_v6:
790 xml_info['gateway_v6'] = gateway_v6 + "/128"
791 if not rescue:869 if not rescue:
792 if instance['kernel_id']:870 if instance['kernel_id']:
793 xml_info['kernel'] = xml_info['basepath'] + "/kernel"871 xml_info['kernel'] = xml_info['basepath'] + "/kernel"
@@ -800,7 +878,6 @@
800 xml = str(Template(self.libvirt_xml, searchList=[xml_info]))878 xml = str(Template(self.libvirt_xml, searchList=[xml_info]))
801 LOG.debug(_('instance %s: finished toXML method'),879 LOG.debug(_('instance %s: finished toXML method'),
802 instance['name'])880 instance['name'])
803
804 return xml881 return xml
805882
806 def get_info(self, instance_name):883 def get_info(self, instance_name):
@@ -1316,7 +1393,7 @@
13161393
13171394
1318class FirewallDriver(object):1395class FirewallDriver(object):
1319 def prepare_instance_filter(self, instance):1396 def prepare_instance_filter(self, instance, network_info=None):
1320 """Prepare filters for the instance.1397 """Prepare filters for the instance.
13211398
1322 At this point, the instance isn't running yet."""1399 At this point, the instance isn't running yet."""
@@ -1350,7 +1427,7 @@
1350 the security group."""1427 the security group."""
1351 raise NotImplementedError()1428 raise NotImplementedError()
13521429
1353 def setup_basic_filtering(self, instance):1430 def setup_basic_filtering(self, instance, network_info=None):
1354 """Create rules to block spoofing and allow dhcp.1431 """Create rules to block spoofing and allow dhcp.
13551432
1356 This gets called when spawning an instance, before1433 This gets called when spawning an instance, before
@@ -1359,11 +1436,6 @@
1359 """1436 """
1360 raise NotImplementedError()1437 raise NotImplementedError()
13611438
1362 def _gateway_v6_for_instance(self, instance):
1363 network = db.network_get_by_instance(context.get_admin_context(),
1364 instance['id'])
1365 return network['gateway_v6']
1366
13671439
1368class NWFilterFirewall(FirewallDriver):1440class NWFilterFirewall(FirewallDriver):
1369 """1441 """
@@ -1455,10 +1527,13 @@
1455 </rule>1527 </rule>
1456 </filter>'''1528 </filter>'''
14571529
1458 def setup_basic_filtering(self, instance):1530 def setup_basic_filtering(self, instance, network_info=None):
1459 """Set up basic filtering (MAC, IP, and ARP spoofing protection)"""1531 """Set up basic filtering (MAC, IP, and ARP spoofing protection)"""
1460 logging.info('called setup_basic_filtering in nwfilter')1532 logging.info('called setup_basic_filtering in nwfilter')
14611533
1534 if not network_info:
1535 network_info = _get_network_info(instance)
1536
1462 if self.handle_security_groups:1537 if self.handle_security_groups:
1463 # No point in setting up a filter set that we'll be overriding1538 # No point in setting up a filter set that we'll be overriding
1464 # anyway.1539 # anyway.
@@ -1467,9 +1542,11 @@
1467 logging.info('ensuring static filters')1542 logging.info('ensuring static filters')
1468 self._ensure_static_filters()1543 self._ensure_static_filters()
14691544
1470 instance_filter_name = self._instance_filter_name(instance)1545 for (network, mapping) in network_info:
1471 self._define_filter(self._filter_container(instance_filter_name,1546 nic_id = mapping['mac'].replace(':', '')
1472 ['nova-base']))1547 instance_filter_name = self._instance_filter_name(instance, nic_id)
1548 self._define_filter(self._filter_container(instance_filter_name,
1549 ['nova-base']))
14731550
1474 def _ensure_static_filters(self):1551 def _ensure_static_filters(self):
1475 if self.static_filters_configured:1552 if self.static_filters_configured:
@@ -1560,48 +1637,60 @@
1560 # Nothing to do1637 # Nothing to do
1561 pass1638 pass
15621639
1563 def prepare_instance_filter(self, instance):1640 def prepare_instance_filter(self, instance, network_info=None):
1564 """1641 """
1565 Creates an NWFilter for the given instance. In the process,1642 Creates an NWFilter for the given instance. In the process,
1566 it makes sure the filters for the security groups as well as1643 it makes sure the filters for the security groups as well as
1567 the base filter are all in place.1644 the base filter are all in place.
1568 """1645 """
1646 if not network_info:
1647 network_info = _get_network_info(instance)
1569 if instance['image_id'] == FLAGS.vpn_image_id:1648 if instance['image_id'] == FLAGS.vpn_image_id:
1570 base_filter = 'nova-vpn'1649 base_filter = 'nova-vpn'
1571 else:1650 else:
1572 base_filter = 'nova-base'1651 base_filter = 'nova-base'
15731652
1574 instance_filter_name = self._instance_filter_name(instance)1653 ctxt = context.get_admin_context()
1575 instance_secgroup_filter_name = '%s-secgroup' % (instance_filter_name,)1654
1576 instance_filter_children = [base_filter, instance_secgroup_filter_name]1655 instance_secgroup_filter_name = \
1656 '%s-secgroup' % (self._instance_filter_name(instance))
1657 #% (instance_filter_name,)
1658
1577 instance_secgroup_filter_children = ['nova-base-ipv4',1659 instance_secgroup_filter_children = ['nova-base-ipv4',
1578 'nova-base-ipv6',1660 'nova-base-ipv6',
1579 'nova-allow-dhcp-server']1661 'nova-allow-dhcp-server']
1580 if FLAGS.use_ipv6:1662
1581 gateway_v6 = self._gateway_v6_for_instance(instance)1663 for security_group in \
1582 if gateway_v6:1664 db.security_group_get_by_instance(ctxt, instance['id']):
1583 instance_secgroup_filter_children += ['nova-allow-ra-server']
1584
1585 ctxt = context.get_admin_context()
1586
1587 if FLAGS.allow_project_net_traffic:
1588 instance_filter_children += ['nova-project']
1589 if FLAGS.use_ipv6:
1590 instance_filter_children += ['nova-project-v6']
1591
1592 for security_group in db.security_group_get_by_instance(ctxt,
1593 instance['id']):
15941665
1595 self.refresh_security_group_rules(security_group['id'])1666 self.refresh_security_group_rules(security_group['id'])
15961667
1597 instance_secgroup_filter_children += [('nova-secgroup-%s' %1668 instance_secgroup_filter_children += [('nova-secgroup-%s' %
1598 security_group['id'])]1669 security_group['id'])]
15991670
1600 self._define_filter(1671 self._define_filter(
1601 self._filter_container(instance_secgroup_filter_name,1672 self._filter_container(instance_secgroup_filter_name,
1602 instance_secgroup_filter_children))1673 instance_secgroup_filter_children))
16031674
1604 self._define_filter(1675 for (network, mapping) in network_info:
1676 nic_id = mapping['mac'].replace(':', '')
1677 instance_filter_name = self._instance_filter_name(instance, nic_id)
1678 instance_filter_children = \
1679 [base_filter, instance_secgroup_filter_name]
1680
1681 if FLAGS.use_ipv6:
1682 gateway_v6 = network['gateway_v6']
1683
1684 if gateway_v6:
1685 instance_secgroup_filter_children += \
1686 ['nova-allow-ra-server']
1687
1688 if FLAGS.allow_project_net_traffic:
1689 instance_filter_children += ['nova-project']
1690 if FLAGS.use_ipv6:
1691 instance_filter_children += ['nova-project-v6']
1692
1693 self._define_filter(
1605 self._filter_container(instance_filter_name,1694 self._filter_container(instance_filter_name,
1606 instance_filter_children))1695 instance_filter_children))
16071696
@@ -1649,8 +1738,10 @@
1649 xml += "chain='ipv4'>%s</filter>" % rule_xml1738 xml += "chain='ipv4'>%s</filter>" % rule_xml
1650 return xml1739 return xml
16511740
1652 def _instance_filter_name(self, instance):1741 def _instance_filter_name(self, instance, nic_id=None):
1653 return 'nova-instance-%s' % instance['name']1742 if not nic_id:
1743 return 'nova-instance-%s' % (instance['name'])
1744 return 'nova-instance-%s-%s' % (instance['name'], nic_id)
16541745
16551746
1656class IptablesFirewallDriver(FirewallDriver):1747class IptablesFirewallDriver(FirewallDriver):
@@ -1665,9 +1756,11 @@
1665 self.iptables.ipv6['filter'].add_chain('sg-fallback')1756 self.iptables.ipv6['filter'].add_chain('sg-fallback')
1666 self.iptables.ipv6['filter'].add_rule('sg-fallback', '-j DROP')1757 self.iptables.ipv6['filter'].add_rule('sg-fallback', '-j DROP')
16671758
1668 def setup_basic_filtering(self, instance):1759 def setup_basic_filtering(self, instance, network_info=None):
1669 """Use NWFilter from libvirt for this."""1760 """Use NWFilter from libvirt for this."""
1670 return self.nwfilter.setup_basic_filtering(instance)1761 if not network_info:
1762 network_info = _get_network_info(instance)
1763 return self.nwfilter.setup_basic_filtering(instance, network_info)
16711764
1672 def apply_instance_filter(self, instance):1765 def apply_instance_filter(self, instance):
1673 """No-op. Everything is done in prepare_instance_filter"""1766 """No-op. Everything is done in prepare_instance_filter"""
@@ -1681,29 +1774,40 @@
1681 LOG.info(_('Attempted to unfilter instance %s which is not '1774 LOG.info(_('Attempted to unfilter instance %s which is not '
1682 'filtered'), instance['id'])1775 'filtered'), instance['id'])
16831776
1684 def prepare_instance_filter(self, instance):1777 def prepare_instance_filter(self, instance, network_info=None):
1778 if not network_info:
1779 network_info = _get_network_info(instance)
1685 self.instances[instance['id']] = instance1780 self.instances[instance['id']] = instance
1686 self.add_filters_for_instance(instance)1781 self.add_filters_for_instance(instance, network_info)
1687 self.iptables.apply()1782 self.iptables.apply()
16881783
1689 def add_filters_for_instance(self, instance):1784 def add_filters_for_instance(self, instance, network_info=None):
1785 if not network_info:
1786 network_info = _get_network_info(instance)
1690 chain_name = self._instance_chain_name(instance)1787 chain_name = self._instance_chain_name(instance)
16911788
1692 self.iptables.ipv4['filter'].add_chain(chain_name)1789 self.iptables.ipv4['filter'].add_chain(chain_name)
1693 ipv4_address = self._ip_for_instance(instance)1790
1694 self.iptables.ipv4['filter'].add_rule('local',1791 ips_v4 = [ip['ip'] for (_, mapping) in network_info
1695 '-d %s -j $%s' %1792 for ip in mapping['ips']]
1696 (ipv4_address, chain_name))1793
1794 for ipv4_address in ips_v4:
1795 self.iptables.ipv4['filter'].add_rule('local',
1796 '-d %s -j $%s' %
1797 (ipv4_address, chain_name))
16971798
1698 if FLAGS.use_ipv6:1799 if FLAGS.use_ipv6:
1699 self.iptables.ipv6['filter'].add_chain(chain_name)1800 self.iptables.ipv6['filter'].add_chain(chain_name)
1700 ipv6_address = self._ip_for_instance_v6(instance)1801 ips_v6 = [ip['ip'] for (_, mapping) in network_info
1701 self.iptables.ipv6['filter'].add_rule('local',1802 for ip in mapping['ip6s']]
1702 '-d %s -j $%s' %1803
1703 (ipv6_address,1804 for ipv6_address in ips_v6:
1704 chain_name))1805 self.iptables.ipv6['filter'].add_rule('local',
17051806 '-d %s -j $%s' %
1706 ipv4_rules, ipv6_rules = self.instance_rules(instance)1807 (ipv6_address,
1808 chain_name))
1809
1810 ipv4_rules, ipv6_rules = self.instance_rules(instance, network_info)
17071811
1708 for rule in ipv4_rules:1812 for rule in ipv4_rules:
1709 self.iptables.ipv4['filter'].add_rule(chain_name, rule)1813 self.iptables.ipv4['filter'].add_rule(chain_name, rule)
@@ -1719,7 +1823,9 @@
1719 if FLAGS.use_ipv6:1823 if FLAGS.use_ipv6:
1720 self.iptables.ipv6['filter'].remove_chain(chain_name)1824 self.iptables.ipv6['filter'].remove_chain(chain_name)
17211825
1722 def instance_rules(self, instance):1826 def instance_rules(self, instance, network_info=None):
1827 if not network_info:
1828 network_info = _get_network_info(instance)
1723 ctxt = context.get_admin_context()1829 ctxt = context.get_admin_context()
17241830
1725 ipv4_rules = []1831 ipv4_rules = []
@@ -1733,28 +1839,36 @@
1733 ipv4_rules += ['-m state --state ESTABLISHED,RELATED -j ACCEPT']1839 ipv4_rules += ['-m state --state ESTABLISHED,RELATED -j ACCEPT']
1734 ipv6_rules += ['-m state --state ESTABLISHED,RELATED -j ACCEPT']1840 ipv6_rules += ['-m state --state ESTABLISHED,RELATED -j ACCEPT']
17351841
1736 dhcp_server = self._dhcp_server_for_instance(instance)1842 dhcp_servers = [network['gateway'] for (network, _m) in network_info]
1737 ipv4_rules += ['-s %s -p udp --sport 67 --dport 68 '1843
1738 '-j ACCEPT' % (dhcp_server,)]1844 for dhcp_server in dhcp_servers:
1845 ipv4_rules.append('-s %s -p udp --sport 67 --dport 68 '
1846 '-j ACCEPT' % (dhcp_server,))
17391847
1740 #Allow project network traffic1848 #Allow project network traffic
1741 if FLAGS.allow_project_net_traffic:1849 if FLAGS.allow_project_net_traffic:
1742 cidr = self._project_cidr_for_instance(instance)1850 cidrs = [network['cidr'] for (network, _m) in network_info]
1743 ipv4_rules += ['-s %s -j ACCEPT' % (cidr,)]1851 for cidr in cidrs:
1852 ipv4_rules.append('-s %s -j ACCEPT' % (cidr,))
17441853
1745 # We wrap these in FLAGS.use_ipv6 because they might cause1854 # We wrap these in FLAGS.use_ipv6 because they might cause
1746 # a DB lookup. The other ones are just list operations, so1855 # a DB lookup. The other ones are just list operations, so
1747 # they're not worth the clutter.1856 # they're not worth the clutter.
1748 if FLAGS.use_ipv6:1857 if FLAGS.use_ipv6:
1749 # Allow RA responses1858 # Allow RA responses
1750 gateway_v6 = self._gateway_v6_for_instance(instance)1859 gateways_v6 = [network['gateway_v6'] for (network, _) in
1751 if gateway_v6:1860 network_info]
1752 ipv6_rules += ['-s %s/128 -p icmpv6 -j ACCEPT' % (gateway_v6,)]1861 for gateway_v6 in gateways_v6:
1862 ipv6_rules.append(
1863 '-s %s/128 -p icmpv6 -j ACCEPT' % (gateway_v6,))
17531864
1754 #Allow project network traffic1865 #Allow project network traffic
1755 if FLAGS.allow_project_net_traffic:1866 if FLAGS.allow_project_net_traffic:
1756 cidrv6 = self._project_cidrv6_for_instance(instance)1867 cidrv6s = [network['cidr_v6'] for (network, _m)
1757 ipv6_rules += ['-s %s -j ACCEPT' % (cidrv6,)]1868 in network_info]
1869
1870 for cidrv6 in cidrv6s:
1871 ipv6_rules.append('-s %s -j ACCEPT' % (cidrv6,))
17581872
1759 security_groups = db.security_group_get_by_instance(ctxt,1873 security_groups = db.security_group_get_by_instance(ctxt,
1760 instance['id'])1874 instance['id'])
@@ -1836,31 +1950,3 @@
18361950
1837 def _instance_chain_name(self, instance):1951 def _instance_chain_name(self, instance):
1838 return 'inst-%s' % (instance['id'],)1952 return 'inst-%s' % (instance['id'],)
1839
1840 def _ip_for_instance(self, instance):
1841 return db.instance_get_fixed_address(context.get_admin_context(),
1842 instance['id'])
1843
1844 def _ip_for_instance_v6(self, instance):
1845 return db.instance_get_fixed_address_v6(context.get_admin_context(),
1846 instance['id'])
1847
1848 def _dhcp_server_for_instance(self, instance):
1849 network = db.network_get_by_instance(context.get_admin_context(),
1850 instance['id'])
1851 return network['gateway']
1852
1853 def _gateway_v6_for_instance(self, instance):
1854 network = db.network_get_by_instance(context.get_admin_context(),
1855 instance['id'])
1856 return network['gateway_v6']
1857
1858 def _project_cidr_for_instance(self, instance):
1859 network = db.network_get_by_instance(context.get_admin_context(),
1860 instance['id'])
1861 return network['cidr']
1862
1863 def _project_cidrv6_for_instance(self, instance):
1864 network = db.network_get_by_instance(context.get_admin_context(),
1865 instance['id'])
1866 return network['cidr_v6']