OpenStack Compute (nova)

Fail to assign private fixed ip with multi_host

Bug #1065004 reported by Mate Lakat on 2012-10-10

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	OpenStack Compute (nova)	Fix Released	High	Brian Elliott	OpenStack Compute (nova) 2013.1 "grizzly"
	Folsom	Fix Released	High	Vish Ishaya	OpenStack Compute (nova) 2012.2.1

Bug Description

The dnsmasq hostsfile does not contain the newly created hosts, when you launch the instances.
Problem since:
https://review.openstack.org/#/c/14174/
https://github.com/openstack/nova/commit/5fd7a9dba127bae812333196a5fa48a933212aeb

0.) See my relevant configuration entries:
stack@DevStackOSDomU:~/devstack$ cat /etc/nova/nova.conf |
grep -e compute_scheduler_driver -e multi_host -e network_manager
compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler
multi_host=True
network_manager=nova.network.manager.FlatDHCPManager

1.) See that there is no dnsmasq running
stack@DevStackOSDomU:~/devstack$ ps ax | grep dnsmasq
12742 pts/19 S+ 0:00 grep dnsmasq

2.) Fire up an instance
stack@DevStackOSDomU:~/devstack$ . openrc admin
stack@DevStackOSDomU:~/devstack$ nova boot --image=tty --flavor=m1.tiny sampleserver
.
.
.
| id | 6e90a769-0bdf-42bd-86a7-da776700bf6a |
.
.
.

4.) And look at the DHCP database - it is empty!!!:
stack@DevStackOSDomU:~/devstack$ ps ax | grep dnsmasq
... --dhcp-hostsfile=/opt/stack/data/nova/networks/nova-xenbr1.conf ...
stack@DevStackOSDomU:~/devstack$ cat /opt/stack/data/nova/networks/nova-xenbr1.conf
stack@DevStackOSDomU:~/devstack$ wc /opt/stack/data/nova/networks/nova-xenbr1.conf
0 0 0 /opt/stack/data/nova/networks/nova-xenbr1.conf

5.) Restart nova-network
stack@DevStackOSDomU:~/devstack$ pkill -HUP -f "nova-network"
stack@DevStackOSDomU:~/devstack$ NL=`echo -ne '\015'`
stack@DevStackOSDomU:~/devstack$ screen -S stack -p n-net -X stuff "cd /opt/stack/nova && /usr/local/bin/nova-network$NL"

6.) See, that dhcp entries are there now:
stack@DevStackOSDomU:~/devstack$ cat /opt/stack/data/nova/networks/nova-xenbr1.conf
fa:16:3e:46:6a:be,sampleserver.novalocal,10.0.0.2
stack@DevStackOSDomU:~/devstack$ wc /opt/stack/data/nova/networks/nova-xenbr1.conf
0 1 49 /opt/stack/data/nova/networks/nova-xenbr1.conf

See original description

Mate Lakat (mate-lakat) on 2012-10-10

Changed in nova:
assignee:	nobody → Mate Lakat (mate-lakat)

Johannes Erdfelt (johannes.erdfelt) on 2012-10-10

summary:

- xenapi - Fail to assing floating ip
+ xenapi - Fail to assign floating ip

Revision history for this message

Armando Migliaccio (armando-migliaccio) wrote on 2012-10-10: Re: xenapi - Fail to assign floating ip

If that's of any help, I personally don't see this in my DevStack instance; I have just built a new instance and run exercise tests against trunk. floating_ips pass just fine. I am running against XCP 1.1 and XCP 1.5

Revision history for this message

Mate Lakat (mate-lakat) wrote on 2012-10-15:

So my setup is using multi_host. It turned out, that even the fixed ips are not allocated. It turned out, that assembling the dhcp lease file depends on the host field. See nova/db/sqlalchemy/api.py:

2196 def network_get_associated_fixed_ips(
....
2223 if host:
2224 query = query.filter(models.Instance.host == host)

And I think it is causing me a headache. If I restart nova-network, everything works, because on startup, the host is already filled in.

Armando, are you using multi_host / custom scheduler?

I am using:
network_manager=nova.network.manager.FlatDHCPManager
compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler

Mate Lakat (mate-lakat) on 2012-10-15

summary:	- xenapi - Fail to assign floating ip + Fail to assign private ip
description:	updated

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-10-15: Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/14464

Changed in nova:
status:	New → In Progress

Mate Lakat (mate-lakat) on 2012-10-16

summary:

- Fail to assign private ip
+ Fail to assign private fixed ip with multi_host

Mate Lakat (mate-lakat) on 2012-10-16

description:

updated

John Garbutt (johngarbutt) on 2012-10-16

Changed in nova:
importance:	Undecided → High

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-10-16:

Fix proposed to branch: master
Review: https://review.openstack.org/14488

Changed in nova:
assignee:	Mate Lakat (mate-lakat) → Brian Elliott (belliott)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-10-16: Fix merged to nova (master)

Reviewed: https://review.openstack.org/14488
Committed: http://github.com/openstack/nova/commit/2649f14673f8ef5ca257583f1cdf5fe57d4734b9
Submitter: Jenkins
Branch: master

commit 2649f14673f8ef5ca257583f1cdf5fe57d4734b9
Author: Brian Elliott <email address hidden>
Date: Tue Oct 16 15:58:20 2012 +0000

Set host prior to allocating network information.

Make sure 'host' field on the instance is set before allocating
network information.

bug 1065004

Change-Id: I7b929fd50a057c8c69a0b83e07cdc5cd9cb55b45

Changed in nova:
status:	In Progress → Fix Committed

Vish Ishaya (vishvananda) on 2012-10-22

tags:

added: folsom-backport-potential

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-10-24: Fix proposed to nova (stable/folsom)

Fix proposed to branch: stable/folsom
Review: https://review.openstack.org/14757

Chuck Short (zulcss) on 2012-10-24

tags:

removed: folsom-backport-potential

Thierry Carrez (ttx) on 2012-11-21

Changed in nova:
milestone:	none → grizzly-1
status:	Fix Committed → Fix Released

Revision history for this message

Vish Ishaya (vishvananda) wrote on 2012-11-21:

Folsom changed squashed into https://review.openstack.org/#/c/16677/

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2012-11-27: Fix merged to nova (stable/folsom)

Reviewed: https://review.openstack.org/16677
Committed: http://github.com/openstack/nova/commit/284f6ea64b0837981a24251825b19abef354a9a8
Submitter: Jenkins
Branch: stable/folsom

commit 284f6ea64b0837981a24251825b19abef354a9a8
Author: Brian Elliott <email address hidden>
Date: Mon Oct 8 18:35:14 2012 +0000

Set instance host field after resource claim

Set the 'host' field on the instance after the resource tracker
on the compute node has accepted the build.

    The field is set after resources are confirmed to be available while the
    COMPUTE_RESOURCES_SEMAPHORE is held. The semaphore ensures the
    resources usage values will be consistent even if the
    update_available_resource periodic task audit runs.

bug 1060255

(cherry picked from commit 5fd7a9dba127bae812333196a5fa48a933212aeb)

Also includes the following bugfix to the original patch:

Set host prior to allocating network information.

Make sure 'host' field on the instance is set before allocating
network information.

bug 1065004

(cherry picked from commit 2649f14673f8ef5ca257583f1cdf5fe57d4734b9)
Change-Id: I92105ec14924960ac8ef7ca8c810783085314e10

Thierry Carrez (ttx) on 2013-04-04