cloud-init fails with "Unknown network_data link type: dvs"

I've marked nova as an 'affects'.
This is really nova's bug. I've submitted a merge proposal to stop this behavior from continuing at

 https://review.openstack.org/#/c/400883/2

The nova changes seem like a good idea.

In order for cloud-init to work with existing openstack deployments, does it make sense to modify sources/helpers/openstack.py to simply default to 'phy' rather than raising a ValueError for unknown link types?

Lars, yes. I'd considered that, but as it is cloud-init is raising a ValueError saying "I dont know what this type is". And then basically not going on. If we change it to assume 'phy', then it will silently leave that block of code and fail somewhere else less obvious when it tries to configure this unknown link type as if it were a 'phy'.

I'll throw up a merge proposal for the DVS type and if you can ack that i'll merge it.
Also, thank you for the ACK on the openstack.

Scott,

Would you be willing to consider adding the “dvs” type to the list of physical types in openstack.py? This is the change I made and it appears to have fixed the issue. It will also allow us to more quickly get a fix via an official channel (download newer version of cloud-init) rather than waiting for our OpenStack vendor to provide a new release of their distribution (which would take some time).

Best regards,

Alain Swanson | Lead Engineer, Cloud Networking & Security | Private Cloud Enablement
1801 Park View Drive | Shoreview, MN 55126 | 612/760-5889 | <email address hidden><mailto:<email address hidden>>

On Apr 3, 2017, at 9:02 AM, Scott Moser <<email address hidden><mailto:<email address hidden>>> wrote:

Lars, yes. I'd considered that, but as it is cloud-init is raising a
ValueError saying "I dont know what this type is". And then basically
not going on. If we change it to assume 'phy', then it will silently
leave that block of code and fail somewhere else less obvious when it
tries to configure this unknown link type as if it were a 'phy'.

I'll throw up a merge proposal for the DVS type and if you can ack that i'll merge it.
Also, thank you for the ACK on the openstack.

--
You received this bug notification because you are subscribed to a
duplicate bug report (1677846).
https://bugs.launchpad.net/bugs/1674946

Title:
 cloud-init fails with "Unknown network_data link type: dvs"

Status in cloud-init:
 Incomplete
Status in OpenStack Compute (nova):
 New

Bug description:
 When booting an OpenStack instance, cloud-init fails with:

 [ 33.307325] cloud-init[445]: Cloud-init v. 0.7.9 running 'init-local' at Mon, 20 Mar 2017 14:42:58 +0000. Up 31.06 seconds.
 [ 33.368434] cloud-init[445]: 2017-03-20 14:43:00,779 - util.py[WARNING]: failed stage init-local
 [ 33.449886] cloud-init[445]: failed run of stage init-local
 [ 33.490863] cloud-init[445]: ------------------------------------------------------------
 [ 33.542214] cloud-init[445]: Traceback (most recent call last):
 [ 33.585204] cloud-init[445]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 513, in status_wrapper
 [ 33.654579] cloud-init[445]: ret = functor(name, args)
 [ 33.696372] cloud-init[445]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 269, in main_init
 [ 33.755593] cloud-init[445]: init.apply_network_config(bring_up=bool(mode != sources.DSMODE_LOCAL))
 [ 33.809124] cloud-init[445]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 622, in apply_network_config
 [ 33.847161] cloud-init[445]: netcfg, src = self._find_networking_config()
 [ 33.876562] cloud-init[445]: File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 609, in _find_networking_config
 [ 33.916335] cloud-init[445]: if self.datasource and hasattr(self.datasource, 'network_config'):
 [ 33.956207] cloud-init[445]: File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceConfigDrive.py", line 147, in network_config
 [ 34.008213] cloud-init[445]: self.network_json, known_macs=self.known_macs)
 [ 34.049714] cloud-in...

Is this fixed in Zesty?

This bug was fixed in the package cloud-init - 0.7.9-90-g61eb03fe-0ubuntu1

---------------
cloud-init (0.7.9-90-g61eb03fe-0ubuntu1) zesty; urgency=medium

  * New upstream snapshot.
    - OpenStack: add 'dvs' to the list of physical link types.
      (LP: #1674946)

 -- Scott Moser <email address hidden> Mon, 03 Apr 2017 11:10:38 -0400

I've dropped the nova task.
This will be fixed in nova hopefully under bug 1642679.
My merge proposal is at https://review.openstack.org/#/c/400883

Hello Radoslav, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-90-g61eb03fe-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Radoslav, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-90-g61eb03fe-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

After upgrading the cloud-init
I have verified in my setup with xenial image . Now i don't see
" cloud-init[445]: ValueError: Unknown network_data link type: dvs"
Error in the VM.
user@xenial-vm2:~$ dpkg -l | grep cloud-init
ii cloud-init 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1 all Init

Thanks for the fix.

Also verified it with xenial and a config-drive that had "type": "dvs" in openstack/latest/network_data.json using the proposed package:

root@x1-fixed:~# apt-cache policy cloud-init
cloud-init:
  Installed: 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1
  Candidate: 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1
  Version table:
 *** 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1 500
        500 http://archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages

$ lxc file pull x1-fixed/run/cloud-init/result.json
{
 "v1": {
  "datasource": "DataSourceNoCloud [seed=/var/lib/cloud/seed/nocloud-net][dsmode=net]",
  "errors": []
 }
}

To make sure my config-drive was being used, I injected a failure by setting an unknown link type of "dvs-andreas-was-here-again":
$ lxc file pull x1-fixed/run/cloud-init/result.json -
{
 "v1": {
  "datasource": null,
  "errors": [
   "Unknown network_data link type: dvs-andreas-was-here-again",
   "Unknown network_data link type: dvs-andreas-was-here-again"
  ]
 }
}

and

$ lxc exec x1-fixed -- grep dvs-andreas-was-here-again /var/log/cloud-init.log
ValueError: Unknown network_data link type: dvs-andreas-was-here-again

Using the package from yakkety-proposed in a yakkety LXD container:

$ lxc exec y1-proposed -- /bin/bash
root@y1-proposed:~# apt-cache policy cloud-init
cloud-init:
  Installed: 0.7.9-90-g61eb03fe-0ubuntu1~16.10.1
  Candidate: 0.7.9-90-g61eb03fe-0ubuntu1~16.10.1
  Version table:
 *** 0.7.9-90-g61eb03fe-0ubuntu1~16.10.1 500
        500 http://archive.ubuntu.com/ubuntu yakkety-proposed/main amd64 Packages

/config-drive with a link of type "dvs":
$ lxc exec y1-proposed -- grep dvs /config-drive/openstack/latest/network_data.json
   "type": "dvs",

No errors:
$ lxc file pull y1-proposed/run/cloud-init/result.json -
{
 "v1": {
  "datasource": "DataSourceConfigDrive [net,ver=2][source=/config-drive]",
  "errors": []
 }
}

logs clean too:
$ lxc exec y1-proposed -- grep dvs /var/log/cloud-init.log
$

To make sure my config-drive was being used, I injected a failure by setting an unknown link type of "dvs-andreas-was-here-again":
$ lxc file pull y1-proposed/run/cloud-init/result.json -
{
 "v1": {
  "datasource": null,
  "errors": [
   "Unknown network_data link type: dvs-andreas-was-here-again",
   "Unknown network_data link type: dvs-andreas-was-here-again",
   "('ssh-authkey-fingerprints', KeyError('getpwnam(): name not found: ubuntu',))"
  ]
 }
}

(the ssh authkey error is irrelevant for this case: the ubuntu user isn't created because of the network_data link type error)

and

$ lxc exec y1-proposed -- grep dvs-andreas-was-here-again /var/log/cloud-init.log
ValueError: Unknown network_data link type: dvs-andreas-was-here-again
ValueError: Unknown network_data link type: dvs-andreas-was-here-again

Using the clout-init :
root@user:/var/lib/cloud# dpkg -l | grep cloud-init
ii cloud-init 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1 all Init scripts for cloud instances
Rebooting the ubuntu server cloud-init fails to start .

Error in the cloud-init.log
2017-04-14 20:49:56,838 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
2017-04-14 20:49:56,838 - util.py[DEBUG]: Read 10 bytes from /proc/uptime
2017-04-14 20:49:56,838 - util.py[DEBUG]: cloud-init mode 'init' took 0.065 seconds (0.06)
2017-04-14 20:49:56,838 - handlers.py[DEBUG]: finish: init-network: SUCCESS: searching for network datasources
2017-04-14 21:03:09,735 - util.py[DEBUG]: Cloud-init v. 0.7.9 running 'init-local' at Sat, 15 Apr 2017 04:03:09 +0000. Up 801.41 seconds.
2017-04-14 21:03:09,735 - main.py[DEBUG]: No kernel command line url found.
2017-04-14 21:03:09,735 - main.py[DEBUG]: Closing stdin.
2017-04-14 21:03:09,737 - util.py[DEBUG]: Writing to /var/log/cloud-init.log - ab: [420] 0 bytes
2017-04-14 21:03:09,738 - util.py[DEBUG]: Changing the ownership of /var/log/cloud-init.log to 104:4
2017-04-14 21:03:09,739 - util.py[DEBUG]: Attempting to remove /var/lib/cloud/instance/boot-finished
2017-04-14 21:03:09,739 - util.py[DEBUG]: Attempting to remove /var/lib/cloud/data/no-net
2017-04-14 21:03:09,739 - handlers.py[DEBUG]: start: init-local/check-cache: attempting to read from cache [check]
2017-04-14 21:03:09,739 - util.py[DEBUG]: Reading from /var/lib/cloud/instance/obj.pkl (quiet=False)
2017-04-14 21:03:09,739 - stages.py[DEBUG]: no cache found
2017-04-14 21:03:09,739 - handlers.py[DEBUG]: finish: init-local/check-cache: SUCCESS: no cache found
2017-04-14 21:03:09,739 - util.py[DEBUG]: Attempting to remove /var/lib/cloud/instance
2017-04-14 21:03:09,739 - util.py[WARNING]: failed stage init-local
2017-04-14 21:03:09,740 - util.py[DEBUG]: failed stage init-local
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 647, in status_wrapper
    ret = functor(name, args)
  File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 332, in main_init
    init.fetch(existing=existing)
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 349, in fetch
    return self._get_data_source(existing=existing)
  File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 250, in _get_data_source
    util.del_file(self.paths.instance_link)
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1627, in del_file
    raise e
  File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 1624, in del_file
    os.unlink(path)
IsADirectoryError: [Errno 21] Is a directory: '/var/lib/cloud/instance'

root@user:/var/lib/cloud# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial

What went wrong and how to fix this ?

I'll try rebooting a xenial node that has the updated cloud-init package and see what happens.

@ashish-kumar-gupta can you please attach your /var/log/cloud-init*.log from this attempt?

Also please the output of:
sudo ls -lah /var/lib/cloud/instance/ /var/lib/cloud/data/ /run/cloud-init/

@ashish-kumar-gupta you seem to be hitting this bug: https://launchpad.net/bugs/1531880

I'm adding back the 'verification-done-xenial'.
the broken state of having a /var/lib/cloud/instance as a directory (rather than a symlink) is not a result of any changes for this bug. bug 1531880 covers that, and we will pursue a fix there.

For the record, I did reboot a maas node that had the config-drive network configuration to use "dvs" and it didn't fail.

This bug was fixed in the package cloud-init - 0.7.9-90-g61eb03fe-0ubuntu1~16.10.1

---------------
cloud-init (0.7.9-90-g61eb03fe-0ubuntu1~16.10.1) yakkety; urgency=medium

  * debian/cloud-init.templates: add Bigstep to list of sources. (LP: #1676460)
  * New upstream snapshot.
    - OpenStack: add 'dvs' to the list of physical link types. (LP: #1674946)
    - Fix bug that resulted in an attempt to rename bonds or vlans.
      (LP: #1669860)
    - tests: update OpenNebula and Digital Ocean to not rely on host
      interfaces.
    - net: in netplan renderer delete known image-builtin content.
      (LP: #1675576)
    - doc: correct grammar in capabilities.rst [David Tagatac]
    - ds-identify: fix detecting of maas datasource. (LP: #1677710)
    - netplan: remove debugging prints, add debug logging [Ryan Harper]
    - ds-identify: do not write None twice to datasource_list.
    - support resizing partition and rootfs on system booted without
      initramfs. [Steve Langasek] (LP: #1677376)
    - apt_configure: run only when needed. (LP: #1675185)
    - OpenStack: identify OpenStack by product 'OpenStack Compute'.
      (LP: #1675349)
    - GCE: Search GCE in ds-identify, consider serial number in check.
      (LP: #1674861)
    - Add support for setting hashed passwords [Tore S. Lonoy] (LP: #1570325)
    - Fix filesystem creation when using "partition: auto"
      [Jonathan Ballet] (LP: #1634678)
    - ConfigDrive: support reading config drive data from /config-drive.
      (LP: #1673411)
    - ds-identify: fix detection of Bigstep datasource. (LP: #1674766)
    - test: add running of pylint [Joshua Powers]
    - ds-identify: fix bug where filename expansion was left on.
    - advertise network config v2 support (NETWORK_CONFIG_V2) in features.
    - Bigstep: fix bug when executing in python3. [root]
    - Fix unit test when running in a system deployed with cloud-init.
    - Bounce network interface for Azure when using the built-in path.
      [Brent Baude] (LP: #1674685)
    - cloudinit.net: add network config v2 parsing and rendering [Ryan Harper]
    - net: Fix incorrect call to isfile [Joshua Powers] (LP: #1674317)
    - net: add renderers for automatically selecting the renderer.
    - doc: fix config drive doc with regard to unpartitioned disks.
      (LP: #1673818)
    - test: Adding integratiron test for password as list [Joshua Powers]
    - render_network_state: switch arguments around, do not require target
    - support 'loopback' as a device type.
    - Integration Testing: improve testcase subclassing [Wesley Wiedenmeier]
    - gitignore: adding doc/rtd_html [Joshua Powers]
    - doc: add instructions for running integration tests via tox.
      [Joshua Powers]
    - test: avoid differences in 'date' output due to daylight savings.
    - Fix chef config module in omnibus install. [Jeremy Melvin] (LP: #1583837)
    - Add feature flags to cloudinit.version. [Wesley Wiedenmeier]
    - tox: add a citest environment
    - Support chpasswd/list being a list in addition to a string.
      [Sergio Lystopad] (LP: #1665694)
    - doc: Fix configuration example for cc_set_passwords module.
      [Sergio Lystopad] (LP: #1665773)
    - ...

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package cloud-init - 0.7.9-90-g61eb03fe-0ubuntu1~16.04.1

---------------
cloud-init (0.7.9-90-g61eb03fe-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/cloud-init.templates: add Bigstep to list of sources. (LP: #1676460)
  * New upstream snapshot.
    - OpenStack: add 'dvs' to the list of physical link types. (LP: #1674946)
    - Fix bug that resulted in an attempt to rename bonds or vlans.
      (LP: #1669860)
    - tests: update OpenNebula and Digital Ocean to not rely on host
      interfaces.
    - net: in netplan renderer delete known image-builtin content.
      (LP: #1675576)
    - doc: correct grammar in capabilities.rst [David Tagatac]
    - ds-identify: fix detecting of maas datasource. (LP: #1677710)
    - netplan: remove debugging prints, add debug logging [Ryan Harper]
    - ds-identify: do not write None twice to datasource_list.
    - support resizing partition and rootfs on system booted without
      initramfs. [Steve Langasek] (LP: #1677376)
    - apt_configure: run only when needed. (LP: #1675185)
    - OpenStack: identify OpenStack by product 'OpenStack Compute'.
      (LP: #1675349)
    - GCE: Search GCE in ds-identify, consider serial number in check.
      (LP: #1674861)
    - Add support for setting hashed passwords [Tore S. Lonoy] (LP: #1570325)
    - Fix filesystem creation when using "partition: auto"
      [Jonathan Ballet] (LP: #1634678)
    - ConfigDrive: support reading config drive data from /config-drive.
      (LP: #1673411)
    - ds-identify: fix detection of Bigstep datasource. (LP: #1674766)
    - test: add running of pylint [Joshua Powers]
    - ds-identify: fix bug where filename expansion was left on.
    - advertise network config v2 support (NETWORK_CONFIG_V2) in features.
    - Bigstep: fix bug when executing in python3. [root]
    - Fix unit test when running in a system deployed with cloud-init.
    - Bounce network interface for Azure when using the built-in path.
      [Brent Baude] (LP: #1674685)
    - cloudinit.net: add network config v2 parsing and rendering [Ryan Harper]
    - net: Fix incorrect call to isfile [Joshua Powers] (LP: #1674317)
    - net: add renderers for automatically selecting the renderer.
    - doc: fix config drive doc with regard to unpartitioned disks.
      (LP: #1673818)
    - test: Adding integratiron test for password as list [Joshua Powers]
    - render_network_state: switch arguments around, do not require target
    - support 'loopback' as a device type.
    - Integration Testing: improve testcase subclassing [Wesley Wiedenmeier]
    - gitignore: adding doc/rtd_html [Joshua Powers]
    - doc: add instructions for running integration tests via tox.
      [Joshua Powers]
    - test: avoid differences in 'date' output due to daylight savings.
    - Fix chef config module in omnibus install. [Jeremy Melvin] (LP: #1583837)
    - Add feature flags to cloudinit.version. [Wesley Wiedenmeier]
    - tox: add a citest environment
    - Support chpasswd/list being a list in addition to a string.
      [Sergio Lystopad] (LP: #1665694)
    - doc: Fix configuration example for cc_set_passwords module.
      [Sergio Lystopad] (LP: #1665773...

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2836