cc_mcollective.py fails on Ubuntu16.04

Bug #1597699 reported by Sergii Golovatiuk
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
High
Unassigned
cloud-init (Ubuntu)
Fix Released
High
Unassigned
Xenial
Fix Released
High
Unassigned

Bug Description

---- Begin SRU Template ----
[Impact]
mcollective usage via cloud-init is not possible.

[Test Case]
launch instance with cloud-config containing 'mcollective' entry.

$ cat >user-data <<"EOF"
#cloud-config
mcollective:
  conf:
    main_collective: mcollective
    collectives: mcollective
    libdir: /usr/share/mcollective/plugins
    logfile: /var/log/mcollective.log
    loglevel: debug
    daemonize: 1
    direct_addressing: 1
    ttl: 4294957
    securityprovider: psk
    plugin.psk: unset
    identity: 2

    connector: rabbitmq
    plugin.rabbitmq.vhost: mcollective
    plugin.rabbitmq.pool.size: 1
    plugin.rabbitmq.pool.1.host: 10.10.0.2
    plugin.rabbitmq.pool.1.port: 61613
    plugin.rabbitmq.pool.1.user: mcollective
    plugin.rabbitmq.pool.1.password: ScwpVo8egrZ0OmT6sRmp9zEA
    plugin.rabbitmq.heartbeat_interval: 30

    factsource: yaml
    plugin.yaml: /etc/mcollective/facts.yaml
EOF

$ keyname=brickies; image=$image;
$ openstack server create \
   --key-name=$keyname --flavor=m1.small \
   --image=$image --user-data=user-data mcollective-test0

# then ssh in and
a.) verify mcollective package is installed
$ dpkg-query --show mcollective
mcollective 2.6.0+dfsg-2.1

b.) verify mcollective service is running
$ systemctl status mcollective

c.) grep WARN /var/log/cloud-init.log && echo FAIL

[Regression Potential]
low to none. This has been unfortunately broken since wily.

[Other Info]
---- End SRU Template ----

How to reproduce:

#cat /etc/issue.net
Ubuntu 16.04 LTS
#( cd /var/lib/cloud/ && rm -rf * )
#cloud-init -d init

#grep ^mcollective: -A25 instance/user-data.txt
mcollective:
  conf:
    main_collective: mcollective
    collectives: mcollective
    libdir: /usr/share/mcollective/plugins
    logfile: /var/log/mcollective.log
    loglevel: debug
    daemonize: 1
    direct_addressing: 1
    ttl: 4294957
    securityprovider: psk
    plugin.psk: unset
    identity: 2

    connector: rabbitmq
    plugin.rabbitmq.vhost: mcollective
    plugin.rabbitmq.pool.size: 1
    plugin.rabbitmq.pool.1.host: 10.10.0.2
    plugin.rabbitmq.pool.1.port: 61613
    plugin.rabbitmq.pool.1.user: mcollective
    plugin.rabbitmq.pool.1.password: ScwpVo8egrZ0OmT6sRmp9zEA
    plugin.rabbitmq.heartbeat_interval: 30

    factsource: yaml
    plugin.yaml: /etc/mcollective/facts.yaml

#cloud-init -d single --name mcollective

The log will contain

Jun 30 08:26:43 node-2 [CLOUDINIT] util.py[DEBUG]: Running module mcollective (<module 'cloudinit.config.cc_mcollective' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_mcollective.py'>) failed#012Traceback (most recent call last):#012 File "/usr/lib/python3/dist-packages/cloudinit/stages.py", line 739, in _run_modules#012 freq=freq)#012 File "/usr/lib/python3/dist-packages/cloudinit/cloud.py", line 70, in run#012 return self._runners.run(name, functor, args, freq, clear_on_fail)#012 File "/usr/lib/python3/dist-packages/cloudinit/helpers.py", line 199, in run#012 results = functor(*args)#012 File "/usr/lib/python3/dist-packages/cloudinit/config/cc_mcollective.py", line 83, in handle#012 mcollective_config.write(contents)#012 File "/usr/lib/python3/dist-packages/configobj.py", line 2126, in write#012 outfile.write(output_bytes)#012TypeError: string argument expected, got 'bytes'

Related branches

Scott Moser (smoser)
Changed in cloud-init:
status: New → Confirmed
importance: Undecided → High
Scott Moser (smoser)
description: updated
Scott Moser (smoser)
description: updated
Changed in cloud-init (Ubuntu):
status: New → Fix Released
importance: Undecided → High
Changed in cloud-init (Ubuntu Xenial):
status: New → Confirmed
importance: Undecided → High
Changed in cloud-init:
status: Confirmed → Fix Committed
Revision history for this message
Chris J Arges (arges) wrote : Please test proposed package

Hello Sergii, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.7~bzr1256-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Xenial):
status: Confirmed → Fix Committed
tags: added: verification-needed
Scott Moser (smoser)
description: updated
Revision history for this message
Scott Moser (smoser) wrote :
Download full text (3.6 KiB)

I've verified this in an lxc container using the method described here.

# launch instance (lxc)
$ name="mcollective0"
$ cat > user-data <<EOF
#cloud-config
mcollective:
  conf:
    main_collective: mcollective
    collectives: mcollective
    libdir: /usr/share/mcollective/plugins
    logfile: /var/log/mcollective.log
    loglevel: debug
    daemonize: 1
    direct_addressing: 1
    ttl: 4294957
    securityprovider: psk
    plugin.psk: unset
    identity: 2

    connector: rabbitmq
    plugin.rabbitmq.vhost: mcollective
    plugin.rabbitmq.pool.size: 1
    plugin.rabbitmq.pool.1.host: 10.10.0.2
    plugin.rabbitmq.pool.1.port: 61613
    plugin.rabbitmq.pool.1.user: mcollective
    plugin.rabbitmq.pool.1.password: ScwpVo8egrZ0OmT6sRmp9zEA
    plugin.rabbitmq.heartbeat_interval: 30

    factsource: yaml
    plugin.yaml: /etc/mcollective/facts.yaml
EOF
$ lxc launch ubuntu-daily:xenial $name "--config=user.user-data=$(cat user-data)"

# wait for boot
$ while ! lxc exec $name -- test -e /run/cloud-init/result.json; do echo .; sleep 1; done

# get in the instance (first boot, unpatched)
$ lxc exec "$name" /bin/bash
# PS1="% "
% cat /etc/cloud/build.info
build_name: server
serial: 20160715

% grep FAIL /var/log/cloud-init.log
Jul 19 17:01:10 ubuntu [CLOUDINIT] handlers.py[DEBUG]: finish: modules-config/config-mcollective: FAIL: running config-mcollective with frequency once-per-instance
Jul 19 17:01:10 ubuntu [CLOUDINIT] handlers.py[DEBUG]: finish: modules-config: FAIL: running modules for config

% dpkg-query --show cloud-init
cloud-init 0.7.7~bzr1246-0ubuntu1~16.04.1

# The failed path results in installation of mcollective
# but with the installed config file moved to '.old'. Just clean
# up to make this more pristine.
% apt-get --purge remove mcollective; rm -Rf /etc/mcollective/

# enable proposed and update
% cat > enable-proposed <<"EOF"
#!/bin/sh
set -e
rel=$(lsb_release -sc)
awk '$1 == "deb" && $2 ~ /ubuntu.com/ {
  printf("%s %s %s-proposed main universe\n", $1, $2, rel); exit(0) };
  ' "rel=$rel" /etc/apt/sources.list |
    tee /etc/apt/sources.list.d/proposed.list
EOF

% sh ./enable-proposed
deb http://archive.ubuntu.com/ubuntu xenial-proposed main universe
% sudo apt-get update -q && sudo apt-get install cloud-init
% dpkg-query --show cloud-init
cloud-init 0.7.7~bzr1256-0ubuntu1~16.04.1

# similate first boot on pristine image.
# avoid the 'seed' dir in /var/lib/cloud as it is used by lxd
% sudo rm -Rf /var/log/cloud-init
% rm -Rf /var/lib/cloud/{data,handlers,instance,instances,scripts,sem}
% sudo reboot

### wait and re-attach to instance
$ while ! lxc exec $name -- test -e /run/cloud-init/result.json; do echo .; sleep 1; done
$ lxc exec "$name" /bin/bash
# PS1="% "

% dpkg-query --show mcollective
mcollective 2.6.0+dfsg-2.1
% grep WARN /var/log/cloud-init.log || echo no WARN
no WARN

% systemctl status mcollective
● mcollective.service - The Marionette Collective
   Loaded: loaded (/lib/systemd/system/mcollective.service; enabled; vendor pres
   Active: active (running) since Tue 2016-07-19 17:03:14 UTC; 3min 8s ago
 Main PID: 739 (ruby)
    Tasks: 3
   Memory: 9.8M
      CPU: 248ms
   CGroup: /system.slice/mcollective.service
 ...

Read more...

tags: added: verification-done
removed: verification-needed
Revision history for this message
Adam Conrad (adconrad) wrote : Update Released

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.7~bzr1256-0ubuntu1~16.04.1

---------------
cloud-init (0.7.7~bzr1256-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * New upstream snapshot.
    - distros/debian.py: fix eni renderer to not render .link files
    - fixes for execution in python2.6.
    - ConfigDrive: fix writing of 'injected' files and legacy networking
      (LP: #1602373)
    - improvements to /etc/network/interfaces rendering including rendering
      of 'lo' devices and sorting attributes within a interface section.
    - fix mcollective module that was completely broken if using python3
      (LP: #1597699)

 -- Scott Moser <email address hidden> Fri, 15 Jul 2016 13:27:04 -0400

Changed in cloud-init (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

This is fixed in cloud-init 0.7.7

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.