cloud-init runs again after reboot in release upgraded instance

Bug #1049146 reported by Andreas Hasenack
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init (Ubuntu)
Fix Released
Medium
Scott Moser
Precise
Fix Released
High
Unassigned

Bug Description

=== Begin SRU Information ===
[Impact]
 * If a user launches a 10.04 cloud-image with a user-data script, and then
   does a upgrade to 12.04 and 'reboot's to complete the upgrade, the
   user-data script will incorrectly execute again on boot.

   This is because the cloud config module that executed user-data changed
   names between 10.04 and 12.04 from 'user-scripts' to 'config-scripts-user',
   but nothing was in place to adjust the "already ran" marker file

[Test Case]
 * launch an instance with some user-data:
   $ cat lp-1049146-user-data.txt
   #!/bin/sh
   echo "====== $(date -R) [$0] =====" | tee -a /root/user-data.log

   $ ubuntu-ec2-run --key=your-keyname --instance-type=m1.medium \
      --user-data-file lp-1049146-user-data.txt lucid daily
   # us-east-1/instance-store/ubuntu-lucid-daily-amd64-server-20121108

 * upgrade the image and then do-release-upgrade it, then reboot
   $ IP=THE_IP_ADDRESS
   $ ssh ubuntu@$IP
   % sudo apt-get update -q && sudo apt-get upgrade -y
   % do-release-upgrade --frontend=DistUpgradeViewNonInteractive

   # you could install the patched new version of cloud-init by hand here.
   % sudo reboot

 * see that the user-data has run more than once
   # upon reboot, you'll see that the user-data script ran again (twice)
   $ sudo cat /root/user-data.log
   ====== Thu, 08 Nov 2012 17:07:52 +0000
   [/var/lib/cloud/data/scripts/part-000] =====
   ====== Thu, 08 Nov 2012 17:26:50 +0000
   [/var/lib/cloud/instance/scripts/part-000] =====
   ====== Thu, 08 Nov 2012 17:26:50 +0000
   [/var/lib/cloud/instance/scripts/part-001] =====

[Regression Potential]
 * There is little potential to make anything worse here. The relevant
   change is really only to touch a file in the post install that will
   serve as a marker for the newer version of cloud-init, so it realize
   that the user-data has already been executed once.

=== End SRU Information ===

I spawned a lucid AWS instance and used some cloud-init config to perform some tasks for me.

All is good.

Then, as part of the task I was performing, I upgraded that lucid instance to precise using do-release-upgrade. It failed, but unrelated to cloud-init (a postinst from a package failed).

I then rebooted the instance. When it came back up, the cloud-init config I supplied the first time was run again, as if this was the first time the instance booted.

smoser had access to before and after reboot, and asked to add this to the bug:
# for x in /var/lib/cloud/instance/sem/*; do [ -f var-lib-cloud/instance/sem/${x##*/} ] || echo ${x##*/}; done
config-byobu
config-ca-certs
config-chef
config-grub-dpkg
config-keys-to-console
config-landscape
config-mcollective
config-phone-home
config-rightscale_userdata
config-rsyslog
config-salt-minion
config-scripts-per-instance
config-scripts-user
config-set-passwords
config-ssh-import-id
config-timezone

$ ls var-lib-cloud/instance/sem/
config-apt-pipelining config-puppet config-updates-check
config-apt-update-upgrade config-runcmd consume_userdata
config-locale config-set_hostname user-scripts
config-mounts config-ssh

Revision history for this message
Andreas Hasenack (ahasenack) wrote :
Download full text (5.7 KiB)

More info:
2012-09-11 13:42:31,743 - cloud-init[INFO]: cloud-init start-local running: Tue, 11 Sep 2012 13:42:31 +0000. up 10.06 seconds
2012-09-11 13:42:33,291 - __init__.py[DEBUG]: searching for data source in ['DataSourceNoCloud', 'DataSourceConfigDrive', 'DataSourceOVF']
2012-09-11 13:42:33,784 - __init__.py[DEBUG]: Did not find data source. searched classes: ['DataSourceNoCloud', 'DataSourceConfigDrive', 'DataSourceOVF']
2012-09-11 13:42:34,487 - cloud-init[INFO]: cloud-init start running: Tue, 11 Sep 2012 13:42:34 +0000. up 13.21 seconds
2012-09-11 13:42:34,529 - __init__.py[DEBUG]: searching for data source in ['DataSourceNoCloudNet', 'DataSourceConfigDriveNet', 'DataSourceOVFNet', 'DataSourceMAAS', 'DataSourceEc2']
2012-09-11 13:42:34,738 - DataSourceEc2.py[DEBUG]: Using metadata source: 'http://169.254.169.254'
2012-09-11 13:42:34,756 - DataSourceEc2.py[DEBUG]: crawl of metadata service took 0s
2012-09-11 13:42:34,757 - __init__.py[DEBUG]: found data source DataSourceEc2
2012-09-11 13:42:34,899 - cloud-init[DEBUG]: found data source: DataSourceEc2
2012-09-11 13:42:34,911 - __init__.py[DEBUG]: consume_userdata already ran once-per-instance
2012-09-11 13:42:34,974 - __init__.py[DEBUG]: handling bootcmd with freq=None and args=[]
2012-09-11 13:42:35,068 - __init__.py[DEBUG]: handling resizefs with freq=None and args=[]
2012-09-11 13:42:35,196 - cc_resizefs.py[DEBUG]: resize took 0.101883888245 seconds
2012-09-11 13:42:35,196 - cc_resizefs.py[DEBUG]: resizing root filesystem (type=ext3, maj=202, min=1, val=True)
2012-09-11 13:42:35,197 - __init__.py[DEBUG]: handling set_hostname with freq=None and args=[]
2012-09-11 13:42:35,198 - __init__.py[DEBUG]: config-set_hostname already ran once-per-instance
2012-09-11 13:42:35,198 - __init__.py[DEBUG]: handling update_hostname with freq=None and args=[]
2012-09-11 13:42:35,200 - cc_update_hostname.py[DEBUG]: wrote ip-10-50-203-219 to /var/lib/cloud/data/previous-hostname
2012-09-11 13:42:35,200 - __init__.py[DEBUG]: handling update_etc_hosts with freq=None and args=[]
2012-09-11 13:42:35,209 - cc_update_etc_hosts.py[DEBUG]: not managing /etc/hosts
2012-09-11 13:42:35,209 - __init__.py[DEBUG]: handling ca-certs with freq=None and args=[]
2012-09-11 13:42:35,210 - __init__.py[DEBUG]: handling rsyslog with freq=None and args=[]
2012-09-11 13:42:35,211 - __init__.py[DEBUG]: handling ssh with freq=None and args=[]
2012-09-11 13:42:35,220 - __init__.py[DEBUG]: config-ssh already ran once-per-instance
2012-09-11 13:42:37,221 - cloud-init-cfg[INFO]: cloud-init-cfg ['all', 'config']
2012-09-11 13:42:37,221 - __init__.py[DEBUG]: handling mounts with freq=None and args=[]
2012-09-11 13:42:37,305 - __init__.py[DEBUG]: config-mounts already ran once-per-instance
2012-09-11 13:42:37,315 - __init__.py[DEBUG]: handling ssh-import-id with freq=None and args=[]
2012-09-11 13:42:37,320 - cc_ssh_import_id.py[DEBUG]: importing ssh ids. cmd = ['sudo', '-Hu', 'ubuntu', 'ssh-import-id', 'ahasenack']
2012-09-11 13:42:39,443 - __init__.py[DEBUG]: handling locale with freq=None and args=[]
2012-09-11 13:42:39,464 - __init__.py[DEBUG]: config-locale already ran once-per-instance
2012-09-11 13:42:39,464 - __init__.py[DEBUG]...

Read more...

Revision history for this message
Scott Moser (smoser) wrote :
description: updated
Revision history for this message
Scott Moser (smoser) wrote :

fix-released in 0.7.0~bzr659-0ubuntu1.

no longer affects: cloud-init
Changed in cloud-init (Ubuntu):
status: New → Fix Released
assignee: nobody → Scott Moser (smoser)
importance: Undecided → Medium
Scott Moser (smoser)
Changed in cloud-init (Ubuntu Precise):
status: New → Triaged
importance: Undecided → High
Scott Moser (smoser)
description: updated
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Andreas, or anyone else affected,

Accepted cloud-init into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cloud-init/0.6.3-0ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Precise):
status: Triaged → Fix Committed
tags: added: verification-needed
Revision history for this message
Scott Moser (smoser) wrote :

verified fixed by running the above 'Test Case', but by adding the proposed pocket before upgrading the initial system.
   echo "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc)-proposed main" | sudo tee -a /etc/apt/sources.list
# then
sudo apt-get update -q && sudo apt-get upgrade -y
do-release-upgrade --frontend=DistUpgradeViewNonInteractive

# then reboot

after reboot,

$ sudo cat /root/user-data.log
====== Wed, 12 Dec 2012 19:18:15 +0000 [/var/lib/cloud/data/scripts/part-000] =====

There was only one entry there, indicating it did not run on reboot

tags: added: verification-done
removed: verification-needed
Revision history for this message
Clint Byrum (clint-fewbar) wrote :

Hello Andreas, or anyone else affected,

Accepted cloud-init into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/cloud-init/0.6.3-0ubuntu1.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: removed: verification-done
tags: added: verification-needed
Scott Moser (smoser)
Changed in cloud-init (Ubuntu Precise):
assignee: nobody → Scott Moser (smoser)
tags: added: verification-done
removed: verification-needed
Changed in cloud-init (Ubuntu Precise):
assignee: Scott Moser (smoser) → nobody
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.6.3-0ubuntu1.3

---------------
cloud-init (0.6.3-0ubuntu1.3) precise-proposed; urgency=low

  * debian/patches/lp-1070345-landscape-restart-after-change.patch,
    debian/patches/lp-1066115-landscape-install-fix-perms.patch:
    fix missing or incorrect imports (LP: #1070345, LP: #1066115).

cloud-init (0.6.3-0ubuntu1.2) precise-proposed; urgency=low

  * debian/patches/lp-978127-maas-oauth-fix-bad-clock.patch: fix usage of
    oauth in maas data source if local system has a bad clock (LP: #978127)
  * debian/cloud-init.preinst: fix bug where user data scripts re-ran on
    upgrade from 10.04 versions (LP: #1049146)
  * debian/patches/lp-974509-detect-dns-server-redirection.patch: detect dns
    server redirection and disable searching dns for a mirror named
    'ubuntu-mirror' (LP: #974509)
  * debian/patches/lp-1018554-shutdown-message-to-console.patch: write a
    message to the console on system shutdown. (LP: #1018554)
  * debian/patches/lp-1066115-landscape-install-fix-perms.patch: install
    landscape package if needed which will ensure proper permissions on config
    file (LP: #1066115).
  * debian/patches/lp-1070345-landscape-restart-after-change.patch: restart
    landscape after modifying config (LP: #1070345)
  * debian/patches/lp-1073077-zsh-workaround-for-locale_warn.patch: avoid
    warning when user's shell is zsh (LP: #1073077)
  * debian/patches/rework-mirror-selection.patch: improve mirror selection by:
    * allowing region/availability-zone to be part of mirror (LP: #1037727)
    * making mirror selection arch aware (LP: #1028501)
    * allow specification of a security mirror (LP: #1006963)
 -- Scott Moser <email address hidden> Thu, 13 Dec 2012 12:16:56 -0500

Changed in cloud-init (Ubuntu Precise):
status: Fix Committed → Fix Released
Revision history for this message
Colin Watson (cjwatson) wrote : Update Released

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.