uec images should wait longer for metadata service

Bug #571271 reported by Scott Moser
20
This bug affects 2 people
Affects Status Importance Assigned to Milestone
cloud-init (Ubuntu)
Fix Released
High
Scott Moser
Lucid
Fix Released
Undecided
Unassigned

Bug Description

Binary package hint: cloud-init

as described in bug 566792 . The uec instances time out on waiting for the metadata service well too early (in ~ 20 seconds).

This needs to be increased so that they boot more reliably.

ProblemType: Bug
DistroRelease: Ubuntu 10.04
Package: cloud-init 0.5.10-0ubuntu1
ProcVersionSignature: User Name 2.6.32-305.9-ec2 2.6.32.11+drm33.2
Uname: Linux 2.6.32-305-ec2 i686
Architecture: i386
Date: Wed Apr 28 14:27:01 2010
Ec2AMI: ami-2d4aa444
Ec2AMIManifest: ubuntu-images-us/ubuntu-lucid-10.04-i386-server-20100427.1.manifest.xml
Ec2AvailabilityZone: us-east-1b
Ec2InstanceType: m1.small
Ec2Kernel: aki-754aa41c
Ec2Ramdisk: unavailable
PackageArchitecture: all
ProcEnviron:
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: cloud-init

======== Begin SRU Information =======
Impact: This bug shows up in UEC instances run when the UEC they are running on is under load.
How this has been addressed: The UEC instances depend on the metadata serviced provided by UEC. When the UEC is under high load, that meta data service may not be available immediately on instance boot. The previous version of cloud-init would wait only 10 seconds and then go on, causing failure. The changes here enable some debug output when that is occuring and wait for considerably longer (1050 seconds total)
Patch: http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/lucid/revision/21?start_revid=21
Regression Potential: The largest potential for regression that I can imagine is the case where 'cloud-init' is installed on a non-cloud instance. Previously, that would have resulted in delaying boot by 10 seconds and some python traces on the console. Now, the delay would be ~ 1050 seconds. There is no trivial way to determine "am I in the cloud", so solving this bug and avoiding problems is not easy. In general, the 'cloud-init' is a cloud specific package, not intended for local installation.

TEST CASE:
 * repeatedly launch instances.
 * previously, failure would be seen by instance not being reachable to ssh and a message such as 'Failed to get instance data' in the console log
======== End SRU Information =======

Revision history for this message
Scott Moser (smoser) wrote :
Changed in cloud-init (Ubuntu):
status: New → Triaged
importance: Undecided → High
Scott Moser (smoser)
Changed in cloud-init (Ubuntu):
assignee: nobody → Scott Moser (smoser)
status: Triaged → In Progress
Scott Moser (smoser)
description: updated
Scott Moser (smoser)
Changed in cloud-init (Ubuntu):
milestone: none → lucid-updates
Revision history for this message
Martin Pitt (pitti) wrote : Please test proposed package

Accepted cloud-init into lucid-proposed, the package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in cloud-init (Ubuntu Lucid):
status: New → Fix Committed
tags: added: verification-needed
Revision history for this message
Scott Moser (smoser) wrote :

I've verified this with:
$ ec2-run-instances ami-2d4aa444
$ ssh host

% echo deb http://archive.ubuntu.com/ubuntu/ lucid-proposed restricted main multiverse universe |
   sudo tee /etc/apt/sources.list.d/lucid-proposed.list

% sudo apt-get update
% sudo apt-get install cloud-init
% sudo rm -f /var/lib/cloud/sem/*
% sudo reboot

# now instance comes up with different ssh fingerprint (indicating that the cloud-init re-ran)

$ ec2-get-console-output i-XXXXXXXX | grep "cloud-init run"
cloud-init running: Fri, 14 May 2010 12:49:55 +0000. up 5.41 seconds^M

That shows new code ran.. I've also verified that the wait is longer if, by chance, the metadata service is not there.

Scott Moser (smoser)
tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.5.10-0ubuntu1.1

---------------
cloud-init (0.5.10-0ubuntu1.1) lucid-proposed; urgency=low

  * wait considerably longer (1050 seconds) for metadata service to come up
    (LP: #571271)
 -- Scott Moser <email address hidden> Wed, 05 May 2010 10:52:12 -0400

Changed in cloud-init (Ubuntu Lucid):
status: Fix Committed → Fix Released
Revision history for this message
Martin Pitt (pitti) wrote :

Copied to maverick.

Changed in cloud-init (Ubuntu):
status: In Progress → Fix Released
tags: added: testcase
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.