couldn't launch an AMI bundled and uploaded using euca2ools

Are you able to retrieve the console output from the instance? That might give a clue about what's failing.

No, it goes almost immediately (10-20 sec after launching) to terminated state. Console output is blank.

One puzzling fact is that ec2-unbundle can successfuly unbundle the defective bundle and restore the original filesystem image.

OK, I found it. The image digest was taken from a wrong file. In Amazon tools, digest is generated from the uncompressed tarball that contains root image, but euca2ools took digest of root image itself. Stand by for an explanatory patch.

BTW, in function "check_image" there were spaces missing, so SHA1 was calculated incorrectly.

Pete,
To accept this patch with the legal ramifications that we have to work with, you'd either need to sign the committer agreement, or simply send me the patch with the MIT license embedded in it. I'm including the license below. I've integrated and tested the patch in euca2ools and will push that to main soon, but I'd like to have the rights sorted out. So you know, I took the code, not totally as submitted, but the working parts of it. Our intention is to re-factor at some point, but thought it was important to get the fix out there sooner.

The MIT License

Copyright (c) <year> <copyright holders>

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

Mitch already e-mailed me about it and pointed at the URL with the form and patch process. I signed the form and might as well try the official route.

There were a couple of technical issues as well. Mitch said that he does not want to use tee. Also, Fedora cloud users had concerns about using this much space for a temporary file. Finally, the code was changed in the trunk, so my patch does not apply (it was based on 1.3.1 that Garrett packages for us).

So, I am going to rework the fix so that it stores the compressed tarball as before, then decompresses it to compute SHA1 digest. This will trade CPU for I/O and disk space.

By way of clarification, Fedora's latest euca2ools update includes the switch to Python's urlparse library as a fix for Red Hat bug 658560, which conflicts with Pete's own URI parsing fix that is included in his patch. AFAICT that part doesn't matter, and the remainder of the patch applies as-is.

Why do we need to store an uncompressed copy on disk just to hash it? That's an awful lot of time lost to I/O.

This is a patch against the tip. It addresses all previous comments:
 - reduces wasted space
 - works faster
 - less intrusive for the code

David's suggestion seems odd to me. Exactly how one attaches the license to a patch? It's a 20-liner anyway, just do whatever with it. MIT - sure, fine by me.

A fix for this in Fedora should appear in Rawhide and updates-testing for Fedora 13 and 14 during the next push. I am not in a position to test this code at the moment, so please check whether or not they fix this bug and give [1, 2] the update +1 or -1 depending on the results you get. Assuming they work, they will (independently) go stable when they hit +3.

If you want to get an early start on this you can also get builds from http://repos.fedorapeople.org/repos/gholms/cloud/.

[1] https://admin.fedoraproject.org/updates/euca2ools-1.3.1-4.fc13
[2] https://admin.fedoraproject.org/updates/euca2ools-1.3.1-4.fc14

Garrett, I tested Fedora 13 version - works great! Thank you!

I had a look at Pets's patch, and it looks good, and would certainly
do the trick. Then, I did what I should have done last week, and wrote
a pipelined version that doesn't touch the disk with a temp file. Have
a look at revision 327 here;
http://bazaar.launchpad.net/~eucalyptus-maintainers/euca2ools/euca2ools-main/annotate/head:/euca2ools/euca2ools/__init__.py

Ideally, I'd get the aes128 stuff worked into the pipeline like the
ec2-ami-tools, but this (I think) is the right direction. I've applied
the changes to euca-bundle-image as well as euca-bundle-vol

David

On Thu, Dec 16, 2010 at 5:02 AM, Marek Goldmann
<email address hidden> wrote:
> Garrett, I tested Fedora 13 version - works great! Thank you!
>
> --
> You received this bug notification because you are a member of
> Eucalyptus Maintainers, which is the registrant for euca2ools.
> https://bugs.launchpad.net/bugs/665667
>
> Title:
>  couldn't launch an AMI bundled and uploaded using euca2ools
>
> Status in Euca2ools:
>  New
>
> Bug description:
>  I try to replace ec2-ami-tools with euca2ools package for BoxGrinder Build. If I switch back to ec2-ami-tools - everything works.
>
> Here is my bug report, see comments: https://jira.jboss.org/browse/BGBUILD-55
>
> I used euca2ools-1.3.1-1.fc13.noarch version.
>
>
>

This issue was fixed in upstream commit 327; marking it as such...

This bug was fixed in the package euca2ools - 1.3.1-0ubuntu5

---------------
euca2ools (1.3.1-0ubuntu5) natty; urgency=low

  * fix odd length string error in euca-bundle-image (LP: #509562)
    which was lost in upgrade to 1.3.1
  * cherry pick 2 upstream commits regarding user id handling. one
    fixing user ids beginning with zero (regressed from LP: #479823)
    one more easily removing '-' chars.
  * cherry pick upstream fix for a bundling bug which caused ec2 instances
    to go straight to terminated (LP: #665667)
 -- Scott Moser <email address hidden> Fri, 14 Jan 2011 11:38:55 -0500

I like David's fix with two subprocess.Popen and a loop that forks the stream.
I would've preferred the same approach myself, but I did not have Python
chops to do it.

BTW, I have RT # now, for next time.

Thanks, Pete. I'm pretty new to Python, I have to admit, but I had
Mitch as a tutor.

On Sat, Jan 15, 2011 at 8:16 PM, Pete Zaitcev <email address hidden> wrote:
> I like David's fix with two subprocess.Popen and a loop that forks the stream.
> I would've preferred the same approach myself, but I did not have Python
> chops to do it.
>
> BTW, I have RT # now, for next time.
>
> --
> You received this bug notification because you are a member of
> Eucalyptus Maintainers, which is the registrant for euca2ools.
> https://bugs.launchpad.net/bugs/665667
>
> Title:
>  couldn't launch an AMI bundled and uploaded using euca2ools
>
> Status in Euca2ools:
>  Fix Committed
> Status in “euca2ools” package in Ubuntu:
>  Fix Released
>
> Bug description:
>  I try to replace ec2-ami-tools with euca2ools package for BoxGrinder
>  Build. If I switch back to ec2-ami-tools - everything works.
>
>  Here is my bug report, see comments:
>  https://jira.jboss.org/browse/BGBUILD-55
>
>  I used euca2ools-1.3.1-1.fc13.noarch version.
>
>
>

I verified this bug is present in euca2ools versions for both maverick and lucid. And thusly, nominated it for that release. Mainly just to track it.

maverick has seen the end of its life and is no longer receiving any updates. Marking the maverick task for this ticket as "Won't Fix".

lucid has seen the end of its life and is no longer receiving any updates. Marking the lucid task for this ticket as "Won't Fix".