unexpected errors after a sequence of tests

Bug #622818 reported by C de-Avillez
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Eucalyptus
Fix Released
Undecided
chris grzegorczyk
eucalyptus (Ubuntu)
Fix Released
High
Dave Walker
Maverick
Fix Released
High
Dave Walker

Bug Description

euca2ools 1.2-0ubuntu10 install ok installed
eucalyptus-cloud 2.0~bzr1233-0ubuntu1 eucalyptus install ok installed
eucalyptus-common 2.0~bzr1233-0ubuntu1 eucalyptus install ok installed
eucalyptus-java-common 2.0~bzr1233-0ubuntu1 eucalyptus install ok installed
libeucalyptus-commons-ext-java 0.5.0-0ubuntu2 eucalyptus-commons-ext install ok installed
uec-component-listener 2.0~bzr1233-0ubuntu1 eucalyptus install ok installed

During this weekend I ran a series of 500-instances tests. I decided on 500 because this makes the tests reasonably short (a few hours at most). Every so often, I would return, have a look and, if the run had finished, I would start a new one.

This morning, when I looked at the terminal, I saw many instance start failures. I then cancelled the test run, and ran a -describe-instances to find out what was running. It failed:

ubuntu@cempedak:/var/log/eucalyptus$ euca-describe-instances --debug
[Errno 104] Connection reset by peer
ubuntu@cempedak:/var/log/eucalyptus$

Experiments showed that no euca-* would return.

I have uploaded the logs to http://bazaar.launchpad.net/~hggdh2/+junk/uec-qa/files at revision 47.

Tags: server-mrs
Mathias Gug (mathiaz)
Changed in eucalyptus (Ubuntu):
importance: Undecided → Medium
Changed in eucalyptus:
assignee: nobody → chris grzegorczyk (chris-grze)
Revision history for this message
Dmitrii Zagorodnov (dmitrii) wrote :

Carlos, when you get a chance, would you try to capture the logs again with " --debug --log-level=DEBUG" appended to CLOUD_OPTS in eucalyptus.conf on the CLC machine? Thank you!

Revision history for this message
C de-Avillez (hggdh2) wrote :

Dmitrii,, done: lp:~hggdh2/+junk/uec-qa:

Using saved push location: bzr+ssh://bazaar.launchpad.net/~hggdh2/%2Bjunk/uec-qa/
Pushed up to revision 50.

Seems we starting having problems at around 20:00 (or earlier, still parsing thru the logs). I noticed it at around 21:30ish, and cancelled the test run (and left it resting until this morning).

Changed in eucalyptus:
status: New → Fix Committed
Revision history for this message
chris grzegorczyk (chris-grze) wrote :

Hi Carlos,

I committed something that was lost in a merge at some point and should address this issue. Let me know how it goes when you have a chance to try reproducing. Thanks.

------------------------------------------------------------
revno: 1238 [merge]
committer: decker <decker@hawaii>
branch nick: 2.0.0-lp
timestamp: Mon 2010-09-06 19:45:17 -0700
message:
  clean up in metadata pipeline; restore fix lost in merge. LP: #622818
------------------------------------------------------------

Thierry Carrez (ttx)
Changed in eucalyptus (Ubuntu Maverick):
importance: Medium → High
tags: added: server-mrs
Revision history for this message
Thierry Carrez (ttx) wrote :

Needs to be rolled out in a package now

Changed in eucalyptus (Ubuntu Maverick):
assignee: nobody → Dave Walker (davewalker)
status: New → Triaged
Dave Walker (davewalker)
Changed in eucalyptus (Ubuntu Maverick):
status: Triaged → In Progress
status: In Progress → Fix Committed
Revision history for this message
Thierry Carrez (ttx) wrote :

eucalyptus (2.0+bzr1239-0ubuntu1) maverick; urgency=low

  [Dave Walker (Daviey)]
  * New upstream stable release, -r1239 Fixes:
    - euca_conf --register-cluster is not idempotent. (LP: #628328)
    - unexpected errors after a sequence of tests (LP: #622818)
  * debian/control:
    - Recommend bittorrent|bittornado for -walrus package, enabling S3
      bittorent API support. (LP: #613535)
    - Raise the minimum version of libvirt-bin needed to eucalyptus-nc.
  * debian/eucalyptus-nc.postinst: Add eucalyptus user to the kvm group to
    enable user to utilise console logs. (LP: #619843)
  * debian/patches/18-priv_security.patch: Added vgscan to util/wrappers.conf,
    as it requires escalated privileges. (LP: #628291)
  * debian/patches/06-UEC-webinterface.patch (and 3 images): Updated and
    refreshed to better reflect Ubuntu branding. (LP: #631451)
  * debian/patches/25-uec-logo-png.patch: Due to a bug in dpkg, we need to use
    a different filename for the logo.png.
  * debian/eucalyptus-{java-,}common.{preinst,postinst}: Update the internal
    database and keys on upgrade from 10.04.

  [Scott Moser]
  * fix determination of 'is_multiboot'.
 -- Dave Walker (Daviey) <email address hidden> Fri, 10 Sep 2010 12:18:02 +0100

Changed in eucalyptus (Ubuntu Maverick):
status: Fix Committed → Fix Released
Changed in eucalyptus:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.