panic setting up 1.18.1 to upgrade to 1.20-beta1

Bug #1323937 reported by Curtis Hovey
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
juju-core
Fix Released
High
Tim Penhey

Bug Description

All the upgrade tests failed
    http://juju-ci.vapour.ws:8080/

The failure happens bootstrapping 1.18, long before the test deploys a stable ant attempts an upgrade. The likely cause is the 1.20-beta1 binaries are in the path, but I don't think this really happen until after the stack is deployed.

This may be a case where 1.18.1 is reading the simple streams metadata and finds 1.20-beta1 an unacceptable version.

juju --show-log bootstrap -e hp-upgrade --constraints mem=4G
2014-05-28 04:10:19 INFO juju.cmd supercommand.go:297 running juju-1.18.1-precise-amd64 [gc]
2014-05-28 04:10:19 INFO juju.provider.openstack provider.go:202 opening environment "hp-upgrade"
2014-05-28 04:10:21 INFO juju.environs.bootstrap bootstrap.go:46 bootstrapping environment "hp-upgrade"
2014-05-28 04:10:21 INFO juju.environs.tools tools.go:86 reading tools with major.minor version 1.18
2014-05-28 04:10:21 INFO juju.environs.tools tools.go:94 filtering tools by version: 1.18.1
2014-05-28 04:10:21 INFO juju.environs.tools tools.go:97 filtering tools by series: precise
2014-05-28 04:10:21 INFO juju.utils http.go:55 hostname SSL verification enabled
2014-05-28 04:10:22 INFO juju.utils http.go:55 hostname SSL verification enabled
2014-05-28 04:10:22 INFO juju.utils http.go:55 hostname SSL verification enabled
2014-05-28 04:10:22 INFO juju.utils http.go:55 hostname SSL verification enabled
panic: invalid version "1.20-beta1"

goroutine 1 [running]:
launchpad.net/juju-core/version.MustParse(0xc20055a010, 0xa, 0xc20049b460, 0xc20049b460, 0xc2005ca101, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/version/version.go:151 +0x7f
launchpad.net/juju-core/environs/tools.appendMatchingTools(0xc2004b2b00, 0xc200495d20, 0xc2001b3cf0, 0x1, 0x1, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/environs/tools/simplestreams.go:224 +0x30e
launchpad.net/juju-core/environs/simplestreams.GetLatestMetadata(0xc2005d3c80, 0xc200495de0, 0xc20049b460, 0xc2004b2b00, 0xc200495d20, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/environs/simplestreams/simplestreams.go:1023 +0x75f
launchpad.net/juju-core/environs/simplestreams.(*IndexReference).getLatestMetadataWithFormat(0xc2005a3630, 0xc200495de0, 0xc20049b460, 0xcfcc90, 0xc, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/environs/simplestreams/simplestreams.go:986 +0x193
launchpad.net/juju-core/environs/simplestreams.getMaybeSignedMetadata(0xc2004b2b00, 0xc200495d20, 0xd53890, 0x10, 0xc200495de0, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/environs/simplestreams/simplestreams.go:469 +0x5b3
launchpad.net/juju-core/environs/simplestreams.GetMetadata(0xc2004b2bc0, 0x3, 0x4, 0xd53890, 0x10, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/environs/simplestreams/simplestreams.go:429 +0x23c
launchpad.net/juju-core/environs/tools.Fetch(0xc2004b2bc0, 0x3, 0x4, 0xd53890, 0x10, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/environs/tools/simplestreams.go:182 +0x168
launchpad.net/juju-core/environs/tools.FindToolsForCloud(0xc2004b2bc0, 0x3, 0x4, 0xc200435080, 0x13, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/environs/tools/tools.go:121 +0x15d
launchpad.net/juju-core/environs/tools.FindTools(0xc200495c30, 0xc200278b00, 0x1, 0x12, 0x0, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/environs/tools/tools.go:106 +0x57a
launchpad.net/juju-core/environs/tools.FindBootstrapTools(0xc200495c30, 0xc200278b00, 0xc2004a1b00, 0x0, 0xc200000b90, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/environs/tools/tools.go:183 +0x1a7
launchpad.net/juju-core/environs/bootstrap.EnsureToolsAvailability(0xc200426900, 0xc200271000, 0xc2005725a0, 0xc200278b00, 0xc200000b90, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/environs/bootstrap/synctools.go:168 +0x2ad
launchpad.net/juju-core/provider/common.EnsureBootstrapTools(0xc200426900, 0xc200271000, 0xc2005725a0, 0xc200278b00, 0xc200000b90, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/provider/common/bootstrap.go:407 +0x72
launchpad.net/juju-core/provider/common.Bootstrap(0xc200426900, 0xc200271000, 0xc2005725a0, 0xc200278b00, 0x0, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/provider/common/bootstrap.go:45 +0x16c
launchpad.net/juju-core/provider/openstack.(*environ).Bootstrap(0xc200278b00, 0xc200426900, 0xc200271000, 0x0, 0x0, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/provider/openstack/provider.go:531 +0xac
launchpad.net/juju-core/environs/bootstrap.Bootstrap(0xc200426900, 0xc200271000, 0xc2005725a0, 0xc200278b00, 0x0, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/environs/bootstrap/bootstrap.go:47 +0x59f
main.(*BootstrapCommand).Run(0xc200275e00, 0xc200271000, 0x0, 0x0)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/cmd/juju/bootstrap.go:143 +0x65f
main.envCmdWrapper.Run(0xc2002778a0, 0xc200275e00, 0xc200271000, 0xc2001ad238, 0x2, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/cmd/juju/main.go:158 +0x49
main.(*envCmdWrapper).Run(0xc2001f4750, 0xc200271000, 0x14, 0xc2001ad238)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/cmd/juju/addmachine.go:0 +0x8e
launchpad.net/juju-core/cmd.(*SuperCommand).Run(0xc200275d80, 0xc200271000, 0xc200271000, 0x0)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/cmd/supercommand.go:298 +0x3d6
launchpad.net/juju-core/cmd.Main(0xc200280e10, 0xc200275d80, 0xc200271000, 0xc200190010, 0x6, ...)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/cmd/cmd.go:244 +0x286
main.Main(0xc200190000, 0x7, 0x7)
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/cmd/juju/main.go:131 +0x5015
main.main()
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/cmd/juju/main.go:173 +0x42

goroutine 2 [syscall]:

goroutine 4 [syscall]:
os/signal.loop()
 /usr/lib/go/src/pkg/os/signal/signal_unix.go:21 +0x1c
created by os/signal.init��1
 /usr/lib/go/src/pkg/os/signal/signal_unix.go:27 +0x2f

goroutine 30 [chan receive]:
net/http.(*persistConn).readLoop(0xc2001cd100)
 /usr/lib/go/src/pkg/net/http/transport.go:761 +0x64b
created by net/http.(*Transport).dialConn
 /usr/lib/go/src/pkg/net/http/transport.go:511 +0x574

goroutine 15 [runnable]:
main.func��002()
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/cmd/juju/bootstrap.go:111 +0x43
created by main.(*BootstrapCommand).Run
 /build/buildd/juju-core-1.18.1/src/launchpad.net/juju-core/cmd/juju/bootstrap.go:114 +0x250

goroutine 31 [select]:
net/http.(*persistConn).writeLoop(0xc2001cd100)
 /usr/lib/go/src/pkg/net/http/transport.go:774 +0x26f
created by net/http.(*Transport).dialConn
 /usr/lib/go/src/pkg/net/http/transport.go:512 +0x58b

goroutine 37 [select]:
net/http.(*persistConn).writeLoop(0xc2004e9500)
 /usr/lib/go/src/pkg/net/http/transport.go:774 +0x26f
created by net/http.(*Transport).dialConn
 /usr/lib/go/src/pkg/net/http/transport.go:512 +0x58b

goroutine 36 [chan receive]:
net/http.(*persistConn).readLoop(0xc2004e9500)
 /usr/lib/go/src/pkg/net/http/transport.go:761 +0x64b
created by net/http.(*Transport).dialConn
 /usr/lib/go/src/pkg/net/http/transport.go:511 +0x574

goroutine 39 [runnable]:
net/http.(*persistConn).readLoop(0xc2004e9d00)
 /usr/lib/go/src/pkg/net/http/transport.go:761 +0x64b
created by net/http.(*Transport).dialConn
 /usr/lib/go/src/pkg/net/http/transport.go:511 +0x574

goroutine 40 [select]:
net/http.(*persistConn).writeLoop(0xc2004e9d00)
 /usr/lib/go/src/pkg/net/http/transport.go:774 +0x26f
created by net/http.(*Transport).dialConn
 /usr/lib/go/src/pkg/net/http/transport.go:512 +0x58b
Traceback (most recent call last):
  File "/var/lib/jenkins/juju-ci-tools/deploy_job.py", line 4, in <module>
    deploy_job()
  File "/mnt/jenkinshome/juju-ci-tools/deploy_stack.py", line 250, in deploy_job
    bootstrap_host, machines, log_dir, debug)
  File "/mnt/jenkinshome/juju-ci-tools/deploy_stack.py", line 293, in _deploy_job
    bootstrap_from_env(juju_home, env)
  File "/mnt/jenkinshome/juju-ci-tools/deploy_stack.py", line 228, in bootstrap_from_env
    env.bootstrap()
  File "/mnt/jenkinshome/juju-ci-tools/jujupy.py", line 236, in bootstrap
    return self.client.bootstrap(self)
  File "/mnt/jenkinshome/juju-ci-tools/jujupy.py", line 110, in bootstrap
    environment.needs_sudo())
  File "/mnt/jenkinshome/juju-ci-tools/jujupy.py", line 160, in juju
    return subprocess.check_call(args)
  File "/usr/lib/python2.7/subprocess.py", line 511, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '('juju', '--show-log', 'bootstrap', '-e', 'hp-upgrade', '--constraints', 'mem=4G')' returned non-zero exit status 2

Tags: upgrade-juju

Related branches

Curtis Hovey (sinzui)
description: updated
Curtis Hovey (sinzui)
Changed in juju-core:
assignee: nobody → Tim Penhey (thumper)
status: Triaged → In Progress
Revision history for this message
Curtis Hovey (sinzui) wrote :

I demoted this to high because CI was the only affected user.

Changed in juju-core:
status: In Progress → Fix Committed
importance: Critical → High
Curtis Hovey (sinzui)
Changed in juju-core:
milestone: 1.19.3 → 1.19.4
Curtis Hovey (sinzui)
Changed in juju-core:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.