LXC / Local provider machines do not boot without default-series

Bug #1309805 reported by Charles Butler
40
This bug affects 8 people
Affects Status Importance Assigned to Milestone
juju-core
Fix Released
High
José Antonio Rey

Bug Description

LXC Machines fail to boot on 1.18 and 1.19 local provider.

machine-0.log repeats missing tool version as the culprit

2014-04-19 00:20:21 ERROR juju runner.go:220 worker: exited "environ-provisioner": failed to process updated machines: cannot start machine 1: no matching tools available

WORK AROUND
The issue appears to be ambiguous series in the env and deployments. The "default-series" option can be added to the environments.yaml to tell juju what you mean.
    default-series: trusty
or
    default-series: precise

See "Working with Ubuntu 12.04 Precise and 14.04 Trusty" in the 1.18.1 section of https://juju.ubuntu.com/docs/reference-release-notes.html

SOLVING

If juju init included a comment about default-series, users could resolve this issue without reporting a bug.

Related branches

José Antonio Rey (jose)
Changed in juju-core:
status: New → Confirmed
Revision history for this message
Curtis Hovey (sinzui) wrote :

We cannot reproduce this. Which version are you using? Where are you deploying too? Is tools-metadata-url set? What is your command to bootstrap?

Changed in juju-core:
status: Confirmed → Incomplete
Revision history for this message
Sridher Jakku (శ్రీధర్ జక్కు) (sridher) wrote :
Download full text (43.8 KiB)

js@js:~$ juju --version
1.19.0-trusty-amd64
js@js:~$
js@js:~$ juju init
A boilerplate environment configuration file has been written to /home/js/.juju/environments.yaml.
Edit the file to configure your juju environment and run bootstrap.
js@js:~$ juju switch local
amazon -> local
js@js:~$ juju bootstrap
uploading tools for series [trusty]
Logging to /home/js/.juju/local/cloud-init-output.log on remote host
Bootstrapping Juju machine agent
Starting Juju machine agent (juju-agent-js-local)
js@js:~$ juju status
environment: local
machines:
  "0":
    agent-state: started
    agent-version: 1.19.0.1
    dns-name: localhost
    instance-id: localhost
    series: trusty
services: {}
js@js:~$ juju-quickstart bundle:~charmers/rails/6/example-scalable
juju quickstart v1.3.1
contents loaded for bundle example-scalable (services: 10)
bootstrapping the local environment (type: local)
sudo privileges will be required to bootstrap the environment
reusing the already bootstrapped local environment
retrieving the environment status
retrieving the Juju API address
connecting to wss://10.0.3.1:17070
requesting juju-gui deployment
charm URL: cs:precise/juju-gui-89
juju-gui deployment request accepted
exposing service juju-gui
requesting new unit deployment
juju-gui/0 deployment request accepted
juju-gui/0 deployment is pending
machine 1 provisioning is pending

===========================Here Hangs======================
output from other terminal:

js@js:~/Repo$ juju status
environment: local
machines:
  "0":
    agent-state: started
    agent-version: 1.19.0.1
    dns-name: localhost
    instance-id: localhost
    series: trusty
  "1":
    instance-id: pending
    series: precise
services:
  juju-gui:
    charm: cs:precise/juju-gui-89
    exposed: true
    units:
      juju-gui/0:
        agent-state: pending
        machine: "1"

===============================
machine-0.log

2014-04-21 13:26:43 INFO juju.cmd supercommand.go:296 running juju-1.19.0.1-trusty-amd64 [gc]
2014-04-21 13:26:43 INFO juju.cmd.jujud machine.go:148 machine agent machine-0 start (1.19.0.1-trusty-amd64 [gc])
2014-04-21 13:26:43 DEBUG juju.agent agent.go:365 read agent config, format "1.18"
2014-04-21 13:26:43 INFO juju runner.go:262 worker: start "api"
2014-04-21 13:26:43 INFO juju runner.go:262 worker: start "statestarter"
2014-04-21 13:26:43 INFO juju runner.go:262 worker: start "termination"
2014-04-21 13:26:43 INFO juju.state.api apiclient.go:201 dialing "wss://localhost:17070/"
2014-04-21 13:26:43 INFO juju runner.go:262 worker: start "state"
2014-04-21 13:26:43 INFO juju.agent.mongo mongo.go:196 Ensuring mongo server is running; dataDir /home/js/.juju/local; port 37017
2014-04-21 13:26:43 INFO juju.state open.go:81 opening state, mongo addresses: ["localhost:37017"]; entity "machine-0"
2014-04-21 13:26:43 DEBUG juju.state open.go:86 dialing mongo
2014-04-21 13:26:43 INFO juju.state.api apiclient.go:209 error dialing "wss://localhost:17070/": websocket.Dial wss://localhost:17070/: dial tcp 127.0.0.1:17070: connection refused
2014-04-21 13:26:43 ERROR juju runner.go:220 worker: exited "api": timed out connecting to "wss://localhost:17070/"
2014-04-21 13:26:43 INFO juju runner.go...

Changed in juju-core:
status: Incomplete → Confirmed
Curtis Hovey (sinzui)
Changed in juju-core:
status: Confirmed → Incomplete
Revision history for this message
Patrick Seemann (patrickseemann) wrote :

$ juju version
1.18.1-trusty-amd64

$ juju init
$ juju switch local
$ juju bootstrap
$ juju deploy juju-gui
$ juju status

environment: local
machines:
  "0":
    agent-state: started
    agent-version: 1.18.1.1
    dns-name: localhost
    instance-id: localhost
    series: trusty
  "1":
    agent-state: started
    agent-version: 1.18.1.1
    dns-name:
    instance-id: seppel-local-machine-1
    series: trusty
    hardware: arch=amd64
services:
  juju-gui:
    charm: cs:trusty/juju-gui-1
    exposed: false
    units:
      juju-gui/0:
        agent-state: started
        agent-version: 1.18.1.1
        machine: "1"
        open-ports:
        - 80/tcp
        - 443/tcp
        public-address:

then after:

$ juju deploy owncloud (or another charm like mysql)
$ juju status

environment: local
machines:
...
  "4":
    instance-id: pending
    series: precise
services:
 ...
  owncloud:
    charm: cs:precise/owncloud-13
    exposed: false
    units:
      owncloud/0:
        agent-state: pending
        machine: "4

$ tail .juju/local/log/all-machines.log

machine-0: 2014-04-21 13:58:13 ERROR juju runner.go:220 worker: exited "environ-provisioner": failed to process updated machines: cannot start machine 4: no matching tools available

tools-metadata-url in .juju/environments/local.jenv is set to: tools-metadata-url: ""

Revision history for this message
Curtis Hovey (sinzui) wrote :

I still cannot reproduce this issue. శ్రీధర్ జక్కు issue is proably different from Charles Butler. A couple issue can be causing this issue:
    1. Tools can be misisng when the local machine doesn't match the series and arch of the env
      and the env cannot talk to streams.canonical.com
      * juju sync-tools will place tools in an env before bootstrap. This can help identify network issues.

    2. Stale lxc image cache will cause a stuck in "Pending" situation
      * Purge the cache. precise images were updated recently to close heartbleed.

    3. quickstart may have issues,
      but that is a separate project.

    4. The juju-gui charm may have issues,
      but that is a separate project

Revision history for this message
Charles Butler (lazypower) wrote :

$ juju --version
1.19.0-saucy-amd64

$ uname -ar
Linux Bushido 3.11.0-19-generic #33-Ubuntu SMP Tue Mar 11 18:48:34 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

charles@Bushido:~/Projects/cabs/cabs-client-cli$ juju bootstrap
uploading tools for series [saucy trusty]
[sudo] password for charles:
Logging to /home/charles/.juju/local/cloud-init-output.log on remote host
Bootstrapping Juju machine agent
Starting Juju machine agent (juju-agent-charles-local)

$ juju switch local
$ juju bootstrap
$ juju deploy cs:precise/ubuntu

Revision history for this message
Matt Bruzek (mbruzek) wrote :

I am not able to get any charms to deploy on my trusty local environment. The bootstrap appears to complete successfully but all my charms stay in the pending state.

uname -r
3.13.0-24-generic
juju version
1.19.0-trusty-amd64
juju bootstrap -e local --upload-tools
uploading tools for series [trusty]
Logging to /home/mbruzek/.juju/local/cloud-init-output.log on remote host
Bootstrapping Juju machine agent
Starting Juju machine agent (juju-agent-mbruzek-local)
tail ~/.juju/local/log/all-machines.log
machine-0: 2014-04-21 14:46:50 ERROR juju runner.go:220 worker: exited "environ-provisioner": failed to process updated machines: cannot start machine 1: no matching tools available

Revision history for this message
Curtis Hovey (sinzui) wrote :

I am not seeing this issue, but maybe my setup is tainted. I have been on trusty for 6 months. I set the default series to precise because I know there are only a few trusty charms.

My env:
  local:
    type: local
    default-series: precise

juju switch local
juju bootstrap
juju deploy ubuntu
juju status
environment: local
machines:
  "0":
    agent-state: started
    agent-version: 1.19.0.1
    dns-name: localhost
    instance-id: localhost
    series: trusty
  "1":
    agent-state: started
    agent-version: 1.19.0.1
    dns-name: 10.0.3.96
    instance-id: curtis-local-machine-1
    series: precise
    hardware: arch=amd64
services:
  ubuntu:
    charm: cs:precise/ubuntu-4
    exposed: false
    units:
      ubuntu/0:
        agent-state: started
        agent-version: 1.19.0.1
        machine: "1"
        public-address: 10.0.3.96

Attached is the log from my successful deploy

Curtis Hovey (sinzui)
description: updated
Curtis Hovey (sinzui)
summary: - LXC / Local provider machines do not boot in 1.18 / 1.19 series
+ LXC / Local provider machines do not boot without default-series
tags: added: config local-provider lxc
Changed in juju-core:
status: Incomplete → Triaged
importance: Undecided → High
description: updated
Changed in juju-core:
milestone: none → 1.19.1
Revision history for this message
Patrick Seemann (patrickseemann) wrote :

Setting "default-series: precise" in environments.yaml and then destroying (juju destroy-environment local) and rebuilding the local environment solved it for me! Thanks.

Ian Booth (wallyworld)
Changed in juju-core:
assignee: nobody → José Antonio Rey (jose)
status: Triaged → Fix Committed
Revision history for this message
Sridher Jakku (శ్రీధర్ జక్కు) (sridher) wrote :

adding " --series=trusty,precise --upload-tools=true " works for me
     $ juju bootstrap --series=trusty,precise --upload-tools=true

Curtis Hovey (sinzui)
Changed in juju-core:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.