unit failing to get unit-get private-address in the install hook

What version of Juju is this?

Marco,

This is 1.25 on xenial. It is possible this is a juju bug, though I have only seen this on the mysql charm so far.

1.25.5-xenial-amd64

I saw this same issue with someone trying to deploy openstack-base in #juju.. same issue and error on multiple charms. Xenial+1.25
It's unclear to me why private-address would fail to fetch, works OK when I try it manually and just returned the public-address

One thing I also saw that may or may not be related, particularly noted with the lxd provider, is that if you have both IPv4+IPv6 then the private/public address randomly has either the IPv4 or IPv6 address - if you deploy 4 units you might get 1 IPv6 and 3 IPv4 or some other permuation. This causes issues with this code where get_host_ip will fail, but the juju get-address part works fine. So probably it's a separate issue but I mention it just in case it twigs something here.

I think this is a juju bug. See https://bugs.launchpad.net/juju-core/+bug/1484105

FYI: This continues to impact openstack charm test automation, in all charms. It's not specific to percona or mysql.

Example error from UOSCI:

Example:

https://openstack-ci-reports.ubuntu.com/artifacts/test_charm_pipeline_amulet_full/openstack/charm-ceph-osd/298412/7/2016-07-06_18-12-23/index.html

Symptomatically the deployment fails with a hook execution error; juju status reveals that the unit is in a problem state:

mysql/0 error idle 1.25.5 10 3306/tcp hook failed: "shared-db-relation-changed" for glance:shared-db

...

10 pending 1.25.5 a8a71066-18db-4d94-83ee-c0bb0de97635 trusty arch=amd64 cpu-cores=1 mem=1536M root-disk=10240M availability-zone=nova

Changed topic as this is not specific to any one charm.

In ServerStack, this is observable as follows. Juju status output indicates:

 - A machine is stuck in a pending state.

 - The machine has a instance ID.

 - The machine instance is alive and well from the cloud and cloudinit perspective.

 - The unit is reachable and the console log shows normal acquisition of addresses and metadata.

 - The unit has no address in juju status.

 - The unit is in an error workload state, with a failed hook.

It appears that hooks are firing before private addresses are available from Juju.

"The unit is reachable and the console log ..." do you mean the machine? (Generally unit has to do with a particular installation of an application.)

It would be interesting to have the context of cloud-init-output.log and machine-X.log as well as the 'machine-0.log' from the controller.

I don't think we can fire any hooks until we have addresses, because the machine agent is the thing that starts the unit agent, and it would have reported its addresses as the first action.

Its possible it is something like the "juju agent fails to get installed", or "the juju agent fails to contact the controller", which leaves us in a state where we know the machine we want to put the Unit on, but as far as we can tell that machine is not ready for us. (As signaled by the machine agent calling back to the controller.)

As soon as you do a "deploy" or an "add-unit", Juju has a record of the Unit you want to be adding, so just having that unit in "pending" state or waiting for "allocation" doesn't mean we've actually gotten to the point where we can do anything with that unit.

Other sources of context are things like "juju status --format=yaml" which includes a lot of details that don't fit well in tabular view.