config-changed fails on inactive rabbitmq-server unit when configured with HA

rabbitmq-server config-changed hook attempts to add a vhost, but will fail on an inactive rabbitmq server when configured in HA because rabbitmq is not running on that node.

unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed subprocess.CalledProcessError: Command '['/usr/sbin/rabbitmqctl', 'add_vhost', 'nagios-rabbitmq-server-0']' returned non-zero exit status 2

$ juju status rabbitmq-server
...
  rabbitmq-server:
    charm: local:precise/rabbitmq-server-118
    exposed: false
    relations:
      amqp:
      - cinder
      - nova-cloud-controller
      - nova-compute
      - quantum-gateway
      ceph:
      - ceph
      cluster:
      - rabbitmq-server
      ha:
      - rabbitmq-hacluster
    units:
      rabbitmq-server/0:
        agent-state: error
        agent-state-info: 'hook failed: "config-changed"'
        agent-version: 1.18.1
        machine: "23"
        open-ports:
        - 5672/tcp
        public-address: rabbitmq001.parsedcloud.com
        subordinates:
          rabbitmq-hacluster/0:
            upgrading-from: local:precise/hacluster-69
            agent-state: started
            agent-version: 1.18.1
            public-address: rabbitmq001.parsedcloud.com
      rabbitmq-server/1:
        agent-state: started
        agent-version: 1.18.1
        machine: "24"
        open-ports:
        - 5672/tcp
        public-address: rabbitmq002.parsedcloud.com
        subordinates:
          rabbitmq-hacluster/1:
            upgrading-from: local:precise/hacluster-69
            agent-state: started
            agent-version: 1.18.1
            public-address: rabbitmq002.parsedcloud.com

$ juju resolved --retry rabbitmq-server/0
$ juju debug-log
...
unit-rabbitmq-server-0: 2014-05-01 15:10:23 INFO config-changed #015Reading package lists... 0%#015#015Reading package lists... 100%#015#015Reading package lists... Done
unit-rabbitmq-server-0: 2014-05-01 15:10:23 INFO config-changed #015Building dependency tree... 0%#015#015Building dependency tree... 0%#015#015Building dependency tree... 50%#015#015Building dependency tree... 50%#015#015Building dependency tree
unit-rabbitmq-server-0: 2014-05-01 15:10:23 INFO config-changed #015Reading state information... 0%#015#015Reading state information... 0%#015#015Reading state information... Done
unit-rabbitmq-server-0: 2014-05-01 15:10:23 INFO juju-log Installing [] with options: ['--option=Dpkg::Options::=--force-confold']
unit-rabbitmq-server-0: 2014-05-01 15:10:23 INFO config-changed Reading package lists...
unit-rabbitmq-server-0: 2014-05-01 15:10:23 INFO config-changed Building dependency tree...
unit-rabbitmq-server-0: 2014-05-01 15:10:23 INFO config-changed Reading state information...
unit-rabbitmq-server-0: 2014-05-01 15:10:23 INFO config-changed The following packages were automatically installed and are no longer required:
unit-rabbitmq-server-0: 2014-05-01 15:10:23 INFO config-changed libgudev-1.0-0 gir1.2-gudev-1.0
unit-rabbitmq-server-0: 2014-05-01 15:10:23 INFO config-changed Use 'apt-get autoremove' to remove them.
unit-rabbitmq-server-0: 2014-05-01 15:10:23 INFO config-changed 0 upgraded, 0 newly installed, 0 to remove and 19 not upgraded.
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Get:1 http://security.ubuntu.com precise-security Release.gpg [198 B]
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Hit http://archive.ubuntu.com precise Release.gpg
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Get:2 http://security.ubuntu.com precise-security Release [49.6 kB]
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Get:3 http://archive.ubuntu.com precise-updates Release.gpg [198 B]
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Get:4 http://ubuntu-cloud.archive.canonical.com precise-updates/havana Release.gpg [543 B]
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Hit http://archive.ubuntu.com precise Release
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Get:5 http://archive.ubuntu.com precise-updates Release [49.6 kB]
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Get:6 http://ubuntu-cloud.archive.canonical.com precise-updates/cloud-tools Release.gpg [543 B]
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Get:7 http://security.ubuntu.com precise-security/main amd64 Packages [381 kB]
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Hit http://ubuntu-cloud.archive.canonical.com precise-updates/havana Release
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Hit http://ubuntu-cloud.archive.canonical.com precise-updates/cloud-tools Release
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Hit http://archive.ubuntu.com precise/main amd64 Packages
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Hit http://archive.ubuntu.com precise/restricted amd64 Packages
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Hit http://ubuntu-cloud.archive.canonical.com precise-updates/havana/main amd64 Packages
unit-rabbitmq-server-0: 2014-05-01 15:10:24 INFO config-changed Hit http://archive.ubuntu.com precise/universe amd64 Packages
unit-rabbitmq-server-0: 2014-05-01 15:10:25 INFO config-changed Hit http://archive.ubuntu.com precise/multiverse amd64 Packages
unit-rabbitmq-server-0: 2014-05-01 15:10:25 INFO config-changed Ign http://ubuntu-cloud.archive.canonical.com precise-updates/havana/main TranslationIndex
unit-rabbitmq-server-0: 2014-05-01 15:10:25 INFO config-changed Hit http://archive.ubuntu.com precise/main TranslationIndex
unit-rabbitmq-server-0: 2014-05-01 15:10:25 INFO config-changed Hit http://ubuntu-cloud.archive.canonical.com precise-updates/cloud-tools/main amd64 Packages
unit-rabbitmq-server-0: 2014-05-01 15:10:25 INFO config-changed Hit http://archive.ubuntu.com precise/multiverse TranslationIndex
unit-rabbitmq-server-0: 2014-05-01 15:10:25 INFO config-changed Hit http://archive.ubuntu.com precise/restricted TranslationIndex
unit-rabbitmq-server-0: 2014-05-01 15:10:25 INFO config-changed Ign http://ubuntu-cloud.archive.canonical.com precise-updates/cloud-tools/main TranslationIndex
unit-rabbitmq-server-0: 2014-05-01 15:10:25 INFO config-changed Hit http://archive.ubuntu.com precise/universe TranslationIndex
unit-rabbitmq-server-0: 2014-05-01 15:10:25 INFO config-changed Get:8 http://archive.ubuntu.com precise-updates/main amd64 Packages [773 kB]
unit-rabbitmq-server-0: 2014-05-01 15:10:25 INFO config-changed Get:9 http://security.ubuntu.com precise-security/restricted amd64 Packages [4627 B]
unit-rabbitmq-server-0: 2014-05-01 15:10:25 INFO config-changed Get:10 http://security.ubuntu.com precise-security/universe amd64 Packages [91.8 kB]
unit-rabbitmq-server-0: 2014-05-01 15:10:26 INFO config-changed Get:11 http://security.ubuntu.com precise-security/multiverse amd64 Packages [2439 B]
unit-rabbitmq-server-0: 2014-05-01 15:10:26 INFO config-changed Get:12 http://security.ubuntu.com precise-security/main TranslationIndex [74 B]
unit-rabbitmq-server-0: 2014-05-01 15:10:26 INFO config-changed Get:13 http://security.ubuntu.com precise-security/multiverse TranslationIndex [72 B]
unit-rabbitmq-server-0: 2014-05-01 15:10:26 INFO config-changed Get:14 http://security.ubuntu.com precise-security/restricted TranslationIndex [72 B]
unit-rabbitmq-server-0: 2014-05-01 15:10:26 INFO config-changed Get:15 http://security.ubuntu.com precise-security/universe TranslationIndex [73 B]
unit-rabbitmq-server-0: 2014-05-01 15:10:26 INFO config-changed Ign http://ubuntu-cloud.archive.canonical.com precise-updates/havana/main Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:26 INFO config-changed Hit http://security.ubuntu.com precise-security/main Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:27 INFO config-changed Ign http://ubuntu-cloud.archive.canonical.com precise-updates/cloud-tools/main Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:27 INFO config-changed Hit http://security.ubuntu.com precise-security/multiverse Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:27 INFO config-changed Hit http://security.ubuntu.com precise-security/restricted Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:27 INFO config-changed Hit http://security.ubuntu.com precise-security/universe Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:27 INFO config-changed Get:16 http://archive.ubuntu.com precise-updates/restricted amd64 Packages [12.2 kB]
unit-rabbitmq-server-0: 2014-05-01 15:10:27 INFO config-changed Get:17 http://archive.ubuntu.com precise-updates/universe amd64 Packages [240 kB]
unit-rabbitmq-server-0: 2014-05-01 15:10:27 INFO config-changed Get:18 http://archive.ubuntu.com precise-updates/multiverse amd64 Packages [15.3 kB]
unit-rabbitmq-server-0: 2014-05-01 15:10:27 INFO config-changed Hit http://archive.ubuntu.com precise-updates/main TranslationIndex
unit-rabbitmq-server-0: 2014-05-01 15:10:27 INFO config-changed Hit http://archive.ubuntu.com precise-updates/multiverse TranslationIndex
unit-rabbitmq-server-0: 2014-05-01 15:10:27 INFO config-changed Hit http://archive.ubuntu.com precise-updates/restricted TranslationIndex
unit-rabbitmq-server-0: 2014-05-01 15:10:27 INFO config-changed Hit http://archive.ubuntu.com precise-updates/universe TranslationIndex
unit-rabbitmq-server-0: 2014-05-01 15:10:28 INFO config-changed Hit http://archive.ubuntu.com precise/main Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:28 INFO config-changed Hit http://archive.ubuntu.com precise/multiverse Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:28 INFO config-changed Hit http://archive.ubuntu.com precise/restricted Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:28 INFO config-changed Hit http://archive.ubuntu.com precise/universe Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:28 INFO config-changed Hit http://archive.ubuntu.com precise-updates/main Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:28 INFO config-changed Hit http://archive.ubuntu.com precise-updates/multiverse Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:28 INFO config-changed Hit http://archive.ubuntu.com precise-updates/restricted Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:28 INFO config-changed Hit http://archive.ubuntu.com precise-updates/universe Translation-en
unit-rabbitmq-server-0: 2014-05-01 15:10:28 INFO config-changed Fetched 1621 kB in 4s (325 kB/s)
unit-rabbitmq-server-0: 2014-05-01 15:10:31 INFO config-changed Reading package lists...
unit-rabbitmq-server-0: 2014-05-01 15:10:31 INFO juju-log Installing ['rabbitmq-server', 'python-amqplib'] with options: ['--option=Dpkg::Options::=--force-confold']
unit-rabbitmq-server-0: 2014-05-01 15:10:31 INFO config-changed Reading package lists...
unit-rabbitmq-server-0: 2014-05-01 15:10:31 INFO config-changed Building dependency tree...
unit-rabbitmq-server-0: 2014-05-01 15:10:31 INFO config-changed Reading state information...
unit-rabbitmq-server-0: 2014-05-01 15:10:31 INFO config-changed python-amqplib is already the newest version.
unit-rabbitmq-server-0: 2014-05-01 15:10:31 INFO config-changed rabbitmq-server is already the newest version.
unit-rabbitmq-server-0: 2014-05-01 15:10:31 INFO config-changed The following packages were automatically installed and are no longer required:
unit-rabbitmq-server-0: 2014-05-01 15:10:31 INFO config-changed libgudev-1.0-0 gir1.2-gudev-1.0
unit-rabbitmq-server-0: 2014-05-01 15:10:31 INFO config-changed Use 'apt-get autoremove' to remove them.
unit-rabbitmq-server-0: 2014-05-01 15:10:32 INFO config-changed 0 upgraded, 0 newly installed, 0 to remove and 20 not upgraded.
unit-rabbitmq-server-0: 2014-05-01 15:10:32 INFO juju-log Changing ownership of path /var/lib/rabbitmq to rabbitmq:rabbitmq
unit-rabbitmq-server-0: 2014-05-01 15:10:32 INFO juju-log Changing perms of path /var/lib/rabbitmq
unit-rabbitmq-server-0: 2014-05-01 15:10:32 INFO config-changed Plugin configuration unchanged.
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO juju-log Deferring action to CRM leader.
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed Error: unable to connect to node 'rabbitmq-server@localhost': nodedown
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed diagnostics:
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed - nodes and their ports on localhost: [{rabbitmqctl2736,36527}]
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed - current node: rabbitmqctl2736@rabbitmq001
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed - current node home dir: /var/lib/rabbitmq
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed - current node cookie hash: I3TrTzbPjkQi0cj4jYlLXg==
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed Creating vhost "nagios-rabbitmq-server-0" ...
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed Error: unable to connect to node 'rabbitmq-server@localhost': nodedown
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed diagnostics:
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed - nodes and their ports on localhost: [{rabbitmqctl2766,52719}]
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed - current node: rabbitmqctl2766@rabbitmq001
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed - current node home dir: /var/lib/rabbitmq
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed - current node cookie hash: I3TrTzbPjkQi0cj4jYlLXg==
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed Traceback (most recent call last):
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed File "/var/lib/juju/agents/unit-rabbitmq-server-0/charm/hooks/config-changed", line 545, in <module>
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed hooks.execute(sys.argv)
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed File "/var/lib/juju/agents/unit-rabbitmq-server-0/charm/hooks/charmhelpers/core/hookenv.py", line 381, in execute
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed self._hooks[hook_name]()
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed File "/var/lib/juju/agents/unit-rabbitmq-server-0/charm/hooks/config-changed", line 534, in config_changed
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed update_nrpe_checks()
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed File "/var/lib/juju/agents/unit-rabbitmq-server-0/charm/hooks/config-changed", line 369, in update_nrpe_checks
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed rabbit.create_vhost(vhost)
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed File "/var/lib/juju/agents/unit-rabbitmq-server-0/charm/hooks/rabbit_utils.py", line 61, in create_vhost
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed subprocess.check_call(cmd)
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed File "/usr/lib/python2.7/subprocess.py", line 511, in check_call
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed raise CalledProcessError(retcode, cmd)
unit-rabbitmq-server-0: 2014-05-01 15:10:34 INFO config-changed subprocess.CalledProcessError: Command '['/usr/sbin/rabbitmqctl', 'add_vhost', 'nagios-rabbitmq-server-0']' returned non-zero exit status 2
unit-rabbitmq-server-0: 2014-05-01 15:10:34 ERROR juju.worker.uniter uniter.go:486 hook failed: exit status 1