Merge ~dmitriis/charm-telegraf/+git/telegraf-charm:master into ~telegraf-charmers/charm-telegraf:master

[Dmitrii Shcherbakov]

make mysql and posgresql rels container-scoped

[Stuart Bishop (stub)]

This needs to be tested against a 2 or 3 unit PostgreSQL deployment before landing. Previously this would fail, with the relations to the standby units never completing, but hopefully that has been fixed with an up to date Juju and PostgreSQL charm.

[Stuart Bishop (stub)]

We also need to test how upgrades work. We may not be able to change the scope of the existing relations without breaking all the existing deployments.

[Dmitrii Shcherbakov (dmitriis)]

Agreed, I'll need to spend some time on that. If you have cycles, please help out.

Most likely I'm not going to have time for it today but we'll see.

I think a test on a localhost provider should be relatively easy to do.

This is what I have for percona:

http://paste.ubuntu.com/25591741/

http://paste.ubuntu.com/25591739/

http://paste.ubuntu.com/25591740/

One unit had a problem - this was the last one I've added.
unit-telegraf-4: 13:42:08 INFO unit.telegraf/4.juju-log mysql:2: Reactive main running for hook mysql-relation-joined

...
unit-telegraf-4: 13:42:10 DEBUG unit.telegraf/4.mysql-relation-joined inactive
unit-telegraf-4: 13:42:10 DEBUG unit.telegraf/4.mysql-relation-joined Failed to execute operation: Invalid argument
unit-telegraf-4: 13:42:10 INFO juju.worker.uniter.operation ran "mysql-relation-joined" hook

It looks more like a reactive framework problem or a charm bug because a Juju unit agent correctly invoked mysql-relation-joined and mysql-relation-changed afterwards, correctly processing the scope:container setting.

The built charm is here:
https://jujucharms.com/u/dmitriis/telegraf/xenial/0

https://api.jujucharms.com/charmstore/v5/~dmitriis/xenial/telegraf-0/archive/metadata.yaml
  "mysql":
    "interface": "mysql-root"
    "scope": "container"
  "postgresql":
    "interface": "pgsql"
    "scope": "container"

[Stuart Bishop (stub)]

Upgrades fail with the scope change:

$ juju upgrade-charm --path ~/charms/repo/builds/telegraf telegraf
Added charm "local:xenial/telegraf-0" to the model.
ERROR cannot upgrade application "telegraf" to charm "local:xenial/telegraf-0": would break relation "telegraf:postgresql postgresql:db-admin"

We can't land this as it is. If we need to proceed, we need to declare a new relation with container scope.

review: Disapprove

[Stuart Bishop (stub)]

In addition, the PostgreSQL relation fails with a replicated deployment. If there are two or more PostgreSQL units, then a relation that declares itself container scoped will fail (which was the case when the postgresql relation was first added to the telegraf charm). The standby PostgreSQL units need to see the settings the master unit has made on the relation to copy them, and this fails because the master unit is not visible and relation-get fails:

ERROR cannot read settings for unit "postgresql/1" in relation "telegraf:postgresql postgresql:db-admin": settings not found

[Stuart Bishop (stub)]

(for anyone trying to reproduce the above, the metrics work on the master but you will find the postgresql metrics missing from the standby units)

[Dmitrii Shcherbakov (dmitriis)]

stub, do you still have full logs for this?

ERROR cannot read settings for unit "postgresql/1" in relation "telegraf:postgresql postgresql:db-admin": settings not found

This qualifies as a Juju bug to me.

The upgrade side of it is something that Juju should know how to do as well in my opinion.

Looking at its code I don't see any handling for that so I am not surprised about the results.

What it should provide is a mechanism to say:

* a relation scope has changed
* I should do remove-relation lifecycle at unit level (-broken and -departed for units that are not in a primary-subordinate position)

[Dmitrii Shcherbakov (dmitriis)]

https://bugs.launchpad.net/juju/+bug/1721289
https://bugs.launchpad.net/juju/+bug/1721295

[Stuart Bishop (stub)]

Logs can be obtained by the following:

juju deploy -n2 cs:postgresql
juju deploy ~/charms/builds/telegraf # My test build
juju add-relation telegraf:juju-info postgresql:juju-info
juju add-relation telegraf:postgresql postgresql:db-admin

You can also trigger it manually as often as you like using any container scoped relation and I think is normal behaviour:

$ juju run --unit postgresql/3 'relation-ids juju-info'
juju-info:12
$ juju run --unit postgresql/3 'relation-get -r juju-info:12 - postgresql/3'
private-address: 10.0.4.141
$ juju run --unit postgresql/3 'relation-get -r juju-info:12 - postgresql/4'
ERROR cannot read settings for unit "postgresql/4" in relation "telegraf:juju-info postgresql:juju-info": settings not found

This was Bug #1560262 , which I closed myself for juju as I decided that both ends of a relation need to agree on the scope just like they need to agree on the rest of the protocol. I think the failures in the logs are correct, in that a container scoped relation is container scoped and units cannot see units not in the same container. If there is a bug, it is that the return code is still 0 rather than an error or in charm-helpers, because even though relation-get emits an error nothing else notices and everything thinks there is just no relation data.

But I have never been able to pin down an actual answer about this stuff. If a charm is implementing a relation, does it need to declare it in the same scope as the other end? Or can charms declare any scope and it is supposed to work? If charms declare different scopes, is it guaranteed that the actual relation scope is 'container' or is the actual relation scope arbitrary and subject to change?
Or can a relation somehow have two scopes, with one end working in container scope and the other in standard scope? Is it a charm bug for a charm such as PostgreSQL to declare a normally scoped relation that fails if a client attempts to use it as a container scoped relation, or is it a bug in the other charm for attempting to use the relation incorrectly?

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

Unmerged commits

a06b274... by Dmitrii Shcherbakov on 2017-09-21

make mysql and posgresql rels container-scoped

On large telegraf models this creates a lot of unnecessary events and
message passing.

pad.lv/1718259

diff --git a/metadata.yaml b/metadata.yaml
index a33ad3f..338e66c 100644
--- a/metadata.yaml
+++ b/metadata.yaml
@@ -37,8 +37,10 @@ subordinate: true
 requires:
   mysql:
     interface: mysql-root
+    scope: container
   postgresql: 
     interface: pgsql
+    scope: container
   mongodb: 
     interface: mongodb
     scope: container