Merge lp:~michael.nelson/charm-helpers/namespace-relation-data into lp:charm-helpers

[Michael Nelson]

This branch namespaces the relationship data stored in the salt grains file (for use in the machine states or template files).

It then additionally makes this data persistent. Rather than re-writing the grains each time update_machine_state() is called, it updates the grains instead (so the relation data is collected and/or updated over time, but always available to the templates/machine_states).

Finally, I just added a small host.apt_add_repository that automatically apt-updates by default to keep my own hooks cleaner. Let me know if this is done elsewhere.

I've QA'd this functionality on my charm.

Thanks!

[lp:~michael.nelson/charm-helpers/namespace-relation-data]

43. By Michael Nelson on 2013-06-20

Re-activated two tests that had been deactivated for debugging.

[Matthew Wedgwood (mew)]

Michael,

This looks to be based on your other MP with the hookenv changes. If you can re-base it to trunk, I'll be glad to take another look.

[Matthew Wedgwood (mew)]

Because of r42, this needs rebasing to trunk.

review: Needs Fixing

[lp:~michael.nelson/charm-helpers/namespace-relation-data]

44. By Michael Nelson on 2013-07-09

Merged ensure_etc_salt_exists into namespace-relation-data.

[Michael Nelson (michael.nelson)]

Yes, the other branch/MP is listed above as the prerequisite (which is why the diff only includes the changes between that prereq and this one).

Anyway, I've remerged, so it now includes r42 and the updates to the prerequisite.

Thanks!

[lp:~michael.nelson/charm-helpers/namespace-relation-data]

45. By Michael Nelson on 2013-07-11

Existing grains are updated from new config, rather than new config updated from existing grains.

[Matthew Wedgwood (mew)]

Sorry I'm just getting around to looking at this again.

As apt operations are related to installation of extra, normally external resources, that functionality lives in charmhelpers.fetch. This also keeps core leaning in the direction of being cross-platform (apt, RPM, Mac, Windows, etc.). There's already a related function in fetch. Can you see if that's good enough?

Also, on the subject of running apt-get update, I think it's best to have it default to False as there may be several repos added and running update each time would be inefficient. I would even prefer that it be a separate function.

review: Needs Fixing

[lp:~michael.nelson/charm-helpers/namespace-relation-data]

46. By Michael Nelson on 2013-07-18

Merge trunk.

47. By Michael Nelson on 2013-07-18

Removed host.apt_add_repository

[Matthew Wedgwood (mew)]

Thanks for the changes! Looks good.

review: Approve

[Michael Nelson (michael.nelson)]

After chatting the other day, I just removed the apt_add_repository helper as I wasn't aware of the fetch module.

=== modified file 'charmhelpers/contrib/saltstack/__init__.py'
--- charmhelpers/contrib/saltstack/__init__.py	2013-06-20 07:37:17 +0000
+++ charmhelpers/contrib/saltstack/__init__.py	2013-07-18 18:23:22 +0000
@@ -115,7 +115,15 @@
     # file resources etc.
     config['charm_dir'] = charm_dir
     config['local_unit'] = charmhelpers.core.hookenv.local_unit()
-    config.update(charmhelpers.core.hookenv.relation_get())
+
+    # Add any relation data prefixed with the relation type.
+    relation_type = charmhelpers.core.hookenv.relation_type()
+    if relation_type is not None:
+        relation_data = charmhelpers.core.hookenv.relation_get()
+        relation_data = dict(
+            ("{}:{}".format(relation_type, key), val)
+            for key, val in relation_data.items())
+        config.update(relation_data)
 
     # Don't use non-standard tags for unicode which will not
     # work when salt uses yaml.load_safe.
@@ -127,5 +135,12 @@
     if not os.path.exists(grains_dir):
         os.makedirs(grains_dir)
 
+    if os.path.exists(salt_grains_path):
+        with open(salt_grains_path, "r") as grain_file:
+            grains = yaml.load(grain_file.read())
+    else:
+        grains = {}
+
+    grains.update(config)
     with open(salt_grains_path, "w+") as fp:
-        fp.write(config.yaml())
+        fp.write(yaml.dump(grains))

=== modified file 'tests/contrib/saltstack/test_saltstates.py'
--- tests/contrib/saltstack/test_saltstates.py	2013-06-20 07:34:15 +0000
+++ tests/contrib/saltstack/test_saltstates.py	2013-07-18 18:23:22 +0000
@@ -96,6 +96,10 @@
         self.mock_relation_get = patcher.start()
         self.mock_relation_get.return_value = {}
         self.addCleanup(patcher.stop)
+        patcher = mock.patch('charmhelpers.core.hookenv.relation_type')
+        self.mock_relation_type = patcher.start()
+        self.mock_relation_type.return_value = None
+        self.addCleanup(patcher.stop)
         patcher = mock.patch('charmhelpers.core.hookenv.local_unit')
         self.mock_local_unit = patcher.start()
         self.addCleanup(patcher.stop)
@@ -137,6 +141,7 @@
             'group_code_owner': 'webops_deploy',
             'user_code_runner': 'ubunet',
         })
+        self.mock_relation_type.return_value = 'wsgi-file'
         self.mock_relation_get.return_value = {
             'relation_key1': 'relation_value1',
             'relation_key2': 'relation_value2',
@@ -151,7 +156,38 @@
                 "charm_dir": "/tmp/charm_dir",
                 "group_code_owner": "webops_deploy",
                 "user_code_runner": "ubunet",
-                "relation_key1": "relation_value1",
-                "relation_key2": "relation_value2",
-                "local_unit": "click-index/3",
+                "wsgi-file:relation_key1": "relation_value1",
+                "wsgi-file:relation_key2": "relation_value2",
+                "local_unit": "click-index/3",
+            }, result)
+
+    def test_updates_existing_values(self):
+        """Data stored in grains is retained.
+
+        This may be helpful so that templates can access information
+        from relations outside the current context.
+        """
+        os.makedirs(os.path.dirname(self.grain_path))
+        with open(self.grain_path, 'w+') as grain_file:
+            grain_file.write(yaml.dump({
+                'solr:hostname': 'example.com',
+                'user_code_runner': 'oldvalue',
+            }))
+
+        self.mock_config.return_value = charmhelpers.core.hookenv.Serializable({
+            'group_code_owner': 'webops_deploy',
+            'user_code_runner': 'newvalue',
+        })
+        self.mock_local_unit.return_value = "click-index/3"
+
+        charmhelpers.contrib.saltstack.juju_config_2_grains()
+
+        with open(self.grain_path, 'r') as grain_file:
+            result = yaml.load(grain_file.read())
+            self.assertEqual({
+                "charm_dir": "/tmp/charm_dir",
+                "group_code_owner": "webops_deploy",
+                "user_code_runner": "newvalue",
+                "local_unit": "click-index/3",
+                "solr:hostname": "example.com",
             }, result)