Merge ~mattyw/prometheus-charm:01-configure-rules into prometheus-charm:master

[Matthew Williams]

prometheus.py: The configure_rules function needs to be called immediately, rather than scheduling for later, the rules for state execution are non deterministic, but we require this to be called before promtool validates them

This is a follow on from https://code.launchpad.net/~mattyw/prometheus-charm/+git/prometheus-charm/+merge/330833

The change in this pr fixes a similar issue where deploying the charm with an initial config setting for custom-rules would cause and install hook error.

cat cfg.yaml
prometheus:
    custom-rules: {}

$ juju deploy --config cfg.yaml cs:prometheus

This was because the configure_rules function (where the file is written) would not be called as the check_reconfig_prometheus() sets the prometheus.do-install during install hook execution which means the configure_rules() function is never called (thanks to a when_not). Meaning that when promtool is called as part of write_prometheus_config_yml, the rules file has never been written (and we get the error described in the above link)

After examining the states it became clear that actually calling the function directly rather than setting states would be a much clearer way of executing this function in the right place, as state execution ordering is not defined in charms.reactive

[Casey Marshall (cmars)]

LGTM

review: Approve

[Jacek Nykis (jacekn)]

Merged, composed and pushed to the edge channel.
final tests are running now and once they complete I'll release to stable

review: Approve

diff --git a/reactive/prometheus.py b/reactive/prometheus.py
index ff8a2b1..31f54df 100644
--- a/reactive/prometheus.py
+++ b/reactive/prometheus.py
@@ -61,8 +61,6 @@ def daemon_args():
     return sorted(args_list)
 
 
-@when('prometheus.do-reconfig-rules')
-@when_not('prometheus.do-install')
 def configure_rules():
     config = hookenv.config()
     # Static rules provided as config options
@@ -112,7 +110,6 @@ def configure_rules():
         with open(PROMETHEUS_RULES_CRON, 'w') as f:
             f.write('{} root {} &>/dev/null\n'.format(schedule, REFRESH_RULES_SCRIPT))
         set_state('prometheus.do-reconfig-yml')
-    remove_state('prometheus.do-reconfig-rules')
 
 
 def remove_external_rules():
@@ -430,7 +427,7 @@ def check_reconfig_prometheus():
         hookenv.log('One of options affecting daemon args has changed, scheduling reconfiguration')
         set_state('prometheus.do-reconfig-def')
     if any(config.changed(opt) for opt in rules_change_opts):
-        set_state('prometheus.do-reconfig-rules')
+        configure_rules()
     if any(config.changed(opt) for opt in yml_change_opts):
         set_state('prometheus.do-reconfig-yml')
     if any((
@@ -446,7 +443,7 @@ def check_reconfig_prometheus():
         hookenv.log('Daemon flags template changed, scheduling reconfiguration')
         set_state('prometheus.do-reconfig-def')
     if templates_changed([REFRESH_RULES_SCRIPT_TMPL]):
-        set_state('prometheus.do-reconfig-rules')
+        configure_rules()
     # Cron can update rules on disk, update-status hook can trigger config update
     if kv.get('prometheus.rule_files') != get_rule_files():
         kv.set('prometheus.rule_files', get_rule_files())