Merge lp:~abentley/charms/precise/charmworld/auto-production into lp:~juju-jitsu/charms/precise/charmworld/trunk

[Aaron Bentley]

Auto-generate production.ini

production.ini is now auto-generated from
1. defaults
2. overrides
3. specific settings from any existing production.ini (session.secret,
   mongo.port, mongo.host)
4. juju environment

Once production.ini is removed from the source tree, this will minimize the
chance of conflicts while ensuring production.ini is up-to-date with the
defaults.

https://codereview.appspot.com/7229057/

[Richard Harding (rharding)]

lgtm

I wonder a little bit if we can reuse the configparser merging code abel
has in charmworld/lib/config, but I suppose it's different enough since
this code also has field limitations defined on what can be overridden.

Should there be some documentation around to help aid other people
working on the charm on how the parts fit together? If we handed this to
abel or jc would it be clear how the ini was generated and used in
production?

https://codereview.appspot.com/7229057/

[Aaron Bentley (abentley)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 13-01-29 11:11 AM, <email address hidden> wrote:
> I wonder a little bit if we can reuse the configparser merging code
> abel has in charmworld/lib/config

The code in charmworld/lib/config doesn't do the merging, it delegates
that to ConfigParser.read, the same as mine does.

The bulk of charmworld/lib/config.py is concerned with emitting
warnings when sections and options are used in the override that are
not present in the default. However, this does not indicate a
problem. For example, the "logger_exc_logger" section in
production_overrides.ini is entirely legitimate, and it would be
inappropriate in the default, because the default logging
configuration is different. So that code is not useful in this context.

If I wanted to use Abel's script anyhow, it would need to take 3
inputs: defaults, production overrides, local configuration. Since it
doesn't, it's not useful here. If it did, I'd still need a Python
script. It would use ConfigParser to write out a temporary config for
local configuration.

> Should there be some documentation around to help aid other people
> working on the charm on how the parts fit together?

To me, the ideal is that the code is clear enough that documentation
is superfluous. I'm not sure whether we've reached that here.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEARECAAYFAlEH+NgACgkQ0F+nu1YWqI2TVwCghD7+Rvvcornf81vhFAJZIlwR
TicAn1rcTH29iQ5IRO8yOfwfKjzD63iD
=8cOq
-----END PGP SIGNATURE-----

=== modified file 'common'
--- common	2013-01-10 20:38:14 +0000
+++ common	2013-01-29 15:55:29 +0000
@@ -1,8 +1,6 @@
 project_dir="/home/ubuntu/charmworld"
 CONFIG_FILE="$project_dir/production.ini"
-configure_mongo(){
-  db_host=${1-}
-  db_port=${2-}
-  sed -i -e "s/mongo.host =.*/mongo.host = ${db_host}/" \
-      -e "s/mongo.port =.*/mongo.port = ${db_port}/" $CONFIG_FILE
+configure(){
+  ./write_config.py $project_dir/default.ini production_overrides.ini\
+      $CONFIG_FILE "$@"
 }

=== modified file 'hooks/config-changed'
--- hooks/config-changed	2013-01-24 17:34:53 +0000
+++ hooks/config-changed	2013-01-29 15:55:29 +0000
@@ -33,6 +33,9 @@
     make clean
     make install
     pip install pyramid_exclog
+    cd $old_dir
+    configure
+    cd $project_dir
     chown ubuntu.ubuntu . -R
     if grep -E 'mongo.host = .+' $CONFIG_FILE; then
         INI=$CONFIG_FILE bin/migrations upgrade
@@ -51,10 +54,6 @@
     update_source
 fi
 
-secret=$(head -c 3 /dev/urandom|sha1sum|cut -f 1 -d ' ')
-# Supply secret, if unspecified.
-sed -i -e "s/session.secret =$/session.secret = ${secret}/" $CONFIG_FILE
-
 interval=`config-get execute-ingest-every`
 : ${interval:-0}
 

=== modified file 'hooks/database-relation-broken'
--- hooks/database-relation-broken	2013-01-10 20:19:05 +0000
+++ hooks/database-relation-broken	2013-01-29 15:55:29 +0000
@@ -4,4 +4,4 @@
 hooks/stop || true
 source common
 
-configure_mongo
+configure '' ''

=== modified file 'hooks/database-relation-changed'
--- hooks/database-relation-changed	2013-01-24 17:34:53 +0000
+++ hooks/database-relation-changed	2013-01-29 15:55:29 +0000
@@ -18,7 +18,7 @@
 relation_port=`relation-get port`
 DB_PORT=${relation_port:-27017}
 
-configure_mongo $DB_HOST $DB_PORT
+configure $DB_HOST $DB_PORT
 
 INI=$CONFIG_FILE $project_dir/bin/migrations upgrade --init
 

=== added file 'production_overrides.ini'
--- production_overrides.ini	1970-01-01 00:00:00 +0000
+++ production_overrides.ini	2013-01-29 15:55:29 +0000
@@ -0,0 +1,46 @@
+[app:main]
+pyramid.debug_templates = false
+pyramid.includes = pyramid_exclog
+pyramid.reload_templates = false
+mongo.host =
+mongo.port =
+session.secret =
+
+[server:main]
+port = 6543
+
+[loggers]
+keys = root, charmworld, exc_logger
+
+[handlers]
+keys = console, filelog
+
+[formatters]
+keys = generic, exception
+
+[logger_root]
+level = WARN
+
+handlers = console, filelog
+[logger_charmworld]
+level = WARN
+
+[logger_exc_logger]
+level = ERROR
+handlers =
+qualname = exc_logger
+
+[handler_filelog]
+class = FileHandler
+args = ('/home/ubuntu/var/app.log', 'a')
+level = INFO
+formatter = generic
+
+[handler_exc_handler]
+class = StreamHandler
+args = (sys.stderr,)
+level = ERROR
+formatter = exc_formatter
+
+[formatter_exception]
+format = %(asctime)s %(message)s

=== modified file 'revision'
--- revision	2013-01-24 17:34:53 +0000
+++ revision	2013-01-29 15:55:29 +0000
@@ -1,1 +1,1 @@
-17
+18

=== added file 'write_config.py'
--- write_config.py	1970-01-01 00:00:00 +0000
+++ write_config.py	2013-01-29 15:55:29 +0000
@@ -0,0 +1,57 @@
+#!/usr/bin/env python
+
+from ConfigParser import ConfigParser, NoSectionError
+from random import getrandbits
+import sys
+
+
+def update_config(parser, items):
+    configurable = set(['session.secret', 'mongo.host', 'mongo.port'])
+    for key, value in items:
+        if key not in configurable:
+            continue
+        parser.set('app:main', key, value)
+
+
+def read_config(files):
+    new_config = ConfigParser()
+    new_config.read(files)
+    return new_config
+
+
+def write_new_config(changes, default, override, target):
+    new_config = read_config([default, override])
+    old_config = read_config(target)
+    try:
+        update_config(new_config, old_config.items('app:main', raw=True))
+    except NoSectionError:
+        pass
+    update_config(new_config, changes)
+    if new_config.get('app:main', 'session.secret') == '':
+        new_config.set('app:main', 'session.secret', str(getrandbits(32)))
+    with open(target, 'w') as output:
+        new_config.write(output)
+
+
+class UsageError(Exception):
+    """An error triggered by incorrect usage"""
+
+
+def main():
+    if len(sys.argv) < 4:
+        raise UsageError('Usage: %s DEFAULTS OVERRIDES TARGET [HOST [PORT]]')
+    changes = {}
+    if len(sys.argv) > 4:
+        changes['mongo.host'] = sys.argv[4]
+    if len(sys.argv) > 5:
+        changes['mongo.port'] = sys.argv[5]
+    parser = write_new_config(changes.items(), default=sys.argv[1],
+                              override=sys.argv[2], target=sys.argv[3])
+
+
+if __name__ == '__main__':
+    try:
+        main()
+    except UsageError as e:
+        sys.stderr.write(str(e))
+        sys.exit(1)