configglue

Merge lp:~ricardokirkner/configglue/environ-vars into lp:configglue

  1. environ-vars
  2. Merge into trunk
Proposed by Ricardo Kirkner
Status: Merged
Merged at revision: 58
Proposed branch: lp:~ricardokirkner/configglue/environ-vars
Merge into: lp:configglue
Diff against target: 292 lines (+146/-20)
6 files modified
configglue/pyschema/glue.py (+20/-7)
configglue/pyschema/parser.py (+18/-0)
configglue/pyschema/schema.py (+1/-1)
configglue/tests/pyschema/test_parser.py (+49/-2)
configglue/tests/pyschema/test_schemaconfig.py (+57/-9)
doc/topics/config-file.rst (+1/-1)
To merge this branch: bzr merge lp:~ricardokirkner/configglue/environ-vars
Reviewer Review Type Date Requested Status
Ricardo Kirkner Approve
Review via email: mp+62584@code.launchpad.net

Commit message

add support for environment variables

Description of the change

This branch adds support for environment variables.

Environment variables are supported as follows:

1. In the commandline glue

Variable interpolation is done according to the precedence:
a. commandline parameter (aka --foo-bar=4)
b. environment variable (aka CONFIGGLUE_FOO_BAR=4)
c. configuration files
d. defaults

2. In the configuration files

Variable interpolation is supported for environment variables using the standard notation

$VAR
${VAR}

If the variable is undefined, it resolves to the default value.

To post a comment you must log in.
Revision history for this message
Ricardo Kirkner (ricardokirkner) wrote :

Yes, I like it :)

review: Approve
Revision history for this message
Ubuntu One Auto Pilot (otto-pilot) wrote :
Download full text (22.2 KiB)

The attempt to merge lp:~ricardokirkner/configglue/environ-vars into lp:configglue failed. Below is the output from the failed tests.

tests.inischema.test_attributed
  TestAttributed
    test_config_after_parsing_is_attributed ... [OK]
    test_config_after_parsing_still_knows_about_empty_values ... [OK]
    test_config_before_parsing_is_plain ... [OK]
tests.inischema.test_glue
  TestCrazyGlue
    test_args_win ... [OK]
    test_help_is_displayed ... [OK]
    test_ini_file_wins_when_no_args ... [OK]
  TestGlue
    test_args_win ... [OK]
    test_help_is_displayed ... [OK]
    test_ini_file_wins_when_no_args ... [OK]
  TestGlue2
    test_main ... [OK]
  TestGlue3
    test_accepts_args_and_filenames ... [OK]
    test_empty ... [OK]
  TestGlueBool
    test_store_false ... [OK]
    test_store_true ... [OK]
  TestGlueLines
    test_append ... [OK]
    test_append_on_empty ... [OK]
    test_no_append ... [OK]
    test_nothing ... [OK]
  TestNoValue
    test_args_win ... [OK]
    test_help_is_displayed ... [OK]
    test_ini_file_wins_when_no_args ... [OK]
tests.inischema.test_glue2glue
  TestGlueConvertor
    test_empty ... [OK]
    test_main ... [OK]
    test_parser_bool ... [OK]
    test_parser_int ... [OK]
    test_parser_none ... [OK]
    test_parser_unicode ... [OK]
    test_simple ... [OK]
tests.inischema.test_parsers
  TestParsers
    test_bool ... [OK]
    test_bool_is_None ... [OK]
    test_bool_not_string ... [OK]
    test_lines ... [OK]
    test_lines_not_string ... [OK]
tests.inischema.test_typed
  TestBackwardsCompat
    test_config_before_parse_is_plain ... [OK]
  TestParserd
    test_add_mult...

Revision history for this message
Ubuntu One Auto Pilot (otto-pilot) wrote :
Download full text (29.8 KiB)

The attempt to merge lp:~ricardokirkner/configglue/environ-vars into lp:configglue failed. Below is the output from the failed tests.

tests.inischema.test_attributed
  TestAttributed
    test_config_after_parsing_is_attributed ... [OK]
    test_config_after_parsing_still_knows_about_empty_values ... [OK]
    test_config_before_parsing_is_plain ... [OK]
tests.inischema.test_glue
  TestCrazyGlue
    test_args_win ... [OK]
    test_help_is_displayed ... [OK]
    test_ini_file_wins_when_no_args ... [OK]
  TestGlue
    test_args_win ... [OK]
    test_help_is_displayed ... [OK]
    test_ini_file_wins_when_no_args ... [OK]
  TestGlue2
    test_main ... [OK]
  TestGlue3
    test_accepts_args_and_filenames ... [OK]
    test_empty ... [OK]
  TestGlueBool
    test_store_false ... [OK]
    test_store_true ... [OK]
  TestGlueLines
    test_append ... [OK]
    test_append_on_empty ... [OK]
    test_no_append ... [OK]
    test_nothing ... [OK]
  TestNoValue
    test_args_win ... [OK]
    test_help_is_displayed ... [OK]
    test_ini_file_wins_when_no_args ... [OK]
tests.inischema.test_glue2glue
  TestGlueConvertor
    test_empty ... [OK]
    test_main ... [OK]
    test_parser_bool ... [OK]
    test_parser_int ... [OK]
    test_parser_none ... [OK]
    test_parser_unicode ... [OK]
    test_simple ... [OK]
tests.inischema.test_parsers
  TestParsers
    test_bool ... [OK]
    test_bool_is_None ... [OK]
    test_bool_not_string ... [OK]
    test_lines ... [OK]
    test_lines_not_string ... [OK]
tests.inischema.test_typed
  TestBackwardsCompat
    test_config_before_parse_is_plain ... [OK]
  TestParserd
    test_add_mult...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'configglue/pyschema/glue.py'
2--- configglue/pyschema/glue.py 2011-06-13 12:57:26 +0000
3+++ configglue/pyschema/glue.py 2011-06-19 16:52:32 +0000
4@@ -15,6 +15,7 @@
5 #
6 ###############################################################################
7
8+import os
9 import sys
10 from optparse import OptionParser
11 from collections import namedtuple
12@@ -67,15 +68,27 @@
13 og.add_option('--' + long_name(option), **kwargs)
14 options, args = op.parse_args(argv)
15
16+ def set_value(section, option, value):
17+ # if value is not of the right type, cast it
18+ if not option.validate(value):
19+ value = option.parse(value)
20+ parser.set(section.name, option.name, value)
21+
22 for section in schema.sections():
23 for option in section.options():
24- value = getattr(options, opt_name(option))
25- if parser.get(section.name, option.name) != value:
26- # the value has been overridden by an argument
27- if isinstance(value, basestring):
28- # parse it to the right type if it's a string
29- value = option.parse(value)
30- parser.set(section.name, option.name, value)
31+ # 1. op value != parser value
32+ # 2. op value == parser value != env value
33+ # 3. op value == parser value == env value or not env value
34+
35+ op_value = getattr(options, opt_name(option))
36+ parser_value = parser.get(section.name, option.name)
37+ env_value = os.environ.get("CONFIGGLUE_{0}".format(
38+ long_name(option).upper()))
39+
40+ if op_value != parser_value:
41+ set_value(section, option, op_value)
42+ elif env_value is not None and env_value != parser_value:
43+ set_value(section, option, env_value)
44
45 return op, options, args
46
47
48=== modified file 'configglue/pyschema/parser.py'
49--- configglue/pyschema/parser.py 2011-06-13 18:22:12 +0000
50+++ configglue/pyschema/parser.py 2011-06-19 16:52:32 +0000
51@@ -19,6 +19,7 @@
52 import collections
53 import copy
54 import os
55+import re
56 import string
57
58 from ConfigParser import (
59@@ -435,6 +436,15 @@
60 assert isinstance(result, basestring)
61 return result
62
63+ def interpolate_environment(self, rawval, raw=False):
64+ if raw:
65+ return rawval
66+ # interpolate environment variables
67+ pattern = re.sub(r'\${([A-Z_]+)}', r'%(\1)s', rawval)
68+ pattern = re.sub(r'\$([A-Z_]+)', r'%(\1)s', pattern)
69+ interpolated = pattern % os.environ
70+ return interpolated
71+
72 def _get_default(self, section, option):
73 # mark the value as not initialized to be able to have a None default
74 marker = object()
75@@ -508,6 +518,14 @@
76 self._sections[section] = {}
77 self.set(section, option, value)
78
79+ # interpolate environment variables
80+ if isinstance(value, basestring):
81+ try:
82+ value = self.interpolate_environment(value, raw=raw)
83+ except KeyError:
84+ # interpolation failed, fallback to default value
85+ value = self._get_default(section, option)
86+
87 if parse:
88 value = self.parse(section, option, value)
89 return value
90
91=== modified file 'configglue/pyschema/schema.py'
92--- configglue/pyschema/schema.py 2011-06-18 15:17:49 +0000
93+++ configglue/pyschema/schema.py 2011-06-19 16:52:32 +0000
94@@ -106,7 +106,7 @@
95
96 To define your own configuration schema you should:
97 1- Inherit from Schema
98- 2- Add Option and Sections as class attributes.
99+ 2- Add Options and Sections as class attributes.
100
101 With that your whole configuration schema is defined, and you can now
102 load configuration files.
103
104=== modified file 'configglue/tests/pyschema/test_parser.py'
105--- configglue/tests/pyschema/test_parser.py 2011-06-13 21:11:16 +0000
106+++ configglue/tests/pyschema/test_parser.py 2011-06-19 16:52:32 +0000
107@@ -32,6 +32,7 @@
108
109 from mock import (
110 Mock,
111+ patch,
112 patch_object,
113 )
114
115@@ -261,6 +262,53 @@
116 self.assertRaises(InterpolationMissingOptionError, parser.get,
117 'foo', 'bar')
118
119+ @patch('configglue.pyschema.parser.os')
120+ def test_interpolate_environment_basic_syntax(self, mock_os):
121+ mock_os.environ = {'PATH': 'foo'}
122+ parser = SchemaConfigParser(Schema())
123+ result = parser.interpolate_environment("$PATH")
124+ self.assertEqual(result, 'foo')
125+
126+ @patch('configglue.pyschema.parser.os')
127+ def test_interpolate_environment_extended_syntax(self, mock_os):
128+ mock_os.environ = {'PATH': 'foo'}
129+ parser = SchemaConfigParser(Schema())
130+ result = parser.interpolate_environment("${PATH}")
131+ self.assertEqual(result, 'foo')
132+
133+ @patch('configglue.pyschema.parser.os')
134+ def test_interpolate_environment_in_config(self, mock_os):
135+ mock_os.environ = {'PYTHONPATH': 'foo', 'PATH': 'bar'}
136+ class MySchema(Schema):
137+ pythonpath = StringOption()
138+ path = StringOption()
139+
140+ config = StringIO("[__main__]\npythonpath=${PYTHONPATH}\npath=$PATH")
141+ parser = SchemaConfigParser(MySchema())
142+ parser.readfp(config)
143+ self.assertEqual(parser.values('__main__'),
144+ {'pythonpath': 'foo', 'path': 'bar'})
145+
146+ @patch('configglue.pyschema.parser.os')
147+ def test_get_with_environment_var(self, mock_os):
148+ mock_os.environ = {'FOO': '42'}
149+ class MySchema(Schema):
150+ foo = IntOption()
151+
152+ config = StringIO("[__main__]\nfoo=$FOO")
153+ parser = SchemaConfigParser(MySchema())
154+ parser.readfp(config)
155+ self.assertEqual(parser.get('__main__', 'foo'), 42)
156+
157+ def test_get_without_environment_var(self):
158+ class MySchema(Schema):
159+ foo = IntOption()
160+
161+ config = StringIO("[__main__]\nfoo=$FOO")
162+ parser = SchemaConfigParser(MySchema())
163+ parser.readfp(config)
164+ self.assertEqual(parser.get('__main__', 'foo'), 0)
165+
166 def test_get_interpolation_keys_string(self):
167 """Test get_interpolation_keys for a string."""
168 class MySchema(Schema):
169@@ -1043,8 +1091,7 @@
170 def test_extra_sections_with_nested_dicts_strict(self):
171 """Test parser.is_valid w/ extra sections in a nested dict (strict)."""
172 class MySchema(Schema):
173- foo = DictOption(spec={'bar': DictOption()},
174- strict=True)
175+ foo = DictOption(spec={'bar': DictOption()}, strict=True)
176
177 config = StringIO("""
178 [__main__]
179
180=== modified file 'configglue/tests/pyschema/test_schemaconfig.py'
181--- configglue/tests/pyschema/test_schemaconfig.py 2011-06-13 21:21:16 +0000
182+++ configglue/tests/pyschema/test_schemaconfig.py 2011-06-19 16:52:32 +0000
183@@ -17,10 +17,15 @@
184 ###############################################################################
185
186 import unittest
187+import os
188 import sys
189 from StringIO import StringIO
190
191-from mock import patch, Mock
192+from mock import (
193+ Mock,
194+ patch,
195+ patch_object,
196+)
197
198 from configglue.pyschema.glue import (
199 configglue,
200@@ -32,6 +37,8 @@
201 ConfigSection,
202 IntOption,
203 Option,
204+ Section,
205+ IntOption,
206 Schema,
207 Section,
208 StringOption,
209@@ -154,14 +161,13 @@
210 self.assertEqual(self.parser.values(),
211 {'foo': {'bar': 0}, '__main__': {'baz': 1}})
212
213- _argv = sys.argv
214- sys.argv = []
215-
216- op, options, args = schemaconfigglue(self.parser)
217- self.assertEqual(self.parser.values(),
218- {'foo': {'bar': 0}, '__main__': {'baz': 1}})
219-
220- sys.argv = _argv
221+ _argv, sys.argv = sys.argv, []
222+ try:
223+ op, options, args = schemaconfigglue(self.parser)
224+ self.assertEqual(self.parser.values(),
225+ {'foo': {'bar': 0}, '__main__': {'baz': 1}})
226+ finally:
227+ sys.argv = _argv
228
229 def test_glue_section_option(self):
230 """Test schemaconfigglue overriding one option."""
231@@ -175,6 +181,48 @@
232 self.assertEqual(self.parser.values(),
233 {'foo': {'bar': 2}, '__main__': {'baz': 0}})
234
235+ @patch('configglue.pyschema.glue.os')
236+ def test_glue_environ(self, mock_os):
237+ mock_os.environ = {'CONFIGGLUE_FOO_BAR': '42', 'CONFIGGLUE_BAZ': 3}
238+ config = StringIO("[foo]\nbar=1")
239+ self.parser.readfp(config)
240+
241+ _argv, sys.argv = sys.argv, ['prognam']
242+ try:
243+ op, options, args = schemaconfigglue(self.parser)
244+ self.assertEqual(self.parser.values(),
245+ {'foo': {'bar': 42}, '__main__': {'baz': 3}})
246+ finally:
247+ sys.argv = _argv
248+
249+ @patch('configglue.pyschema.glue.os')
250+ def test_glue_environ_bad_name(self, mock_os):
251+ mock_os.environ = {'FOO_BAR': 2, 'BAZ': 3}
252+ config = StringIO("[foo]\nbar=1")
253+ self.parser.readfp(config)
254+
255+ _argv, sys.argv = sys.argv, ['prognam']
256+ try:
257+ op, options, args = schemaconfigglue(self.parser)
258+ self.assertEqual(self.parser.values(),
259+ {'foo': {'bar': 1}, '__main__': {'baz': 0}})
260+ finally:
261+ sys.argv = _argv
262+
263+ def test_glue_environ_precedence(self):
264+ with patch_object(os, 'environ',
265+ {'CONFIGGLUE_FOO_BAR': '42', 'BAR': '1'}):
266+
267+ config = StringIO("[foo]\nbar=$BAR")
268+ self.parser.readfp(config)
269+
270+ _argv, sys.argv = sys.argv, ['prognam']
271+ try:
272+ op, options, args = schemaconfigglue(self.parser)
273+ self.assertEqual(self.parser.get('foo', 'bar'), 42)
274+ finally:
275+ sys.argv = _argv
276+
277 def test_ambiguous_option(self):
278 """Test schemaconfigglue when an ambiguous option is specified."""
279 class MySchema(Schema):
280
281=== modified file 'doc/topics/config-file.rst'
282--- doc/topics/config-file.rst 2011-06-13 19:21:19 +0000
283+++ doc/topics/config-file.rst 2011-06-19 16:52:32 +0000
284@@ -35,7 +35,7 @@
285
286 A few special considerations have to be kept in mind while working with these
287 configuration files. As ConfigParser requires a config file to have at least
288-one section defined, any top-level Option are added to an implicitely
289+one section defined, any top-level Options are added to an implicitely
290 defined section called ``__main__``.
291
292 Therefore, if you have a schema like::

Subscribers

People subscribed via source and target branches