Merge lp:~stub/charms/precise/postgresql/pg93 into lp:charms/postgresql

Proposed by Stuart Bishop
Status: Merged
Merged at revision: 86
Proposed branch: lp:~stub/charms/precise/postgresql/pg93
Merge into: lp:charms/postgresql
Prerequisite: lp:~stub/charms/precise/postgresql/tests
Diff against target: 1315 lines (+538/-184)
19 files modified
.bzrignore (+3/-0)
Makefile (+20/-3)
README.md (+6/-3)
config.yaml (+38/-15)
hooks/charmhelpers/fetch/__init__.py (+2/-2)
hooks/hooks.py (+283/-132)
lib/ACCC4CF8.asc (+45/-0)
templates/pg_hba.conf.tmpl (+1/-1)
templates/pg_ident.conf.tmpl (+1/-0)
templates/postgresql.conf.tmpl (+5/-0)
templates/recovery.conf.tmpl (+1/-1)
test.py (+91/-21)
testing/jujufixture.py (+17/-5)
tests/00_setup.test (+12/-0)
tests/01_lint.test (+3/-0)
tests/02_unit_test.test (+1/-1)
tests/91_integration_test_91.test (+3/-0)
tests/92_integration_test_92.test (+3/-0)
tests/93_integration_test_93.test (+3/-0)
To merge this branch: bzr merge lp:~stub/charms/precise/postgresql/pg93
Reviewer Review Type Date Requested Status
Marco Ceppi Approve
Review via email: mp+202841@code.launchpad.net

Description of the change

Run tests for all supported versions of PostgreSQL.

This involved several drive-bys, including:

- Allowing specifying a non-standard port to actually work.
- Realizing that the mechanism for specifying external sources was broken,
  and replacing it with the charm-helpers tool. I was using a space separated
  list, but unless you are using a PPA your source probably contains a space.
- Realizing that the current mechanism for charm-helpers to retrieve keys
  is insecure, so implement a shortcut to add the official PG backports
  securely that also makes it easier to use.
- Using null instead of empty string to indicate default/magic, probably
  for personal and unjustifiable aethetic reasons. Empty string still works.
- Config validation, to catch dangerously bad config changes before code
  that doesn't expect them destroys data. We cannot support version
  downgrades, and we don't yet support version upgrades.

To post a comment you must log in.
148. By Stuart Bishop on 2014-01-27

9.2 tests need PGDG archive or packages might not be found

149. By Stuart Bishop on 2014-01-27

Replication fixes

150. By Stuart Bishop on 2014-01-29

Fix admin test to work on non-local providers

151. By Stuart Bishop on 2014-01-29

charm-helpers for lsb_release

152. By Stuart Bishop on 2014-01-29

distro_release -> distro_codename

153. By Stuart Bishop on 2014-01-29

Merged tests into pg93.

154. By Stuart Bishop on 2014-01-30

Refactoring startup, shutdown & reload to better avoid config failures and race conditions like lp:1273636, and some fixes

155. By Stuart Bishop on 2014-01-30

Fixes

156. By Stuart Bishop on 2014-01-31

Fix PG 9.2 & 9.3 replication

157. By Stuart Bishop on 2014-01-31

Improve config validation

158. By Stuart Bishop on 2014-01-31

Make basic replication test more robust

159. By Stuart Bishop on 2014-01-31

Bump timeout to decrease flakeyness due to LP:1200267

160. By Stuart Bishop on 2014-02-04

Resolve conflicts

161. By Stuart Bishop on 2014-02-05

Merged tests into pg93.

162. By Stuart Bishop on 2014-02-05

Merged tests into pg93.

Revision history for this message
Marco Ceppi (marcoceppi) wrote :

Fantastic work, thank you for submitting this.

make test fails however with the following:

Lint check (flake8)
directory hooks
checking hooks/helpers.py
checking hooks/hooks.py
hooks/hooks.py:2109:5: E125 continuation line does not distinguish itself from next logical line
checking hooks/test_hooks.py
directory testing
checking testing/__init__.py
checking testing/jujufixture.py
directory tests
checking test.py
make: *** [test] Error 1

Seems simple enough, otherwise LGTM

163. By Stuart Bishop on 2014-02-07

Better fit with juju test

164. By Stuart Bishop on 2014-02-07

delint

Revision history for this message
Stuart Bishop (stub) wrote :

Thanks. I've delinted, and will land it once the dependent MP https://code.launchpad.net/~stub/charms/precise/postgresql/cleanups/+merge/203701 is ready.

Revision history for this message
Marco Ceppi (marcoceppi) wrote :

LGTM +1

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2012-10-30 16:41:12 +0000
3+++ .bzrignore 2014-02-07 12:21:04 +0000
4@@ -0,0 +1,3 @@
5+_trial_temp
6+hooks/_trial_temp
7+hooks/local_state.pickle
8
9=== modified file 'Makefile'
10--- Makefile 2014-01-09 09:08:11 +0000
11+++ Makefile 2014-02-07 12:21:04 +0000
12@@ -1,13 +1,30 @@
13 CHARM_DIR := $(shell pwd)
14+TEST_TIMEOUT := 900
15
16 test: lint unit_test integration_test
17
18 unit_test:
19+ @echo "Unit tests of hooks"
20 cd hooks && trial test_hooks.py
21
22 integration_test:
23- echo "Integration tests using Juju deployed units"
24- TEST_TIMEOUT=900 ./test.py -v
25+ @echo "PostgreSQL integration tests, all versions"
26+ trial test
27+
28+integration_test_91:
29+ @echo "PostgreSQL 9.1 integration tests"
30+ trial test.PG91Tests
31+
32+integration_test_92:
33+ @echo "PostgreSQL 9.2 integration tests"
34+ trial test.PG92Tests
35+
36+integration_test_93:
37+ @echo "PostgreSQL 9.3 integration tests"
38+ trial test.PG93Tests
39
40 lint:
41- @flake8 --exclude hooks/charmhelpers hooks # requires python-flakes8
42+ @echo "Lint check (flake8)"
43+ @flake8 -v \
44+ --exclude hooks/charmhelpers,hooks/_trial_temp \
45+ hooks testing tests test.py
46
47=== modified file 'README.md'
48--- README.md 2014-02-07 12:21:03 +0000
49+++ README.md 2014-02-07 12:21:04 +0000
50@@ -62,9 +62,12 @@
51 To setup a client using a PostgreSQL database, in this case a vanilla Django
52 installation listening on port 8080::
53
54- juju deploy postgresql juju deploy python-django juju deploy gunicorn juju
55-add-relation python-django postgresql:db juju add-relation python-django
56-gunicorn juju expose python-django
57+ juju deploy postgresql
58+ juju deploy python-django
59+ juju deploy gunicorn
60+ juju add-relation python-django postgresql:db
61+ juju add-relation python-django gunicorn
62+ juju expose python-django
63
64
65 ## Known Limitations and Issues
66
67=== modified file 'config.yaml'
68--- config.yaml 2014-01-22 07:09:38 +0000
69+++ config.yaml 2014-02-07 12:21:04 +0000
70@@ -35,17 +35,20 @@
71 default: "reload"
72 type: string
73 description: |
74- The command to run whenever config has changed. Accepted values are
75- "reload" or "restart" - any other value will mean neither is executed
76- after a config change (which may be desired, if you're running a
77- production server and would rather handle these out of band). Note that
78- postgresql will still need to be reloaded whenever authentication and
79- access details are updated, so disabling either doesn't mean PostgreSQL
80- will never be reloaded.
81+ The command to run whenever config has changed. Accepted values
82+ are "reload" or "restart" - any other value will mean neither is
83+ executed after a config change (which may be desired, if you're
84+ running a production server and would rather handle these out of
85+ band). Note that postgresql will still need to be reloaded
86+ whenever authentication and access details are updated, so
87+ disabling either doesn't mean PostgreSQL will never be reloaded.
88 version:
89- default: ""
90+ default: null
91 type: string
92- description: Version of PostgreSQL that we want to install
93+ description: |
94+ Version of PostgreSQL that we want to install. Supported versions
95+ are "9.1", "9.2", "9.3". The default version for the deployed Ubuntu
96+ release is used when the version is not specified.
97 cluster_name:
98 default: "main"
99 type: string
100@@ -55,9 +58,9 @@
101 type: string
102 description: IP to listen on
103 listen_port:
104- default: 5432
105+ default: null
106 type: int
107- description: Port to listen on
108+ description: Port to listen on. Default is automatically assigned.
109 max_connections:
110 default: 100
111 type: int
112@@ -120,7 +123,9 @@
113 autovacuum:
114 default: True
115 type: boolean
116- description: Autovacuum should almost always be running.
117+ description: |
118+ Autovacuum should almost always be running. If you want to turn this
119+ off, you are probably following out of date documentation.
120 log_autovacuum_min_duration:
121 default: -1
122 type: int
123@@ -329,13 +334,31 @@
124 juju-postgresql-0
125 If you're running multiple environments with the same services in them
126 this allows you to differentiate between them.
127+ pgdg:
128+ description: |
129+ Enable the PostgreSQL Global Development Group APT repository
130+ (https://wiki.postgresql.org/wiki/Apt). This package source provides
131+ official PostgreSQL packages for Ubuntu LTS releases beyond those
132+ provided by the main Ubuntu archive.
133+ type: boolean
134+ default: false
135+ install_sources:
136+ description: |
137+ List of extra package sources, per charm-helpers standard.
138+ YAML format.
139+ type: string
140+ default: ""
141+ install_keys:
142+ description: |
143+ List of signing keys for install_sources package sources, per
144+ charmhelpers standard. YAML format.
145+ type: string
146+ default: ""
147 extra_archives:
148 default: ""
149 type: string
150 description: |
151- Extra archives to add, space separated. Supports ppa:, http:, cloud:
152- URIs, as well as other schemes and keywords supported by
153- charmhelpers.fetch.add_source() such as "proposed".
154+ DEPRECATED & IGNORED. Use install_sources and install_keys.
155 advisory_lock_restart_key:
156 default: 765
157 type: int
158
159=== modified file 'hooks/charmhelpers/fetch/__init__.py'
160--- hooks/charmhelpers/fetch/__init__.py 2013-09-24 11:09:35 +0000
161+++ hooks/charmhelpers/fetch/__init__.py 2014-02-07 12:21:04 +0000
162@@ -117,8 +117,8 @@
163
164 Note that 'null' (a.k.a. None) should not be quoted.
165 """
166- sources = safe_load(config(sources_var))
167- keys = safe_load(config(keys_var))
168+ sources = safe_load(config(sources_var)) or []
169+ keys = safe_load(config(keys_var)) or []
170 if isinstance(sources, basestring) and isinstance(keys, basestring):
171 add_source(sources, keys)
172 else:
173
174=== modified file 'hooks/hooks.py'
175--- hooks/hooks.py 2014-02-07 12:21:03 +0000
176+++ hooks/hooks.py 2014-02-07 12:21:04 +0000
177@@ -60,24 +60,22 @@
178 package candidate, saving it in local_state for later.
179 '''
180 config_data = hookenv.config()
181- if config_data['version']:
182+ if 'pg_version' in local_state:
183+ version = local_state['pg_version']
184+ elif 'version' in config_data:
185 version = config_data['version']
186- elif 'pg_version' in local_state:
187- version = local_state['pg_version']
188 else:
189 log("map version from distro release ...")
190- distro_release = run("lsb_release -sc")
191- distro_release = distro_release.rstrip()
192 version_map = {'precise': '9.1',
193 'trusty': '9.3'}
194- version = version_map.get(distro_release)
195+ version = version_map.get(distro_codename())
196 if not version:
197- log("No PG version map for distro_release={}, "
198- "you'll need to explicitly set it".format(distro_release),
199+ log("No PG version map for distro_codename={}, "
200+ "you'll need to explicitly set it".format(distro_codename()),
201 CRITICAL)
202 sys.exit(1)
203- log("version={} from distro_release='{}'".format(
204- version, distro_release))
205+ log("version={} from distro_codename='{}'".format(
206+ version, distro_codename()))
207 # save it for later
208 local_state.setdefault('pg_version', version)
209 local_state.save()
210@@ -86,6 +84,11 @@
211 return version
212
213
214+def distro_codename():
215+ """Return the distro release code name, eg. 'precise' or 'trusty'."""
216+ return host.lsb_release()['DISTRIB_CODENAME']
217+
218+
219 class State(dict):
220 """Encapsulate state common to the unit for republishing to relations."""
221 def __init__(self, state_file):
222@@ -128,6 +131,7 @@
223 replication_state = dict(client_state)
224
225 add(replication_state, 'replication_password')
226+ add(replication_state, 'port')
227 add(replication_state, 'wal_received_offset')
228 add(replication_state, 'following')
229 add(replication_state, 'client_relations')
230@@ -242,73 +246,101 @@
231 startup_file, contents, 'postgres', 'postgres', perms=0o644)
232
233
234-def run(command, exit_on_error=True):
235+def run(command, exit_on_error=True, quiet=False):
236 '''Run a command and return the output.'''
237- try:
238- log(command, DEBUG)
239- return subprocess.check_output(
240- command, stderr=subprocess.STDOUT, shell=True)
241- except subprocess.CalledProcessError, e:
242- log("status=%d, output=%s" % (e.returncode, e.output), ERROR)
243- if exit_on_error:
244- sys.exit(e.returncode)
245- else:
246- raise
247+ log("Running {!r}".format(command), DEBUG)
248+ p = subprocess.Popen(
249+ command, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
250+ shell=isinstance(command, basestring))
251+ p.stdin.close()
252+ lines = []
253+ for line in p.stdout:
254+ if line:
255+ # LP:1274460 & LP:1259490 mean juju-log is no where near as
256+ # useful as we would like, so just shove a copy of the
257+ # output to stdout for logging.
258+ # log("> {}".format(line), DEBUG)
259+ if not quiet:
260+ print line
261+ lines.append(line)
262+ elif p.poll() is not None:
263+ break
264+
265+ p.wait()
266+
267+ if p.returncode == 0:
268+ return '\n'.join(lines)
269+
270+ if p.returncode != 0 and exit_on_error:
271+ log("ERROR: {}".format(p.returncode), ERROR)
272+ sys.exit(p.returncode)
273+
274+ raise subprocess.CalledProcessError(
275+ p.returncode, command, '\n'.join(lines))
276
277
278 def postgresql_is_running():
279 '''Return true if PostgreSQL is running.'''
280- # init script always return true (9.1), add extra check to make it useful
281- status, output = commands.getstatusoutput("invoke-rc.d postgresql status")
282- if status != 0:
283- return False
284- # e.g. output: "Running clusters: 9.1/main"
285- vc = "%s/%s" % (pg_version(), hookenv.config("cluster_name"))
286- return vc in output.decode('utf8').split()
287+ for version, name, _, status in lsclusters(slice(4)):
288+ if (version, name) == (pg_version(), hookenv.config('cluster_name')):
289+ if 'online' in status.split(','):
290+ log('PostgreSQL is running', DEBUG)
291+ return True
292+ else:
293+ log('PostgreSQL is not running', DEBUG)
294+ return False
295+ assert False, 'Cluster {} {} not found'.format(
296+ pg_version(), hookenv.config('cluster_name'))
297
298
299 def postgresql_stop():
300 '''Shutdown PostgreSQL.'''
301- success = host.service_stop('postgresql')
302- return not (success and postgresql_is_running())
303+ if postgresql_is_running():
304+ run([
305+ 'pg_ctlcluster', '--force',
306+ pg_version(), hookenv.config('cluster_name'), 'stop'])
307+ log('PostgreSQL shut down')
308
309
310 def postgresql_start():
311 '''Start PostgreSQL if it is not already running.'''
312- success = host.service_start('postgresql')
313- return success and postgresql_is_running()
314+ if not postgresql_is_running():
315+ run([
316+ 'pg_ctlcluster', pg_version(),
317+ hookenv.config('cluster_name'), 'start'])
318+ log('PostgreSQL started')
319
320
321 def postgresql_restart():
322 '''Restart PostgreSQL, or start it if it is not already running.'''
323 if postgresql_is_running():
324 with restart_lock(hookenv.local_unit(), True):
325- # 'service postgresql restart' fails; it only does a reload.
326- # success = host.service_restart('postgresql')
327- try:
328- run('pg_ctlcluster -force {} {} '
329- 'restart'.format(pg_version(),
330- hookenv.config('cluster_name')))
331- success = True
332- except subprocess.CalledProcessError:
333- success = False
334+ run([
335+ 'pg_ctlcluster', '--force',
336+ pg_version(), hookenv.config('cluster_name'), 'restart'])
337+ log('PostgreSQL restarted')
338 else:
339- success = host.service_start('postgresql')
340+ postgresql_start()
341+
342+ assert postgresql_is_running()
343
344 # Store a copy of our known live configuration so
345 # postgresql_reload_or_restart() can make good choices.
346- if success and 'saved_config' in local_state:
347+ if 'saved_config' in local_state:
348 local_state['live_config'] = local_state['saved_config']
349 local_state.save()
350
351- return success and postgresql_is_running()
352-
353
354 def postgresql_reload():
355 '''Make PostgreSQL reload its configuration.'''
356 # reload returns a reliable exit status
357- status, output = commands.getstatusoutput("invoke-rc.d postgresql reload")
358- return (status == 0)
359+ if postgresql_is_running():
360+ # I'm using the PostgreSQL function to avoid as much indirection
361+ # as possible.
362+ success = run_select_as_postgres('SELECT pg_reload_conf()')[1][0][0]
363+ assert success, 'Failed to reload PostgreSQL configuration'
364+ log('PostgreSQL configuration reloaded')
365+ return postgresql_start()
366
367
368 def requires_restart():
369@@ -340,38 +372,38 @@
370 # A setting has changed that requires PostgreSQL to be
371 # restarted before it will take effect.
372 restart = True
373+ log('{} changed from {} to {}. Restart required.'.format(
374+ name, live_value, new_value), DEBUG)
375 return restart
376
377
378 def postgresql_reload_or_restart():
379 """Reload PostgreSQL configuration, restarting if necessary."""
380 if requires_restart():
381- log("Configuration change requires PostgreSQL restart. Restarting.",
382- WARNING)
383- success = postgresql_restart()
384- if not success or requires_restart():
385- log("Configuration changes failed to apply", WARNING)
386- success = False
387+ log("Configuration change requires PostgreSQL restart", WARNING)
388+ postgresql_restart()
389+ assert not requires_restart(), "Configuration changes failed to apply"
390 else:
391- success = host.service_reload('postgresql')
392-
393- if success:
394- local_state['saved_config'] = local_state['live_config']
395- local_state.save()
396-
397- return success
398-
399-
400-def get_service_port(config_file):
401+ postgresql_reload()
402+
403+ local_state['saved_config'] = local_state['live_config']
404+ local_state.save()
405+
406+
407+def get_service_port():
408 '''Return the port PostgreSQL is listening on.'''
409- if not os.path.exists(config_file):
410- return None
411- postgresql_config = open(config_file, 'r').read()
412- port = re.search("port.*=(.*)", postgresql_config).group(1).strip()
413- try:
414- return int(port)
415- except (ValueError, TypeError):
416- return None
417+ for version, name, port in lsclusters(slice(3)):
418+ if (version, name) == (pg_version(), hookenv.config('cluster_name')):
419+ return int(port)
420+
421+ assert False, 'No port found for {!r} {!r}'.format(
422+ pg_version(), hookenv.config['cluster_name'])
423+
424+
425+def lsclusters(s=slice(0, -1)):
426+ for line in run('pg_lsclusters', quiet=True).splitlines()[1:]:
427+ if line:
428+ yield line.split()[s]
429
430
431 def _get_system_ram():
432@@ -388,6 +420,8 @@
433 def create_postgresql_config(config_file):
434 '''Create the postgresql.conf file'''
435 config_data = hookenv.config()
436+ if not config_data.get('listen_port', None):
437+ config_data['listen_port'] = get_service_port()
438 if config_data["performance_tuning"] == "auto":
439 # Taken from:
440 # http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server
441@@ -439,7 +473,7 @@
442 # Return it as pg_config
443 charm_dir = hookenv.charm_dir()
444 template_file = "{}/templates/postgresql.conf.tmpl".format(charm_dir)
445- if not config_data['version']:
446+ if not config_data.get('version', None):
447 config_data['version'] = pg_version()
448 pg_config = Template(
449 open(template_file).read()).render(config_data)
450@@ -615,7 +649,7 @@
451 host.write_file(output_file, crontab_template, perms=0600)
452
453
454-def create_recovery_conf(master_host, restart_on_change=False):
455+def create_recovery_conf(master_host, master_port, restart_on_change=False):
456 version = pg_version()
457 cluster_name = hookenv.config('cluster_name')
458 postgresql_cluster_dir = os.path.join(
459@@ -632,6 +666,7 @@
460 template_file = "{}/templates/recovery.conf.tmpl".format(charm_dir)
461 recovery_conf = Template(open(template_file).read()).render({
462 'host': master_host,
463+ 'port': master_port,
464 'password': local_state['replication_password'],
465 'streaming_replication': streaming_replication})
466 log(recovery_conf, DEBUG)
467@@ -657,17 +692,16 @@
468 return(None)
469
470
471-#------------------------------------------------------------------------------
472-# update_service_ports: Convenience function that evaluate the old and new
473-# service ports to decide which ports need to be
474-# opened and which to close
475-#------------------------------------------------------------------------------
476-def update_service_port(old_service_port=None, new_service_port=None):
477- if old_service_port is None or new_service_port is None:
478- return(None)
479- if new_service_port != old_service_port:
480- hookenv.close_port(old_service_port)
481- hookenv.open_port(new_service_port)
482+def update_service_port():
483+ old_port = local_state.get('listen_port', None)
484+ new_port = get_service_port()
485+ if old_port != new_port:
486+ if new_port:
487+ hookenv.open_port(new_port)
488+ if old_port:
489+ hookenv.close_port(old_port)
490+ local_state['listen_port'] = new_port
491+ local_state.save()
492
493
494 def create_ssl_cert(cluster_dir):
495@@ -702,12 +736,15 @@
496
497
498 def db_cursor(autocommit=False, db='postgres', user='postgres',
499- host=None, timeout=30):
500+ host=None, port=None, timeout=30):
501 import psycopg2
502+ if port is None:
503+ port = get_service_port()
504 if host:
505- conn_str = "dbname={} host={} user={}".format(db, host, user)
506+ conn_str = "dbname={} host={} port={} user={}".format(
507+ db, host, port, user)
508 else:
509- conn_str = "dbname={} user={}".format(db, user)
510+ conn_str = "dbname={} port={} user={}".format(db, port, user)
511 # There are often race conditions in opening database connections,
512 # such as a reload having just happened to change pg_hba.conf
513 # settings or a hot standby being restarted and needing to catch up
514@@ -750,6 +787,47 @@
515 return (cur.rowcount, results)
516
517
518+def validate_config():
519+ """
520+ Sanity check charm configuration, aborting the script if
521+ we have bogus config values or config changes the charm does not yet
522+ (or cannot) support.
523+ """
524+ valid = True
525+ config_data = hookenv.config()
526+
527+ version = config_data.get('version', None)
528+ if version:
529+ if version not in ('9.1', '9.2', '9.3'):
530+ valid = False
531+ log("Invalid or unsupported version {!r} requested".format(
532+ version), CRITICAL)
533+
534+ if config_data['cluster_name'] != 'main':
535+ valid = False
536+ log("Cluster names other than 'main' do not work per LP:1271835",
537+ CRITICAL)
538+
539+ if config_data['listen_ip'] != '*':
540+ valid = False
541+ log("listen_ip values other than '*' do not work per LP:1271837",
542+ CRITICAL)
543+
544+ unchangeable_config = [
545+ 'locale', 'encoding', 'version', 'cluster_name', 'pgdg']
546+
547+ for name in unchangeable_config:
548+ if (name in local_state
549+ and local_state[name] != config_data.get(name, None)):
550+ valid = False
551+ log("Cannot change {!r} setting after install.".format(name))
552+ local_state[name] = config_data.get(name, None)
553+ local_state.save()
554+
555+ if not valid:
556+ sys.exit(99)
557+
558+
559 #------------------------------------------------------------------------------
560 # Core logic for permanent storage changes:
561 # NOTE the only 2 "True" return points:
562@@ -869,6 +947,7 @@
563
564 @hooks.hook()
565 def config_changed(force_restart=False):
566+ validate_config()
567 config_data = hookenv.config()
568 update_repos_and_packages()
569
570@@ -908,19 +987,16 @@
571 postgresql_hba = os.path.join(postgresql_config_dir, "pg_hba.conf")
572 postgresql_ident = os.path.join(postgresql_config_dir, "pg_ident.conf")
573
574- current_service_port = get_service_port(postgresql_config)
575 create_postgresql_config(postgresql_config)
576+ create_postgresql_ident(postgresql_ident) # Do this before pg_hba.conf.
577 generate_postgresql_hba(postgresql_hba)
578- create_postgresql_ident(postgresql_ident)
579 create_ssl_cert(os.path.join(
580 postgresql_data_dir, pg_version(), config_data['cluster_name']))
581-
582- updated_service_port = config_data["listen_port"]
583- update_service_port(current_service_port, updated_service_port)
584+ update_service_port()
585 update_nrpe_checks()
586 if force_restart:
587- return postgresql_restart()
588- return postgresql_reload_or_restart()
589+ postgresql_restart()
590+ postgresql_reload_or_restart()
591
592
593 @hooks.hook()
594@@ -930,6 +1006,8 @@
595 if os.path.isfile(f) and os.access(f, os.X_OK):
596 subprocess.check_call(['sh', '-c', f])
597
598+ validate_config()
599+
600 config_data = hookenv.config()
601 update_repos_and_packages()
602 if not 'state' in local_state:
603@@ -938,14 +1016,35 @@
604 # any non-idempotent setup. We should probably fix this; it
605 # seems rather fragile.
606 local_state.setdefault('state', 'standalone')
607- local_state.publish()
608
609 # Drop the cluster created when the postgresql package was
610 # installed, and rebuild it with the requested locale and encoding.
611 version = pg_version()
612- run("pg_dropcluster --stop {} main".format(version))
613- run("pg_createcluster --locale='{}' --encoding='{}' {} main".format(
614- config_data['locale'], config_data['encoding'], version))
615+ for ver, name in lsclusters(slice(2)):
616+ if version == ver and name == 'main':
617+ run("pg_dropcluster --stop {} main".format(version))
618+ listen_port = config_data.get('listen_port', None)
619+ if listen_port:
620+ port_opt = "--port={}".format(config_data['listen_port'])
621+ else:
622+ port_opt = ''
623+ with switch_cwd('/tmp'):
624+ create_cmd = [
625+ "pg_createcluster",
626+ "--locale", config_data['locale'],
627+ "-e", config_data['encoding']]
628+ if listen_port:
629+ create_cmd.extend(["-p", str(config_data['listen_port'])])
630+ create_cmd.append(pg_version())
631+ create_cmd.append(config_data['cluster_name'])
632+ run(create_cmd)
633+ assert (
634+ not port_opt
635+ or get_service_port() == config_data['listen_port']), (
636+ 'allocated port {!r} != {!r}'.format(
637+ get_service_port(), config_data['listen_port']))
638+ local_state['port'] = get_service_port()
639+ local_state.publish()
640
641 postgresql_backups_dir = (
642 config_data['backup_dir'].strip() or
643@@ -972,7 +1071,7 @@
644 '{}/pg_backup_job'.format(postgresql_scripts_dir),
645 backup_job, perms=0755)
646 install_postgresql_crontab(postgresql_crontab)
647- hookenv.open_port(5432)
648+ hookenv.open_port(get_service_port())
649
650 # Ensure at least minimal access granted for hooks to run.
651 # Reload because we are using the default cluster setup and started
652@@ -990,16 +1089,14 @@
653
654 @hooks.hook()
655 def start():
656- if not postgresql_reload_or_restart():
657- raise SystemExit(1)
658+ postgresql_reload_or_restart()
659
660
661 @hooks.hook()
662 def stop():
663 if postgresql_is_running():
664 with restart_lock(hookenv.local_unit(), True):
665- if not postgresql_stop():
666- raise SystemExit(1)
667+ postgresql_stop()
668
669
670 def quote_identifier(identifier):
671@@ -1262,7 +1359,7 @@
672 schema_password = create_user(schema_user)
673 ensure_database(user, schema_user, database)
674 host = hookenv.unit_private_ip()
675- port = hookenv.config('listen_port')
676+ port = get_service_port()
677 state = local_state['state'] # master, hot standby, standalone
678
679 # Publish connection details.
680@@ -1301,7 +1398,7 @@
681
682 password = create_user(user, admin=True)
683 host = hookenv.unit_private_ip()
684- port = hookenv.config('listen_port')
685+ port = get_service_port()
686 state = local_state['state'] # master, hot standby, standalone
687
688 # Publish connection details.
689@@ -1386,18 +1483,62 @@
690
691
692 def update_repos_and_packages():
693- extra_repos = hookenv.config('extra_archives')
694- extra_repos_added = local_state.setdefault('extra_repos_added', set())
695- if extra_repos:
696- repos_added = False
697- for repo in extra_repos.split():
698- if repo not in extra_repos_added:
699- fetch.add_source(repo)
700- extra_repos_added.add(repo)
701- repos_added = True
702- if repos_added:
703- fetch.apt_update(fatal=True)
704- local_state.save()
705+ need_upgrade = False
706+
707+ # Add the PGDG APT repository if it is enabled. Setting this boolean
708+ # is simpler than requiring the magic URL and key be added to
709+ # install_sources and install_keys. In addition, per Bug #1271148,
710+ # install_keys is likely a security hole for this sort of remote
711+ # archive. Instead, we keep a copy of the signing key in the charm
712+ # and can add it securely.
713+ pgdg_list = '/etc/apt/sources.list.d/pgdg_{}.list'.format(
714+ sanitize(hookenv.local_unit()))
715+ pgdg_key = 'ACCC4CF8'
716+
717+ if hookenv.config('pgdg'):
718+ if not os.path.exists(pgdg_list):
719+ # We need to upgrade, as if we have Ubuntu main packages
720+ # installed they may be incompatible with the PGDG ones.
721+ # This is unlikely to ever happen outside of the test suite,
722+ # and never if you don't reuse machines.
723+ need_upgrade = True
724+ run("apt-key add lib/{}.asc".format(pgdg_key))
725+ open(pgdg_list, 'w').write('deb {} {}-pgdg main'.format(
726+ 'http://apt.postgresql.org/pub/repos/apt/', distro_codename()))
727+ elif os.path.exists(pgdg_list):
728+ log(
729+ "PGDG apt source not requested, but already in place in this "
730+ "container", WARNING)
731+ # We can't just remove a source, as we may have packages
732+ # installed that conflict with ones from the other configured
733+ # sources. In particular, if we have postgresql-common installed
734+ # from the PGDG Apt source, PostgreSQL packages from Ubuntu main
735+ # will fail to install.
736+ # os.unlink(pgdg_list)
737+
738+ # Try to optimize our calls to fetch.configure_sources(), as it
739+ # cannot do this itself due to lack of state.
740+ if (need_upgrade
741+ or local_state.get('install_sources', None)
742+ != hookenv.config('install_sources')
743+ or local_state.get('install_keys', None)
744+ != hookenv.config('install_keys')):
745+ # Support the standard mechanism implemented by charm-helpers. Pulls
746+ # from the default 'install_sources' and 'install_keys' config
747+ # options. This also does 'apt-get update', pulling in the PGDG data
748+ # if we just configured it.
749+ fetch.configure_sources(True)
750+ local_state['install_sources'] = hookenv.config('install_sources')
751+ local_state['install_keys'] = hookenv.config('install_keys')
752+ local_state.save()
753+
754+ # Ensure that the desired database locale is possible.
755+ if hookenv.config('locale') != 'C':
756+ run(["locale-gen", "{}.{}".format(
757+ hookenv.config('locale'), hookenv.config('encoding'))])
758+
759+ if need_upgrade:
760+ run("apt-get -y upgrade")
761
762 version = pg_version()
763 # It might have been better for debversion and plpython to only get
764@@ -1405,11 +1546,14 @@
765 # but they predate this feature.
766 packages = ["python-psutil", # to obtain system RAM from python
767 "libc-bin", # for getconf
768- "postgresql-%s" % version,
769- "postgresql-contrib-%s" % version,
770- "postgresql-plpython-%s" % version,
771- "postgresql-%s-debversion" % version,
772+ "postgresql-{}".format(version),
773+ "postgresql-contrib-{}".format(version),
774+ "postgresql-plpython-{}".format(version),
775 "python-jinja2", "syslinux", "python-psycopg2"]
776+ # PGDG currently doesn't have debversion for 9.3. Put this back when
777+ # it does.
778+ if not (hookenv.config('pgdg') and version == '9.3'):
779+ "postgresql-{}-debversion".format(version)
780 packages.extend((hookenv.config('extra-packages') or '').split())
781 packages = fetch.filter_installed_packages(packages)
782 fetch.apt_install(packages, fatal=True)
783@@ -1472,7 +1616,8 @@
784 '''Connect the database as a streaming replica of the master.'''
785 master_relation = hookenv.relation_get(unit=master)
786 create_recovery_conf(
787- master_relation['private-address'], restart_on_change=True)
788+ master_relation['private-address'],
789+ master_relation['port'], restart_on_change=True)
790
791
792 def elected_master():
793@@ -1618,6 +1763,7 @@
794 local_state['replication_password'] = replication_password
795 local_state['client_relations'] = ' '.join(
796 hookenv.relation_ids('db') + hookenv.relation_ids('db-admin'))
797+ local_state.publish()
798
799 else:
800 log("I am master and remain master")
801@@ -1635,8 +1781,10 @@
802 master))
803
804 master_ip = hookenv.relation_get('private-address', master)
805+ master_port = hookenv.relation_get('port', master)
806+ assert master_port is not None, 'No master port set'
807
808- clone_database(master, master_ip)
809+ clone_database(master, master_ip, master_port)
810
811 local_state['state'] = 'hot standby'
812 local_state['following'] = master
813@@ -1711,7 +1859,7 @@
814
815 # Override unit specific connection details
816 connection_settings['host'] = hookenv.unit_private_ip()
817- connection_settings['port'] = hookenv.config('listen_port')
818+ connection_settings['port'] = get_service_port()
819 connection_settings['state'] = local_state['state']
820
821 # Block until users and database has replicated, so we know the
822@@ -1833,9 +1981,10 @@
823 cur = db_cursor(autocommit=True)
824 else:
825 host = hookenv.relation_get('private-address', unit)
826+ port = hookenv.relation_get('port', unit)
827 cur = db_cursor(
828- autocommit=True, db='postgres',
829- user='juju_replication', host=host)
830+ autocommit=True, db='postgres', user='juju_replication',
831+ host=host, port=port)
832 cur.execute(q)
833 break
834 except psycopg2.Error:
835@@ -1853,7 +2002,7 @@
836 pass
837
838
839-def clone_database(master_unit, master_host):
840+def clone_database(master_unit, master_host, master_port):
841 with restart_lock(master_unit, False):
842 postgresql_stop()
843 log("Cloning master {}".format(master_unit))
844@@ -1868,7 +2017,8 @@
845 'sudo', '-E', # -E needed to locate pgpass file.
846 '-u', 'postgres', 'pg_basebackup', '-D', postgresql_cluster_dir,
847 '--xlog', '--checkpoint=fast', '--no-password',
848- '-h', master_host, '-p', '5432', '--username=juju_replication']
849+ '-h', master_host, '-p', master_port,
850+ '--username=juju_replication']
851 log(' '.join(cmd), DEBUG)
852
853 if os.path.isdir(postgresql_cluster_dir):
854@@ -1883,7 +2033,7 @@
855 log(output, DEBUG)
856 # Debian by default expects SSL certificates in the datadir.
857 create_ssl_cert(postgresql_cluster_dir)
858- create_recovery_conf(master_host)
859+ create_recovery_conf(master_host, master_port)
860 except subprocess.CalledProcessError as x:
861 # We failed, and this cluster is broken. Rebuild a
862 # working cluster so start/stop etc. works and we
863@@ -1955,9 +2105,10 @@
864 return int(logid, 16) * 16 * 1024 * 1024 * 255 + int(offset, 16)
865
866
867-def wait_for_db(timeout=120, db='postgres', user='postgres', host=None):
868+def wait_for_db(
869+ timeout=120, db='postgres', user='postgres', host=None, port=None):
870 '''Wait until the db is fully up.'''
871- db_cursor(db=db, user=user, host=host, timeout=timeout)
872+ db_cursor(db=db, user=user, host=host, port=port, timeout=timeout)
873
874
875 def unit_sorted(units):
876@@ -2016,7 +2167,7 @@
877 nrpe_check_config.write("# check pgsql\n")
878 nrpe_check_config.write(
879 "command[check_pgsql]=/usr/lib/nagios/plugins/check_pgsql -P {}"
880- .format(config_data['listen_port']))
881+ .format(get_service_port()))
882 # pgsql backups
883 nrpe_check_file = '/etc/nagios/nrpe.d/check_pgsql_backups.cfg'
884 backup_log = "{}/backups.log".format(postgresql_logs_dir)
885
886=== added directory 'lib'
887=== added file 'lib/ACCC4CF8.asc'
888--- lib/ACCC4CF8.asc 1970-01-01 00:00:00 +0000
889+++ lib/ACCC4CF8.asc 2014-02-07 12:21:04 +0000
890@@ -0,0 +1,45 @@
891+This is the signing key for the PostgreSQL Global Development Group
892+APT repository. See https://wiki.postgresql.org/wiki/Apt for details.
893+
894+-----BEGIN PGP PUBLIC KEY BLOCK-----
895+Version: GnuPG v1.4.12 (GNU/Linux)
896+
897+mQINBE6XR8IBEACVdDKT2HEH1IyHzXkb4nIWAY7echjRxo7MTcj4vbXAyBKOfjja
898+UrBEJWHN6fjKJXOYWXHLIYg0hOGeW9qcSiaa1/rYIbOzjfGfhE4x0Y+NJHS1db0V
899+G6GUj3qXaeyqIJGS2z7m0Thy4Lgr/LpZlZ78Nf1fliSzBlMo1sV7PpP/7zUO+aA4
900+bKa8Rio3weMXQOZgclzgeSdqtwKnyKTQdXY5MkH1QXyFIk1nTfWwyqpJjHlgtwMi
901+c2cxjqG5nnV9rIYlTTjYG6RBglq0SmzF/raBnF4Lwjxq4qRqvRllBXdFu5+2pMfC
902+IZ10HPRdqDCTN60DUix+BTzBUT30NzaLhZbOMT5RvQtvTVgWpeIn20i2NrPWNCUh
903+hj490dKDLpK/v+A5/i8zPvN4c6MkDHi1FZfaoz3863dylUBR3Ip26oM0hHXf4/2U
904+A/oA4pCl2W0hc4aNtozjKHkVjRx5Q8/hVYu+39csFWxo6YSB/KgIEw+0W8DiTII3
905+RQj/OlD68ZDmGLyQPiJvaEtY9fDrcSpI0Esm0i4sjkNbuuh0Cvwwwqo5EF1zfkVj
906+Tqz2REYQGMJGc5LUbIpk5sMHo1HWV038TWxlDRwtOdzw08zQA6BeWe9FOokRPeR2
907+AqhyaJJwOZJodKZ76S+LDwFkTLzEKnYPCzkoRwLrEdNt1M7wQBThnC5z6wARAQAB
908+tBxQb3N0Z3JlU1FMIERlYmlhbiBSZXBvc2l0b3J5iQI9BBMBCAAnAhsDBQsJCAcD
909+BRUKCQgLBRYCAwEAAh4BAheABQJRKm2VBQkINsBBAAoJEH/MfUaszEz4RTEP/1sQ
910+HyjHaUiAPaCAv8jw/3SaWP/g8qLjpY6ROjLnDMvwKwRAoxUwcIv4/TWDOMpwJN+C
911+JIbjXsXNYvf9OX+UTOvq4iwi4ADrAAw2xw+Jomc6EsYla+hkN2FzGzhpXfZFfUsu
912+phjY3FKL+4hXH+R8ucNwIz3yrkfc17MMn8yFNWFzm4omU9/JeeaafwUoLxlULL2z
913+Y7H3+QmxCl0u6t8VvlszdEFhemLHzVYRY0Ro/ISrR78CnANNsMIy3i11U5uvdeWV
914+CoWV1BXNLzOD4+BIDbMB/Do8PQCWiliSGZi8lvmj/sKbumMFQonMQWOfQswTtqTy
915+Q3yhUM1LaxK5PYq13rggi3rA8oq8SYb/KNCQL5pzACji4TRVK0kNpvtxJxe84X8+
916+9IB1vhBvF/Ji/xDd/3VDNPY+k1a47cON0S8Qc8DA3mq4hRfcgvuWy7ZxoMY7AfSJ
917+Ohleb9+PzRBBn9agYgMxZg1RUWZazQ5KuoJqbxpwOYVFja/stItNS4xsmi0lh2I4
918+MNlBEDqnFLUxSvTDc22c3uJlWhzBM/f2jH19uUeqm4jaggob3iJvJmK+Q7Ns3Wcf
919+huWwCnc1+58diFAMRUCRBPeFS0qd56QGk1r97B6+3UfLUslCfaaA8IMOFvQSHJwD
920+O87xWGyxeRTYIIP9up4xwgje9LB7fMxsSkCDTHOkiEYEEBEIAAYFAk6XSO4ACgkQ
921+xa93SlhRC1qmjwCg9U7U+XN7Gc/dhY/eymJqmzUGT/gAn0guvoX75Y+BsZlI6dWn
922+qaFU6N8HiQIcBBABCAAGBQJOl0kLAAoJEExaa6sS0qeuBfEP/3AnLrcKx+dFKERX
923+o4NBCGWr+i1CnowupKS3rm2xLbmiB969szG5TxnOIvnjECqPz6skK3HkV3jTZaju
924+v3sR6M2ItpnrncWuiLnYcCSDp9TEMpCWzTEgtrBlKdVuTNTeRGILeIcvqoZX5w+u
925+i0eBvvbeRbHEyUsvOEnYjrqoAjqUJj5FUZtR1+V9fnZp8zDgpOSxx0LomnFdKnhj
926+uyXAQlRCA6/roVNR9ruRjxTR5ubteZ9ubTsVYr2/eMYOjQ46LhAgR+3Alblu/WHB
927+MR/9F9//RuOa43R5Sjx9TiFCYol+Ozk8XRt3QGweEH51YkSYY3oRbHBb2Fkql6N6
928+YFqlLBL7/aiWnNmRDEs/cdpo9HpFsbjOv4RlsSXQfvvfOayHpT5nO1UQFzoyMVpJ
929+615zwmQDJT5Qy7uvr2eQYRV9AXt8t/H+xjQsRZCc5YVmeAo91qIzI/tA2gtXik49
930+6yeziZbfUvcZzuzjjxFExss4DSAwMgorvBeIbiz2k2qXukbqcTjB2XqAlZasd6Ll
931+nLXpQdqDV3McYkP/MvttWh3w+J/woiBcA7yEI5e3YJk97uS6+ssbqLEd0CcdT+qz
932++Waw0z/ZIU99Lfh2Qm77OT6vr//Zulw5ovjZVO2boRIcve7S97gQ4KC+G/+QaRS+
933+VPZ67j5UMxqtT/Y4+NHcQGgwF/1i
934+=Iugu
935+-----END PGP PUBLIC KEY BLOCK-----
936
937=== modified file 'templates/pg_hba.conf.tmpl'
938--- templates/pg_hba.conf.tmpl 2013-11-19 12:51:43 +0000
939+++ templates/pg_hba.conf.tmpl 2014-02-07 12:21:04 +0000
940@@ -3,7 +3,7 @@
941 #------------------------------------------------------------------------------
942
943 # Database administrative login by UNIX sockets
944-local all postgres ident map=superusers
945+local all root,postgres ident map=superusers
946 local replication root,postgres ident map=superusers
947 local all nagios md5
948
949
950=== modified file 'templates/pg_ident.conf.tmpl'
951--- templates/pg_ident.conf.tmpl 2012-10-05 13:21:43 +0000
952+++ templates/pg_ident.conf.tmpl 2014-02-07 12:21:04 +0000
953@@ -1,2 +1,3 @@
954+# This file is managed by juju
955 superusers root postgres
956 superusers postgres postgres
957
958=== modified file 'templates/postgresql.conf.tmpl'
959--- templates/postgresql.conf.tmpl 2014-01-17 03:18:52 +0000
960+++ templates/postgresql.conf.tmpl 2014-02-07 12:21:04 +0000
961@@ -23,6 +23,11 @@
962 unix_socket_directory = '/var/run/postgresql'
963 {% endif -%}
964
965+{% if version >= "9.2" -%}
966+ssl_cert_file = '/etc/ssl/certs/ssl-cert-snakeoil.pem'
967+ssl_key_file = '/etc/ssl/private/ssl-cert-snakeoil.key'
968+{% endif -%}
969+
970 {% if listen_ip != "" -%}
971 listen_addresses = '{{listen_ip}}'
972 {% endif -%}
973
974=== modified file 'templates/recovery.conf.tmpl'
975--- templates/recovery.conf.tmpl 2013-11-28 08:43:35 +0000
976+++ templates/recovery.conf.tmpl 2014-02-07 12:21:04 +0000
977@@ -4,7 +4,7 @@
978 standby_mode = on
979 recovery_target_timeline = latest
980 {% if streaming_replication %}
981-primary_conninfo = 'host={{host}} user=juju_replication password={{password}} requirepeer=postgres'
982+primary_conninfo = 'host={{host}} port={{port}} user=juju_replication password={{password}} requirepeer=postgres'
983 {% endif %}
984 {% if restore_command %}
985 restore_command = '{{restore_command}}'
986
987=== modified file 'test.py'
988--- test.py 2013-12-23 10:38:13 +0000
989+++ test.py 2014-02-07 12:21:04 +0000
990@@ -25,13 +25,30 @@
991
992 SERIES = 'precise'
993 TEST_CHARM = 'local:postgresql'
994-PSQL_CHARM = 'local:postgresql-psql'
995-
996-
997-class PostgreSQLCharmTestCase(testtools.TestCase, fixtures.TestWithFixtures):
998+PSQL_CHARM = 'cs:postgresql-psql'
999+
1000+
1001+class PostgreSQLCharmBaseTestCase(object):
1002+
1003+ # Override these in subclasses to run these tests multiple times
1004+ # for different PostgreSQL versions.
1005+
1006+ # PostgreSQL version for tests. One of the subclasses leaves the
1007+ # VERSION as None to test automatic version selection.
1008+ VERSION = None
1009+
1010+ # Use the PGDG Apt archive or not. One of the subclasses sets this
1011+ # to False to test the Ubuntu main packages. The rest set this to
1012+ # True to pull packages from the PGDG (only one PostgreSQL version
1013+ # exists in main).
1014+ PGDG = None
1015
1016 def setUp(self):
1017- super(PostgreSQLCharmTestCase, self).setUp()
1018+ super(PostgreSQLCharmBaseTestCase, self).setUp()
1019+
1020+ # Generate a basic config for all PostgreSQL charm deploys.
1021+ # Tests may add or change options.
1022+ self.pg_config = dict(version=self.VERSION, pgdg=self.PGDG)
1023
1024 self.juju = self.useFixture(JujuFixture(
1025 reuse_machines=True,
1026@@ -99,7 +116,7 @@
1027
1028 def test_basic(self):
1029 '''Connect to a a single unit service via the db relationship.'''
1030- self.juju.deploy(TEST_CHARM, 'postgresql')
1031+ self.juju.deploy(TEST_CHARM, 'postgresql', config=self.pg_config)
1032 self.juju.deploy(PSQL_CHARM, 'psql')
1033 self.juju.do(['add-relation', 'postgresql:db', 'psql:db'])
1034 self.juju.wait_until_ready()
1035@@ -107,11 +124,27 @@
1036 result = self.sql('SELECT TRUE')
1037 self.assertEqual(result, [['t']])
1038
1039+ def test_streaming_replication(self):
1040+ self.juju.deploy(
1041+ TEST_CHARM, 'postgresql', num_units=2, config=self.pg_config)
1042+ self.juju.deploy(PSQL_CHARM, 'psql')
1043+ self.juju.do(['add-relation', 'postgresql:db', 'psql:db'])
1044+ self.juju.wait_until_ready()
1045+
1046+ # Confirm that the slave has successfully opened a streaming
1047+ # replication connection.
1048+ num_slaves = self.sql(
1049+ 'SELECT COUNT(*) FROM pg_stat_replication',
1050+ postgres_unit='master')[0][0]
1051+
1052+ self.assertEqual(num_slaves, '1', 'Slave not connected')
1053+
1054 def test_basic_admin(self):
1055 '''Connect to a single unit service via the db-admin relationship.'''
1056- self.juju.deploy(TEST_CHARM, 'postgresql')
1057+ self.juju.deploy(TEST_CHARM, 'postgresql', config=self.pg_config)
1058 self.juju.deploy(PSQL_CHARM, 'psql')
1059 self.juju.do(['add-relation', 'postgresql:db-admin', 'psql:db-admin'])
1060+ self.juju.do(['expose', 'postgresql'])
1061 self.juju.wait_until_ready()
1062
1063 result = self.sql('SELECT TRUE', dbname='postgres')
1064@@ -125,7 +158,8 @@
1065
1066 def test_failover(self):
1067 """Set up a multi-unit service and perform failovers."""
1068- self.juju.deploy(TEST_CHARM, 'postgresql', num_units=3)
1069+ self.juju.deploy(
1070+ TEST_CHARM, 'postgresql', num_units=3, config=self.pg_config)
1071 self.juju.deploy(PSQL_CHARM, 'psql')
1072 self.juju.do(['add-relation', 'postgresql:db', 'psql:db'])
1073 self.juju.wait_until_ready()
1074@@ -213,7 +247,8 @@
1075
1076 def test_failover_election(self):
1077 """Ensure master elected in a failover is the best choice"""
1078- self.juju.deploy(TEST_CHARM, 'postgresql', num_units=3)
1079+ self.juju.deploy(
1080+ TEST_CHARM, 'postgresql', num_units=3, config=self.pg_config)
1081 self.juju.deploy(PSQL_CHARM, 'psql')
1082 self.juju.do(['add-relation', 'postgresql:db-admin', 'psql:db-admin'])
1083 self.juju.wait_until_ready()
1084@@ -261,7 +296,15 @@
1085 self.assertIs(False, self.is_master(standby_unit_1, 'postgres'))
1086
1087 def test_admin_addresses(self):
1088- self.juju.deploy(TEST_CHARM, 'postgresql')
1089+
1090+ # This test also tests explicit port assignment. We need
1091+ # a different port for each PostgreSQL version we might be
1092+ # testing, because clusters from previous tests of different
1093+ # versions may be hanging around.
1094+ port = 7400 + int((self.VERSION or '66').replace('.', ''))
1095+ self.pg_config['listen_port'] = port
1096+
1097+ self.juju.deploy(TEST_CHARM, 'postgresql', config=self.pg_config)
1098 self.juju.deploy(PSQL_CHARM, 'psql')
1099 self.juju.do(['add-relation', 'postgresql:db-admin', 'psql:db-admin'])
1100 self.juju.wait_until_ready()
1101@@ -271,7 +314,7 @@
1102 unit_ip = self.juju.status['services']['postgresql']['units'][
1103 unit]['public-address']
1104 s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
1105- s.connect((unit_ip, 5432))
1106+ s.connect((unit_ip, port))
1107 my_ip = s.getsockname()[0]
1108 del s
1109
1110@@ -280,8 +323,9 @@
1111 "ALTER USER postgres ENCRYPTED PASSWORD 'foo'", dbname='postgres')
1112
1113 # Direct connection string to the unit's database.
1114- conn_str = 'dbname=postgres user=postgres password=foo host={}'.format(
1115- unit_ip)
1116+ conn_str = (
1117+ 'dbname=postgres user=postgres password=foo '
1118+ 'host={} port={}'.format(unit_ip, port))
1119
1120 # Direct database connections should fail at the moment.
1121 self.assertRaises(
1122@@ -297,7 +341,7 @@
1123 self.assertEquals(1, cur.fetchone()[0])
1124
1125 def test_explicit_database(self):
1126- self.juju.deploy(TEST_CHARM, 'postgresql')
1127+ self.juju.deploy(TEST_CHARM, 'postgresql', config=self.pg_config)
1128 self.juju.deploy(PSQL_CHARM, 'psql')
1129 self.juju.do(['set', 'psql', 'database=explicit'])
1130 self.juju.do(['add-relation', 'postgresql:db', 'psql:db'])
1131@@ -306,11 +350,12 @@
1132 result = self.sql('SELECT current_database()')
1133 self.assertEqual(result, [['explicit']])
1134
1135-
1136 def test_roles_granted(self):
1137- self.juju.deploy(TEST_CHARM, 'postgresql')
1138- self.juju.deploy(PSQL_CHARM, 'psql')
1139- self.juju.do(['set', 'psql', 'roles=role_a'])
1140+ # We use two units to confirm that there is no attempts to
1141+ # grant roles on the hot standby.
1142+ self.juju.deploy(
1143+ TEST_CHARM, 'postgresql', num_units=2, config=self.pg_config)
1144+ self.juju.deploy(PSQL_CHARM, 'psql', config={'roles': 'role_a'})
1145 self.juju.do(['add-relation', 'postgresql:db', 'psql:db'])
1146 self.juju.wait_until_ready()
1147
1148@@ -330,9 +375,11 @@
1149 self.assertEqual(result, [['t', 't']])
1150
1151 def test_roles_revoked(self):
1152- self.juju.deploy(TEST_CHARM, 'postgresql')
1153- self.juju.deploy(PSQL_CHARM, 'psql')
1154- self.juju.do(['set', 'psql', 'roles=role_a,role_b'])
1155+ # We use two units to confirm that there is no attempts to
1156+ # grant roles on the hot standby.
1157+ self.juju.deploy(
1158+ TEST_CHARM, 'postgresql', num_units=2, config=self.pg_config)
1159+ self.juju.deploy(PSQL_CHARM, 'psql', config={'roles': 'role_a,role_b'})
1160 self.juju.do(['add-relation', 'postgresql:db', 'psql:db'])
1161 self.juju.wait_until_ready()
1162
1163@@ -366,6 +413,29 @@
1164 self.assertEqual(result, [['f', 'f', 'f']])
1165
1166
1167+class PG91Tests(
1168+ PostgreSQLCharmBaseTestCase,
1169+ testtools.TestCase, fixtures.TestWithFixtures):
1170+ # Test automatic version selection under precise.
1171+ VERSION = None if SERIES == 'precise' else '9.1'
1172+ PGDG = False if SERIES == 'precise' else True
1173+
1174+
1175+class PG92Tests(
1176+ PostgreSQLCharmBaseTestCase,
1177+ testtools.TestCase, fixtures.TestWithFixtures):
1178+ VERSION = '9.2'
1179+ PGDG = True
1180+
1181+
1182+class PG93Tests(
1183+ PostgreSQLCharmBaseTestCase,
1184+ testtools.TestCase, fixtures.TestWithFixtures):
1185+ # Test automatic version selection under trusty.
1186+ VERSION = None if SERIES == 'trusty' else '9.3'
1187+ PGDG = False if SERIES == 'trusty' else True
1188+
1189+
1190 def unit_sorted(units):
1191 """Return a correctly sorted list of unit names."""
1192 return sorted(
1193
1194=== modified file 'testing/jujufixture.py'
1195--- testing/jujufixture.py 2013-12-23 10:38:13 +0000
1196+++ testing/jujufixture.py 2014-02-07 12:21:04 +0000
1197@@ -1,9 +1,11 @@
1198 import json
1199+import os.path
1200 import subprocess
1201 import time
1202
1203 import fixtures
1204 from testtools.content import text_content
1205+import yaml
1206
1207
1208 __all__ = ['JujuFixture', 'run']
1209@@ -42,7 +44,7 @@
1210 return json.loads(out)
1211 return None
1212
1213- def deploy(self, charm, name=None, num_units=1):
1214+ def deploy(self, charm, name=None, num_units=1, config=None):
1215 # The first time we deploy a local: charm in the test run, it
1216 # needs to deploy with --update to ensure we are testing the
1217 # desired revision of the charm. Subsequent deploys we do not
1218@@ -54,6 +56,14 @@
1219 cmd = ['deploy', '-u']
1220 self._deployed_charms.add(charm)
1221
1222+ if config:
1223+ config_path = os.path.join(
1224+ self.useFixture(fixtures.TempDir()).path, 'config.yaml')
1225+ cmd.append('--config={}'.format(config_path))
1226+ config = yaml.safe_dump({name: config}, default_flow_style=False)
1227+ open(config_path, 'w').write(config)
1228+ self.addDetail('pgconfig', text_content(config))
1229+
1230 cmd.append(charm)
1231
1232 if name is None:
1233@@ -93,10 +103,10 @@
1234 self.status = self.get_result(['status'])
1235
1236 self._free_machines = set(
1237- int(k) for k, m in self.status['machines'].items() if
1238- k != '0'
1239- and m.get('life', None) not in ('dead', 'dying')
1240- and m.get('agent-state', 'pending') in ('started', 'ready'))
1241+ int(k) for k, m in self.status['machines'].items()
1242+ if k != '0'
1243+ and m.get('life', None) not in ('dead', 'dying')
1244+ and m.get('agent-state', 'pending') in ('started', 'ready'))
1245 for service in self.status.get('services', {}).values():
1246 for unit in service.get('units', []):
1247 if 'machine' in unit:
1248@@ -201,6 +211,8 @@
1249 raise
1250
1251 (out, err) = proc.communicate(input)
1252+ detail_collector.addDetail(
1253+ 'cmd', text_content('{}: {}'.format(proc.returncode, ' '.join(cmd))))
1254 if out:
1255 detail_collector.addDetail('stdout', text_content(out))
1256 if err:
1257
1258=== added file 'tests/00_setup.test'
1259--- tests/00_setup.test 1970-01-01 00:00:00 +0000
1260+++ tests/00_setup.test 2014-02-07 12:21:04 +0000
1261@@ -0,0 +1,12 @@
1262+#!/bin/sh
1263+
1264+sudo apt-get install -y \
1265+ python-flake8 \
1266+ python-fixtures \
1267+ python-jinja2 \
1268+ python-mocker \
1269+ python-psycopg2 \
1270+ python-testtools \
1271+ python-twisted-core \
1272+ python-yaml
1273+
1274
1275=== added file 'tests/01_lint.test'
1276--- tests/01_lint.test 1970-01-01 00:00:00 +0000
1277+++ tests/01_lint.test 2014-02-07 12:21:04 +0000
1278@@ -0,0 +1,3 @@
1279+#!/bin/bash
1280+
1281+make -C $(dirname $0)/.. lint
1282
1283=== renamed file 'tests/01_unittests.test' => 'tests/02_unit_test.test'
1284--- tests/01_unittests.test 2014-01-15 10:18:42 +0000
1285+++ tests/02_unit_test.test 2014-02-07 12:21:04 +0000
1286@@ -1,3 +1,3 @@
1287 #!/bin/bash
1288
1289-cd $(dirname $0)/../hooks && trial test_hooks.py
1290+make -C $(dirname $0)/.. unit_test
1291
1292=== renamed symlink 'tests/10_juju_integration.test' => 'tests/91_integration_test_91.test' (properties changed: -x to +x)
1293=== target was u'../test.py'
1294--- tests/10_juju_integration.test 1970-01-01 00:00:00 +0000
1295+++ tests/91_integration_test_91.test 2014-02-07 12:21:04 +0000
1296@@ -0,0 +1,3 @@
1297+#!/bin/bash
1298+
1299+make -C $(dirname $0)/.. integration_test_91
1300
1301=== added file 'tests/92_integration_test_92.test'
1302--- tests/92_integration_test_92.test 1970-01-01 00:00:00 +0000
1303+++ tests/92_integration_test_92.test 2014-02-07 12:21:04 +0000
1304@@ -0,0 +1,3 @@
1305+#!/bin/bash
1306+
1307+make -C $(dirname $0)/.. integration_test_92
1308
1309=== added file 'tests/93_integration_test_93.test'
1310--- tests/93_integration_test_93.test 1970-01-01 00:00:00 +0000
1311+++ tests/93_integration_test_93.test 2014-02-07 12:21:04 +0000
1312@@ -0,0 +1,3 @@
1313+#!/bin/bash
1314+
1315+make -C $(dirname $0)/.. integration_test_93

Subscribers

People subscribed via source and target branches