Merge lp:~stub/charms/precise/postgresql/use-swiftwal into lp:charms/postgresql

Proposed by Stuart Bishop
Status: Merged
Merged at revision: 81
Proposed branch: lp:~stub/charms/precise/postgresql/use-swiftwal
Merge into: lp:charms/postgresql
Prerequisite: lp:~stub/charms/precise/postgresql/tests
Diff against target: 154 lines (+88/-5)
7 files modified
config.yaml (+57/-0)
templates/pg_hba.conf.tmpl (+3/-2)
templates/postgres.cron.tmpl (+13/-1)
templates/recovery.conf.tmpl (+6/-0)
templates/swiftwal.conf.tmpl (+6/-0)
test.py (+2/-1)
testing/jujufixture.py (+1/-1)
To merge this branch: bzr merge lp:~stub/charms/precise/postgresql/use-swiftwal
Reviewer Review Type Date Requested Status
Marco Ceppi Approve
Review via email: mp+199942@code.launchpad.net

Commit message

Experimental SwiftWAL support, for PostgreSQL log shipping and PITR into Swift.

Description of the change

SwiftWAL is a tool I've written to do PostgreSQL log shipping into Swift. This branch adds experimental support for the tool. Experimental because there are no tests, and we need it in the charm to test it with real staging and production environments.

Unfortunatly, the OpenStack Swift charms do not work with the local provider. I'd like to have a test suite for this functionality before promoting this from experimental to supported.

To post a comment you must log in.
118. By Stuart Bishop on 2014-01-08

Merged tests into use-swiftwal.

119. By Stuart Bishop on 2014-01-08

Merge trunk

120. By Stuart Bishop on 2014-01-08

Merged tests into use-swiftwal.

121. By Stuart Bishop on 2014-01-08

Merged tests into use-swiftwal.

Marco Ceppi (marcoceppi) wrote :

Approved, please make sure when this is no longer experimental that the README gets updated with instructions

review: Approve
122. By Stuart Bishop on 2014-01-15

Merged tests into use-swiftwal.

123. By Stuart Bishop on 2014-01-22

Merged tests into use-swiftwal.

124. By Stuart Bishop on 2014-01-22

Remove cruft

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'config.yaml'
2--- config.yaml 2014-01-17 03:18:52 +0000
3+++ config.yaml 2014-01-22 07:21:40 +0000
4@@ -343,3 +343,60 @@
5 An advisory lock key used internally by the charm. You do not need
6 to change it unless it happens to conflict with an advisory lock key
7 being used by your applications.
8+
9+ # Swift backups and PITR via SwiftWAL
10+ swiftwal_container_prefix:
11+ type: string
12+ default: ""
13+ description: |
14+ EXPERIMENTAL.
15+ Swift container prefix for SwiftWAL to use. Must be set if any
16+ SwiftWAL features are enabled.
17+ swiftwal_backup_schedule:
18+ type: string
19+ default: ""
20+ description: |
21+ EXPERIMENTAL.
22+ Cron-formatted schedule for SwiftWAL database backups.
23+ swiftwal_backup_retention:
24+ type: int
25+ default: 2
26+ description: |
27+ EXPERIMENTAL.
28+ Number of recent base backups to retain. You need enough space in
29+ Swift for this many backups plus one more, as an old backup will only
30+ be removed after a new one has been successfully made to replace it.
31+ swiftwal_log_shipping:
32+ type: boolean
33+ default: false
34+ description: |
35+ EXPERIMENTAL.
36+ Archive WAL files into Swift. If swiftwal_backup_schedule is set,
37+ this allows point-in-time recovery and WAL files are removed
38+ automatically with old backups. If swiftwal_backup_schedule is not set
39+ then WAL files are never removed. Enabling this option will override
40+ the archive_mode and archive_command settings.
41+ streaming_replication:
42+ type: boolean
43+ default: true
44+ description: |
45+ EXPERIMENTAL.
46+ Enable streaming replication. Normally, streaming replication is
47+ always used, and any log shipping configured is used as a fallback.
48+ Turning this off without configuring log shipping is an error.
49+ os_username:
50+ type: string
51+ default: ""
52+ description: EXPERIMENTAL. OpenStack Swift username.
53+ os_password:
54+ type: string
55+ default: ""
56+ description: EXPERIMENTAL. OpenStack Swift password.
57+ os_auth_url:
58+ type: string
59+ default: ""
60+ description: EXPERIMENTAL. OpenStack Swift authentication URL.
61+ os_tenant_name:
62+ type: string
63+ default: ""
64+ description: EXPERIMENTAL. OpenStack Swift tenant name.
65
66=== modified file 'hooks/hooks.py' (properties changed: +x to -x)
67=== modified file 'templates/pg_hba.conf.tmpl'
68--- templates/pg_hba.conf.tmpl 2013-04-22 20:55:37 +0000
69+++ templates/pg_hba.conf.tmpl 2014-01-22 07:21:40 +0000
70@@ -3,8 +3,9 @@
71 #------------------------------------------------------------------------------
72
73 # Database administrative login by UNIX sockets
74-local all postgres ident map=superusers
75-local all nagios md5
76+local all postgres ident map=superusers
77+local replication root,postgres ident map=superusers
78+local all nagios md5
79
80 {% if access_list is defined -%}
81 {% for unit in access_list -%}
82
83=== modified file 'templates/postgres.cron.tmpl'
84--- templates/postgres.cron.tmpl 2013-02-05 14:09:18 +0000
85+++ templates/postgres.cron.tmpl 2014-01-22 07:21:40 +0000
86@@ -1,2 +1,14 @@
87 {{backup_schedule}} postgres {{scripts_dir}}/pg_backup_job {{backup_days}}
88-
89+{% if swiftwal_container -%}
90+{% if swiftwal_backup_schedule -%}
91+{% if swiftwal_log_shipping -%}
92+{{swiftwal_backup_schedule}} postgres \
93+ swiftwal --config={{swiftwal_config}} backup && \
94+ swiftwal --config={{swiftwal_config}} prune -n {{swiftwal_backup_retention}}
95+{% else -%}
96+{{swiftwal_backup_schedule}} postgres \
97+ swiftwal --config={{swiftwal_config}} backup --xlog && \
98+ swiftwal --config={{swiftwal_config}} prune -n {{swiftwal_backup_retention}}
99+{% endif -%}
100+{% endif -%}
101+{% endif -%}
102
103=== modified file 'templates/recovery.conf.tmpl'
104--- templates/recovery.conf.tmpl 2013-04-03 14:25:47 +0000
105+++ templates/recovery.conf.tmpl 2014-01-22 07:21:40 +0000
106@@ -2,4 +2,10 @@
107 # This file is managed by Juju
108 #------------------------------------------------------------------------------
109 standby_mode = on
110+recovery_target_timeline = latest
111+{% if streaming_replication %}
112 primary_conninfo = 'host={{host}} user=juju_replication password={{password}} requirepeer=postgres'
113+{% endif %}
114+{% if restore_command %}
115+restore_command = '{{restore_command}}'
116+{% endif %}
117
118=== added file 'templates/swiftwal.conf.tmpl'
119--- templates/swiftwal.conf.tmpl 1970-01-01 00:00:00 +0000
120+++ templates/swiftwal.conf.tmpl 2014-01-22 07:21:40 +0000
121@@ -0,0 +1,6 @@
122+# Generated and maintained by juju
123+OS_USERNAME: {{config.os_username}}
124+OS_TENANT_NAME: {{config.os_tenant_name}}
125+OS_PASSWORD: {{config.os_password}}
126+OS_AUTH_URL: {{config.os_auth_url}}
127+CONTAINER: {{vars.container}}
128
129=== modified file 'test.py'
130--- test.py 2013-12-23 10:35:30 +0000
131+++ test.py 2014-01-22 07:21:40 +0000
132@@ -39,7 +39,8 @@
133
134 # If the charms fail, we don't want tests to hang indefinitely.
135 timeout = int(os.environ.get('TEST_TIMEOUT', 900))
136- self.useFixture(fixtures.Timeout(timeout, gentle=True))
137+ if timeout > 0:
138+ self.useFixture(fixtures.Timeout(timeout, gentle=True))
139
140 def sql(self, sql, postgres_unit=None, psql_unit=None, dbname=None):
141 '''Run some SQL on postgres_unit from psql_unit.
142
143=== modified file 'testing/jujufixture.py'
144--- testing/jujufixture.py 2013-12-23 10:35:30 +0000
145+++ testing/jujufixture.py 2014-01-22 07:21:40 +0000
146@@ -104,7 +104,7 @@
147
148 return self.status
149
150- def wait_until_ready(self, extra=45):
151+ def wait_until_ready(self, extra=60):
152 ready = False
153 while not ready:
154 self.refresh_status()

Subscribers

People subscribed via source and target branches