Merge ~dash3/charm-duplicity:lp/1981596 into charm-duplicity:master

Proposed by Nishant Dash
Status: Merged
Approved by: Ramesh Sattaru
Approved revision: 81897f77ba78c3d8db73fe188253f742fc659e7b
Merged at revision: b6789e90b7f9c7089822fecd915f85ae920a7d98
Proposed branch: ~dash3/charm-duplicity:lp/1981596
Merge into: charm-duplicity:master
Diff against target: 1374 lines (+715/-284)
13 files modified
src/README.md (+13/-5)
src/actions.yaml (+38/-1)
src/actions/actions.py (+36/-10)
src/actions/remove-all-but-n-full (+1/-0)
src/actions/remove-all-inc-of-but-n-full (+1/-0)
src/actions/remove-older-than (+1/-0)
src/lib/lib_duplicity.py (+52/-56)
src/tests/functional/requirements.txt (+0/-1)
src/tests/functional/tests/test_duplicity.py (+337/-194)
src/tests/functional/tests/tests.yaml (+3/-0)
src/tests/unit/test_actions.py (+106/-3)
src/tests/unit/test_lib_duplicity.py (+123/-13)
src/tox.ini (+4/-1)
Reviewer Review Type Date Requested Status
Erhan Sunar (community) Approve
🤖 prod-jenkaas-bootstack (community) continuous-integration Approve
Martin Kalcok Pending
BootStack Reviewers Pending
Chi Wai CHAN Pending
Robert Gildein Pending
Eric Chen Pending
Review via email: mp+433766@code.launchpad.net

This proposal supersedes a proposal from 2022-11-28.

Commit message

MR-2 for lp 1981596, updated MR 431126; Refactored lib_duplicity + unit and functional tests; small fix for tox
+ removed old dict constructors from test_duplicity functional test
+ Improvements and Cleanup

Description of the change

To fully support a retention period (lp:1981596), deletion command support will first need
to be added to the charm. After that, cron jobs can be setup along with added values in
config.yaml to have the charm inherently support a retention period.
This commit hopes to address part of the feature request.

+ Refactor and rebased from master since last MR
+ Improvements and Cleanup

Modified:
 src/
  README: added notes on new deletion commands + examples
  actions.yaml: new entries for the new run actions to support deletion commands

 src/actions/
  actions.py: added and modified to support deletion commands through juju run action
  symlinks to added actions

 src/lib/
  lib_duplicity.py: added fns to wrap and build the duplicity commands for deletion, Refactored gtom previous MR

 src/tests/
  added functional and unit tests; Refactored

To post a comment you must log in.
Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : Posted in a previous version of this proposal

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

Revision history for this message
Nishant Dash (dash3) wrote : Posted in a previous version of this proposal

Another note:

If using [charm] in charmcraft.yaml, it will currently only build with charmcraft/2.1/candidate
To make it build with charmcraft/2.0/stable, you will need to specify [charm/2.x/stable]

Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Nishant Dash (dash3) wrote : Posted in a previous version of this proposal

FAIL: test_encryption_true_no_key_no_passphrase_blocks is a test that is known to fail with the current workaround of adding an artificial delay for it failing as well.

Revision history for this message
Eric Chen (eric-chen) wrote : Posted in a previous version of this proposal

This MR is quite big. I need more time to review it. But I check some simple thing quickly. Please check my inline comment.

review: Needs Fixing
Revision history for this message
Martin Kalcok (martin-kalcok) wrote : Posted in a previous version of this proposal

Thanks for your work on this. It was not an easy first task. Overall this change looks good to me. I left some in-line comments, please review them.

review: Needs Fixing
Revision history for this message
Chi Wai CHAN (raychan96) wrote : Posted in a previous version of this proposal

I will fix the functional tests in here: https://code.launchpad.net/~raychan96/charm-duplicity/+git/charm-duplicity/+merge/431319, please rebase you patch after that has been merged.

review: Needs Fixing
Revision history for this message
Chi Wai CHAN (raychan96) wrote : Posted in a previous version of this proposal

Please see inline comment related to functional tests. Thanks

Revision history for this message
Eric Chen (eric-chen) wrote : Posted in a previous version of this proposal

see inline message

review: Needs Fixing
Revision history for this message
Erhan Sunar (esunar) : Posted in a previous version of this proposal
Revision history for this message
Nishant Dash (dash3) wrote : Posted in a previous version of this proposal

Acknowledged the concerns and left a question below with regards to testing.

Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : Posted in a previous version of this proposal

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Approve (continuous-integration)
Revision history for this message
Martin Kalcok (martin-kalcok) wrote : Posted in a previous version of this proposal

Overall looks good. Thanks. Just last few nit picks I found in code that I highlighted inline.

Revision history for this message
Nishant Dash (dash3) wrote : Posted in a previous version of this proposal

ok noted and addressed.

Revision history for this message
Nishant Dash (dash3) : Posted in a previous version of this proposal
Revision history for this message
Chi Wai CHAN (raychan96) wrote : Posted in a previous version of this proposal

Thanks for the patch; overall LGTM, I left some suggestions to improve the readability (just me being picky).

Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : Posted in a previous version of this proposal

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Approve (continuous-integration)
Revision history for this message
Robert Gildein (rgildein) : Posted in a previous version of this proposal
review: Needs Fixing
Revision history for this message
Martin Kalcok (martin-kalcok) wrote : Posted in a previous version of this proposal

LGTM. Thanks for this new functionality and also cleaning up a lot of old tech debt.

review: Approve
Revision history for this message
Mert Kirpici (mertkirpici) wrote : Posted in a previous version of this proposal

Hey Nishant, I left a small suggestion regarding the flake8 issue below.

Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Approve (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote :

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Approve (continuous-integration)
Revision history for this message
Robert Gildein (rgildein) wrote : Posted in a previous version of this proposal

LGTM, good job. Thanks

review: Approve
Revision history for this message
Chi Wai CHAN (raychan96) : Posted in a previous version of this proposal
review: Approve
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Mert Kirpici (mertkirpici) wrote :

Hi Nishant, thanks for taking care of the linting issue! Linked the bug to the MP for better visibility.

Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Approve (continuous-integration)
Revision history for this message
Przemyslaw Lal (przemeklal) wrote : Posted in a previous version of this proposal

LGTM, we confirmed that it works as expected with GPG encryption enabled

review: Approve
Revision history for this message
Erhan Sunar (esunar) :
review: Approve
Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote :

Change successfully merged at revision b6789e90b7f9c7089822fecd915f85ae920a7d98

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/src/README.md b/src/README.md
2index 9666142..443c0c9 100644
3--- a/src/README.md
4+++ b/src/README.md
5@@ -110,17 +110,25 @@ juju add-relation nrpe ubuntu # required on host
6 juju add-relation nrpe duplicity
7 ```
8
9+### Cleaning up backups
10+
11+Deletion commands such as `remove-older-than`, `remove-all-but-n-full` and
12+`remove-all-inc-of-but-n-full` can be used with run action and a specified
13+parameter.
14+For example:
15+ `juju run-action duplicity/0 remove-older-than time=2022-10-02T19:44:00+00:00`
16+ where time follows the same w3 standard as duplicity and yaml
17+ `juju run-action duplicity/0 remove-all-but-n-full count=1`
18+
19 # Known Limitations and Future Features
20
21-This charm is currently still under development. The only supported Duplicity action right now
22-is full backups (through both an action and periodic backups). The following is the list
23-of future Duplicity functionality:
24+This charm is currently still under development. The only supported Duplicity actions right now is full backups (through both an action and periodic backups) and removal of backups.
25+The following is the list of future Duplicity functionality:
26
27 - incremental backups
28 - restoring backups
29 - verifying backups
30-- listing backed-up files
31-- cleaning up backed files
32+- listing backed-up files
33 - additional supported backends
34
35 # Upstream and Bugs
36diff --git a/src/actions.yaml b/src/actions.yaml
37index a683112..8a6d230 100644
38--- a/src/actions.yaml
39+++ b/src/actions.yaml
40@@ -11,4 +11,41 @@ do-backup:
41 # checksum saved during backup
42 list-current-files:
43 description: |
44- Lists the latest backed up files on the remote repository
45\ No newline at end of file
46+ Lists the latest backed up files on the remote repository
47+remove-older-than:
48+ description: |
49+ Delete all backup sets older than the given time on the remote repository
50+ params:
51+ time:
52+ type:
53+ - string
54+ - number
55+ format: date-time
56+ description: |
57+ Time string follows the same time format (w3) as duplicity. For example:
58+ now, 2022-09-30T13:31:15+00:00, 1665058250, 3D4H are all valid.
59+ required: [time]
60+remove-all-but-n-full:
61+ description: |
62+ Keep only the most recent 'count' number of full backup(s) and any
63+ associated incremental sets and delete the rest from the remote repository.
64+ params:
65+ count:
66+ type: integer
67+ minimum: 1
68+ description: |
69+ Count must be larger than zero. A value of 1 means that only the single
70+ most recent backup chain will be kept.
71+ required: [count]
72+remove-all-inc-of-but-n-full:
73+ description: |
74+ Keep only the most recent 'count' number of full backup(s) but NOT any of
75+ their incremental sets and delete the rest from the remote repository.
76+ params:
77+ count:
78+ type: integer
79+ minimum: 1
80+ description: |
81+ Count must be larger than zero. A value of 1 means that only the single
82+ most recent backup chain will be kept intact.
83+ required: [count]
84\ No newline at end of file
85diff --git a/src/actions/actions.py b/src/actions/actions.py
86index 6722ab3..68b9243 100755
87--- a/src/actions/actions.py
88+++ b/src/actions/actions.py
89@@ -19,19 +19,43 @@ helper = DuplicityHelper()
90 error_file = "/var/run/periodic_backup.error"
91
92
93-def do_backup(*args):
94+def do_backup(*_):
95 """do-backup action."""
96 output = helper.do_backup()
97- hookenv.function_set(dict(output=output.decode("utf-8")))
98+ hookenv.action_set({"output": output.decode("utf-8")})
99
100
101-def list_current_files(*args):
102+def list_current_files(*_):
103 """list-current-files action."""
104 output = helper.list_current_files()
105- hookenv.function_set(dict(output=output.decode("utf-8")))
106+ hookenv.action_set({"output": output.decode("utf-8")})
107
108
109-ACTIONS = {"do-backup": do_backup, "list-current-files": list_current_files}
110+def remove_older_than(*_):
111+ """remove-older-than action."""
112+ output = helper.remove_older_than(hookenv.action_get("time"))
113+ hookenv.action_set({"output": output.decode("utf-8")})
114+
115+
116+def remove_all_but_n_full(*_):
117+ """remove-all-but-n-full action."""
118+ output = helper.remove_all_but_n_full(hookenv.action_get("count"))
119+ hookenv.action_set({"output": output.decode("utf-8")})
120+
121+
122+def remove_all_inc_of_but_n_full(*_):
123+ """remove-all-inc-of-but-n-full action."""
124+ output = helper.remove_all_inc_of_but_n_full(hookenv.action_get("count"))
125+ hookenv.action_set({"output": output.decode("utf-8")})
126+
127+
128+ACTIONS = {
129+ "do-backup": do_backup,
130+ "list-current-files": list_current_files,
131+ "remove-older-than": remove_older_than,
132+ "remove-all-but-n-full": remove_all_but_n_full,
133+ "remove-all-inc-of-but-n-full": remove_all_inc_of_but_n_full,
134+}
135
136
137 def main(args):
138@@ -45,18 +69,20 @@ def main(args):
139 except CalledProcessError as e:
140 err_msg = (
141 'Command "{}" failed with return code "{}" '
142- "and error output:\n{}".format(
143- e.cmd, e.returncode, e.output.decode("utf-8")
144+ "and error output:{}{}".format(
145+ e.cmd, e.returncode, os.linesep, e.output.decode("utf-8")
146 )
147 )
148 hookenv.log(err_msg, level=hookenv.ERROR)
149- hookenv.function_fail(err_msg)
150+ hookenv.action_fail(err_msg)
151 except Exception as e:
152 hookenv.log(
153- "do-backup action failed: {}\n{}".format(e, traceback.print_exc()),
154+ "{} action failed: {}{}{}".format(
155+ action_name, e, os.linesep, traceback.print_exc()
156+ ),
157 level=hookenv.ERROR,
158 )
159- hookenv.function_fail(str(e))
160+ hookenv.action_fail(str(e))
161 else:
162 if action_name == "do-backup":
163 clear_flag("duplicity.failed_backup")
164diff --git a/src/actions/remove-all-but-n-full b/src/actions/remove-all-but-n-full
165new file mode 120000
166index 0000000..405a394
167--- /dev/null
168+++ b/src/actions/remove-all-but-n-full
169@@ -0,0 +1 @@
170+actions.py
171\ No newline at end of file
172diff --git a/src/actions/remove-all-inc-of-but-n-full b/src/actions/remove-all-inc-of-but-n-full
173new file mode 120000
174index 0000000..405a394
175--- /dev/null
176+++ b/src/actions/remove-all-inc-of-but-n-full
177@@ -0,0 +1 @@
178+actions.py
179\ No newline at end of file
180diff --git a/src/actions/remove-older-than b/src/actions/remove-older-than
181new file mode 120000
182index 0000000..405a394
183--- /dev/null
184+++ b/src/actions/remove-older-than
185@@ -0,0 +1 @@
186+actions.py
187\ No newline at end of file
188diff --git a/src/lib/lib_duplicity.py b/src/lib/lib_duplicity.py
189index ad2f768..3b6bd57 100644
190--- a/src/lib/lib_duplicity.py
191+++ b/src/lib/lib_duplicity.py
192@@ -28,42 +28,20 @@ class DuplicityHelper:
193 """Introduce configurations."""
194 self.charm_config = hookenv.config()
195
196- @property
197- def backup_cmd(self):
198- """Juju action do-backup handler."""
199- cmd = ["duplicity"]
200- if self.charm_config.get("private_ssh_key"):
201- if self.charm_config.get("backend") == "rsync":
202- cmd.append(
203- '--rsync-options=-e "ssh -i {}"'.format(PRIVATE_SSH_KEY_PATH)
204- )
205- else:
206- cmd.append(
207- "--ssh-options=-oIdentityFile={}".format(PRIVATE_SSH_KEY_PATH)
208- )
209- # later switch to
210- # cmd.append('full' if self.charm_config.get('full_backup') else 'incr')
211- # when full_backup implemented
212- cmd.append("full")
213- cmd.extend([self.charm_config.get("aux_backup_directory"), self._backup_url()])
214+ def _build_cmd(self, duplicity_command, *args):
215+ """Duplicity command builder."""
216+ cmd = ["duplicity", duplicity_command]
217+ cmd.extend([str(arg) for arg in args])
218+ cmd.append(self._backup_url())
219 cmd.extend(self._additional_options())
220+ if "remove" in duplicity_command:
221+ cmd.append("--force")
222 return cmd
223
224- @property
225- def list_files_cmd(self):
226- """Juju action list-current-files handler."""
227- cmd = ["duplicity", "list-current-files", self._backup_url()]
228- if self.charm_config.get("private_ssh_key"):
229- if self.charm_config.get("backend") == "rsync":
230- cmd.append(
231- '--rsync-options=-e "ssh -i {}"'.format(PRIVATE_SSH_KEY_PATH)
232- )
233- else:
234- cmd.append(
235- "--ssh-options=-oIdentityFile={}".format(PRIVATE_SSH_KEY_PATH)
236- )
237- cmd.extend(self._additional_options())
238- return cmd
239+ def _executor(self, cmd):
240+ self._set_environment_vars()
241+ self.safe_log("Duplicity Command: {}".format(cmd))
242+ return subprocess.check_output(cmd, stderr=subprocess.STDOUT)
243
244 def _backup_url(self):
245 """Remote URL.
246@@ -128,6 +106,16 @@ class DuplicityHelper:
247 # backups named after the unit
248 cmd = []
249
250+ if self.charm_config.get("private_ssh_key"):
251+ if self.charm_config.get("backend") == "rsync":
252+ cmd.append(
253+ '--rsync-options=-e "ssh -i {}"'.format(PRIVATE_SSH_KEY_PATH)
254+ )
255+ else:
256+ cmd.append(
257+ "--ssh-options=-oIdentityFile={}".format(PRIVATE_SSH_KEY_PATH)
258+ )
259+
260 if self.charm_config.get("disable_encryption"):
261 cmd.append("--no-encryption")
262 elif self.charm_config.get("gpg_public_key"):
263@@ -175,12 +163,10 @@ class DuplicityHelper:
264 :param: kwargs
265 :type: dictionary of values that may be used instead of config values
266 """
267- self._set_environment_vars()
268- cmd = self.backup_cmd
269- self.safe_log("Duplicity Command: {}".format(cmd))
270+ cmd = self._build_cmd("full", self.charm_config.get("aux_backup_directory"))
271 if self.charm_config.get("backend") == "rsync":
272 self.create_remote_dirs()
273- return subprocess.check_output(cmd, stderr=subprocess.STDOUT)
274+ return self._executor(cmd)
275
276 def safe_log(self, message, level=hookenv.INFO):
277 """Replace password in the log with ***."""
278@@ -241,10 +227,8 @@ class DuplicityHelper:
279 :param: kwargs
280 :type: dictionary of values that may be used instead of config values
281 """
282- self._set_environment_vars()
283- cmd = self.list_files_cmd
284- self.safe_log("Duplicity Command: {}".format(cmd))
285- return subprocess.check_output(cmd, stderr=subprocess.STDOUT)
286+ cmd = self._build_cmd("list-current-files")
287+ return self._executor(cmd)
288
289 def restore(self):
290 # TODO
291@@ -252,20 +236,32 @@ class DuplicityHelper:
292 """Restore the full monty or selected folders/files."""
293 raise NotImplementedError()
294
295- def remove_older_than(self):
296- # TODO
297- # duplicity remove-older-than time [options] target_url
298- """Delete all backup sets older than the given time."""
299- raise NotImplementedError()
300+ def remove_older_than(self, time):
301+ """Delete all backup sets older than the given time.
302
303- def remove_all_but_n_full(self):
304- # TODO
305- # duplicity remove-all-but-n-full <count> <target_url>
306- """Keep the last count full backups and associated incremental sets."""
307- raise NotImplementedError()
308+ :param: kwargs
309+ :type: dictionary of values that may be used instead of config values
310+ - used types from kwargs: time
311+ """
312+ cmd = self._build_cmd("remove-older-than", time)
313+ return self._executor(cmd)
314
315- def remove_all_inc_of_but_n_full(self):
316- # TODO
317- # duplicity remove-all-inc-of-but-n-full <count> <target_url>
318- """Keep only old full backups and not their increments."""
319- raise NotImplementedError()
320+ def remove_all_but_n_full(self, count):
321+ """Keep the last count full backups and associated incremental sets.
322+
323+ :param: kwargs
324+ :type: dictionary of values that may be used instead of config values
325+ - used types from kwargs: count
326+ """
327+ cmd = self._build_cmd("remove-all-but-n-full", count)
328+ return self._executor(cmd)
329+
330+ def remove_all_inc_of_but_n_full(self, count):
331+ """Keep only old full backups and not their increments.
332+
333+ :param: kwargs
334+ :type: dictionary of values that may be used instead of config values
335+ - used types from kwargs: count
336+ """
337+ cmd = self._build_cmd("remove-all-inc-of-but-n-full", count)
338+ return self._executor(cmd)
339diff --git a/src/tests/functional/requirements.txt b/src/tests/functional/requirements.txt
340index d4871a5..4a6e1d8 100644
341--- a/src/tests/functional/requirements.txt
342+++ b/src/tests/functional/requirements.txt
343@@ -1,5 +1,4 @@
344 flake8
345-juju
346 mock
347 git+https://github.com/openstack-charmers/zaza.git#egg=zaza
348 python-openstackclient
349diff --git a/src/tests/functional/tests/test_duplicity.py b/src/tests/functional/tests/test_duplicity.py
350index 29200cd..a926ab5 100644
351--- a/src/tests/functional/tests/test_duplicity.py
352+++ b/src/tests/functional/tests/test_duplicity.py
353@@ -38,7 +38,7 @@ class DuplicityBackupCronTest(BaseDuplicityTest):
354 """Verify cron job creation."""
355 options = ["daily", "weekly", "monthly"]
356 for option in options:
357- new_config = dict(backup_frequency=option)
358+ new_config = {"backup_frequency": option}
359 zaza.model.set_application_config(self.application_name, new_config)
360 try:
361 zaza.model.block_until_file_has_contents(
362@@ -57,7 +57,7 @@ class DuplicityBackupCronTest(BaseDuplicityTest):
363 def test_cron_creation_cron_string(self):
364 """Verify cron job creation."""
365 cron_string = "* * * * *"
366- new_config = dict(backup_frequency=cron_string)
367+ new_config = {"backup_frequency": cron_string}
368 zaza.model.set_application_config(self.application_name, new_config)
369 try:
370 zaza.model.block_until_file_has_contents(
371@@ -76,7 +76,7 @@ class DuplicityBackupCronTest(BaseDuplicityTest):
372 def test_cron_invalid_cron_string(self):
373 """Verify cron job creation with invalid frequency."""
374 cron_string = "* * * *"
375- new_config = dict(backup_frequency=cron_string)
376+ new_config = {"backup_frequency": cron_string}
377 zaza.model.set_application_config(self.application_name, new_config)
378 try:
379 duplicity_workload_checker = utils.get_workload_application_status_checker(
380@@ -96,7 +96,7 @@ class DuplicityBackupCronTest(BaseDuplicityTest):
381 """Verify manual or invalid cron job frequency."""
382 options = ["manual"]
383 for option in options:
384- new_config = dict(backup_frequency=option)
385+ new_config = {"backup_frequency": option}
386 zaza.model.set_application_config(self.application_name, new_config)
387 try:
388 zaza.model.block_until_file_missing(
389@@ -123,9 +123,11 @@ class DuplicityEncryptionValidationTest(BaseDuplicityTest):
390 @utils.config_restore("duplicity")
391 def test_encryption_true_no_key_no_passphrase_blocks(self):
392 """Verify unit is blocked with no passphrase or key."""
393- new_config = dict(
394- encryption_passphrase="", gpg_public_key="", disable_encryption="False"
395- )
396+ new_config = {
397+ "encryption_passphrase": "",
398+ "gpg_public_key": "",
399+ "disable_encryption": "False",
400+ }
401 zaza.model.set_application_config(
402 self.application_name, new_config, self.model_name
403 )
404@@ -150,7 +152,7 @@ class DuplicityEncryptionValidationTest(BaseDuplicityTest):
405 def test_encryption_true_with_key(self):
406 """Verify encryption with a valid gpg key."""
407 zaza.model.set_application_config(
408- self.application_name, dict(disable_encryption="False"), self.model_name
409+ self.application_name, {"disable_encryption": "False"}, self.model_name
410 )
411 try:
412 duplicity_workload_checker = utils.get_workload_application_status_checker(
413@@ -163,7 +165,7 @@ class DuplicityEncryptionValidationTest(BaseDuplicityTest):
414 "no passphrase or key."
415 )
416 zaza.model.set_application_config(
417- self.application_name, dict(gpg_public_key="S0M3k3Y")
418+ self.application_name, {"gpg_public_key": "S0M3k3Y"}
419 )
420 try:
421 zaza.model.block_until_all_units_idle()
422@@ -177,7 +179,7 @@ class DuplicityEncryptionValidationTest(BaseDuplicityTest):
423 def test_encryption_true_with_passphrase(self):
424 """Verify encryption with a valid passphrase."""
425 zaza.model.set_application_config(
426- self.application_name, dict(disable_encryption="False"), self.model_name
427+ self.application_name, {"disable_encryption": "False"}, self.model_name
428 )
429 try:
430 duplicity_workload_checker = utils.get_workload_application_status_checker(
431@@ -190,7 +192,7 @@ class DuplicityEncryptionValidationTest(BaseDuplicityTest):
432 "no passphrase or key."
433 )
434 zaza.model.set_application_config(
435- self.application_name, dict(encryption_passphrase="somephrase")
436+ self.application_name, {"encryption_passphrase": "somephrase"}
437 )
438 try:
439 zaza.model.block_until_all_units_idle()
440@@ -201,33 +203,25 @@ class DuplicityEncryptionValidationTest(BaseDuplicityTest):
441 )
442
443
444-class DuplicityBackupCommandTest(BaseDuplicityTest):
445- """Verify do-backup command."""
446+class BaseDuplicityCommandTest(BaseDuplicityTest):
447+ """VHelper class to use for duplicity command tests."""
448
449 @classmethod
450 def setUpClass(cls):
451 """Set up do-backup command tests."""
452 super().setUpClass()
453 cls.backup_host = zaza.model.get_units("backup-host")[0]
454- cls.duplicity_unit = zaza.model.get_units("duplicity")[0]
455- cls.backup_host_ip = cls.backup_host.public_address
456+ cls.duplicity_unit = zaza.model.get_units("duplicity")[0].name
457 user_pass_pair = ubuntu_user_pass.split(":")
458- cls.remote_user = user_pass_pair[0]
459- cls.remote_pass = user_pass_pair[1]
460- cls.action = "do-backup"
461 cls.ssh_priv_key = cls.get_ssh_priv_key()
462-
463- def get_config(self, **kwargs):
464- """Return app config."""
465- base_config = dict(
466- remote_backup_url=self.backup_host_ip,
467- aux_backup_directory=ubuntu_backup_directory_source,
468- remote_user=self.remote_user,
469- remote_password=self.remote_pass,
470- )
471- for key, value in kwargs.items():
472- base_config[key] = value
473- return base_config
474+ cls.base_config = {
475+ "remote_backup_url": cls.backup_host.public_address,
476+ "aux_backup_directory": ubuntu_backup_directory_source,
477+ "remote_user": user_pass_pair[0],
478+ "remote_password": user_pass_pair[1],
479+ }
480+ cls.auxiliary_actions = []
481+ cls.action_params = None
482
483 @staticmethod
484 def get_ssh_priv_key():
485@@ -237,208 +231,357 @@ class DuplicityBackupCommandTest(BaseDuplicityTest):
486 encoded_ssh_private_key = base64.b64encode(ssh_private_key)
487 return encoded_ssh_private_key.decode("utf-8")
488
489- @utils.config_restore("duplicity")
490- def test_scp_full_do_backup_action(self):
491- """Verify do-backup action with scp."""
492- additional_config = dict(backend="scp")
493- new_config = self.get_config(**additional_config)
494- utils.set_config_and_wait(self.application_name, new_config)
495+ def _run(self, **config):
496+ """Run action on zaza model."""
497+ config.update(self.base_config)
498+ utils.set_config_and_wait(self.application_name, config)
499+ for a in self.auxiliary_actions:
500+ zaza.model.run_action(self.duplicity_unit, a, raise_on_failure=True)
501 zaza.model.run_action(
502- self.duplicity_unit.name, self.action, raise_on_failure=True
503+ self.duplicity_unit,
504+ self.action,
505+ action_params=self.action_params,
506+ raise_on_failure=True,
507 )
508
509+
510+class DuplicityBackupCommandTest(BaseDuplicityCommandTest):
511+ """Verify do-backup command."""
512+
513+ @classmethod
514+ def setUpClass(cls):
515+ """Set up do-backup command tests."""
516+ super().setUpClass()
517+ cls.action = "do-backup"
518+
519 @utils.config_restore("duplicity")
520- def test_file_full_do_backup_action(self):
521- """Verify do-backup action with ftp."""
522- additional_config = dict(
523- backend="file", remote_backup_url="/home/ubuntu/test-backups"
524- )
525- new_config = self.get_config(**additional_config)
526- utils.set_config_and_wait(self.application_name, new_config)
527- zaza.model.run_action(
528- self.duplicity_unit.name, self.action, raise_on_failure=True
529- )
530+ def test_scp_full(self):
531+ """Verify do-backup action with scp."""
532+ additional_config = {"backend": "scp"}
533+ self._run(**additional_config)
534
535 @utils.config_restore("duplicity")
536- def test_scp_full_ssh_key_auth_backup_action(self):
537+ def test_file_full(self):
538+ """Verify do-backup action with file."""
539+ additional_config = {
540+ "backend": "file",
541+ "remote_backup_url": "/home/ubuntu/test-backups",
542+ }
543+ self._run(**additional_config)
544+
545+ @utils.config_restore("duplicity")
546+ def test_scp_full_ssh_key(self):
547 """Verify do-backup action with scp and private key."""
548- additional_config = dict(
549- backend="scp", private_ssh_key=self.ssh_priv_key, remote_password=""
550- )
551- new_config = self.get_config(**additional_config)
552- utils.set_config_and_wait(self.application_name, new_config)
553- zaza.model.run_action(
554- self.duplicity_unit.name, self.action, raise_on_failure=True
555- )
556+ additional_config = {
557+ "backend": "scp",
558+ "private_ssh_key": self.ssh_priv_key,
559+ "remote_password": "",
560+ }
561+ self._run(**additional_config)
562
563 @utils.config_restore("duplicity")
564- def test_rsync_full_ssh_key_auth_backup_action(self):
565+ def test_rsync_full_ssh_key(self):
566 """Verify do-backup action with rsync and private key."""
567- additional_config = dict(
568- backend="rsync", private_ssh_key=self.ssh_priv_key, remote_password=""
569- )
570- new_config = self.get_config(**additional_config)
571- utils.set_config_and_wait(self.application_name, new_config)
572- zaza.model.run_action(
573- self.duplicity_unit.name, self.action, raise_on_failure=True
574- )
575+ additional_config = {
576+ "backend": "rsync",
577+ "private_ssh_key": self.ssh_priv_key,
578+ "remote_password": "",
579+ }
580+ self._run(**additional_config)
581
582 @utils.config_restore("duplicity")
583- def test_sftp_full_do_backup(self):
584+ def test_sftp_full(self):
585 """Verify do-backup action with sftp and password."""
586- additional_config = dict(backend="sftp")
587- new_config = self.get_config(**additional_config)
588- utils.set_config_and_wait(self.application_name, new_config)
589- zaza.model.run_action(
590- self.duplicity_unit.name, self.action, raise_on_failure=True
591- )
592+ additional_config = {"backend": "sftp"}
593+ self._run(**additional_config)
594
595 @utils.config_restore("duplicity")
596- def test_sftp_full_ssh_key_do_backup(self):
597- """Verify do-backup action with sftp with private key."""
598- additional_config = dict(
599- backend="sftp", private_ssh_key=self.ssh_priv_key, remote_password=""
600- )
601- new_config = self.get_config(**additional_config)
602- utils.set_config_and_wait(self.application_name, new_config)
603- zaza.model.run_action(
604- self.duplicity_unit.name, self.action, raise_on_failure=True
605- )
606+ def test_sftp_full_ssh_key(self):
607+ """Verify do-backup action with sftp and private key."""
608+ additional_config = {
609+ "backend": "sftp",
610+ "private_ssh_key": self.ssh_priv_key,
611+ "remote_password": "",
612+ }
613+ self._run(**additional_config)
614
615 @utils.config_restore("duplicity")
616- def test_ftp_full_do_backup(self):
617- """Verify do-backup action with ftp."""
618- additional_config = dict(backend="ftp")
619- new_config = self.get_config(**additional_config)
620- utils.set_config_and_wait(self.application_name, new_config)
621- zaza.model.run_action(
622- self.duplicity_unit.name, self.action, raise_on_failure=True
623- )
624+ def test_ftp_full(self):
625+ """Verify do-backup action with ftp and password."""
626+ additional_config = {"backend": "ftp"}
627+ self._run(**additional_config)
628
629
630-class DuplicityListFilesCommandTest(BaseDuplicityTest):
631- """Verify list-current-files action."""
632+class DuplicityListFilesCommandTest(BaseDuplicityCommandTest):
633+ """Verify list-current-files command."""
634
635 @classmethod
636 def setUpClass(cls):
637- """Set up list-current-files action tests."""
638+ """Set up list-current-files command tests."""
639 super().setUpClass()
640- cls.backup_host = zaza.model.get_units("backup-host")[0]
641- cls.duplicity_unit = zaza.model.get_units("duplicity")[0]
642- cls.backup_host_ip = cls.backup_host.public_address
643- user_pass_pair = ubuntu_user_pass.split(":")
644- cls.remote_user = user_pass_pair[0]
645- cls.remote_pass = user_pass_pair[1]
646 cls.action = "list-current-files"
647- cls.ssh_priv_key = cls.get_ssh_priv_key()
648-
649- def get_config(self, **kwargs):
650- """Get charm config."""
651- base_config = dict(
652- remote_backup_url=self.backup_host_ip,
653- aux_backup_directory=ubuntu_backup_directory_source,
654- remote_user=self.remote_user,
655- remote_password=self.remote_pass,
656- )
657- base_config.update(kwargs)
658- return base_config
659-
660- @staticmethod
661- def get_ssh_priv_key():
662- """Get ssh private key."""
663- with open("./tests/resources/testing_id_rsa", "rb") as f:
664- ssh_private_key = f.read()
665- encoded_ssh_private_key = base64.b64encode(ssh_private_key)
666- return encoded_ssh_private_key.decode("utf-8")
667+ cls.auxiliary_actions = ["do-backup"]
668
669 @utils.config_restore("duplicity")
670- def test_scp_full_list_current_files_action(self):
671- """Verify list-current-files work with scp backend."""
672- new_config = self.get_config(backend="scp")
673- utils.set_config_and_wait(self.application_name, new_config)
674- zaza.model.run_action(
675- self.duplicity_unit.name, "do-backup", raise_on_failure=True
676- )
677- zaza.model.run_action(
678- self.duplicity_unit.name, self.action, raise_on_failure=True
679- )
680+ def test_scp_full_list_current_files(self):
681+ """Verify list-current-files action with scp."""
682+ additional_config = {"backend": "scp"}
683+ self._run(**additional_config)
684
685 @utils.config_restore("duplicity")
686- def test_file_full_list_current_files_action(self):
687- """Verify list-current-files work with file backend."""
688- new_config = self.get_config(
689- backend="file", remote_backup_url="/home/ubuntu/test-backups"
690- )
691- utils.set_config_and_wait(self.application_name, new_config)
692- zaza.model.run_action(
693- self.duplicity_unit.name, "do-backup", raise_on_failure=True
694- )
695- zaza.model.run_action(
696- self.duplicity_unit.name, self.action, raise_on_failure=True
697- )
698+ def test_file_full_list_current_files(self):
699+ """Verify list-current-files action with file."""
700+ additional_config = {
701+ "backend": "file",
702+ "remote_backup_url": "/home/ubuntu/test-backups",
703+ }
704+ self._run(**additional_config)
705
706 @utils.config_restore("duplicity")
707- def test_scp_full_ssh_key_auth_list_current_files_action(self):
708- """Verify list-current-files work after do-backup run."""
709- new_config = self.get_config(
710- backend="scp", private_ssh_key=self.ssh_priv_key, remote_password=""
711- )
712- utils.set_config_and_wait(self.application_name, new_config)
713- zaza.model.run_action(
714- self.duplicity_unit.name, "do-backup", raise_on_failure=True
715- )
716- zaza.model.run_action(
717- self.duplicity_unit.name, self.action, raise_on_failure=True
718- )
719+ def test_scp_full_ssh_key_list_current_files(self):
720+ """Verify list-current-files action with scp and private key."""
721+ additional_config = {
722+ "backend": "scp",
723+ "private_ssh_key": self.ssh_priv_key,
724+ "remote_password": "",
725+ }
726+ self._run(**additional_config)
727
728 @utils.config_restore("duplicity")
729- def test_rsync_full_ssh_key_auth_list_current_files_action(self):
730- """Verify list-current-files work with rsync backend after do-backup run."""
731- new_config = self.get_config(
732- backend="rsync", private_ssh_key=self.ssh_priv_key, remote_password=""
733- )
734- utils.set_config_and_wait(self.application_name, new_config)
735- zaza.model.run_action(
736- self.duplicity_unit.name, "do-backup", raise_on_failure=True
737- )
738- zaza.model.run_action(
739- self.duplicity_unit.name, self.action, raise_on_failure=True
740- )
741+ def test_rsync_full_ssh_key_list_current_files(self):
742+ """Verify list-current-files action with rsync and private key."""
743+ additional_config = {
744+ "backend": "rsync",
745+ "private_ssh_key": self.ssh_priv_key,
746+ "remote_password": "",
747+ }
748+ self._run(**additional_config)
749
750 @utils.config_restore("duplicity")
751 def test_sftp_full_list_current_files(self):
752- """Verify list-current-files work with sftp backend after do-backup run."""
753- new_config = self.get_config(backend="sftp")
754- utils.set_config_and_wait(self.application_name, new_config)
755- zaza.model.run_action(
756- self.duplicity_unit.name, "do-backup", raise_on_failure=True
757- )
758- zaza.model.run_action(
759- self.duplicity_unit.name, self.action, raise_on_failure=True
760- )
761+ """Verify list-current-files action with sftp and password."""
762+ additional_config = {"backend": "sftp"}
763+ self._run(**additional_config)
764
765 @utils.config_restore("duplicity")
766 def test_sftp_full_ssh_key_list_current_files(self):
767- """Verify list-current-files work with sftp backend after do-backup run."""
768- new_config = self.get_config(
769- backend="sftp", private_ssh_key=self.ssh_priv_key, remote_password=""
770- )
771- utils.set_config_and_wait(self.application_name, new_config)
772- zaza.model.run_action(
773- self.duplicity_unit.name, "do-backup", raise_on_failure=True
774- )
775- zaza.model.run_action(
776- self.duplicity_unit.name, self.action, raise_on_failure=True
777- )
778+ """Verify list-current-files action with sftp and private key."""
779+ additional_config = {
780+ "backend": "sftp",
781+ "private_ssh_key": self.ssh_priv_key,
782+ "remote_password": "",
783+ }
784+ self._run(**additional_config)
785
786 @utils.config_restore("duplicity")
787 def test_ftp_full_list_current_files(self):
788- """Verify list-current-files work with ftp backend after do-backup run."""
789- new_config = self.get_config(backend="ftp")
790- utils.set_config_and_wait(self.application_name, new_config)
791- zaza.model.run_action(
792- self.duplicity_unit.name, "do-backup", raise_on_failure=True
793- )
794- zaza.model.run_action(
795- self.duplicity_unit.name, self.action, raise_on_failure=True
796- )
797+ """Verify list-current-files action with ftp and password."""
798+ additional_config = {"backend": "ftp"}
799+ self._run(**additional_config)
800+
801+
802+class DuplicityRemoveOlderThanCommandTest(BaseDuplicityCommandTest):
803+ """Verify remove-older-than command."""
804+
805+ @classmethod
806+ def setUpClass(cls):
807+ """Set up remove-older-than command tests."""
808+ super().setUpClass()
809+ cls.action = "remove-older-than"
810+ cls.auxiliary_actions = ["do-backup", "do-backup"]
811+ cls.action_params = {"time": "now"}
812+
813+ @utils.config_restore("duplicity")
814+ def test_scp_full_list_remove_older_than(self):
815+ """Verify remove-older-than action with scp."""
816+ additional_config = {"backend": "scp"}
817+ self._run(**additional_config)
818+
819+ @utils.config_restore("duplicity")
820+ def test_file_full_remove_older_than(self):
821+ """Verify remove-older-than action with file."""
822+ additional_config = {
823+ "backend": "file",
824+ "remote_backup_url": "/home/ubuntu/test-backups",
825+ }
826+ self._run(**additional_config)
827+
828+ @utils.config_restore("duplicity")
829+ def test_scp_full_ssh_key_remove_older_than(self):
830+ """Verify remove-older-than action with scp and private key."""
831+ additional_config = {
832+ "backend": "scp",
833+ "private_ssh_key": self.ssh_priv_key,
834+ "remote_password": "",
835+ }
836+ self._run(**additional_config)
837+
838+ @utils.config_restore("duplicity")
839+ def test_rsync_full_ssh_key_remove_older_than(self):
840+ """Verify remove-older-than action with rsync and private key."""
841+ additional_config = {
842+ "backend": "rsync",
843+ "private_ssh_key": self.ssh_priv_key,
844+ "remote_password": "",
845+ }
846+ self._run(**additional_config)
847+
848+ @utils.config_restore("duplicity")
849+ def test_sftp_full_remove_older_than(self):
850+ """Verify remove-older-than action with sftp and password."""
851+ additional_config = {"backend": "sftp"}
852+ self._run(**additional_config)
853+
854+ @utils.config_restore("duplicity")
855+ def test_sftp_full_ssh_key_remove_older_than(self):
856+ """Verify remove-older-than action with sftp and private key."""
857+ additional_config = {
858+ "backend": "sftp",
859+ "private_ssh_key": self.ssh_priv_key,
860+ "remote_password": "",
861+ }
862+ self._run(**additional_config)
863+
864+ @utils.config_restore("duplicity")
865+ def test_ftp_full_remove_older_than(self):
866+ """Verify remove-older-than action with ftp and password."""
867+ additional_config = {"backend": "ftp"}
868+ self._run(**additional_config)
869+
870+
871+class DuplicityRemoveAllButNFullCommandTest(BaseDuplicityCommandTest):
872+ """Verify remove-all-but-n-full command."""
873+
874+ @classmethod
875+ def setUpClass(cls):
876+ """Set up remove-all-but-n-full command tests."""
877+ super().setUpClass()
878+ cls.action = "remove-all-but-n-full"
879+ cls.auxiliary_actions = ["do-backup", "do-backup"]
880+ cls.action_params = {"count": 1}
881+
882+ @utils.config_restore("duplicity")
883+ def test_scp_full_list_remove_all_but_n_full(self):
884+ """Verify remove-all-but-n-full action with scp."""
885+ additional_config = {"backend": "scp"}
886+ self._run(**additional_config)
887+
888+ @utils.config_restore("duplicity")
889+ def test_file_full_remove_all_but_n_full(self):
890+ """Verify remove-all-but-n-full action with file."""
891+ additional_config = {
892+ "backend": "file",
893+ "remote_backup_url": "/home/ubuntu/test-backups",
894+ }
895+ self._run(**additional_config)
896+
897+ @utils.config_restore("duplicity")
898+ def test_scp_full_ssh_key_remove_all_but_n_full(self):
899+ """Verify remove-all-but-n-full action with scp and private key."""
900+ additional_config = {
901+ "backend": "scp",
902+ "private_ssh_key": self.ssh_priv_key,
903+ "remote_password": "",
904+ }
905+ self._run(**additional_config)
906+
907+ @utils.config_restore("duplicity")
908+ def test_rsync_full_ssh_key_remove_all_but_n_full(self):
909+ """Verify remove-all-but-n-full action with rsync and private key."""
910+ additional_config = {
911+ "backend": "rsync",
912+ "private_ssh_key": self.ssh_priv_key,
913+ "remote_password": "",
914+ }
915+ self._run(**additional_config)
916+
917+ @utils.config_restore("duplicity")
918+ def test_sftp_full_remove_all_but_n_full(self):
919+ """Verify remove-all-but-n-full action with sftp and password."""
920+ additional_config = {"backend": "sftp"}
921+ self._run(**additional_config)
922+
923+ @utils.config_restore("duplicity")
924+ def test_sftp_full_ssh_key_remove_all_but_n_full(self):
925+ """Verify remove-all-but-n-full action with sftp and private key."""
926+ additional_config = {
927+ "backend": "sftp",
928+ "private_ssh_key": self.ssh_priv_key,
929+ "remote_password": "",
930+ }
931+ self._run(**additional_config)
932+
933+ @utils.config_restore("duplicity")
934+ def test_ftp_full_remove_all_but_n_full(self):
935+ """Verify remove-all-but-n-full action with ftp and password."""
936+ additional_config = {"backend": "ftp"}
937+ self._run(**additional_config)
938+
939+
940+class DuplicityRemoveAllIncOfButNFullCommandTest(BaseDuplicityCommandTest):
941+ """Verify remove-all-inc-of-but-n-fullcommand."""
942+
943+ @classmethod
944+ def setUpClass(cls):
945+ """Set up remove-all-inc-of-but-n-full command tests."""
946+ super().setUpClass()
947+ cls.action = "remove-all-inc-of-but-n-full"
948+ cls.auxiliary_actions = ["do-backup", "do-backup"]
949+ cls.action_params = {"count": 1}
950+
951+ @utils.config_restore("duplicity")
952+ def test_scp_full_list_remove_all_inc_of_but_n_full(self):
953+ """Verify remove-all-inc-of-but-n-full action with scp."""
954+ additional_config = {"backend": "scp"}
955+ self._run(**additional_config)
956+
957+ @utils.config_restore("duplicity")
958+ def test_file_full_remove_all_inc_of_but_n_full(self):
959+ """Verify remove-all-inc-of-but-n-full action with file."""
960+ additional_config = {
961+ "backend": "file",
962+ "remote_backup_url": "/home/ubuntu/test-backups",
963+ }
964+ self._run(**additional_config)
965+
966+ @utils.config_restore("duplicity")
967+ def test_scp_full_ssh_key_remove_all_inc_of_but_n_full(self):
968+ """Verify remove-all-inc-of-but-n-full action with scp and private key."""
969+ additional_config = {
970+ "backend": "scp",
971+ "private_ssh_key": self.ssh_priv_key,
972+ "remote_password": "",
973+ }
974+ self._run(**additional_config)
975+
976+ @utils.config_restore("duplicity")
977+ def test_rsync_full_ssh_key_remove_all_inc_of_but_n_full(self):
978+ """Verify remove-all-inc-of-but-n-full action with rsync and private key."""
979+ additional_config = {
980+ "backend": "rsync",
981+ "private_ssh_key": self.ssh_priv_key,
982+ "remote_password": "",
983+ }
984+ self._run(**additional_config)
985+
986+ @utils.config_restore("duplicity")
987+ def test_sftp_full_remove_all_inc_of_but_n_full(self):
988+ """Verify remove-all-inc-of-but-n-full action with sftp and password."""
989+ additional_config = {"backend": "sftp"}
990+ self._run(**additional_config)
991+
992+ @utils.config_restore("duplicity")
993+ def test_sftp_full_ssh_remove_all_inc_of_but_n_full(self):
994+ """Verify remove-all-inc-of-but-n-full action with sftp and private key."""
995+ additional_config = {
996+ "backend": "sftp",
997+ "private_ssh_key": self.ssh_priv_key,
998+ "remote_password": "",
999+ }
1000+ self._run(**additional_config)
1001+
1002+ @utils.config_restore("duplicity")
1003+ def test_ftp_full_remove_all_inc_of_but_n_full(self):
1004+ """Verify remove-all-inc-of-but-n-full action with ftp and password."""
1005+ additional_config = {"backend": "ftp"}
1006+ self._run(**additional_config)
1007diff --git a/src/tests/functional/tests/tests.yaml b/src/tests/functional/tests/tests.yaml
1008index 155c4e9..74aa1bd 100644
1009--- a/src/tests/functional/tests/tests.yaml
1010+++ b/src/tests/functional/tests/tests.yaml
1011@@ -4,6 +4,9 @@ tests:
1012 - tests.test_duplicity.DuplicityEncryptionValidationTest
1013 - tests.test_duplicity.DuplicityBackupCommandTest
1014 - tests.test_duplicity.DuplicityListFilesCommandTest
1015+ - tests.test_duplicity.DuplicityRemoveOlderThanCommandTest
1016+ - tests.test_duplicity.DuplicityRemoveAllButNFullCommandTest
1017+ - tests.test_duplicity.DuplicityRemoveAllIncOfButNFullCommandTest
1018 configure:
1019 - tests.configure.set_ubuntu_password_on_backup_host
1020 - tests.configure.set_ssh_password_access_on_backup_host
1021diff --git a/src/tests/unit/test_actions.py b/src/tests/unit/test_actions.py
1022index 632b762..d24aa4c 100644
1023--- a/src/tests/unit/test_actions.py
1024+++ b/src/tests/unit/test_actions.py
1025@@ -52,6 +52,64 @@ class TestActions:
1026 mock_list_current_files.assert_called_with(action_args)
1027 assert mock_remove.called == error_path_exists
1028
1029+ @pytest.mark.parametrize("error_path_exists", [True, False])
1030+ @patch("actions.remove_older_than")
1031+ @patch("actions.os.path.exists")
1032+ @patch("actions.os.remove")
1033+ def test_remove_older_than_action_run_success(
1034+ self,
1035+ mock_remove,
1036+ mock_exists,
1037+ mock_remove_older_than,
1038+ error_path_exists,
1039+ ):
1040+ """Verify remove_older_than action."""
1041+ action_args = ["actions/remove_older_than"]
1042+ mock_exists.return_value = error_path_exists
1043+ actions.ACTIONS["remove_older_than"] = mock_remove_older_than
1044+ actions.main(action_args)
1045+ mock_remove_older_than.assert_called_with(action_args)
1046+ assert mock_remove.called == error_path_exists
1047+
1048+ @pytest.mark.parametrize("error_path_exists", [True, False])
1049+ @patch("actions.remove_all_but_n_full")
1050+ @patch("actions.os.path.exists")
1051+ @patch("actions.os.remove")
1052+ def test_remove_all_but_n_full_run_success(
1053+ self,
1054+ mock_remove,
1055+ mock_exists,
1056+ mock_remove_all_but_n_full,
1057+ error_path_exists,
1058+ ):
1059+ """Verify remove_all_but_n_full action."""
1060+ action_args = ["actions/remove_all_but_n_full"]
1061+ mock_exists.return_value = error_path_exists
1062+ actions.ACTIONS["remove_all_but_n_full"] = mock_remove_all_but_n_full
1063+ actions.main(action_args)
1064+ mock_remove_all_but_n_full.assert_called_with(action_args)
1065+ assert mock_remove.called == error_path_exists
1066+
1067+ @pytest.mark.parametrize("error_path_exists", [True, False])
1068+ @patch("actions.remove_all_inc_of_but_n_full")
1069+ @patch("actions.os.path.exists")
1070+ @patch("actions.os.remove")
1071+ def test_remove_all_inc_of_but_n_full_run_success(
1072+ self,
1073+ mock_remove,
1074+ mock_exists,
1075+ mock_remove_all_inc_of_but_n_full,
1076+ error_path_exists,
1077+ ):
1078+ """Verify remove_all_inc_of_but_n_full action."""
1079+ action_args = ["actions/remove_all_inc_of_but_n_full"]
1080+ mock_exists.return_value = error_path_exists
1081+ mock_temp = mock_remove_all_inc_of_but_n_full
1082+ actions.ACTIONS["remove_all_inc_of_but_n_full"] = mock_temp
1083+ actions.main(action_args)
1084+ mock_remove_all_inc_of_but_n_full.assert_called_with(action_args)
1085+ assert mock_remove.called == error_path_exists
1086+
1087 @patch("actions.do_backup")
1088 @patch("actions.clear_flag")
1089 @patch("actions.os.remove")
1090@@ -101,7 +159,7 @@ class TestActions:
1091 assert type(exception_raised) == type(e)
1092 for expected_contain in expected_fail_contains:
1093 assert expected_contain in str(e)
1094- mock_hookenv.function_fail.assert_called()
1095+ mock_hookenv.action_fail.assert_called()
1096 mock_remove.assert_not_called()
1097 mock_clear_flag.assert_not_called()
1098
1099@@ -118,7 +176,7 @@ class TestDoBackupAction:
1100 expected_dict_input = dict(output=result.decode("utf-8"))
1101 actions.do_backup()
1102 mock_helper.do_backup.assert_called_once()
1103- mock_hookenv.function_set.called_with(expected_dict_input)
1104+ mock_hookenv.action_set.called_with(expected_dict_input)
1105
1106
1107 class TestListCurrentFilesAction:
1108@@ -133,4 +191,49 @@ class TestListCurrentFilesAction:
1109 expected_dict_input = dict(output=result.decode("utf-8"))
1110 actions.list_current_files()
1111 mock_helper.list_current_files.assert_called_once()
1112- mock_hookenv.function_set.called_with(expected_dict_input)
1113+ mock_hookenv.action_set.called_with(expected_dict_input)
1114+
1115+
1116+class TestRemoveOlderThanAction:
1117+ """Verify remove-older-than action."""
1118+
1119+ @patch("actions.helper")
1120+ @patch("actions.hookenv")
1121+ def test_remove_older_than(self, mock_hookenv, mock_helper):
1122+ """Verify remove-older-than action."""
1123+ result = "action_output".encode("utf-8")
1124+ mock_helper.remove_older_than.return_value = result
1125+ expected_dict_input = dict(output=result.decode("utf-8"))
1126+ actions.remove_older_than()
1127+ mock_helper.remove_older_than.assert_called_once()
1128+ mock_hookenv.action_set.called_with(expected_dict_input)
1129+
1130+
1131+class TestRemoveAllButNFullAction:
1132+ """Verify remove-all-but-n-full action."""
1133+
1134+ @patch("actions.helper")
1135+ @patch("actions.hookenv")
1136+ def test_remove_all_but_n_full(self, mock_hookenv, mock_helper):
1137+ """Verify remove-all-but-n-full action."""
1138+ result = "action_output".encode("utf-8")
1139+ mock_helper.remove_all_but_n_full.return_value = result
1140+ expected_dict_input = dict(output=result.decode("utf-8"))
1141+ actions.remove_all_but_n_full()
1142+ mock_helper.remove_all_but_n_full.assert_called_once()
1143+ mock_hookenv.action_set.called_with(expected_dict_input)
1144+
1145+
1146+class TestRemoveAllIncOfButNFullAction:
1147+ """Verify remove-all-inc-of-but-n-full action."""
1148+
1149+ @patch("actions.helper")
1150+ @patch("actions.hookenv")
1151+ def test_remove_all_inc_of_but_n_full(self, mock_hookenv, mock_helper):
1152+ """Verify remove-all-inc-of-but-n-full action."""
1153+ result = "action_output".encode("utf-8")
1154+ mock_helper.remove_all_inc_of_but_n_full.return_value = result
1155+ expected_dict_input = dict(output=result.decode("utf-8"))
1156+ actions.remove_all_inc_of_but_n_full()
1157+ mock_helper.remove_all_inc_of_but_n_full.assert_called_once()
1158+ mock_hookenv.action_set.called_with(expected_dict_input)
1159diff --git a/src/tests/unit/test_lib_duplicity.py b/src/tests/unit/test_lib_duplicity.py
1160index 90edf7a..06375c2 100644
1161--- a/src/tests/unit/test_lib_duplicity.py
1162+++ b/src/tests/unit/test_lib_duplicity.py
1163@@ -22,15 +22,17 @@ class TestDuplicityHelper:
1164 ("else", "host", None),
1165 ],
1166 )
1167- def test_backup_cmd(
1168+ def test_run_cmd_backup_and_list(
1169 self, duplicity_helper, backend, remote_backup_url, expected_destination
1170 ):
1171- """Verify backup command."""
1172+ """Verify full-backup and list-current-files _run_cmd."""
1173 duplicity_helper.charm_config["backend"] = backend
1174 duplicity_helper.charm_config["remote_backup_url"] = remote_backup_url
1175- command = duplicity_helper.backup_cmd
1176- assert "duplicity" in command
1177+ command = duplicity_helper._build_cmd("full", "/tmp/duplicity")
1178+ assert expected_destination in command
1179 assert "/tmp/duplicity" in command
1180+ command = duplicity_helper._build_cmd("list-current-files")
1181+ assert "duplicity" in command
1182 assert expected_destination in command
1183
1184 @pytest.mark.parametrize(
1185@@ -45,17 +47,95 @@ class TestDuplicityHelper:
1186 ("else", "host", None),
1187 ],
1188 )
1189- def test_list_current_files_cmd(
1190- self, duplicity_helper, backend, remote_backup_url, expected_destination
1191+ @pytest.mark.parametrize(
1192+ "time",
1193+ [
1194+ ("now"),
1195+ (328974),
1196+ ("20220910T15:15:15+02:00"),
1197+ ("3D4S"),
1198+ ],
1199+ )
1200+ def test_run_cmd_remove_older_than(
1201+ self, duplicity_helper, backend, remote_backup_url, expected_destination, time
1202 ):
1203- """Verify list-current-files command."""
1204+ """Verify remove-older-than command through _run_cmd."""
1205 duplicity_helper.charm_config["backend"] = backend
1206 duplicity_helper.charm_config["remote_backup_url"] = remote_backup_url
1207- command = duplicity_helper.backup_cmd
1208+ command = duplicity_helper._build_cmd("remove-older-than", time)
1209+ assert "duplicity" in command
1210 assert expected_destination in command
1211- command = duplicity_helper.list_files_cmd
1212+ assert "--force" in command
1213+ assert str(time) in command
1214+ assert "remove-older-than" in command
1215+
1216+ @pytest.mark.parametrize(
1217+ "backend,remote_backup_url,expected_destination",
1218+ [
1219+ ("scp", "some.host/backups", "scp://some.host/backups/unit-mock-0"),
1220+ ("rsync", "some.host/backups", "rsync://some.host/backups/unit-mock-0"),
1221+ ("ftp", "some.host/backups", "ftp://some.host/backups/unit-mock-0"),
1222+ ("sftp", "some.host/backups", "sftp://some.host/backups/unit-mock-0"),
1223+ ("s3", "some.host/backups", "s3://some.host/backups/unit-mock-0"),
1224+ ("file", "some.host/backups", "file://some.host/backups/unit-mock-0"),
1225+ ("else", "host", None),
1226+ ],
1227+ )
1228+ @pytest.mark.parametrize(
1229+ "count",
1230+ [
1231+ (0),
1232+ (1),
1233+ (2204),
1234+ (999999999999),
1235+ ],
1236+ )
1237+ def test_run_cmd_remove_all_but_n_full(
1238+ self, duplicity_helper, backend, remote_backup_url, expected_destination, count
1239+ ):
1240+ """Verify remove-all-but-n-full command through _run_cmd."""
1241+ duplicity_helper.charm_config["backend"] = backend
1242+ duplicity_helper.charm_config["remote_backup_url"] = remote_backup_url
1243+ command = duplicity_helper._build_cmd("remove-all-but-n-full", count)
1244 assert "duplicity" in command
1245 assert expected_destination in command
1246+ assert "--force" in command
1247+ assert str(count) in command
1248+ assert "remove-all-but-n-full" in command
1249+
1250+ @pytest.mark.parametrize(
1251+ "backend,remote_backup_url,expected_destination",
1252+ [
1253+ ("scp", "some.host/backups", "scp://some.host/backups/unit-mock-0"),
1254+ ("rsync", "some.host/backups", "rsync://some.host/backups/unit-mock-0"),
1255+ ("ftp", "some.host/backups", "ftp://some.host/backups/unit-mock-0"),
1256+ ("sftp", "some.host/backups", "sftp://some.host/backups/unit-mock-0"),
1257+ ("s3", "some.host/backups", "s3://some.host/backups/unit-mock-0"),
1258+ ("file", "some.host/backups", "file://some.host/backups/unit-mock-0"),
1259+ ("else", "host", None),
1260+ ],
1261+ )
1262+ @pytest.mark.parametrize(
1263+ "count",
1264+ [
1265+ (0),
1266+ (1),
1267+ (2204),
1268+ (999999999999),
1269+ ],
1270+ )
1271+ def test_run_cmd_remove_all_inc_of_but_n_full(
1272+ self, duplicity_helper, backend, remote_backup_url, expected_destination, count
1273+ ):
1274+ """Verify remove-all-inc-of-but-n-full command through _run_cmd."""
1275+ duplicity_helper.charm_config["backend"] = backend
1276+ duplicity_helper.charm_config["remote_backup_url"] = remote_backup_url
1277+ command = duplicity_helper._build_cmd("remove-all-inc-of-but-n-full", count)
1278+ assert "duplicity" in command
1279+ assert expected_destination in command
1280+ assert "--force" in command
1281+ assert str(count) in command
1282+ assert "remove-all-inc-of-but-n-full" in command
1283
1284 @pytest.mark.parametrize(
1285 "user,password,expected_destination",
1286@@ -78,7 +158,7 @@ class TestDuplicityHelper:
1287 duplicity_helper.charm_config["remote_backup_url"] = remote_backup_url
1288 duplicity_helper.charm_config["remote_user"] = user
1289 duplicity_helper.charm_config["remote_password"] = password
1290- command = duplicity_helper.backup_cmd
1291+ command = duplicity_helper._build_cmd("full")
1292 assert expected_destination in command
1293
1294 @pytest.mark.parametrize(
1295@@ -124,7 +204,7 @@ class TestDuplicityHelper:
1296 duplicity_helper.charm_config["disable_encryption"] = disable_encryption
1297 duplicity_helper.charm_config["gpg_public_key"] = gpg_public_key
1298 duplicity_helper.charm_config["private_ssh_key"] = private_ssh_key
1299- command = duplicity_helper.backup_cmd
1300+ command = duplicity_helper._build_cmd("full")
1301 for expected_option in expected_options:
1302 assert expected_option in command
1303
1304@@ -134,7 +214,7 @@ class TestDuplicityHelper:
1305 )
1306 @patch("lib_duplicity.subprocess")
1307 @patch("lib_duplicity.os")
1308- def test_do_backup(
1309+ def test_executor(
1310 self,
1311 mock_os,
1312 mock_subprocess,
1313@@ -143,7 +223,7 @@ class TestDuplicityHelper:
1314 aws_access_key_id,
1315 encryption_passphrase,
1316 ):
1317- """Verify do backup action."""
1318+ """Verify executor function."""
1319 duplicity_helper.charm_config["aws_secret_access_key"] = aws_secret_access_key
1320 duplicity_helper.charm_config["aws_access_key_id"] = aws_access_key_id
1321 duplicity_helper.charm_config["encryption_passphrase"] = encryption_passphrase
1322@@ -156,6 +236,36 @@ class TestDuplicityHelper:
1323 ]
1324 mock_os.environ.__setitem__.assert_has_calls(calls, any_order=True)
1325
1326+ @patch("lib_duplicity.subprocess")
1327+ def test_do_backup(self, mock_subprocess, duplicity_helper):
1328+ """Verify do_backup action."""
1329+ duplicity_helper.do_backup()
1330+ mock_subprocess.check_output.assert_called_once()
1331+
1332+ @patch("lib_duplicity.subprocess")
1333+ def test_list_current_files(self, mock_subprocess, duplicity_helper):
1334+ """Verify list_current_files action."""
1335+ duplicity_helper.list_current_files()
1336+ mock_subprocess.check_output.assert_called_once()
1337+
1338+ @patch("lib_duplicity.subprocess")
1339+ def test_remove_older_than(self, mock_subprocess, duplicity_helper):
1340+ """Verify remove_older_than action."""
1341+ duplicity_helper.remove_older_than(time="now")
1342+ mock_subprocess.check_output.assert_called_once()
1343+
1344+ @patch("lib_duplicity.subprocess")
1345+ def test_remove_all_but_n_full(self, mock_subprocess, duplicity_helper):
1346+ """Verify remove_all_but_n_full action."""
1347+ duplicity_helper.remove_all_but_n_full(count=1)
1348+ mock_subprocess.check_output.assert_called_once()
1349+
1350+ @patch("lib_duplicity.subprocess")
1351+ def test_remove_all_inc_of_but_n_full(self, mock_subprocess, duplicity_helper):
1352+ """Verify remove_all_inc_of_but_n_full action."""
1353+ duplicity_helper.remove_all_inc_of_but_n_full(count=1)
1354+ mock_subprocess.check_output.assert_called_once()
1355+
1356 @pytest.mark.parametrize(
1357 "backup_frequency,expected_frequency",
1358 [
1359diff --git a/src/tox.ini b/src/tox.ini
1360index 25b8687..3b939b5 100644
1361--- a/src/tox.ini
1362+++ b/src/tox.ini
1363@@ -55,7 +55,10 @@ exclude =
1364
1365 max-line-length = 88
1366 max-complexity = 10
1367-ignore = E402 #TODO remove it.
1368+# multiple lines because of https://bugs.launchpad.net/charm-duplicity/+bug/1998094
1369+ignore =
1370+ #TODO remove it.
1371+ E402
1372
1373 [testenv:black]
1374 commands =

Subscribers

People subscribed via source and target branches

to all changes: