Merge ~dashmage/charm-logrotated:bug/2017798-update-cron-daily-location into charm-logrotated:master

Proposed by Ashley James
Status: Merged
Approved by: Eric Chen
Approved revision: 13acd7b2a53e0663f8fe35ee458bb25be4b6035e
Merged at revision: 10ab2f6b55b60798c08a40c4a86014c542a7bac1
Proposed branch: ~dashmage/charm-logrotated:bug/2017798-update-cron-daily-location
Merge into: charm-logrotated:master
Diff against target: 201 lines (+70/-25)
4 files modified
src/lib/lib_cron.py (+16/-14)
src/tests/functional/requirements.txt (+1/-0)
src/tests/functional/test_logrotate.py (+13/-5)
src/tests/unit/test_logrotate.py (+40/-6)
Reviewer Review Type Date Requested Status
JamesLin Approve
🤖 prod-jenkaas-bootstack (community) continuous-integration Approve
Chi Wai CHAN Needs Information
Eric Chen Approve
BootStack Reviewers Pending
BootStack Reviewers Pending
Robert Gildein Pending
Tianqi Xiao Pending
Review via email: mp+443242@code.launchpad.net

This proposal supersedes a proposal from 2023-05-09.

Commit message

Manage charm's daily cronjob from /etc/crontab instead of root user's crontab.

Description of the change

`update-cron-daily-schedule` currently updates the charm's daily cronjob with the root user crontab present in `/var/spool/cron/crontabs/root`. This change was performed so that the cron.daily script only runs from a single crontab file (/etc/crotab) and to fix `hookenv.log` which wasn't correctly executed when running with sudo user cronjob (messages did not show up in juju debug-log).

Changes performed by MP
-----------------------
- Use tenacity.retry while checking whether a file exists in the functional tests. This prevents failure in cases of performance issues on the CI server.

- In the `update_cron_daily_schedule` method -- prevent writing to root user's crontab. Instead create a separate function (write_to_crontab) where /etc/crontab daily cron job time is modified.

- Replace dangling `open` function calls with a context manager in `install_cronjob`.

- Fix existing unit tests in order mock the file and not write directly to the crontab of the system where it's being run.

- Add unit test for the new function `write_to_crontab`.

Fixes LP#2017798

To post a comment you must log in.
Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote : Posted in a previous version of this proposal

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

Revision history for this message
Ashley James (dashmage) wrote : Posted in a previous version of this proposal

I see one potential problem now - in an upgrade scenario, even after implementing this fix, the sudo user crontab would still exist if cron-update-daily-schedule was configured. This would mean that the script would still run twice.

I propose that we add a function that checks whether the `/var/spool/cron/crontabs/root` crontab file exists and remove it in case it is present since we are only managing through `/etc/crontab`.

Revision history for this message
Robert Gildein (rgildein) : Posted in a previous version of this proposal
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Ashley James (dashmage) wrote (last edit ): Posted in a previous version of this proposal

> Is this really wisely to overwritte system crontab? Why not put new file to /etc/cron.d/<name>?

Even if we add this as a new file to `/etc/cron.d/<name>`, the script (placed in cron.daily) would be run twice - once for `/etc/crontab` and then for `/etc/cron.d/<name>`.

In case we're modifying the time for running cron.daily scripts and want to add it as a separate file in `/etc/cron.d`, we would need to remove the line from `/etc/crontab`. I prefer we make the change directly in `/etc/crontab` itself for changing the time.

Edit: Just to clarify, the `update-cron-daily-schedule` will only change the time for running the cron.daily scripts in the /etc/crontab file.

Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Approve (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Robert Gildein (rgildein) : Posted in a previous version of this proposal
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Approve (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Robert Gildein (rgildein) wrote : Posted in a previous version of this proposal

LGTM, thanks

review: Approve
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Approve (continuous-integration)
Revision history for this message
Eric Chen (eric-chen) wrote : Posted in a previous version of this proposal

I am curious

1. who is the runner when this library being called. (Ubuntu? Juju?) why they have root permission.
2. what's the different between f.flush and f.seek.
3. The logic looks like a magic if we want to copy the temp file. Is there better way to avoid the magic flush() or seek() function? I prefer not to use some implictly ability of flush() or seek() here. I prefer to make it just like a human do here.

#1 create temp file
#2 write data and close
#3 copy file
#4 delete temp file

review: Needs Information
Revision history for this message
Ashley James (dashmage) wrote : Posted in a previous version of this proposal

> I am curious
>
> 1. who is the runner when this library being called. (Ubuntu? Juju?) why they
> have root permission.

After looking into this, the script is called in cron.daily (or weekly/monthly depending on config) and it runs as root. Here is the relevant part in the code [1]. Which is why the copy command works without "sudo". I propose we can remove the sudo while copying from the tmp file.

```
$ cat /etc/cron.daily/charm-logrotated
#!/bin/bash
/usr/bin/sudo /usr/bin/juju-run logrotated/1 "/var/lib/juju/agents/unit-logrotated-1/.venv/bin/python3 /var/lib/juju/agents/unit-logrotated-1/charm/lib/lib_cron.py"
```

[1]: https://git.launchpad.net/charm-logrotated/tree/src/lib/lib_cron.py#n66

> 2. what's the different between f.flush and f.seek.

f.seek() would make the file pointer return to zero so that we can read contents from the start of the file. f.flush() is used to clear the internal buffer and put contents into the file explicitly before f.close() or reaching the end of the context manager (where the file is automatically closed). In this situation we could also use f.flush and push the contents to the tmp file before we read it, I just chose to pick f.seek here.

> 3. The logic looks like a magic if we want to copy the temp file. Is there
> better way to avoid the magic flush() or seek() function? I prefer not to use
> some implictly ability of flush() or seek() here. I prefer to make it just
> like a human do here.
>
> #1 create temp file
> #2 write data and close
> #3 copy file
> #4 delete temp file

In case you suggest to avoid using seek/flush, we can do it this way to make it more like how a human would do it:

```
with open(r"/tmp/crontab","w") as tmp_crontab:
    tmp_crontab.write(data)

try:
    cmd = ["cp", "/tmp/crontab", "/etc/crontab"]
    subprocess.run(cmd, check=True)

except subprocess.CalledProcessError as exception:
    hookenv.log(
        f"Writing to system crontab failed with error: {exception}",
        level=hookenv.ERROR,
    )
    raise
os.remove("/tmp/crontab")
```

Revision history for this message
Eric Chen (eric-chen) : Posted in a previous version of this proposal
Revision history for this message
Eric Chen (eric-chen) wrote : Posted in a previous version of this proposal

Based on the description you provided in the document, f.flush() is more reasonable. There is no clue that f.seek() will flush the data into the disk. Anyway, I prefer we handle it like a human way unless there is any specific reason we must copy the file before we close the file handle.

review: Needs Fixing
Revision history for this message
Eric Chen (eric-chen) wrote : Posted in a previous version of this proposal

I'm so surprise that we will execute /var/lib/juju/agents/unit-logrotated-1/charm/lib/lib_cron.py from the cronjob. It is very unreasonable to me. I suppose that every library should be used during

- event handler under src/reactive
- juju actions under src/actions

Could you check the reason why we need to design like this?

BTW, update_cron_daily_schedule is not only used in /etc/cron.daily/charm-logrotated, it may be triggered from

src/actions -> update_cronjob -> cron.install_cronjob -> update_cron_daily_schedule

The runner of these two behaviors may be different. Could you also confirm it?

1. from cron.daily
2. from juju actions

review: Needs Information
Revision history for this message
Chi Wai CHAN (raychan96) wrote : Posted in a previous version of this proposal

LGTM

review: Approve
Revision history for this message
Eric Chen (eric-chen) wrote : Posted in a previous version of this proposal

Change the status to "Needs Fixing"

review: Needs Fixing
Revision history for this message
Ashley James (dashmage) wrote (last edit ): Posted in a previous version of this proposal

> I'm so surprise that we will execute /var/lib/juju/agents/unit-
> logrotated-1/charm/lib/lib_cron.py from the cronjob. It is very unreasonable
> to me. I suppose that every library should be used during
>
> - event handler under src/reactive
> - juju actions under src/actions
>
> Could you check the reason why we need to design like this?

After going through the code, this is what I could infer ->

The cron_lib.py code should be executed by the cronjob because the charm is used to rotate the logrotate.d files at the regular interval that is determined by the time configured in the cronjob. The event handlers under src/reactive defines install/config-changed hook to run logrotate.modify_configs once during that event but to have it run regularly by the charm it would need to be present in the cronjob so that the script can be invoked at the hourly/daily/monthly times. Are you suggesting that instead of invoking the library script from the cronjob, there's another way to trigger the charm functionality at the different intervals?

In my view - the juju action allows to install the cronjob/ update logrotate files manually by the operator if they don't want to wait for the cronjob trigger time.

======

> BTW, update_cron_daily_schedule is not only used in /etc/cron.daily/charm-
> logrotated, it may be triggered from
>
> src/actions -> update_cronjob -> cron.install_cronjob ->
> update_cron_daily_schedule
>
> The runner of these two behaviors may be different. Could you also confirm it?
>
> 1. from cron.daily
> 2. from juju actions

So even if this functionality is triggered from src/actions (with update_cronjob) or during execution of the install hook event handler or config changed handler, they utilize the same `CronHelper.install_cronjob` method which then runs update_cron_daily_schedule`. Therefore, the runner in both cases will be root since sudo is used in the crontab entry to call the library script.

cron.daily is set with the `logrotate-cronjob-frequency` config and is used both by the action and by the other event handlers of the charm to run the same `update_cron_daily_schedule` function.

Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Approve (continuous-integration)
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote : Posted in a previous version of this proposal
review: Approve (continuous-integration)
Revision history for this message
Tianqi Xiao (txiao) wrote : Posted in a previous version of this proposal

Have you considered using a mature library (e.g. python-crontab [1]) to interact with crontab? I'm a bit worried that there are some consequences we have not considered when directly overwriting the cron file.

[1]: https://pypi.org/project/python-crontab/

review: Needs Information
Revision history for this message
Eric Chen (eric-chen) wrote : Posted in a previous version of this proposal

I cannot fully understand your answer above. I want to know why we need to execute cron_lib.py regularly. But I didn't find the real reason in your explanation.

My major concern is

1. In general case, the library in charm is used under the juju framework. The design in logrotate breaks the common situation and we should know the real reason. Maybe the solution is not to copy the file, we should just avoid this behavior.

2. Sudo is a dangerous command, we should only use it when needed. We should also need to know how the executor gets the permission?

3. subprocess is always my last choice to handle the program. because it implies many assumptions of the environment. So I agree with Tianqi's comment, we should consider the mature library. Even when we copy the file, we should think about how to use shutil.move if possible. (In the previous commit, you didn't use shutil.move when there is no "sudo").

4. "Retry" is also something I don't like a lot. Please write down the reason why we cannot avoid it. If there is a bug that only happen with 30% possibility, it will be concealed by the "Retry" mechanism.

I suggest that you can write a spec to clarify all the things and purpose the proposal again.

review: Needs Fixing
Revision history for this message
Ashley James (dashmage) wrote : Posted in a previous version of this proposal

Problem
-------
Currently when the logrotate-cronjob-frequency config option is set to daily, then the configured value for update-cron-daily-schedule (eg: set,08:00) is used to update the job execution time for the logrotated cron.daily job. This is done using the root user’s crontab (with `sudo crontab -u root <file_with_new_crontab_contents>`). But this is in conflict with the job that already exists in the system crontab (/etc/crontab) which means that the cron.daily job would run twice. It also

Fixes LP#2017798 [1]

Proposal
--------
For daily cronjobs, create a new cron file in /etc/cron.d where the charm will be able to manage the execution time without messing with the system crontab (/etc/crontab) or the root user’s crontab. It also prevents the charm from running commands with sudo.

I’m proposing to add a new file to /etc/cron.d instead of modifying the cron.daily execution time directly in the system crontab (/etc/crontab) because this would affect the execution of other scripts in cron.daily.

Changes performed by MP
-----------------------
- Use tenacity.retry while checking whether a file exists in the functional tests. This prevents failure in cases of performance issues on the CI server.

- 2 new files are now managed by the charm --> `cronjob_daily_command_path` which has the juju-run command that executes the lib_cron.py script and is called from the cronjob. `cronjob_daily_file` is the file being added in /etc/cron.d.

- In the `update_cron_daily_schedule` method
    - Renames the function to something more appropriate (`create_cron_daily_job`) since we're creating a new cron job and not updating the cron.daily job.
    - Replaces updating root user crontab with creating a new cron file in /etc/cron.d for daily cronjob

- Replace dangling `open` function calls with a context manager in `install_cronjob`.

- `cleanup_cronjob_files` also cleans up the new daily cronjob files.

I'm still working on these:
- Fix existing unit tests in order to not write directly to the crontab of the system where it's being run. Mock the required files which are being accessed.
- Re-write unit test for `create_cron_daily_job` method.

Other Notes
------------
I couldn’t use the python-crontab [2] library in order to manage the crontab changes since it does not support the random execution time syntax (00~45 08~09). This feature is not supported by the cron package which is used by Ubuntu in the first place [3]. A separate bug report has been raised for this issue [4].

-------------

[1]: https://bugs.launchpad.net/charm-logrotated/+bug/2017798
[2]: https://pypi.org/project/python-crontab/
[3]: https://packages.ubuntu.com/jammy/cron
[4]: https://bugs.launchpad.net/charm-logrotated/+bug/2019990

Revision history for this message
Mert Kirpici (mertkirpici) wrote : Posted in a previous version of this proposal

I am a little bit late to the discussion here, but I think I found the reason for the self-installing cronjob design decision in this charm, that Eric, Ray, Ashley and I were wondering at different times.

Taking from the original bug report:

LP #1832296: Create functionality to manage logrotate files
https://bugs.launchpad.net/charm-logrotated/+bug/1832296

"""
- As some packages might change the logrotate configurations - a cronjob option needs to be defined, where it will constantly update the configurations with regards to the retention period defined
"""

So it is for making sure, our retention settings are taking precedence if any other package/operator makes changes to the /etc/logrotate.d/ files. While I am not sure that it would be the right place to do this, but i suppose we could move this functionality to the "update-status" hook, since it is fired periodically.

Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote :

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
Ashley James (dashmage) wrote :

I've updated the MP with an important change - reverting from adding a file in /etc/cron.d and instead updating `/etc/crontab` itself. This is because the charm would also need to manage the `/etc/cron.daily/logrotate` file and this would involve changing the system crontab. Since we can't move the logrotate cronjob (provided by the apt package) out from cron.daily, we need to directly modify the time schedule in `/etc/crontab`.

Another important observation is that the `lib_cron.install_cronjob` and `lib_logrotate.modify_configs` are called in 3 situations - install/config changed hook, running an action and from the cronjob itself. In all these three places, the user running the code is root.

Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Approve (continuous-integration)
Revision history for this message
Eric Chen (eric-chen) wrote :

Please provide the final result we write into /etc/crontab for the verificiation

There are also inline comment. Not big change.

review: Needs Fixing
Revision history for this message
Eric Chen (eric-chen) :
Revision history for this message
JamesLin (jneo8) wrote :

One inline comment.

review: Needs Fixing
Revision history for this message
Ashley James (dashmage) wrote (last edit ):

> ditto, 0o755 or 0o644 ? , 0x744 seems weird

Yeah, I agree. I've made this `0o755` now.

> It seems we add function argument `target` is just because we need to test it.
But the value will never change in real case.
I will suggest to remove it and mock the `open` syntax.

I've now modified the unit test so that it mocks the open context manager instead of relying on the `target` argument which would never change. Good suggestion!

> Please provide the final result we write into /etc/crontab for the verificiation

As part of the updated unit tests, the final results are now specified and verified.

Revision history for this message
Eric Chen (eric-chen) wrote :

LGTM

review: Approve
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Approve (continuous-integration)
Revision history for this message
Chi Wai CHAN (raychan96) wrote (last edit ):

Thanks for working on the fix.

After thinking this a bit, I am wondering why we are insist on using the default /etc/crontab and modify the cron.daily schedule, or copy the default /etc/crontab and modify the cron.daily schedule and put it under root's crontab. I think this is undesired because it will affect all cron jobs under /etc/cron.daily, not just the logrotate charm's cronjob.

One of the possible alternative is to just maintain our own cron job without using the default crontab.

For example, we can either 1. create a crontab for root user with only one entry, or 2. append an entry to default /etc/crontab without modifying original content

```
17 * * * * root path_to_charm_logrotate
```

This way, we can be sure that even if we add to root user's cronjob or /etc/crontab, it will not have big impacts. What do you think?

review: Needs Information
Revision history for this message
🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack) wrote :
review: Approve (continuous-integration)
Revision history for this message
Ashley James (dashmage) wrote :

> Thanks for working on the fix.
>
> After thinking this a bit, I am wondering why we are insist on using the
> default /etc/crontab and modify the cron.daily schedule, or copy the default
> /etc/crontab and modify the cron.daily schedule and put it under root's
> crontab. I think this is undesired because it will affect all cron jobs under
> /etc/cron.daily, not just the logrotate charm's cronjob.
>
> One of the possible alternative is to just maintain our own cron job without
> using the default crontab.
>
> For example, we can either 1. create a crontab for root user with only one
> entry, or 2. append an entry to default /etc/crontab without modifying
> original content
>
> ```
> 17 * * * * root path_to_charm_logrotate
> ```
>
> This way, we can be sure that even if we add to root user's cronjob or
> /etc/crontab, it will not have big impacts. What do you think?

Thank you for your comment!

So the requirement for using /etc/crontab is because the cron.daily job is used by the charm for managing the execution of `/etc/cron.daily/logrotate` along with charm's `/etc/cron.daily/charm-logrotate` file. This logrotate cronjob is installed by the apt package and therefore we can't modify its location (it will be overwritten). So when the charm modifies the daily cronjob time config, it also needs to modify it for the `logrotate` file. Which is why we would need to directly modify the existing cron.daily time.

Revision history for this message
JamesLin (jneo8) wrote :

Overall LGTM now.

review: Approve
Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote :

Change successfully merged at revision 10ab2f6b55b60798c08a40c4a86014c542a7bac1

Revision history for this message
Chi Wai CHAN (raychan96) wrote :

> > Thanks for working on the fix.
> >
> > After thinking this a bit, I am wondering why we are insist on using the
> > default /etc/crontab and modify the cron.daily schedule, or copy the default
> > /etc/crontab and modify the cron.daily schedule and put it under root's
> > crontab. I think this is undesired because it will affect all cron jobs
> under
> > /etc/cron.daily, not just the logrotate charm's cronjob.
> >
> > One of the possible alternative is to just maintain our own cron job without
> > using the default crontab.
> >
> > For example, we can either 1. create a crontab for root user with only one
> > entry, or 2. append an entry to default /etc/crontab without modifying
> > original content
> >
> > ```
> > 17 * * * * root path_to_charm_logrotate
> > ```
> >
> > This way, we can be sure that even if we add to root user's cronjob or
> > /etc/crontab, it will not have big impacts. What do you think?
>
> Thank you for your comment!
>
> So the requirement for using /etc/crontab is because the cron.daily job is
> used by the charm for managing the execution of `/etc/cron.daily/logrotate`
> along with charm's `/etc/cron.daily/charm-logrotate` file. This logrotate
> cronjob is installed by the apt package and therefore we can't modify its
> location (it will be overwritten). So when the charm modifies the daily
> cronjob time config, it also needs to modify it for the `logrotate` file.
> Which is why we would need to directly modify the existing cron.daily time.

That's a good observation! and it's exactly the why I mentioned "it will affect all cron jobs
under /etc/cron.daily". The question is do we really want to change the `logrotate` script provided by the apt package? I guess is not because from the config.yaml, it appears to me that the "update-cron-daily-schedule" is designed for "logrotate-cronjob" and "logrotate-cronjob-frequency", and that's related to `charm-logrotate` script not the `logrotate` script.

And what's more is that the logic in "lib_cron.py" is weird / incorrect: when "logrotate-cronjob" is enabled, it creates a cronjob that runs "lib_cron.py", and "lib_cron.py" will create a cronjob that runs "lib_cron.py" and update logrotate config file. And the cycle keep going, so that's a logic loop.

That being said, the logic is already there before you create this MR, so it's not on you. If it's possible, we should raise a bug to refactor the code's logic first, before attempting a new features.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/src/lib/lib_cron.py b/src/lib/lib_cron.py
index 1f30ddc..f2383cc 100644
--- a/src/lib/lib_cron.py
+++ b/src/lib/lib_cron.py
@@ -1,7 +1,6 @@
1"""Cron helper module."""1"""Cron helper module."""
2import os2import os
3import re3import re
4import subprocess
54
6from charmhelpers.core import hookenv5from charmhelpers.core import hookenv
76
@@ -62,15 +61,14 @@ class CronHelper:
62 logrotate_unit = hookenv.local_unit()61 logrotate_unit = hookenv.local_unit()
63 python_venv_path = os.getcwd().replace("charm", "") + ".venv/bin/python3"62 python_venv_path = os.getcwd().replace("charm", "") + ".venv/bin/python3"
64 # upgrade to template if logic increases63 # upgrade to template if logic increases
65 cron_file = open(cron_file_path, "w")
66 cron_job = """#!/bin/bash64 cron_job = """#!/bin/bash
67/usr/bin/sudo /usr/bin/juju-run {} "{} {}"65/usr/bin/sudo /usr/bin/juju-run {} "{} {}"
68""".format(66""".format(
69 logrotate_unit, python_venv_path, cronjob_path67 logrotate_unit, python_venv_path, cronjob_path
70 )68 )
71 cron_file.write(cron_job)69 with open(cron_file_path, "w") as cron_file:
72 cron_file.close()70 cron_file.write(cron_job)
73 os.chmod(cron_file_path, 700)71 os.chmod(cron_file_path, 0o755)
7472
75 # update cron.daily schedule if logrotate-cronjob-frequency set to "daily"73 # update cron.daily schedule if logrotate-cronjob-frequency set to "daily"
76 if self.cronjob_frequency == 1 and self.validate_cron_conf():74 if self.cronjob_frequency == 1 and self.validate_cron_conf():
@@ -123,7 +121,13 @@ class CronHelper:
123 else:121 else:
124 raise RuntimeError("Unknown daily schedule type: {}".format(schedule_type))122 raise RuntimeError("Unknown daily schedule type: {}".format(schedule_type))
125123
126 cron_daily_timestamp = cron_daily_minute + " " + cron_daily_hour + "\t"124 cron_daily_timestamp = cron_daily_minute + " " + cron_daily_hour
125 self.write_to_crontab(cron_daily_timestamp)
126
127 return cron_daily_timestamp
128
129 def write_to_crontab(self, cron_daily_timestamp):
130 """Write daily cronjob with provided timestamp to /etc/crontab."""
127 cron_pattern = re.compile(r".*\/etc\/cron.daily.*")131 cron_pattern = re.compile(r".*\/etc\/cron.daily.*")
128 with open(r"/etc/crontab", "r") as crontab:132 with open(r"/etc/crontab", "r") as crontab:
129 data = crontab.read()133 data = crontab.read()
@@ -132,15 +136,13 @@ class CronHelper:
132 updated_cron_daily = ""136 updated_cron_daily = ""
133 if cron_daily:137 if cron_daily:
134 updated_cron_daily = re.sub(138 updated_cron_daily = re.sub(
135 r"\d?\d(~\d\d)?\s\d?\d(~\d\d)?\t", cron_daily_timestamp, cron_daily[0]139 r"\d?\d(~\d\d)?\s\d?\d(~\d\d)?\t",
140 cron_daily_timestamp + "\t",
141 cron_daily[0],
136 )142 )
137 data = data.replace(cron_daily[0], updated_cron_daily)143 updated_data = data.replace(cron_daily[0], updated_cron_daily)
138 with open(r"/tmp/crontab", "w") as crontab:144 with open(r"/etc/crontab", "w") as crontab:
139 crontab.write(data)145 crontab.write(updated_data)
140
141 subprocess.check_output(["sudo", "crontab", "-u", "root", "/tmp/crontab"])
142
143 return updated_cron_daily
144146
145 def validate_cron_conf(self):147 def validate_cron_conf(self):
146 """Block the unit and exit the hook if there is invalid configuration."""148 """Block the unit and exit the hook if there is invalid configuration."""
diff --git a/src/tests/functional/requirements.txt b/src/tests/functional/requirements.txt
index 6c4de97..4128a5d 100644
--- a/src/tests/functional/requirements.txt
+++ b/src/tests/functional/requirements.txt
@@ -7,3 +7,4 @@ mock
7pytest7pytest
8pytest-asyncio8pytest-asyncio
9requests9requests
10tenacity
diff --git a/src/tests/functional/test_logrotate.py b/src/tests/functional/test_logrotate.py
index 0afbb82..7088371 100644
--- a/src/tests/functional/test_logrotate.py
+++ b/src/tests/functional/test_logrotate.py
@@ -9,6 +9,8 @@ import pytest
99
10import pytest_asyncio10import pytest_asyncio
1111
12import tenacity
13
12pytestmark = pytest.mark.asyncio14pytestmark = pytest.mark.asyncio
13SERIES = ["bionic", "focal", "jammy"]15SERIES = ["bionic", "focal", "jammy"]
1416
@@ -118,10 +120,17 @@ async def test_reconfigure_cronjob_frequency(model, deploy_app, unit, jujutools)
118 await model.block_until(lambda: deploy_app.status == "active")120 await model.block_until(lambda: deploy_app.status == "active")
119 config = await deploy_app.get_config()121 config = await deploy_app.get_config()
120122
121 result = await jujutools.run_command(123 # Retry because test fails sometimes when cron file isn't ready yet
122 "test -f /etc/cron.weekly/charm-logrotate", unit124 for attempt in tenacity.Retrying(
123 )125 stop=tenacity.stop_after_attempt(3),
124 weekly_cronjob_exists = result["return-code"] == 0126 wait=tenacity.wait_exponential(multiplier=1, min=2, max=10),
127 ):
128 with attempt:
129 result = await jujutools.run_command(
130 "test -f /etc/cron.weekly/charm-logrotate", unit
131 )
132 weekly_cronjob_exists = result["return-code"] == 0
133 assert weekly_cronjob_exists
125134
126 result = await jujutools.run_command(135 result = await jujutools.run_command(
127 "test -f /etc/cron.daily/charm-logrotate", unit136 "test -f /etc/cron.daily/charm-logrotate", unit
@@ -130,7 +139,6 @@ async def test_reconfigure_cronjob_frequency(model, deploy_app, unit, jujutools)
130139
131 assert config["logrotate-cronjob-frequency"]["value"] == "weekly"140 assert config["logrotate-cronjob-frequency"]["value"] == "weekly"
132 assert not daily_cronjob_exists141 assert not daily_cronjob_exists
133 assert weekly_cronjob_exists
134142
135143
136async def test_configure_override_01(model, deploy_app, jujutools, unit):144async def test_configure_override_01(model, deploy_app, jujutools, unit):
diff --git a/src/tests/unit/test_logrotate.py b/src/tests/unit/test_logrotate.py
index 8d329e5..3af2c13 100644
--- a/src/tests/unit/test_logrotate.py
+++ b/src/tests/unit/test_logrotate.py
@@ -245,18 +245,54 @@ class TestCronHelper:
245 ("unset", "25 6"),245 ("unset", "25 6"),
246 ],246 ],
247 )247 )
248 def test_cron_daily_schedule(self, cron, cron_schedule, exp_pattern):248 def test_cron_daily_schedule(self, cron, cron_schedule, exp_pattern, mocker):
249 """Test the validate and update random cron.daily schedule."""249 """Test the validate and update random cron.daily schedule."""
250 cron_config = cron()250 cron_config = cron()
251 cron_config.cronjob_enabled = True251 cron_config.cronjob_enabled = True
252 cron_config.cronjob_frequency = 1252 cron_config.cronjob_frequency = 1
253 cron_config.cron_daily_schedule = cron_schedule253 cron_config.cron_daily_schedule = cron_schedule
254254
255 assert cron_config.validate_cron_conf()255 mock_method = mocker.Mock()
256 mocker.patch.object(cron, "write_to_crontab", new=mock_method)
256257
257 updated_cron_daily = cron_config.update_cron_daily_schedule()258 updated_cron_daily = cron_config.update_cron_daily_schedule()
258259
260 assert cron_config.validate_cron_conf()
259 assert updated_cron_daily.split("\t")[0] == exp_pattern261 assert updated_cron_daily.split("\t")[0] == exp_pattern
262 mock_method.assert_called_once_with(exp_pattern)
263
264 @pytest.mark.parametrize("cron_daily_timestamp", ["00 08", "25 6", "00~50 06~07"])
265 def test_write_to_crontab(self, cron, cron_daily_timestamp, mocker):
266 """Test function that writes updated data to /etc/crontab."""
267 cron_config = cron()
268 mock_open = mocker.patch("lib_cron.open")
269 mock_handle = mock_open.return_value.__enter__.return_value
270 default_crontab_contents = dedent(
271 """
272 # some comment
273 17 *\t* * * root cd / && run-parts --report /etc/cron.hourly
274 25 6\t* * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
275 47 6\t* * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
276 52 6\t1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
277 #
278 """ # noqa
279 )
280 updated_crontab_contents = dedent(
281 f"""
282 # some comment
283 17 *\t* * * root cd / && run-parts --report /etc/cron.hourly
284 {cron_daily_timestamp}\t* * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
285 47 6\t* * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
286 52 6\t1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
287 #
288 """ # noqa
289 )
290 mock_handle.read.return_value = default_crontab_contents
291 cron_config.write_to_crontab(cron_daily_timestamp)
292
293 mock_open.assert_any_call("/etc/crontab", "r")
294 mock_open.assert_any_call("/etc/crontab", "w")
295 mock_handle.write.assert_called_with(updated_crontab_contents)
260296
261 @pytest.mark.parametrize(297 @pytest.mark.parametrize(
262 ("cron_schedule"),298 ("cron_schedule"),
@@ -293,8 +329,7 @@ class TestCronHelper:
293 )329 )
294 mocker.patch("lib_cron.os.getcwd", return_value=mock_charm_dir)330 mocker.patch("lib_cron.os.getcwd", return_value=mock_charm_dir)
295 mock_open = mocker.patch("lib_cron.open")331 mock_open = mocker.patch("lib_cron.open")
296 mock_handle = mock_open.return_value332 mock_handle = mock_open.return_value.__enter__.return_value
297
298 expected_files_to_be_removed = [333 expected_files_to_be_removed = [
299 "/etc/cron.hourly/charm-logrotate",334 "/etc/cron.hourly/charm-logrotate",
300 "/etc/cron.daily/charm-logrotate",335 "/etc/cron.daily/charm-logrotate",
@@ -322,8 +357,7 @@ class TestCronHelper:
322 """ # noqa357 """ # noqa
323 )358 )
324 )359 )
325 mock_handle.close.assert_called_once()360 mock_chmod.assert_called_once_with("/etc/cron.weekly/charm-logrotate", 0o755)
326 mock_chmod.assert_called_once_with("/etc/cron.weekly/charm-logrotate", 700)
327361
328 def test_install_cronjob_removes_etc_config_when_cronjob_disabled(362 def test_install_cronjob_removes_etc_config_when_cronjob_disabled(
329 self, cron, mocker363 self, cron, mocker

Subscribers

People subscribed via source and target branches

to all changes: