[MIR] run-one

Bug #888770 reported by Dustin Kirkland 
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
run-one (Ubuntu)
Fix Released
Medium
Didier Roche-Tolomelli

Bug Description

This is a Main Inclusion Report for the run-one source and binary package, per:
 * https://wiki.ubuntu.com/UbuntuMainInclusionRequirements

1. Availability: run-one is already in Ubuntu universe, and builds one package (shell code) for all architectures:
 * https://launchpad.net/ubuntu/+source/run-one

2. Rationale: run-one is a very useful, simple set of best practices around process locking. It's particularly useful in conjunction with long-running cronjobs, which might overrun one another. It would be nice if Ubuntu's cron packages recommended run-one. I need run-one to solve a couple of problems in the Byobu package (in Main). As such I'd like to depend or recommend run-one to do so, such that I don't have to duplicate this lock handling code there.

3. Security: There have been no security issues to date with the run-one utilities. I am the active maintainer of both the upstream source and the Ubuntu packaging, and I commit to active maintenance of any security issues that are raise. The run-one code base is less than 100 lines of shell code, and I would welcome an audit from the Security Team, if they would like to do so. The project is more or less "complete", and I don't expect for it to undergo much change over time.

4. Quality Assurance: The package is functional upon installation. It's quite easy to use, right out of the box. It's tremendously useful in conjunction with cronjobs. It does not ask any debconf questions. It has no open bugs in Launchpad.
 * https://bugs.launchpad.net/ubuntu/+source/run-one

5. UI Standards: Text based application. Very little in terms of interface to consider.

6. Dependencies: All dependencies (procps) are in Main already.

7. Maintenance: Packaging is clean and simple. Very simple installation layout, FHS-clean.

8. Background Information: None that applies here.

Changed in run-one (Ubuntu):
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
Dave Walker (davewalker) wrote :

@Dustin, Please can you provide some information on what run-one does that cronutils doesn't?

Thanks.

Revision history for this message
Dustin Kirkland  (kirkland) wrote : Re: [Bug 888770] Re: [MIR] run-one

Sure, Dave.

See:
 * http://blog.dustinkirkland.com/2011/02/introducing-run-one-and-run-this-one.html

Basically, imagine having a long running cronjob, like an rsync, that
might take an hour or more sometimes. But you want to run this
cronjob every, say 10 minutes. something like:
 */10 * * * * rsync -a foo:/bar bar:/foo

If that rsync takes longer than 10 minutes, you'll end up with a bunch
of rsync processes over time.

Instead, you can use the run-one utility like so:
 */10 * * * * run-one rsync -a foo:/bar bar:/foo

This will ensure that no more than one copy of that rsync is running
as this user at any time.

Alternatively, the run-this-one utility will kill any previous
existing copies, and run *this* one, which might be useful for killing
a hung process.
 */10 * * * * run-this-one rsync -a foo:/bar bar:/foo

Thanks,
Dustin

Revision history for this message
Dave Walker (davewalker) wrote :

@Dustin,

I understand the problem that this MIR is trying to resolve, so a how-to wasn't what I was asking. :)

I was really asking, why run-one is superior to the tools cronutils provide?:

runlock: Prevent concurrent runs of a process.
       runlock tries to hold an exclusive lock while it executes a command.
       Subsequent execution of runlock with the same lock, while that lock is
       held, will cause the new instance of runlock to terminate with a
       faliure exit code. Otherwise, the exit code of the subprocess is
       returned.
http://manpages.ubuntu.com/manpages/precise/en/man1/runlock.1.html

runalarm - enforce a time limit on execution of a process
       runalarm tries to execute a command and, if the subprocess does not
       exit before a timer expires, tries to terminate that subprocess.
       Otherwise, the exit status of the command is returned.
http://manpages.ubuntu.com/manpages/precise/en/man1/runalarm.1.html

Some stat information:
       runstat tries to execute a command in a subprocess, and upon
       termination of the subprocess collects some statics about that
       subrocess, and writes them to a file.

       These statistics include time of execution, exit status, and elapsed
       time to execute, as well as use of various OS resources.
http://manpages.ubuntu.com/manpages/precise/en/man1/runstat.1.html

... The syslog integration seems to fit the Orchestra model quite nicely - particularly posting runalarm warnings.

It *doesn't* have the run-this-one functionality, with runalarm being the nearest option. It seems to me that this is actually a superior suggested workflow of resolving this problem, as I fear people will routinely be happy to essentially "killall foo ; foo" as part of a cron task.

Thoughts?

Revision history for this message
Dave Walker (davewalker) wrote :

Dustin, did you see my question?

Thanks.

Changed in run-one (Ubuntu):
status: Triaged → Incomplete
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

runlock is insufficient, because it only takes into account the process name, and not the arguments.

For instance, try this in terminal 1:
 $ runlock sleep 100

And in terminal 2:
 $ runlock sleep 200

Your second terminal's runlock will terminate with an error condition because the "sleep" process is running (nevermind that you're running a different screen). Imagine something like "rsync", where you might have several different backup jobs that use rsync, each with their own arguments.

Instead, try this with run-one, terminal 1:
 $ run-one sleep 100

And terminal 2:
 $ run-one sleep 200

Both of these are allowed to run, because they're unique commands with different arguments.

Open a third terminal and try to 'run-one sleep 100', and you'll see that it terminates, because one of those specifically is already running.

I don't see anything like 'run-this-one' in cronutils. Again, in terminal 3 try 'run-this-one sleep 200'. It should kill the other one, and run THIS one.

And keep-one-running is different from nohup (in case you're wondering), because keep-one-running will restart the command (and parameters) again, in ANY case of exit (zero, non-zero, as well as interrupt signals). nohup just ignores signals.

Changed in run-one (Ubuntu):
status: Incomplete → New
Michael Terry (mterry)
Changed in run-one (Ubuntu):
assignee: nobody → Didier Roche (didrocks)
Revision history for this message
Didier Roche-Tolomelli (didrocks) wrote :

looked at the package and the code, all looks fine for promoting in main.
I would like to request that debian/copyright follows DEP-5 (see http://dep.debian.net/deps/dep5/)

Just a small note, looking from the code, it's quite clear that the matching is done in a unique set of argument. You are telling it in the man page, but I would really emphase it</suggestion> ;)

However, I would like a clear conscientious with the server team before putting it to main, Dave, can you have a look again? Thanks!

Changed in run-one (Ubuntu):
status: New → Incomplete
Revision history for this message
Didier Roche-Tolomelli (didrocks) wrote :

(please reassign to myself when there is an answer from Dave/Dustin)

Changed in run-one (Ubuntu):
assignee: Didier Roche (didrocks) → nobody
Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for run-one (Ubuntu) because there has been no activity for 60 days.]

Changed in run-one (Ubuntu):
status: Incomplete → Expired
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Hi Didier,

I completely lost track of this. I have reformatted the copyright to DEP-5, and release/uploaded run-one version 1.15, which should be building in Trusty shortly. Thanks!

Dustin

Changed in run-one (Ubuntu):
status: Expired → In Progress
assignee: nobody → Didier Roche (didrocks)
Revision history for this message
Didier Roche-Tolomelli (didrocks) wrote :

Hey Dustin, can you grab someone from the server team to ack this change? All the rest looks ok to me, just want to have the consensus with them. Please just reassign to me once done and I'll just +1 ;)

Changed in run-one (Ubuntu):
assignee: Didier Roche (didrocks) → Dustin Kirkland  (kirkland)
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

Thanks, @didrocks.

I have subscribed @jamespage, and he says he'll take a look.

Changed in run-one (Ubuntu):
assignee: Dustin Kirkland  (kirkland) → James Page (james-page)
Revision history for this message
James Page (james-page) wrote :

Re-assigning to Didier post ack from server-team.

Cheers

James

Changed in run-one (Ubuntu):
status: In Progress → Confirmed
assignee: James Page (james-page) → Didier Roche (didrocks)
Revision history for this message
Didier Roche-Tolomelli (didrocks) wrote :

Great!

Approved then. Dustin, just ping me back once byobu is depending on it and I'll just promote it.

Changed in run-one (Ubuntu):
status: Confirmed → Fix Committed
Revision history for this message
Dustin Kirkland  (kirkland) wrote :

@didrocks,

[ubuntu/trusty-proposed] byobu 5.65-0ubuntu1 (Accepted)

Thanks!
:-Dustin

On Mon, Nov 18, 2013 at 12:43 PM, Launchpad Bug Tracker
<email address hidden> wrote:
> ** Branch linked: lp:ubuntu/trusty-proposed/byobu
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/888770
>
> Title:
> [MIR] run-one
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/run-one/+bug/888770/+subscriptions

Revision history for this message
Didier Roche-Tolomelli (didrocks) wrote :

$ ./change-override -c main -S run-one
Override component to main
run-one 1.16-0ubuntu1 in trusty: universe/admin -> main
run-one 1.16-0ubuntu1 in trusty amd64: universe/admin/optional/100% -> main
run-one 1.16-0ubuntu1 in trusty arm64: universe/admin/optional/100% -> main
run-one 1.16-0ubuntu1 in trusty armhf: universe/admin/optional/100% -> main
run-one 1.16-0ubuntu1 in trusty i386: universe/admin/optional/100% -> main
run-one 1.16-0ubuntu1 in trusty powerpc: universe/admin/optional/100% -> main
Override [y|N]? y
6 publications overridden.

Changed in run-one (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.