Use Twisted's thread support instead of the threading module in checkwatches
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| Launchpad itself |
Fix Released
|
High
|
Gavin Panella | ||
Bug Description
If only to get Twisted's foot in the door with regards to
checkwatches, the direct use of the threading module in checkwatches
should be replaced by Twisted.
As a first step, the same approach can be taken, which is to update
each bug tracker in a thread. This avoids converting the blocking
networking code in lp.bugs.
figure out a clear hand-off between the transactional database code
and the networking code.
However, by getting Twisted in there, and a reactor running, it does
open the door to later conversion, perhaps one bug tracker type at a
time.
The following two articles outline some approaches to running a
limited number of jobs in parallel with Twisted:
Async Batching with Twisted: A Walkthrough
http://
Limiting Parallelism
http://
Sprinkle some deferToThread into the above, et voila.
Related branches
- Eleanor Berger (community): Approve (code)
-
Diff: 516 lines (+266/-143)5 files modifiedlib/lp/bugs/doc/checkwatches-cli-switches.txt (+1/-1)
lib/lp/bugs/doc/externalbugtracker.txt (+0/-107)
lib/lp/bugs/scripts/checkwatches.py (+88/-32)
lib/lp/bugs/scripts/tests/test_checkwatches.py (+176/-3)
lib/lp/scripts/utilities/importfascist.py (+1/-0)
- Jonathan Lange (community): Approve
- Björn Tillenius (community): Abstain
-
Diff: 319 lines (+217/-15)4 files modifiedlib/lp/services/job/runner.py (+3/-2)
lib/lp/services/job/tests/test_runner.py (+15/-13)
lib/lp/testing/__init__.py (+68/-0)
lib/lp/testing/tests/test_zope_test_in_subprocess.py (+131/-0)
| tags: | added: bugwatch |
| Aaron Bentley (abentley) wrote Re: [Bug 491870] [NEW] Use Twisted's thread support instead of the threading module in checkwatches | #1 |
| tags: | added: story-reliable-bug-syncing |
| Changed in malone: | |
| assignee: | nobody → Gavin Panella (allenap) |
| status: | Triaged → In Progress |
| Ursula Junque (ursinha) wrote Bug fixed by a commit | #2 |
| Changed in malone: | |
| milestone: | none → 10.03 |
| status: | In Progress → Fix Committed |
| tags: | added: qa-needstesting |
| tags: |
added: qa-ok removed: qa-needstesting |
| Changed in malone: | |
| status: | Fix Committed → Fix Released |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Gavin Panella wrote:
> Public bug reported:
>
> If only to get Twisted's foot in the door with regards to
> checkwatches, the direct use of the threading module in checkwatches
> should be replaced by Twisted.
The code team is pursuing lp:ampoule, which another approach to
exploiting parallelism in Twisted, using a process pool. It might also
work here.
Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://
iEYEARECAAYFAks
t2YAn1gvISy5J+
=Eane
-----END PGP SIGNATURE-----