Merge lp:~jelmer/bzr-builddeb/parallel-test into lp:bzr-builddeb

[Jelmer Vernooij]

Enable parallel running of the tests during package build if enabled using DEB_BUILD_OPTIONS=parallel=X. This makes quite a bit of a difference on quad-core machines.

[James Westby (james-w)]

What does --parallel=fork do if BZR_CONCURRENCY is unset?

Thanks,

James

[Robert Collins (lifeless)]

On Sun, Feb 27, 2011 at 8:33 AM, James Westby <email address hidden> wrote:
> What does --parallel=fork do if BZR_CONCURRENCY is unset?

probes the machine for available cores. E.g. on a 4-core machine, runs
4 workers.

-Rob

[Jelmer Vernooij (jelmer)]

Hmm, perhaps we should force it to BZR_CONCURRENCY=1 if parallel isn't in DEB_BUILD_OPTIONS?

[Robert Collins (lifeless)]

On Sun, Feb 27, 2011 at 11:07 AM, Jelmer Vernooij <email address hidden> wrote:
> Hmm, perhaps we should force it to BZR_CONCURRENCY=1 if parallel isn't in DEB_BUILD_OPTIONS?

why? That would be slower than just not passing --parallel in. And
unlike make -j, bzr won't spawn until death, it creates precisely the
number of workers as there are cpus.

[James Westby (james-w)]

On Sat, 26 Feb 2011 22:17:08 -0000, Robert Collins <email address hidden> wrote:
> On Sun, Feb 27, 2011 at 11:07 AM, Jelmer Vernooij <email address hidden> wrote:
> > Hmm, perhaps we should force it to BZR_CONCURRENCY=1 if parallel isn't in DEB_BUILD_OPTIONS?
>
> why? That would be slower than just not passing --parallel in. And
> unlike make -j, bzr won't spawn until death, it creates precisely the
> number of workers as there are cpus.

Well, don't pass --parallel in that case then.

I would think that the implied meaning of not passing parallel=n would
be parallel=1.

Thanks,

James

[Robert Collins (lifeless)]

On Sun, Feb 27, 2011 at 11:32 AM, James Westby
<email address hidden> wrote:
> On Sat, 26 Feb 2011 22:17:08 -0000, Robert Collins <email address hidden> wrote:
>> On Sun, Feb 27, 2011 at 11:07 AM, Jelmer Vernooij <email address hidden> wrote:
>> > Hmm, perhaps we should force it to BZR_CONCURRENCY=1 if parallel isn't in DEB_BUILD_OPTIONS?
>>
>> why? That would be slower than just not passing --parallel in. And
>> unlike make -j, bzr won't spawn until death, it creates precisely the
>> number of workers as there are cpus.
>
> Well, don't pass --parallel in that case then.
>
> I would think that the implied meaning of not passing parallel=n would
> be parallel=1.

I don't recall debian policy saying anything specific. Anyhow, as I
say, bzr handles default parallelism much better than make and many
other similar tools.

-Rob

[James Westby (james-w)]

To be clear, I'm happy to merge this if the lack of parallel= gives
a non-parallel run.

Thanks,

James

review: Needs Fixing

[Jelmer Vernooij (jelmer)]

On Mon, 2011-02-28 at 16:37 +0000, James Westby wrote:
> Review: Needs Fixing
> To be clear, I'm happy to merge this if the lack of parallel= gives
> a non-parallel run.
Thanks. I've pushed a new revision that only specifies --parallel and
BZR_CONCURRENCY if parallel was set in DEB_BUILD_OPTIONS.

Cheers,

Jelmer

[lp:~jelmer/bzr-builddeb/parallel-test]

528. By Jelmer Vernooij on 2011-02-28

Only use --parallel=fork if DEB_BUILD_OPTIONS had parallel= set.

[James Westby (james-w)]

review: Approve

=== modified file 'debian/changelog'
--- debian/changelog	2011-02-27 11:16:51 +0000
+++ debian/changelog	2011-02-28 17:21:27 +0000
@@ -1,4 +1,4 @@
-bzr-builddeb (2.7) UNRELEASED; urgency=low
+bzr-builddeb (2.7) unstable; urgency=low
 
   [ James Westby ]
 
@@ -34,6 +34,7 @@
     changelog and larstiq mode. LP: #619295
   * The deb: directory service has been renamed to apt: to avoid
     confusion.
+<<<<<<< TREE
   * When building with distribution set to UNRELEASED, it is now assumed
     the build is targetted at the same distribution as the build before.
 
@@ -42,6 +43,11 @@
   * Add devscripts to Build-Depends-Indep, the test suite runs dch.
 
  -- Jelmer Vernooij <[e-mail]>  Sun, 27 Feb 2011 03:08:09 +0100
+=======
+  * Support running the test suite in parallel during package build.
+
+ -- Jelmer Vernooij <[e-mail]>  Sat, 26 Feb 2011 15:45:12 +0100
+>>>>>>> MERGE-SOURCE
 
 bzr-builddeb (2.6) unstable; urgency=low
 

=== modified file 'debian/control'
--- debian/control	2011-02-26 18:41:42 +0000
+++ debian/control	2011-02-28 17:21:27 +0000
@@ -4,7 +4,11 @@
 Maintainer: Debian Bazaar Maintainers <[e-mail]>
 Uploaders: Reinhard Tartler <[e-mail]>, James Westby <[e-mail]>, Jelmer Vernooij <[e-mail]>
 Build-Depends: debhelper (>= 5.0.37.2), python-all (>= 2.3.5-11)
+<<<<<<< TREE
 Build-Depends-Indep: bzr (>= 2.1~), python-central (>= 0.5.8), python-docutils, python-debian (>= 0.1.11), python-apt, patchutils, python-testtools (>= 0.9.2), pristine-tar, devscripts
+=======
+Build-Depends-Indep: bzr (>= 2.1~), python-central (>= 0.5.8), python-docutils, python-debian (>= 0.1.11), python-apt, patchutils, python-testtools (>= 0.9.2), pristine-tar, python-subunit
+>>>>>>> MERGE-SOURCE
 Vcs-Bzr: http://bzr.debian.org/pkg-bazaar/bzr-builddeb/unstable
 Vcs-Browser: http://bzr.debian.org/loggerhead/pkg-bazaar/bzr-builddeb/unstable
 XS-Python-Version: >= 2.4

=== modified file 'debian/rules'
--- debian/rules	2011-02-03 13:38:09 +0000
+++ debian/rules	2011-02-28 17:21:27 +0000
@@ -19,10 +19,19 @@
 	rst2html README > README.html
 	touch $@
 
+ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
+SELFTEST_CONCURRENCY = BZR_CONCURRENCY=$(patsubst parallel=%,%,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
+SELFTEST_PARALLEL = --parallel=fork
+else
+SELFTEST_CONCURRENCY =
+SELFTEST_PARALLEL =
+endif
+
 build-python%:
 	python$* setup.py build
 ifeq (,$(filter nocheck,$(DEB_BUILD_OPTIONS)))
-	BZR_PLUGINS_AT=builddeb@$(CURDIR) python$* /usr/bin/bzr selftest -s bp.builddeb
+	$(SELFTEST_CONCURRENCY) BZR_PLUGINS_AT=builddeb@$(CURDIR) python$* /usr/bin/bzr selftest \
+	    $(SELFTEST_PARALLEL) -v -s bp.builddeb
 endif
 	touch $@-stamp