buildd is unnecessarily coupled to the launchpad tree through tac file and readyservice

This was mentioned in deploying a new build for bug 884997 and friends.

It looks like readyservice is barely used by the actual buildd code, and so perhaps it can be dispensed with. Alternatively, it's so small that perhaps it can just be copied in. This would be continuing from bug 663828.

The tac file seems like an implementation detail of the buildd that Launchpad itself shouldn't need to know about.

see also https://dev.launchpad.net/CreatingNewProjects for splitting out the buildds

copying it in would need care to prevent skew on the literal string
that test code looks for.

> copying it in would need care to prevent skew on the literal string
> that test code looks for.

That kind of thing is going to become interesting in general if/as lp
moves to having several separate trees and deployments. But in this
particular case it seems like I can just remove it...

The TacTestSetup depends on it to detect service ready when bringing
up the fixture....

The buildmaster tests rely on bringing up a slave. If we're going to separate launchpad-buildd from Launchpad, we need to either:
 - make it bring up the slave, by running a separately-installed slave
 - not have any code-level tests that rely on starting an actual slave, and count on testing that during integration
The former seems better.

So we possibly want a separate launchpad-buildd-fixtures package. For the moment I'll just try to get the tests to work in-tree, but not duplicated.

With the current <https://code.launchpad.net/~mbp/launchpad/800295-buildd-split/+merge/81111> the tac file is moved into the buildd directory, and nothing outside depends on it.

The buildd no longer depends on readyservice, and instead it just waits for the service to start listening, which is arguably cleaner anyhow. Perhaps I should update all the tachandler tests to do that.

The buildd tests still rely on tachandler, plus os.services.lputils, canonical.launchpad.ftests, canonical.testing, and it seems like at least test_generate_translation_templates relies on the lp test factory.

So at the moment this branch fixes the actual deployment/runtime dependency, but to run the buildd tests you still need to have a Launchpad tree, and also a Launchpad database and whole working environment.

Possibly the tests that do depend on having a full Launchpad ought to be moved outside of the buildd tree and then just depend on buildds to run.

I moved the test.

The other constraint as bigjools pointed out last night is that the buildd's own tests, which we probably would want to run without Launchpad, rely on TacTestSetup. It's not enormous but it's not trivial. It is fairly generic. Robert suggests making a new txfixtures which seems reasonable.

ok, https://launchpad.net/txfixtures now exists and the followon my changes the buildds to use that.

Fixed in stable r14248 (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/14248) by a commit, but not testable.

The incremental landing hasn't totally fixed this. It's a bit arbitrary where we draw the line but I'm going to say it's done when the buildd is cut out entirely.

https://code.launchpad.net/~mbp/launchpad/use-txfixtures/+merge/81243 changes lp to use external txfixtures.

lp:launchpad-buildd now has a copy of the current code, and you can build packages from it.

I think we could now delete it from the Launchpad tree, except then Launchpad wouldn't be able to run its integration tests for buildds. So instead this needs to become an external dependency of Launchpad. wgrant suggests putting it in sourcecode, which makes more sense than making it an egg.

I also need to make it actually possible to run 'make check' or similar within the buildd package, to test those things that can be done in isolation.

I ran a diff between the contents of the built packages after the change, because I'm concerned the deployment work the first time. The difference is that the python package has moved and that the tests are included (which is not necessary, but possibly useful.)

https://code.launchpad.net/~mbp/launchpad-buildd/rename-package/+merge/81692

FWIW I'd rather an egg than putting it in sourcecode: I want to nuke
sourcecode entirely.

I think the buildd code is not really "like" a python package and it
wouldn't for example make sense to have in pypi. I will see about
adding a dpkg dependency, since it's already packaged that way.

To add launchpad's dependency on this as a dpkg, we need it to be something that can reasonably be installed on developer machines, which probably means that just installing it should not cause the buildd to start. It will be started as a fixture by the tests that need it.

Possibly we should split out the buildd libraries in to a separate package from the 'actually acts as a buildd' task-like package.

Launchpad's tests do need to be able to start the buildd through its tac file.

http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html

For ease of getting at the module from other packages, the python module should probably move from /usr/share/launchpad-buildd to /usr/lib, and being installed by python-central. Though that may be hard if we want the same package to work across hardy..oneiric. It might not be urgent, as arguable launchpad is part of "the same suite of programs" and it can just know to look there.

 I need to make Launchpad search that path which I guess can be done from buildout and the makefile.

Fixed in stable r14311 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/14311>.

this hit some bumps in landing; it needs one more bit of dependency decoupling and then for the actual deletion to be resubmitted.