Merge lp:~vila/bzr/move-test-servers into lp:bzr
| Status: | Merged |
|---|---|
| Approved by: | John A Meinel on 2010-02-10 |
| Approved revision: | not available |
| Merged at revision: | not available |
| Proposed branch: | lp:~vila/bzr/move-test-servers |
| Merge into: | lp:bzr |
| Diff against target: |
2151 lines (+544/-488) 49 files modified
NEWS (+7/-0) bzrlib/builtins.py (+5/-4) bzrlib/smart/server.py (+0/-85) bzrlib/tests/__init__.py (+18/-13) bzrlib/tests/blackbox/test_branch.py (+2/-2) bzrlib/tests/blackbox/test_cat.py (+2/-1) bzrlib/tests/blackbox/test_info.py (+1/-0) bzrlib/tests/blackbox/test_push.py (+1/-0) bzrlib/tests/blackbox/test_selftest.py (+2/-4) bzrlib/tests/blackbox/test_send.py (+2/-1) bzrlib/tests/ftp_server/medusa_based.py (+4/-4) bzrlib/tests/ftp_server/pyftpdlib_based.py (+4/-4) bzrlib/tests/http_server.py (+4/-2) bzrlib/tests/per_branch/__init__.py (+8/-13) bzrlib/tests/per_branch/test_bound_sftp.py (+4/-4) bzrlib/tests/per_branch/test_branch.py (+2/-2) bzrlib/tests/per_branch/test_hooks.py (+3/-4) bzrlib/tests/per_branch/test_http.py (+5/-6) bzrlib/tests/per_branch/test_push.py (+6/-4) bzrlib/tests/per_bzrdir/__init__.py (+12/-17) bzrlib/tests/per_interbranch/test_push.py (+2/-2) bzrlib/tests/per_pack_repository.py (+5/-6) bzrlib/tests/per_repository/__init__.py (+12/-17) bzrlib/tests/per_repository/test_repository.py (+6/-5) bzrlib/tests/per_repository/test_write_group.py (+6/-3) bzrlib/tests/per_transport.py (+2/-2) bzrlib/tests/stub_sftp.py (+4/-4) bzrlib/tests/test_bzrdir.py (+7/-5) bzrlib/tests/test_remote.py (+8/-7) bzrlib/tests/test_repository.py (+0/-1) bzrlib/tests/test_selftest.py (+11/-13) bzrlib/tests/test_server.py (+312/-0) bzrlib/tests/test_smart.py (+6/-6) bzrlib/tests/test_transport.py (+5/-2) bzrlib/transport/__init__.py (+1/-28) bzrlib/transport/brokenrename.py (+4/-11) bzrlib/transport/chroot.py (+2/-16) bzrlib/transport/decorator.py (+0/-43) bzrlib/transport/fakenfs.py (+6/-13) bzrlib/transport/fakevfat.py (+4/-14) bzrlib/transport/local.py (+4/-20) bzrlib/transport/log.py (+6/-20) bzrlib/transport/memory.py (+10/-9) bzrlib/transport/nosmart.py (+6/-12) bzrlib/transport/pathfilter.py (+2/-17) bzrlib/transport/readonly.py (+6/-14) bzrlib/transport/remote.py (+2/-2) bzrlib/transport/trace.py (+7/-13) bzrlib/transport/unlistable.py (+6/-13) |
| To merge this branch: | bzr merge lp:~vila/bzr/move-test-servers |
| Related bugs: |
| Reviewer | Review Type | Date Requested | Status |
|---|---|---|---|
| John A Meinel | 2010-02-09 | Approve on 2010-02-10 | |
|
Review via email:
|
|||
| Vincent Ladeuil (vila) wrote : | # |
| John A Meinel (jameinel) wrote : | # |
This change looks ok except:
1) I expect it will cause some significant fallout with Launchpad and possibly some plugins.
2) We should make sure to have a nice big NEWS entry about how to update your code. It looks mostly like just importing 'test_server' and using whatever server you were using previously. (I would expect MemoryServer to be the only one causing problems.)
I wonder if we should just leave in a deprecated thunk for MemoryServer...
3) We should probably try to coordinate with someone like MWH for (1).
| Andrew Bennetts (spiv) wrote : | # |
John A Meinel wrote:
> Review: Approve
> This change looks ok except:
>
> 1) I expect it will cause some significant fallout with Launchpad and possibly some plugins.
> 2) We should make sure to have a nice big NEWS entry about how to update your code. It looks mostly like just importing 'test_server' and using whatever server you were using previously. (I would expect MemoryServer to be the only one causing problems.)
>
> I wonder if we should just leave in a deprecated thunk for MemoryServer...
I think MemoryServer is useful for more than just testing, so I think it should
stay in bzrlib.
filesystem, e.g. Launchpad uses it in non-test code for the automatic bzrdirs
with stacking directives.
This implies to me that the Server base class should remain in bzrlib.transport
too.
| Robert Collins (lifeless) wrote : | # |
On Wed, 2010-02-10 at 23:51 +0000, Andrew Bennetts wrote:
>
> I think MemoryServer is useful for more than just testing, so I think
> it should
> stay in bzrlib.
> virtual
> filesystem, e.g. Launchpad uses it in non-test code for the automatic
> bzrdirs
> with stacking directives.
>
> This implies to me that the Server base class should remain in
> bzrlib.transport
> too.
I agree with this. I'll go further and say that *most* of our servers
are really quite useful for non-test code. I don't think they should
move under tests at all.
I think the following *should* move under tests:
- servers that are unsafe to use outside of tests
- servers that are deliberately broken (and thus their raison d'etre is
for testing)
And these *could* move under tests:
- the mapping of 'To test transport T use servers A, B and C'
-Rob
| Martin Pool (mbp) wrote : | # |
On 11 February 2010 12:48, Robert Collins <email address hidden> wrote:
> On Wed, 2010-02-10 at 23:51 +0000, Andrew Bennetts wrote:
>>
>> I think MemoryServer is useful for more than just testing, so I think
>> it should
>> stay in bzrlib.
>> virtual
>> filesystem, e.g. Launchpad uses it in non-test code for the automatic
>> bzrdirs
>> with stacking directives.
>>
>> This implies to me that the Server base class should remain in
>> bzrlib.transport
>> too.
>
> I agree with this. I'll go further and say that *most* of our servers
> are really quite useful for non-test code. I don't think they should
> move under tests at all.
>
> I think the following *should* move under tests:
> - servers that are unsafe to use outside of tests
> - servers that are deliberately broken (and thus their raison d'etre is
> for testing)
>
> And these *could* move under tests:
> - the mapping of 'To test transport T use servers A, B and C'
Perhaps we should move them all to bzrlib.servers. Then we can assert
that module is not loaded when it is not needed, and we can allow for
currently limited servers to grow into being more generally useful. I
think we have some intentionally limited transports in b.transport and
I don't see that as essentially problematic. It's more important that
the class name make its limitations clear.
--
Martin <http://
| Robert Collins (lifeless) wrote : | # |
On Thu, 2010-02-11 at 01:54 +0000, Martin Pool wrote:
>
>
> Perhaps we should move them all to bzrlib.servers. Then we can assert
> that module is not loaded when it is not needed, and we can allow for
> currently limited servers to grow into being more generally useful. I
> think we have some intentionally limited transports in b.transport and
> I don't see that as essentially problematic. It's more important that
> the class name make its limitations clear.
This sounds fine to me too.
Perhaps bzrlib.
-Rob
| Martin Pool (mbp) wrote : | # |
On 11 February 2010 13:42, Robert Collins <email address hidden> wrote:
> On Thu, 2010-02-11 at 01:54 +0000, Martin Pool wrote:
>>
>>
>> Perhaps we should move them all to bzrlib.servers. Then we can assert
>> that module is not loaded when it is not needed, and we can allow for
>> currently limited servers to grow into being more generally useful. I
>> think we have some intentionally limited transports in b.transport and
>> I don't see that as essentially problematic. It's more important that
>> the class name make its limitations clear.
>
> This sounds fine to me too.
>
> Perhaps bzrlib.
I'd slightly prefer .servers:
1- submodules seem to occasionally cause more annoying python import
behaviour (circularity etc) than sibling modules
2- bzrlib.
--
Martin <http://
| Robert Collins (lifeless) wrote : | # |
On Thu, 2010-02-11 at 02:51 +0000, Martin Pool wrote:
>
> > Perhaps bzrlib.
>
> I'd slightly prefer .servers:
>
> 1- submodules seem to occasionally cause more annoying python import
> behaviour (circularity etc) than sibling modules
Could you enlarge on this? I haven't seen that, unless you mean the
behaviour of 'import foo' in a submodule trying for
'parent.
This should be pretty fast after it is tried once, because after that
the dentry cache is hot.
> 2- bzrlib.
I proposed transport.servers because:
- They are specifically transport servers, not smart server servers, or
git servers, or CVS servers; and I wouldn't really want to see a git
server mixed in. I guess we could do servers.
equally long winded, and IMO less clear.
- I see the transport servers as being generally coupled to transports,
not coupled to bzrlib as a whole.
That said, its up to whomever proposes the patch :)
-Rob
| Martin Pool (mbp) wrote : | # |
On 11 February 2010 14:06, Robert Collins <email address hidden> wrote:
> On Thu, 2010-02-11 at 02:51 +0000, Martin Pool wrote:
>>
>> > Perhaps bzrlib.
>>
>> I'd slightly prefer .servers:
>>
>> 1- submodules seem to occasionally cause more annoying python import
>> behaviour (circularity etc) than sibling modules
>
> Could you enlarge on this? I haven't seen that, unless you mean the
> behaviour of 'import foo' in a submodule trying for
> 'parent.
> This should be pretty fast after it is tried once, because after that
> the dentry cache is hot.
You can't do 'import bzrlib.ui.text' in top-level code in 'bzrlib.ui'.
>> 2- bzrlib.
>
> I proposed transport.servers because:
> - They are specifically transport servers, not smart server servers, or
> git servers, or CVS servers; and I wouldn't really want to see a git
> server mixed in. I guess we could do servers.
> equally long winded, and IMO less clear.
Are they really? They are things that can be started, stopped, and
that have a URL. Their essentially connection to transports (as
opposed to the fact that they are only used for testing transports)
seems quite weak.
I would be ok with having bzrlib.server.ftp alongside
bzrlib.server.git for example.
> - I see the transport servers as being generally coupled to transports,
> not coupled to bzrlib as a whole.
>
> That said, its up to whomever proposes the patch :)
--
Martin <http://
| Robert Collins (lifeless) wrote : | # |
On Thu, 2010-02-11 at 03:45 +0000, Martin Pool wrote:
> On 11 February 2010 14:06, Robert Collins <email address hidden> wrote:
> > On Thu, 2010-02-11 at 02:51 +0000, Martin Pool wrote:
> >>
> >> > Perhaps bzrlib.
> >>
> >> I'd slightly prefer .servers:
> >>
> >> 1- submodules seem to occasionally cause more annoying python import
> >> behaviour (circularity etc) than sibling modules
> >
> > Could you enlarge on this? I haven't seen that, unless you mean the
> > behaviour of 'import foo' in a submodule trying for
> > 'parent.
> > This should be pretty fast after it is tried once, because after that
> > the dentry cache is hot.
>
> You can't do 'import bzrlib.ui.text' in top-level code in 'bzrlib.ui'.
Thats true. That doesn't seem to apply here though, does it? I don't
think we should shun all subpackages and submodules because that can
happen. In fact, its the case in all imports that you can't import from
within the currently importing package at the top level - this is why
circular imports are a problem. bzrlib.servers seems no more or less
inclined to suffer this than bzrlib.
case do we seem to have an a-priori need to expose things from within
the namespace.
> >> 2- bzrlib.
> >
> > I proposed transport.servers because:
> > - They are specifically transport servers, not smart server servers, or
> > git servers, or CVS servers; and I wouldn't really want to see a git
> > server mixed in. I guess we could do servers.
> > equally long winded, and IMO less clear.
>
> Are they really? They are things that can be started, stopped, and
> that have a URL. Their essentially connection to transports (as
> opposed to the fact that they are only used for testing transports)
> seems quite weak.
They are all servers for VFS's, rather than servers for non-FS-like file
systems.
-Rob
| Vincent Ladeuil (vila) wrote : | # |
Cough, so to clear up a few points:
- bzrlib.
- only chroot and pathfilter are really used as servers,
- there is still various grey areas around inheriting from
b.t.Server or just implementing {start|
get_url() is only used for tests,
- I expect most fallouts from LocalUrlServer and to a lesser
extent MemoryServer (if the bzr code base usage can serve
as a comparison point),
- given the above, only *tests* should be failing from there.
Also, I first started to deprecate b.t.Server and it helped me
find all obscure usages, but in the end I had to leave it alone to
separate "real" server usages (chroot and pathfilter) from true test
usages. And since a server is just implementing {start|
I already wondered if it wasn't already too much (or said otherwise
a small enough API to not even bother having a base class for it, which
is exactly what smart_server does).
About starting a bzrlib.
- YAGNI,
- a slight preference for bzrlib.servers when needed but so far,
I don't even consider our ftp and http servers like true servers.
Their intent is to be hacked on to be able to test any crazy
failure mode. As such, if there is ever a decision needed to be
made between making them easy to customize or making them faster,
I will chose the former.
I'll add a NEWS entry explaining the migration path (mostly,
as John said changing an import). Since the fix is trivial,
I don't think we need to make an exception for launchpad
(which may already have other API skews to fix).
Since it has been approved, I'll land that patch which is already
massive and we can improve from there.
| Vincent Ladeuil (vila) wrote : | # |
I forgot to mention that I've put all test servers into
bzrlib.
to not need their own dedicated server.
Semy-arbitrarily I left the ftp and http servers out of that
because they needed a bit more code.
The smart servers for testing for a bit between the two but
they needed to leave the bzrlib.transport hierarchy because
they were intended for *test*.
- 5062. By Vincent Ladeuil on 2010-02-11
-
Move MemoryServer back into bzrlib.
transport. memory as it's needed as soon as a MemoryTransport is used. Add a NEWS entry. - 5063. By Vincent Ladeuil on 2010-02-11
-
Update NEWS entry.
| John A Meinel (jameinel) wrote : | # |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
...
>>>> 2- bzrlib.
>>> I proposed transport.servers because:
>>> - They are specifically transport servers, not smart server servers, or
>>> git servers, or CVS servers; and I wouldn't really want to see a git
>>> server mixed in. I guess we could do servers.
>>> equally long winded, and IMO less clear.
>> Are they really? They are things that can be started, stopped, and
>> that have a URL. Their essentially connection to transports (as
>> opposed to the fact that they are only used for testing transports)
>> seems quite weak.
>
> They are all servers for VFS's, rather than servers for non-FS-like file
> systems.
>
> -Rob
>
Just to mention that the whole reason this was brought up at all, is
because the SftpServer implementation depended on something in
'bzrlib.tests' which caused a *runtime* dependency for sftp to require
having 'testtools' installed.
If these servers require 'bzrlib.tests' to be available, that certainly
seems like it belongs as 'test_servers'. (If you can't really use them
without loading the testing infrastructure.)
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://
iEYEARECAAYFAkt
MfYAnifbuaB2Yxa
=asMl
-----END PGP SIGNATURE-----
| John A Meinel (jameinel) wrote : | # |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
...
> I'll add a NEWS entry explaining the migration path (mostly,
> as John said changing an import). Since the fix is trivial,
> I don't think we need to make an exception for launchpad
> (which may already have other API skews to fix).
>
> Since it has been approved, I'll land that patch which is already
> massive and we can improve from there.
>
>
I would probably make sure to do something about MemoryServer, as that
is likely to actually impact 3rd-party code. The rest can be debated ad
nauseum.
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://
iEYEARECAAYFAkt
cVQAn1pgkeERuoC
=QGYJ
-----END PGP SIGNATURE-----
| Vincent Ladeuil (vila) wrote : | # |
>>>>> "jam" == John A Meinel <email address hidden> writes:
<snip/>
jam> I would probably make sure to do something about MemoryServer, as that
jam> is likely to actually impact 3rd-party code. The rest can be debated ad
jam> nauseum.
I fixed that before landing after discussing it on IRC.
So MemoryServer is still bzrlib.
you can't use a Memory Transport without it.
Vincent

This patch moves test servers out of the bzrlib.transport hierarchy test.test_ server and all related fallouts.
to bzrlib.
It spun off bug #516183 (testools required for sftp use) to make
sure we don't run into similar problems with other transports.
It's quite big but mostly mechanical.
Given that I relied on the test suite, I don't expect a lot of fallouts.
Some plugins may break if they relied on bzrlib. transport. Server but the fix
will be trivial anyway.
They should be fine if they use our ftp or http servers though.
This patch may be easier to review commit by commit rather than as a whole...