Merge lp:~mbp/bzr/715000-stacking into lp:bzr
- 715000-stacking
- Merge into bzr.dev
Status: | Merged | ||||
---|---|---|---|---|---|
Merged at revision: | 5657 | ||||
Proposed branch: | lp:~mbp/bzr/715000-stacking | ||||
Merge into: | lp:bzr | ||||
Diff against target: |
243 lines (+146/-3) (has conflicts) 8 files modified
bzr (+4/-0) bzrlib/__init__.py (+4/-0) bzrlib/groupcompress.py (+2/-2) bzrlib/knit.py (+1/-1) bzrlib/tests/per_repository_reference/__init__.py (+1/-0) bzrlib/tests/per_repository_reference/test_graph.py (+45/-0) bzrlib/versionedfile.py (+13/-0) doc/en/release-notes/bzr-2.3.txt (+76/-0) Text conflict in bzr Text conflict in bzrlib/__init__.py Text conflict in doc/en/release-notes/bzr-2.3.txt |
||||
To merge this branch: | bzr merge lp:~mbp/bzr/715000-stacking | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Robert Collins (community) | Approve | ||
Review via email: mp+48886@code.launchpad.net |
Commit message
Description of the change
This tries to fix bug 715000 by making sure we get the graph from all fallback repositories.
This is done off lp:bzr/2.2 to keep our options open, but perhaps we should just merge it to 2.3 or 2.4 and deploy that to Launchpad.
Andrew Bennetts (spiv) wrote : | # |
Vincent Ladeuil (vila) wrote : | # |
Fwiw, merging up in 2.3 and trunk pass the whole test suite.
As mentioned by poolie in https:/
Note to reviewers: you can safely ignore the conflicts which are caused by the mp being based on 2.2 while targeting trunk.
John A Meinel (jameinel) wrote : | # |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 2/8/2011 3:26 AM, Andrew Bennetts wrote:
> Some quick comments (rather than a comprehensive review):
>
> 1. "for br in [branch_c]:" is a weird expression (why have a loop when there's exactly 1 item?)
>
> 2. this fix appears reasonable, but you only fix one method when there are many other uses of self._fallback_vfs in e.g. GroupCompressVe
I agree, I think he meant:
for br in [branch_a, branch_b, branch_c]:
Though I'm fine with a test of just branch_c, if we are just wanting to
test the transitivity of operations.
I wouldn't be surprised if there are some bugs in other places, but I
think things like "get_parent_map" already recurse via the public api,
rather than recursing to find the transitive fallbacks, and then
iterating. (so branch_
branch_
branch_
branch_
I sort of like the public api recursion approach more than the
_transitive_
RepoA.do_
only looks at its own private attributes, and a fallbacks public attributes.
transitive_
function is actually a public Repository api. (Which I think it should
be, and should be tested as such, with a per_repository implementation
test.)
Anyway, I'm fine with this as a fix for what we are currently
encountering. I think we can do an audit of the codebase for everywhere
that _fallback_vfs or _fallback_
that can be separate to Martin's patch.
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://
iEYEARECAAYFAk1
edEAniQplUIN6to
=HM+5
-----END PGP SIGNATURE-----
Martin Pool (mbp) wrote : | # |
On 8 February 2011 20:26, Andrew Bennetts <email address hidden> wrote:
> Some quick comments (rather than a comprehensive review):
>
> 1. "for br in [branch_c]:" is a weird expression (why have a loop when there's exactly 1 item?)
Ah, for the sake of easier testing I had cut out the others. It
should test all of them.
> 2. this fix appears reasonable, but you only fix one method when there are many other uses of self._fallback_vfs in e.g. GroupCompressVe
I was hoping someone would tell me :-)
More seriously, I do think all the others need to be checked too, and
also by searching the bug tracker we might find reports where similar
things are failing.
Martin Pool (mbp) wrote : | # |
On 9 February 2011 07:50, John Arbash Meinel <email address hidden> wrote:
> transitive_
> function is actually a public Repository api. (Which I think it should
> be, and should be tested as such, with a per_repository implementation
> test.)
I was going to recurse through the public API, and originally took
that approach, but the problem was that in this case the public API
returns a KnownGraph, and I couldn't see how to combine several of
those objects. Perhaps there is a way?
John A Meinel (jameinel) wrote : | # |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 2/8/2011 4:49 PM, Martin Pool wrote:
> On 9 February 2011 07:50, John Arbash Meinel <email address hidden> wrote:
>
>> transitive_
>> function is actually a public Repository api. (Which I think it should
>> be, and should be tested as such, with a per_repository implementation
>> test.)
>
> I was going to recurse through the public API, and originally took
> that approach, but the problem was that in this case the public API
> returns a KnownGraph, and I couldn't see how to combine several of
> those objects. Perhaps there is a way?
>
Not trivially. We have code to add new entries one-by-one, but that
isn't great.
If we wanted to tease this apart more, something that returns a dict,
and then wrapping that in a KnownGraph would be enough. Or expose more
of _find_ancestor (I don't remember the exact 'take one step' function
name).
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://
iEYEARECAAYFAk1
9moAoLpnFYmQ0Nn
=YQNy
-----END PGP SIGNATURE-----
John A Meinel (jameinel) wrote : | # |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 2/8/2011 4:46 PM, Martin Pool wrote:
> On 8 February 2011 20:26, Andrew Bennetts <email address hidden> wrote:
>> Some quick comments (rather than a comprehensive review):
>>
>> 1. "for br in [branch_c]:" is a weird expression (why have a loop when there's exactly 1 item?)
>
> Ah, for the sake of easier testing I had cut out the others. It
> should test all of them.
>
>> 2. this fix appears reasonable, but you only fix one method when there are many other uses of self._fallback_vfs in e.g. GroupCompressVe
>
> I was hoping someone would tell me :-)
>
> More seriously, I do think all the others need to be checked too, and
> also by searching the bug tracker we might find reports where similar
> things are failing.
>
I honestly haven't seen much failure of this. Mostly because stacking is
rare outside of launchpad, and stacking on stacked is a whole next-level
of unlikely. Since not even launchpad creates branches like this easily.
I wouldn't be surprised to have internal bugs that are latent. Doing a
quick grep doesn't show anything obvious.
_fallback_vfs is only mentioned in bzrlib/
(since those are the only VersionedFiles that actually support it, it
should make sense.)
Specific references in groupcompress:
get_known_
_get_parent_
_find_from_
keys(), public_api
in knit.py:
_logical_check(), public_api
get_known_
_get_parent_
_get_remaining_
get_sha1s(), public_api
insert_
boundary, and need to do extra sanity checks on the stream
iter_lines_
deprecated anyway
keys(), public_api
_ContentMapGene
_KnitAnnotator.
non-optimized code
In the end, what we could do is promote "self._
be a public attribute of VersionedFiles rather than hidden on the _index
object, and then it could be done via public api, rather than chaining
the transitive closure of fallback_vfs. That would allow us to
consistently go via public api. Albeit by using a slightly different api
that can chain.
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://
iEYEARECAAYFAk1
T9IAnAl/
=i3cD
-----END PGP SIGNATURE-----
Robert Collins (lifeless) wrote : | # |
This looks fine to me; I agree that renaming _fallback_vfs to _immediate_
Martin Pool (mbp) wrote : | # |
Since lp is running 2.2 and wants to deploy this on top of 2.2, and since people seem to agree this is safe (and the change is smaller than I suspected it might be), I'll merge it there.
Preview Diff
1 | === modified file 'bzr' |
2 | --- bzr 2011-02-08 13:56:49 +0000 |
3 | +++ bzr 2011-02-09 04:20:36 +0000 |
4 | @@ -23,7 +23,11 @@ |
5 | import warnings |
6 | |
7 | # update this on each release |
8 | +<<<<<<< TREE |
9 | _script_version = (2, 4, 0) |
10 | +======= |
11 | +_script_version = (2, 2, 5) |
12 | +>>>>>>> MERGE-SOURCE |
13 | |
14 | try: |
15 | version_info = sys.version_info |
16 | |
17 | === modified file 'bzrlib/__init__.py' |
18 | --- bzrlib/__init__.py 2011-02-08 13:56:49 +0000 |
19 | +++ bzrlib/__init__.py 2011-02-09 04:20:36 +0000 |
20 | @@ -52,7 +52,11 @@ |
21 | # Python version 2.0 is (2, 0, 0, 'final', 0)." Additionally we use a |
22 | # releaselevel of 'dev' for unreleased under-development code. |
23 | |
24 | +<<<<<<< TREE |
25 | version_info = (2, 4, 0, 'dev', 1) |
26 | +======= |
27 | +version_info = (2, 2, 5, 'dev', 0) |
28 | +>>>>>>> MERGE-SOURCE |
29 | |
30 | # API compatibility version |
31 | api_minimum_version = (2, 4, 0) |
32 | |
33 | === modified file 'bzrlib/groupcompress.py' |
34 | --- bzrlib/groupcompress.py 2010-09-26 20:45:28 +0000 |
35 | +++ bzrlib/groupcompress.py 2011-02-09 04:20:36 +0000 |
36 | @@ -1,4 +1,4 @@ |
37 | -# Copyright (C) 2008, 2009, 2010 Canonical Ltd |
38 | +# Copyright (C) 2008-2011 Canonical Ltd |
39 | # |
40 | # This program is free software; you can redistribute it and/or modify |
41 | # it under the terms of the GNU General Public License as published by |
42 | @@ -1312,7 +1312,7 @@ |
43 | # KnitVersionedFiles.get_known_graph_ancestry, but they don't share |
44 | # ancestry. |
45 | parent_map, missing_keys = self._index.find_ancestry(keys) |
46 | - for fallback in self._fallback_vfs: |
47 | + for fallback in self._transitive_fallbacks(): |
48 | if not missing_keys: |
49 | break |
50 | (f_parent_map, f_missing_keys) = fallback._index.find_ancestry( |
51 | |
52 | === modified file 'bzrlib/knit.py' |
53 | --- bzrlib/knit.py 2011-01-11 20:42:36 +0000 |
54 | +++ bzrlib/knit.py 2011-02-09 04:20:36 +0000 |
55 | @@ -1196,7 +1196,7 @@ |
56 | def get_known_graph_ancestry(self, keys): |
57 | """Get a KnownGraph instance with the ancestry of keys.""" |
58 | parent_map, missing_keys = self._index.find_ancestry(keys) |
59 | - for fallback in self._fallback_vfs: |
60 | + for fallback in self._transitive_fallbacks(): |
61 | if not missing_keys: |
62 | break |
63 | (f_parent_map, f_missing_keys) = fallback._index.find_ancestry( |
64 | |
65 | === modified file 'bzrlib/tests/per_repository_reference/__init__.py' |
66 | --- bzrlib/tests/per_repository_reference/__init__.py 2011-01-10 22:47:59 +0000 |
67 | +++ bzrlib/tests/per_repository_reference/__init__.py 2011-02-09 04:20:36 +0000 |
68 | @@ -105,6 +105,7 @@ |
69 | 'bzrlib.tests.per_repository_reference.test_fetch', |
70 | 'bzrlib.tests.per_repository_reference.test_get_record_stream', |
71 | 'bzrlib.tests.per_repository_reference.test_get_rev_id_for_revno', |
72 | + 'bzrlib.tests.per_repository_reference.test_graph', |
73 | 'bzrlib.tests.per_repository_reference.test_initialize', |
74 | 'bzrlib.tests.per_repository_reference.test_unlock', |
75 | ] |
76 | |
77 | === added file 'bzrlib/tests/per_repository_reference/test_graph.py' |
78 | --- bzrlib/tests/per_repository_reference/test_graph.py 1970-01-01 00:00:00 +0000 |
79 | +++ bzrlib/tests/per_repository_reference/test_graph.py 2011-02-09 04:20:36 +0000 |
80 | @@ -0,0 +1,45 @@ |
81 | +# Copyright (C) 2011 Canonical Ltd |
82 | +# |
83 | +# This program is free software; you can redistribute it and/or modify |
84 | +# it under the terms of the GNU General Public License as published by |
85 | +# the Free Software Foundation; either version 2 of the License, or |
86 | +# (at your option) any later version. |
87 | +# |
88 | +# This program is distributed in the hope that it will be useful, |
89 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
90 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
91 | +# GNU General Public License for more details. |
92 | +# |
93 | +# You should have received a copy of the GNU General Public License |
94 | +# along with this program; if not, write to the Free Software |
95 | +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA |
96 | + |
97 | + |
98 | +"""Tests for graph operations on stacked repositories.""" |
99 | + |
100 | + |
101 | +from bzrlib.tests.per_repository import TestCaseWithRepository |
102 | + |
103 | + |
104 | +class TestGraph(TestCaseWithRepository): |
105 | + |
106 | + def test_get_known_graph_ancestry_stacked(self): |
107 | + """get_known_graph_ancestry works correctly on stacking. |
108 | + |
109 | + See <https://bugs.launchpad.net/bugs/715000>. |
110 | + """ |
111 | + branch_a, branch_b, branch_c, revid_1 = self.make_double_stacked_branches() |
112 | + for br in [branch_c]: |
113 | + self.assertEquals( |
114 | + [revid_1], |
115 | + br.repository.get_known_graph_ancestry([revid_1]).topo_sort()) |
116 | + |
117 | + def make_double_stacked_branches(self): |
118 | + wt_a = self.make_branch_and_tree('a') |
119 | + branch_a = wt_a.branch |
120 | + branch_b = self.make_branch('b') |
121 | + branch_b.set_stacked_on_url('../a') |
122 | + branch_c = self.make_branch('c') |
123 | + branch_c.set_stacked_on_url('../b') |
124 | + revid_1 = wt_a.commit('first commit') |
125 | + return branch_a, branch_b, branch_c, revid_1 |
126 | |
127 | === modified file 'bzrlib/versionedfile.py' |
128 | --- bzrlib/versionedfile.py 2010-08-28 15:34:24 +0000 |
129 | +++ bzrlib/versionedfile.py 2011-02-09 04:20:36 +0000 |
130 | @@ -1193,6 +1193,19 @@ |
131 | def _extract_blocks(self, version_id, source, target): |
132 | return None |
133 | |
134 | + def _transitive_fallbacks(self): |
135 | + """Return the whole stack of fallback versionedfiles. |
136 | + |
137 | + This VersionedFiles may have a list of fallbacks, but it doesn't |
138 | + necessarily know about the whole stack going down, and it can't know |
139 | + at open time because they may change after the objects are opened. |
140 | + """ |
141 | + all_fallbacks = [] |
142 | + for a_vfs in self._fallback_vfs: |
143 | + all_fallbacks.append(a_vfs) |
144 | + all_fallbacks.extend(a_vfs._transitive_fallbacks()) |
145 | + return all_fallbacks |
146 | + |
147 | |
148 | class ThunkedVersionedFiles(VersionedFiles): |
149 | """Storage for many versioned files thunked onto a 'VersionedFile' class. |
150 | |
151 | === modified file 'doc/en/release-notes/bzr-2.3.txt' |
152 | --- doc/en/release-notes/bzr-2.3.txt 2011-02-08 13:56:49 +0000 |
153 | +++ doc/en/release-notes/bzr-2.3.txt 2011-02-09 04:20:36 +0000 |
154 | @@ -2,6 +2,7 @@ |
155 | Bazaar Release Notes |
156 | #################### |
157 | |
158 | +<<<<<<< TREE |
159 | .. toctree:: |
160 | :maxdepth: 1 |
161 | |
162 | @@ -14,6 +15,81 @@ |
163 | ***************************** |
164 | |
165 | .. These may require users to change the way they use Bazaar. |
166 | +======= |
167 | +.. contents:: List of Releases |
168 | + :depth: 1 |
169 | + |
170 | +bzr 2.2.5 |
171 | +######### |
172 | + |
173 | +:Codename: Suggestions welcome |
174 | +:2.2.5: NOT RELEASED YET |
175 | + |
176 | +Compatibility Breaks |
177 | +******************** |
178 | + |
179 | +New Features |
180 | +************ |
181 | + |
182 | +Bug Fixes |
183 | +********* |
184 | + |
185 | +* Correctly handle ``bzr log`` and `get_known_graph_ancestry` on a |
186 | + doubly-stacked branch. |
187 | + (James Westby, Martin Pool, #715000) |
188 | + |
189 | +Improvements |
190 | +************ |
191 | + |
192 | +Documentation |
193 | +************* |
194 | + |
195 | +API Changes |
196 | +*********** |
197 | + |
198 | +Internals |
199 | +********* |
200 | + |
201 | +Testing |
202 | +******* |
203 | + |
204 | + |
205 | +bzr 2.2.4 |
206 | +######### |
207 | + |
208 | +:2.2.4: 2011-02-04 |
209 | + |
210 | +This is a bugfix release. Only one bug has been fixed, a regression from 2.2.3 |
211 | +involving only certain operations with launchpad. Upgrading is recommended for |
212 | +all users on earlier 2.2 releases. |
213 | + |
214 | +Bug Fixes |
215 | +********* |
216 | + |
217 | +* Fix communications with the Launchpad web service when using |
218 | + launchpadlib >= 1.5.5. This was a latent bug in bzr's communication |
219 | + with Launchpad's production instance, which only became a problem when |
220 | + the default instance was switched from edge to production in bzr 2.2.3. |
221 | + (Max Bowsher, #707075) |
222 | + |
223 | + |
224 | +bzr 2.2.3 |
225 | +######### |
226 | + |
227 | +:2.2.3: 2011-01-20 |
228 | + |
229 | +This is a bugfix release. Upgrading is recommended for all users |
230 | +on earlier 2.2 releases. |
231 | + |
232 | +Compatibility Breaks |
233 | +******************** |
234 | + |
235 | +* Launchpad has announced that the ``edge.launchpad.net`` instance is |
236 | + deprecated and may be shut down in the future |
237 | + <http://blog.launchpad.net/general/edge-is-deprecated>. Bazaar has therefore |
238 | + been updated in this release to talk to the main (``launchpad.net``) servers, |
239 | + rather than the ``edge`` ones. (Vincent Ladeuil, #583667) |
240 | +>>>>>>> MERGE-SOURCE |
241 | |
242 | New Features |
243 | ************ |
Some quick comments (rather than a comprehensive review):
1. "for br in [branch_c]:" is a weird expression (why have a loop when there's exactly 1 item?)
2. this fix appears reasonable, but you only fix one method when there are many other uses of self._fallback_vfs in e.g. GroupCompressVe rsionedFiles. Either there are more bugs waiting to be fixed here (hopefully easy ones to reveal with tests similar to the one you add in this patch), or the reason why only get_known_ graph_ancestry was broken is subtle/unclear. Do you know which? I'm a bit worried that this may correct the immediate traceback without actually making the Launchpad scanner functional because of further issues.