Status: | Needs review |
---|---|
Proposed branch: | lp:~jelmer/brz/depth |
Merge into: | lp:brz/3.3 |
Diff against target: |
766 lines (+185/-77) 16 files modified
breezy/branch.py (+9/-4) breezy/bzr/bzrdir.py (+5/-1) breezy/controldir.py (+2/-1) breezy/errors.py (+8/-0) breezy/git/branch.py (+17/-10) breezy/git/dir.py (+3/-2) breezy/git/fetch.py (+3/-1) breezy/git/interrepo.py (+53/-41) breezy/git/remote.py (+13/-11) breezy/git/repository.py (+1/-0) breezy/plugins/weave_fmt/bzrdir.py (+3/-1) breezy/repository.py (+4/-0) breezy/tests/per_controldir/test_controldir.py (+17/-2) breezy/tests/per_interbranch/test_fetch.py (+41/-1) breezy/tests/per_repository/test_repository.py (+3/-0) breezy/tests/test_foreign.py (+3/-2) |
To merge this branch: | bzr merge lp:~jelmer/brz/depth |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Jelmer Vernooij | Approve | ||
Martin Packman | Pending | ||
Review via email: mp+432254@code.launchpad.net |
This proposal supersedes a proposal from 2019-02-14.
Commit message
Add a depth argument to ControlDir.sprout() and Repository.fetch().
Description of the change
Add a depth argument to ControlDir.sprout() and Repository.fetch().
Martin Packman (gz) wrote : Posted in a previous version of this proposal | # |
Jelmer Vernooij (jelmer) : Posted in a previous version of this proposal | # |
Martin Packman (gz) wrote : Posted in a previous version of this proposal | # |
Restamping this one for clarity.
The Breezy Bot (the-breezy-bot) wrote : Posted in a previous version of this proposal | # |
Running landing tests failed
https:/
The Breezy Bot (the-breezy-bot) wrote : Posted in a previous version of this proposal | # |
Running landing tests failed
https:/
Jelmer Vernooij (jelmer) wrote : Posted in a previous version of this proposal | # |
This is blocked on support for depth between local repositories in Dulwich.
The Breezy Bot (the-breezy-bot) wrote : Posted in a previous version of this proposal | # |
Merging failed
https:/
The Breezy Bot (the-breezy-bot) wrote : Posted in a previous version of this proposal | # |
Merging failed
https:/
The Breezy Bot (the-breezy-bot) wrote : Posted in a previous version of this proposal | # |
Merging failed
https:/
The Breezy Bot (the-breezy-bot) wrote : | # |
The attempt to merge lp:~jelmer/brz/depth into lp:brz/3.3 failed. Command exited with 1.
Below is the output from the failed tests.
Collecting setuptools-gettext
Downloading setuptools_
Requirement already satisfied: setuptools>=46.1 in ./lib/python3.
Installing collected packages: setuptools-gettext
Successfully installed setuptools-
Obtaining file://
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Checking if build backend supports build_editable: started
Checking if build backend supports build_editable: finished with status 'done'
Getting requirements to build editable: started
Getting requirements to build editable: finished with status 'done'
Preparing editable metadata (pyproject.toml): started
Preparing editable metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: pyyaml in /usr/lib/
Requirement already satisfied: fastbencode in /usr/lib/
Requirement already satisfied: urllib3>=1.24.1 in /usr/lib/
Requirement already satisfied: dulwich>=0.20.46 in /usr/lib/
Requirement already satisfied: merge3 in /usr/lib/
Requirement already satisfied: configobj in /usr/lib/
Requirement already satisfied: patiencediff in /usr/lib/
Requirement already satisfied: gpg in /usr/lib/
Collecting sphinx
Downloading sphinx-
━━
Requirement already satisfied: setuptools in ./lib/python3.
Collecting sphinx-epytext
Downloading sphinx-
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting fastimport
Downloading fastimport-
━━
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: launchpadlib>=1.6.3 in /usr/lib/
Collecting testtools
Downloading testtools-
━━
Collecting python-subunit
Downloading python_
━━
Requirement already satisfied: httplib2 in /usr/lib/
Collecting keyring
Downloading keyring-
Unmerged revisions
- 7158. By Jelmer Vernooij
-
Merge lp:brz/3.3
- 7157. By Jelmer Vernooij
-
Merge trunk.
- 7156. By Jelmer Vernooij
-
Merge lp:brz/3.2.
- 7155. By Jelmer Vernooij
-
Merge lp:brz/3.1.
- 7154. By Jelmer Vernooij
-
Fix tests.
- 7153. By Jelmer Vernooij
-
add depth argument to ControlDir.sprout.
- 7152. By Jelmer Vernooij
-
Enable depth fetching for Git repositories.
- 7151. By Jelmer Vernooij
-
For the moment, Dulwich doesn't support depth fetching locally.
- 7150. By Jelmer Vernooij
-
Add Repository.
support_ fetch_depth. - 7149. By Jelmer Vernooij
-
Merge dulwich-compat.
Preview Diff
1 | === modified file 'breezy/branch.py' |
2 | --- breezy/branch.py 2022-09-12 13:37:25 +0000 |
3 | +++ breezy/branch.py 2022-10-27 12:03:56 +0000 |
4 | @@ -680,18 +680,20 @@ |
5 | raise errors.UpgradeRequired(self.user_url) |
6 | self.get_config_stack().set('append_revisions_only', enabled) |
7 | |
8 | - def fetch(self, from_branch, stop_revision=None, limit=None, lossy=False): |
9 | + def fetch(self, from_branch, stop_revision=None, limit=None, lossy=False, |
10 | + depth=None): |
11 | """Copy revisions from from_branch into this branch. |
12 | |
13 | :param from_branch: Where to copy from. |
14 | :param stop_revision: What revision to stop at (None for at the end |
15 | of the branch. |
16 | :param limit: Optional rough limit of revisions to fetch |
17 | + :param depth: Revision depth |
18 | :return: None |
19 | """ |
20 | with self.lock_write(): |
21 | return InterBranch.get(from_branch, self).fetch( |
22 | - stop_revision, limit=limit, lossy=lossy) |
23 | + stop_revision, limit=limit, lossy=lossy, depth=depth) |
24 | |
25 | def get_bound_location(self): |
26 | """Return the URL of the branch we are bound to. |
27 | @@ -2105,11 +2107,12 @@ |
28 | """ |
29 | raise NotImplementedError(self.copy_content_into) |
30 | |
31 | - def fetch(self, stop_revision=None, limit=None, lossy=False): |
32 | + def fetch(self, stop_revision=None, limit=None, lossy=False, depth=None): |
33 | """Fetch revisions. |
34 | |
35 | :param stop_revision: Last revision to fetch |
36 | :param limit: Optional rough limit of revisions to fetch |
37 | + :param depth: Optional revision depth |
38 | :return: FetchResult object |
39 | """ |
40 | raise NotImplementedError(self.fetch) |
41 | @@ -2167,9 +2170,11 @@ |
42 | if self.source._push_should_merge_tags(): |
43 | self.source.tags.merge_to(self.target.tags, selector=tag_selector) |
44 | |
45 | - def fetch(self, stop_revision=None, limit=None, lossy=False): |
46 | + def fetch(self, stop_revision=None, limit=None, lossy=False, depth=None): |
47 | if self.target.base == self.source.base: |
48 | return (0, []) |
49 | + if depth is not None: |
50 | + raise errors.FetchDepthUnsupported(self) |
51 | with self.source.lock_read(), self.target.lock_write(): |
52 | fetch_spec_factory = fetch.FetchSpecFactory() |
53 | fetch_spec_factory.source_branch = self.source |
54 | |
55 | === modified file 'breezy/bzr/bzrdir.py' |
56 | --- breezy/bzr/bzrdir.py 2022-08-22 18:19:46 +0000 |
57 | +++ breezy/bzr/bzrdir.py 2022-10-27 12:03:56 +0000 |
58 | @@ -352,7 +352,7 @@ |
59 | recurse='down', possible_transports=None, |
60 | accelerator_tree=None, hardlink=False, stacked=False, |
61 | source_branch=None, create_tree_if_local=True, |
62 | - lossy=False): |
63 | + depth=None): |
64 | """Create a copy of this controldir prepared for use as a new line of |
65 | development. |
66 | |
67 | @@ -376,8 +376,12 @@ |
68 | location of this control directory. |
69 | :param create_tree_if_local: If true, a working-tree will be created |
70 | when working locally. |
71 | + :param depth: Optional fetch depth |
72 | :return: The created control directory |
73 | """ |
74 | + if depth is not None: |
75 | + raise errors.FetchDepthUnsupported(self) |
76 | + |
77 | with contextlib.ExitStack() as stack: |
78 | fetch_spec_factory = fetch.FetchSpecFactory() |
79 | if revision_id is not None: |
80 | |
81 | === modified file 'breezy/controldir.py' |
82 | --- breezy/controldir.py 2022-09-12 18:35:44 +0000 |
83 | +++ breezy/controldir.py 2022-10-27 12:03:56 +0000 |
84 | @@ -414,7 +414,7 @@ |
85 | recurse='down', possible_transports=None, |
86 | accelerator_tree=None, hardlink=False, stacked=False, |
87 | source_branch=None, create_tree_if_local=True, |
88 | - lossy=False): |
89 | + depth=None): |
90 | """Create a copy of this controldir prepared for use as a new line of |
91 | development. |
92 | |
93 | @@ -438,6 +438,7 @@ |
94 | location of this control directory. |
95 | create_tree_if_local: If true, a working-tree will be created |
96 | when working locally. |
97 | + :param depth: Possible fetch depth |
98 | """ |
99 | raise NotImplementedError(self.sprout) |
100 | |
101 | |
102 | === modified file 'breezy/errors.py' |
103 | --- breezy/errors.py 2022-08-22 18:19:46 +0000 |
104 | +++ breezy/errors.py 2022-10-27 12:03:56 +0000 |
105 | @@ -1658,6 +1658,14 @@ |
106 | self.tname = type(method_self).__name__ |
107 | |
108 | |
109 | +class FetchDepthUnsupported(UnsupportedOperation): |
110 | + |
111 | + fmt = ("InterBranch %(interbranch)r does not support fetching depths.") |
112 | + |
113 | + def __init__(self, interbranch): |
114 | + BzrError.__init__(self, interbranch=interbranch) |
115 | + |
116 | + |
117 | class FetchLimitUnsupported(UnsupportedOperation): |
118 | |
119 | fmt = ("InterBranch %(interbranch)r does not support fetching limits.") |
120 | |
121 | === modified file 'breezy/git/branch.py' |
122 | --- breezy/git/branch.py 2022-08-28 18:54:11 +0000 |
123 | +++ breezy/git/branch.py 2022-10-27 12:03:56 +0000 |
124 | @@ -1007,12 +1007,15 @@ |
125 | return False |
126 | return True |
127 | |
128 | - def fetch(self, stop_revision=None, fetch_tags=None, limit=None, lossy=False): |
129 | + def fetch(self, stop_revision=None, fetch_tags=None, limit=None, |
130 | + lossy=False, depth=None): |
131 | self.fetch_objects( |
132 | - stop_revision, fetch_tags=fetch_tags, limit=limit, lossy=lossy) |
133 | + stop_revision, fetch_tags=fetch_tags, limit=limit, lossy=lossy, |
134 | + depth=depth) |
135 | return _mod_repository.FetchResult() |
136 | |
137 | - def fetch_objects(self, stop_revision, fetch_tags, limit=None, lossy=False, tag_selector=None): |
138 | + def fetch_objects(self, stop_revision, fetch_tags, limit=None, lossy=False, |
139 | + tag_selector=None, depth=None): |
140 | interrepo = self._get_interrepo(self.source, self.target) |
141 | if fetch_tags is None: |
142 | c = self.source.get_config_stack() |
143 | @@ -1033,7 +1036,7 @@ |
144 | [self._last_revid], include_tags=fetch_tags, tag_selector=tag_selector) |
145 | return real(heads) |
146 | pack_hint, head, refs = interrepo.fetch_objects( |
147 | - determine_wants, self.source.mapping, limit=limit, |
148 | + determine_wants, self.source.mapping, limit=limit, depth=depth, |
149 | lossy=lossy) |
150 | if (pack_hint is not None and |
151 | self.target.repository._format.pack_compresses): |
152 | @@ -1168,7 +1171,8 @@ |
153 | class InterGitBranch(branch.GenericInterBranch): |
154 | """InterBranch implementation that pulls between Git branches.""" |
155 | |
156 | - def fetch(self, stop_revision=None, fetch_tags=None, limit=None, lossy=False): |
157 | + def fetch(self, stop_revision=None, fetch_tags=None, limit=None, lossy=False, |
158 | + depth=None): |
159 | raise NotImplementedError(self.fetch) |
160 | |
161 | |
162 | @@ -1248,7 +1252,8 @@ |
163 | return (isinstance(source, GitBranch) and |
164 | isinstance(target, LocalGitBranch)) |
165 | |
166 | - def fetch(self, stop_revision=None, fetch_tags=None, limit=None, lossy=False): |
167 | + def fetch(self, stop_revision=None, fetch_tags=None, limit=None, lossy=False, |
168 | + depth=None): |
169 | interrepo = _mod_repository.InterRepository.get( |
170 | self.source.repository, self.target.repository) |
171 | if stop_revision is None: |
172 | @@ -1258,7 +1263,8 @@ |
173 | fetch_tags = c.get('branch.fetch_tags') |
174 | determine_wants = interrepo.get_determine_wants_revids( |
175 | [stop_revision], include_tags=fetch_tags) |
176 | - interrepo.fetch_objects(determine_wants, limit=limit, lossy=lossy) |
177 | + interrepo.fetch_objects( |
178 | + determine_wants, limit=limit, lossy=lossy, depth=depth) |
179 | return _mod_repository.FetchResult() |
180 | |
181 | def _basic_push(self, overwrite=False, stop_revision=None, tag_selector=None): |
182 | @@ -1455,7 +1461,7 @@ |
183 | return ret |
184 | |
185 | def fetch(self, stop_revision=None, fetch_tags=None, lossy=False, |
186 | - limit=None): |
187 | + limit=None, depth=None): |
188 | if stop_revision is None: |
189 | stop_revision = self.source.last_revision() |
190 | ret = [] |
191 | @@ -1465,7 +1471,8 @@ |
192 | ret.append((None, stop_revision)) |
193 | if getattr(self.interrepo, 'fetch_revs', None): |
194 | try: |
195 | - revidmap = self.interrepo.fetch_revs(ret, lossy=lossy, limit=limit) |
196 | + revidmap = self.interrepo.fetch_revs( |
197 | + ret, lossy=lossy, limit=limit, depth=depth) |
198 | except NoPushSupport: |
199 | raise errors.NoRoundtrippingSupport(self.source, self.target) |
200 | return _mod_repository.FetchResult(revidmap={ |
201 | @@ -1481,7 +1488,7 @@ |
202 | return wants |
203 | |
204 | self.interrepo.fetch_objects( |
205 | - determine_wants, lossy=lossy, limit=limit) |
206 | + determine_wants, lossy=lossy, limit=limit, depth=depth) |
207 | return _mod_repository.FetchResult() |
208 | |
209 | def pull(self, overwrite=False, stop_revision=None, local=False, |
210 | |
211 | === modified file 'breezy/git/dir.py' |
212 | --- breezy/git/dir.py 2022-08-22 18:19:46 +0000 |
213 | +++ breezy/git/dir.py 2022-10-27 12:03:56 +0000 |
214 | @@ -157,7 +157,7 @@ |
215 | def sprout(self, url, revision_id=None, force_new_repo=False, |
216 | recurse='down', possible_transports=None, |
217 | accelerator_tree=None, hardlink=False, stacked=False, |
218 | - source_branch=None, create_tree_if_local=True): |
219 | + source_branch=None, create_tree_if_local=True, depth=None): |
220 | from ..repository import InterRepository |
221 | from ..transport.local import LocalTransport |
222 | from ..transport import get_transport |
223 | @@ -187,7 +187,8 @@ |
224 | else: |
225 | determine_wants = interrepo.determine_wants_all |
226 | interrepo.fetch_objects(determine_wants=determine_wants, |
227 | - mapping=source_branch.mapping) |
228 | + mapping=source_branch.mapping, |
229 | + depth=depth) |
230 | result_branch = source_branch.sprout( |
231 | result, revision_id=revision_id, repository=result_repo) |
232 | if (create_tree_if_local and |
233 | |
234 | === modified file 'breezy/git/fetch.py' |
235 | --- breezy/git/fetch.py 2022-08-22 18:19:46 +0000 |
236 | +++ breezy/git/fetch.py 2022-10-27 12:03:56 +0000 |
237 | @@ -453,7 +453,7 @@ |
238 | |
239 | def import_git_objects(repo, mapping, object_iter, |
240 | target_git_object_retriever, heads, pb=None, |
241 | - limit=None): |
242 | + limit=None, depth=None): |
243 | """Import a set of git objects into a bzr repository. |
244 | |
245 | :param repo: Target Bazaar repository |
246 | @@ -461,6 +461,8 @@ |
247 | :param object_iter: Iterator over Git objects. |
248 | :return: Tuple with pack hints and last imported revision id |
249 | """ |
250 | + if depth is None: |
251 | + raise NotImplementedError |
252 | def lookup_object(sha): |
253 | try: |
254 | return object_iter[sha] |
255 | |
256 | === modified file 'breezy/git/interrepo.py' |
257 | --- breezy/git/interrepo.py 2022-01-16 17:15:16 +0000 |
258 | +++ breezy/git/interrepo.py 2022-10-27 12:03:56 +0000 |
259 | @@ -37,6 +37,7 @@ |
260 | |
261 | from ..errors import ( |
262 | DivergedBranches, |
263 | + FetchDepthUnsupported, |
264 | FetchLimitUnsupported, |
265 | InvalidRevisionId, |
266 | LossyPushToSameVCS, |
267 | @@ -121,9 +122,9 @@ |
268 | """ |
269 | raise NotImplementedError(self.fetch_refs) |
270 | |
271 | - def search_missing_revision_ids(self, |
272 | - find_ghosts=True, revision_ids=None, |
273 | - if_present_ids=None, limit=None): |
274 | + def search_missing_revision_ids( |
275 | + self, find_ghosts=True, revision_ids=None, if_present_ids=None, |
276 | + limit=None, depth=None): |
277 | if limit is not None: |
278 | raise FetchLimitUnsupported(self) |
279 | git_shas = [] |
280 | @@ -187,7 +188,7 @@ |
281 | return False |
282 | return self._commit_needs_fetching(sha_id) |
283 | |
284 | - def missing_revisions(self, stop_revisions): |
285 | + def missing_revisions(self, stop_revisions, depth=None): |
286 | """Find the revisions that are missing from the target repository. |
287 | |
288 | :param stop_revisions: Revisions to check for (tuples with |
289 | @@ -201,31 +202,34 @@ |
290 | for (sha1, revid) in stop_revisions: |
291 | if sha1 is not None and revid is not None: |
292 | revid_sha_map[revid] = sha1 |
293 | - stop_revids.append(revid) |
294 | + stop_revids.append((revid, 1)) |
295 | elif sha1 is not None: |
296 | if self._commit_needs_fetching(sha1): |
297 | for (kind, (revid, tree_sha, verifiers)) in self.source_store.lookup_git_sha(sha1): |
298 | revid_sha_map[revid] = sha1 |
299 | - stop_revids.append(revid) |
300 | + stop_revids.append((revid, 1)) |
301 | else: |
302 | if revid is None: |
303 | raise AssertionError |
304 | - stop_revids.append(revid) |
305 | + stop_revids.append((revid, 1)) |
306 | missing = set() |
307 | graph = self.source.get_graph() |
308 | with ui.ui_factory.nested_progress_bar() as pb: |
309 | while stop_revids: |
310 | - new_stop_revids = [] |
311 | - for revid in stop_revids: |
312 | + new_stop_revids = {} |
313 | + for revid, revid_depth in stop_revids: |
314 | sha1 = revid_sha_map.get(revid) |
315 | if (revid not in missing and |
316 | self._revision_needs_fetching(sha1, revid)): |
317 | missing.add(revid) |
318 | - new_stop_revids.append(revid) |
319 | + if depth is None or revid_depth < depth: |
320 | + new_stop_revids[revid] = revid_depth |
321 | stop_revids = set() |
322 | - parent_map = graph.get_parent_map(new_stop_revids) |
323 | - for parent_revids in parent_map.values(): |
324 | - stop_revids.update(parent_revids) |
325 | + parent_map = graph.get_parent_map(new_stop_revids.keys()) |
326 | + for revid, parent_revids in parent_map.items(): |
327 | + stop_revids.update( |
328 | + [(parent_revid, new_stop_revids[revid] + 1) |
329 | + for parent_revid in parent_revids]) |
330 | pb.update("determining revisions to fetch", len(missing)) |
331 | return graph.iter_topo_order(missing) |
332 | |
333 | @@ -286,7 +290,7 @@ |
334 | result_refs[name] = (gitid, revid if not lossy else self.mapping.revision_id_foreign_to_bzr(gitid)) |
335 | return revidmap, old_refs, result_refs |
336 | |
337 | - def fetch_revs(self, revs, lossy, limit=None): |
338 | + def fetch_revs(self, revs, lossy, limit=None, depth=None): |
339 | if not lossy and not self.mapping.roundtripping: |
340 | for git_sha, bzr_revid in revs: |
341 | if (bzr_revid is not None and |
342 | @@ -294,7 +298,7 @@ |
343 | raise NoPushSupport(self.source, self.target, self.mapping, |
344 | bzr_revid) |
345 | with self.source_store.lock_read(): |
346 | - todo = list(self.missing_revisions(revs))[:limit] |
347 | + todo = list(self.missing_revisions(revs, depth=depth))[:limit] |
348 | revidmap = {} |
349 | with ui.ui_factory.nested_progress_bar() as pb: |
350 | object_generator = MissingObjectsIterator( |
351 | @@ -315,7 +319,7 @@ |
352 | return revidmap |
353 | |
354 | def fetch(self, revision_id=None, pb=None, find_ghosts=False, |
355 | - fetch_spec=None, mapped_refs=None, lossy=False): |
356 | + fetch_spec=None, mapped_refs=None, lossy=False, depth=None): |
357 | if mapped_refs is not None: |
358 | stop_revisions = mapped_refs |
359 | elif revision_id is not None: |
360 | @@ -332,7 +336,7 @@ |
361 | for revid in self.source.all_revision_ids()] |
362 | self._warn_slow() |
363 | try: |
364 | - revidmap = self.fetch_revs(stop_revisions, lossy=lossy) |
365 | + revidmap = self.fetch_revs(stop_revisions, lossy=lossy, depth=depth) |
366 | except NoPushSupport: |
367 | raise NoRoundtrippingSupport(self.source, self.target) |
368 | return FetchResult(revidmap) |
369 | @@ -501,7 +505,8 @@ |
370 | 'Fetching from Git to Bazaar repository. ' |
371 | 'For better performance, fetch into a Git repository.') |
372 | |
373 | - def fetch_objects(self, determine_wants, mapping, limit=None, lossy=False): |
374 | + def fetch_objects(self, determine_wants, mapping, limit=None, lossy=False, |
375 | + depth=None): |
376 | """Fetch objects from a remote server. |
377 | |
378 | :param determine_wants: determine_wants callback |
379 | @@ -523,7 +528,8 @@ |
380 | wants, include_tags=include_tags, tag_selector=tag_selector) |
381 | |
382 | def fetch(self, revision_id=None, find_ghosts=False, |
383 | - mapping=None, fetch_spec=None, include_tags=False, lossy=False): |
384 | + mapping=None, fetch_spec=None, include_tags=False, |
385 | + lossy=False, depth=None): |
386 | if mapping is None: |
387 | mapping = self.source.get_mapping() |
388 | if revision_id is not None: |
389 | @@ -545,7 +551,7 @@ |
390 | determine_wants = self.determine_wants_all |
391 | |
392 | (pack_hint, _, remote_refs) = self.fetch_objects( |
393 | - determine_wants, mapping, lossy=lossy) |
394 | + determine_wants, mapping, lossy=lossy, depth=depth) |
395 | if pack_hint is not None and self.target._format.pack_compresses: |
396 | self.target.pack(hint=pack_hint) |
397 | result = FetchResult() |
398 | @@ -566,7 +572,8 @@ |
399 | all_parents.update(values) |
400 | return set(all_revs) - all_parents |
401 | |
402 | - def fetch_objects(self, determine_wants, mapping, limit=None, lossy=False): |
403 | + def fetch_objects(self, determine_wants, mapping, limit=None, lossy=False, |
404 | + depth=None): |
405 | """See `InterGitNonGitRepository`.""" |
406 | self._warn_slow() |
407 | store = get_object_store(self.target, mapping) |
408 | @@ -579,7 +586,8 @@ |
409 | |
410 | with ui.ui_factory.nested_progress_bar() as pb: |
411 | objects_iter = self.source.fetch_objects( |
412 | - wants_recorder, graph_walker, store.get_raw) |
413 | + wants_recorder, graph_walker, store.get_raw, |
414 | + depth=depth) |
415 | trace.mutter("Importing %d new revisions", |
416 | len(wants_recorder.wants)) |
417 | (pack_hint, last_rev) = import_git_objects( |
418 | @@ -605,21 +613,22 @@ |
419 | """InterRepository that copies revisions from a local Git into a non-Git |
420 | repository.""" |
421 | |
422 | - def fetch_objects(self, determine_wants, mapping, limit=None, lossy=False): |
423 | + def fetch_objects(self, determine_wants, mapping, limit=None, lossy=False, |
424 | + depth=None): |
425 | """See `InterGitNonGitRepository`.""" |
426 | + if depth is not None: |
427 | + raise FetchDepthUnsupported(self) |
428 | self._warn_slow() |
429 | remote_refs = self.source.controldir.get_refs_container().as_dict() |
430 | wants = determine_wants(remote_refs) |
431 | target_git_object_retriever = get_object_store(self.target, mapping) |
432 | - with ui.ui_factory.nested_progress_bar() as pb: |
433 | - target_git_object_retriever.lock_write() |
434 | - try: |
435 | - (pack_hint, last_rev) = import_git_objects( |
436 | - self.target, mapping, self.source._git.object_store, |
437 | - target_git_object_retriever, wants, pb, limit) |
438 | - return (pack_hint, last_rev, remote_refs) |
439 | - finally: |
440 | - target_git_object_retriever.unlock() |
441 | + with ui.ui_factory.nested_progress_bar() as pb, \ |
442 | + target_git_object_retriever.lock_write(): |
443 | + (pack_hint, last_rev) = import_git_objects( |
444 | + self.target, mapping, self.source._git.object_store, |
445 | + target_git_object_retriever, wants, pb, limit=limit, |
446 | + depth=depth) |
447 | + return (pack_hint, last_rev, remote_refs) |
448 | |
449 | @staticmethod |
450 | def is_compatible(source, target): |
451 | @@ -657,7 +666,7 @@ |
452 | return None, old_refs, new_refs |
453 | |
454 | def fetch_objects(self, determine_wants, mapping=None, limit=None, |
455 | - lossy=False): |
456 | + lossy=False, depth=None): |
457 | raise NotImplementedError(self.fetch_objects) |
458 | |
459 | def _target_has_shas(self, shas): |
460 | @@ -666,7 +675,7 @@ |
461 | |
462 | def fetch(self, revision_id=None, find_ghosts=False, |
463 | mapping=None, fetch_spec=None, branches=None, limit=None, |
464 | - include_tags=False, lossy=False): |
465 | + include_tags=False, lossy=False, depth=None): |
466 | if mapping is None: |
467 | mapping = self.source.get_mapping() |
468 | if revision_id is not None: |
469 | @@ -688,7 +697,8 @@ |
470 | determine_wants = self.get_determine_wants_revids( |
471 | args, include_tags=include_tags) |
472 | wants_recorder = DetermineWantsRecorder(determine_wants) |
473 | - self.fetch_objects(wants_recorder, mapping, limit=limit, lossy=lossy) |
474 | + self.fetch_objects( |
475 | + wants_recorder, mapping, limit=limit, lossy=lossy, depth=depth) |
476 | result = FetchResult() |
477 | result.refs = wants_recorder.remote_refs |
478 | return result |
479 | @@ -727,7 +737,7 @@ |
480 | class InterLocalGitLocalGitRepository(InterGitGitRepository): |
481 | |
482 | def fetch_objects(self, determine_wants, mapping=None, limit=None, |
483 | - lossy=False): |
484 | + lossy=False, depth=None): |
485 | if lossy: |
486 | raise LossyPushToSameVCS(self.source, self.target) |
487 | if limit is not None: |
488 | @@ -737,7 +747,7 @@ |
489 | progress = DefaultProgressReporter(pb).progress |
490 | refs = self.source._git.fetch( |
491 | self.target._git, determine_wants, |
492 | - progress=progress) |
493 | + progress=progress, depth=depth) |
494 | return (None, None, refs) |
495 | |
496 | @staticmethod |
497 | @@ -750,7 +760,7 @@ |
498 | class InterRemoteGitLocalGitRepository(InterGitGitRepository): |
499 | |
500 | def fetch_objects(self, determine_wants, mapping=None, limit=None, |
501 | - lossy=False): |
502 | + lossy=False, depth=None): |
503 | if lossy: |
504 | raise LossyPushToSameVCS(self.source, self.target) |
505 | if limit is not None: |
506 | @@ -772,10 +782,12 @@ |
507 | else: |
508 | f, commit, abort = self.target._git.object_store.add_pack() |
509 | try: |
510 | - refs = self.source.controldir.fetch_pack( |
511 | - determine_wants, graphwalker, f.write) |
512 | + fetch_result = self.source.controldir.fetch_pack( |
513 | + determine_wants, graphwalker, f.write, depth=depth) |
514 | commit() |
515 | - return (None, None, refs) |
516 | + self.target._git.update_shallow( |
517 | + fetch_result.new_shallow, fetch_result.new_unshallow) |
518 | + return (None, None, fetch_result.refs) |
519 | except BaseException: |
520 | abort() |
521 | raise |
522 | |
523 | === modified file 'breezy/git/remote.py' |
524 | --- breezy/git/remote.py 2022-09-14 19:01:01 +0000 |
525 | +++ breezy/git/remote.py 2022-10-27 12:03:56 +0000 |
526 | @@ -475,8 +475,8 @@ |
527 | if pb is not None: |
528 | pb.finished() |
529 | |
530 | - def fetch_pack(self, determine_wants, graph_walker, pack_data, |
531 | - progress=None): |
532 | + def fetch_pack(self, determine_wants, graph_walker, pack_data, progress=None, |
533 | + depth=None): |
534 | if progress is None: |
535 | pb = ui.ui_factory.nested_progress_bar() |
536 | progress = DefaultProgressReporter(pb).progress |
537 | @@ -485,7 +485,7 @@ |
538 | try: |
539 | result = self._client.fetch_pack( |
540 | self._client_path, determine_wants, graph_walker, pack_data, |
541 | - progress) |
542 | + progress, depth=depth) |
543 | if result.refs is None: |
544 | result.refs = {} |
545 | self._refs = remote_refs_dict_to_container( |
546 | @@ -631,9 +631,10 @@ |
547 | def get_refs_container(self): |
548 | if self._refs is not None: |
549 | return self._refs |
550 | - result = self.fetch_pack(lambda x: None, None, |
551 | - lambda x: None, |
552 | - lambda x: trace.mutter("git: %s" % x)) |
553 | + result = self.fetch_pack( |
554 | + lambda x: None, None, |
555 | + lambda x: None, lambda x: trace.mutter("git: %s" % x), |
556 | + depth=None) |
557 | self._refs = remote_refs_dict_to_container( |
558 | result.refs, result.symrefs) |
559 | return self._refs |
560 | @@ -995,20 +996,21 @@ |
561 | return self.controldir.archive(*args, **kwargs) |
562 | |
563 | def fetch_pack(self, determine_wants, graph_walker, pack_data, |
564 | - progress=None): |
565 | + progress=None, depth=None): |
566 | return self.controldir.fetch_pack( |
567 | - determine_wants, graph_walker, pack_data, progress) |
568 | + determine_wants, graph_walker, pack_data, progress, depth=depth) |
569 | |
570 | def send_pack(self, get_changed_refs, generate_pack_data): |
571 | return self.controldir.send_pack(get_changed_refs, generate_pack_data) |
572 | |
573 | def fetch_objects(self, determine_wants, graph_walker, resolve_ext_ref, |
574 | - progress=None): |
575 | + progress=None, depth=None): |
576 | import tempfile |
577 | fd, path = tempfile.mkstemp(suffix=".pack") |
578 | try: |
579 | - self.fetch_pack(determine_wants, graph_walker, |
580 | - lambda x: os.write(fd, x), progress) |
581 | + self.fetch_pack( |
582 | + determine_wants, graph_walker, |
583 | + lambda x: os.write(fd, x), progress, depth=depth) |
584 | finally: |
585 | os.close(fd) |
586 | if os.path.getsize(path) == 0: |
587 | |
588 | === modified file 'breezy/git/repository.py' |
589 | --- breezy/git/repository.py 2022-01-24 20:22:05 +0000 |
590 | +++ breezy/git/repository.py 2022-10-27 12:03:56 +0000 |
591 | @@ -118,6 +118,7 @@ |
592 | _serializer = None |
593 | vcs = foreign_vcs_git |
594 | chk_bytes = None |
595 | + supports_fetch_depth = True |
596 | |
597 | def __init__(self, gitdir): |
598 | self._transport = gitdir.root_transport |
599 | |
600 | === modified file 'breezy/plugins/weave_fmt/bzrdir.py' |
601 | --- breezy/plugins/weave_fmt/bzrdir.py 2022-08-22 18:19:46 +0000 |
602 | +++ breezy/plugins/weave_fmt/bzrdir.py 2022-10-27 12:03:56 +0000 |
603 | @@ -890,8 +890,10 @@ |
604 | def sprout(self, url, revision_id=None, force_new_repo=False, |
605 | recurse=None, possible_transports=None, accelerator_tree=None, |
606 | hardlink=False, stacked=False, create_tree_if_local=True, |
607 | - source_branch=None): |
608 | + source_branch=None, depth=None): |
609 | """See ControlDir.sprout().""" |
610 | + if depth is not None: |
611 | + raise errors.FetchDepthUnsupported(self) |
612 | if source_branch is not None: |
613 | my_branch = self.open_branch() |
614 | if source_branch.base != my_branch.base: |
615 | |
616 | === modified file 'breezy/repository.py' |
617 | --- breezy/repository.py 2022-07-02 13:44:01 +0000 |
618 | +++ breezy/repository.py 2022-10-27 12:03:56 +0000 |
619 | @@ -281,6 +281,10 @@ |
620 | # items in the tree, or just bulk fetching/pushing of data? |
621 | supports_random_access = True |
622 | |
623 | + # Does this repository implementation support fetching with |
624 | + # a certain graph depth? |
625 | + supports_fetch_depth = False |
626 | + |
627 | def abort_write_group(self, suppress_errors=False): |
628 | """Commit the contents accrued within the current write group. |
629 | |
630 | |
631 | === modified file 'breezy/tests/per_controldir/test_controldir.py' |
632 | --- breezy/tests/per_controldir/test_controldir.py 2022-08-22 18:19:46 +0000 |
633 | +++ breezy/tests/per_controldir/test_controldir.py 2022-10-27 12:03:56 +0000 |
634 | @@ -87,7 +87,7 @@ |
635 | |
636 | def sproutOrSkip(self, from_bzrdir, to_url, revision_id=None, |
637 | force_new_repo=False, accelerator_tree=None, |
638 | - create_tree_if_local=True): |
639 | + create_tree_if_local=True, depth=None): |
640 | """Sprout from_bzrdir into to_url, or raise TestSkipped. |
641 | |
642 | A simple wrapper for from_bzrdir.sprout that translates NotLocalUrl into |
643 | @@ -100,7 +100,8 @@ |
644 | force_new_repo=force_new_repo, |
645 | possible_transports=[to_transport], |
646 | accelerator_tree=accelerator_tree, |
647 | - create_tree_if_local=create_tree_if_local) |
648 | + create_tree_if_local=create_tree_if_local, |
649 | + depth=depth) |
650 | return target |
651 | |
652 | def test_uninitializable(self): |
653 | @@ -1065,6 +1066,20 @@ |
654 | self.addCleanup(repo.lock_read().unlock) |
655 | self.assertEqual(None, repo.get_parent_map([rev1]).get(rev1)) |
656 | |
657 | + def test_sprout_with_depth(self): |
658 | + tree = self.make_branch_and_tree('source') |
659 | + self.build_tree(['source/foo']) |
660 | + tree.add('foo') |
661 | + tree.commit('revision 1') |
662 | + rev2 = tree.commit('revision 2', allow_pointless=True) |
663 | + dir = tree.controldir |
664 | + try: |
665 | + target = self.sproutOrSkip(dir, self.get_url('target'), depth=1) |
666 | + except errors.FetchDepthUnsupported: |
667 | + self.assertFalse(tree.branch.repository.supports_fetch_depth) |
668 | + else: |
669 | + self.assertEqual({rev2}, target.open_repository().all_revision_ids()) |
670 | + |
671 | def test_format_initialize_find_open(self): |
672 | # loopback test to check the current format initializes to itself. |
673 | if not self.bzrdir_format.is_initializable(): |
674 | |
675 | === modified file 'breezy/tests/per_interbranch/test_fetch.py' |
676 | --- breezy/tests/per_interbranch/test_fetch.py 2018-11-11 04:08:32 +0000 |
677 | +++ breezy/tests/per_interbranch/test_fetch.py 2022-10-27 12:03:56 +0000 |
678 | @@ -16,7 +16,11 @@ |
679 | |
680 | """Tests for InterBranch.fetch.""" |
681 | |
682 | -from breezy.errors import FetchLimitUnsupported, NoRoundtrippingSupport |
683 | +from breezy.errors import ( |
684 | + FetchDepthUnsupported, |
685 | + FetchLimitUnsupported, |
686 | + NoRoundtrippingSupport, |
687 | + ) |
688 | from breezy.revision import NULL_REVISION |
689 | from breezy.tests import TestNotApplicable |
690 | from breezy.tests.per_interbranch import ( |
691 | @@ -113,3 +117,39 @@ |
692 | self.assertEqual( |
693 | {rev1, rev2}, |
694 | b2.repository.has_revisions([rev1, rev2, rev3])) |
695 | + |
696 | + def test_fetch_revisions_depth(self): |
697 | + """Test fetch-revision operation.""" |
698 | + builder = self.make_branch_builder( |
699 | + 'b1', format=self.branch_format_from._matchingcontroldir) |
700 | + builder.start_series() |
701 | + rev1 = builder.build_commit() |
702 | + rev2 = builder.build_commit() |
703 | + rev3 = builder.build_commit() |
704 | + builder.finish_series() |
705 | + b1 = builder.get_branch() |
706 | + b2 = self.make_to_branch('b2') |
707 | + try: |
708 | + if b2.repository.supports_fetch_depth: |
709 | + b2.fetch(b1, depth=1) |
710 | + else: |
711 | + self.assertRaises(FetchDepthUnsupported, b2.fetch, b1, depth=1) |
712 | + raise TestNotApplicable( |
713 | + 'interbranch does not support fetch depths') |
714 | + except NoRoundtrippingSupport: |
715 | + raise TestNotApplicable( |
716 | + 'lossless cross-vcs fetch %r to %r not supported' % |
717 | + (b1, b2)) |
718 | + |
719 | + self.assertEqual( |
720 | + {rev3}, |
721 | + b2.repository.has_revisions([rev1, rev2, rev3])) |
722 | + |
723 | + # fetch does not update the last revision |
724 | + self.assertEqual(NULL_REVISION, b2.last_revision()) |
725 | + |
726 | + # Incrementally fetch one more |
727 | + b2.fetch(b1, depth=2) |
728 | + |
729 | + self.assertEqual( |
730 | + {rev2, rev3}, b2.repository.has_revisions([rev1, rev2, rev3])) |
731 | |
732 | === modified file 'breezy/tests/per_repository/test_repository.py' |
733 | --- breezy/tests/per_repository/test_repository.py 2022-07-02 13:44:01 +0000 |
734 | +++ breezy/tests/per_repository/test_repository.py 2022-10-27 12:03:56 +0000 |
735 | @@ -126,6 +126,9 @@ |
736 | def test_attribute_format_supports_random_access(self): |
737 | self.assertRepositoryAttribute('supports_random_access', (True, False)) |
738 | |
739 | + def test_attribute_format_supports_fetch_depth(self): |
740 | + self.assertRepositoryAttribute('supports_fetch_depth', (True, False)) |
741 | + |
742 | def test_attribute_format_supports_setting_revision_ids(self): |
743 | self.assertFormatAttribute('supports_setting_revision_ids', |
744 | (True, False)) |
745 | |
746 | === modified file 'breezy/tests/test_foreign.py' |
747 | --- breezy/tests/test_foreign.py 2022-08-22 18:19:46 +0000 |
748 | +++ breezy/tests/test_foreign.py 2022-10-27 12:03:56 +0000 |
749 | @@ -340,14 +340,15 @@ |
750 | def sprout(self, url, revision_id=None, force_new_repo=False, |
751 | recurse='down', possible_transports=None, |
752 | accelerator_tree=None, hardlink=False, stacked=False, |
753 | - source_branch=None): |
754 | + source_branch=None, depth=None): |
755 | # dirstate doesn't cope with accelerator_trees well |
756 | # that have a different control dir |
757 | return super(DummyForeignVcsDir, self).sprout( |
758 | url=url, |
759 | revision_id=revision_id, force_new_repo=force_new_repo, |
760 | recurse=recurse, possible_transports=possible_transports, |
761 | - hardlink=hardlink, stacked=stacked, source_branch=source_branch) |
762 | + hardlink=hardlink, stacked=stacked, source_branch=source_branch, |
763 | + depth=depth) |
764 | |
765 | |
766 | def register_dummy_foreign_for_test(testcase): |
Thanks!