Merge ~nacc/git-ubuntu:lp1717965-lint-applied-branches into git-ubuntu:master
- Git
- lp:~nacc/git-ubuntu
- lp1717965-lint-applied-branches
- Merge into master
Status: | Merged | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Merged at revision: | 262a939c67cd2913867966b7f5b368b0a45b5c47 | ||||||||||||||||
Proposed branch: | ~nacc/git-ubuntu:lp1717965-lint-applied-branches | ||||||||||||||||
Merge into: | git-ubuntu:master | ||||||||||||||||
Diff against target: |
2730 lines (+1245/-463) 23 files modified
doc/README.md (+1/-1) doc/gitubuntu-completion.sh (+3/-0) gitubuntu/__main__.py (+1/-1) gitubuntu/build.py (+278/-192) gitubuntu/buildsource.py (+25/-20) gitubuntu/git_repository.py (+546/-41) gitubuntu/importer.py (+98/-77) gitubuntu/importppa.py (+8/-0) gitubuntu/lint.py (+29/-4) gitubuntu/merge.py (+11/-12) gitubuntu/remote.py (+128/-31) gitubuntu/review.py (+4/-0) gitubuntu/run.py (+31/-14) gitubuntu/versioning.py (+14/-5) man/man1/git-ubuntu-build-source.1 (+8/-1) man/man1/git-ubuntu-build.1 (+8/-1) man/man1/git-ubuntu-import-ppa.1 (+16/-1) man/man1/git-ubuntu-import.1 (+16/-1) man/man1/git-ubuntu.1 (+1/-1) setup.py (+1/-0) snap/snapcraft.yaml (+4/-60) snap/wrappers/git-merge-changelogs (+7/-0) snap/wrappers/git-reconstruct-changelog (+7/-0) |
||||||||||||||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Server Team CI bot | continuous-integration | Approve | |
Robie Basak | Pending | ||
Review via email: mp+332160@code.launchpad.net |
Commit message
Description of the change
Make Jenkins happy.
This resolves several first-time user issues with the build/buildsource code, and allows us to toggle the default branch for clone to be patches-applied.
Server Team CI bot (server-team-bot) wrote : | # |
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:79700c89631
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:79700c89631
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:41bd8600400
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:ece83566572
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:98c5c3c522f
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:9657a342abe
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:e7e4018d118
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:9fed1c0078c
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:0ea422567c9
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
SUCCESS: Integration Tests
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:c216f0968a5
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
SUCCESS: Integration Tests
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:72c44168170
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:72c44168170
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
SUCCESS: Integration Tests
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:a8da6ba278c
https:/
Executed test runs:
SUCCESS: Checkout
FAILED: Style Check
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:5ffa1cee5ea
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:5ffa1cee5ea
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
SUCCESS: Integration Tests
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:60ffc3cc1be
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:faee1277ab4
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
SUCCESS: Integration Tests
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:0ff756c2542
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
SUCCESS: Integration Tests
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:ce1f0689665
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:0036188e37d
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
SUCCESS: Integration Tests
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:ce1f0689665
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
SUCCESS: Integration Tests
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:0036188e37d
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
SUCCESS: Integration Tests
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:262a939c67c
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
SUCCESS: Integration Tests
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild:
https:/
Preview Diff
1 | diff --git a/doc/README.md b/doc/README.md |
2 | index f525112..0e33343 100644 |
3 | --- a/doc/README.md |
4 | +++ b/doc/README.md |
5 | @@ -21,7 +21,7 @@ or |
6 | 3. Get necessary dependencies |
7 | |
8 | $ sudo apt update -qy |
9 | - $ deps="dpkg-dev git-buildpackage python3-argcomplete python3-lazr.restfulclient python3-debian python3-distro-info python3-launchpadlib python3-pygit2 python3-ubuntutools python3-pkg-resources python3-pytest quilt" |
10 | + $ deps="dpkg-dev git-buildpackage python3-argcomplete python3-lazr.restfulclient python3-debian python3-distro-info python3-launchpadlib python3-pygit2 python3-ubuntutools python3-pkg-resources python3-pytest python3-petname quilt" |
11 | $ sudo apt install -qy $deps |
12 | |
13 | ## Getting via snap ## |
14 | diff --git a/doc/gitubuntu-completion.sh b/doc/gitubuntu-completion.sh |
15 | index 5449b54..4f1d9a2 100644 |
16 | --- a/doc/gitubuntu-completion.sh |
17 | +++ b/doc/gitubuntu-completion.sh |
18 | @@ -43,6 +43,7 @@ _git_ubuntu () |
19 | case "$subcommand,$cur" in |
20 | build,--*) |
21 | __gitcomp " |
22 | + --commitish |
23 | --dl-cache |
24 | --for-merge |
25 | --keep-build-env |
26 | @@ -58,6 +59,7 @@ _git_ubuntu () |
27 | ;; |
28 | build-source,--*) |
29 | __gitcomp " |
30 | + --commitish |
31 | --dl-cache |
32 | --for-merge |
33 | --keep-build-env |
34 | @@ -81,6 +83,7 @@ _git_ubuntu () |
35 | import,--*) |
36 | __gitcomp " |
37 | --active-series-only |
38 | + --allow-applied-failures |
39 | -d --directory |
40 | --dl-cache |
41 | --fixup-devel |
42 | diff --git a/gitubuntu/__main__.py b/gitubuntu/__main__.py |
43 | index ea3bcba..867b581 100644 |
44 | --- a/gitubuntu/__main__.py |
45 | +++ b/gitubuntu/__main__.py |
46 | @@ -24,7 +24,7 @@ TopLevelDefaults = namedtuple( |
47 | ) |
48 | top_level_defaults = TopLevelDefaults( |
49 | verbose=False, |
50 | - retries=3, |
51 | + retries=5, |
52 | retry_backoffs = [2 ** i for i in range(3)], |
53 | proto='https', |
54 | parentfile=pkg_resources.resource_filename( |
55 | diff --git a/gitubuntu/build.py b/gitubuntu/build.py |
56 | index 1eb8b5f..6c86374 100644 |
57 | --- a/gitubuntu/build.py |
58 | +++ b/gitubuntu/build.py |
59 | @@ -30,17 +30,11 @@ import sys |
60 | import tempfile |
61 | import time |
62 | import traceback |
63 | -import uuid |
64 | from gitubuntu.__main__ import top_level_defaults |
65 | from gitubuntu.cache import CACHE_PATH |
66 | from gitubuntu.dsc import GitUbuntuDsc |
67 | -from gitubuntu.git_repository import ( |
68 | - Changelog, |
69 | - GitUbuntuRepository, |
70 | - MultiplePristineTarFoundError, |
71 | - PristineTarNotFoundError, |
72 | -) |
73 | from gitubuntu.lint import derive_target_branch |
74 | +import gitubuntu.git_repository |
75 | from gitubuntu.run import decode_binary, run, runq |
76 | from gitubuntu.source_information import ( |
77 | GitUbuntuSourceInformation, |
78 | @@ -51,6 +45,10 @@ from gitubuntu.source_information import ( |
79 | try: |
80 | pkg = 'python3-debian' |
81 | from debian.debfile import PART_EXTS |
82 | + pkg = 'python3-petname' |
83 | + import petname |
84 | + pkg = 'python3-pygit2' |
85 | + import pygit2 |
86 | pkg = 'python3-pytest' |
87 | import pytest |
88 | except ImportError: |
89 | @@ -265,7 +263,7 @@ def fetch_orig_from_pristine_tar(changelog, source, repo): |
90 | changelog.srcpkg, |
91 | changelog.upstream_version, |
92 | ) |
93 | - except MultiplePristineTarFoundError as e: |
94 | + except gitubuntu.git_repository.MultiplePristineTarFoundError as e: |
95 | logging.warning("%s. This is often because the orig " |
96 | "tarball compression changed and it is not possible " |
97 | "to determine which tarball to use automatically.", e |
98 | @@ -387,12 +385,12 @@ def fetch_orig_from_launchpad(changelog, source, pullfile, retries, |
99 | |
100 | def check_repository(): |
101 | try: |
102 | - cp = run(['git', 'status', '--porcelain']) |
103 | + stdout, _ = run(['git', 'status', '--porcelain']) |
104 | except CalledProcessError: |
105 | logging.error('Is the current directory a git repository?') |
106 | return False |
107 | |
108 | - if len(cp.stdout) > 0: |
109 | + if len(stdout) > 0: |
110 | logging.warning( |
111 | "Working tree is not clean. `git status` output follows." |
112 | ) |
113 | @@ -410,9 +408,11 @@ def check_repository(): |
114 | |
115 | |
116 | def main( |
117 | + repo, |
118 | + commitish, |
119 | search_list, |
120 | - changelog, |
121 | rem_args, |
122 | + for_merge, |
123 | sign, |
124 | use_lxd=True, |
125 | keep_build_env=False, |
126 | @@ -423,9 +423,11 @@ def main( |
127 | ): |
128 | """main entry point for build |
129 | |
130 | + @param repo: a gitubuntu.git_repository.GitUbuntuRepository object |
131 | + @param commitish: string commitish to build |
132 | @param search_list: list of OrigSearchListEntry namedtuples |
133 | - @param changelog: gitubuntu.git_repository.Changelog object for source package to build |
134 | @param rem_args: namespace object of remaining arguments to pass onto dpkg-buildpackage |
135 | + @param for_merge: if True, the build is an Ubuntu merge |
136 | @param sign: if True, sign the resulting changes file |
137 | @param use_lxd: if True, use a LXD container to build the package |
138 | @param keep_build_env: if True, do not purge the build environment |
139 | @@ -443,13 +445,10 @@ def main( |
140 | return [] |
141 | |
142 | try: |
143 | - repo = GitUbuntuRepository('.') |
144 | - pkg_remote_branch_string = derive_target_branch(repo, 'HEAD') |
145 | - if len(pkg_remote_branch_string) == 0: |
146 | - sys.exit(1) |
147 | + pkg_remote_branch_string = derive_target_branch(repo, commitish) |
148 | |
149 | # Implies a merge |
150 | - if 'debian' in pkg_remote_branch_string: |
151 | + if 'debian' in pkg_remote_branch_string or for_merge: |
152 | oldubuntu_version, _ = repo.get_changelog_versions_from_treeish( |
153 | 'pkg/ubuntu/devel', |
154 | ) |
155 | @@ -462,8 +461,9 @@ def main( |
156 | ) |
157 | |
158 | files = fetch_orig_and_build( |
159 | + repo, |
160 | + commitish, |
161 | search_list, |
162 | - changelog, |
163 | rem_args, |
164 | use_lxd, |
165 | keep_build_env, |
166 | @@ -482,7 +482,7 @@ def main( |
167 | changes_file = changes_files.pop() |
168 | try: |
169 | logging.info("Signing changes file %s", changes_file) |
170 | - run(['debsign', changes_file,]) |
171 | + run(['debsign', '-p%s' % _gpg_abspath(), changes_file,]) |
172 | except CalledProcessError as e: |
173 | raise RuntimeError( |
174 | "Failed to sign changes file %s" % changes_file |
175 | @@ -556,6 +556,12 @@ def parse_args(subparsers=None, base_subparsers=None): |
176 | help="LXD image to use. If not specified, the image will be " |
177 | "derived from distribution value of HEAD:debian/changelog.", |
178 | ) |
179 | + parser.add_argument( |
180 | + '--commitish', |
181 | + type=str, |
182 | + help="Git commitish to build.", |
183 | + default='HEAD', |
184 | + ) |
185 | parser.add_argument('rem_args', help=argparse.SUPPRESS, |
186 | nargs=argparse.REMAINDER) |
187 | if not subparsers: |
188 | @@ -563,8 +569,31 @@ def parse_args(subparsers=None, base_subparsers=None): |
189 | return 'build - %s' % kwargs['description'] |
190 | |
191 | |
192 | -def derive_orig_search_list_from_args(args): |
193 | - source = 'debian' if args.for_merge else 'ubuntu' |
194 | +def derive_orig_search_list_from_args( |
195 | + repo, |
196 | + commitish, |
197 | + for_merge, |
198 | + no_pristine_tar, |
199 | + pullfile=top_level_defaults.pullfile, |
200 | + retries=top_level_defaults.retries, |
201 | + retry_backoffs=top_level_defaults.retry_backoffs, |
202 | + dl_cache=None, |
203 | +): |
204 | + native = is_native_package( |
205 | + repo.get_changelog_from_treeish(commitish) |
206 | + ) |
207 | + |
208 | + if native: |
209 | + # No orig tarball required |
210 | + return [ |
211 | + OrigSearchListEntry( |
212 | + mechanism=fetch_orig_noop, |
213 | + source=[], |
214 | + must_build=True, |
215 | + ), |
216 | + ] |
217 | + |
218 | + source = 'debian' if for_merge else 'ubuntu' |
219 | orig_search_list = [ |
220 | OrigSearchListEntry( |
221 | mechanism=fetch_orig_from_parent_dir, |
222 | @@ -574,34 +603,29 @@ def derive_orig_search_list_from_args(args): |
223 | OrigSearchListEntry( |
224 | mechanism=fetch_orig_from_cache, |
225 | source=source, |
226 | - must_build=True, |
227 | + must_build=False, |
228 | ), |
229 | ] |
230 | - if not args.no_pristine_tar: |
231 | + if not no_pristine_tar: |
232 | orig_search_list.append( |
233 | OrigSearchListEntry( |
234 | mechanism=functools.partial(fetch_orig_from_pristine_tar, |
235 | - repo=GitUbuntuRepository('.'), |
236 | + repo=repo, |
237 | ), |
238 | source=source, |
239 | - must_build=True, |
240 | + must_build=False, |
241 | ) |
242 | ) |
243 | orig_search_list.extend([ |
244 | OrigSearchListEntry( |
245 | mechanism=functools.partial(fetch_orig_from_launchpad, |
246 | - pullfile=args.pullfile, |
247 | - retries=args.retries, |
248 | - retry_backoffs=args.retry_backoffs, |
249 | - dl_cache=args.dl_cache, |
250 | + pullfile=pullfile, |
251 | + retries=retries, |
252 | + retry_backoffs=retry_backoffs, |
253 | + dl_cache=dl_cache, |
254 | ), |
255 | source=source, |
256 | - must_build=True, |
257 | - ), |
258 | - OrigSearchListEntry( |
259 | - mechanism=fetch_orig_noop, |
260 | - source=[], |
261 | - must_build=True, |
262 | + must_build=False, |
263 | ), |
264 | ]) |
265 | |
266 | @@ -615,32 +639,31 @@ def cli_main(args): |
267 | else: |
268 | rem_args = default_rem_args |
269 | |
270 | - changelog = Changelog.from_path('debian/changelog') |
271 | + repo = gitubuntu.git_repository.GitUbuntuRepository('.') |
272 | |
273 | try: |
274 | - native = is_native_package(changelog) |
275 | + orig_search_list = derive_orig_search_list_from_args( |
276 | + repo, |
277 | + args.commitish, |
278 | + args.for_merge, |
279 | + args.no_pristine_tar, |
280 | + args.pullfile, |
281 | + args.retries, |
282 | + args.retry_backoffs, |
283 | + args.dl_cache, |
284 | + ) |
285 | except NativenessMismatchError as e: |
286 | logging.error("%s" % e) |
287 | return 1 |
288 | |
289 | - if native: |
290 | - # No orig tarball required |
291 | - orig_search_list = [ |
292 | - OrigSearchListEntry( |
293 | - mechanism=fetch_orig_noop, |
294 | - source=[], |
295 | - must_build=True, |
296 | - ), |
297 | - ] |
298 | - else: |
299 | - orig_search_list = derive_orig_search_list_from_args(args) |
300 | - |
301 | # See http://pad.ubuntu.com/KKB1kMR0JH for logic |
302 | if len( |
303 | main( |
304 | + repo, |
305 | + args.commitish, |
306 | orig_search_list, |
307 | - changelog, |
308 | rem_args, |
309 | + for_merge=args.for_merge, |
310 | sign=args.sign, |
311 | use_lxd=not args.no_lxd, |
312 | keep_build_env=args.keep_build_env, |
313 | @@ -667,7 +690,7 @@ def derive_source_from_changelog(changelog): |
314 | ]) |
315 | def test_derive_source_from_changelog(changelog_path, expected): |
316 | assert derive_source_from_changelog( |
317 | - Changelog.from_path(changelog_path) |
318 | + gitubuntu.git_repository.Changelog.from_path(changelog_path) |
319 | ) == expected |
320 | |
321 | def derive_codename_from_changelog(changelog): |
322 | @@ -681,11 +704,11 @@ def derive_codename_from_changelog(changelog): |
323 | ]) |
324 | def test_derive_codename_from_changelog(changelog_path, expected): |
325 | assert derive_codename_from_changelog( |
326 | - Changelog.from_path(changelog_path) |
327 | + gitubuntu.git_repository.Changelog.from_path(changelog_path) |
328 | ) == expected |
329 | with pytest.raises(ValueError): |
330 | derive_codename_from_changelog( |
331 | - Changelog.from_path( |
332 | + gitubuntu.git_repository.Changelog.from_path( |
333 | 'tests/changelogs/test_distribution_source_4', |
334 | ) |
335 | ) |
336 | @@ -706,7 +729,8 @@ def expand_changelog_source_aliases(orig_search_list, changelog): |
337 | |
338 | |
339 | def do_build( |
340 | - changelog, |
341 | + repo, |
342 | + commitish, |
343 | tarballs, |
344 | rem_args, |
345 | use_lxd, |
346 | @@ -716,66 +740,89 @@ def do_build( |
347 | retries, |
348 | retry_backoffs, |
349 | ): |
350 | - if use_lxd: |
351 | - return do_build_lxd( |
352 | - changelog, |
353 | - tarballs, |
354 | - rem_args, |
355 | - keep_build_env, |
356 | - lxd_profile, |
357 | - lxd_image, |
358 | - retries, |
359 | - retry_backoffs, |
360 | + with ExitStack() as stack: |
361 | + changelog = repo.get_changelog_from_treeish(commitish) |
362 | + |
363 | + commitish_tree_hash = str(repo.get_commitish(commitish).peel(pygit2.Tree).id) |
364 | + |
365 | + if gitubuntu.git_repository.is_3_0_quilt(repo, commitish): |
366 | + tree_hash = repo.quiltify_and_changelogify_tree_hash(commitish) |
367 | + else: |
368 | + tree_hash = commitish_tree_hash |
369 | + |
370 | + archive_tarball = stack.enter_context( |
371 | + tempfile.NamedTemporaryFile( |
372 | + suffix='.tar.gz', |
373 | + ) |
374 | ) |
375 | - else: |
376 | - return do_build_host( |
377 | - changelog, |
378 | - tarballs, |
379 | - rem_args, |
380 | - keep_build_env, |
381 | + archive_prefix = '%s-%s' % ( |
382 | + changelog.srcpkg, |
383 | + changelog.version, |
384 | ) |
385 | + cmd = [ |
386 | + 'git', |
387 | + 'archive', |
388 | + '-o', archive_tarball.name, |
389 | + '--prefix=%s/' % archive_prefix, |
390 | + tree_hash, |
391 | + ] |
392 | + try: |
393 | + run(cmd) |
394 | + except Exception as e: |
395 | + raise RuntimeError("Failed to archive the repository") from e |
396 | + |
397 | + if use_lxd: |
398 | + built_files = do_build_lxd_exitstack( |
399 | + changelog, |
400 | + archive_tarball.name, |
401 | + tarballs, |
402 | + rem_args, |
403 | + keep_build_env, |
404 | + lxd_profile, |
405 | + lxd_image, |
406 | + retries, |
407 | + retry_backoffs, |
408 | + stack, |
409 | + ) |
410 | + else: |
411 | + built_files = do_build_host_exitstack( |
412 | + changelog, |
413 | + archive_prefix, |
414 | + archive_tarball.name, |
415 | + tarballs, |
416 | + rem_args, |
417 | + keep_build_env, |
418 | + stack, |
419 | + ) |
420 | |
421 | + # Did we do a quiltify/changelogify fixup? |
422 | + if commitish_tree_hash != tree_hash: |
423 | + fixup_commit_hash = repo.commit_tree_hash( |
424 | + tree_hash=tree_hash, |
425 | + parents=[commitish], |
426 | + log_message=b'Automatically generated git-ubuntu fixup.', |
427 | + ) |
428 | + logging.info( |
429 | +"""We automatically generated fixup changes relative to |
430 | +%s as commit %s. |
431 | +If you would like to create a branch at that commit, run: |
432 | +\tgit checkout -b <branch name> %s""", |
433 | + commitish, |
434 | + fixup_commit_hash, |
435 | + fixup_commit_hash, |
436 | + ) |
437 | |
438 | -def do_build_host(changelog, tarballs, rem_args, keep_build_env): |
439 | - with ExitStack() as stack: |
440 | - return _do_build_host_exitstack( |
441 | - changelog, |
442 | - tarballs, |
443 | - rem_args, |
444 | - keep_build_env, |
445 | - stack, |
446 | - ) |
447 | + return built_files |
448 | |
449 | -def _do_build_host_exitstack( |
450 | +def do_build_host_exitstack( |
451 | changelog, |
452 | + archive_prefix, |
453 | + archive_tarball_name, |
454 | tarballs, |
455 | rem_args, |
456 | keep_build_env, |
457 | stack, |
458 | ): |
459 | - commitish = 'HEAD' |
460 | - |
461 | - archive_tarball = stack.enter_context( |
462 | - tempfile.NamedTemporaryFile( |
463 | - suffix='.tar.gz', |
464 | - ) |
465 | - ) |
466 | - archive_prefix = '%s-%s' % ( |
467 | - changelog.srcpkg, |
468 | - changelog.version, |
469 | - ) |
470 | - cmd = [ |
471 | - 'git', |
472 | - 'archive', |
473 | - '-o', archive_tarball.name, |
474 | - '--prefix=%s/' % archive_prefix, |
475 | - commitish |
476 | - ] |
477 | - try: |
478 | - run(cmd) |
479 | - except Exception as e: |
480 | - raise RuntimeError("Failed to archive the repository") from e |
481 | - |
482 | tempdir = tempfile.mkdtemp() |
483 | if keep_build_env: |
484 | logging.info( |
485 | @@ -790,7 +837,7 @@ def _do_build_host_exitstack( |
486 | shutil.copy(tarball, tempdir) |
487 | |
488 | try: |
489 | - run(['tar', 'zxCf', tempdir, archive_tarball.name,]) |
490 | + run(['tar', 'zxCf', tempdir, archive_tarball_name,]) |
491 | except Exception as e: |
492 | raise RuntimeError( |
493 | "Failed to untar archive tarball in temporary directory" |
494 | @@ -814,29 +861,6 @@ def _do_build_host_exitstack( |
495 | |
496 | return built_pardir_contents |
497 | |
498 | -def do_build_lxd( |
499 | - changelog, |
500 | - tarballs, |
501 | - rem_args, |
502 | - keep_build_env, |
503 | - profile, |
504 | - image, |
505 | - retries, |
506 | - retry_backoffs, |
507 | -): |
508 | - with ExitStack() as stack: |
509 | - return _do_build_lxd_exitstack( |
510 | - changelog, |
511 | - tarballs, |
512 | - rem_args, |
513 | - keep_build_env, |
514 | - profile, |
515 | - image, |
516 | - retries, |
517 | - retry_backoffs, |
518 | - stack, |
519 | - ) |
520 | - |
521 | def get_cmd_in_origpath(cmd): |
522 | return shutil.which( |
523 | cmd, |
524 | @@ -847,20 +871,58 @@ def get_cmd_in_origpath(cmd): |
525 | ) |
526 | |
527 | @functools.lru_cache() |
528 | +def _gpg_abspath(): |
529 | + orig_gpg = get_cmd_in_origpath('gpg') |
530 | + if orig_gpg: |
531 | + return orig_gpg |
532 | + raise RuntimeError("Unable to find a gpg binary, is it installed?") |
533 | + |
534 | +@functools.lru_cache() |
535 | def _lxc_abspath(): |
536 | orig_lxc = get_cmd_in_origpath('lxc') |
537 | - return orig_lxc if orig_lxc else 'lxc' |
538 | + if orig_lxc: |
539 | + return orig_lxc |
540 | + raise RuntimeError("Unable to find a lxc binary, is it installed?") |
541 | |
542 | def _cleanup_lxd(container_name): |
543 | run([_lxc_abspath(), 'stop', '--force', container_name]) |
544 | |
545 | -def _run_in_lxd(container_name, args): |
546 | +def _run_in_lxd(container_name, args, user=None, **kwargs): |
547 | cmd = [_lxc_abspath(), 'exec', container_name, '--',] |
548 | + # user == None means run as root |
549 | + if user: |
550 | + try: |
551 | + _run_in_lxd( |
552 | + container_name, |
553 | + [ |
554 | + 'grep', |
555 | + '-q', |
556 | + user, |
557 | + '/etc/passwd', |
558 | + ], |
559 | + user=None, |
560 | + verbose_on_failure=False, |
561 | + ) |
562 | + except CalledProcessError as e: |
563 | + if e.returncode == 1: |
564 | + _run_in_lxd( |
565 | + container_name, |
566 | + [ |
567 | + 'adduser', |
568 | + user, |
569 | + '--disabled-password', |
570 | + '--gecos=""', |
571 | + ], |
572 | + ) |
573 | + else: |
574 | + raise |
575 | + cmd.extend(['sudo', '-s', '-H', '-u', user,]) |
576 | cmd.extend(args) |
577 | - return run(cmd) |
578 | + return run(cmd, **kwargs) |
579 | |
580 | -def _do_build_lxd_exitstack( |
581 | +def do_build_lxd_exitstack( |
582 | changelog, |
583 | + archive_tarball_name, |
584 | tarballs, |
585 | rem_args, |
586 | keep_build_env, |
587 | @@ -876,26 +938,7 @@ def _do_build_lxd_exitstack( |
588 | except Exception as e: |
589 | raise RuntimeError("LXD running?") from e |
590 | |
591 | - commitish = 'HEAD' |
592 | - |
593 | - archive_tarball = stack.enter_context( |
594 | - tempfile.NamedTemporaryFile( |
595 | - suffix='.tar.gz', |
596 | - ) |
597 | - ) |
598 | - cmd = [ |
599 | - 'git', |
600 | - 'archive', |
601 | - '-o', archive_tarball.name, |
602 | - '--prefix=%s-%s/' % (changelog.srcpkg, changelog.version), |
603 | - commitish |
604 | - ] |
605 | - try: |
606 | - run(cmd) |
607 | - except Exception as e: |
608 | - raise RuntimeError("Failed to archive the repository") from e |
609 | - |
610 | - container_name = 'gu-build-%s' % uuid.uuid4() |
611 | + container_name = petname.Generate(2, '-') |
612 | |
613 | cmd = [lxc, 'launch', '-e'] |
614 | if profile: |
615 | @@ -937,9 +980,6 @@ def _do_build_lxd_exitstack( |
616 | "manually stopped with:\n\tlxc stop --force %s", |
617 | container_name, |
618 | ) |
619 | - # wait for the LXD container to have networking, which always takes |
620 | - # a bit of time |
621 | - time.sleep(10) |
622 | |
623 | for i in range(retries+1): |
624 | try: |
625 | @@ -950,6 +990,7 @@ def _do_build_lxd_exitstack( |
626 | '-y', |
627 | 'devscripts', |
628 | 'equivs', |
629 | + 'sudo', |
630 | ]) |
631 | break |
632 | except Exception as e: |
633 | @@ -970,7 +1011,7 @@ def _do_build_lxd_exitstack( |
634 | lxc, |
635 | 'file', |
636 | 'push', |
637 | - archive_tarball.name, |
638 | + archive_tarball_name, |
639 | '%s/tmp/' % container_name, |
640 | ]) |
641 | except Exception as e: |
642 | @@ -978,17 +1019,9 @@ def _do_build_lxd_exitstack( |
643 | "Failed to push archive tarball to ephemeral build container" |
644 | ) from e |
645 | |
646 | - try: |
647 | - cp = _run_in_lxd(container_name, ['mktemp' , '-d',]) |
648 | - containertemp = decode_binary(cp.stdout).strip() |
649 | - except Exception as e: |
650 | - raise RuntimeError( |
651 | - "Failed to create temporary build directory in container" |
652 | - ) from e |
653 | - |
654 | cmd = [lxc, 'file', 'push'] |
655 | cmd.extend(tarballs) |
656 | - cmd.extend(['%s%s/' % (container_name, containertemp),]) |
657 | + cmd.extend(['%s/tmp/' % container_name,]) |
658 | try: |
659 | run(cmd) |
660 | except Exception as e: |
661 | @@ -997,29 +1030,36 @@ def _do_build_lxd_exitstack( |
662 | logging.info("Copied build files to %s", container_name) |
663 | |
664 | try: |
665 | - _run_in_lxd(container_name, [ |
666 | - 'tar', |
667 | - 'zxCf', |
668 | - containertemp, |
669 | - '/tmp/%s' % os.path.basename(archive_tarball.name), |
670 | - ]) |
671 | + _run_in_lxd( |
672 | + container_name, |
673 | + [ |
674 | + 'tar', |
675 | + 'zxCf', |
676 | + '/tmp', |
677 | + '/tmp/%s' % os.path.basename(archive_tarball_name), |
678 | + ], |
679 | + user='ubuntu', |
680 | + ) |
681 | except Exception as e: |
682 | raise RuntimeError( |
683 | "Failed to untar archive tarball in container" |
684 | ) from e |
685 | |
686 | - cp = _run_in_lxd(container_name, ['ls', containertemp,]) |
687 | + stdout, _ = _run_in_lxd( |
688 | + container_name, |
689 | + ['ls', '/tmp',], |
690 | + user='ubuntu', |
691 | + ) |
692 | orig_temp_contents = set( |
693 | filename_stripped |
694 | for filename_stripped in ( |
695 | filename.strip() for filename in |
696 | - decode_binary(cp.stdout).splitlines() |
697 | + stdout.splitlines() |
698 | ) |
699 | if filename_stripped |
700 | ) |
701 | |
702 | - container_path = '%s/%s-%s' % ( |
703 | - containertemp, |
704 | + container_path = '/tmp/%s-%s' % ( |
705 | changelog.srcpkg, |
706 | changelog.version, |
707 | ) |
708 | @@ -1033,11 +1073,14 @@ def _do_build_lxd_exitstack( |
709 | # ), |
710 | # except that Trusty does not support apt-get build-dep with a |
711 | # local dir. |
712 | - _run_in_lxd(container_name, [ |
713 | - 'sh', |
714 | - '-c', |
715 | - 'cd %s && (export DEBIAN_FRONTEND=noninteractive; mk-build-deps -i -t "apt-get -y -o DPkg::options::=\'--force-confdef\' -o DPkg::options::=\'--force-confold\' -o Debug::pkgProblemResolver=yes --no-install-recommends" -r debian/control)' % container_path, |
716 | - ]) |
717 | + _run_in_lxd( |
718 | + container_name, |
719 | + [ |
720 | + 'sh', |
721 | + '-c', |
722 | + 'cd %s && (export DEBIAN_FRONTEND=noninteractive; mk-build-deps -i -t "apt-get -y -o DPkg::options::=\'--force-confdef\' -o DPkg::options::=\'--force-confold\' -o Debug::pkgProblemResolver=yes --no-install-recommends" -r debian/control)' % container_path, |
723 | + ], |
724 | + ) |
725 | except Exception as e: |
726 | raise RuntimeError( |
727 | "Failed to install build dependencies in container" |
728 | @@ -1046,23 +1089,31 @@ def _do_build_lxd_exitstack( |
729 | logging.info("Running dpkg-buildpackage in %s", container_name) |
730 | |
731 | try: |
732 | - _run_in_lxd(container_name, [ |
733 | - 'sh', |
734 | - '-c', |
735 | - 'cd %s && dpkg-buildpackage %s' % ( |
736 | - container_path, |
737 | - ' '.join(rem_args), |
738 | - ), |
739 | - ]) |
740 | + _run_in_lxd( |
741 | + container_name, |
742 | + [ |
743 | + 'sh', |
744 | + '-c', |
745 | + 'cd %s && dpkg-buildpackage %s' % ( |
746 | + container_path, |
747 | + ' '.join(rem_args), |
748 | + ), |
749 | + ], |
750 | + user='ubuntu', |
751 | + ) |
752 | except Exception as e: |
753 | raise RuntimeError("dpkg-buildpackage failed in the container") from e |
754 | |
755 | - cp = _run_in_lxd(container_name, ['ls', containertemp]) |
756 | + stdout, _ = _run_in_lxd( |
757 | + container_name, |
758 | + ['ls', '/tmp',], |
759 | + user='ubuntu', |
760 | + ) |
761 | new_temp_contents = set( |
762 | filename_stripped |
763 | for filename_stripped in ( |
764 | filename.strip() for filename in |
765 | - decode_binary(cp.stdout).splitlines() |
766 | + stdout.splitlines() |
767 | ) |
768 | if filename_stripped |
769 | ) |
770 | @@ -1070,7 +1121,7 @@ def _do_build_lxd_exitstack( |
771 | built_temp_contents = new_temp_contents - orig_temp_contents |
772 | cmd = [lxc, 'file', 'pull'] |
773 | cmd.extend([ |
774 | - '%s%s' % (container_name, os.path.join(containertemp, f)) |
775 | + '%s%s' % (container_name, os.path.join('/tmp', f)) |
776 | for f in built_temp_contents |
777 | ] |
778 | ) |
779 | @@ -1080,9 +1131,39 @@ def _do_build_lxd_exitstack( |
780 | return [os.path.join(os.pardir, f) for f in built_temp_contents] |
781 | |
782 | |
783 | -def fetch_orig_and_build( |
784 | +def fetch_orig( |
785 | orig_search_list, |
786 | changelog, |
787 | +): |
788 | + unaliased_orig_search_list = expand_changelog_source_aliases( |
789 | + orig_search_list, |
790 | + changelog, |
791 | + ) |
792 | + # Follow searches already attempted as a 'changelog' source alias |
793 | + # may expand to a duplicate. |
794 | + for entry in unique_everseen(unaliased_orig_search_list): |
795 | + try: |
796 | + mechanism_name = entry.mechanism.__name__ |
797 | + except AttributeError: |
798 | + mechanism_name = entry.mechanism.func.__name__ |
799 | + tarballs = entry.mechanism(changelog, entry.source) |
800 | + if tarballs is None: |
801 | + logging.debug('%s(source=%s) failed', |
802 | + mechanism_name, |
803 | + entry.source, |
804 | + ) |
805 | + continue # search returned negative; try next search entry |
806 | + logging.info('Successfully fetched%susing %s(source=%s)', |
807 | + ':\n' + '\n'.join(tarballs) + '\n' if tarballs else ' ', |
808 | + mechanism_name, |
809 | + entry.source, |
810 | + ) |
811 | + return tarballs |
812 | + |
813 | +def fetch_orig_and_build( |
814 | + repo, |
815 | + commitish, |
816 | + orig_search_list, |
817 | rem_args=[], |
818 | use_lxd=True, |
819 | keep_build_env=False, |
820 | @@ -1091,6 +1172,7 @@ def fetch_orig_and_build( |
821 | retries=top_level_defaults.retries, |
822 | retry_backoffs=top_level_defaults.retry_backoffs, |
823 | ): |
824 | + changelog = repo.get_changelog_from_treeish(commitish) |
825 | unaliased_orig_search_list = expand_changelog_source_aliases( |
826 | orig_search_list, |
827 | changelog, |
828 | @@ -1111,7 +1193,8 @@ def fetch_orig_and_build( |
829 | continue # search returned negative; try next search entry |
830 | try: |
831 | built_files = do_build( |
832 | - changelog, |
833 | + repo, |
834 | + commitish, |
835 | tarballs, |
836 | rem_args, |
837 | use_lxd, |
838 | @@ -1133,6 +1216,9 @@ def fetch_orig_and_build( |
839 | # Build failed, but the search list entry declares that |
840 | # this should not be fatal and so we try the next search |
841 | # entry |
842 | + # First, remove any tarballs fetched with this mechanism |
843 | + for tarball in tarballs: |
844 | + os.remove(tarball) |
845 | logging.debug( |
846 | "Ignoring failure to build using %s(source=%s)", |
847 | mechanism_name, |
848 | diff --git a/gitubuntu/buildsource.py b/gitubuntu/buildsource.py |
849 | index 833ae74..dfbef8c 100644 |
850 | --- a/gitubuntu/buildsource.py |
851 | +++ b/gitubuntu/buildsource.py |
852 | @@ -8,7 +8,7 @@ from subprocess import CalledProcessError |
853 | import sys |
854 | import gitubuntu.build |
855 | from gitubuntu.cache import CACHE_PATH |
856 | -from gitubuntu.git_repository import Changelog |
857 | +import gitubuntu.git_repository |
858 | from gitubuntu.run import run |
859 | |
860 | def parse_args(subparsers=None, base_subparsers=None): |
861 | @@ -74,8 +74,16 @@ def parse_args(subparsers=None, base_subparsers=None): |
862 | help="LXD image to use. If not specified, the image will be " |
863 | "derived from distribution value of HEAD:debian/changelog.", |
864 | ) |
865 | - parser.add_argument('rem_args', help=argparse.SUPPRESS, |
866 | - nargs=argparse.REMAINDER |
867 | + parser.add_argument( |
868 | + '--commitish', |
869 | + type=str, |
870 | + help="Git commitish to build.", |
871 | + default='HEAD', |
872 | + ) |
873 | + parser.add_argument( |
874 | + 'rem_args', |
875 | + help=argparse.SUPPRESS, |
876 | + nargs=argparse.REMAINDER, |
877 | ) |
878 | if not subparsers: |
879 | return parser.parse_args() |
880 | @@ -96,33 +104,30 @@ def cli_main(args): |
881 | if args.for_merge: |
882 | args.rem_args += ['-sa'] |
883 | |
884 | - changelog = Changelog.from_path('debian/changelog') |
885 | + repo = gitubuntu.git_repository.GitUbuntuRepository('.') |
886 | |
887 | try: |
888 | - native = gitubuntu.build.is_native_package(changelog) |
889 | + orig_search_list = gitubuntu.build.derive_orig_search_list_from_args( |
890 | + repo, |
891 | + args.commitish, |
892 | + args.for_merge, |
893 | + args.no_pristine_tar, |
894 | + args.pullfile, |
895 | + args.retries, |
896 | + args.retry_backoffs, |
897 | + args.dl_cache, |
898 | + ) |
899 | except gitubuntu.build.NativenessMismatchError as e: |
900 | logging.error("%s" % e) |
901 | return 1 |
902 | |
903 | - if native: |
904 | - # No orig tarball required |
905 | - orig_search_list = [ |
906 | - gitubuntu.build.OrigSearchListEntry( |
907 | - mechanism=gitubuntu.build.fetch_orig_noop, |
908 | - source=[], |
909 | - must_build=True, |
910 | - ), |
911 | - ] |
912 | - else: |
913 | - orig_search_list = gitubuntu.build.derive_orig_search_list_from_args( |
914 | - args, |
915 | - ) |
916 | - |
917 | if len( |
918 | gitubuntu.build.main( |
919 | + repo, |
920 | + args.commitish, |
921 | orig_search_list, |
922 | - changelog, |
923 | rem_args, |
924 | + for_merge=args.for_merge, |
925 | sign=args.sign, |
926 | use_lxd=not args.no_lxd, |
927 | keep_build_env=args.keep_build_env, |
928 | diff --git a/gitubuntu/git_repository.py b/gitubuntu/git_repository.py |
929 | index 58d5a09..dffeaae 100644 |
930 | --- a/gitubuntu/git_repository.py |
931 | +++ b/gitubuntu/git_repository.py |
932 | @@ -16,7 +16,9 @@ from subprocess import CalledProcessError |
933 | import sys |
934 | import tempfile |
935 | import time |
936 | +import gitubuntu.lint |
937 | from gitubuntu.__main__ import top_level_defaults |
938 | +import gitubuntu.build |
939 | from gitubuntu.dsc import component_tarball_matches |
940 | from gitubuntu.run import run, runq, decode_binary |
941 | try: |
942 | @@ -149,23 +151,23 @@ class Changelog: |
943 | |
944 | @lru_cache() |
945 | def _dpkg_parsechangelog(self, parse_params): |
946 | - cp = run( |
947 | + stdout, _ = run( |
948 | 'dpkg-parsechangelog -l- %s' % parse_params, |
949 | input=self._contents, |
950 | shell=True, |
951 | verbose_on_failure=False, |
952 | ) |
953 | - return decode_binary(cp.stdout).strip() |
954 | + return stdout |
955 | |
956 | @lru_cache() |
957 | def _shell(self, cmd): |
958 | - cp = run( |
959 | + stdout, _ = run( |
960 | cmd, |
961 | input=self._contents, |
962 | shell=True, |
963 | verbose_on_failure=False, |
964 | ) |
965 | - return decode_binary(cp.stdout).strip() |
966 | + return stdout |
967 | |
968 | @property |
969 | def _shell_version(self): |
970 | @@ -408,6 +410,27 @@ def upstream_tag(version, namespace): |
971 | def orphan_tag(version, namespace): |
972 | return '%s/orphan/%s' % (namespace, git_dep14_tag(version)) |
973 | |
974 | +def is_dir_3_0_quilt(_dir=None): |
975 | + _dir = _dir if _dir else '.' |
976 | + try: |
977 | + fmt, _ = run(['dpkg-source', '--print-format', _dir]) |
978 | + if '3.0 (quilt)' in fmt: |
979 | + return True |
980 | + except CalledProcessError as e: |
981 | + try: |
982 | + with open(os.path.join(_dir, 'debian/source/format'), 'r') as f: |
983 | + for line in f: |
984 | + if re.match(r'3.0 (.*)', line): |
985 | + return True |
986 | + # `man dpkg-source` indicates no d/s/format implies 1.0 |
987 | + except OSError: |
988 | + pass |
989 | + |
990 | + return False |
991 | + |
992 | +def is_3_0_quilt(repo, commitish='HEAD'): |
993 | + with repo.temporary_worktree(commitish): |
994 | + return is_dir_3_0_quilt() |
995 | |
996 | class GitUbuntuRepositoryFetchError(Exception): |
997 | pass |
998 | @@ -465,11 +488,10 @@ class GitUbuntuRepository: |
999 | self._lp_user = lp_user |
1000 | else: |
1001 | try: |
1002 | - cp = self.git_run( |
1003 | + self._lp_user, _ = self.git_run( |
1004 | ['config', 'gitubuntu.lpuser'], |
1005 | verbose_on_failure=False, |
1006 | ) |
1007 | - self._lp_user = decode_binary(cp.stdout).strip() |
1008 | except CalledProcessError: |
1009 | logging.error("Unable to determine Launchpad user") |
1010 | sys.exit(1) |
1011 | @@ -554,8 +576,8 @@ class GitUbuntuRepository: |
1012 | |
1013 | def pristine_tar_list(self, dist, namespace='pkg'): |
1014 | with self.pristine_tar_branches(dist, namespace): |
1015 | - cp = run(['pristine-tar', 'list']) |
1016 | - return decode_binary(cp.stdout).strip().splitlines() |
1017 | + stdout, _ = run(['pristine-tar', 'list']) |
1018 | + return stdout.splitlines() |
1019 | |
1020 | def pristine_tar_extract(self, pkgname, version, dist=None, namespace='pkg'): |
1021 | '''Extract orig tarballs for a given package and upstream version |
1022 | @@ -737,6 +759,26 @@ class GitUbuntuRepository: |
1023 | # https://github.com/libgit2/pygit2/issues/671 |
1024 | return any(remote.name == remote_name for remote in self.raw_repo.remotes) |
1025 | |
1026 | + def _add_remote_by_fetch_url(self, remote_name, fetch_url): |
1027 | + if not self._fetch_proto: |
1028 | + raise Exception('Cannot fetch using an object without a protocol') |
1029 | + |
1030 | + logging.debug('Adding %s as remote %s', fetch_url, remote_name) |
1031 | + |
1032 | + if not self.remote_exists(remote_name): |
1033 | + self.raw_repo.remotes.create(remote_name, fetch_url, |
1034 | + 'refs/heads/*:refs/remotes/%s/*' % remote_name) |
1035 | + # grab unreachable tags (orphans) |
1036 | + self.raw_repo.remotes.add_fetch(remote_name, |
1037 | + 'refs/tags/*:refs/tags/%s/*' % remote_name) |
1038 | + self.git_run( |
1039 | + [ |
1040 | + 'config', |
1041 | + 'remote.%s.tagOpt' % remote_name, |
1042 | + '--no-tags', |
1043 | + ] |
1044 | + ) |
1045 | + |
1046 | def _add_remote(self, remote_name, remote_url): |
1047 | if not self._fetch_proto: |
1048 | raise Exception('Cannot fetch using an object without a protocol') |
1049 | @@ -770,6 +812,12 @@ class GitUbuntuRepository: |
1050 | |
1051 | self._add_remote(remote_name, remote_url) |
1052 | |
1053 | + def add_remote_by_url(self, remote_name, fetch_url): |
1054 | + if not self._fetch_proto: |
1055 | + raise Exception('Cannot fetch using an object without a protocol') |
1056 | + |
1057 | + self._add_remote_by_fetch_url(remote_name, fetch_url) |
1058 | + |
1059 | def add_base_remotes(self, pkgname, repo_owner='usd-import-team'): |
1060 | self.add_remote(pkgname, repo_owner, 'pkg') |
1061 | |
1062 | @@ -798,7 +846,7 @@ class GitUbuntuRepository: |
1063 | kwargs['stderr'] = None |
1064 | try: |
1065 | logging.debug("Fetching remote %s", remote_name) |
1066 | - cp = self.git_run( |
1067 | + self.git_run( |
1068 | args=['fetch', remote_name], |
1069 | env={'GIT_TERMINAL_PROMPT': '0',}, |
1070 | **kwargs |
1071 | @@ -950,7 +998,8 @@ class GitUbuntuRepository: |
1072 | |
1073 | Note that the hash may still become ambiguous in the future. |
1074 | """ |
1075 | - return self.git_run(['rev-parse', '--short', hash]).stdout.decode().strip() |
1076 | + stdout, _ = self.git_run(['rev-parse', '--short', hash]) |
1077 | + return stdout |
1078 | |
1079 | def git_run(self, args, env=None, **kwargs): |
1080 | # Explicitly take a copy of self._env so the following update doesn't |
1081 | @@ -1140,6 +1189,29 @@ class GitUbuntuRepository: |
1082 | return set() |
1083 | |
1084 | |
1085 | + def nearest_tag( |
1086 | + self, |
1087 | + commitish_string, |
1088 | + prefix, |
1089 | + max_commits=100, |
1090 | + ): |
1091 | + # 1) cache all patterned tag names by commit |
1092 | + pattern_tags_by_commit = collections.defaultdict(set) |
1093 | + for t in self.tags: |
1094 | + if t.name.startswith('refs/tags/' + prefix): |
1095 | + pattern_tags_by_commit[t.peel(pygit2.Commit).id].add(t) |
1096 | + |
1097 | + commits = self.raw_repo.walk( |
1098 | + self.get_commitish(commitish_string).id, |
1099 | + pygit2.GIT_SORT_TOPOLOGICAL, |
1100 | + ) |
1101 | + for commit in itertools.islice(commits, max_commits): |
1102 | + if commit.id not in pattern_tags_by_commit: |
1103 | + continue |
1104 | + |
1105 | + return pattern_tags_by_commit[commit.id].pop() |
1106 | + |
1107 | + return None |
1108 | |
1109 | @staticmethod |
1110 | def tag_to_pretty_name(tag): |
1111 | @@ -1349,7 +1421,8 @@ class GitUbuntuRepository: |
1112 | fp.flush() |
1113 | commit_tree += ['-F', fp.name] |
1114 | try: |
1115 | - cp = run(commit_tree, env=commit_env) |
1116 | + stdout, _ = run(commit_tree, env=commit_env) |
1117 | + return stdout |
1118 | except CalledProcessError: |
1119 | _, t = tempfile.mkstemp() |
1120 | shutil.copy(fp.name, t) |
1121 | @@ -1357,8 +1430,6 @@ class GitUbuntuRepository: |
1122 | "temp commit message as %s", t) |
1123 | sys.exit(1) |
1124 | |
1125 | - return decode_binary(cp.stdout).strip() |
1126 | - |
1127 | @classmethod |
1128 | def _create_replacement_tree_builder(cls, repo, treeish, sub_path): |
1129 | '''Create a replacement TreeBuilder |
1130 | @@ -1467,40 +1538,51 @@ class GitUbuntuRepository: |
1131 | with tempfile.TemporaryDirectory() as index_dir: |
1132 | index_path = os.path.join(index_dir, 'index') |
1133 | index_env = {'GIT_INDEX_FILE': index_path} |
1134 | - git_dir = os.path.join(path, '.git') |
1135 | - if os.path.exists(git_dir): |
1136 | - logging.warning('.git directory found in source ' |
1137 | - 'package. Will remove it.') |
1138 | - shutil.rmtree(git_dir) |
1139 | self.git_run( |
1140 | ['--work-tree', path, 'add', '-f', '-A'], |
1141 | env=index_env, |
1142 | ) |
1143 | - cp = self.git_run( |
1144 | + self.git_run( |
1145 | + ['--work-tree', path, 'reset', 'HEAD', '--', '.git',], |
1146 | + env=index_env, |
1147 | + ) |
1148 | + self.git_run( |
1149 | + ['--work-tree', path, 'reset', 'HEAD', '--', '.pc',], |
1150 | + env=index_env, |
1151 | + ) |
1152 | + tree_hash_str, _ = self.git_run( |
1153 | ['--work-tree', path, 'write-tree'], |
1154 | env=index_env, |
1155 | ) |
1156 | - tree_hash_str = decode_binary(cp.stdout).strip() |
1157 | - tree = self.raw_repo.get(tree_hash_str) |
1158 | - |
1159 | - # Add any empty directories that git did not import. Workaround for LP: |
1160 | - # #1687057. |
1161 | - replacement_oid = self._add_missing_tree_dirs( |
1162 | - repo=self.raw_repo, |
1163 | - top_path=path, |
1164 | - top_tree_object=tree, |
1165 | - ) |
1166 | - if replacement_oid: |
1167 | - # Empty directories had to be added |
1168 | - return str(replacement_oid) # return the replacement instead |
1169 | - else: |
1170 | - # No empty directories were added |
1171 | - return tree_hash_str # no replacement was needed |
1172 | + tree = self.raw_repo.get(tree_hash_str) |
1173 | + |
1174 | + # Add any empty directories that git did not import. Workaround for LP: |
1175 | + # #1687057. |
1176 | + replacement_oid = self._add_missing_tree_dirs( |
1177 | + repo=self.raw_repo, |
1178 | + top_path=path, |
1179 | + top_tree_object=tree, |
1180 | + ) |
1181 | + if replacement_oid: |
1182 | + # Empty directories had to be added |
1183 | + return str(replacement_oid) # return the replacement instead |
1184 | + else: |
1185 | + # No empty directories were added |
1186 | + return tree_hash_str # no replacement was needed |
1187 | |
1188 | @contextmanager |
1189 | - def temporary_worktree(self, commitish): |
1190 | - with tempfile.TemporaryDirectory() as tempdir: |
1191 | - self.git_run(['worktree', 'add', '--force', tempdir, commitish]) |
1192 | + def temporary_worktree(self, commitish, prefix=None): |
1193 | + with tempfile.TemporaryDirectory(prefix=prefix) as tempdir: |
1194 | + self.git_run( |
1195 | + [ |
1196 | + 'worktree', |
1197 | + 'add', |
1198 | + '--detach', |
1199 | + '--force', |
1200 | + tempdir, |
1201 | + commitish, |
1202 | + ] |
1203 | + ) |
1204 | |
1205 | oldcwd = os.getcwd() |
1206 | os.chdir(tempdir) |
1207 | @@ -1523,6 +1605,429 @@ class GitUbuntuRepository: |
1208 | raise |
1209 | |
1210 | run(["git", "add", "-f", ".",]) |
1211 | - cp = run(["git", "write-tree"]) |
1212 | - tree_hash = decode_binary(cp.stdout).strip() |
1213 | - return tree_hash |
1214 | + tree_hash, _ = run(["git", "write-tree"]) |
1215 | + return tree_hash |
1216 | + |
1217 | + def tree_hash_subpath(self, treeish_string, path): |
1218 | + """Get the tree hash for path at a given treeish |
1219 | + |
1220 | + Arguments: |
1221 | + @treeish_string: a string Git treeish |
1222 | + @path: a string path present in @treeish_string |
1223 | + |
1224 | + Returns: |
1225 | + String hash of Git tree corresponding to @path in @treeish_string |
1226 | + """ |
1227 | + tree_obj = self.raw_repo.revparse_single(treeish_string).peel( |
1228 | + pygit2.Tree |
1229 | + ) |
1230 | + return str(tree_obj[path].id) |
1231 | + |
1232 | + def paths_are_identical(self, treeish1_string, treeish2_string, path): |
1233 | + """Determine if a given path is the same in two treeishs |
1234 | + |
1235 | + Arguments: |
1236 | + @treeish1_string: a string Git treeish |
1237 | + @treeish2_string: a string Git treeish |
1238 | + @path: a string path present in @treeish1_string and @treeish2_string |
1239 | + |
1240 | + Returns: |
1241 | + True, if @path is the same in @treeish1_string and @treeish2_string |
1242 | + False, otherwise |
1243 | + """ |
1244 | + return self.tree_hash_subpath( |
1245 | + treeish1_string, |
1246 | + path, |
1247 | + ) == self.tree_hash_subpath( |
1248 | + treeish2_string, |
1249 | + path, |
1250 | + ) |
1251 | + |
1252 | + @lru_cache() |
1253 | + def quilt_env(self, commit_hash): |
1254 | + with self.temporary_worktree(commit_hash): |
1255 | + combined_env = self.env.copy() |
1256 | + combined_env.update( |
1257 | + { |
1258 | + 'QUILT_PATCHES': 'debian/patches', |
1259 | + 'QUILT_NO_DIFF_INDEX': '1', |
1260 | + 'QUILT_NO_DIFF_TIMESTAMPS': '1', |
1261 | + 'EDITOR': 'true', |
1262 | + } |
1263 | + ) |
1264 | + for path in [ |
1265 | + os.path.join('debian', 'patches', 'debian.series'), |
1266 | + os.path.join('debian', 'patches', 'series'), |
1267 | + ]: |
1268 | + if os.path.exists(path): |
1269 | + combined_env.update({'QUILT_SERIES': path,}) |
1270 | + break |
1271 | + else: |
1272 | + logging.debug( |
1273 | + "Unable to find a series file in %s", |
1274 | + commit_hash |
1275 | + ) |
1276 | + |
1277 | + return combined_env |
1278 | + |
1279 | + def is_patches_applied(self, commit_hash, regenerated_pc_path): |
1280 | + # first see if quilt push -a would do anything to |
1281 | + # differentiate between applied an unapplied |
1282 | + with self.temporary_worktree(commit_hash): |
1283 | + try: |
1284 | + run(['quilt', 'push', '-a'], env=self.quilt_env(commit_hash)) |
1285 | + # False if in an unapplied state, which is signified by |
1286 | + # successful push (rc=0) |
1287 | + return False |
1288 | + except CalledProcessError as e: |
1289 | + # non-zero return might be an error or it might mean no |
1290 | + # patches exist |
1291 | + if e.returncode == 1: |
1292 | + # an error may occur if we need to recreate the .pc |
1293 | + # first |
1294 | + try: |
1295 | + # the first quilt push may have created a .pc/ |
1296 | + shutil.rmtree('.pc') |
1297 | + shutil.copytree( |
1298 | + regenerated_pc_path, |
1299 | + '.pc', |
1300 | + ) |
1301 | + except FileNotFoundError: |
1302 | + # if there was no .pc directory, then the first |
1303 | + # quilt push failure was a real error |
1304 | + raise e |
1305 | + |
1306 | + try: |
1307 | + run(['quilt', 'push', '-a'], env=self.quilt_env(commit_hash)) |
1308 | + # False if in an unapplied state |
1309 | + return False |
1310 | + except CalledProcessError as e: |
1311 | + # True if in a patches-applied state or |
1312 | + # there are no patches to apply |
1313 | + if e.returncode == 2: |
1314 | + return True |
1315 | + else: |
1316 | + raise |
1317 | + # True if in a patches-applied state or there are |
1318 | + # no patches to apply |
1319 | + elif e.returncode == 2: |
1320 | + return True |
1321 | + else: |
1322 | + raise |
1323 | + |
1324 | + def _maybe_quiltify_tree_hash(self, commit_hash): |
1325 | + """Determine if quiltify is needed and yield the quiltify'd tree hash |
1326 | + |
1327 | + The imported patches-applied trees do not contain .pc |
1328 | + directories. To determine if an additional quilt patch is |
1329 | + necessary, we have to first regenerate the .pc directory, then |
1330 | + see if dpkg-source --commit generates a new quilt patch. |
1331 | + |
1332 | + In order for dpkg-source --commit to function, we need to know |
1333 | + if the commit we are building is patches-unapplied or |
1334 | + patches-applied. In the latter case, we can build the commit |
1335 | + directly after copying the regenerated .pc directory. In the |
1336 | + former case, we do not want to copy the regenerated .pc |
1337 | + directory, as dpkg-source will do this for us, as it applies the |
1338 | + current patches. We determine if patches are applied or |
1339 | + unapplied by relying `quilt push -a`'s exit status at |
1340 | + @commit_hash. |
1341 | + |
1342 | + This is a common method used by multiple callers. |
1343 | + |
1344 | + Arguments: |
1345 | + @commit_hash: a string Git commit hash |
1346 | + |
1347 | + Returns: |
1348 | + String tree hash of quiltify'ing @commit_hash. |
1349 | + If no quiltify is needed, the return value is @commit_hash's |
1350 | + tree hash |
1351 | + """ |
1352 | + # we need the orig tarballs for quilt and dpkg-source |
1353 | + # but suppress any logging |
1354 | + logger = logging.getLogger() |
1355 | + oldLevel = logger.getEffectiveLevel() |
1356 | + logger.setLevel(logging.WARNING) |
1357 | + tarballs = gitubuntu.build.fetch_orig( |
1358 | + orig_search_list=gitubuntu.build.derive_orig_search_list_from_args( |
1359 | + self, |
1360 | + commitish=commit_hash, |
1361 | + for_merge=False, |
1362 | + no_pristine_tar=False, |
1363 | + ), |
1364 | + changelog=Changelog.from_path('debian/changelog'), |
1365 | + ) |
1366 | + logger.setLevel(oldLevel) |
1367 | + commit_tree_hash = str( |
1368 | + self.raw_repo.get(commit_hash).peel(pygit2.Tree).id |
1369 | + ) |
1370 | + # the tarballs need to be in the parent directory from where we |
1371 | + # run dpkg-source |
1372 | + with tempfile.TemporaryDirectory() as tempdir: |
1373 | + # copy the generated tarballs |
1374 | + new_tarballs = [] |
1375 | + for tarball in tarballs: |
1376 | + new_tarballs.append(shutil.copy(tarball, tempdir)) |
1377 | + tarballs = new_tarballs |
1378 | + |
1379 | + # create a nested temporary directory where we will recreate |
1380 | + # the .pc directory |
1381 | + with tempfile.TemporaryDirectory(prefix=tempdir+'/') as ttempdir: |
1382 | + oldcwd = os.getcwd() |
1383 | + os.chdir(ttempdir) |
1384 | + |
1385 | + for tarball in tarballs: |
1386 | + run(['tar', '-x', '--strip-components=1', '-f', tarball,]) |
1387 | + |
1388 | + # need the debia/patches |
1389 | + shutil.copytree( |
1390 | + os.path.join(self.local_dir, 'debian',), |
1391 | + 'debian', |
1392 | + ) |
1393 | + |
1394 | + # generate the equivalent .pc directory |
1395 | + run( |
1396 | + ['quilt', 'push', '-a'], |
1397 | + env=self.quilt_env(commit_hash), |
1398 | + rcs=[2], |
1399 | + ) |
1400 | + |
1401 | + regenerated_pc_path = os.path.join(tempdir, '.pc') |
1402 | + shutil.copytree( |
1403 | + '.pc', |
1404 | + regenerated_pc_path, |
1405 | + ) |
1406 | + |
1407 | + os.chdir(oldcwd) |
1408 | + |
1409 | + patches_applied = self.is_patches_applied( |
1410 | + commit_hash, |
1411 | + regenerated_pc_path, |
1412 | + ) |
1413 | + |
1414 | + with self.temporary_worktree(commit_hash, prefix=tempdir+'/'): |
1415 | + # we only need to copy the generated .pc directory |
1416 | + # if we are building a patches-applied tree, which |
1417 | + # we determine by comparing our current tree hash to |
1418 | + # the generated tree hash. |
1419 | + if patches_applied: |
1420 | + try: |
1421 | + shutil.copytree( |
1422 | + regenerated_pc_path, |
1423 | + '.pc', |
1424 | + ) |
1425 | + except FileNotFoundError: |
1426 | + # it is possible no quilt patches exist yet |
1427 | + pass |
1428 | + |
1429 | + fixup_patch_path = os.path.join( |
1430 | + 'debian', |
1431 | + 'patches', |
1432 | + 'git-ubuntu-fixup.patch' |
1433 | + ) |
1434 | + |
1435 | + if os.path.exists(fixup_patch_path): |
1436 | + raise ValueError( |
1437 | + "A quilt patch with the name git-ubuntu-fixup.patch " |
1438 | + "already exists in %s" % commit_hash |
1439 | + ) |
1440 | + |
1441 | + run( |
1442 | + [ |
1443 | + 'dpkg-source', |
1444 | + '--commit', |
1445 | + '.', |
1446 | + 'git-ubuntu-fixup.patch', |
1447 | + ], |
1448 | + env=self.quilt_env(commit_hash), |
1449 | + ) |
1450 | + |
1451 | + # do not want the .pc directory in the resulting |
1452 | + # treeish |
1453 | + shutil.rmtree('.pc') |
1454 | + |
1455 | + if os.path.exists(fixup_patch_path): |
1456 | + # dpkg-source uses debian/changelog to generate some |
1457 | + # fields. We do not know yet if the changelog has |
1458 | + # been updated, so elide that section of comments. |
1459 | + with open(fixup_patch_path, 'r+') as f: |
1460 | + for line in f: |
1461 | + if '---' in line: |
1462 | + break |
1463 | + text = """Description: git-ubuntu generated quilt fixup patch |
1464 | +TODO: Put a short summary on the line above and replace this paragraph |
1465 | +with a longer explanation of this change. Complete the meta-information |
1466 | +with other relevant fields (see below for details). |
1467 | +---\n""" |
1468 | + for line in f: |
1469 | + text += line |
1470 | + f.seek(0) |
1471 | + f.write(text) |
1472 | + f.truncate() |
1473 | + |
1474 | + # If we are on a patches-unapplied tree, then we |
1475 | + # need to reset ourselves back to @commit_hash with |
1476 | + # our new patch. |
1477 | + # In order for this to be buildable, we have to |
1478 | + # reverse-apply our patch, to undo the git-commited |
1479 | + # upstream changes. |
1480 | + if not patches_applied: |
1481 | + run(['git', 'add', '-f', 'debian/patches',]) |
1482 | + # if any patches add files that are untracked, |
1483 | + # remove them |
1484 | + run(['git', 'clean', '-f', '-d',]) |
1485 | + # reset all the other files to their status in |
1486 | + # HEAD |
1487 | + run(['git', 'checkout', commit_hash, '--', '*',]) |
1488 | + with open(fixup_patch_path, 'rb') as f: |
1489 | + run(['patch', '-Rp1',], input=f.read()) |
1490 | + |
1491 | + return self.dir_to_tree('.') |
1492 | + else: |
1493 | + return commit_tree_hash |
1494 | + |
1495 | + def maybe_quiltify_tree_hash(self, commitish_string): |
1496 | + """Determine if quiltify is needed and return the quiltify'd tree hash |
1497 | + |
1498 | + See _maybe_quiltify_tree_hash for details. |
1499 | + |
1500 | + Arguments: |
1501 | + @commitish_string: a string Git commitish |
1502 | + |
1503 | + Returns: |
1504 | + String tree hash of quiltify'ing @commitish_string. |
1505 | + If no quiltify is needed, the return value is the tree hash of |
1506 | + @commitish_string. |
1507 | + """ |
1508 | + commit_hash = str( |
1509 | + self.get_commitish(commitish_string).peel(pygit2.Commit).id |
1510 | + ) |
1511 | + return self._maybe_quiltify_tree_hash(commit_hash) |
1512 | + |
1513 | + def maybe_changelogify_tree_hash(self, commit_hash): |
1514 | + """Determine if changelogify is needed and yield the changelogify'd tree hash |
1515 | + |
1516 | + Given a commit, we need to detect if the user has inserted a |
1517 | + changelog entry relative to a published version for the purpose |
1518 | + of test builds. |
1519 | + |
1520 | + Arguments: |
1521 | + @commit_hash: a string Git commit hash |
1522 | + |
1523 | + Returns: |
1524 | + String tree hash of changelogify'ing @commit_hash. |
1525 | + If no changelogify is needed, the return value is the tree hash of |
1526 | + @commit_hash. |
1527 | + """ |
1528 | + commit_tree_hash = str( |
1529 | + self.raw_repo.get(commit_hash).peel(pygit2.Tree).id |
1530 | + ) |
1531 | + |
1532 | + # one of these are the "base" pkg that @commit_hash's changes |
1533 | + # are based on |
1534 | + remote_tag = self.nearest_tag( |
1535 | + commit_hash, |
1536 | + prefix='pkg/', |
1537 | + ) |
1538 | + remote_branch = gitubuntu.lint.derive_target_branch( |
1539 | + self, |
1540 | + commit_hash, |
1541 | + ) |
1542 | + |
1543 | + assert remote_tag or remote_branch |
1544 | + |
1545 | + if remote_tag: |
1546 | + if remote_branch: |
1547 | + try: |
1548 | + self.git_run( |
1549 | + [ |
1550 | + 'merge-base', |
1551 | + '--is-ancestor', |
1552 | + remote_tag.name, |
1553 | + remote_branch, |
1554 | + ], |
1555 | + verbose_on_failure=False, |
1556 | + ) |
1557 | + parent_ref = remote_branch |
1558 | + except CalledProcessError as e: |
1559 | + if e.returncode == 1: |
1560 | + parent_ref = remote_tag.name |
1561 | + else: |
1562 | + raise |
1563 | + else: |
1564 | + parent_ref = remote_tag.name |
1565 | + else: |
1566 | + parent_ref = remote_branch |
1567 | + |
1568 | + # If there are any changes relative to parent_ref but there are |
1569 | + # not any changelog changes, insert a snapshot changelog entry, |
1570 | + # starting from parent_ref, and return the resulting tree hash. |
1571 | + if str(self.raw_repo.revparse_single(parent_ref).peel( |
1572 | + pygit2.Tree |
1573 | + ).id) != commit_tree_hash and self.paths_are_identical( |
1574 | + parent_ref, |
1575 | + commit_hash, |
1576 | + 'debian/changelog', |
1577 | + ): |
1578 | + with self.temporary_worktree(commit_hash): |
1579 | + run( |
1580 | + [ |
1581 | + 'gbp', |
1582 | + 'dch', |
1583 | + '--snapshot', |
1584 | + '--ignore-branch', |
1585 | + '--since=%s' % str(parent_ref), |
1586 | + ] |
1587 | + ) |
1588 | + return self.dir_to_tree('.') |
1589 | + |
1590 | + # otherwise, return @commit_hash's tree hash |
1591 | + return commit_tree_hash |
1592 | + |
1593 | + def quiltify_and_changelogify_tree_hash(self, commitish_string): |
1594 | + """Given a commitish, possibly quiltify and changelogify its tree |
1595 | + |
1596 | + Definitions: |
1597 | + quiltify: generate a quilt patch from untracked upstream |
1598 | + changes |
1599 | + changelogify: generate a snapshot changelog entry if any |
1600 | + changes exist, and no new changelog entry yet exists |
1601 | + |
1602 | + Arguments: |
1603 | + @commitish_string: string Git commitish |
1604 | + |
1605 | + Returns: |
1606 | + string Git tree hash of quiltify-ing and changelogify-ing |
1607 | + @commitish_string, if needed |
1608 | + if neither quiltify or changelogify are needed, return |
1609 | + @commitish_string's tree hash |
1610 | + """ |
1611 | + commit_hash = str( |
1612 | + self.get_commitish(commitish_string).peel(pygit2.Commit).id |
1613 | + ) |
1614 | + quiltify_tree_hash = self._maybe_quiltify_tree_hash(commit_hash) |
1615 | + changelogify_tree_hash = self.maybe_changelogify_tree_hash(commit_hash) |
1616 | + |
1617 | + quiltify_tree_obj = self.raw_repo.get(quiltify_tree_hash) |
1618 | + changelogify_tree_obj = self.raw_repo.get(changelogify_tree_hash) |
1619 | + |
1620 | + # There are multiple ways to solve this problem, but the |
1621 | + # simplest is to use a TreeBuilder to merge the quiltify tree |
1622 | + # with the changelog from the changelogify tree |
1623 | + # top-level TreeBuilder |
1624 | + tb = self.raw_repo.TreeBuilder(quiltify_tree_obj) |
1625 | + te = tb.get('debian') |
1626 | + # TreeBuilder for debian/ |
1627 | + dtb = self.raw_repo.TreeBuilder(self.raw_repo.get(te.id)) |
1628 | + dtb.insert( # does not take kwargs |
1629 | + 'changelog', # name |
1630 | + changelogify_tree_obj['debian/changelog'].oid, # oid |
1631 | + pygit2.GIT_FILEMODE_BLOB, # attr |
1632 | + ) |
1633 | + # insert can replace |
1634 | + tb.insert( # does not take kwargs |
1635 | + 'debian', # name |
1636 | + dtb.write(), # oid |
1637 | + pygit2.GIT_FILEMODE_TREE, # attr |
1638 | + ) |
1639 | + return str(tb.write()) |
1640 | diff --git a/gitubuntu/importer.py b/gitubuntu/importer.py |
1641 | index 7248ee6..f438739 100644 |
1642 | --- a/gitubuntu/importer.py |
1643 | +++ b/gitubuntu/importer.py |
1644 | @@ -50,6 +50,7 @@ from gitubuntu.git_repository import ( |
1645 | import_tag, |
1646 | upstream_tag, |
1647 | PristineTarError, |
1648 | + is_dir_3_0_quilt, |
1649 | ) |
1650 | from gitubuntu.run import decode_binary, run, runq |
1651 | from gitubuntu.source_information import ( |
1652 | @@ -141,6 +142,7 @@ def main( |
1653 | skip_orig, |
1654 | skip_applied, |
1655 | reimport, |
1656 | + allow_applied_failures, |
1657 | directory=None, |
1658 | dl_cache=None, |
1659 | user=None, |
1660 | @@ -167,6 +169,9 @@ def main( |
1661 | publishes |
1662 | @reimport: if True, import the source package from scratch and |
1663 | delete/recreate the target repository. |
1664 | + @allow_import_failures: if True, and patches fail to apply for any |
1665 | + publish, that patches-applied import will be skipped rather than an |
1666 | + error |
1667 | @directory: string path for local repository |
1668 | @user: string user to authenticate to Launchpad as |
1669 | @proto: string protocol to use (one of 'http', 'https', 'git') |
1670 | @@ -329,6 +334,7 @@ def main( |
1671 | workdir=workdir, |
1672 | fixup_devel=fixup_devel, |
1673 | skip_orig=skip_orig, |
1674 | + allow_applied_failures=allow_applied_failures, |
1675 | ) |
1676 | |
1677 | if not history_found: |
1678 | @@ -350,6 +356,7 @@ def main( |
1679 | workdir=workdir, |
1680 | fixup_devel=fixup_devel, |
1681 | skip_orig=skip_orig, |
1682 | + allow_applied_failures=allow_applied_failures, |
1683 | ) |
1684 | |
1685 | os.chdir(oldcwd) |
1686 | @@ -424,11 +431,11 @@ def get_changelog_for_commit( |
1687 | if changelog_parent_commit is not None: |
1688 | cmd = ['diff-tree', '-p', changelog_parent_commit, |
1689 | tree_hash, '--', 'debian/changelog'] |
1690 | - raw_clog_entry = repo.git_run(cmd).stdout |
1691 | + raw_clog_entry, _ = repo.git_run(cmd, decode=False) |
1692 | elif publish_parent_commit is not None: |
1693 | cmd = ['diff-tree', '-p', publish_parent_commit, |
1694 | tree_hash, '--', 'debian/changelog'] |
1695 | - raw_clog_entry = repo.git_run(cmd).stdout |
1696 | + raw_clog_entry, _ = repo.git_run(cmd, decode=False) |
1697 | |
1698 | changelog_entry = b'' |
1699 | changelog_entry_found = False |
1700 | @@ -754,10 +761,9 @@ def import_patches_applied_tree(repo, dsc_pathname): |
1701 | repo.git_run( |
1702 | ['--work-tree', extracted_dir, 'rm', '-r', '-f', '.pc'] |
1703 | ) |
1704 | - cp = repo.git_run( |
1705 | + import_tree_hash, _ = repo.git_run( |
1706 | ['--work-tree', extracted_dir, 'write-tree'] |
1707 | ) |
1708 | - import_tree_hash = decode_binary(cp.stdout).strip() |
1709 | yield ( |
1710 | import_tree_hash, |
1711 | None, |
1712 | @@ -765,31 +771,15 @@ def import_patches_applied_tree(repo, dsc_pathname): |
1713 | ) |
1714 | |
1715 | try: |
1716 | - try: |
1717 | - cp = run(['dpkg-source', '--print-format', extracted_dir]) |
1718 | - fmt = decode_binary(cp.stdout).strip() |
1719 | - if '3.0 (quilt)' not in fmt: |
1720 | - raise StopIteration() |
1721 | - except CalledProcessError as e: |
1722 | - try: |
1723 | - with open('debian/source/format', 'r') as f: |
1724 | - for line in f: |
1725 | - if re.match(r'3.0 (.*)', line): |
1726 | - break |
1727 | - else: |
1728 | - raise StopIteration() |
1729 | - # `man dpkg-source` indicates no d/s/format implies 1.0 |
1730 | - except OSError: |
1731 | - raise StopIteration() |
1732 | + if not is_dir_3_0_quilt(extracted_dir): |
1733 | + raise StopIteration() |
1734 | |
1735 | while True: |
1736 | try: |
1737 | os.chdir(extracted_dir) |
1738 | - run(['quilt', 'push'], rcs=[2]) |
1739 | - cp = run(['quilt', 'top']) |
1740 | - patch_name = decode_binary(cp.stdout).strip() |
1741 | - cp = run(['quilt', 'header']) |
1742 | - header = decode_binary(cp.stdout).strip() |
1743 | + run(['quilt', 'push'], verbose_on_failure=False,) |
1744 | + patch_name, _ = run(['quilt', 'top']) |
1745 | + header, _ = run(['quilt', 'header']) |
1746 | patch_desc = None |
1747 | for regex in (r'Subject:\s*(.*?)$', |
1748 | r'Description:\s*(.*?)$' |
1749 | @@ -818,8 +808,9 @@ def import_patches_applied_tree(repo, dsc_pathname): |
1750 | '--', |
1751 | '.pc', |
1752 | ]) |
1753 | - cp = repo.git_run(['--work-tree', extracted_dir, 'write-tree']) |
1754 | - import_tree_hash = decode_binary(cp.stdout).strip() |
1755 | + import_tree_hash, _ = repo.git_run( |
1756 | + ['--work-tree', extracted_dir, 'write-tree'] |
1757 | + ) |
1758 | |
1759 | yield (import_tree_hash, patch_name, patch_desc) |
1760 | except CalledProcessError as e: |
1761 | @@ -1195,14 +1186,6 @@ def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo): |
1762 | unapplied_publish_parent_commit = str(unapplied_publish_parent_head.peel().id) |
1763 | except AttributeError: |
1764 | pass |
1765 | - finally: |
1766 | - try: |
1767 | - unapplied_publish_parent_tag = repo.nearest_import_tag(unapplied_publish_parent_commit, namespace) |
1768 | - logging.debug('Publishing parent (tag) is %s', |
1769 | - repo.tag_to_pretty_name(unapplied_publish_parent_tag) |
1770 | - ) |
1771 | - except AttributeError: |
1772 | - pass |
1773 | |
1774 | if version_compare(str(spi.version), unapplied_tip_version) <= 0: |
1775 | logging.warn('Version to import (%s) is not after %s tip (%s)', |
1776 | @@ -1265,11 +1248,15 @@ def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo): |
1777 | upload_parent_commit = str(upload_tag.peel().id) |
1778 | if unapplied_publish_parent_commit is not None: |
1779 | try: |
1780 | - repo.git_run(['merge-base', '--is-ancestor', |
1781 | - unapplied_publish_parent_commit, |
1782 | - upload_parent_commit |
1783 | - ] |
1784 | - ) |
1785 | + repo.git_run( |
1786 | + [ |
1787 | + 'merge-base', |
1788 | + '--is-ancestor', |
1789 | + unapplied_publish_parent_commit, |
1790 | + upload_parent_commit, |
1791 | + ], |
1792 | + verbose_on_failure=False, |
1793 | + ) |
1794 | unapplied_publish_parent_commit = None |
1795 | except CalledProcessError as e: |
1796 | if e.returncode != 1: |
1797 | @@ -1277,11 +1264,15 @@ def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo): |
1798 | |
1799 | if unapplied_changelog_parent_commit is not None: |
1800 | try: |
1801 | - repo.git_run(['merge-base', '--is-ancestor', |
1802 | - unapplied_changelog_parent_commit, |
1803 | - upload_parent_commit |
1804 | - ] |
1805 | - ) |
1806 | + repo.git_run( |
1807 | + [ |
1808 | + 'merge-base', |
1809 | + '--is-ancestor', |
1810 | + unapplied_changelog_parent_commit, |
1811 | + upload_parent_commit, |
1812 | + ], |
1813 | + verbose_on_failure=False, |
1814 | + ) |
1815 | unapplied_changelog_parent_commit = None |
1816 | except CalledProcessError as e: |
1817 | if e.returncode != 1: |
1818 | @@ -1305,7 +1296,13 @@ def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo): |
1819 | spi=spi, |
1820 | ) |
1821 | |
1822 | -def import_applied_spi(repo, spi, namespace, ubuntu_sinfo): |
1823 | +def import_applied_spi( |
1824 | + repo, |
1825 | + spi, |
1826 | + namespace, |
1827 | + ubuntu_sinfo, |
1828 | + allow_applied_failures |
1829 | +): |
1830 | """Imports a source package from Launchpad into the git |
1831 | repository |
1832 | |
1833 | @@ -1424,35 +1421,40 @@ def import_applied_spi(repo, spi, namespace, ubuntu_sinfo): |
1834 | # Assume no patches to apply |
1835 | applied_import_tree_hash = unapplied_import_tree_hash |
1836 | # get tree id from above commit |
1837 | - for ( |
1838 | - applied_import_tree_hash, |
1839 | - patch_name, |
1840 | - patch_desc |
1841 | - ) in import_patches_applied_tree(repo, spi.dsc_pathname): |
1842 | - # special case for .pc removal |
1843 | - if patch_name is None: |
1844 | - msg = b'%b.' % (patch_desc.encode()) |
1845 | - else: |
1846 | - if patch_desc is None: |
1847 | - patch_desc = ( |
1848 | - "%s\n\nNo DEP3 Subject or Description header found" % |
1849 | - patch_name |
1850 | + try: |
1851 | + for ( |
1852 | + applied_import_tree_hash, |
1853 | + patch_name, |
1854 | + patch_desc |
1855 | + ) in import_patches_applied_tree(repo, spi.dsc_pathname): |
1856 | + # special case for .pc removal |
1857 | + if patch_name is None: |
1858 | + msg = b'%b.' % (patch_desc.encode()) |
1859 | + else: |
1860 | + if patch_desc is None: |
1861 | + patch_desc = ( |
1862 | + "%s\n\nNo DEP3 Subject or Description header found" % |
1863 | + patch_name |
1864 | + ) |
1865 | + msg = b'%b\n\nGbp-Pq: %b.' % ( |
1866 | + patch_desc.encode(), |
1867 | + patch_name.encode() |
1868 | ) |
1869 | - msg = b'%b\n\nGbp-Pq: %b.' % ( |
1870 | - patch_desc.encode(), |
1871 | - patch_name.encode() |
1872 | + unapplied_parent_commit = repo.commit_tree_hash( |
1873 | + applied_import_tree_hash, |
1874 | + [unapplied_parent_commit], |
1875 | + msg, |
1876 | + spi |
1877 | ) |
1878 | - unapplied_parent_commit = repo.commit_tree_hash( |
1879 | - applied_import_tree_hash, |
1880 | - [unapplied_parent_commit], |
1881 | - msg, |
1882 | - spi |
1883 | - ) |
1884 | |
1885 | - logging.debug( |
1886 | - "Committed patch-application of %s as %s", |
1887 | - patch_name, unapplied_parent_commit |
1888 | - ) |
1889 | + logging.debug( |
1890 | + "Committed patch-application of %s as %s", |
1891 | + patch_name, unapplied_parent_commit |
1892 | + ) |
1893 | + except CalledProcessError as e: |
1894 | + if allow_applied_failures: |
1895 | + return |
1896 | + raise |
1897 | |
1898 | commit_applied_patches_import( |
1899 | repo, |
1900 | @@ -1472,10 +1474,20 @@ def import_applied_spi(repo, spi, namespace, ubuntu_sinfo): |
1901 | spi=spi, |
1902 | ) |
1903 | |
1904 | -def import_publishes(repo, pkgname, namespace, patches_applied, |
1905 | - debian_head_versions, ubuntu_head_versions, debian_sinfo, |
1906 | - ubuntu_sinfo, active_series_only, workdir, fixup_devel, |
1907 | +def import_publishes( |
1908 | + repo, |
1909 | + pkgname, |
1910 | + namespace, |
1911 | + patches_applied, |
1912 | + debian_head_versions, |
1913 | + ubuntu_head_versions, |
1914 | + debian_sinfo, |
1915 | + ubuntu_sinfo, |
1916 | + active_series_only, |
1917 | + workdir, |
1918 | + fixup_devel, |
1919 | skip_orig, |
1920 | + allow_applied_failures, |
1921 | ): |
1922 | history_found = False |
1923 | only_debian = False |
1924 | @@ -1484,13 +1496,16 @@ def import_publishes(repo, pkgname, namespace, patches_applied, |
1925 | _namespace = namespace |
1926 | namespace = '%s/applied' % namespace |
1927 | import_type = 'patches-applied' |
1928 | - import_func = import_applied_spi |
1929 | + import_func = functools.partial( |
1930 | + import_applied_spi, |
1931 | + allow_applied_failures=allow_applied_failures, |
1932 | + ) |
1933 | else: |
1934 | _namespace = namespace |
1935 | import_type = 'patches-unapplied' |
1936 | import_func = functools.partial( |
1937 | import_unapplied_spi, |
1938 | - skip_orig=skip_orig |
1939 | + skip_orig=skip_orig, |
1940 | ) |
1941 | for distname, versions, dist_sinfo in ( |
1942 | ("debian", debian_head_versions, debian_sinfo), |
1943 | @@ -1592,6 +1607,11 @@ def parse_args(subparsers=None, base_subparsers=None): |
1944 | help=argparse.SUPPRESS) |
1945 | parser.add_argument('--reimport', action='store_true', |
1946 | help=argparse.SUPPRESS) |
1947 | + parser.add_argument( |
1948 | + '--allow-applied-failures', |
1949 | + action='store_true', |
1950 | + help=argparse.SUPPRESS, |
1951 | + ) |
1952 | if not subparsers: |
1953 | return parser.parse_args() |
1954 | return 'import - %s' % kwargs['description'] |
1955 | @@ -1631,6 +1651,7 @@ def cli_main(args): |
1956 | skip_orig=args.skip_orig, |
1957 | skip_applied=args.skip_applied, |
1958 | reimport=args.reimport, |
1959 | + allow_applied_failures=args.allow_applied_failures, |
1960 | directory=directory, |
1961 | user=user, |
1962 | proto=args.proto, |
1963 | diff --git a/gitubuntu/importppa.py b/gitubuntu/importppa.py |
1964 | index 3d5ba65..1f3283c 100644 |
1965 | --- a/gitubuntu/importppa.py |
1966 | +++ b/gitubuntu/importppa.py |
1967 | @@ -52,6 +52,7 @@ def main( |
1968 | retries, |
1969 | retry_backoffs, |
1970 | skip_orig, |
1971 | + allow_applied_failures, |
1972 | ): |
1973 | if re.match(r'ppa:\w+', ppa) is None: |
1974 | logging.error( |
1975 | @@ -159,6 +160,7 @@ def main( |
1976 | spi=srcpkg_information, |
1977 | namespace=namespace, |
1978 | ubuntu_sinfo=source_information, |
1979 | + allow_applied_failures=allow_applied_failures, |
1980 | ) |
1981 | except DownloadError: |
1982 | # it is non-fatal for a PPA to not have files for older |
1983 | @@ -215,6 +217,11 @@ def parse_args(subparsers=None, base_subparsers=None): |
1984 | action='store_true', |
1985 | help=argparse.SUPPRESS, |
1986 | ) |
1987 | + parser.add_argument( |
1988 | + '--allow-applied-failures', |
1989 | + action='store_true', |
1990 | + help=argparse.SUPPRESS, |
1991 | + ) |
1992 | if not subparsers: |
1993 | return parser.parse_args() |
1994 | return 'import-ppa - %s' % kwargs['description'] |
1995 | @@ -241,4 +248,5 @@ def cli_main(args): |
1996 | args.retries, |
1997 | args.retry_backoffs, |
1998 | args.skip_orig, |
1999 | + args.allow_applied_failures, |
2000 | ) |
2001 | diff --git a/gitubuntu/lint.py b/gitubuntu/lint.py |
2002 | index 0897eef..dda55e8 100644 |
2003 | --- a/gitubuntu/lint.py |
2004 | +++ b/gitubuntu/lint.py |
2005 | @@ -625,14 +625,39 @@ def do_change_lint(repo, commitish_string, pkg_remote_branch_string): |
2006 | # Don"t need most arguments here as we"re only grabbing some |
2007 | # data from launchpad about the active series |
2008 | ubuntu_source_information = GitUbuntuSourceInformation("ubuntu") |
2009 | - # 1) does changelog in new branch only have additions at the top |
2010 | + # 1) Are there upstream changes not stored in a quilt patch |
2011 | + try: |
2012 | + quiltify_needed = not repo.paths_are_identical( |
2013 | + commitish_string, |
2014 | + repo.maybe_quiltify_tree_hash(commitish_string), |
2015 | + 'debian/patches', |
2016 | + ) |
2017 | + except ValueError: |
2018 | + quiltify_needed = False |
2019 | + if quiltify_needed: |
2020 | + # 1a) if so, emit an error |
2021 | + error( |
2022 | + "Upstream changes exist that have not yet " |
2023 | + "been converted into a quilt patch. Consider using " |
2024 | + "`git ubuntu build` to generate a quilt patch." |
2025 | + ) |
2026 | + ret = False |
2027 | + elif 'applied' in pkg_remote_branch_string: |
2028 | + # 1b) if not, emit a warning recommending if they are targetting |
2029 | + # a patches-applied branch |
2030 | + warning( |
2031 | + "All upstream changes are correctly tracked in quilt " |
2032 | + "patches, but changes are targetting a " |
2033 | + "patches-applied tree." |
2034 | + ) |
2035 | + # 2) does changelog in new branch only have additions at the top |
2036 | # relative to target |
2037 | ret = _check_changelog_addition( |
2038 | repo, |
2039 | pkg_remote_branch_string, |
2040 | commitish_string, |
2041 | ) and ret |
2042 | - # 2) does versioning make sense? |
2043 | + # 3) does versioning make sense? |
2044 | if dist in ["devel", ubuntu_source_information.active_series_name_list[0]]: |
2045 | ret = _check_versioning( |
2046 | repo, |
2047 | @@ -655,9 +680,9 @@ def do_change_lint(repo, commitish_string, pkg_remote_branch_string): |
2048 | else: |
2049 | error("Targetted distribution (%s) is not active", dist) |
2050 | ret = False |
2051 | - # 3) has update-maintainer been run? |
2052 | + # 4) has update-maintainer been run? |
2053 | ret = _check_update_maintainer(repo, commitish_string) and ret |
2054 | - # 4) does target branch match changelog? |
2055 | + # 5) does target branch match changelog? |
2056 | pkg_branch_series = pkg_remote_branch_string[len('pkg/'):].split('/')[1].split('-')[0] |
2057 | if dist != pkg_branch_series: |
2058 | error( |
2059 | diff --git a/gitubuntu/merge.py b/gitubuntu/merge.py |
2060 | index 3baa748..c16fe4b 100644 |
2061 | --- a/gitubuntu/merge.py |
2062 | +++ b/gitubuntu/merge.py |
2063 | @@ -129,15 +129,16 @@ def do_tag(repo, tag_prefix, commitish, merge_base_id, onto, force): |
2064 | def do_reconstruct(repo, tag_prefix, commitish, merge_base_id, force): |
2065 | versions=() |
2066 | repo.checkout_commitish(merge_base_id) |
2067 | - cp = repo.git_run( |
2068 | + stdout, _ = repo.git_run( |
2069 | [ |
2070 | 'rev-list', |
2071 | '--ancestry-path', |
2072 | '--reverse', |
2073 | '%s..%s' % (merge_base_id, commitish), |
2074 | - ] |
2075 | + ], |
2076 | + decode=False, |
2077 | ) |
2078 | - for commit in cp.stdout.split(b'\n'): |
2079 | + for commit in stdout.split(b'\n'): |
2080 | commit = decode_binary(commit).strip() |
2081 | # empty newline |
2082 | if len(commit) == 0: |
2083 | @@ -153,14 +154,13 @@ def do_reconstruct(repo, tag_prefix, commitish, merge_base_id, force): |
2084 | args += [str(obj.id)] |
2085 | repo.git_run(args) |
2086 | try: |
2087 | - cp = repo.git_run(['diff', '--exit-code', commitish]) |
2088 | + repo.git_run(['diff', '--exit-code', commitish]) |
2089 | except: |
2090 | logging.error( |
2091 | "Resulting cleaned-up commit is not " |
2092 | "source-identical to %s", |
2093 | commitish |
2094 | ) |
2095 | - logging.error(decode_binary(cp.stdout)) |
2096 | raise ReconstructException("Failed to reconstruct commit.") |
2097 | |
2098 | old_head_version, _ = repo.get_changelog_versions_from_treeish(commitish) |
2099 | @@ -237,14 +237,13 @@ def do_reconstruct_changelog(repo, onto, release, bug): |
2100 | fp.write(' (LP: #%s)' % bug) |
2101 | fp.write('. Remaining changes:\n') |
2102 | # merge-changelogs is HEAD |
2103 | - cp = repo.git_run( |
2104 | + stdout, _ = repo.git_run( |
2105 | ['rev-list', '--reverse', '%s..HEAD^' % onto,] |
2106 | ) |
2107 | - for rev in decode_binary(cp.stdout).splitlines(): |
2108 | - cp = repo.git_run( |
2109 | + for rev in stdout.splitlines(): |
2110 | + out, _ = repo.git_run( |
2111 | ['log', '--pretty=%B', '-n', '1', rev,] |
2112 | ) |
2113 | - out = decode_binary(cp.stdout) |
2114 | look_for_marker = False |
2115 | if '--CL--' in out: |
2116 | look_for_marker = True |
2117 | @@ -432,10 +431,10 @@ def main( |
2118 | return 1 |
2119 | onto_obj = repo.get_commitish(onto) |
2120 | |
2121 | - cp = run(['git', 'status', '--porcelain']) |
2122 | - if len(cp.stdout) > 0: |
2123 | + stdout, _ = run(['git', 'status', '--porcelain']) |
2124 | + if len(stdout) > 0: |
2125 | logging.error('Working tree must be clean to continue:') |
2126 | - logging.error(decode_binary(cp.stdout)) |
2127 | + logging.error(stdout) |
2128 | return 1 |
2129 | |
2130 | merge_base_id = repo.raw_repo.merge_base( |
2131 | diff --git a/gitubuntu/remote.py b/gitubuntu/remote.py |
2132 | index 4526e3f..6e21b5f 100644 |
2133 | --- a/gitubuntu/remote.py |
2134 | +++ b/gitubuntu/remote.py |
2135 | @@ -25,37 +25,66 @@ def parse_args(subparsers=None, base_subparsers=None): |
2136 | parser.set_defaults(func=cli_main) |
2137 | else: |
2138 | parser = argparse.ArgumentParser(**kwargs) |
2139 | - parser.add_argument('subsubcommand', |
2140 | - help='add - Add a launchpad user\'s repository as a remote', |
2141 | - metavar='add', |
2142 | - choices=['add']) |
2143 | - parser.add_argument('user', type=str, |
2144 | - help='Launchpad user to add as a remote') |
2145 | - parser.add_argument('url', type=str, |
2146 | - help='Specify URL to add as a remote. If ' |
2147 | - 'not specified, a URL is derived from ' |
2148 | - 'USER and the repository source package.', |
2149 | - default=None, |
2150 | - nargs='?') |
2151 | - parser.add_argument('--directory', type=str, |
2152 | - help='Local git directory to modify. If not ' |
2153 | - 'specified, the current directory is ' |
2154 | - 'assumed') |
2155 | - parser.add_argument('-l', '--lp-user', type=str, |
2156 | - help=argparse.SUPPRESS) |
2157 | - parser.add_argument('-r', '--remote-name', type=str, |
2158 | - help='Specify remote name. If not ' |
2159 | - 'specified the remote will be named ' |
2160 | - 'after the user argument.') |
2161 | - parser.add_argument('--no-fetch', action='store_true', |
2162 | - help='Do not fetch the remote after adding') |
2163 | - parser.add_argument('--package', type=str, |
2164 | - help='Specify the source package rather than ' |
2165 | - 'automatically determining it from ' |
2166 | - 'debian/changelog') |
2167 | + known_subcommands = { |
2168 | + 'add': 'Add a launchpad user\'s repository as a remote', |
2169 | + } |
2170 | + subsubparsers = parser.add_subparsers( |
2171 | + dest='subsubcommand', |
2172 | + help='', |
2173 | + # this will look off until more than one subcommand exists |
2174 | + metavar='%s' % '|'.join(sorted(known_subcommands.keys())), |
2175 | + ) |
2176 | + add_parser = subsubparsers.add_parser( |
2177 | + 'add', |
2178 | + help=known_subcommands['add'] |
2179 | + ) |
2180 | + add_parser.add_argument( |
2181 | + 'user', |
2182 | + type=str, |
2183 | + help="Launchpad user to add as a remote. If 'debian', parse " |
2184 | + "pkg/ubuntu/devel:debian/control for either " |
2185 | + "X-Debian-Vcs-Git or Vcs-Git. If found, add it as a remote " |
2186 | + "named 'debian'.", |
2187 | + ) |
2188 | + add_parser.add_argument( |
2189 | + 'url', |
2190 | + type=str, |
2191 | + help="Specify URL to add as a remote. If not specified, a URL is " |
2192 | + "derived from USER and the repository source package.", |
2193 | + default=None, |
2194 | + nargs='?', |
2195 | + ) |
2196 | + add_parser.add_argument( |
2197 | + '--directory', |
2198 | + type=str, |
2199 | + help="Local git directory to modify. If not specified, the " |
2200 | + "current directory is assumed.", |
2201 | + ) |
2202 | + add_parser.add_argument( |
2203 | + '-l', '--lp-user', |
2204 | + type=str, |
2205 | + help=argparse.SUPPRESS |
2206 | + ) |
2207 | + add_parser.add_argument( |
2208 | + '-r', '--remote-name', |
2209 | + type=str, |
2210 | + help="Specify remote name. If not specified the remote will be " |
2211 | + "named after the user argument.", |
2212 | + ) |
2213 | + add_parser.add_argument( |
2214 | + '--no-fetch', |
2215 | + action='store_true', |
2216 | + help="Do not fetch the remote after adding", |
2217 | + ) |
2218 | + add_parser.add_argument( |
2219 | + '--package', |
2220 | + type=str, |
2221 | + help="Specify the source package rather than automatically " |
2222 | + "determining it from debian/changelog.", |
2223 | + ) |
2224 | if not subparsers: |
2225 | return parser.parse_args() |
2226 | - return 'remote - %s' % kwargs['description'] |
2227 | + return "remote - %s" % kwargs['description'] |
2228 | |
2229 | def cli_main(args): |
2230 | if args.directory is not None: |
2231 | @@ -75,6 +104,68 @@ def cli_main(args): |
2232 | args.proto, |
2233 | ) |
2234 | |
2235 | +def do_add_debian(repo, package, no_fetch=False): |
2236 | + """add a Debian remote to a repository |
2237 | + |
2238 | + Parses debian/control in pkg/ubuntu/devel, and uses the value of: |
2239 | + 1) X-Debian-Vcs-Git |
2240 | + or |
2241 | + 2) Vcs-Git, only if there is no X-Debian-Vcs-Git |
2242 | + |
2243 | + as the URL for a remote named Debian. |
2244 | + |
2245 | + @repo: GitUbuntuRepository to modify |
2246 | + @package: source package name (XXX: is this derive-able from @repo?) |
2247 | + @no_fetch: if True, do not fetch the remote after adding it |
2248 | + """ |
2249 | + remote_name = 'debian' |
2250 | + |
2251 | + control_file = None |
2252 | + try: |
2253 | + control_file = repo.extract_file_from_treeish( |
2254 | + treeish_string='pkg/ubuntu/devel', |
2255 | + filename='debian/control', |
2256 | + ) |
2257 | + control_file.seek(0) |
2258 | + x_debian_vcs_git = None |
2259 | + vcs_git = None |
2260 | + for line in control_file: |
2261 | + line = decode_binary(line) |
2262 | + if line.startswith('X-Debian-Vcs-Git:'): |
2263 | + _, x_debian_vcs_git = line.split(':', 1) |
2264 | + x_debian_vcs_git = x_debian_vcs_git.strip() |
2265 | + break |
2266 | + if line.startswith('Vcs-Git:'): |
2267 | + _, vcs_git = line.split(':', 1) |
2268 | + vcs_git = vcs_git.strip() |
2269 | + |
2270 | + if not x_debian_vcs_git and not vcs_git: |
2271 | + logging.error( |
2272 | + "Unable to find any Vcs metadata in " |
2273 | + "pkg/ubuntu/devel:debian/control." |
2274 | + ) |
2275 | + return 1 |
2276 | + |
2277 | + url = x_debian_vcs_git if x_debian_vcs_git else vcs_git |
2278 | + repo.add_remote_by_url(remote_name, url) |
2279 | + finally: |
2280 | + if control_file: |
2281 | + control_file.close() |
2282 | + |
2283 | + if not no_fetch: |
2284 | + try: |
2285 | + repo.fetch_remote(remote_name, verbose=True) |
2286 | + except GitUbuntuRepositoryFetchError: |
2287 | + pass |
2288 | + |
2289 | + logging.debug( |
2290 | + "added remote '%s' -> %s", |
2291 | + remote_name, |
2292 | + repo.raw_repo.remotes[remote_name].url, |
2293 | + ) |
2294 | + |
2295 | + return 0 |
2296 | + |
2297 | def do_add(repo, package, user, url=None, remote_name=None, no_fetch=False): |
2298 | """add a remote to a repository |
2299 | |
2300 | @@ -94,7 +185,7 @@ def do_add(repo, package, user, url=None, remote_name=None, no_fetch=False): |
2301 | remote_name = user |
2302 | |
2303 | if url: |
2304 | - repo.git_run(['remote', 'add', remote_name, url]) |
2305 | + repo.add_remote_by_url(remote_name, url) |
2306 | else: |
2307 | repo.add_remote( |
2308 | pkgname=package, |
2309 | @@ -133,6 +224,9 @@ def main( |
2310 | @package: string name of source package |
2311 | @url: the string URL of the remote to add |
2312 | @lp_user: string user to authenticate to Launchpad as |
2313 | + @directory: directory containing git repository to modify |
2314 | + @remote_name: name of remote |
2315 | + @no_fetch: do not fetch remote after modifying it |
2316 | @proto: string protocol to use (one of 'http', 'https', 'git') |
2317 | |
2318 | If package is None, it will be derived from HEAD's debian/changelog. |
2319 | @@ -181,7 +275,10 @@ def main( |
2320 | return 1 |
2321 | |
2322 | if subcommand == 'add': |
2323 | - return do_add(repo, package, user, url, remote_name, no_fetch) |
2324 | + if user == 'debian': |
2325 | + return do_add_debian(repo, package, no_fetch) |
2326 | + else: |
2327 | + return do_add(repo, package, user, url, remote_name, no_fetch) |
2328 | |
2329 | return 1 |
2330 | |
2331 | diff --git a/gitubuntu/review.py b/gitubuntu/review.py |
2332 | index 8e252d5..3a02c62 100644 |
2333 | --- a/gitubuntu/review.py |
2334 | +++ b/gitubuntu/review.py |
2335 | @@ -91,6 +91,10 @@ def main(clone, url, add_comment): |
2336 | target_user |
2337 | ) |
2338 | return 1 |
2339 | + if 'applied' in target_branch: |
2340 | + logging.error( |
2341 | + "Upload tagging a patches-applied upload is not supported." |
2342 | + ) |
2343 | |
2344 | source_url = mp.source_git_repository.git_https_url |
2345 | source_branch = mp.source_git_path[len('refs/heads/'):] |
2346 | diff --git a/gitubuntu/run.py b/gitubuntu/run.py |
2347 | index 098438c..2f6fa53 100644 |
2348 | --- a/gitubuntu/run.py |
2349 | +++ b/gitubuntu/run.py |
2350 | @@ -54,34 +54,51 @@ def run( |
2351 | try: |
2352 | logging.debug("Executing: %s", pcmd) |
2353 | if input: |
2354 | - return subprocess.run( |
2355 | + cp = subprocess.run( |
2356 | args, env=env, check=check, shell=shell, |
2357 | input=input, |
2358 | stdout=stdout, stderr=stderr) |
2359 | else: |
2360 | - return subprocess.run( |
2361 | + cp = subprocess.run( |
2362 | args, env=env, check=check, shell=shell, |
2363 | stdout=stdout, stderr=stderr, stdin=stdin) |
2364 | + ret = cp |
2365 | except subprocess.CalledProcessError as e: |
2366 | not_captured = "[Not captured]\n" |
2367 | - err = not_captured |
2368 | out = not_captured |
2369 | - if stderr is subprocess.PIPE: |
2370 | - err = e.stderr.decode(errors="replace") |
2371 | + err = not_captured |
2372 | if stdout is subprocess.PIPE: |
2373 | - out = e.stdout.decode(errors="replace") |
2374 | - if verbose_on_failure and e.returncode not in rcs: |
2375 | - logging.error("Command exited %d: %s", e.returncode, pcmd) |
2376 | - logging.error("stdout: %s", |
2377 | - out.rstrip().replace("\n", "\n ")) |
2378 | - logging.error("stderr: %s", |
2379 | - err.rstrip().replace("\n", "\n ")) |
2380 | - raise e |
2381 | + out = e.stdout.decode(errors='replace') |
2382 | + if stderr is subprocess.PIPE: |
2383 | + err = e.stderr.decode(errors='replace') |
2384 | + if e.returncode not in rcs: |
2385 | + if verbose_on_failure: |
2386 | + logging.error("Command exited %d: %s", e.returncode, pcmd) |
2387 | + logging.error( |
2388 | + "stdout: %s", |
2389 | + out.replace("\n", "\n "), |
2390 | + ) |
2391 | + logging.error( |
2392 | + "stderr: %s", |
2393 | + err.replace("\n", "\n "), |
2394 | + ) |
2395 | + raise e |
2396 | + ret = e |
2397 | + pass |
2398 | |
2399 | + if decode: |
2400 | + out = decode_binary(ret.stdout) |
2401 | + err = decode_binary(ret.stderr) |
2402 | + return ( |
2403 | + out if out is None else out.strip(), |
2404 | + err if err is None else err.strip(), |
2405 | + ) |
2406 | + else: |
2407 | + return (ret.stdout, ret.stderr) |
2408 | |
2409 | def decode_binary(binary, verbose=True): |
2410 | try: |
2411 | - return binary.decode('utf-8') |
2412 | + return binary if binary is None else binary.decode('utf-8') |
2413 | except UnicodeDecodeError as e: |
2414 | if verbose: |
2415 | logging.warning("Failed to decode blob: %s", e) |
2416 | diff --git a/gitubuntu/versioning.py b/gitubuntu/versioning.py |
2417 | index 3a5dc53..7b1eb7b 100644 |
2418 | --- a/gitubuntu/versioning.py |
2419 | +++ b/gitubuntu/versioning.py |
2420 | @@ -4,7 +4,10 @@ import functools |
2421 | import logging |
2422 | import re |
2423 | import sys |
2424 | -from gitubuntu.source_information import GitUbuntuSourceInformation |
2425 | +from gitubuntu.source_information import ( |
2426 | + GitUbuntuSourceInformation, |
2427 | + NoPublicationHistoryException, |
2428 | +) |
2429 | |
2430 | try: |
2431 | pkg = 'python3-debian' |
2432 | @@ -218,10 +221,13 @@ def max_version(lp_series_object, pkgname): |
2433 | and series |
2434 | """ |
2435 | ubuntu_source_information = GitUbuntuSourceInformation('ubuntu', pkgname) |
2436 | - for spi in ubuntu_source_information.launchpad_versions_published( |
2437 | - sorted_by_version=True, series=lp_series_object |
2438 | - ): |
2439 | - return Version(spi.version) |
2440 | + try: |
2441 | + for spi in ubuntu_source_information.launchpad_versions_published( |
2442 | + sorted_by_version=True, series=lp_series_object |
2443 | + ): |
2444 | + return Version(spi.version) |
2445 | + except NoPublicationHistoryException: |
2446 | + return None |
2447 | |
2448 | |
2449 | def next_sru_version_string(lp_series_object, pkgname): |
2450 | @@ -301,6 +307,9 @@ def test_next_development_version_string(test_input, expected): |
2451 | (['1.0-1'], '16.04', '1.0-1ubuntu1', ['1.0-1ubuntu11'], '1.0-1ubuntu1.1'), |
2452 | (['1.0-1'], '16.04', '1.0-2ubuntu1', ['1.0-2ubuntu11'], '1.0-2ubuntu1.1'), |
2453 | ([], '16.04', '1.0-2ubuntu1', ['2.0-2ubuntu1'], '1.0-2ubuntu1.1'), |
2454 | + (['1.0.1-0ubuntu0.16.04.1', None, None,], '17.04', '1.0.1-0ubuntu1', ['1.0.1-1ubuntu2', '1.0.1-1ubuntu2',], '1.0.1-0ubuntu1.1'), |
2455 | + ([None, None,], '17.04', '1.0.1-0ubuntu1', ['1.0.1-1ubuntu2', '1.0.1-1ubuntu2',], '1.0.1-0ubuntu1.1'), |
2456 | + ([None, None,], '17.04', '1.0.1-0ubuntu1', [None, None,], '1.0.1-0ubuntu1.1'), |
2457 | ]) |
2458 | def test__next_sru_version(befores, series_string, current, afters, expected): |
2459 | assert _next_sru_version( |
2460 | diff --git a/man/man1/git-ubuntu-build-source.1 b/man/man1/git-ubuntu-build-source.1 |
2461 | index a9131d8..6ee03ec 100644 |
2462 | --- a/man/man1/git-ubuntu-build-source.1 |
2463 | +++ b/man/man1/git-ubuntu-build-source.1 |
2464 | @@ -7,7 +7,8 @@ git-ubuntu build-source \- Build source packages |
2465 | .NF |
2466 | \fIgit ubuntu build-source\fR [\-\-dl-cache <dl_cache>] [\-\-sign] |
2467 | [\-\-for-merge] [\-\-no-pristine-tar] [\-\-no-lxd] [\-\-keep-build-env] |
2468 | -[\-\-lxd-profile <profile>] [\-\-lxd-image <image>] [-- \&.\&.\&.] |
2469 | +[\-\-lxd-profile <profile>] [\-\-lxd-image <image>] |
2470 | +[\-\-commitish <commitish>] [-- \&.\&.\&.] |
2471 | .FI |
2472 | .SP |
2473 | .SH "DESCRIPTION" |
2474 | @@ -83,6 +84,12 @@ By default, the image is derived from the distribution and series |
2475 | targetted by debian/changelog of the source package to build\&. |
2476 | .RE |
2477 | .PP |
2478 | +\-\-commitish <commitish> |
2479 | +.RS 4 |
2480 | +The commitish to build in the Git repository\&. |
2481 | +If not specified, HEAD will be built\&. |
2482 | +.RE |
2483 | +.PP |
2484 | \-\- \&.\&.\&. |
2485 | .RS 4 |
2486 | Any arguments after "--" are passed on verbatim to |
2487 | diff --git a/man/man1/git-ubuntu-build.1 b/man/man1/git-ubuntu-build.1 |
2488 | index 3a24ccf..9b719cb 100644 |
2489 | --- a/man/man1/git-ubuntu-build.1 |
2490 | +++ b/man/man1/git-ubuntu-build.1 |
2491 | @@ -7,7 +7,8 @@ git-ubuntu build \- Build packages |
2492 | .NF |
2493 | \fIgit ubuntu build\fR [\-\-dl-cache <dl_cache>] [\-\-sign] |
2494 | [\-\-for-merge] [\-\-no-pristine-tar] [\-\-no-lxd] [\-\-keep-build-env] |
2495 | -[\-\-lxd-profile <profile>] [\-\-lxd-image <image>] [-- \&.\&.\&.] |
2496 | +[\-\-lxd-profile <profile>] [\-\-lxd-image <image>] |
2497 | +[\-\-commitish <commitish>] [-- \&.\&.\&.] |
2498 | .FI |
2499 | .SP |
2500 | .SH "DESCRIPTION" |
2501 | @@ -81,6 +82,12 @@ By default, the image is derived from the distribution and series |
2502 | targetted by debian/changelog of the source package to build\&. |
2503 | .RE |
2504 | .PP |
2505 | +\-\-commitish <commitish> |
2506 | +.RS 4 |
2507 | +The commitish to build in the Git repository\&. |
2508 | +If not specified, HEAD will be built\&. |
2509 | +.RE |
2510 | +.PP |
2511 | \-\- \&.\&.\&. |
2512 | .RS 4 |
2513 | Any arguments after "--" are passed on verbatim to |
2514 | diff --git a/man/man1/git-ubuntu-import-ppa.1 b/man/man1/git-ubuntu-import-ppa.1 |
2515 | index d302464..318cd5a 100644 |
2516 | --- a/man/man1/git-ubuntu-import-ppa.1 |
2517 | +++ b/man/man1/git-ubuntu-import-ppa.1 |
2518 | @@ -7,7 +7,7 @@ git-ubuntu import-ppa\- Import a DSC file to Git |
2519 | .NF |
2520 | \fIgit ubuntu import-ppa\fR [\-l | \-\-lp-user <user>] |
2521 | [\-d | \-\-directory <directory>] [\-\-skip-orig] |
2522 | -[\-\-dl-cache <dl_cache>] <ppa> <package> |
2523 | +[\-\-allow-applied-failures] [\-\-dl-cache <dl_cache>] <ppa> <package> |
2524 | .FI |
2525 | .SP |
2526 | .SH "DESCRIPTION" |
2527 | @@ -48,6 +48,21 @@ This option can be useful for \fBgit-ubuntu\fR(1) developers when |
2528 | debugging issues with "gbp" or "pristine-tar"\&. |
2529 | .RE |
2530 | .PP |
2531 | +\-\-allow-applied-failures |
2532 | +.RS 4 |
2533 | +Do not treat a patch application as a hard failure when importing the |
2534 | +patches-applied history\&. |
2535 | +As \fBdpkg-source\fR(1)'s (or other tool's) behavior has evolved over |
2536 | +time, it is possible that the version used by \fBgit ubuntu import\fR |
2537 | +is not able to reproduce the patch application of a historical |
2538 | +publication\&. |
2539 | +By default, such a failure will stop the historical import for the |
2540 | +patches-applied branch, to allow a \fBgit ubuntu\fR developer to |
2541 | +investigate\&. |
2542 | +After investigation, this flag can be used to indicate the importer is |
2543 | +allowed to ignore such a failure\&. |
2544 | +.RE |
2545 | +.PP |
2546 | <ppa> |
2547 | .RS 4 |
2548 | The name of the PPA to import from\&. |
2549 | diff --git a/man/man1/git-ubuntu-import.1 b/man/man1/git-ubuntu-import.1 |
2550 | index 129ecd9..c73f7ce 100644 |
2551 | --- a/man/man1/git-ubuntu-import.1 |
2552 | +++ b/man/man1/git-ubuntu-import.1 |
2553 | @@ -9,7 +9,7 @@ git-ubuntu import \- Import Launchpad publishing history to Git |
2554 | <user>] [\-\-dl-cache <dl_cache>] [\-\-no-fetch] [\-\-no-push] |
2555 | [\-\-no-clean] [\-d | \-\-directory <directory>] [\-\-fixup-devel] |
2556 | [\-\-active-series-only] [\-\-skip-applied] [\-\-skip-orig] |
2557 | -[\-\-reimport] <package> |
2558 | +[\-\-reimport] [\-\-allow-applied-failures] <package> |
2559 | .FI |
2560 | .SP |
2561 | .SH "DESCRIPTION" |
2562 | @@ -198,6 +198,21 @@ occurred) and forcibly \fBgit-push\fR the resulting repository to |
2563 | .RE |
2564 | .RE |
2565 | .PP |
2566 | +\-\-allow-applied-failures |
2567 | +.RS 4 |
2568 | +Do not treat a patch application as a hard failure when importing the |
2569 | +patches-applied history\&. |
2570 | +As \fBdpkg-source\fR(1)'s (or other tool's) behavior has evolved over |
2571 | +time, it is possible that the version used by \fBgit ubuntu import\fR |
2572 | +is not able to reproduce the patch application of a historical |
2573 | +publication\&. |
2574 | +By default, such a failure will stop the historical import for the |
2575 | +patches-applied branch, to allow a \fBgit ubuntu\fR developer to |
2576 | +investigate\&. |
2577 | +After investigation, this flag can be used to indicate the importer is |
2578 | +allowed to ignore such a failure\&. |
2579 | +.RE |
2580 | +.PP |
2581 | <package> |
2582 | .RS 4 |
2583 | The name of the source package to import\&. |
2584 | diff --git a/man/man1/git-ubuntu.1 b/man/man1/git-ubuntu.1 |
2585 | index 09230bb..9817e87 100644 |
2586 | --- a/man/man1/git-ubuntu.1 |
2587 | +++ b/man/man1/git-ubuntu.1 |
2588 | @@ -43,7 +43,7 @@ Useful for debugging and seeing what underlying commands are run\&. |
2589 | \-\-retries <num_retries> |
2590 | .RS 4 |
2591 | Number of retries to attempt for network interactions with Launchpad\&. |
2592 | -The default is 3 retry attempts\&. |
2593 | +The default is 5 retry attempts\&. |
2594 | .RE |
2595 | .PP |
2596 | \-\-retry-backoffs <backoff,\&.\&.\&.> |
2597 | diff --git a/setup.py b/setup.py |
2598 | index 75222db..ab91ab9 100644 |
2599 | --- a/setup.py |
2600 | +++ b/setup.py |
2601 | @@ -15,6 +15,7 @@ setup(name='gitubuntu', |
2602 | 'python-debian==0.1.31', |
2603 | 'pygit2==0.24.2', |
2604 | 'launchpadlib==1.10.5', |
2605 | + 'petname', |
2606 | 'setuptools', |
2607 | 'keyrings.alt==2.2', |
2608 | 'lazr.restfulclient==0.13.5', |
2609 | diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml |
2610 | index b18d430..6ddda16 100644 |
2611 | --- a/snap/snapcraft.yaml |
2612 | +++ b/snap/snapcraft.yaml |
2613 | @@ -25,6 +25,10 @@ apps: |
2614 | environment: |
2615 | LD_LIBRARY_PATH: "$SNAP/usr/lib/man-db:$SNAP/lib/x86_64-linux-gnu:$SNAP/usr/lib/x86_64-linux-gnu" |
2616 | PATH: "$SNAP/usr/local/sbin:$SNAP/usr/local/bin:$SNAP/usr/sbin:$SNAP/usr/bin:$SNAP/sbin:$SNAP/bin:$CORE_SNAP_PATH" |
2617 | + reconstruct-changelog: |
2618 | + command: wrappers/git-reconstruct-changelog |
2619 | + merge-changelogs: |
2620 | + command: wrappers/git-merge-changelogs |
2621 | |
2622 | parts: |
2623 | gbp: |
2624 | @@ -206,57 +210,6 @@ parts: |
2625 | stage-packages: |
2626 | - distro-info |
2627 | after: [python3] |
2628 | - libgpg-error: |
2629 | - plugin: autotools |
2630 | - source: https://gnupg.org/ftp/gcrypt/libgpg-error/libgpg-error-1.27.tar.bz2 |
2631 | - source-type: tar |
2632 | - configflags: [--prefix=/usr] |
2633 | - stage: |
2634 | - - -usr/share/info |
2635 | - libgcrypt: |
2636 | - plugin: autotools |
2637 | - source: https://gnupg.org/ftp/gcrypt/libgcrypt/libgcrypt-1.8.1.tar.bz2 |
2638 | - source-type: tar |
2639 | - configflags: [--prefix=/usr] |
2640 | - stage: |
2641 | - - -usr/share/info |
2642 | - libassuan: |
2643 | - plugin: autotools |
2644 | - source: https://gnupg.org/ftp/gcrypt/libassuan/libassuan-2.4.3.tar.bz2 |
2645 | - source-type: tar |
2646 | - configflags: [--prefix=/usr] |
2647 | - after: [libgpg-error] |
2648 | - stage: |
2649 | - - -usr/share/info |
2650 | - libksba: |
2651 | - plugin: autotools |
2652 | - source: https://gnupg.org/ftp/gcrypt/libksba/libksba-1.3.5.tar.bz2 |
2653 | - source-type: tar |
2654 | - configflags: [--prefix=/usr] |
2655 | - stage: |
2656 | - - -usr/share/info |
2657 | - npth: |
2658 | - plugin: autotools |
2659 | - source: https://gnupg.org/ftp/gcrypt/npth/npth-1.5.tar.bz2 |
2660 | - source-type: tar |
2661 | - configflags: [--prefix=/usr] |
2662 | - stage: |
2663 | - - -usr/share/info |
2664 | - gnupg2: |
2665 | - plugin: autotools |
2666 | - source: https://gnupg.org/ftp/gcrypt/gnupg/gnupg-2.2.1.tar.bz2 |
2667 | - source-type: tar |
2668 | - configflags: |
2669 | - - --prefix=/usr |
2670 | - - --enable-gpg2-is-gpg |
2671 | - after: |
2672 | - - libgpg-error |
2673 | - - libgcrypt |
2674 | - - libassuan |
2675 | - - libksba |
2676 | - - npth |
2677 | - stage: |
2678 | - - -usr/share/info |
2679 | devscripts: |
2680 | plugin: nil |
2681 | stage-packages: |
2682 | @@ -276,7 +229,6 @@ parts: |
2683 | - -usr/bin/xzgrep |
2684 | - -usr/bin/xzless |
2685 | - -usr/bin/xzmore |
2686 | - after: [gnupg2] |
2687 | git-ubuntu: |
2688 | plugin: python |
2689 | python-version: python3 |
2690 | @@ -299,14 +251,6 @@ parts: |
2691 | - -lib/python3.6/site-packages/oauth* |
2692 | - -lib/python3.6/site-packages/launchpadlib* |
2693 | - -usr/lib/python3.6 |
2694 | - - -usr/bin/gpg-error* |
2695 | - - -usr/share/aclocal/gpg-error.m4 |
2696 | - - -usr/bin/dumpsexp |
2697 | - - -usr/bin/hmac256 |
2698 | - - -usr/bin/libgcrypt-config |
2699 | - - -usr/bin/mpicalc |
2700 | - - -usr/include/gcrypt.h |
2701 | - - -usr/share/aclocal/libgcrypt.m4 |
2702 | prime: |
2703 | - -usr/share/doc |
2704 | install: | |
2705 | diff --git a/snap/wrappers/git-merge-changelogs b/snap/wrappers/git-merge-changelogs |
2706 | new file mode 100755 |
2707 | index 0000000..89af38d |
2708 | --- /dev/null |
2709 | +++ b/snap/wrappers/git-merge-changelogs |
2710 | @@ -0,0 +1,7 @@ |
2711 | +#!/snap/core/current/bin/bash |
2712 | + |
2713 | +CORE_SNAP="/snap/core/current" |
2714 | +CORE_SNAP_PATH="$CORE_SNAP/usr/local/sbin:$CORE_SNAP/usr/local/bin:$CORE_SNAP/usr/sbin:$CORE_SNAP/usr/bin:$CORE_SNAP/sbin:$CORE_SNAP/bin" |
2715 | +export PATH="$SNAP/bin/snap:$SNAP/usr/local/sbin:$SNAP/usr/local/bin:$SNAP/usr/sbin:$SNAP/usr/bin:$SNAP/sbin:$SNAP/bin:$CORE_SNAP_PATH" |
2716 | +export LD_LIBRARY_PATH="$SNAP/lib/x86_64-linux-gnu:$SNAP/usr/lib/x86_64-linux-gnu:$CORE_SNAP/lib/x86_64-linux-gnu:$CORE_SNAP/usr/lib/x86_64-linux-gnu" |
2717 | +exec "$SNAP/bin/git-merge-changelogs" "$@" |
2718 | diff --git a/snap/wrappers/git-reconstruct-changelog b/snap/wrappers/git-reconstruct-changelog |
2719 | new file mode 100755 |
2720 | index 0000000..356b1eb |
2721 | --- /dev/null |
2722 | +++ b/snap/wrappers/git-reconstruct-changelog |
2723 | @@ -0,0 +1,7 @@ |
2724 | +#!/snap/core/current/bin/bash |
2725 | + |
2726 | +CORE_SNAP="/snap/core/current" |
2727 | +CORE_SNAP_PATH="$CORE_SNAP/usr/local/sbin:$CORE_SNAP/usr/local/bin:$CORE_SNAP/usr/sbin:$CORE_SNAP/usr/bin:$CORE_SNAP/sbin:$CORE_SNAP/bin" |
2728 | +export PATH="$SNAP/bin/snap:$SNAP/usr/local/sbin:$SNAP/usr/local/bin:$SNAP/usr/sbin:$SNAP/usr/bin:$SNAP/sbin:$SNAP/bin:$CORE_SNAP_PATH" |
2729 | +export LD_LIBRARY_PATH="$SNAP/lib/x86_64-linux-gnu:$SNAP/usr/lib/x86_64-linux-gnu:$CORE_SNAP/lib/x86_64-linux-gnu:$CORE_SNAP/usr/lib/x86_64-linux-gnu" |
2730 | +exec "$SNAP/bin/git-reconstruct-changelog" "$@" |
FAILED: Continuous integration, rev:2b64fa48d7e 149c40179065753 d520224572f394 /jenkins. ubuntu. com/server/ job/git- ubuntu- ci/142/
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Style Check
SUCCESS: Unit Tests
FAILED: Integration Tests
Click here to trigger a rebuild: /jenkins. ubuntu. com/server/ job/git- ubuntu- ci/142/ rebuild
https:/