Merge ~nacc/git-ubuntu:gu-review into git-ubuntu:master
- Git
- lp:~nacc/git-ubuntu
- gu-review
- Merge into master
Status: | Superseded | ||||||||
---|---|---|---|---|---|---|---|---|---|
Proposed branch: | ~nacc/git-ubuntu:gu-review | ||||||||
Merge into: | git-ubuntu:master | ||||||||
Diff against target: |
8362 lines (+4219/-3553) 15 files modified
gitubuntu/__main__.py (+142/-123) gitubuntu/build.py (+132/-129) gitubuntu/buildsource.py (+55/-59) gitubuntu/clone.py (+109/-105) gitubuntu/importer.py (+1388/-1176) gitubuntu/importlocal.py (+312/-260) gitubuntu/importppa.py (+256/-175) gitubuntu/lint.py (+529/-506) gitubuntu/merge.py (+423/-355) gitubuntu/queue.py (+299/-281) gitubuntu/remote.py (+120/-97) gitubuntu/review.py (+117/-0) gitubuntu/source_information.py (+15/-0) gitubuntu/submit.py (+190/-174) gitubuntu/tag.py (+132/-113) |
||||||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Robie Basak | Needs Fixing | ||
Review via email: mp+328203@code.launchpad.net |
This proposal has been superseded by a proposal from 2017-09-06.
Commit message
Description of the change
This is probably not perfect, but it's basically (right now) just a wrapper around other commands.
Robie Basak (racb) : | # |
- 00c4243... by Nish Aravamudan
-
build/buildsource: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- 9f2c0f3... by Nish Aravamudan
-
clone: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- 980b55b... by Nish Aravamudan
-
import: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- ab02d0a... by Nish Aravamudan
-
importlocal: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- d37db34... by Nish Aravamudan
-
git_repository: move derive_
source_ from_series from build It will be used by multiple modules, move it to common code.
- 17d1c2d... by Nish Aravamudan
-
import: refactor import_orig
Rather than passing a spi in, pass the parts of the spi we immediately
derive. This allows the function to be re-used by importppa. - dc060d7... by Nish Aravamudan
-
source_information: raise ValueError if no person can be found
When we do PPA lookups, it's possible we are unable to find the supposed
PPA owner. Rather than failing in launchpadlib later, raise a ValueError
as the PPA is invalid. - 3909203... by Nish Aravamudan
-
importppa: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- 02928c7... by Nish Aravamudan
-
lint: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- 7b56298... by Nish Aravamudan
-
merge: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- 83da23b... by Nish Aravamudan
-
queue: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- 1eaa251... by Nish Aravamudan
-
remote: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- a0c296d... by Nish Aravamudan
-
submit: use lint.derive_
target_ branch submit can be used for distributed development among team members by
proposing MPs against each other's repositories, via the --target-user
flag. Rather than having submit-specific code for deriving the target
branch, use the existing code from lint, after extending it to take a
namespace argument. - b1d970f... by Nish Aravamudan
-
tag: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- cfcdc7a... by Nish Aravamudan
-
submit: drop unused pullfile member variable
- 2937467... by Nish Aravamudan
-
submit: fix implementation of --branch
We currently default to using HEAD, but allow the user to specify a
different branch to submit. However, we use HEAD in a few places
anyways, even if the user did specify --branch, which is incorrect. - cdc1cb0... by Nish Aravamudan
-
submit: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- b3ae5b8... by Nish Aravamudan
-
submit: fix stray tab
- 877e70f... by Nish Aravamudan
-
git ubuntu: refactor main to call module parse_args
This is the final step in the refactor -- rather than using per-module
classes, just import each module in __main__ in order to call its
parse_args and func (which in the current implementation is that
module's cli_main).Before merging with master, I think we will squash all of these
together.At this point, all errors from pylint3 are from pygit2 imports.
LP: #1707321
- f39ea92... by Nish Aravamudan
-
build: fix use of undefined variables
- 6591602... by Nish Aravamudan
-
lint: fix use of head_is_detached
- 9def1cf... by Nish Aravamudan
-
importppa: fix use of undefined variables
- a54e9ee... by Nish Aravamudan
-
lint: drop unused pullfile variable
- dbfe7bc... by Nish Aravamudan
-
clone: have main return the resulting directory path
- 16cab88... by Nish Aravamudan
-
clone: fix whitespace issue
- e2e5cce... by Nish Aravamudan
-
lint: linting historical merges with default arguments can fail
Typically, this is because pkg/debian/sid now is not the same as
pkg/debian/sid when the MP was proposed. This can lead to all sorts of
oddness, but should not be fatal to lint.One specific case is when the to-be-linted branch is older than the
target branch (e.g., a missed MP) and thus it only contains changelog
deletions.
Unmerged commits
- e0831bb... by Nish Aravamudan
-
git ubuntu review: add new subcommand
Given a MP URL, this attempts to provide a local directory to use for
reviewing the MP (and allows one to auto-approve with lint results).LP: #1702960
- 16cab88... by Nish Aravamudan
-
clone: fix whitespace issue
- e2e5cce... by Nish Aravamudan
-
lint: linting historical merges with default arguments can fail
Typically, this is because pkg/debian/sid now is not the same as
pkg/debian/sid when the MP was proposed. This can lead to all sorts of
oddness, but should not be fatal to lint.One specific case is when the to-be-linted branch is older than the
target branch (e.g., a missed MP) and thus it only contains changelog
deletions. - dbfe7bc... by Nish Aravamudan
-
clone: have main return the resulting directory path
- a54e9ee... by Nish Aravamudan
-
lint: drop unused pullfile variable
- 877e70f... by Nish Aravamudan
-
git ubuntu: refactor main to call module parse_args
This is the final step in the refactor -- rather than using per-module
classes, just import each module in __main__ in order to call its
parse_args and func (which in the current implementation is that
module's cli_main).Before merging with master, I think we will squash all of these
together.At this point, all errors from pylint3 are from pygit2 imports.
LP: #1707321
- cdc1cb0... by Nish Aravamudan
-
submit: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- a0c296d... by Nish Aravamudan
-
submit: use lint.derive_
target_ branch submit can be used for distributed development among team members by
proposing MPs against each other's repositories, via the --target-user
flag. Rather than having submit-specific code for deriving the target
branch, use the existing code from lint, after extending it to take a
namespace argument. - b1d970f... by Nish Aravamudan
-
tag: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
- 1eaa251... by Nish Aravamudan
-
remote: refactor class
Leave only parse_args and cli_main in the class. Make cli_main nothing
more than a wrapper to the module's main function.LP: #1707321
Preview Diff
1 | diff --git a/gitubuntu/__main__.py b/gitubuntu/__main__.py |
2 | index ba4a435..ea2de9f 100644 |
3 | --- a/gitubuntu/__main__.py |
4 | +++ b/gitubuntu/__main__.py |
5 | @@ -3,24 +3,13 @@ |
6 | def main(): |
7 | try: |
8 | import argparse |
9 | + from importlib import import_module |
10 | import logging |
11 | from os import isatty |
12 | import shutil |
13 | from subprocess import CalledProcessError |
14 | import sys |
15 | import textwrap |
16 | - from gitubuntu.importer import GitUbuntuImport |
17 | - from gitubuntu.build import GitUbuntuBuild |
18 | - from gitubuntu.buildsource import GitUbuntuBuildSource |
19 | - from gitubuntu.merge import GitUbuntuMerge |
20 | - from gitubuntu.clone import GitUbuntuClone |
21 | - from gitubuntu.tag import GitUbuntuTag |
22 | - from gitubuntu.queue import GitUbuntuQueue |
23 | - from gitubuntu.importlocal import GitUbuntuImportLocal |
24 | - from gitubuntu.importppa import GitUbuntuImportPPA |
25 | - from gitubuntu.remote import GitUbuntuRemote |
26 | - from gitubuntu.submit import GitUbuntuSubmit |
27 | - from gitubuntu.lint import GitUbuntuLint |
28 | from gitubuntu.run import run |
29 | |
30 | try: |
31 | @@ -34,123 +23,149 @@ def main(): |
32 | |
33 | logging.getLogger('keyring').setLevel(logging.WARNING) |
34 | |
35 | - known_subcommands = {'import':GitUbuntuImport, |
36 | - 'build':GitUbuntuBuild, |
37 | - 'build-source':GitUbuntuBuildSource, |
38 | - 'merge':GitUbuntuMerge, |
39 | - 'clone':GitUbuntuClone, |
40 | - 'tag':GitUbuntuTag, |
41 | - 'queue':GitUbuntuQueue, |
42 | - 'import-local':GitUbuntuImportLocal, |
43 | - 'import-ppa':GitUbuntuImportPPA, |
44 | - 'remote':GitUbuntuRemote, |
45 | - 'submit':GitUbuntuSubmit, |
46 | - 'lint':GitUbuntuLint, |
47 | - } |
48 | + known_subcommands = { |
49 | + 'import':'gitubuntu.importer', |
50 | + 'build':'gitubuntu.build', |
51 | + 'build-source':'gitubuntu.buildsource', |
52 | + 'merge':'gitubuntu.merge', |
53 | + 'clone':'gitubuntu.clone', |
54 | + 'tag':'gitubuntu.tag', |
55 | + 'queue':'gitubuntu.queue', |
56 | + 'import-local':'gitubuntu.importlocal', |
57 | + 'import-ppa':'gitubuntu.importppa', |
58 | + 'remote':'gitubuntu.remote', |
59 | + 'submit':'gitubuntu.submit', |
60 | + 'lint':'gitubuntu.lint', |
61 | + 'review':'gitubuntu.review', |
62 | + } |
63 | |
64 | - known_network_subcommands = {'import', 'import-local', |
65 | - 'import-ppa', 'clone', 'build', |
66 | - 'build-source', 'queue', |
67 | - 'remote', 'submit'} |
68 | + known_network_subcommands = { |
69 | + 'import', |
70 | + 'import-local', |
71 | + 'import-ppa', |
72 | + 'clone', |
73 | + 'build', |
74 | + 'build-source', |
75 | + 'queue', |
76 | + 'remote', |
77 | + 'submit', |
78 | + 'review', |
79 | + } |
80 | |
81 | parser = argparse.ArgumentParser( |
82 | description='Ubuntu git development tool', |
83 | formatter_class=argparse.RawTextHelpFormatter, |
84 | - epilog='More information is available at https://wiki.ubuntu.com/UbuntuDevelopment/Merging/GitWorkflow.' |
85 | - ) |
86 | - subparsers = parser.add_subparsers(dest='subcommand', |
87 | - help='', |
88 | - metavar='%s' % '|'.join(sorted(known_subcommands)) |
89 | - ) |
90 | + epilog='More information is available at https://wiki.ubuntu.com/UbuntuDevelopment/Merging/GitWorkflow.', |
91 | + ) |
92 | + subparsers = parser.add_subparsers( |
93 | + dest='subcommand', |
94 | + help='', |
95 | + metavar='%s' % '|'.join(sorted(known_subcommands)), |
96 | + ) |
97 | # This help ends up not being very useful to the end user |
98 | # subparsers.required = True |
99 | |
100 | # common flags to all subcommands |
101 | base_subparser = argparse.ArgumentParser(add_help=False) |
102 | - base_subparser.add_argument('-v', '--verbose', action='store_true', |
103 | - help='Increase verbosity') |
104 | + base_subparser.add_argument( |
105 | + '-v', '--verbose', |
106 | + action='store_true', |
107 | + help='Increase verbosity' |
108 | + ) |
109 | network_base_subparser = argparse.ArgumentParser(add_help=False) |
110 | network_base_subparser.add_argument( |
111 | - '--retries', type=int, |
112 | - help='Number of times to attempt to retry ' |
113 | - 'downloading from Launchpad.', |
114 | - default=3 |
115 | - ) |
116 | + '--retries', |
117 | + type=int, |
118 | + help='Number of times to attempt to retry ' |
119 | + 'downloading from Launchpad.', |
120 | + default=3, |
121 | + ) |
122 | network_base_subparser.add_argument( |
123 | - '--retry-backoffs', |
124 | - type=lambda s : [int(item) for item in s.split(',')], |
125 | - help='Comma-separated list of backoffs in ' |
126 | - 'seconds to use between each retry ' |
127 | - 'attempt. Default is exponential backoff.', |
128 | - default=argparse.SUPPRESS |
129 | - ) |
130 | + '--retry-backoffs', |
131 | + type=lambda s : [int(item) for item in s.split(',')], |
132 | + help='Comma-separated list of backoffs in ' |
133 | + 'seconds to use between each retry ' |
134 | + 'attempt. Default is exponential backoff.', |
135 | + default=argparse.SUPPRESS, |
136 | + ) |
137 | known_protos = ['git', 'http', 'https'] |
138 | network_base_subparser.add_argument( |
139 | - '--proto', |
140 | - metavar='[%s]' % '|'.join(known_protos), |
141 | - choices=known_protos, |
142 | - help='Specify protocol to use for fetch. Default: %(default)s', |
143 | - default='git' |
144 | - ) |
145 | + '--proto', |
146 | + metavar='[%s]' % '|'.join(known_protos), |
147 | + choices=known_protos, |
148 | + help='Specify protocol to use for fetch. Default: %(default)s', |
149 | + default='git', |
150 | + ) |
151 | |
152 | width, _ = shutil.get_terminal_size() |
153 | subhelp_width = width - 30 |
154 | for sub in sorted(known_subcommands): |
155 | - s = known_subcommands[sub]() |
156 | + m = import_module(known_subcommands[sub]) |
157 | if sub in known_network_subcommands: |
158 | - help_text = s.parse_args(subparsers, [base_subparser, network_base_subparser]) |
159 | + help_text = m.parse_args( |
160 | + subparsers, |
161 | + [base_subparser, network_base_subparser], |
162 | + ) |
163 | else: |
164 | - help_text = s.parse_args(subparsers, [base_subparser]) |
165 | + help_text = m.parse_args(subparsers, [base_subparser]) |
166 | subparsers.help += '\n' |
167 | - subparsers.help += '\n'.join(textwrap.wrap(help_text, |
168 | - subhelp_width, |
169 | - break_on_hyphens=False |
170 | - ) |
171 | - ) |
172 | - help_text = '\n'.join(textwrap.wrap('File specifying ' |
173 | - 'parent-child relationship ' |
174 | - 'overrides as:', |
175 | - subhelp_width |
176 | - ) |
177 | - ) |
178 | + subparsers.help += '\n'.join( |
179 | + textwrap.wrap( |
180 | + help_text, |
181 | + subhelp_width, |
182 | + break_on_hyphens=False, |
183 | + ) |
184 | + ) |
185 | + help_text = '\n'.join( |
186 | + textwrap.wrap( |
187 | + "File specifying parent-child relationship overrides as:", |
188 | + subhelp_width, |
189 | + ) |
190 | + ) |
191 | help_text += '\n' |
192 | help_text += '<pkgname> <child version> <publish parent version> <changelog parent version>\n' |
193 | - help_text += '\n'.join(textwrap.wrap('with one override per ' |
194 | - 'line.', |
195 | - subhelp_width |
196 | - ) |
197 | - ) |
198 | + help_text += '\n'.join( |
199 | + textwrap.wrap( |
200 | + "with one override per line.", |
201 | + subhelp_width, |
202 | + ) |
203 | + ) |
204 | help_text += '\ndefault=%(default)s' |
205 | - parser.add_argument('-P', '--parentfile', type=str, |
206 | - help=argparse.SUPPRESS, |
207 | - default=pkg_resources.resource_filename( |
208 | - 'gitubuntu', |
209 | - 'parent_overrides.txt' |
210 | - ) |
211 | - ) |
212 | - help_text = '\n'.join(textwrap.wrap('File specifying source ' |
213 | - 'files to use instead of ' |
214 | - 'Launchpad to work around ' |
215 | - 'source file shadowing: ', |
216 | - subhelp_width |
217 | - ) |
218 | - ) |
219 | + parser.add_argument( |
220 | + '-P', '--parentfile', |
221 | + type=str, |
222 | + help=argparse.SUPPRESS, |
223 | + default=pkg_resources.resource_filename( |
224 | + 'gitubuntu', |
225 | + 'parent_overrides.txt', |
226 | + ), |
227 | + ) |
228 | + help_text = '\n'.join( |
229 | + textwrap.wrap( |
230 | + "File specifying source files to use instead of " |
231 | + "Launchpad to work around source file shadowing: ", |
232 | + subhelp_width, |
233 | + ) |
234 | + ) |
235 | help_text += '\n' |
236 | help_text += '<pkgname> <version> <URL of DSC file> <URLs of orig files>\n' |
237 | - help_text += '\n'.join(textwrap.wrap('with one package per line. ' |
238 | - '\'-\' for the DSC file ' |
239 | - 'indicates to use the one ' |
240 | - 'from Launchpad.' |
241 | - ) |
242 | - ) |
243 | + help_text += '\n'.join( |
244 | + textwrap.wrap( |
245 | + "with one package per line. '-' for the DSC file " |
246 | + "indicates to use the one from Launchpad.", |
247 | + subhelp_width, |
248 | + ) |
249 | + ) |
250 | help_text += '\ndefault=%(default)s' |
251 | - parser.add_argument('-L', '--pullfile', type=str, |
252 | - help=argparse.SUPPRESS, |
253 | - default=pkg_resources.resource_filename( |
254 | - 'gitubuntu', |
255 | - 'pull_overrides.txt' |
256 | - ) |
257 | - ) |
258 | + parser.add_argument( |
259 | + '-L', '--pullfile', |
260 | + type=str, |
261 | + help=argparse.SUPPRESS, |
262 | + default=pkg_resources.resource_filename( |
263 | + 'gitubuntu', |
264 | + 'pull_overrides.txt', |
265 | + ), |
266 | + ) |
267 | |
268 | argcomplete.autocomplete(parser) |
269 | args = parser.parse_args() |
270 | @@ -161,22 +176,22 @@ def main(): |
271 | # fragile, assumes developers will pass the base_subparser above |
272 | # should make this structural in the classes |
273 | if args.verbose: |
274 | - logging.basicConfig(level=logging.DEBUG, |
275 | - format='%(asctime)s - %(levelname)s:%(message)s', |
276 | - datefmt='%m/%d/%Y %H:%M:%S', |
277 | - ) |
278 | + level=logging.DEBUG |
279 | else: |
280 | - logging.basicConfig(level=logging.INFO, |
281 | - format='%(asctime)s - %(levelname)s:%(message)s', |
282 | - datefmt='%m/%d/%Y %H:%M:%S', |
283 | - ) |
284 | + level=logging.INFO |
285 | + logging.basicConfig( |
286 | + level=level, |
287 | + format='%(asctime)s - %(levelname)s:%(message)s', |
288 | + datefmt='%m/%d/%Y %H:%M:%S', |
289 | + ) |
290 | if args.subcommand in known_network_subcommands: |
291 | try: |
292 | retry_backoffs = args.retry_backoffs |
293 | if len(retry_backoffs) != args.retries: |
294 | - logging.error('Number of backoffs specified in ' |
295 | - '--retry-backoffs must match --retries.' |
296 | - ) |
297 | + logging.error( |
298 | + "Number of backoffs specified in " |
299 | + "--retry-backoffs must match --retries." |
300 | + ) |
301 | sys.exit(1) |
302 | except AttributeError: |
303 | args.retry_backoffs = [2 ** i for i in range(args.retries)] |
304 | @@ -188,26 +203,30 @@ def main(): |
305 | ) |
306 | except CalledProcessError: |
307 | if isatty(sys.stdin.fileno()): |
308 | - user = input("gitubuntu.lpuser is not set. What is your " |
309 | + user = input( |
310 | + "gitubuntu.lpuser is not set. What is your " |
311 | "Launchpad username? We will set this in your " |
312 | "personally global git configuration for you. " |
313 | "To abort, press Enter.\nUsername: " |
314 | ) |
315 | if len(user) == 0: |
316 | - logging.error('git-ubuntu requires a launchpad user') |
317 | + logging.error("git-ubuntu requires a launchpad user") |
318 | sys.exit(1) |
319 | else: |
320 | - logging.warning('Setting the Launchpad user for ' |
321 | - 'git-ubuntu to %s with ' |
322 | - '`git config --global gitubuntu.lpuser %s`.', |
323 | - user, user) |
324 | + logging.warning( |
325 | + "Setting the Launchpad user for git-ubuntu to %s " |
326 | + "with `git config --global gitubuntu.lpuser %s`.", |
327 | + user, |
328 | + user, |
329 | + ) |
330 | run(['git', 'config', '--global', 'gitubuntu.lpuser', user]) |
331 | else: |
332 | - logging.error('Not connected to a TTY and no ' |
333 | - 'git-ubuntu lpuser defined. Please run ' |
334 | - '`git config --global gitubuntu.lpuser ' |
335 | - '<Launchpad username>` and re-run this ' |
336 | - 'command.') |
337 | + logging.error( |
338 | + "Not connected to a TTY and no git-ubuntu lpuser " |
339 | + "defined. Please run `git config --global " |
340 | + "gitubuntu.lpuser <Launchpad username>` and re-run " |
341 | + "this command." |
342 | + ) |
343 | sys.exit(1) |
344 | |
345 | args.func(args) |
346 | diff --git a/gitubuntu/build.py b/gitubuntu/build.py |
347 | index 8143995..d3d292e 100644 |
348 | --- a/gitubuntu/build.py |
349 | +++ b/gitubuntu/build.py |
350 | @@ -38,12 +38,11 @@ from gitubuntu.run import run, runq |
351 | from gitubuntu.source_information import ( |
352 | GitUbuntuSourceInformation, |
353 | NoPublicationHistoryException, |
354 | + derive_source_from_series, |
355 | ) |
356 | try: |
357 | pkg = 'python3-debian' |
358 | from debian.debfile import PART_EXTS |
359 | - pkg = 'python3-distro-info' |
360 | - from distro_info import DebianDistroInfo, UbuntuDistroInfo |
361 | pkg = 'python3-pytest' |
362 | import pytest |
363 | except ImportError: |
364 | @@ -228,8 +227,7 @@ def fetch_orig_from_cache(changelog, source, dl_cache=None): |
365 | return None |
366 | if dsc.verify() is None: |
367 | logging.warn( |
368 | - "Cache dir found, but verification of orig tarball %s failed.", |
369 | - orig, |
370 | + "Cache dir found, but verification of orig tarball(s) failed." |
371 | ) |
372 | return None # XXX decide - see docstring |
373 | _symlink_paths_into_parent_dir(dsc.all_tarball_paths) |
374 | @@ -320,8 +318,8 @@ def fetch_orig_from_launchpad(changelog, source, pullfile, retries, |
375 | except NoPublicationHistoryException: |
376 | logging.warning( |
377 | "No publication history found for %s in %s. ", |
378 | - pkgname, |
379 | - dist_name, |
380 | + changelog.srcpkg, |
381 | + source, |
382 | ) |
383 | return None |
384 | |
385 | @@ -374,151 +372,156 @@ def fetch_orig_from_launchpad(changelog, source, pullfile, retries, |
386 | return None |
387 | |
388 | |
389 | -class GitUbuntuBuild: |
390 | - def parse_args(self, subparsers=None, base_subparsers=None): |
391 | - kwargs = dict( |
392 | - description='Build a git-ubuntu-cloned tree with dpkg-buildpackage', |
393 | - formatter_class=argparse.ArgumentDefaultsHelpFormatter, |
394 | - usage='%(prog)s [options] -- ...', |
395 | - epilog='Any arguments after -- are passed to dpkg-buildpackage.' |
396 | - ) |
397 | - if base_subparsers: |
398 | - kwargs['parents'] = base_subparsers |
399 | - if subparsers: |
400 | - parser = subparsers.add_parser('build', **kwargs) |
401 | - parser.set_defaults(func=self.main) |
402 | - else: |
403 | - parser = argparse.ArgumentParser(**kwargs) |
404 | - parser.add_argument('--dl-cache', type=str, |
405 | - help='Cache directory for downloads.', |
406 | - default=os.path.join(os.getcwd(), '.git', CACHE_PATH), |
407 | - ) |
408 | - parser.add_argument( |
409 | - '--for-merge', |
410 | - action='store_true', |
411 | - help="Build an Ubuntu merge. Implies searching Debian for " |
412 | - "published versions first.", |
413 | +def check_repository(): |
414 | + try: |
415 | + cp = run(['git', 'status', '--porcelain']) |
416 | + except CalledProcessError: |
417 | + logging.error('Is the current directory a git repository?') |
418 | + sys.exit(1) |
419 | + |
420 | + if len(cp.stdout) > 0: |
421 | + logging.warning( |
422 | + "Working tree is not clean. `git status` output follows." |
423 | ) |
424 | - parser.add_argument( |
425 | - '--no-pristine-tar', |
426 | - action='store_true', |
427 | - help="Do not use pristine-tar branches to generate orig " |
428 | - "tarballs. This can be necessary as we stabilize the pristine-tar " |
429 | - "data generation, particularly if multiple component tarballs are " |
430 | - "present.", |
431 | + run(['git', 'status'], stdout=None) |
432 | + sys.exit(1) |
433 | |
434 | + if not os.path.isfile('debian/changelog'): |
435 | + logging.error( |
436 | + "No debian/changelog found, is this a source " |
437 | + "package repository? See `git ubuntu clone`." |
438 | ) |
439 | - parser.add_argument('rem_args', help=argparse.SUPPRESS, |
440 | - nargs=argparse.REMAINDER) |
441 | - if not subparsers: |
442 | - return parser.parse_args() |
443 | - return 'build - %s' % kwargs['description'] |
444 | + sys.exit(1) |
445 | |
446 | - def check_repository(self): |
447 | - try: |
448 | - cp = run(['git', 'status', '--porcelain']) |
449 | - except CalledProcessError: |
450 | - logging.error('Is the current directory a git repository?') |
451 | - sys.exit(1) |
452 | |
453 | - if len(cp.stdout) > 0: |
454 | - logging.warning( |
455 | - "Working tree is not clean. `git status` output follows." |
456 | - ) |
457 | - run(['git', 'status'], stdout=None) |
458 | - sys.exit(1) |
459 | +def main(search_list, changelog, rem_args): |
460 | + """main entry point for build |
461 | |
462 | - if not os.path.isfile('debian/changelog'): |
463 | - logging.error( |
464 | - "No debian/changelog found, is this a source " |
465 | - "package repository? See `git ubuntu clone`." |
466 | - ) |
467 | - sys.exit(1) |
468 | + @param search_list: list of OrigSearchListEntry namedtuples |
469 | + @param changelog: gitubuntu.git_repository.Changelog object for source package to build |
470 | + @param rem_args: namespace object of remaining arguments to pass onto dpkg-buildpackage |
471 | + """ |
472 | + check_repository() |
473 | + |
474 | + fetch_orig_and_build(search_list, changelog, rem_args) |
475 | + |
476 | + |
477 | +def parse_args(subparsers=None, base_subparsers=None): |
478 | + kwargs = dict( |
479 | + description='Build a git-ubuntu-cloned tree with dpkg-buildpackage', |
480 | + formatter_class=argparse.ArgumentDefaultsHelpFormatter, |
481 | + usage='%(prog)s [options] -- ...', |
482 | + epilog='Any arguments after -- are passed to dpkg-buildpackage.' |
483 | + ) |
484 | + if base_subparsers: |
485 | + kwargs['parents'] = base_subparsers |
486 | + if subparsers: |
487 | + parser = subparsers.add_parser('build', **kwargs) |
488 | + parser.set_defaults(func=cli_main) |
489 | + else: |
490 | + parser = argparse.ArgumentParser(**kwargs) |
491 | + parser.add_argument('--dl-cache', type=str, |
492 | + help='Cache directory for downloads.', |
493 | + default=os.path.join(os.getcwd(), '.git', CACHE_PATH), |
494 | + ) |
495 | + parser.add_argument( |
496 | + '--for-merge', |
497 | + action='store_true', |
498 | + help="Build an Ubuntu merge. Implies searching Debian for " |
499 | + "published versions first.", |
500 | + ) |
501 | + parser.add_argument( |
502 | + '--no-pristine-tar', |
503 | + action='store_true', |
504 | + help="Do not use pristine-tar branches to generate orig " |
505 | + "tarballs. This can be necessary as we stabilize the pristine-tar " |
506 | + "data generation, particularly if multiple component tarballs are " |
507 | + "present.", |
508 | |
509 | - def derive_orig_search_list_from_args(self, args): |
510 | - source = 'debian' if args.for_merge else 'ubuntu' |
511 | - orig_search_list = [ |
512 | - OrigSearchListEntry( |
513 | - mechanism=fetch_orig_from_parent_dir, |
514 | - source=None, |
515 | - must_build=True, |
516 | - ), |
517 | - OrigSearchListEntry( |
518 | - mechanism=fetch_orig_from_cache, |
519 | - source=source, |
520 | - must_build=True, |
521 | - ), |
522 | - ] |
523 | - if not args.no_pristine_tar: |
524 | - orig_search_list.append( |
525 | - OrigSearchListEntry( |
526 | - mechanism=functools.partial(fetch_orig_from_pristine_tar, |
527 | - repo=self.local_repo, |
528 | - ), |
529 | - source=source, |
530 | - must_build=True, |
531 | - ) |
532 | - ) |
533 | - orig_search_list.extend([ |
534 | + ) |
535 | + parser.add_argument('rem_args', help=argparse.SUPPRESS, |
536 | + nargs=argparse.REMAINDER) |
537 | + if not subparsers: |
538 | + return parser.parse_args() |
539 | + return 'build - %s' % kwargs['description'] |
540 | + |
541 | + |
542 | +def derive_orig_search_list_from_args(args): |
543 | + source = 'debian' if args.for_merge else 'ubuntu' |
544 | + orig_search_list = [ |
545 | + OrigSearchListEntry( |
546 | + mechanism=fetch_orig_from_parent_dir, |
547 | + source=None, |
548 | + must_build=True, |
549 | + ), |
550 | + OrigSearchListEntry( |
551 | + mechanism=fetch_orig_from_cache, |
552 | + source=source, |
553 | + must_build=True, |
554 | + ), |
555 | + ] |
556 | + if not args.no_pristine_tar: |
557 | + orig_search_list.append( |
558 | OrigSearchListEntry( |
559 | - mechanism=functools.partial(fetch_orig_from_launchpad, |
560 | - pullfile=args.pullfile, |
561 | - retries=args.retries, |
562 | - retry_backoffs=args.retry_backoffs, |
563 | - dl_cache=args.dl_cache, |
564 | + mechanism=functools.partial(fetch_orig_from_pristine_tar, |
565 | + repo=GitUbuntuRepository('.'), |
566 | ), |
567 | source=source, |
568 | must_build=True, |
569 | + ) |
570 | + ) |
571 | + orig_search_list.extend([ |
572 | + OrigSearchListEntry( |
573 | + mechanism=functools.partial(fetch_orig_from_launchpad, |
574 | + pullfile=args.pullfile, |
575 | + retries=args.retries, |
576 | + retry_backoffs=args.retry_backoffs, |
577 | + dl_cache=args.dl_cache, |
578 | ), |
579 | - OrigSearchListEntry( |
580 | - mechanism=fetch_orig_noop, |
581 | - source=[], |
582 | - must_build=True, |
583 | - ), |
584 | - ]) |
585 | + source=source, |
586 | + must_build=True, |
587 | + ), |
588 | + OrigSearchListEntry( |
589 | + mechanism=fetch_orig_noop, |
590 | + source=[], |
591 | + must_build=True, |
592 | + ), |
593 | + ]) |
594 | |
595 | - return orig_search_list |
596 | + return orig_search_list |
597 | |
598 | |
599 | - def main(self, args): |
600 | - rem_args = args.rem_args |
601 | - if len(rem_args) > 0: |
602 | - rem_args = rem_args[1:] |
603 | +def cli_main(args): |
604 | + rem_args = args.rem_args |
605 | + if len(rem_args) > 0: |
606 | + rem_args = rem_args[1:] |
607 | |
608 | - self.check_repository() |
609 | - self.local_repo = GitUbuntuRepository('.') |
610 | + changelog = Changelog.from_path('debian/changelog') |
611 | |
612 | - changelog = Changelog.from_path('debian/changelog') |
613 | + try: |
614 | + native = is_native_package(changelog) |
615 | + except NativenessMismatchError as e: |
616 | + logging.error("%s" % e) |
617 | + sys.exit(1) |
618 | |
619 | - try: |
620 | - native = is_native_package(changelog) |
621 | - except NativenessMismatchError as e: |
622 | - logging.error("%s" % e) |
623 | - sys.exit(1) |
624 | - |
625 | - if native: |
626 | - # No orig tarball required |
627 | - orig_search_list = [ |
628 | - OrigSearchListEntry( |
629 | - mechanism=fetch_orig_noop, |
630 | - source=[], |
631 | - must_build=True, |
632 | - ), |
633 | - ] |
634 | - else: |
635 | - orig_search_list = self.derive_orig_search_list_from_args(args) |
636 | + if native: |
637 | + # No orig tarball required |
638 | + orig_search_list = [ |
639 | + OrigSearchListEntry( |
640 | + mechanism=fetch_orig_noop, |
641 | + source=[], |
642 | + must_build=True, |
643 | + ), |
644 | + ] |
645 | + else: |
646 | + orig_search_list = derive_orig_search_list_from_args(args) |
647 | |
648 | - # See http://pad.ubuntu.com/KKB1kMR0JH for logic |
649 | - fetch_orig_and_build(orig_search_list, changelog, rem_args) |
650 | + # See http://pad.ubuntu.com/KKB1kMR0JH for logic |
651 | + main(orig_search_list, changelog, rem_args) |
652 | |
653 | |
654 | def derive_source_from_changelog(changelog): |
655 | - series = changelog.distribution |
656 | - if DebianDistroInfo().valid(codename=series): |
657 | - return 'debian' |
658 | - if UbuntuDistroInfo().valid(codename=series): |
659 | - return 'ubuntu' |
660 | - raise ValueError("Unable to determine distribution from %s" % series) |
661 | + return derive_source_from_series(changelog.distribution) |
662 | |
663 | def expand_changelog_source_aliases(orig_search_list, changelog): |
664 | """Replace 'changelog' sources by reading debian/changelog |
665 | diff --git a/gitubuntu/buildsource.py b/gitubuntu/buildsource.py |
666 | index 4e9b69b..1735129 100644 |
667 | --- a/gitubuntu/buildsource.py |
668 | +++ b/gitubuntu/buildsource.py |
669 | @@ -5,66 +5,62 @@ import argparse |
670 | import logging |
671 | import os |
672 | import sys |
673 | -from gitubuntu.build import GitUbuntuBuild |
674 | +from gitubuntu.build import cli_main as build_cli_main |
675 | from gitubuntu.cache import CACHE_PATH |
676 | |
677 | -class GitUbuntuBuildSource(GitUbuntuBuild): |
678 | - def __init__(self): |
679 | - super().__init__() |
680 | +def parse_args(subparsers=None, base_subparsers=None): |
681 | + kwargs = dict( |
682 | + description="Build a source package and changes file", |
683 | + formatter_class=argparse.ArgumentDefaultsHelpFormatter |
684 | + ) |
685 | + if base_subparsers: |
686 | + kwargs['parents'] = base_subparsers |
687 | + if subparsers: |
688 | + parser = subparsers.add_parser('build-source', **kwargs) |
689 | + help_text = "build-source - Build a source package" |
690 | + parser.set_defaults(func=cli_main) |
691 | + else: |
692 | + parser = argparse.ArgumentParser(**kwargs) |
693 | + parser.add_argument('--dl-cache', type=str, |
694 | + help=("Cache directory for downloads."), |
695 | + default=os.path.join(os.getcwd(), '.git', CACHE_PATH), |
696 | + ) |
697 | + parser.add_argument('--sign', action='store_true', |
698 | + help="Sign the source package.", |
699 | + ) |
700 | + parser.add_argument('--for-merge', action='store_true', |
701 | + help=("Build an Ubuntu merge. Implies searching Debian for " |
702 | + "published versions first and passing -sa to " |
703 | + "dpkg-buildpackage." |
704 | + ), |
705 | + ) |
706 | + parser.add_argument( |
707 | + '--no-pristine-tar', |
708 | + action='store_true', |
709 | + help="Do not use pristine-tar branches to generate orig " |
710 | + "tarballs. This can be necessary as we stabilize the pristine-tar " |
711 | + "data generation, particularly if multiple component tarballs are " |
712 | + "present.", |
713 | |
714 | - def parse_args(self, subparsers=None, base_subparsers=None): |
715 | - kwargs = dict( |
716 | - description="Build a source package and changes file", |
717 | - formatter_class=argparse.ArgumentDefaultsHelpFormatter |
718 | - ) |
719 | - if base_subparsers: |
720 | - kwargs['parents'] = base_subparsers |
721 | - if subparsers: |
722 | - parser = subparsers.add_parser('build-source', **kwargs) |
723 | - help_text = "build-source - Build a source package" |
724 | - parser.set_defaults(func=self.main) |
725 | - else: |
726 | - parser = argparse.ArgumentParser(**kwargs) |
727 | - parser.add_argument('--dl-cache', type=str, |
728 | - help=("Cache directory for downloads."), |
729 | - default=os.path.join(os.getcwd(), '.git', CACHE_PATH), |
730 | - ) |
731 | - parser.add_argument('--sign', action='store_true', |
732 | - help="Sign the source package.", |
733 | - ) |
734 | - parser.add_argument('--for-merge', action='store_true', |
735 | - help=("Build an Ubuntu merge. Implies searching Debian for " |
736 | - "published versions first and passing -sa to " |
737 | - "dpkg-buildpackage." |
738 | - ), |
739 | - ) |
740 | - parser.add_argument( |
741 | - '--no-pristine-tar', |
742 | - action='store_true', |
743 | - help="Do not use pristine-tar branches to generate orig " |
744 | - "tarballs. This can be necessary as we stabilize the pristine-tar " |
745 | - "data generation, particularly if multiple component tarballs are " |
746 | - "present.", |
747 | + ) |
748 | + parser.add_argument('rem_args', help=argparse.SUPPRESS, |
749 | + nargs=argparse.REMAINDER |
750 | + ) |
751 | + if not subparsers: |
752 | + return parser.parse_args() |
753 | + return "build-source - %s" % kwargs['description'] |
754 | |
755 | - ) |
756 | - parser.add_argument('rem_args', help=argparse.SUPPRESS, |
757 | - nargs=argparse.REMAINDER |
758 | - ) |
759 | - if not subparsers: |
760 | - return parser.parse_args() |
761 | - return "build-source - %s" % kwargs['description'] |
762 | - |
763 | - def main(self, args): |
764 | - if len(args.rem_args) != 0: |
765 | - logging.warning('Passing specified flags (%s) on to ' |
766 | - 'dpkg-buildpackage. Did you mean to ' |
767 | - 'call `git ubuntu build` instead?' % |
768 | - ' '.join(args.rem_args) |
769 | - ) |
770 | - else: |
771 | - args.rem_args = ['--', '-S', '-nc', '-d', '-i', '-I'] |
772 | - if args.for_merge: |
773 | - args.rem_args += ['-sa'] |
774 | - if not args.sign: |
775 | - args.rem_args += ['-us', '-uc'] |
776 | - super().main(args) |
777 | +def cli_main(args): |
778 | + if len(args.rem_args) != 0: |
779 | + logging.warning('Passing specified flags (%s) on to ' |
780 | + 'dpkg-buildpackage. Did you mean to ' |
781 | + 'call `git ubuntu build` instead?' % |
782 | + ' '.join(args.rem_args) |
783 | + ) |
784 | + else: |
785 | + args.rem_args = ['--', '-S', '-nc', '-d', '-i', '-I'] |
786 | + if args.for_merge: |
787 | + args.rem_args += ['-sa'] |
788 | + if not args.sign: |
789 | + args.rem_args += ['-us', '-uc'] |
790 | + build_cli_main(args) |
791 | diff --git a/gitubuntu/clone.py b/gitubuntu/clone.py |
792 | index a1bc8cb..6fdc8e9 100644 |
793 | --- a/gitubuntu/clone.py |
794 | +++ b/gitubuntu/clone.py |
795 | @@ -18,12 +18,82 @@ except ImportError: |
796 | logging.error('Is %s installed?', pkg) |
797 | sys.exit(1) |
798 | |
799 | -class GitUbuntuClone: |
800 | - def parse_args(self, subparsers=None, base_subparsers=None): |
801 | - kwargs = dict( |
802 | - description='Clone a source package git repository to a directory', |
803 | - formatter_class=argparse.RawTextHelpFormatter, |
804 | - epilog=''' |
805 | +def copy_hooks(src, dst): |
806 | + try: |
807 | + os.mkdir(dst) |
808 | + except FileExistsError: |
809 | + pass |
810 | + |
811 | + for hook in os.listdir(src): |
812 | + shutil.copy2( |
813 | + os.path.join(src, hook), |
814 | + dst, |
815 | + ) |
816 | + |
817 | +def main(directory, lp_user, proto, package): |
818 | + if os.path.isdir(directory): |
819 | + logging.error('directory %s exists' % directory) |
820 | + sys.exit(1) |
821 | + |
822 | + local_repo = GitUbuntuRepository( |
823 | + local_dir=directory, |
824 | + lp_user=lp_user, |
825 | + fetch_proto=proto, |
826 | + ) |
827 | + |
828 | + copy_hooks( |
829 | + os.path.join(os.path.dirname(__file__), os.path.pardir, 'hooks'), |
830 | + os.path.join(directory, '.git', 'hooks') |
831 | + ) |
832 | + |
833 | + try: |
834 | + local_repo.add_base_remotes(package) |
835 | + local_repo.fetch_base_remotes(verbose=True) |
836 | + except GitUbuntuRepositoryFetchError: |
837 | + logging.error("Unable to find an imported repository for %s. " |
838 | + "Please request an import by e-mailing " |
839 | + "usd-import-team@lists.launchpad.net.", |
840 | + package |
841 | + ) |
842 | + shutil.rmtree(local_repo.local_dir) |
843 | + raise |
844 | + |
845 | + try: |
846 | + local_repo.add_lpuser_remote(pkgname=package) |
847 | + local_repo.fetch_lpuser_remote(verbose=True) |
848 | + |
849 | + logging.debug("added remote '%s' -> %s", local_repo.lp_user, |
850 | + local_repo.raw_repo.remotes[local_repo.lp_user].url |
851 | + ) |
852 | + except GitUbuntuRepositoryFetchError: |
853 | + pass |
854 | + |
855 | + try: |
856 | + local_repo.create_tracking_branch( |
857 | + 'ubuntu/devel', |
858 | + 'pkg/ubuntu/devel' |
859 | + ) |
860 | + local_repo.checkout_commitish('ubuntu/devel') |
861 | + except: |
862 | + logging.error('Unable to checkout ubuntu/devel, does ' |
863 | + 'pkg/ubuntu/devel branch exist?' |
864 | + ) |
865 | + |
866 | + if os.path.isfile(os.path.join(directory, '.gitignore')): |
867 | + logging.warning('A .gitignore file exists in the source ' |
868 | + 'package. This will affect the behavior of git. Consider ' |
869 | + 'backing up the gitignore while working on this package ' |
870 | + 'to ensure all changes are tracked or passing appropriate ' |
871 | + 'flags to git commands (e.g., git status --ignored).' |
872 | + ) |
873 | + |
874 | + return directory |
875 | + |
876 | +def parse_args(subparsers=None, base_subparsers=None): |
877 | + kwargs = dict( |
878 | + description='Clone a source package git repository to a directory', |
879 | + formatter_class=argparse.RawTextHelpFormatter, |
880 | + epilog=''' |
881 | Example: |
882 | * clone to open-iscsi/ |
883 | %(prog)s open-iscsi |
884 | @@ -32,107 +102,41 @@ Example: |
885 | * use https rather than git protocol for read-only remotes: |
886 | %(prog)s --https open-iscsi |
887 | ''' |
888 | - ) |
889 | - if base_subparsers: |
890 | - kwargs['parents'] = base_subparsers |
891 | - if subparsers: |
892 | - parser = subparsers.add_parser('clone', **kwargs) |
893 | - parser.set_defaults(func=self.main) |
894 | - else: |
895 | - parser = argparse.ArgumentParser(**kwargs) |
896 | - parser.add_argument('package', type=str, |
897 | - help='Name of source package to clone' |
898 | - ) |
899 | - parser.add_argument('directory', type=str, |
900 | - help='Local directory to clone to. If not specified, a ' |
901 | - ' directory with the same name as PACKAGE will be ' |
902 | - 'used', |
903 | - default=None, |
904 | - nargs='?' |
905 | - ) |
906 | - parser.add_argument('-l', '--lp-user', type=str, help=argparse.SUPPRESS) |
907 | - if not subparsers: |
908 | - return parser.parse_args() |
909 | - return 'clone - %s' % kwargs['description'] |
910 | - |
911 | - @staticmethod |
912 | - def copy_hooks(src, dst): |
913 | - try: |
914 | - os.mkdir(dst) |
915 | - except FileExistsError: |
916 | - pass |
917 | - |
918 | - for hook in os.listdir(src): |
919 | - shutil.copy2( |
920 | - os.path.join(src, hook), |
921 | - dst, |
922 | - ) |
923 | - |
924 | - def main(self, args): |
925 | - try: |
926 | - lp_user = args.lp_user |
927 | - except AttributeError: |
928 | - lp_user = None |
929 | - directory = ( |
930 | - os.path.abspath(args.directory) |
931 | - if args.directory |
932 | - else os.path.join(os.path.abspath(os.getcwd()), args.package) |
933 | - ) |
934 | + ) |
935 | + if base_subparsers: |
936 | + kwargs['parents'] = base_subparsers |
937 | + if subparsers: |
938 | + parser = subparsers.add_parser('clone', **kwargs) |
939 | + parser.set_defaults(func=cli_main) |
940 | + else: |
941 | + parser = argparse.ArgumentParser(**kwargs) |
942 | + parser.add_argument('package', type=str, |
943 | + help='Name of source package to clone' |
944 | + ) |
945 | + parser.add_argument('directory', type=str, |
946 | + help='Local directory to clone to. If not specified, a ' |
947 | + 'directory with the same name as PACKAGE will be ' |
948 | + 'used', |
949 | + default=None, |
950 | + nargs='?' |
951 | + ) |
952 | + parser.add_argument('-l', '--lp-user', type=str, help=argparse.SUPPRESS) |
953 | + if not subparsers: |
954 | + return parser.parse_args() |
955 | + return 'clone - %s' % kwargs['description'] |
956 | |
957 | - if os.path.isdir(directory): |
958 | - logging.error('directory %s exists' % directory) |
959 | - sys.exit(1) |
960 | +def cli_main(args): |
961 | + try: |
962 | + lp_user = args.lp_user |
963 | + except AttributeError: |
964 | + lp_user = None |
965 | + directory = ( |
966 | + os.path.abspath(args.directory) |
967 | + if args.directory |
968 | + else os.path.join(os.path.abspath(os.getcwd()), args.package) |
969 | + ) |
970 | |
971 | - local_repo = GitUbuntuRepository( |
972 | - local_dir=directory, |
973 | - lp_user=lp_user, |
974 | - fetch_proto=args.proto, |
975 | - ) |
976 | - |
977 | - self.copy_hooks( |
978 | - os.path.join(os.path.dirname(__file__), os.path.pardir, 'hooks'), |
979 | - os.path.join(directory, '.git', 'hooks') |
980 | - ) |
981 | + main(directory, lp_user, args.proto, args.package) |
982 | |
983 | - try: |
984 | - local_repo.add_base_remotes(args.package) |
985 | - local_repo.fetch_base_remotes(verbose=True) |
986 | - except GitUbuntuRepositoryFetchError: |
987 | - logging.error("Unable to find an imported repository for %s. " |
988 | - "Please request an import by e-mailing " |
989 | - "usd-import-team@lists.launchpad.net.", |
990 | - args.package |
991 | - ) |
992 | - shutil.rmtree(local_repo.local_dir) |
993 | - raise |
994 | - |
995 | - try: |
996 | - local_repo.add_lpuser_remote(pkgname=args.package) |
997 | - local_repo.fetch_lpuser_remote(verbose=True) |
998 | - |
999 | - logging.debug("added remote '%s' -> %s", local_repo.lp_user, |
1000 | - local_repo.raw_repo.remotes[local_repo.lp_user].url |
1001 | - ) |
1002 | - except GitUbuntuRepositoryFetchError: |
1003 | - pass |
1004 | - |
1005 | - try: |
1006 | - local_repo.create_tracking_branch( |
1007 | - 'ubuntu/devel', |
1008 | - 'pkg/ubuntu/devel' |
1009 | - ) |
1010 | - local_repo.checkout_commitish('ubuntu/devel') |
1011 | - except: |
1012 | - logging.error('Unable to checkout ubuntu/devel, does ' |
1013 | - 'pkg/ubuntu/devel branch exist?' |
1014 | - ) |
1015 | - |
1016 | - if os.path.isfile(os.path.join(directory, '.gitignore')): |
1017 | - logging.warning('A .gitignore file exists in the source ' |
1018 | - 'package. This will affect the behavior of git. Consider ' |
1019 | - 'backing up the gitignore while working on this package ' |
1020 | - 'to ensure all changes are tracked or passing appropriate ' |
1021 | - 'flags to git commands (e.g., git status --ignored).' |
1022 | - ) |
1023 | |
1024 | # vi: ts=4 expandtab |
1025 | diff --git a/gitubuntu/importer.py b/gitubuntu/importer.py |
1026 | index 22fd08a..c3903cb 100644 |
1027 | --- a/gitubuntu/importer.py |
1028 | +++ b/gitubuntu/importer.py |
1029 | @@ -26,6 +26,7 @@ |
1030 | |
1031 | import argparse |
1032 | import atexit |
1033 | +import functools |
1034 | import getpass |
1035 | import logging |
1036 | import os |
1037 | @@ -50,7 +51,12 @@ from gitubuntu.git_repository import ( |
1038 | PristineTarError, |
1039 | ) |
1040 | from gitubuntu.run import decode_binary, run, runq |
1041 | -from gitubuntu.source_information import GitUbuntuSourceInformation, NoPublicationHistoryException, SourceExtractionException, launchpad_login_auth |
1042 | +from gitubuntu.source_information import ( |
1043 | + GitUbuntuSourceInformation, |
1044 | + NoPublicationHistoryException, |
1045 | + SourceExtractionException, |
1046 | + launchpad_login_auth, |
1047 | +) |
1048 | from gitubuntu.version import VERSION |
1049 | from gitubuntu.versioning import version_compare |
1050 | try: |
1051 | @@ -86,16 +92,18 @@ def dsc_to_tree_hash(repo, dsc_path): |
1052 | # extracted_dir. |
1053 | |
1054 | extracted_dir = os.path.join(temp_dir, 'x') |
1055 | - run(['dpkg-source', |
1056 | - '-x', |
1057 | - '--skip-patches', |
1058 | - dsc_path, |
1059 | - extracted_dir, # this is <temp_dir>/x |
1060 | + run( |
1061 | + [ |
1062 | + 'dpkg-source', |
1063 | + '-x', |
1064 | + '--skip-patches', |
1065 | + dsc_path, |
1066 | + extracted_dir, # this is <temp_dir>/x |
1067 | ] |
1068 | - ) |
1069 | + ) |
1070 | |
1071 | if not os.path.exists(extracted_dir): |
1072 | - logging.exception('No source extracted?') |
1073 | + logging.error("No source extracted?") |
1074 | raise SourceExtractionException( |
1075 | "Failed to find an extracted directory from " |
1076 | "dpkg-source -x" |
1077 | @@ -104,1258 +112,1462 @@ def dsc_to_tree_hash(repo, dsc_path): |
1078 | return repo.dir_to_tree(extracted_dir) |
1079 | |
1080 | |
1081 | -class GitUbuntuImport: |
1082 | - def __init__(self): |
1083 | - self.parent_overrides = None |
1084 | - |
1085 | - def parse_args(self, subparsers=None, base_subparsers=None): |
1086 | - kwargs = dict(description='Update a launchpad git tree based upon ' |
1087 | - 'the state of the Ubuntu and Debian archives', |
1088 | - formatter_class=argparse.ArgumentDefaultsHelpFormatter |
1089 | - ) |
1090 | - if base_subparsers: |
1091 | - kwargs['parents'] = base_subparsers |
1092 | - if subparsers: |
1093 | - parser = subparsers.add_parser('import', **kwargs) |
1094 | - parser.set_defaults(func=self.main) |
1095 | - else: |
1096 | - parser = argparse.ArgumentParser(**kwargs) |
1097 | - parser.add_argument('package', type=str, |
1098 | - help='Which package to update in the git tree') |
1099 | - parser.add_argument('-o', '--lp-owner', type=str, |
1100 | - help=argparse.SUPPRESS, |
1101 | - default='usd-import-team') |
1102 | - parser.add_argument('-l', '--lp-user', type=str, |
1103 | - help=argparse.SUPPRESS) |
1104 | - parser.add_argument('--dl-cache', type=str, |
1105 | - help=('Cache directory for downloads. Default is to ' |
1106 | - 'put downloads in <directory>/.git/%s' % |
1107 | - CACHE_PATH |
1108 | - ), |
1109 | - default=argparse.SUPPRESS) |
1110 | - parser.add_argument('--no-fetch', action='store_true', |
1111 | - help='Do not fetch from the remote (DANGEROUS; ' |
1112 | - 'only useful for debugging); implies ' |
1113 | - '--no-push') |
1114 | - parser.add_argument('--no-push', action='store_true', |
1115 | - help='Do not push to the remote') |
1116 | - parser.add_argument('--no-clean', action='store_true', |
1117 | - help='Do not clean the temporary directory') |
1118 | - parser.add_argument('-d', '--directory', type=str, |
1119 | - help='Use git repository at specified location rather ' |
1120 | - 'than a temporary directory (implies --no-clean). ' |
1121 | - 'Will create the local repository if needed.', |
1122 | - default=argparse.SUPPRESS |
1123 | - ) |
1124 | - parser.add_argument('--fixup-devel', action='store_true', |
1125 | - help=argparse.SUPPRESS) |
1126 | - parser.add_argument('--active-series-only', action='store_true', |
1127 | - help='Do an import of only active Ubuntu series ' |
1128 | - 'history, implies --no-push.') |
1129 | - parser.add_argument('--skip-applied', action='store_true', |
1130 | - help=argparse.SUPPRESS) |
1131 | - parser.add_argument('--skip-orig', action='store_true', |
1132 | - help=argparse.SUPPRESS) |
1133 | - parser.add_argument('--reimport', action='store_true', |
1134 | - help=argparse.SUPPRESS) |
1135 | - if not subparsers: |
1136 | - return parser.parse_args() |
1137 | - return 'import - %s' % kwargs['description'] |
1138 | - |
1139 | - def cleanup(self, no_clean, local_dir): |
1140 | - """Recursively remove local_dir if no_clean is not True""" |
1141 | - if not no_clean: |
1142 | - shutil.rmtree(local_dir) |
1143 | - else: |
1144 | - logging.info('Leaving %s as directed' % local_dir) |
1145 | - |
1146 | - def get_changelog_for_commit(self, tree_hash, publish_parent_commit, changelog_parent_commit): |
1147 | - """Extract changes to debian/changelog in this publish |
1148 | - |
1149 | - LP: #1633114 -- favor the changelog parent's diff |
1150 | - """ |
1151 | - raw_clog_entry = b'' |
1152 | - if changelog_parent_commit is not None: |
1153 | - cmd = ['diff-tree', '-p', changelog_parent_commit, |
1154 | - tree_hash, '--', 'debian/changelog'] |
1155 | - raw_clog_entry = self.local_repo.git_run(cmd).stdout |
1156 | - elif publish_parent_commit is not None: |
1157 | - cmd = ['diff-tree', '-p', publish_parent_commit, |
1158 | - tree_hash, '--', 'debian/changelog'] |
1159 | - raw_clog_entry = self.local_repo.git_run(cmd).stdout |
1160 | - |
1161 | - changelog_entry = b'' |
1162 | - changelog_entry_found = False |
1163 | - i = 0 |
1164 | - for line in raw_clog_entry.split(b'\n'): |
1165 | - # skip the header lines |
1166 | - if i < 4: |
1167 | - i += 1 |
1168 | - continue |
1169 | - if not line.startswith(b'+'): |
1170 | - # in case there are stray changelog changes, just take |
1171 | - # the first complete section of changelog additions |
1172 | - if changelog_entry_found: |
1173 | - break |
1174 | - continue |
1175 | - line = re.sub(b'^[+]', b'', line) |
1176 | - if not line.startswith(b' '): |
1177 | - continue |
1178 | - if not changelog_entry_found: |
1179 | - changelog_entry = b'\n\nNew changelog entries:\n' |
1180 | - changelog_entry += line + b'\n' |
1181 | - changelog_entry_found = True |
1182 | - return changelog_entry |
1183 | - |
1184 | - def _commit_import(self, spi, tree_hash, publish_parent_commit, changelog_parent_commit, upload_parent_commit, unapplied_parent_commit): |
1185 | - """Commit a tree object into the repository with the specified |
1186 | - parents |
1187 | - |
1188 | - The importer algorithm works by 'staging' the import of a |
1189 | - publication as a tree, then using that tree to determine the |
1190 | - appropriate parents from the publishing history and |
1191 | - debian/changelog. |
1192 | - |
1193 | - Arguments: |
1194 | - spi - A SourcePackageInformation instance for this publish |
1195 | - tree_hash - SHA1 from git-dsc-commit --tree-only of this publish |
1196 | - publish_parent_commit = SHA1 of publishing parent |
1197 | - changelog_parent_commit = SHA1 of changelog parent |
1198 | - upload_parent_commit = SHA1 of upload parent |
1199 | - unapplied_parent_commit = SHA1 of unapplied-patches import parent |
1200 | - """ |
1201 | - tag = None |
1202 | - |
1203 | - if unapplied_parent_commit: |
1204 | - import_type = 'patches-applied' |
1205 | - target_head_name = spi.applied_head_name(self.namespace) |
1206 | - if self.local_repo.get_applied_tag(spi.version, self.namespace) is None: |
1207 | - # Not imported before |
1208 | - tag = applied_tag(spi.version, self.namespace) |
1209 | - else: |
1210 | - import_type = 'patches-unapplied' |
1211 | - target_head_name = spi.head_name(self.namespace) |
1212 | - if self.local_repo.get_import_tag(spi.version, self.namespace) is None: |
1213 | - # Not imported before |
1214 | - tag = import_tag(spi.version, self.namespace) |
1215 | - |
1216 | - # Do not show importer/ namespace to user |
1217 | - _, _, pretty_head_name = target_head_name.partition('%s/' % self.namespace) |
1218 | - |
1219 | - changelog_entry = self.get_changelog_for_commit( |
1220 | - tree_hash, |
1221 | - publish_parent_commit, |
1222 | - changelog_parent_commit |
1223 | - ) |
1224 | - msg = (b'Import %s version %b to %b\n\nImported using git-ubuntu import.' % |
1225 | - (import_type.encode(), spi.version.encode(), pretty_head_name.encode()) |
1226 | - ) |
1227 | - |
1228 | - parents = [] |
1229 | - |
1230 | - if publish_parent_commit is None and \ |
1231 | - changelog_parent_commit is None and \ |
1232 | - upload_parent_commit is None and \ |
1233 | - unapplied_parent_commit is None and \ |
1234 | - target_head_name in self.local_repo.local_branch_names: |
1235 | - # No parents and not creating a new branch |
1236 | - tag = orphan_tag(spi.version, self.namespace) |
1237 | - else: |
1238 | - msg += b'\n' |
1239 | - |
1240 | - if publish_parent_commit is not None: |
1241 | - parents.append(publish_parent_commit) |
1242 | - msg += b'\nPublish parent: %b' % publish_parent_commit.encode() |
1243 | - if changelog_parent_commit is not None: |
1244 | - parents.append(changelog_parent_commit) |
1245 | - msg += b'\nChangelog parent: %b' % changelog_parent_commit.encode() |
1246 | - if upload_parent_commit is not None: |
1247 | - parents.append(upload_parent_commit) |
1248 | - msg += b'\nUpload parent: %b' % upload_parent_commit.encode() |
1249 | - if unapplied_parent_commit is not None: |
1250 | - parents.append(unapplied_parent_commit) |
1251 | - msg += b'\nUnapplied parent: %b' % unapplied_parent_commit.encode() |
1252 | - |
1253 | - msg += b'%b' % changelog_entry |
1254 | - |
1255 | - commit_hash = self.local_repo.commit_tree_hash( |
1256 | - tree_hash, |
1257 | - parents, |
1258 | - msg, |
1259 | - environment_spi=spi |
1260 | - ) |
1261 | +def cleanup(no_clean, local_dir): |
1262 | + '''Remove a local directory, conditionally |
1263 | |
1264 | - self.local_repo.update_head_to_commit(target_head_name, commit_hash) |
1265 | + no_clean: if True, do not remove @local_dir and print a message instead |
1266 | + local_dir: directory to remove |
1267 | + ''' |
1268 | + if not no_clean: |
1269 | + shutil.rmtree(local_dir) |
1270 | + else: |
1271 | + logging.info('Leaving %s as directed' % local_dir) |
1272 | |
1273 | - logging.debug('Committed %s import of %s as %s in %s', |
1274 | - import_type, spi.version, commit_hash, pretty_head_name |
1275 | - ) |
1276 | |
1277 | - if tag is not None: |
1278 | - # should be annotated to use create_tag API |
1279 | - logging.debug('Creating tag %s pointing to %s', tag, commit_hash) |
1280 | - self.local_repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION, |
1281 | - tag, commit_hash |
1282 | - ] |
1283 | - ) |
1284 | +# XXX: need a namedtuple to hold common arguments |
1285 | +def main(pkgname, owner, no_clean, directory, user, proto, no_fetch, |
1286 | + no_push, dl_cache, fixup_devel, active_series_only, skip_orig, |
1287 | + pullfile, parentfile, retries, retry_backoffs, skip_applied, |
1288 | + reimport, |
1289 | +): |
1290 | + if owner == 'usd-import-team': |
1291 | + namespace = 'importer' |
1292 | + else: |
1293 | + namespace = owner |
1294 | |
1295 | - def commit_unapplied_patches_import(self, spi, import_tree_hash, publish_parent_commit, changelog_parent_commit, upload_parent_commit): |
1296 | - self._commit_import( |
1297 | - spi, |
1298 | - import_tree_hash, |
1299 | - publish_parent_commit, |
1300 | - changelog_parent_commit, |
1301 | - upload_parent_commit, |
1302 | - # unapplied trees do not have a distinct unapplied parent |
1303 | - None |
1304 | - ) |
1305 | + logging.info('Ubuntu Server Team importer v%s' % VERSION) |
1306 | |
1307 | - def commit_applied_patches_import(self, spi, import_tree_hash, publish_parent_commit, changelog_parent_commit, unapplied_parent_commit): |
1308 | - self._commit_import( |
1309 | - spi, |
1310 | - import_tree_hash, |
1311 | - publish_parent_commit, |
1312 | - changelog_parent_commit, |
1313 | - # uploads will be unapplied trees currently, so applied trees |
1314 | - # can not have them as direct parents |
1315 | - None, |
1316 | - unapplied_parent_commit |
1317 | - ) |
1318 | + repo = GitUbuntuRepository(directory, user, proto) |
1319 | + if not directory: |
1320 | + logging.info('Using git repository at %s', repo.local_dir) |
1321 | + |
1322 | + atexit.register(cleanup, no_clean, repo.local_dir) |
1323 | |
1324 | - def import_dsc(self, dsc, version, dist): |
1325 | - """Imports the dsc file to importer/{debian,ubuntu}/dsc |
1326 | - |
1327 | - Arguments: |
1328 | - spi - A GitUbuntuSourcePackageInformation instance |
1329 | - """ |
1330 | - |
1331 | - # Add DSC file to the appropriate branch |
1332 | - runq(['git', 'checkout', '%s/importer/%s/dsc' % (self.namespace, dist)], env=self.local_repo.env) |
1333 | - # It is possible the same-named DSC is published multiple times |
1334 | - # with different contents, e.g. on epoch bumps |
1335 | - local_dir = os.path.join(self.local_repo.local_dir, version) |
1336 | - os.makedirs(local_dir, exist_ok=True) |
1337 | - shutil.copy(dsc.dsc_path, |
1338 | - local_dir) |
1339 | - self.local_repo.git_run(['add', self.local_repo.local_dir]) |
1340 | + repo.add_base_remotes(pkgname, repo_owner=owner) |
1341 | + if not no_fetch and not reimport: |
1342 | try: |
1343 | - # Anything to commit? (this will be 0 if, for instance, the |
1344 | - # same dsc is being imported again) |
1345 | - runq(['git', 'diff', '--exit-code', 'HEAD'], env=self.local_repo.env) |
1346 | - except: |
1347 | - self.local_repo.git_run(['commit', '-m', 'DSC file for %s' % version]) |
1348 | + repo.fetch_base_remotes() |
1349 | + except GitUbuntuRepositoryFetchError: |
1350 | + pass |
1351 | |
1352 | - self.local_repo.clean_repository_state() |
1353 | + if not no_fetch: |
1354 | + repo.delete_branches_in_namespace(namespace) |
1355 | + repo.delete_tags_in_namespace(namespace) |
1356 | + repo.copy_base_references(namespace) |
1357 | |
1358 | - def orig_imported(self, orig_tarball_paths, dist): |
1359 | - """Determines if a list of paths to tarballs have already been imported to @dist |
1360 | + if not no_fetch and not reimport: |
1361 | + try: |
1362 | + repo.fetch_remote_refspecs('pkg', |
1363 | + refspecs=['refs/tags/*:refs/tags/%s/*' % namespace], |
1364 | + ) |
1365 | + except GitUbuntuRepositoryFetchError: |
1366 | + pass |
1367 | |
1368 | - Assumes that the pristine-tar branch exists |
1369 | - """ |
1370 | + if reimport: |
1371 | + if no_fetch: |
1372 | + logging.warning( |
1373 | + "--reimport specified with --no-fetch. Upload " |
1374 | + "tags would not be incorporated into the new import " |
1375 | + "and would be lost forever. This is not currently " |
1376 | + "supported and --no-fetch will be ignored for upload " |
1377 | + "tags." |
1378 | + ) |
1379 | try: |
1380 | - return self.local_repo.verify_pristine_tar( |
1381 | - orig_tarball_paths, |
1382 | - dist, |
1383 | - self.namespace |
1384 | + repo.fetch_remote_refspecs('pkg', |
1385 | + refspecs=['refs/tags/upload/*:refs/tags/%s/upload/*' % namespace], |
1386 | ) |
1387 | - except PristineTarError as e: |
1388 | - raise GitUbuntuImportOrigError from e |
1389 | + except GitUbuntuRepositoryFetchError: |
1390 | + pass |
1391 | |
1392 | - def import_orig(self, spi): |
1393 | - """Imports the orig-tarball using gbp import-org --pristine-tar |
1394 | + repo.ensure_importer_branches_exist(namespace) |
1395 | + |
1396 | + debian_sinfo = GitUbuntuSourceInformation( |
1397 | + 'debian', |
1398 | + pkgname, |
1399 | + os.path.abspath(pullfile), |
1400 | + retries, |
1401 | + retry_backoffs, |
1402 | + ) |
1403 | + |
1404 | + ubuntu_sinfo = GitUbuntuSourceInformation( |
1405 | + 'ubuntu', |
1406 | + pkgname, |
1407 | + os.path.abspath(pullfile), |
1408 | + retries, |
1409 | + retry_backoffs, |
1410 | + ) |
1411 | + |
1412 | + debian_head_versions = ( |
1413 | + repo.get_heads_and_versions('debian', namespace) |
1414 | + ) |
1415 | + for head_name in sorted(debian_head_versions): |
1416 | + _, _, pretty_head_name = head_name.partition('%s/' % namespace) |
1417 | + logging.debug('Last imported %s version is %s', |
1418 | + pretty_head_name, |
1419 | + debian_head_versions[head_name]['version'] |
1420 | + ) |
1421 | |
1422 | - Arguments: |
1423 | - spi - A GitUbuntuSourcePackageInformation instance |
1424 | - """ |
1425 | - dsc = GitUbuntuDsc(spi.dsc_pathname) |
1426 | - upstream_version = str(spi.upstream_version) |
1427 | - version = str(spi.version) |
1428 | - dist = spi.distribution.name.lower() |
1429 | + ubuntu_head_versions = ( |
1430 | + repo.get_heads_and_versions('ubuntu', namespace) |
1431 | + ) |
1432 | + for head_name in sorted(ubuntu_head_versions): |
1433 | + _, _, pretty_head_name = head_name.partition('%s/' % namespace) |
1434 | + logging.debug('Last imported %s version is %s', |
1435 | + pretty_head_name, |
1436 | + ubuntu_head_versions[head_name]['version'] |
1437 | + ) |
1438 | |
1439 | - try: |
1440 | - orig_tarball_path = dsc.orig_tarball_path |
1441 | - except GitUbuntuDscError as e: |
1442 | - raise GitUbuntuImportOrigError( |
1443 | - 'Unable to import orig tarball in DSC for version %s' % version |
1444 | - ) from e |
1445 | - |
1446 | - if orig_tarball_path is None: |
1447 | - # native packages only have a .tar.* |
1448 | - native_re = re.compile(r'.*\.tar\.[^.]+$') |
1449 | - for entry in dsc['Files']: |
1450 | - if native_re.match(entry['name']): |
1451 | - return |
1452 | - raise GitUbuntuImportOrigError('No orig tarball in ' |
1453 | - 'DSC for version %s.' % version) |
1454 | + if not skip_applied: |
1455 | + applied_debian_head_versions = ( |
1456 | + repo.get_heads_and_versions( |
1457 | + 'applied/debian', namespace |
1458 | + ) |
1459 | + ) |
1460 | + for head_name in sorted(applied_debian_head_versions): |
1461 | + _, _, pretty_head_name = head_name.partition( |
1462 | + '%s/' % namespace |
1463 | + ) |
1464 | + logging.debug('Last applied %s version is %s', |
1465 | + pretty_head_name, |
1466 | + applied_debian_head_versions[head_name]['version'] |
1467 | + ) |
1468 | |
1469 | - try: |
1470 | - orig_component_paths = dsc.component_tarball_paths |
1471 | - except GitUbuntuDscError as e: |
1472 | - raise GitUbuntuImportOrigError( |
1473 | - 'Unable to import component tarballs in DSC for version ' |
1474 | - '%s' % version |
1475 | - ) from e |
1476 | - |
1477 | - orig_paths = [orig_tarball_path] + list(orig_component_paths.values()) |
1478 | - if (not orig_tarball_path or |
1479 | - self.orig_imported(orig_paths, dist)): |
1480 | - return |
1481 | - if not all(map(os.path.exists, orig_paths)): |
1482 | - raise GitUbuntuImportOrigError( |
1483 | - 'Unable to find tarball: %s' % |
1484 | - [p for p in orig_paths if not os.path.exists(p)] |
1485 | + applied_ubuntu_head_versions = ( |
1486 | + repo.get_heads_and_versions( |
1487 | + 'applied/ubuntu', namespace |
1488 | + ) |
1489 | + ) |
1490 | + for head_name in sorted(applied_ubuntu_head_versions): |
1491 | + _, _, pretty_head_name = head_name.partition( |
1492 | + '%s/' % namespace |
1493 | + ) |
1494 | + logging.debug('Last applied %s version is %s', |
1495 | + pretty_head_name, |
1496 | + applied_ubuntu_head_versions[head_name]['version'] |
1497 | ) |
1498 | |
1499 | - try: |
1500 | - with self.local_repo.pristine_tar_branches(dist, self.namespace): |
1501 | - oldcwd = os.getcwd() |
1502 | - # gbp does not support running from arbitrary git trees or |
1503 | - # working directories |
1504 | - # https://github.com/agx/git-buildpackage/pull/16 |
1505 | - os.chdir(self.local_repo.local_dir) |
1506 | - |
1507 | - ext = os.path.splitext(dsc.orig_tarball_path)[1] |
1508 | - |
1509 | - # make this non-fatal if upstream already exist as tagged? |
1510 | - cmd = ['gbp', 'import-orig', '--no-merge', |
1511 | - '--upstream-branch', 'do-not-push', |
1512 | - '--pristine-tar', '--no-interactive', |
1513 | - '--no-symlink-orig', |
1514 | - '--upstream-tag=%s/upstream/%s/%%(version)s%s' % |
1515 | - (self.namespace, dist, ext)] |
1516 | - cmd.extend(map(lambda x: '--component=%s' % x, |
1517 | - list(orig_component_paths.keys())) |
1518 | - ) |
1519 | - cmd.extend([orig_tarball_path,]) |
1520 | - run(cmd, env=self.local_repo.env) |
1521 | - except CalledProcessError as e: |
1522 | - raise GitUbuntuImportOrigError( |
1523 | - 'Unable to import tarballs: %s' % orig_paths |
1524 | - ) from e |
1525 | - finally: |
1526 | - os.chdir(oldcwd) |
1527 | + oldcwd = os.getcwd() |
1528 | + os.chdir(repo.local_dir) |
1529 | + |
1530 | + if dl_cache is None: |
1531 | + workdir = os.path.join(repo.git_dir, CACHE_PATH) |
1532 | + else: |
1533 | + workdir = dl_cache |
1534 | + |
1535 | + os.makedirs(workdir, exist_ok=True) |
1536 | + |
1537 | + # now sets a global _PARENT_OVERRIDES |
1538 | + parse_parentfile(parentfile, pkgname) |
1539 | + |
1540 | + only_debian, history_found = import_publishes( |
1541 | + repo=repo, |
1542 | + pkgname=pkgname, |
1543 | + namespace=namespace, |
1544 | + patches_applied=False, |
1545 | + debian_head_versions=debian_head_versions, |
1546 | + ubuntu_head_versions=ubuntu_head_versions, |
1547 | + debian_sinfo=debian_sinfo, |
1548 | + ubuntu_sinfo=ubuntu_sinfo, |
1549 | + active_series_only=active_series_only, |
1550 | + workdir=workdir, |
1551 | + fixup_devel=fixup_devel, |
1552 | + skip_orig=skip_orig, |
1553 | + ) |
1554 | + |
1555 | + if not history_found: |
1556 | + logging.error("No publication history for '%s' in debian or ubuntu. " |
1557 | + "Wrong source package name?", pkgname) |
1558 | + sys.exit(1) |
1559 | + |
1560 | + if not skip_applied: |
1561 | + import_publishes( |
1562 | + repo=repo, |
1563 | + pkgname=pkgname, |
1564 | + namespace=namespace, |
1565 | + patches_applied=True, |
1566 | + debian_head_versions=applied_debian_head_versions, |
1567 | + ubuntu_head_versions=applied_ubuntu_head_versions, |
1568 | + debian_sinfo=debian_sinfo, |
1569 | + ubuntu_sinfo=ubuntu_sinfo, |
1570 | + active_series_only=active_series_only, |
1571 | + workdir=workdir, |
1572 | + fixup_devel=fixup_devel, |
1573 | + skip_orig=skip_orig, |
1574 | + ) |
1575 | |
1576 | - self.local_repo.clean_repository_state() |
1577 | + os.chdir(oldcwd) |
1578 | |
1579 | - def import_patches_unapplied_tree(self, dsc_pathname): |
1580 | - """Imports the patches-unapplied source package and writes the |
1581 | - corresponding working tree for a given srcpkg |
1582 | + repo.garbage_collect() |
1583 | |
1584 | - Arguments: |
1585 | - dsc_pathname - A string path to a DSC file |
1586 | - """ |
1587 | + if no_push: |
1588 | + logging.info('Not pushing to remote as specified') |
1589 | + else: |
1590 | + lp = launchpad_login_auth() |
1591 | + repo_path = '~%s/ubuntu/+source/%s/+git/%s' % ( |
1592 | + owner, |
1593 | + pkgname, |
1594 | + pkgname, |
1595 | + ) |
1596 | + lp_git_repo = lp.git_repositories.getByPath(path=repo_path) |
1597 | + if reimport: |
1598 | + if not lp_git_repo.canBeDeleted(): |
1599 | + logging.warning( |
1600 | + "There are linked MPs to the pkg repository, " |
1601 | + "which will be deleted:" |
1602 | + ) |
1603 | + for mp in lp_git_repo.landing_candidates: |
1604 | + logging.warning( |
1605 | + "Adding comment to %s indicating reimport", |
1606 | + mp.web_link |
1607 | + ) |
1608 | + mp.createComment( |
1609 | + content="%s has been reimported and this MP " |
1610 | + "must be manually resubmitted." % |
1611 | + pkgname, |
1612 | + subject="%s has been reimported" % pkgname, |
1613 | + ) |
1614 | + # repo deletion will delete the MP, but hopefully |
1615 | + # the above comment will be sufficient to explain |
1616 | + # why |
1617 | + lp_git_repo.lp_delete() |
1618 | + |
1619 | + repo.git_run([ |
1620 | + 'push', '--atomic', 'pkg', |
1621 | + 'refs/heads/%s/*:refs/heads/*' % namespace, |
1622 | + 'refs/tags/%s/*:refs/tags/*' % namespace, |
1623 | + ]) |
1624 | + # update our reference |
1625 | + lp_git_repo = lp.git_repositories.getByPath(path=repo_path) |
1626 | + for i in range(retries): |
1627 | + try: |
1628 | + if only_debian: |
1629 | + lp_git_repo.default_branch = 'refs/heads/debian/sid' |
1630 | + else: |
1631 | + lp_git_repo.default_branch = 'refs/heads/ubuntu/devel' |
1632 | + lp_git_repo.lp_save() |
1633 | + break |
1634 | + except (NotFound, PreconditionFailed) as e: |
1635 | + time.sleep(retry_backoffs[i]) |
1636 | + lp_git_repo.lp_refresh() |
1637 | + |
1638 | + |
1639 | +def get_changelog_for_commit( |
1640 | + repo, |
1641 | + tree_hash, |
1642 | + publish_parent_commit, |
1643 | + changelog_parent_commit, |
1644 | +): |
1645 | + """Extract changes to debian/changelog in this publish |
1646 | + |
1647 | + LP: #1633114 -- favor the changelog parent's diff |
1648 | + """ |
1649 | + raw_clog_entry = b'' |
1650 | + if changelog_parent_commit is not None: |
1651 | + cmd = ['diff-tree', '-p', changelog_parent_commit, |
1652 | + tree_hash, '--', 'debian/changelog'] |
1653 | + raw_clog_entry = repo.git_run(cmd).stdout |
1654 | + elif publish_parent_commit is not None: |
1655 | + cmd = ['diff-tree', '-p', publish_parent_commit, |
1656 | + tree_hash, '--', 'debian/changelog'] |
1657 | + raw_clog_entry = repo.git_run(cmd).stdout |
1658 | + |
1659 | + changelog_entry = b'' |
1660 | + changelog_entry_found = False |
1661 | + i = 0 |
1662 | + for line in raw_clog_entry.split(b'\n'): |
1663 | + # skip the header lines |
1664 | + if i < 4: |
1665 | + i += 1 |
1666 | + continue |
1667 | + if not line.startswith(b'+'): |
1668 | + # in case there are stray changelog changes, just take |
1669 | + # the first complete section of changelog additions |
1670 | + if changelog_entry_found: |
1671 | + break |
1672 | + continue |
1673 | + line = re.sub(b'^[+]', b'', line) |
1674 | + if not line.startswith(b' '): |
1675 | + continue |
1676 | + if not changelog_entry_found: |
1677 | + changelog_entry = b'\n\nNew changelog entries:\n' |
1678 | + changelog_entry += line + b'\n' |
1679 | + changelog_entry_found = True |
1680 | + return changelog_entry |
1681 | + |
1682 | +def _commit_import(repo, spi, tree_hash, namespace, |
1683 | + publish_parent_commit, |
1684 | + changelog_parent_commit, upload_parent_commit, |
1685 | + unapplied_parent_commit |
1686 | +): |
1687 | + """Commit a tree object into the repository with the specified |
1688 | + parents |
1689 | + |
1690 | + The importer algorithm works by 'staging' the import of a |
1691 | + publication as a tree, then using that tree to determine the |
1692 | + appropriate parents from the publishing history and |
1693 | + debian/changelog. |
1694 | + |
1695 | + Arguments: |
1696 | + spi - A SourcePackageInformation instance for this publish |
1697 | + tree_hash - SHA1 from git-dsc-commit --tree-only of this publish |
1698 | + publish_parent_commit = SHA1 of publishing parent |
1699 | + changelog_parent_commit = SHA1 of changelog parent |
1700 | + upload_parent_commit = SHA1 of upload parent |
1701 | + unapplied_parent_commit = SHA1 of unapplied-patches import parent |
1702 | + """ |
1703 | + tag = None |
1704 | + |
1705 | + if unapplied_parent_commit: |
1706 | + import_type = 'patches-applied' |
1707 | + target_head_name = spi.applied_head_name(namespace) |
1708 | + if repo.get_applied_tag(spi.version, namespace) is None: |
1709 | + # Not imported before |
1710 | + tag = applied_tag(spi.version, namespace) |
1711 | + else: |
1712 | + import_type = 'patches-unapplied' |
1713 | + target_head_name = spi.head_name(namespace) |
1714 | + if repo.get_import_tag(spi.version, namespace) is None: |
1715 | + # Not imported before |
1716 | + tag = import_tag(spi.version, namespace) |
1717 | + |
1718 | + # Do not show importer/ namespace to user |
1719 | + _, _, pretty_head_name = target_head_name.partition('%s/' % namespace) |
1720 | + |
1721 | + changelog_entry = get_changelog_for_commit( |
1722 | + repo, |
1723 | + tree_hash, |
1724 | + publish_parent_commit, |
1725 | + changelog_parent_commit |
1726 | + ) |
1727 | + msg = (b'Import %s version %b to %b\n\nImported using git-ubuntu import.' % |
1728 | + (import_type.encode(), spi.version.encode(), pretty_head_name.encode()) |
1729 | + ) |
1730 | + |
1731 | + parents = [] |
1732 | + |
1733 | + if publish_parent_commit is None and \ |
1734 | + changelog_parent_commit is None and \ |
1735 | + upload_parent_commit is None and \ |
1736 | + unapplied_parent_commit is None and \ |
1737 | + target_head_name in repo.local_branch_names: |
1738 | + # No parents and not creating a new branch |
1739 | + tag = orphan_tag(spi.version, namespace) |
1740 | + else: |
1741 | + msg += b'\n' |
1742 | + |
1743 | + if publish_parent_commit is not None: |
1744 | + parents.append(publish_parent_commit) |
1745 | + msg += b'\nPublish parent: %b' % publish_parent_commit.encode() |
1746 | + if changelog_parent_commit is not None: |
1747 | + parents.append(changelog_parent_commit) |
1748 | + msg += b'\nChangelog parent: %b' % changelog_parent_commit.encode() |
1749 | + if upload_parent_commit is not None: |
1750 | + parents.append(upload_parent_commit) |
1751 | + msg += b'\nUpload parent: %b' % upload_parent_commit.encode() |
1752 | + if unapplied_parent_commit is not None: |
1753 | + parents.append(unapplied_parent_commit) |
1754 | + msg += b'\nUnapplied parent: %b' % unapplied_parent_commit.encode() |
1755 | + |
1756 | + msg += b'%b' % changelog_entry |
1757 | + |
1758 | + commit_hash = repo.commit_tree_hash( |
1759 | + tree_hash, |
1760 | + parents, |
1761 | + msg, |
1762 | + environment_spi=spi |
1763 | + ) |
1764 | + |
1765 | + repo.update_head_to_commit(target_head_name, commit_hash) |
1766 | + |
1767 | + logging.debug('Committed %s import of %s as %s in %s', |
1768 | + import_type, spi.version, commit_hash, pretty_head_name |
1769 | + ) |
1770 | |
1771 | - import_tree_hash = dsc_to_tree_hash(self.local_repo, dsc_pathname) |
1772 | - self.local_repo.clean_repository_state() |
1773 | + if tag is not None: |
1774 | + # should be annotated to use create_tag API |
1775 | + logging.debug('Creating tag %s pointing to %s', tag, commit_hash) |
1776 | + repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION, |
1777 | + tag, commit_hash |
1778 | + ] |
1779 | + ) |
1780 | |
1781 | - return import_tree_hash |
1782 | +def commit_unapplied_patches_import( |
1783 | + repo, |
1784 | + spi, |
1785 | + import_tree_hash, |
1786 | + namespace, |
1787 | + publish_parent_commit, |
1788 | + changelog_parent_commit, |
1789 | + upload_parent_commit, |
1790 | +): |
1791 | + _commit_import( |
1792 | + repo, |
1793 | + spi, |
1794 | + import_tree_hash, |
1795 | + namespace, |
1796 | + publish_parent_commit, |
1797 | + changelog_parent_commit, |
1798 | + upload_parent_commit, |
1799 | + # unapplied trees do not have a distinct unapplied parent |
1800 | + None, |
1801 | + ) |
1802 | + |
1803 | +def commit_applied_patches_import( |
1804 | + repo, |
1805 | + spi, |
1806 | + import_tree_hash, |
1807 | + namespace, |
1808 | + publish_parent_commit, |
1809 | + changelog_parent_commit, |
1810 | + unapplied_parent_commit |
1811 | +): |
1812 | + _commit_import( |
1813 | + repo, |
1814 | + spi, |
1815 | + import_tree_hash, |
1816 | + namespace, |
1817 | + publish_parent_commit, |
1818 | + changelog_parent_commit, |
1819 | + # uploads will be unapplied trees currently, so applied trees |
1820 | + # can not have them as direct parents |
1821 | + None, |
1822 | + unapplied_parent_commit, |
1823 | + ) |
1824 | + |
1825 | +def import_dsc(repo, dsc, namespace, version, dist): |
1826 | + """Imports the dsc file to importer/{debian,ubuntu}/dsc |
1827 | + |
1828 | + Arguments: |
1829 | + spi - A GitUbuntuSourcePackageInformation instance |
1830 | + """ |
1831 | + |
1832 | + # Add DSC file to the appropriate branch |
1833 | + runq(['git', 'checkout', '%s/importer/%s/dsc' % (namespace, dist)], env=repo.env) |
1834 | + # It is possible the same-named DSC is published multiple times |
1835 | + # with different contents, e.g. on epoch bumps |
1836 | + local_dir = os.path.join(repo.local_dir, version) |
1837 | + os.makedirs(local_dir, exist_ok=True) |
1838 | + shutil.copy(dsc.dsc_path, local_dir) |
1839 | + repo.git_run(['add', repo.local_dir]) |
1840 | + try: |
1841 | + # Anything to commit? (this will be 0 if, for instance, the |
1842 | + # same dsc is being imported again) |
1843 | + runq(['git', 'diff', '--exit-code', 'HEAD'], env=repo.env) |
1844 | + except: |
1845 | + repo.git_run(['commit', '-m', 'DSC file for %s' % version]) |
1846 | + |
1847 | + repo.clean_repository_state() |
1848 | + |
1849 | +def orig_imported(repo, orig_tarball_paths, namespace, dist): |
1850 | + """Determines if a list of paths to tarballs have already been imported to @dist |
1851 | + |
1852 | + Assumes that the pristine-tar branch exists |
1853 | + """ |
1854 | + try: |
1855 | + return repo.verify_pristine_tar( |
1856 | + orig_tarball_paths, |
1857 | + dist, |
1858 | + namespace, |
1859 | + ) |
1860 | + except PristineTarError as e: |
1861 | + raise GitUbuntuImportOrigError from e |
1862 | + |
1863 | +def import_orig(repo, dsc, namespace, version, dist): |
1864 | + """Imports the orig-tarball using gbp import-org --pristine-tar |
1865 | + |
1866 | + Arguments: |
1867 | + repo - a GitUbuntuRepository object |
1868 | + dsc - a GitUbuntuDsc object |
1869 | + namespace - the string namespace in which to search for tags, etc. |
1870 | + version - the string package version |
1871 | + dist - the string distribution, either ubuntu or debian, to import to |
1872 | + """ |
1873 | + try: |
1874 | + orig_tarball_path = dsc.orig_tarball_path |
1875 | + except GitUbuntuDscError as e: |
1876 | + raise GitUbuntuImportOrigError( |
1877 | + 'Unable to import orig tarball in DSC for version %s' % version |
1878 | + ) from e |
1879 | + |
1880 | + if orig_tarball_path is None: |
1881 | + # native packages only have a .tar.* |
1882 | + native_re = re.compile(r'.*\.tar\.[^.]+$') |
1883 | + for entry in dsc['Files']: |
1884 | + if native_re.match(entry['name']): |
1885 | + return |
1886 | + raise GitUbuntuImportOrigError('No orig tarball in ' |
1887 | + 'DSC for version %s.' % version |
1888 | + ) |
1889 | |
1890 | - def import_patches_applied_tree(self, dsc_pathname): |
1891 | - """Imports the patches-applied source package and writes the |
1892 | - corresponding working tree for a given srcpkg, patch by patch |
1893 | + try: |
1894 | + orig_component_paths = dsc.component_tarball_paths |
1895 | + except GitUbuntuDscError as e: |
1896 | + raise GitUbuntuImportOrigError( |
1897 | + 'Unable to import component tarballs in DSC for version ' |
1898 | + '%s' % version |
1899 | + ) from e |
1900 | + |
1901 | + orig_paths = [orig_tarball_path] + list(orig_component_paths.values()) |
1902 | + if (not orig_tarball_path or |
1903 | + orig_imported(repo, orig_paths, namespace, dist) |
1904 | + ): |
1905 | + return |
1906 | + if not all(map(os.path.exists, orig_paths)): |
1907 | + raise GitUbuntuImportOrigError('Unable to find tarball: ' |
1908 | + '%s' % [p for p in orig_paths if not os.path.exists(p)]) |
1909 | + |
1910 | + try: |
1911 | + with repo.pristine_tar_branches(dist, namespace): |
1912 | + oldcwd = os.getcwd() |
1913 | + # gbp does not support running from arbitrary git trees or |
1914 | + # working directories |
1915 | + # https://github.com/agx/git-buildpackage/pull/16 |
1916 | + os.chdir(repo.local_dir) |
1917 | + |
1918 | + ext = os.path.splitext(dsc.orig_tarball_path)[1] |
1919 | + |
1920 | + # make this non-fatal if upstream already exist as tagged? |
1921 | + cmd = ['gbp', 'import-orig', '--no-merge', |
1922 | + '--upstream-branch', 'do-not-push', |
1923 | + '--pristine-tar', '--no-interactive', |
1924 | + '--no-symlink-orig', |
1925 | + '--upstream-tag=%s/upstream/%s/%%(version)s%s' % |
1926 | + (namespace, dist, ext)] |
1927 | + cmd.extend(map(lambda x: '--component=%s' % x, |
1928 | + list(orig_component_paths.keys())) |
1929 | + ) |
1930 | + cmd.extend([orig_tarball_path,]) |
1931 | + run(cmd, env=repo.env) |
1932 | + except CalledProcessError as e: |
1933 | + raise GitUbuntuImportOrigError( |
1934 | + 'Unable to import tarballs: %s' % orig_paths |
1935 | + ) from e |
1936 | + finally: |
1937 | + os.chdir(oldcwd) |
1938 | |
1939 | - Arguments: |
1940 | - dsc_pathname - A string path to a DSC file |
1941 | - """ |
1942 | - oldcwd = os.getcwd() |
1943 | + repo.clean_repository_state() |
1944 | |
1945 | - extract_dir = tempfile.mkdtemp() |
1946 | - os.chdir(extract_dir) |
1947 | +def import_patches_unapplied_tree(repo, dsc_pathname): |
1948 | + """Imports the patches-unapplied source package and writes the |
1949 | + corresponding working tree for a given srcpkg |
1950 | |
1951 | - run(['dpkg-source', '-x', |
1952 | - '--skip-patches', |
1953 | - os.path.join(oldcwd, dsc_pathname) |
1954 | - ] |
1955 | - ) |
1956 | + Arguments: |
1957 | + dsc_pathname - A string path to a DSC file |
1958 | + """ |
1959 | |
1960 | - extracted_dir = None |
1961 | - for path in os.listdir(extract_dir): |
1962 | - if os.path.isdir(path): |
1963 | - extracted_dir = os.path.join(extract_dir, path) |
1964 | - break |
1965 | - if extracted_dir is None: |
1966 | - logging.exception('No source extracted?') |
1967 | - raise SourceExtractionException("Failed to find an extracted " |
1968 | - "directory from dpkg-source -x") |
1969 | - |
1970 | - if os.path.isdir(os.path.join(extracted_dir, '.pc')): |
1971 | - self.local_repo.git_run( |
1972 | - ['--work-tree', extracted_dir, 'add', '-f', '-A'] |
1973 | - ) |
1974 | - self.local_repo.git_run( |
1975 | - ['--work-tree', extracted_dir, 'rm', '-r', '-f', '.pc'] |
1976 | - ) |
1977 | - cp = self.local_repo.git_run( |
1978 | - ['--work-tree', extracted_dir, 'write-tree'] |
1979 | - ) |
1980 | - import_tree_hash = decode_binary(cp.stdout).strip() |
1981 | - yield ( |
1982 | - import_tree_hash, |
1983 | - None, |
1984 | - 'Remove .pc directory from source package', |
1985 | - ) |
1986 | + import_tree_hash = dsc_to_tree_hash(repo, dsc_pathname) |
1987 | + repo.clean_repository_state() |
1988 | |
1989 | - try: |
1990 | - try: |
1991 | - cp = run(['dpkg-source', '--print-format', extracted_dir]) |
1992 | - fmt = decode_binary(cp.stdout).strip() |
1993 | - if '3.0 (quilt)' not in fmt: |
1994 | - raise StopIteration() |
1995 | - except CalledProcessError as e: |
1996 | - try: |
1997 | - with open('debian/source/format', 'r') as f: |
1998 | - for line in f: |
1999 | - if re.match(r'3.0 (.*)', line): |
2000 | - break |
2001 | - else: |
2002 | - raise StopIteration() |
2003 | - # `man dpkg-source` indicates no d/s/format implies 1.0 |
2004 | - except OSError: |
2005 | - raise StopIteration() |
2006 | - |
2007 | - while True: |
2008 | - try: |
2009 | - os.chdir(extracted_dir) |
2010 | - run(['quilt', 'push'], rcs=[2]) |
2011 | - cp = run(['quilt', 'top']) |
2012 | - patch_name = decode_binary(cp.stdout).strip() |
2013 | - cp = run(['quilt', 'header']) |
2014 | - header = decode_binary(cp.stdout).strip() |
2015 | - patch_desc = None |
2016 | - for regex in (r'Subject:\s*(.*?)$', |
2017 | - r'Description:\s*(.*?)$' |
2018 | - ): |
2019 | - try: |
2020 | - m = re.search(regex, header, re.MULTILINE) |
2021 | - patch_desc = m.group(1) |
2022 | - except AttributeError: |
2023 | - pass |
2024 | - if patch_desc is None: |
2025 | - logging.debug('Unable to find Subject or Description in %s' % patch_name) |
2026 | - self.local_repo.git_run(['--work-tree', extracted_dir, 'add', '-f', '-A']) |
2027 | - self.local_repo.git_run(['--work-tree', extracted_dir, 'reset', 'HEAD', '--', '.pc']) |
2028 | - cp = self.local_repo.git_run(['--work-tree', extracted_dir, 'write-tree']) |
2029 | - import_tree_hash = decode_binary(cp.stdout).strip() |
2030 | - |
2031 | - yield (import_tree_hash, patch_name, patch_desc) |
2032 | - except CalledProcessError as e: |
2033 | - # quilt returns 2 when done pushing |
2034 | - if e.returncode != 2: |
2035 | - raise |
2036 | - raise StopIteration() |
2037 | - finally: |
2038 | - os.chdir(oldcwd) |
2039 | - shutil.rmtree(extract_dir) |
2040 | - self.local_repo.clean_repository_state() |
2041 | + return import_tree_hash |
2042 | |
2043 | - def parse_parentfile(self, parentfile): |
2044 | - """Extract parent overrides from a file |
2045 | +def import_patches_applied_tree(repo, dsc_pathname): |
2046 | + """Imports the patches-applied source package and writes the |
2047 | + corresponding working tree for a given srcpkg, patch by patch |
2048 | |
2049 | - The parent overrides file specifies parent-child relationship(s), |
2050 | - where the importer may not be able to determine them automatically. |
2051 | - The format of the file is: |
2052 | - <pkgname> <child version> <publish parent version> <changelog parent version> |
2053 | - with one override per line. |
2054 | + Arguments: |
2055 | + dsc_pathname - A string path to a DSC file |
2056 | + """ |
2057 | + oldcwd = os.getcwd() |
2058 | |
2059 | - <child version> is the published version to which this override |
2060 | - applies. |
2061 | + extract_dir = tempfile.mkdtemp() |
2062 | + os.chdir(extract_dir) |
2063 | |
2064 | - The <publish parent version> is the prior version published in the |
2065 | - same series/pocket, or the immediately preceding series/pocket, if |
2066 | - this is the first publish in a series/pocket. |
2067 | + run(['dpkg-source', '-x', |
2068 | + '--skip-patches', |
2069 | + os.path.join(oldcwd, dsc_pathname) |
2070 | + ] |
2071 | + ) |
2072 | |
2073 | - The <changelog parent version> is the prior version in the |
2074 | - child publish's debian/changelog. |
2075 | + extracted_dir = None |
2076 | + for path in os.listdir(extract_dir): |
2077 | + if os.path.isdir(path): |
2078 | + extracted_dir = os.path.join(extract_dir, path) |
2079 | + break |
2080 | + if extracted_dir is None: |
2081 | + logging.exception('No source extracted?') |
2082 | + raise SourceExtractionException("Failed to find an extracted " |
2083 | + "directory from dpkg-source -x") |
2084 | |
2085 | - Keyword Arguments: |
2086 | - parentfile -- Path to parent overrides file |
2087 | - """ |
2088 | - if self.parent_overrides: |
2089 | - return |
2090 | - parent_overrides = dict() |
2091 | + if os.path.isdir(os.path.join(extracted_dir, '.pc')): |
2092 | + repo.git_run( |
2093 | + ['--work-tree', extracted_dir, 'add', '-f', '-A'] |
2094 | + ) |
2095 | + repo.git_run( |
2096 | + ['--work-tree', extracted_dir, 'rm', '-r', '-f', '.pc'] |
2097 | + ) |
2098 | + cp = repo.git_run( |
2099 | + ['--work-tree', extracted_dir, 'write-tree'] |
2100 | + ) |
2101 | + import_tree_hash = decode_binary(cp.stdout).strip() |
2102 | + yield ( |
2103 | + import_tree_hash, |
2104 | + None, |
2105 | + 'Remove .pc directory from source package', |
2106 | + ) |
2107 | + |
2108 | + try: |
2109 | try: |
2110 | - with open(parentfile) as f: |
2111 | - for line in f: |
2112 | - if line.startswith('#'): |
2113 | - continue |
2114 | - m = re.match( |
2115 | - r'(?P<pkgname>\S*)\s*(?P<childversion>\S*)\s*(?P<parent1version>\S*)\s*(?P<parent2version>\S*)', |
2116 | - line |
2117 | - ) |
2118 | - if m is None: |
2119 | - continue |
2120 | - if m.group('pkgname') != self.pkgname: |
2121 | - continue |
2122 | - parent_overrides[m.group('childversion')] = { |
2123 | - 'publish_parent':m.group('parent1version'), |
2124 | - 'changelog_parent':m.group('parent2version') |
2125 | - } |
2126 | - except FileNotFoundError: |
2127 | - pass |
2128 | - self.parent_overrides = parent_overrides |
2129 | + cp = run(['dpkg-source', '--print-format', extracted_dir]) |
2130 | + fmt = decode_binary(cp.stdout).strip() |
2131 | + if '3.0 (quilt)' not in fmt: |
2132 | + raise StopIteration() |
2133 | + except CalledProcessError as e: |
2134 | + try: |
2135 | + with open('debian/source/format', 'r') as f: |
2136 | + for line in f: |
2137 | + if re.match(r'3.0 (.*)', line): |
2138 | + break |
2139 | + else: |
2140 | + raise StopIteration() |
2141 | + # `man dpkg-source` indicates no d/s/format implies 1.0 |
2142 | + except OSError: |
2143 | + raise StopIteration() |
2144 | |
2145 | - def override_parents(self, spi): |
2146 | - unapplied_publish_parent_commit = None |
2147 | - unapplied_changelog_parent_commit = None |
2148 | - applied_publish_parent_commit = None |
2149 | - applied_changelog_parent_commit = None |
2150 | + while True: |
2151 | + try: |
2152 | + os.chdir(extracted_dir) |
2153 | + run(['quilt', 'push'], rcs=[2]) |
2154 | + cp = run(['quilt', 'top']) |
2155 | + patch_name = decode_binary(cp.stdout).strip() |
2156 | + cp = run(['quilt', 'header']) |
2157 | + header = decode_binary(cp.stdout).strip() |
2158 | + patch_desc = None |
2159 | + for regex in (r'Subject:\s*(.*?)$', |
2160 | + r'Description:\s*(.*?)$' |
2161 | + ): |
2162 | + try: |
2163 | + m = re.search(regex, header, re.MULTILINE) |
2164 | + patch_desc = m.group(1) |
2165 | + except AttributeError: |
2166 | + pass |
2167 | + if patch_desc is None: |
2168 | + logging.debug('Unable to find Subject or Description ' |
2169 | + 'in %s' % patch_name |
2170 | + ) |
2171 | + repo.git_run([ |
2172 | + '--work-tree', |
2173 | + extracted_dir, |
2174 | + 'add', |
2175 | + '-f', |
2176 | + '-A', |
2177 | + ]) |
2178 | + repo.git_run([ |
2179 | + '--work-tree', |
2180 | + extracted_dir, |
2181 | + 'reset', |
2182 | + 'HEAD', |
2183 | + '--', |
2184 | + '.pc', |
2185 | + ]) |
2186 | + cp = repo.git_run(['--work-tree', extracted_dir, 'write-tree']) |
2187 | + import_tree_hash = decode_binary(cp.stdout).strip() |
2188 | + |
2189 | + yield (import_tree_hash, patch_name, patch_desc) |
2190 | + except CalledProcessError as e: |
2191 | + # quilt returns 2 when done pushing |
2192 | + if e.returncode != 2: |
2193 | + raise |
2194 | + raise StopIteration() |
2195 | + finally: |
2196 | + os.chdir(oldcwd) |
2197 | + shutil.rmtree(extract_dir) |
2198 | + repo.clean_repository_state() |
2199 | + |
2200 | +_PARENT_OVERRIDES = None |
2201 | +def parse_parentfile(parentfile, pkgname): |
2202 | + """Extract parent overrides from a file |
2203 | + |
2204 | + The parent overrides file specifies parent-child relationship(s), |
2205 | + where the importer may not be able to determine them automatically. |
2206 | + The format of the file is: |
2207 | + <pkgname> <child version> <publish parent version> <changelog parent version> |
2208 | + with one override per line. |
2209 | + |
2210 | + <child version> is the published version to which this override |
2211 | + applies. |
2212 | + |
2213 | + The <publish parent version> is the prior version published in the |
2214 | + same series/pocket, or the immediately preceding series/pocket, if |
2215 | + this is the first publish in a series/pocket. |
2216 | + |
2217 | + The <changelog parent version> is the prior version in the |
2218 | + child publish's debian/changelog. |
2219 | + |
2220 | + Keyword Arguments: |
2221 | + parentfile -- Path to parent overrides file |
2222 | + """ |
2223 | + global _PARENT_OVERRIDES |
2224 | + if _PARENT_OVERRIDES: |
2225 | + return |
2226 | + _PARENT_OVERRIDES = dict() |
2227 | + try: |
2228 | + with open(parentfile) as f: |
2229 | + for line in f: |
2230 | + if line.startswith('#'): |
2231 | + continue |
2232 | + m = re.match( |
2233 | + r'(?P<pkgname>\S*)\s*(?P<childversion>\S*)\s*(?P<parent1version>\S*)\s*(?P<parent2version>\S*)', |
2234 | + line |
2235 | + ) |
2236 | + if m is None: |
2237 | + continue |
2238 | + if m.group('pkgname') != pkgname: |
2239 | + continue |
2240 | + _PARENT_OVERRIDES[m.group('childversion')] = { |
2241 | + 'publish_parent':m.group('parent1version'), |
2242 | + 'changelog_parent':m.group('parent2version') |
2243 | + } |
2244 | + except FileNotFoundError: |
2245 | + pass |
2246 | + |
2247 | +def override_parents(repo, spi, namespace): |
2248 | + unapplied_publish_parent_commit = None |
2249 | + unapplied_changelog_parent_commit = None |
2250 | + applied_publish_parent_commit = None |
2251 | + applied_changelog_parent_commit = None |
2252 | + |
2253 | + unapplied_publish_parent_tag = repo.get_import_tag( |
2254 | + _PARENT_OVERRIDES[spi.version]['publish_parent'], |
2255 | + namespace |
2256 | + ) |
2257 | + applied_publish_parent_tag = repo.get_applied_tag( |
2258 | + _PARENT_OVERRIDES[spi.version]['publish_parent'], |
2259 | + namespace |
2260 | + ) |
2261 | + if unapplied_publish_parent_tag is not None: |
2262 | + # sanity check that version from d/changelog of the |
2263 | + # tagged commit matches ours |
2264 | + parent_publish_version, _ = \ |
2265 | + repo.get_changelog_versions_from_treeish( |
2266 | + str(unapplied_publish_parent_tag.peel().id), |
2267 | + ) |
2268 | + if parent_publish_version != _PARENT_OVERRIDES[spi.version]['publish_parent']: |
2269 | + logging.error('Found a tag corresponding to publish parent ' |
2270 | + 'override version %s, but d/changelog version (%s) ' |
2271 | + 'differs. Will orphan commit.', |
2272 | + _PARENT_OVERRIDES[spi.version]['publish_parent'], |
2273 | + parent_publish_version |
2274 | + ) |
2275 | + else: |
2276 | + unapplied_publish_parent_commit = str(unapplied_publish_parent_tag.peel().id) |
2277 | + if applied_publish_parent_tag is not None: |
2278 | + applied_publish_parent_commit = str(applied_publish_parent_tag.peel().id) |
2279 | + logging.debug('Overriding publish parent (tag) to %s', |
2280 | + repo.tag_to_pretty_name(unapplied_publish_parent_tag) |
2281 | + ) |
2282 | + else: |
2283 | + if _PARENT_OVERRIDES[spi.version]['publish_parent'] == '-': |
2284 | + logging.debug('Not setting publish parent as ' |
2285 | + 'specified in override file.' |
2286 | + ) |
2287 | + else: |
2288 | + logging.error('Specified publish parent override ' |
2289 | + '(%s) for version (%s) not found in tags. ' |
2290 | + 'Unable to proceed.' % ( |
2291 | + _PARENT_OVERRIDES[spi.version]['publish_parent'], |
2292 | + spi.version |
2293 | + ) |
2294 | + ) |
2295 | + sys.exit(1) |
2296 | |
2297 | - unapplied_publish_parent_tag = self.local_repo.get_import_tag( |
2298 | - self.parent_overrides[spi.version]['publish_parent'], |
2299 | - self.namespace) |
2300 | - applied_publish_parent_tag = self.local_repo.get_applied_tag( |
2301 | - self.parent_overrides[spi.version]['publish_parent'], |
2302 | - self.namespace) |
2303 | - if unapplied_publish_parent_tag is not None: |
2304 | - # sanity check that version from d/changelog of the |
2305 | - # tagged commit matches ours |
2306 | - parent_publish_version, _ = \ |
2307 | - self.local_repo.get_changelog_versions_from_treeish( |
2308 | - str(unapplied_publish_parent_tag.peel().id), |
2309 | - ) |
2310 | - if parent_publish_version != self.parent_overrides[spi.version]['publish_parent']: |
2311 | - logging.error('Found a tag corresponding to ' |
2312 | - 'publish parent override ' |
2313 | - 'version %s, but d/changelog ' |
2314 | - 'version (%s) differs. Will ' |
2315 | - 'orphan commit.' % ( |
2316 | - self.parent_overrides[spi.version]['publish_parent'], |
2317 | - parent_publish_version |
2318 | - ) |
2319 | - ) |
2320 | - else: |
2321 | - unapplied_publish_parent_commit = str(unapplied_publish_parent_tag.peel().id) |
2322 | - if applied_publish_parent_tag is not None: |
2323 | - applied_publish_parent_commit = str(applied_publish_parent_tag.peel().id) |
2324 | - logging.debug('Overriding publish parent (tag) to %s', |
2325 | - self.local_repo.tag_to_pretty_name(unapplied_publish_parent_tag) |
2326 | - ) |
2327 | + unapplied_changelog_parent_tag = repo.get_import_tag( |
2328 | + _PARENT_OVERRIDES[spi.version]['changelog_parent'], |
2329 | + namespace |
2330 | + ) |
2331 | + applied_changelog_parent_tag = repo.get_applied_tag( |
2332 | + _PARENT_OVERRIDES[spi.version]['changelog_parent'], |
2333 | + namespace |
2334 | + ) |
2335 | + if unapplied_changelog_parent_tag is not None: |
2336 | + # sanity check that version from d/changelog of the |
2337 | + # tagged commit matches ours |
2338 | + parent_changelog_version, _ = repo.get_changelog_versions_from_treeish( |
2339 | + str(unapplied_changelog_parent_tag.peel().id), |
2340 | + ) |
2341 | + if parent_changelog_version != _PARENT_OVERRIDES[spi.version]['changelog_parent']: |
2342 | + logging.error('Found a tag corresponding to ' |
2343 | + 'changelog parent override ' |
2344 | + 'version %s, but d/changelog ' |
2345 | + 'version (%s) differs. Will ' |
2346 | + 'orphan commit.' % ( |
2347 | + _PARENT_OVERRIDES[spi.version]['changelog_parent'], |
2348 | + parent_changelog_version |
2349 | + ) |
2350 | + ) |
2351 | else: |
2352 | - if self.parent_overrides[spi.version]['publish_parent'] == '-': |
2353 | - logging.debug('Not setting publish parent as ' |
2354 | - 'specified in override file.' |
2355 | - ) |
2356 | - else: |
2357 | - logging.error('Specified publish parent override ' |
2358 | - '(%s) for version (%s) not found in tags. ' |
2359 | - 'Unable to proceed.' % ( |
2360 | - self.parent_overrides[spi.version]['publish_parent'], |
2361 | - spi.version |
2362 | - ) |
2363 | - ) |
2364 | - sys.exit(1) |
2365 | - |
2366 | - unapplied_changelog_parent_tag = self.local_repo.get_import_tag( |
2367 | - self.parent_overrides[spi.version]['changelog_parent'], |
2368 | - self.namespace) |
2369 | - applied_changelog_parent_tag = self.local_repo.get_applied_tag( |
2370 | - self.parent_overrides[spi.version]['changelog_parent'], |
2371 | - self.namespace) |
2372 | - if unapplied_changelog_parent_tag is not None: |
2373 | - # sanity check that version from d/changelog of the |
2374 | - # tagged commit matches ours |
2375 | - parent_changelog_version, _ = \ |
2376 | - self.local_repo.get_changelog_versions_from_treeish( |
2377 | - str(unapplied_changelog_parent_tag.peel().id), |
2378 | - ) |
2379 | - if parent_changelog_version != self.parent_overrides[spi.version]['changelog_parent']: |
2380 | - logging.error('Found a tag corresponding to ' |
2381 | - 'changelog parent override ' |
2382 | - 'version %s, but d/changelog ' |
2383 | - 'version (%s) differs. Will ' |
2384 | - 'orphan commit.' % ( |
2385 | - self.parent_overrides[spi.version]['changelog_parent'], |
2386 | - parent_changelog_version |
2387 | - ) |
2388 | - ) |
2389 | - else: |
2390 | - unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id) |
2391 | - if applied_changelog_parent_tag is not None: |
2392 | - applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id) |
2393 | - logging.debug('Overriding changelog parent (tag) to %s', |
2394 | - self.local_repo.tag_to_pretty_name(unapplied_changelog_parent_tag) |
2395 | - ) |
2396 | + unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id) |
2397 | + if applied_changelog_parent_tag is not None: |
2398 | + applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id) |
2399 | + logging.debug('Overriding changelog parent (tag) to %s', |
2400 | + repo.tag_to_pretty_name(unapplied_changelog_parent_tag) |
2401 | + ) |
2402 | + else: |
2403 | + if _PARENT_OVERRIDES[spi.version]['changelog_parent'] == '-': |
2404 | + logging.debug('Not setting changelog parent as specified ' |
2405 | + 'in override file.' |
2406 | + ) |
2407 | else: |
2408 | - if self.parent_overrides[spi.version]['changelog_parent'] == '-': |
2409 | - logging.debug('Not setting changelog parent as ' |
2410 | - 'specified in override file.' |
2411 | - ) |
2412 | - else: |
2413 | - logging.error('Specified changelog parent override ' |
2414 | - '(%s) for version (%s) not found in tags. ' |
2415 | - 'Unable to proceed.' % ( |
2416 | - self.parent_overrides[spi.version]['changelog_parent'], |
2417 | - spi.version |
2418 | - ) |
2419 | - ) |
2420 | - sys.exit(1) |
2421 | - return (unapplied_publish_parent_commit, |
2422 | - unapplied_changelog_parent_commit, |
2423 | - applied_publish_parent_commit, |
2424 | - applied_changelog_parent_commit) |
2425 | - |
2426 | - def _update_devel_branches(self, namespace, spi=None): |
2427 | - if spi: |
2428 | - # we do not maintain -devel pointers for debian |
2429 | - if str(spi.distribution.name.lower()) != "ubuntu": |
2430 | + logging.error('Specified changelog parent override (%s) ' |
2431 | + 'for version (%s) not found in tags. Unable to proceed.', |
2432 | + _PARENT_OVERRIDES[spi.version]['changelog_parent'], |
2433 | + spi.version |
2434 | + ) |
2435 | + sys.exit(1) |
2436 | + return ( |
2437 | + unapplied_publish_parent_commit, |
2438 | + unapplied_changelog_parent_commit, |
2439 | + applied_publish_parent_commit, |
2440 | + applied_changelog_parent_commit, |
2441 | + ) |
2442 | + |
2443 | +def _update_devel_branches( |
2444 | + repo, |
2445 | + namespace, |
2446 | + pkgname, |
2447 | + ubuntu_sinfo, |
2448 | + spi=None, |
2449 | +): |
2450 | + if spi: |
2451 | + # we do not maintain -devel pointers for debian |
2452 | + if str(spi.distribution.name.lower()) != "ubuntu": |
2453 | + return |
2454 | + for devel_head in ( |
2455 | + '%s/ubuntu/%s-devel' % (namespace, spi.series.name.lower()), |
2456 | + '%s/ubuntu/devel' % namespace |
2457 | + ): |
2458 | + try: |
2459 | + devel_head_hash = repo.head_to_commit(devel_head) |
2460 | + devel_head_version, _ = repo.get_changelog_versions_from_treeish(devel_head_hash) |
2461 | + except AttributeError: |
2462 | + devel_head_version = None |
2463 | + if version_compare(spi.version, devel_head_version) <= 0: |
2464 | return |
2465 | - for devel_head in ( |
2466 | - '%s/ubuntu/%s-devel' % (namespace, spi.series.name.lower()), |
2467 | - '%s/ubuntu/devel' % namespace |
2468 | - ): |
2469 | - try: |
2470 | - devel_head_hash = self.local_repo.head_to_commit(devel_head) |
2471 | - devel_head_version, _ = self.local_repo.get_changelog_versions_from_treeish(devel_head_hash) |
2472 | - except AttributeError: |
2473 | - devel_head_version = None |
2474 | - if version_compare(spi.version, devel_head_version) <= 0: |
2475 | - return |
2476 | - logging.debug("Updating %s to %s" % (devel_head, |
2477 | - spi.head_name(namespace))) |
2478 | - self.local_repo.merge_commit_to_devel_head( |
2479 | - namespace, devel_head, |
2480 | - self.local_repo.head_to_commit(spi.head_name(namespace)), |
2481 | - environment_spi=spi) |
2482 | + logging.debug("Updating %s to %s" % (devel_head, |
2483 | + spi.head_name(namespace))) |
2484 | + repo.merge_commit_to_devel_head( |
2485 | + namespace, |
2486 | + devel_head, |
2487 | + repo.head_to_commit(spi.head_name(namespace)), |
2488 | + environment_spi=spi, |
2489 | + ) |
2490 | + else: |
2491 | + ubuntu_head_versions = repo.get_heads_and_versions( |
2492 | + 'ubuntu', |
2493 | + namespace, |
2494 | + ) |
2495 | + devel_hash = None |
2496 | + devel_name = None |
2497 | + for rel in ubuntu_sinfo.active_series_name_list: |
2498 | + series_devel_hash = None |
2499 | + series_devel_name = None |
2500 | + series_devel_version = None |
2501 | + # Find highest version publish in these pockets |
2502 | + for suff in ("-proposed", "-updates", "-security", ""): |
2503 | + name = "%s/ubuntu/%s%s" % (namespace, rel, suff) |
2504 | + if name in ubuntu_head_versions and version_compare( |
2505 | + ubuntu_head_versions[name]['version'], |
2506 | + series_devel_version |
2507 | + ) > 0: |
2508 | + series_devel_name = name |
2509 | + series_devel_version = ubuntu_head_versions[name]['version'] |
2510 | + series_devel_hash = str(ubuntu_head_versions[name]['head'].peel().id) |
2511 | + if series_devel_hash: |
2512 | + if not devel_hash: |
2513 | + devel_hash = series_devel_hash |
2514 | + devel_name = series_devel_name |
2515 | + logging.debug("Updating %s/ubuntu/%s-devel branch to '%s' (%s)", |
2516 | + namespace, rel, series_devel_name, series_devel_hash) |
2517 | + repo.merge_commit_to_devel_head(namespace, |
2518 | + "%s/ubuntu/%s-devel" % (namespace, rel), series_devel_hash) |
2519 | + if devel_hash is None: |
2520 | + logging.warn("Package '%s' does not appear to have been published " |
2521 | + "in Ubuntu. No %s/ubuntu/devel branch created.", |
2522 | + pkgname, namespace) |
2523 | else: |
2524 | - ubuntu_head_versions = self.local_repo.get_heads_and_versions( |
2525 | - 'ubuntu', namespace |
2526 | + logging.debug("Setting %s/ubuntu/devel branch to '%s' (%s)", |
2527 | + namespace, devel_name, devel_hash) |
2528 | + repo.merge_commit_to_devel_head( |
2529 | + namespace, "%s/ubuntu/devel" % namespace, devel_hash |
2530 | ) |
2531 | - devel_hash = None |
2532 | - devel_name = None |
2533 | - for rel in self.ubuntu_sinfo.active_series_name_list: |
2534 | - series_devel_hash = None |
2535 | - series_devel_name = None |
2536 | - series_devel_version = None |
2537 | - # Find highest version publish in these pockets |
2538 | - for suff in ("-proposed", "-updates", "-security", ""): |
2539 | - name = "%s/ubuntu/%s%s" % (namespace, rel, suff) |
2540 | - if name in ubuntu_head_versions and version_compare( |
2541 | - ubuntu_head_versions[name]['version'], |
2542 | - series_devel_version |
2543 | - ) > 0: |
2544 | - series_devel_name = name |
2545 | - series_devel_version = ubuntu_head_versions[name]['version'] |
2546 | - series_devel_hash = str(ubuntu_head_versions[name]['head'].peel().id) |
2547 | - if series_devel_hash: |
2548 | - if not devel_hash: |
2549 | - devel_hash = series_devel_hash |
2550 | - devel_name = series_devel_name |
2551 | - logging.debug("Updating %s/ubuntu/%s-devel branch to '%s' (%s)", |
2552 | - namespace, rel, series_devel_name, series_devel_hash) |
2553 | - self.local_repo.merge_commit_to_devel_head(namespace, |
2554 | - "%s/ubuntu/%s-devel" % (namespace, rel), series_devel_hash) |
2555 | - if devel_hash is None: |
2556 | - logging.warn("Package '%s' does not appear to have been published " |
2557 | - "in Ubuntu. No %s/ubuntu/devel branch created.", |
2558 | - self.pkgname, namespace) |
2559 | - else: |
2560 | - logging.debug("Setting %s/ubuntu/devel branch to '%s' (%s)", |
2561 | - namespace, devel_name, devel_hash) |
2562 | - self.local_repo.merge_commit_to_devel_head( |
2563 | - namespace, "%s/ubuntu/devel" % namespace, devel_hash |
2564 | - ) |
2565 | - |
2566 | - def update_devel_branches(self, spi=None): |
2567 | - self._update_devel_branches(self.namespace, spi) |
2568 | - |
2569 | - def update_applied_devel_branches(self, spi=None): |
2570 | - self._update_devel_branches('%s/applied' % self.namespace, spi) |
2571 | - |
2572 | - # imports a package based upon source package information |
2573 | - def import_unapplied_spi(self, spi): |
2574 | - """Imports a source package from Launchpad into the git |
2575 | - repository |
2576 | - |
2577 | - Arguments: |
2578 | - spi - a SourcePackageInformation instance |
2579 | - """ |
2580 | - pretty_head_name = spi.pretty_head_name |
2581 | |
2582 | - logging.info('Importing patches-unapplied %s to %s' % |
2583 | - (spi.version, pretty_head_name)) |
2584 | - unapplied_tip_head = self.local_repo.get_head_by_name(spi.head_name(self.namespace)) |
2585 | - |
2586 | - spi.pull() |
2587 | - |
2588 | - self.local_repo.clean_repository_state() |
2589 | - |
2590 | - self.import_dsc( |
2591 | +def update_devel_branches( |
2592 | + repo, |
2593 | + namespace, |
2594 | + pkgname, |
2595 | + ubuntu_sinfo, |
2596 | + spi=None, |
2597 | +): |
2598 | + _update_devel_branches( |
2599 | + repo=repo, |
2600 | + namespace=namespace, |
2601 | + pkgname=pkgname, |
2602 | + ubuntu_sinfo=ubuntu_sinfo, |
2603 | + spi=spi, |
2604 | + ) |
2605 | + |
2606 | +def update_applied_devel_branches( |
2607 | + repo, |
2608 | + namespace, |
2609 | + pkgname, |
2610 | + ubuntu_sinfo, |
2611 | + spi=None, |
2612 | +): |
2613 | + _update_devel_branches( |
2614 | + repo=repo, |
2615 | + namespace='%s/applied' % namespace, |
2616 | + pkgname=pkgname, |
2617 | + ubuntu_sinfo=ubuntu_sinfo, |
2618 | + spi=spi, |
2619 | + ) |
2620 | + |
2621 | +# imports a package based upon source package information |
2622 | +def import_unapplied_spi(repo, spi, namespace, skip_orig, ubuntu_sinfo): |
2623 | + """Imports a source package from Launchpad into the git |
2624 | + repository |
2625 | + |
2626 | + Arguments: |
2627 | + spi - a SourcePackageInformation instance |
2628 | + """ |
2629 | + pretty_head_name = spi.pretty_head_name |
2630 | + |
2631 | + logging.info('Importing patches-unapplied %s to %s', |
2632 | + spi.version, pretty_head_name |
2633 | + ) |
2634 | + unapplied_tip_head = repo.get_head_by_name(spi.head_name(namespace)) |
2635 | + |
2636 | + spi.pull() |
2637 | + |
2638 | + repo.clean_repository_state() |
2639 | + |
2640 | + import_dsc( |
2641 | + repo, |
2642 | + GitUbuntuDsc(spi.dsc_pathname), |
2643 | + namespace, |
2644 | + str(spi.version), |
2645 | + spi.distribution.name.lower(), |
2646 | + ) |
2647 | + if not skip_orig: |
2648 | + import_orig( |
2649 | + repo, |
2650 | GitUbuntuDsc(spi.dsc_pathname), |
2651 | + namespace, |
2652 | str(spi.version), |
2653 | - spi.distribution.name.lower() |
2654 | + spi.distribution.name.lower(), |
2655 | ) |
2656 | - if not self.skip_orig: |
2657 | - self.import_orig(spi) |
2658 | |
2659 | - unapplied_import_tree_hash = self.import_patches_unapplied_tree(spi.dsc_pathname) |
2660 | - logging.debug('Imported patches-unapplied version %s as tree %s', |
2661 | - spi.version, unapplied_import_tree_hash |
2662 | - ) |
2663 | + unapplied_import_tree_hash = import_patches_unapplied_tree( |
2664 | + repo, |
2665 | + spi.dsc_pathname |
2666 | + ) |
2667 | + logging.debug( |
2668 | + "Imported patches-unapplied version %s as tree %s", |
2669 | + spi.version, |
2670 | + unapplied_import_tree_hash |
2671 | + ) |
2672 | + |
2673 | + import_tree_versions = repo.get_all_changelog_versions_from_treeish( |
2674 | + unapplied_import_tree_hash |
2675 | + ) |
2676 | + changelog_version = import_tree_versions[0] |
2677 | + |
2678 | + logging.debug( |
2679 | + "Found changelog version %s in newly imported tree" % |
2680 | + changelog_version |
2681 | + ) |
2682 | + |
2683 | + # sanity check on spph and d/changelog agreeing on the latest version |
2684 | + if spi.version not in _PARENT_OVERRIDES: |
2685 | + assert spi.version == changelog_version, ( |
2686 | + 'source pkg version: {} != changelog version: {}'.format( |
2687 | + spi.version, changelog_version)) |
2688 | + |
2689 | + unapplied_tip_version = None |
2690 | + # check if the version to import has already been imported to |
2691 | + # this head |
2692 | + if unapplied_tip_head is not None: |
2693 | + if repo.treeishs_identical( |
2694 | + unapplied_import_tree_hash, str(unapplied_tip_head.peel().id) |
2695 | + ): |
2696 | + logging.warn('%s is identical to %s', |
2697 | + pretty_head_name, spi.version |
2698 | + ) |
2699 | + return |
2700 | + unapplied_tip_version, _ = repo.get_changelog_versions_from_treeish( |
2701 | + str(unapplied_tip_head.peel().id), |
2702 | + ) |
2703 | |
2704 | - import_tree_versions = self.local_repo.get_all_changelog_versions_from_treeish( |
2705 | - unapplied_import_tree_hash |
2706 | - ) |
2707 | - changelog_version = import_tree_versions[0] |
2708 | + logging.debug('Tip version is %s', unapplied_tip_version) |
2709 | |
2710 | + unapplied_publish_parent_commit = None |
2711 | + unapplied_changelog_parent_commit = None |
2712 | + upload_parent_commit = None |
2713 | + unapplied_parent_commit = None |
2714 | + |
2715 | + if spi.version in _PARENT_OVERRIDES: |
2716 | logging.debug( |
2717 | - 'Found changelog version %s in newly imported tree' % |
2718 | - changelog_version |
2719 | - ) |
2720 | + '%s is specified in the parent override file.', |
2721 | + spi.version |
2722 | + ) |
2723 | |
2724 | - # sanity check on spph and d/changelog agreeing on the latest version |
2725 | - if spi.version not in self.parent_overrides: |
2726 | - assert spi.version == changelog_version, ( |
2727 | - 'source pkg version: {} != changelog version: {}'.format( |
2728 | - spi.version, changelog_version)) |
2729 | - |
2730 | - unapplied_tip_version = None |
2731 | - # check if the version to import has already been imported to |
2732 | - # this head |
2733 | - if unapplied_tip_head is not None: |
2734 | - if self.local_repo.treeishs_identical( |
2735 | - unapplied_import_tree_hash, str(unapplied_tip_head.peel().id) |
2736 | - ): |
2737 | - logging.warn('%s is identical to %s', |
2738 | - pretty_head_name, spi.version |
2739 | + ( |
2740 | + unapplied_publish_parent_commit, |
2741 | + unapplied_changelog_parent_commit, |
2742 | + _, |
2743 | + _ |
2744 | + ) = override_parents(repo, spi, namespace) |
2745 | + else: |
2746 | + # Get parent from publishing history (which is the last |
2747 | + # published version in the corresponding series) |
2748 | + try: |
2749 | + unapplied_publish_parent_commit = str(unapplied_tip_head.peel().id) |
2750 | + except AttributeError: |
2751 | + try: |
2752 | + unapplied_publish_parent_head = repo.get_head_by_name(spi.parent_head_name(namespace)) |
2753 | + unapplied_publish_parent_commit = str(unapplied_publish_parent_head.peel().id) |
2754 | + except AttributeError: |
2755 | + pass |
2756 | + finally: |
2757 | + try: |
2758 | + unapplied_publish_parent_tag = repo.nearest_import_tag(unapplied_publish_parent_commit, namespace) |
2759 | + logging.debug('Publishing parent (tag) is %s', |
2760 | + repo.tag_to_pretty_name(unapplied_publish_parent_tag) |
2761 | ) |
2762 | - return |
2763 | - unapplied_tip_version, _ = self.local_repo.get_changelog_versions_from_treeish( |
2764 | - str(unapplied_tip_head.peel().id), |
2765 | - ) |
2766 | + except AttributeError: |
2767 | + pass |
2768 | |
2769 | - logging.debug('Tip version is %s', unapplied_tip_version) |
2770 | + if version_compare(str(spi.version), unapplied_tip_version) <= 0: |
2771 | + logging.warn('Version to import (%s) is not after %s tip (%s)', |
2772 | + spi.version, pretty_head_name, unapplied_tip_version |
2773 | + ) |
2774 | + |
2775 | + # Walk changelog backwards until we find an imported version |
2776 | + for version in import_tree_versions: |
2777 | + unapplied_changelog_parent_tag = repo.get_import_tag(version, namespace) |
2778 | + if unapplied_changelog_parent_tag is not None: |
2779 | + # sanity check that version from d/changelog of the |
2780 | + # tagged parent matches ours |
2781 | + parent_changelog_version, _ = \ |
2782 | + repo.get_changelog_versions_from_treeish( |
2783 | + str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id), |
2784 | + ) |
2785 | + # if the parent was imported via a parent override |
2786 | + # itself, assume it is valid without checking its |
2787 | + # tree's debian/changelog, as it wasn't used |
2788 | + if (version not in _PARENT_OVERRIDES and |
2789 | + parent_changelog_version != version |
2790 | + ): |
2791 | + logging.error('Found a tag corresponding to ' |
2792 | + 'parent version %s, but d/changelog ' |
2793 | + 'version (%s) differs. Will ' |
2794 | + 'orphan commit.' % ( |
2795 | + version, |
2796 | + parent_changelog_version |
2797 | + ) |
2798 | + ) |
2799 | + else: |
2800 | + unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id) |
2801 | + logging.debug('Changelog parent (tag) is %s', |
2802 | + repo.tag_to_pretty_name(unapplied_changelog_parent_tag) |
2803 | + ) |
2804 | + break |
2805 | |
2806 | - unapplied_publish_parent_commit = None |
2807 | + # If the two parents are tree-identical, then favor publication |
2808 | + # history. This deals with copy-forwards between series and with |
2809 | + # syncs. |
2810 | + if repo.treeishs_identical(unapplied_publish_parent_commit, unapplied_changelog_parent_commit): |
2811 | unapplied_changelog_parent_commit = None |
2812 | - upload_parent_commit = None |
2813 | - unapplied_parent_commit = None |
2814 | |
2815 | - if spi.version in self.parent_overrides: |
2816 | - logging.debug('%s is specified in the parent override file.' % |
2817 | + # check if the version to import has already been uploaded to |
2818 | + # this head -- we leverage the above code to perform a 'dry-run' |
2819 | + # of the import tree, to obtain the (up to) two parents |
2820 | + upload_tag = repo.get_upload_tag(spi.version, namespace) |
2821 | + import_tag = repo.get_import_tag(spi.version, namespace) |
2822 | + if upload_tag and not import_tag: |
2823 | + if str(upload_tag.peel().tree.id) != unapplied_import_tree_hash: |
2824 | + logging.warn('Found upload tag %s, ' |
2825 | + 'but the corresponding tree ' |
2826 | + 'does not match the published ' |
2827 | + 'version. Will import %s as ' |
2828 | + 'normal, ignoring the upload tag.', |
2829 | + repo.tag_to_pretty_name(upload_tag), |
2830 | spi.version |
2831 | ) |
2832 | - |
2833 | - (unapplied_publish_parent_commit, |
2834 | - unapplied_changelog_parent_commit, |
2835 | - _, _) = self.override_parents(spi) |
2836 | else: |
2837 | - # Get parent from publishing history (which is the last |
2838 | - # published version in the corresponding series) |
2839 | - try: |
2840 | - unapplied_publish_parent_commit = str(unapplied_tip_head.peel().id) |
2841 | - except AttributeError: |
2842 | - try: |
2843 | - unapplied_publish_parent_head = self.local_repo.get_head_by_name(spi.parent_head_name(self.namespace)) |
2844 | - unapplied_publish_parent_commit = str(unapplied_publish_parent_head.peel().id) |
2845 | - except AttributeError: |
2846 | - pass |
2847 | - finally: |
2848 | + upload_parent_commit = str(upload_tag.peel().id) |
2849 | + if unapplied_publish_parent_commit is not None: |
2850 | try: |
2851 | - unapplied_publish_parent_tag = self.local_repo.nearest_import_tag(unapplied_publish_parent_commit, self.namespace) |
2852 | - logging.debug('Publishing parent (tag) is %s', |
2853 | - self.local_repo.tag_to_pretty_name(unapplied_publish_parent_tag) |
2854 | + repo.git_run(['merge-base', '--is-ancestor', |
2855 | + unapplied_publish_parent_commit, |
2856 | + upload_parent_commit |
2857 | + ] |
2858 | ) |
2859 | - except AttributeError: |
2860 | - pass |
2861 | - |
2862 | - if version_compare(str(spi.version), unapplied_tip_version) <= 0: |
2863 | - logging.warn('Version to import (%s) is not after %s tip (%s)', |
2864 | - spi.version, pretty_head_name, unapplied_tip_version |
2865 | - ) |
2866 | - |
2867 | - # Walk changelog backwards until we find an imported version |
2868 | - for version in import_tree_versions: |
2869 | - unapplied_changelog_parent_tag = self.local_repo.get_import_tag(version, self.namespace) |
2870 | - if unapplied_changelog_parent_tag is not None: |
2871 | - # sanity check that version from d/changelog of the |
2872 | - # tagged parent matches ours |
2873 | - parent_changelog_version, _ = \ |
2874 | - self.local_repo.get_changelog_versions_from_treeish( |
2875 | - str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id), |
2876 | - ) |
2877 | - # if the parent was imported via a parent override |
2878 | - # itself, assume it is valid without checking its |
2879 | - # tree's debian/changelog, as it wasn't used |
2880 | - if (version not in self.parent_overrides and |
2881 | - parent_changelog_version != version |
2882 | - ): |
2883 | - logging.error('Found a tag corresponding to ' |
2884 | - 'parent version %s, but d/changelog ' |
2885 | - 'version (%s) differs. Will ' |
2886 | - 'orphan commit.' % ( |
2887 | - version, |
2888 | - parent_changelog_version |
2889 | - ) |
2890 | - ) |
2891 | - else: |
2892 | - unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id) |
2893 | - logging.debug('Changelog parent (tag) is %s', |
2894 | - self.local_repo.tag_to_pretty_name(unapplied_changelog_parent_tag) |
2895 | - ) |
2896 | - break |
2897 | - |
2898 | - # If the two parents are tree-identical, then favor publication |
2899 | - # history. This deals with copy-forwards between series and with |
2900 | - # syncs. |
2901 | - if self.local_repo.treeishs_identical(unapplied_publish_parent_commit, unapplied_changelog_parent_commit): |
2902 | - unapplied_changelog_parent_commit = None |
2903 | - |
2904 | - # check if the version to import has already been uploaded to |
2905 | - # this head -- we leverage the above code to perform a 'dry-run' |
2906 | - # of the import tree, to obtain the (up to) two parents |
2907 | - upload_tag = self.local_repo.get_upload_tag(spi.version, self.namespace) |
2908 | - import_tag = self.local_repo.get_import_tag(spi.version, self.namespace) |
2909 | - if upload_tag and not import_tag: |
2910 | - if str(upload_tag.peel().tree.id) != unapplied_import_tree_hash: |
2911 | - logging.warn('Found upload tag %s, ' |
2912 | - 'but the corresponding tree ' |
2913 | - 'does not match the published ' |
2914 | - 'version. Will import %s as ' |
2915 | - 'normal, ignoring the upload tag.', |
2916 | - self.local_repo.tag_to_pretty_name(upload_tag), |
2917 | - spi.version |
2918 | - ) |
2919 | - else: |
2920 | - upload_parent_commit = str(upload_tag.peel().id) |
2921 | - if unapplied_publish_parent_commit is not None: |
2922 | - try: |
2923 | - self.local_repo.git_run(['merge-base', '--is-ancestor', |
2924 | - unapplied_publish_parent_commit, |
2925 | - upload_parent_commit |
2926 | - ] |
2927 | - ) |
2928 | - unapplied_publish_parent_commit = None |
2929 | - except CalledProcessError as e: |
2930 | - if e.returncode != 1: |
2931 | - raise |
2932 | - |
2933 | - if unapplied_changelog_parent_commit is not None: |
2934 | - try: |
2935 | - self.local_repo.git_run(['merge-base', '--is-ancestor', |
2936 | - unapplied_changelog_parent_commit, |
2937 | - upload_parent_commit |
2938 | - ] |
2939 | - ) |
2940 | - unapplied_changelog_parent_commit = None |
2941 | - except CalledProcessError as e: |
2942 | - if e.returncode != 1: |
2943 | - raise |
2944 | - |
2945 | - self.commit_unapplied_patches_import( |
2946 | - spi, |
2947 | - unapplied_import_tree_hash, |
2948 | - unapplied_publish_parent_commit, |
2949 | - unapplied_changelog_parent_commit, |
2950 | - upload_parent_commit, |
2951 | - ) |
2952 | - |
2953 | - self.update_devel_branches(spi) |
2954 | - |
2955 | - def import_applied_spi(self, spi): |
2956 | - """Imports a source package from Launchpad into the git |
2957 | - repository |
2958 | - |
2959 | - Arguments: |
2960 | - spi - a SourcePackageInformation instance |
2961 | - """ |
2962 | - pretty_head_name = spi.pretty_head_name |
2963 | - |
2964 | - logging.info('Importing patches-applied %s to %s' % (spi.version, pretty_head_name)) |
2965 | - applied_tip_head = self.local_repo.get_head_by_name(spi.applied_head_name(self.namespace)) |
2966 | - |
2967 | - spi.pull() |
2968 | - |
2969 | - self.local_repo.clean_repository_state() |
2970 | - |
2971 | - unapplied_parent_tag = self.local_repo.get_import_tag(spi.version, self.namespace) |
2972 | - unapplied_parent_commit = str(unapplied_parent_tag.peel().id) |
2973 | - unapplied_import_tree_hash = str(unapplied_parent_tag.peel(pygit2.Tree).id) |
2974 | - logging.debug('Found patches-unapplied version %s as commit %s', |
2975 | - str(spi.version), unapplied_parent_commit) |
2976 | - |
2977 | - import_tree_versions = self.local_repo.get_all_changelog_versions_from_treeish( |
2978 | - unapplied_import_tree_hash |
2979 | - ) |
2980 | - changelog_version = import_tree_versions[0] |
2981 | - |
2982 | - applied_tip_version = None |
2983 | - # check if the version to import has already been imported to |
2984 | - # this head |
2985 | - if applied_tip_head is not None: |
2986 | - if self.local_repo.treeishs_identical( |
2987 | - unapplied_import_tree_hash, str(applied_tip_head.peel().id) |
2988 | - ): |
2989 | - logging.warn('%s is identical to %s', |
2990 | - pretty_head_name, spi.version |
2991 | - ) |
2992 | - return |
2993 | - applied_tip_version, _ = self.local_repo.get_changelog_versions_from_treeish( |
2994 | - str(applied_tip_head.peel().id), |
2995 | - ) |
2996 | - |
2997 | - logging.debug('Tip version is %s', applied_tip_version) |
2998 | + unapplied_publish_parent_commit = None |
2999 | + except CalledProcessError as e: |
3000 | + if e.returncode != 1: |
3001 | + raise |
3002 | |
3003 | - applied_publish_parent_commit = None |
3004 | - applied_changelog_parent_commit = None |
3005 | + if unapplied_changelog_parent_commit is not None: |
3006 | + try: |
3007 | + repo.git_run(['merge-base', '--is-ancestor', |
3008 | + unapplied_changelog_parent_commit, |
3009 | + upload_parent_commit |
3010 | + ] |
3011 | + ) |
3012 | + unapplied_changelog_parent_commit = None |
3013 | + except CalledProcessError as e: |
3014 | + if e.returncode != 1: |
3015 | + raise |
3016 | |
3017 | - if spi.version in self.parent_overrides: |
3018 | - logging.debug('%s is specified in the parent override file.' % |
3019 | - spi.version |
3020 | + commit_unapplied_patches_import( |
3021 | + repo, |
3022 | + spi, |
3023 | + unapplied_import_tree_hash, |
3024 | + namespace, |
3025 | + unapplied_publish_parent_commit, |
3026 | + unapplied_changelog_parent_commit, |
3027 | + upload_parent_commit, |
3028 | + ) |
3029 | + |
3030 | + update_devel_branches( |
3031 | + repo=repo, |
3032 | + namespace=namespace, |
3033 | + pkgname=spi.name, |
3034 | + ubuntu_sinfo=ubuntu_sinfo, |
3035 | + spi=spi, |
3036 | + ) |
3037 | + |
3038 | +def import_applied_spi(repo, spi, namespace, ubuntu_sinfo): |
3039 | + """Imports a source package from Launchpad into the git |
3040 | + repository |
3041 | + |
3042 | + Arguments: |
3043 | + spi - a SourcePackageInformation instance |
3044 | + """ |
3045 | + pretty_head_name = spi.pretty_head_name |
3046 | + |
3047 | + logging.info('Importing patches-applied %s to %s' % (spi.version, pretty_head_name)) |
3048 | + applied_tip_head = repo.get_head_by_name(spi.applied_head_name(namespace)) |
3049 | + |
3050 | + spi.pull() |
3051 | + |
3052 | + repo.clean_repository_state() |
3053 | + |
3054 | + unapplied_parent_tag = repo.get_import_tag(spi.version, namespace) |
3055 | + unapplied_parent_commit = str(unapplied_parent_tag.peel().id) |
3056 | + unapplied_import_tree_hash = str(unapplied_parent_tag.peel(pygit2.Tree).id) |
3057 | + logging.debug('Found patches-unapplied version %s as commit %s', |
3058 | + str(spi.version), unapplied_parent_commit) |
3059 | + |
3060 | + import_tree_versions = repo.get_all_changelog_versions_from_treeish( |
3061 | + unapplied_import_tree_hash |
3062 | + ) |
3063 | + changelog_version = import_tree_versions[0] |
3064 | + |
3065 | + applied_tip_version = None |
3066 | + # check if the version to import has already been imported to |
3067 | + # this head |
3068 | + if applied_tip_head is not None: |
3069 | + if repo.treeishs_identical( |
3070 | + unapplied_import_tree_hash, str(applied_tip_head.peel().id) |
3071 | + ): |
3072 | + logging.warn('%s is identical to %s', |
3073 | + pretty_head_name, spi.version |
3074 | ) |
3075 | + return |
3076 | + applied_tip_version, _ = repo.get_changelog_versions_from_treeish( |
3077 | + str(applied_tip_head.peel().id), |
3078 | + ) |
3079 | |
3080 | - (_, _, |
3081 | - applied_publish_parent_commit, |
3082 | - applied_changelog_parent_commit) = self.override_parents(spi) |
3083 | - else: |
3084 | - # Get parent from publishing history (which is the last |
3085 | - # published version in the corresponding series) |
3086 | - try: |
3087 | - applied_publish_parent_commit = str(applied_tip_head.peel().id) |
3088 | - except AttributeError: |
3089 | - try: |
3090 | - applied_publish_parent_head = self.local_repo.get_head_by_name(spi.applied_parent_head_name(self.namespace)) |
3091 | - applied_publish_parent_head_commit = str(applied_publish_parent_head.peel().id) |
3092 | - except AttributeError: |
3093 | - pass |
3094 | - |
3095 | - if version_compare(str(spi.version), applied_tip_version) <= 0: |
3096 | - logging.warn('Version to import (%s) is not after %s tip (%s)', |
3097 | - spi.version, pretty_head_name, applied_tip_version |
3098 | - ) |
3099 | + logging.debug('Tip version is %s', applied_tip_version) |
3100 | |
3101 | - # Walk changelog backwards until we find an imported version |
3102 | - for version in import_tree_versions: |
3103 | - applied_changelog_parent_tag = self.local_repo.get_applied_tag(version, self.namespace) |
3104 | - if applied_changelog_parent_tag is not None: |
3105 | - # sanity check that version from d/changelog of the |
3106 | - # tagged parent matches ours |
3107 | - parent_changelog_version, _ = \ |
3108 | - self.local_repo.get_changelog_versions_from_treeish( |
3109 | - str(applied_changelog_parent_tag.peel(pygit2.Tree).id), |
3110 | - ) |
3111 | - # if the parent was imported via a parent override |
3112 | - # itself, assume it is valid without checking its |
3113 | - # tree's debian/changelog, as it wasn't used |
3114 | - if (version not in self.parent_overrides and |
3115 | - parent_changelog_version != version |
3116 | - ): |
3117 | - logging.error('Found a tag corresponding to ' |
3118 | - 'parent version %s, but d/changelog ' |
3119 | - 'version (%s) differs. Will ' |
3120 | - 'orphan commit.' % ( |
3121 | - version, |
3122 | - parent_changelog_version |
3123 | - ) |
3124 | - ) |
3125 | - else: |
3126 | - applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id) |
3127 | - logging.debug('Changelog parent (tag) is %s', |
3128 | - self.local_repo.tag_to_pretty_name(applied_changelog_parent_tag) |
3129 | - ) |
3130 | - break |
3131 | - |
3132 | - # If the two parents are tree-identical, then favor publication |
3133 | - # history. This deals with copy-forwards between series and with |
3134 | - # syncs. |
3135 | - if self.local_repo.treeishs_identical(applied_publish_parent_commit, applied_changelog_parent_commit): |
3136 | - applied_changelog_parent_commit = None |
3137 | - |
3138 | - # Assume no patches to apply |
3139 | - applied_import_tree_hash = unapplied_import_tree_hash |
3140 | - # get tree id from above commit |
3141 | - for (applied_import_tree_hash, patch_name, patch_desc) in self.import_patches_applied_tree(spi.dsc_pathname): |
3142 | - # special case for .pc removal |
3143 | - if patch_name is None: |
3144 | - msg = b'%b.' % (patch_desc.encode()) |
3145 | - else: |
3146 | - if patch_desc is None: |
3147 | - patch_desc = '%s\n\nNo DEP3 Subject or Description header found' % patch_name |
3148 | - msg = b'%b\n\nGbp-Pq: %b.' % (patch_desc.encode(), patch_name.encode()) |
3149 | - unapplied_parent_commit = self.local_repo.commit_tree_hash( |
3150 | - applied_import_tree_hash, |
3151 | - [unapplied_parent_commit], |
3152 | - msg, |
3153 | - spi) |
3154 | - |
3155 | - logging.debug('Committed patch-application of %s as %s', |
3156 | - patch_name, unapplied_parent_commit |
3157 | - ) |
3158 | + applied_publish_parent_commit = None |
3159 | + applied_changelog_parent_commit = None |
3160 | |
3161 | - self.commit_applied_patches_import( |
3162 | - spi, |
3163 | - applied_import_tree_hash, |
3164 | + if spi.version in _PARENT_OVERRIDES: |
3165 | + logging.debug('%s is specified in the parent override file.' % |
3166 | + spi.version |
3167 | + ) |
3168 | + |
3169 | + ( |
3170 | + _, |
3171 | + _, |
3172 | applied_publish_parent_commit, |
3173 | applied_changelog_parent_commit, |
3174 | - unapplied_parent_commit |
3175 | - ) |
3176 | - |
3177 | - self.update_applied_devel_branches(spi) |
3178 | - |
3179 | - def import_publishes(self, patches_applied, |
3180 | - debian_head_versions, ubuntu_head_versions): |
3181 | - history_found = False |
3182 | - only_debian = False |
3183 | - srcpkg_information = None |
3184 | - if patches_applied: |
3185 | - namespace = '%s/applied' % self.namespace |
3186 | - import_type = 'patches-applied' |
3187 | - import_func = self.import_applied_spi |
3188 | - else: |
3189 | - namespace = self.namespace |
3190 | - import_type = 'patches-unapplied' |
3191 | - import_func = self.import_unapplied_spi |
3192 | - for distname, versions, dist_sinfo in ( |
3193 | - ("debian", debian_head_versions, self.debian_sinfo), |
3194 | - ("ubuntu", ubuntu_head_versions, self.ubuntu_sinfo)): |
3195 | - if self.active_series_only and distname == "debian": |
3196 | - continue |
3197 | - try: |
3198 | - for srcpkg_information in dist_sinfo.launchpad_versions_published_after( |
3199 | - versions, |
3200 | - namespace, |
3201 | - workdir=self.workdir, |
3202 | - active_series_only=self.active_series_only |
3203 | - ): |
3204 | - history_found = True |
3205 | - import_func(srcpkg_information) |
3206 | - # Do not push upstream |
3207 | - if self.fixup_devel and distname == "ubuntu": |
3208 | - self.update_devel_branches(spi=None) |
3209 | - except NoPublicationHistoryException: |
3210 | - logging.warning("No publication history found for %s in %s. ", |
3211 | - self.pkgname, distname) |
3212 | - if distname == 'ubuntu': |
3213 | - only_debian = True |
3214 | - except Exception as e: |
3215 | - if srcpkg_information is None: |
3216 | - msg = 'Unable to import %s to %s' % (import_type, distname) |
3217 | - else: |
3218 | - msg = 'Unable to import %s %s to %s' % (import_type, |
3219 | - str(srcpkg_information.version), distname) |
3220 | - if not patches_applied: |
3221 | - raise GitUbuntuImportError(msg) from e |
3222 | - else: |
3223 | - logging.error(msg) |
3224 | - else: |
3225 | - history_found = True |
3226 | - |
3227 | - return (only_debian, history_found) |
3228 | - |
3229 | - def main(self, args): |
3230 | - self.pkgname = args.package |
3231 | - owner = args.lp_owner |
3232 | - no_clean = args.no_clean |
3233 | - try: |
3234 | - directory = args.directory |
3235 | - no_clean = True |
3236 | - except AttributeError: |
3237 | - directory = None |
3238 | - try: |
3239 | - user = args.lp_user |
3240 | - except AttributeError: |
3241 | - user = None |
3242 | - # --active-series-only/--no-fetch to a tmpdir/--skip-orig |
3243 | - # implies --no-push |
3244 | - no_push = ( |
3245 | - args.no_push or |
3246 | - (args.no_fetch and not directory) or |
3247 | - args.active_series_only or |
3248 | - args.skip_orig |
3249 | - ) |
3250 | + ) = override_parents( |
3251 | + repo, |
3252 | + spi, |
3253 | + namespace, |
3254 | + ) |
3255 | + else: |
3256 | + # Get parent from publishing history (which is the last |
3257 | + # published version in the corresponding series) |
3258 | try: |
3259 | - dl_cache = args.dl_cache |
3260 | + applied_publish_parent_commit = str(applied_tip_head.peel().id) |
3261 | except AttributeError: |
3262 | - dl_cache = None |
3263 | - self.fixup_devel = args.fixup_devel |
3264 | - self.active_series_only = args.active_series_only |
3265 | - self.skip_orig = args.skip_orig |
3266 | - |
3267 | - if owner == 'usd-import-team': |
3268 | - self.namespace = 'importer' |
3269 | - else: |
3270 | - self.namespace = owner |
3271 | - |
3272 | - logging.info('Ubuntu Server Team importer v%s' % VERSION) |
3273 | - |
3274 | - self.local_repo = GitUbuntuRepository(directory, user, args.proto) |
3275 | - if not directory: |
3276 | - logging.info('Using git repository at %s', self.local_repo.local_dir) |
3277 | - |
3278 | - atexit.register(self.cleanup, no_clean, self.local_repo.local_dir) |
3279 | - |
3280 | - self.local_repo.add_base_remotes(self.pkgname, repo_owner=owner) |
3281 | - if not args.no_fetch and not args.reimport: |
3282 | - try: |
3283 | - self.local_repo.fetch_base_remotes() |
3284 | - except GitUbuntuRepositoryFetchError: |
3285 | - pass |
3286 | - |
3287 | - if not args.no_fetch: |
3288 | - self.local_repo.delete_branches_in_namespace(self.namespace) |
3289 | - self.local_repo.delete_tags_in_namespace(self.namespace) |
3290 | - self.local_repo.copy_base_references(self.namespace) |
3291 | - |
3292 | - if not args.no_fetch and not args.reimport: |
3293 | - try: |
3294 | - self.local_repo.fetch_remote_refspecs('pkg', |
3295 | - refspecs=['refs/tags/*:refs/tags/%s/*' % self.namespace], |
3296 | - ) |
3297 | - except GitUbuntuRepositoryFetchError: |
3298 | - pass |
3299 | - |
3300 | - if args.reimport: |
3301 | - if args.no_fetch: |
3302 | - logging.warning( |
3303 | - "--reimport specified with --no-fetch. Upload " |
3304 | - "tags would not be incorporated into the new import " |
3305 | - "and would be lost forever. This is not currently " |
3306 | - "supported and --no-fetch will be ignored for upload " |
3307 | - "tags." |
3308 | - ) |
3309 | try: |
3310 | - self.local_repo.fetch_remote_refspecs('pkg', |
3311 | - refspecs=['refs/tags/upload/*:refs/tags/%s/upload/*' % self.namespace], |
3312 | - ) |
3313 | - except GitUbuntuRepositoryFetchError: |
3314 | + applied_publish_parent_head = repo.get_head_by_name(spi.applied_parent_head_name(namespace)) |
3315 | + applied_publish_parent_head_commit = str(applied_publish_parent_head.peel().id) |
3316 | + except AttributeError: |
3317 | pass |
3318 | |
3319 | - self.local_repo.ensure_importer_branches_exist(self.namespace) |
3320 | + if version_compare(str(spi.version), applied_tip_version) <= 0: |
3321 | + logging.warn('Version to import (%s) is not after %s tip (%s)', |
3322 | + spi.version, pretty_head_name, applied_tip_version |
3323 | + ) |
3324 | |
3325 | - self.debian_sinfo = GitUbuntuSourceInformation('debian', |
3326 | - self.pkgname, os.path.abspath(args.pullfile), |
3327 | - args.retries, args.retry_backoffs) |
3328 | + # Walk changelog backwards until we find an imported version |
3329 | + for version in import_tree_versions: |
3330 | + applied_changelog_parent_tag = repo.get_applied_tag(version, namespace) |
3331 | + if applied_changelog_parent_tag is not None: |
3332 | + # sanity check that version from d/changelog of the |
3333 | + # tagged parent matches ours |
3334 | + parent_changelog_version, _ = \ |
3335 | + repo.get_changelog_versions_from_treeish( |
3336 | + str(applied_changelog_parent_tag.peel(pygit2.Tree).id), |
3337 | + ) |
3338 | + # if the parent was imported via a parent override |
3339 | + # itself, assume it is valid without checking its |
3340 | + # tree's debian/changelog, as it wasn't used |
3341 | + if (version not in _PARENT_OVERRIDES and |
3342 | + parent_changelog_version != version |
3343 | + ): |
3344 | + logging.error('Found a tag corresponding to ' |
3345 | + 'parent version %s, but d/changelog ' |
3346 | + 'version (%s) differs. Will ' |
3347 | + 'orphan commit.' % ( |
3348 | + version, |
3349 | + parent_changelog_version |
3350 | + ) |
3351 | + ) |
3352 | + else: |
3353 | + applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id) |
3354 | + logging.debug('Changelog parent (tag) is %s', |
3355 | + repo.tag_to_pretty_name(applied_changelog_parent_tag) |
3356 | + ) |
3357 | + break |
3358 | |
3359 | - self.ubuntu_sinfo = GitUbuntuSourceInformation('ubuntu', |
3360 | - self.pkgname, os.path.abspath(args.pullfile), |
3361 | - args.retries, args.retry_backoffs) |
3362 | + # If the two parents are tree-identical, then favor publication |
3363 | + # history. This deals with copy-forwards between series and with |
3364 | + # syncs. |
3365 | + if repo.treeishs_identical(applied_publish_parent_commit, applied_changelog_parent_commit): |
3366 | + applied_changelog_parent_commit = None |
3367 | |
3368 | - debian_head_versions = ( |
3369 | - self.local_repo.get_heads_and_versions('debian', self.namespace) |
3370 | - ) |
3371 | - for head_name in sorted(debian_head_versions): |
3372 | - _, _, pretty_head_name = head_name.partition('%s/' % self.namespace) |
3373 | - logging.debug('Last imported %s version is %s', |
3374 | - pretty_head_name, |
3375 | - debian_head_versions[head_name]['version'] |
3376 | + # Assume no patches to apply |
3377 | + applied_import_tree_hash = unapplied_import_tree_hash |
3378 | + # get tree id from above commit |
3379 | + for ( |
3380 | + applied_import_tree_hash, |
3381 | + patch_name, |
3382 | + patch_desc |
3383 | + ) in import_patches_applied_tree(repo, spi.dsc_pathname): |
3384 | + # special case for .pc removal |
3385 | + if patch_name is None: |
3386 | + msg = b'%b.' % (patch_desc.encode()) |
3387 | + else: |
3388 | + if patch_desc is None: |
3389 | + patch_desc = ( |
3390 | + "%s\n\nNo DEP3 Subject or Description header found" % |
3391 | + patch_name |
3392 | + ) |
3393 | + msg = b'%b\n\nGbp-Pq: %b.' % ( |
3394 | + patch_desc.encode(), |
3395 | + patch_name.encode() |
3396 | ) |
3397 | + unapplied_parent_commit = repo.commit_tree_hash( |
3398 | + applied_import_tree_hash, |
3399 | + [unapplied_parent_commit], |
3400 | + msg, |
3401 | + spi |
3402 | + ) |
3403 | |
3404 | - ubuntu_head_versions = ( |
3405 | - self.local_repo.get_heads_and_versions('ubuntu', self.namespace) |
3406 | + logging.debug( |
3407 | + "Committed patch-application of %s as %s", |
3408 | + patch_name, unapplied_parent_commit |
3409 | ) |
3410 | - for head_name in sorted(ubuntu_head_versions): |
3411 | - _, _, pretty_head_name = head_name.partition('%s/' % self.namespace) |
3412 | - logging.debug('Last imported %s version is %s', |
3413 | - pretty_head_name, |
3414 | - ubuntu_head_versions[head_name]['version'] |
3415 | - ) |
3416 | |
3417 | - if not args.skip_applied: |
3418 | - applied_debian_head_versions = ( |
3419 | - self.local_repo.get_heads_and_versions( |
3420 | - 'applied/debian', self.namespace |
3421 | - ) |
3422 | - ) |
3423 | - for head_name in sorted(applied_debian_head_versions): |
3424 | - _, _, pretty_head_name = head_name.partition( |
3425 | - '%s/' % self.namespace |
3426 | - ) |
3427 | - logging.debug('Last applied %s version is %s', |
3428 | - pretty_head_name, |
3429 | - applied_debian_head_versions[head_name]['version'] |
3430 | + commit_applied_patches_import( |
3431 | + repo, |
3432 | + spi, |
3433 | + applied_import_tree_hash, |
3434 | + namespace, |
3435 | + applied_publish_parent_commit, |
3436 | + applied_changelog_parent_commit, |
3437 | + unapplied_parent_commit, |
3438 | + ) |
3439 | + |
3440 | + update_applied_devel_branches( |
3441 | + repo=repo, |
3442 | + pkgname=spi.name, |
3443 | + namespace=namespace, |
3444 | + ubuntu_sinfo=ubuntu_sinfo, |
3445 | + spi=spi, |
3446 | + ) |
3447 | + |
3448 | +def import_publishes(repo, pkgname, namespace, patches_applied, |
3449 | + debian_head_versions, ubuntu_head_versions, debian_sinfo, |
3450 | + ubuntu_sinfo, active_series_only, workdir, fixup_devel, |
3451 | + skip_orig, |
3452 | +): |
3453 | + history_found = False |
3454 | + only_debian = False |
3455 | + srcpkg_information = None |
3456 | + if patches_applied: |
3457 | + _namespace = namespace |
3458 | + namespace = '%s/applied' % namespace |
3459 | + import_type = 'patches-applied' |
3460 | + import_func = import_applied_spi |
3461 | + else: |
3462 | + _namespace = namespace |
3463 | + import_type = 'patches-unapplied' |
3464 | + import_func = functools.partial( |
3465 | + import_unapplied_spi, |
3466 | + skip_orig=skip_orig |
3467 | + ) |
3468 | + for distname, versions, dist_sinfo in ( |
3469 | + ("debian", debian_head_versions, debian_sinfo), |
3470 | + ("ubuntu", ubuntu_head_versions, ubuntu_sinfo)): |
3471 | + if active_series_only and distname == "debian": |
3472 | + continue |
3473 | + try: |
3474 | + for srcpkg_information in dist_sinfo.launchpad_versions_published_after( |
3475 | + versions, |
3476 | + namespace, |
3477 | + workdir=workdir, |
3478 | + active_series_only=active_series_only |
3479 | + ): |
3480 | + history_found = True |
3481 | + import_func( |
3482 | + repo=repo, |
3483 | + spi=srcpkg_information, |
3484 | + namespace=_namespace, |
3485 | + ubuntu_sinfo=ubuntu_sinfo, |
3486 | ) |
3487 | - |
3488 | - applied_ubuntu_head_versions = ( |
3489 | - self.local_repo.get_heads_and_versions( |
3490 | - 'applied/ubuntu', self.namespace |
3491 | + # Do not push upstream |
3492 | + if fixup_devel and distname == "ubuntu": |
3493 | + update_devel_branches( |
3494 | + repo=repo, |
3495 | + namespace=namespace, |
3496 | + pkgname=pkgname, |
3497 | + ubuntu_sinfo=ubuntu_sinfo, |
3498 | + spi=None, |
3499 | ) |
3500 | + except NoPublicationHistoryException: |
3501 | + logging.warning("No publication history found for %s in %s.", |
3502 | + pkgname, distname |
3503 | ) |
3504 | - for head_name in sorted(applied_ubuntu_head_versions): |
3505 | - _, _, pretty_head_name = head_name.partition( |
3506 | - '%s/' % self.namespace |
3507 | - ) |
3508 | - logging.debug('Last applied %s version is %s', |
3509 | - pretty_head_name, |
3510 | - applied_ubuntu_head_versions[head_name]['version'] |
3511 | - ) |
3512 | - |
3513 | - oldcwd = os.getcwd() |
3514 | - os.chdir(self.local_repo.local_dir) |
3515 | - |
3516 | - if dl_cache is None: |
3517 | - self.workdir = os.path.join(self.local_repo.git_dir, CACHE_PATH) |
3518 | - else: |
3519 | - self.workdir = dl_cache |
3520 | - |
3521 | - os.makedirs(self.workdir, exist_ok=True) |
3522 | - |
3523 | - self.parse_parentfile(args.parentfile) |
3524 | - |
3525 | - only_debian, history_found = self.import_publishes( |
3526 | - patches_applied=False, |
3527 | - debian_head_versions=debian_head_versions, |
3528 | - ubuntu_head_versions=ubuntu_head_versions) |
3529 | - |
3530 | - if not history_found: |
3531 | - logging.error("No publication history for '%s' in debian or ubuntu." |
3532 | - " Wrong source package name?", self.pkgname) |
3533 | - sys.exit(1) |
3534 | - |
3535 | - if not args.skip_applied: |
3536 | - self.import_publishes(patches_applied=True, |
3537 | - debian_head_versions=applied_debian_head_versions, |
3538 | - ubuntu_head_versions=applied_ubuntu_head_versions) |
3539 | - |
3540 | - os.chdir(oldcwd) |
3541 | - |
3542 | - self.local_repo.garbage_collect() |
3543 | - |
3544 | - if no_push: |
3545 | - logging.info('Not pushing to remote as specified') |
3546 | + if distname == 'ubuntu': |
3547 | + only_debian = True |
3548 | + except Exception as e: |
3549 | + if srcpkg_information is None: |
3550 | + msg = 'Unable to import %s to %s' % (import_type, distname) |
3551 | + else: |
3552 | + msg = 'Unable to import %s %s to %s' % (import_type, |
3553 | + str(srcpkg_information.version), distname) |
3554 | + #if not patches_applied: |
3555 | + raise GitUbuntuImportError(msg) from e |
3556 | + #else: |
3557 | + # logging.error(msg) |
3558 | else: |
3559 | - lp = launchpad_login_auth() |
3560 | - repo_path = '~%s/ubuntu/+source/%s/+git/%s' % ( |
3561 | - owner, |
3562 | - self.pkgname, |
3563 | - self.pkgname, |
3564 | - ) |
3565 | - lp_git_repo = lp.git_repositories.getByPath(path=repo_path) |
3566 | - if args.reimport: |
3567 | - if not lp_git_repo.canBeDeleted(): |
3568 | - logging.warning( |
3569 | - "There are linked MPs to the pkg repository, " |
3570 | - "which will be deleted:" |
3571 | - ) |
3572 | - for mp in lp_git_repo.landing_candidates: |
3573 | - logging.warning( |
3574 | - "Adding comment to %s indicating reimport", |
3575 | - mp.web_link |
3576 | - ) |
3577 | - mp.createComment( |
3578 | - content="%s has been reimported and this MP " |
3579 | - "must be manually resubmitted." % |
3580 | - self.pkgname, |
3581 | - subject="%s has been reimported" % self.pkgname, |
3582 | + history_found = True |
3583 | + |
3584 | + return (only_debian, history_found) |
3585 | + |
3586 | + |
3587 | +def parse_args(subparsers=None, base_subparsers=None): |
3588 | + kwargs = dict(description='Update a launchpad git tree based upon ' |
3589 | + 'the state of the Ubuntu and Debian archives', |
3590 | + formatter_class=argparse.ArgumentDefaultsHelpFormatter |
3591 | + ) |
3592 | + if base_subparsers: |
3593 | + kwargs['parents'] = base_subparsers |
3594 | + if subparsers: |
3595 | + parser = subparsers.add_parser('import', **kwargs) |
3596 | + parser.set_defaults(func=cli_main) |
3597 | + else: |
3598 | + parser = argparse.ArgumentParser(**kwargs) |
3599 | + parser.add_argument('package', type=str, |
3600 | + help='Which package to update in the git tree') |
3601 | + parser.add_argument('-o', '--lp-owner', type=str, |
3602 | + help=argparse.SUPPRESS, |
3603 | + default='usd-import-team') |
3604 | + parser.add_argument('-l', '--lp-user', type=str, |
3605 | + help=argparse.SUPPRESS) |
3606 | + parser.add_argument('--dl-cache', type=str, |
3607 | + help=('Cache directory for downloads. Default is to ' |
3608 | + 'put downloads in <directory>/.git/%s' % |
3609 | + CACHE_PATH |
3610 | + ), |
3611 | + default=argparse.SUPPRESS) |
3612 | + parser.add_argument('--no-fetch', action='store_true', |
3613 | + help='Do not fetch from the remote (DANGEROUS; ' |
3614 | + 'only useful for debugging); implies ' |
3615 | + '--no-push') |
3616 | + parser.add_argument('--no-push', action='store_true', |
3617 | + help='Do not push to the remote') |
3618 | + parser.add_argument('--no-clean', action='store_true', |
3619 | + help='Do not clean the temporary directory') |
3620 | + parser.add_argument('-d', '--directory', type=str, |
3621 | + help='Use git repository at specified location rather ' |
3622 | + 'than a temporary directory (implies --no-clean). ' |
3623 | + 'Will create the local repository if needed.', |
3624 | + default=argparse.SUPPRESS |
3625 | ) |
3626 | - # repo deletion will delete the MP, but hopefully |
3627 | - # the above comment will be sufficient to explain |
3628 | - # why |
3629 | - lp_git_repo.lp_delete() |
3630 | - |
3631 | - self.local_repo.git_run([ |
3632 | - 'push', '--atomic', 'pkg', |
3633 | - 'refs/heads/%s/*:refs/heads/*' % self.namespace, |
3634 | - 'refs/tags/%s/*:refs/tags/*' % self.namespace, |
3635 | - ]) |
3636 | - # update our reference |
3637 | - lp_git_repo = lp.git_repositories.getByPath(path=repo_path) |
3638 | - for i in range(args.retries): |
3639 | - try: |
3640 | - if only_debian: |
3641 | - lp_git_repo.default_branch = 'refs/heads/debian/sid' |
3642 | - else: |
3643 | - lp_git_repo.default_branch = 'refs/heads/ubuntu/devel' |
3644 | - lp_git_repo.lp_save() |
3645 | - break |
3646 | - except (NotFound, PreconditionFailed) as e: |
3647 | - time.sleep(args.retry_backoffs[i]) |
3648 | - lp_git_repo.lp_refresh() |
3649 | + parser.add_argument('--fixup-devel', action='store_true', |
3650 | + help=argparse.SUPPRESS) |
3651 | + parser.add_argument('--active-series-only', action='store_true', |
3652 | + help='Do an import of only active Ubuntu series ' |
3653 | + 'history, implies --no-push.') |
3654 | + parser.add_argument('--skip-applied', action='store_true', |
3655 | + help=argparse.SUPPRESS) |
3656 | + parser.add_argument('--skip-orig', action='store_true', |
3657 | + help=argparse.SUPPRESS) |
3658 | + parser.add_argument('--reimport', action='store_true', |
3659 | + help=argparse.SUPPRESS) |
3660 | + if not subparsers: |
3661 | + return parser.parse_args() |
3662 | + return 'import - %s' % kwargs['description'] |
3663 | + |
3664 | +def cli_main(args): |
3665 | + no_clean = args.no_clean |
3666 | + try: |
3667 | + directory = args.directory |
3668 | + no_clean = True |
3669 | + except AttributeError: |
3670 | + directory = None |
3671 | + try: |
3672 | + user = args.lp_user |
3673 | + except AttributeError: |
3674 | + user = None |
3675 | + # --active-series-only/--no-fetch to a tmpdir/--skip-orig |
3676 | + # implies --no-push |
3677 | + no_push = ( |
3678 | + args.no_push or |
3679 | + (args.no_fetch and not directory) or |
3680 | + args.active_series_only or |
3681 | + args.skip_orig |
3682 | + ) |
3683 | + try: |
3684 | + dl_cache = args.dl_cache |
3685 | + except AttributeError: |
3686 | + dl_cache = None |
3687 | + |
3688 | + main( |
3689 | + args.package, |
3690 | + args.lp_owner, |
3691 | + no_clean, |
3692 | + directory, |
3693 | + user, |
3694 | + args.proto, |
3695 | + args.no_fetch, |
3696 | + no_push, |
3697 | + dl_cache, |
3698 | + args.fixup_devel, |
3699 | + args.active_series_only, |
3700 | + args.skip_orig, |
3701 | + args.pullfile, |
3702 | + args.parentfile, |
3703 | + args.retries, |
3704 | + args.retry_backoffs, |
3705 | + args.skip_applied, |
3706 | + args.reimport, |
3707 | + ) |
3708 | diff --git a/gitubuntu/importlocal.py b/gitubuntu/importlocal.py |
3709 | index bf1700e..c237551 100644 |
3710 | --- a/gitubuntu/importlocal.py |
3711 | +++ b/gitubuntu/importlocal.py |
3712 | @@ -2,15 +2,22 @@ import argparse |
3713 | import glob |
3714 | import logging |
3715 | import os |
3716 | -import re |
3717 | -from subprocess import CalledProcessError |
3718 | import sys |
3719 | import tempfile |
3720 | from gitubuntu.dsc import GitUbuntuDsc |
3721 | from gitubuntu.git_repository import GitUbuntuRepository, git_dep14_tag |
3722 | -from gitubuntu.run import decode_binary, run, runq |
3723 | +from gitubuntu.run import run, runq |
3724 | from gitubuntu.version import VERSION |
3725 | -from gitubuntu.importer import GitUbuntuImport |
3726 | +from gitubuntu.importer import ( |
3727 | + parse_parentfile, |
3728 | + _PARENT_OVERRIDES, |
3729 | + import_orig, |
3730 | + import_dsc, |
3731 | + get_changelog_for_commit, |
3732 | + import_patches_unapplied_tree, |
3733 | + import_patches_applied_tree, |
3734 | +) |
3735 | +from gitubuntu.source_information import derive_source_from_series |
3736 | from gitubuntu.versioning import version_compare |
3737 | try: |
3738 | pkg = 'python3-pygit2' |
3739 | @@ -20,264 +27,309 @@ except ImportError: |
3740 | sys.exit(1) |
3741 | |
3742 | |
3743 | -class GitUbuntuImportLocal(GitUbuntuImport): |
3744 | - def __init__(self): |
3745 | - self.parent_overrides = None |
3746 | - |
3747 | - def parse_args(self, subparsers=None, base_subparsers=None): |
3748 | - kwargs = dict(description='Import a DSC file locally', |
3749 | - #usage='%(prog)s [options] -- ...', |
3750 | - epilog='Unlike Launchpad imports, local imports ' |
3751 | - 'require specifying a namespace to uniquely ' |
3752 | - 'identify the source of the DSC file. This ' |
3753 | - 'means that if you want to tie history ' |
3754 | - 'together for several DSCs from the same ' |
3755 | - 'source, you must use the same namespace value.', |
3756 | - formatter_class=argparse.ArgumentDefaultsHelpFormatter |
3757 | - ) |
3758 | - if base_subparsers: |
3759 | - kwargs['parents'] = base_subparsers |
3760 | - if subparsers: |
3761 | - parser = subparsers.add_parser('import-local', **kwargs) |
3762 | - parser.set_defaults(func=self.main) |
3763 | - else: |
3764 | - parser = argparse.ArgumentParser(**kwargs) |
3765 | - parser.add_argument('namespace', |
3766 | - help='String namespace to use for tagging the imports') |
3767 | - parser.add_argument('-d', '--directory', type=str, |
3768 | - help='Use git repository at specified location rather ' |
3769 | - 'than a temporary directory. ' |
3770 | - 'Will create the local repository if needed.', |
3771 | - default=argparse.SUPPRESS |
3772 | - ) |
3773 | - parser.add_argument('dsc', type=str, |
3774 | - help='Path to or URL of DSC file') |
3775 | - parser.add_argument('--skip-applied', action='store_true', |
3776 | - help=argparse.SUPPRESS) |
3777 | - parser.add_argument('--skip-orig', action='store_true', |
3778 | - help=argparse.SUPPRESS) |
3779 | - if not subparsers: |
3780 | - return parser.parse_args() |
3781 | - return 'import-local - %s' % kwargs['description'] |
3782 | - |
3783 | - def main(self, args): |
3784 | - try: |
3785 | - directory = args.directory |
3786 | - no_clean = True |
3787 | - except AttributeError: |
3788 | - directory = None |
3789 | - self.namespace = args.namespace |
3790 | - self.dsc_path = args.dsc |
3791 | - self.skip_applied = args.skip_applied |
3792 | - self.skip_orig = args.skip_orig |
3793 | - |
3794 | - self.local_repo = GitUbuntuRepository(directory) |
3795 | - if not directory: |
3796 | - logging.info('Using git repository at %s', self.local_repo.local_dir) |
3797 | - |
3798 | - self.local_repo.ensure_importer_branches_exist(self.namespace) |
3799 | - if len(self.local_repo.raw_repo.status()) != 0: |
3800 | - logging.error('Working tree must be clean to continue.') |
3801 | - run(['git', 'status'], stdout=None) |
3802 | - sys.exit(1) |
3803 | - |
3804 | - if os.path.exists(self.dsc_path): |
3805 | - self.dsc_path = os.path.abspath(self.dsc_path) |
3806 | - else: |
3807 | - dsc_dir = tempfile.mkdtemp() |
3808 | - |
3809 | - oldcwd = os.getcwd() |
3810 | - os.chdir(dsc_dir) |
3811 | - |
3812 | - runq(['dget', '--quiet', '--allow-unauthenticated', |
3813 | - '--download-only', self.dsc_path]) |
3814 | - dscs = glob.glob(os.path.join(dsc_dir, '*.dsc')) |
3815 | - if len(dscs) == 0: |
3816 | - logging.error('Unable to obtain DSC file.') |
3817 | - sys.exit(1) |
3818 | - if len(dscs) > 1: |
3819 | - logging.error('Multiple DSCs found in temporary directory.') |
3820 | - sys.exit(1) |
3821 | - self.dsc_path = os.path.join(dsc_dir, dscs[0]) |
3822 | - os.chdir(oldcwd) |
3823 | - logging.debug('Importing DSC at local path %s' % self.dsc_path) |
3824 | - self.dsc = GitUbuntuDsc(self.dsc_path) |
3825 | - if not self.dsc.verify(): |
3826 | - logging.error('Unable to verify orig tarball specified by DSC %s', |
3827 | - self.dsc_path) |
3828 | - sys.exit(1) |
3829 | - |
3830 | - self.pkgname = self.dsc['Source'] |
3831 | +def main( |
3832 | + directory, |
3833 | + namespace, |
3834 | + dsc_path, |
3835 | + skip_applied, |
3836 | + skip_orig, |
3837 | + parentfile, |
3838 | +): |
3839 | + repo = GitUbuntuRepository(directory) |
3840 | + if not directory: |
3841 | + logging.info('Using git repository at %s', repo.local_dir) |
3842 | + |
3843 | + repo.ensure_importer_branches_exist(namespace) |
3844 | + if len(repo.raw_repo.status()) != 0: |
3845 | + logging.error('Working tree must be clean to continue.') |
3846 | + run(['git', 'status'], stdout=None) |
3847 | + sys.exit(1) |
3848 | + |
3849 | + if os.path.exists(dsc_path): |
3850 | + dsc_path = os.path.abspath(dsc_path) |
3851 | + else: |
3852 | + dsc_dir = tempfile.mkdtemp() |
3853 | |
3854 | oldcwd = os.getcwd() |
3855 | - os.chdir(self.local_repo.local_dir) |
3856 | - |
3857 | - self.local_repo.clean_repository_state() |
3858 | + os.chdir(dsc_dir) |
3859 | |
3860 | - self.parse_parentfile(args.parentfile) |
3861 | - |
3862 | - unapplied_import_tree_hash = self.import_patches_unapplied_tree(self.dsc_path) |
3863 | - |
3864 | - import_tree_versions = self.local_repo.get_all_changelog_versions_from_treeish( |
3865 | - unapplied_import_tree_hash |
3866 | - ) |
3867 | - |
3868 | - changelog_version = import_tree_versions[0] |
3869 | - |
3870 | - logging.debug( |
3871 | - 'Found changelog version %s in newly imported tree' % |
3872 | - changelog_version |
3873 | - ) |
3874 | - |
3875 | - head = None |
3876 | - head_version = None |
3877 | - if not self.local_repo.raw_repo.head_is_unborn: |
3878 | - head = self.local_repo.raw_repo.head |
3879 | - # check if the version to import has already been imported to |
3880 | - # this head |
3881 | - if head is not None: |
3882 | - head_version, _ = self.local_repo.get_changelog_versions_from_treeish( |
3883 | - str(head.peel().id)) |
3884 | - |
3885 | - logging.debug('Head version is %s', head_version) |
3886 | - |
3887 | - unapplied_changelog_parent_commit = None |
3888 | - applied_changelog_parent_commit = None |
3889 | - |
3890 | - if version_compare(changelog_version, head_version) <= 0: |
3891 | - logging.warn('Version to import (%s) is not after HEAD ' |
3892 | - 'version (%s), history may not make sense.', |
3893 | - changelog_version, head_version |
3894 | - ) |
3895 | - |
3896 | - # Walk changelog backwards until we find an imported version |
3897 | - for version in import_tree_versions: |
3898 | - unapplied_tag_ref = '%s/import/%s' % (self.namespace, git_dep14_tag(version)) |
3899 | - applied_tag_ref = '%s/applied/%s' % (self.namespace, git_dep14_tag(version)) |
3900 | - unapplied_changelog_parent_tag = self.local_repo.get_tag_reference(unapplied_tag_ref) |
3901 | - applied_changelog_parent_tag = self.local_repo.get_tag_reference(applied_tag_ref) |
3902 | - if unapplied_changelog_parent_tag is not None: |
3903 | - # sanity check that version from d/changelog of the |
3904 | - # tagged parent matches ours |
3905 | - parent_changelog_version, _ = \ |
3906 | - self.local_repo.get_changelog_versions_from_treeish( |
3907 | - str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id), |
3908 | - ) |
3909 | - # if the parent was imported via a parent override |
3910 | - # itself, assume it is valid without checking its |
3911 | - # tree's debian/changelog, as it wasn't used |
3912 | - if (version not in self.parent_overrides and |
3913 | - parent_changelog_version != version |
3914 | - ): |
3915 | - logging.error('Found a tag corresponding to ' |
3916 | - 'parent version %s, but d/changelog ' |
3917 | - 'version (%s) differs. Will ' |
3918 | - 'orphan commit.' % ( |
3919 | - version, |
3920 | - parent_changelog_version |
3921 | - ) |
3922 | - ) |
3923 | - else: |
3924 | - unapplied_changelog_parent_commit = str(unapplied_changelog_parent_tag.peel().id) |
3925 | - if applied_changelog_parent_tag: |
3926 | - applied_changelog_parent_commit = str(applied_changelog_parent_tag.peel().id) |
3927 | - logging.debug('Changelog parent (tag) is %s', |
3928 | - self.local_repo.tag_to_pretty_name(unapplied_changelog_parent_tag) |
3929 | - ) |
3930 | - break |
3931 | - |
3932 | - distribution = self.local_repo.get_changelog_distribution_from_treeish(unapplied_import_tree_hash) |
3933 | - # do lookup of series for dist on lp? |
3934 | - dist = 'ubuntu' |
3935 | - if distribution in ("stable", "testing", "unstable", "experimental"): |
3936 | - dist = 'debian' |
3937 | - self.import_dsc(self.dsc, changelog_version, dist) |
3938 | - try: |
3939 | - if not self.skip_orig: |
3940 | - self.import_orig(self.dsc, changelog_version, dist) |
3941 | - except Exception as e: |
3942 | - logging.error('Unable to import orig tarball for %s: %s', |
3943 | - changelog_version, e) |
3944 | + runq(['dget', '--quiet', '--allow-unauthenticated', |
3945 | + '--download-only', dsc_path]) |
3946 | + dscs = glob.glob(os.path.join(dsc_dir, '*.dsc')) |
3947 | + if len(dscs) == 0: |
3948 | + logging.error('Unable to obtain DSC file.') |
3949 | sys.exit(1) |
3950 | - |
3951 | - msg = (b'Import unapplied version %b\n\nImported using git-ubuntu-import.' % |
3952 | - (changelog_version.encode()) |
3953 | - ) |
3954 | - |
3955 | - parents = [] |
3956 | - |
3957 | - tag = None |
3958 | - tag_ref = '%s/import/%s' % (self.namespace, |
3959 | - git_dep14_tag(changelog_version)) |
3960 | - if self.local_repo.get_tag_reference(tag_ref) is None: |
3961 | - # Not imported before |
3962 | - tag = tag_ref |
3963 | - |
3964 | - if unapplied_changelog_parent_commit is not None: |
3965 | - msg += b'\n' |
3966 | - parents.append(unapplied_changelog_parent_commit) |
3967 | - msg += b'\nChangelog parent: %b' % unapplied_changelog_parent_commit.encode() |
3968 | - |
3969 | - changelog_entry = self.get_changelog_for_commit( |
3970 | - unapplied_import_tree_hash, |
3971 | - None, |
3972 | - unapplied_changelog_parent_commit |
3973 | - ) |
3974 | - |
3975 | - msg += b'%b' % changelog_entry |
3976 | - unapplied_parent_commit = self.local_repo.commit_tree_hash( |
3977 | - unapplied_import_tree_hash, parents, msg) |
3978 | - if tag is None: |
3979 | - logging.info('Committed unapplied import of %s as %s', |
3980 | - changelog_version, unapplied_parent_commit) |
3981 | - else: |
3982 | - logging.debug('Creating tag %s pointing to %s', tag, |
3983 | - unapplied_parent_commit) |
3984 | - self.local_repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION, |
3985 | - tag, unapplied_parent_commit |
3986 | - ] |
3987 | - ) |
3988 | - |
3989 | - if not args.skip_applied: |
3990 | - # Assume no patches to apply |
3991 | - applied_import_tree_hash = unapplied_import_tree_hash |
3992 | - for (applied_import_tree_hash, patch_name, patch_desc) in self.import_patches_applied_tree(self.dsc_path): |
3993 | - # special case for .pc removal |
3994 | - if patch_name is None: |
3995 | - msg = b'%b.' % (patch_desc.encode()) |
3996 | - else: |
3997 | - if patch_desc is None: |
3998 | - patch_desc = '%s\n\nNo DEP3 Subject or Description header found' % patch_name |
3999 | - msg = b'%b\n\nGbp-Pq: %b.' % (patch_desc.encode(), patch_name.encode()) |
4000 | - unapplied_parent_commit = self.local_repo.commit_tree_hash( |
4001 | - applied_import_tree_hash, |
4002 | - [unapplied_parent_commit], |
4003 | - msg) |
4004 | - |
4005 | - logging.debug('Committed patch-application of %s as %s', |
4006 | - patch_name, unapplied_parent_commit |
4007 | - ) |
4008 | - |
4009 | - tag = None |
4010 | - tag_ref = '%s/applied/%s' % (self.namespace, |
4011 | - git_dep14_tag(changelog_version)) |
4012 | - if self.local_repo.get_tag_reference(tag_ref) is None: |
4013 | - # Not imported before |
4014 | - tag= tag_ref |
4015 | - |
4016 | - if tag is None: |
4017 | - logging.info('Committed applied import of %s as %s', |
4018 | - changelog_version, unapplied_parent_commit) |
4019 | - else: |
4020 | - logging.debug('Creating tag %s pointing to %s', tag, |
4021 | - unapplied_parent_commit) |
4022 | - self.local_repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION, |
4023 | - tag, unapplied_parent_commit |
4024 | - ] |
4025 | - ) |
4026 | - |
4027 | - self.local_repo.clean_repository_state() |
4028 | - |
4029 | + if len(dscs) > 1: |
4030 | + logging.error('Multiple DSCs found in temporary directory.') |
4031 | + sys.exit(1) |
4032 | + dsc_path = os.path.join(dsc_dir, dscs[0]) |
4033 | os.chdir(oldcwd) |
4034 | - |
4035 | - self.local_repo.garbage_collect() |
4036 | + logging.debug('Importing DSC at local path %s', dsc_path) |
4037 | + dsc = GitUbuntuDsc(dsc_path) |
4038 | + if not dsc.verify(): |
4039 | + logging.error( |
4040 | + 'Unable to verify orig tarball specified by DSC %s', |
4041 | + dsc_path, |
4042 | + ) |
4043 | + sys.exit(1) |
4044 | + |
4045 | + pkgname = dsc['Source'] |
4046 | + |
4047 | + oldcwd = os.getcwd() |
4048 | + os.chdir(repo.local_dir) |
4049 | + |
4050 | + repo.clean_repository_state() |
4051 | + |
4052 | + parse_parentfile(parentfile, pkgname) |
4053 | + |
4054 | + unapplied_import_tree_hash = import_patches_unapplied_tree(repo, dsc_path) |
4055 | + |
4056 | + import_tree_versions = repo.get_all_changelog_versions_from_treeish( |
4057 | + unapplied_import_tree_hash |
4058 | + ) |
4059 | + |
4060 | + changelog_version = import_tree_versions[0] |
4061 | + |
4062 | + logging.debug( |
4063 | + 'Found changelog version %s in newly imported tree', |
4064 | + changelog_version |
4065 | + ) |
4066 | + |
4067 | + head = None |
4068 | + head_version = None |
4069 | + if not repo.raw_repo.head_is_unborn: |
4070 | + head = repo.raw_repo.head |
4071 | + # check if the version to import has already been imported to |
4072 | + # this head |
4073 | + if head is not None: |
4074 | + head_version, _ = repo.get_changelog_versions_from_treeish( |
4075 | + str(head.peel().id) |
4076 | + ) |
4077 | + |
4078 | + logging.debug('Head version is %s', head_version) |
4079 | + |
4080 | + unapplied_changelog_parent_commit = None |
4081 | + applied_changelog_parent_commit = None |
4082 | + |
4083 | + if version_compare(changelog_version, head_version) <= 0: |
4084 | + logging.warn('Version to import (%s) is not after HEAD ' |
4085 | + 'version (%s), history may not make sense.', |
4086 | + changelog_version, head_version |
4087 | + ) |
4088 | + |
4089 | + # Walk changelog backwards until we find an imported version |
4090 | + for version in import_tree_versions: |
4091 | + unapplied_tag_ref = '%s/import/%s' % (namespace, git_dep14_tag(version)) |
4092 | + applied_tag_ref = '%s/applied/%s' % (namespace, git_dep14_tag(version)) |
4093 | + unapplied_changelog_parent_tag = repo.get_tag_reference(unapplied_tag_ref) |
4094 | + applied_changelog_parent_tag = repo.get_tag_reference(applied_tag_ref) |
4095 | + if unapplied_changelog_parent_tag is not None: |
4096 | + # sanity check that version from d/changelog of the |
4097 | + # tagged parent matches ours |
4098 | + parent_changelog_version, _ = \ |
4099 | + repo.get_changelog_versions_from_treeish( |
4100 | + str(unapplied_changelog_parent_tag.peel(pygit2.Tree).id), |
4101 | + ) |
4102 | + # if the parent was imported via a parent override |
4103 | + # itself, assume it is valid without checking its |
4104 | + # tree's debian/changelog, as it wasn't used |
4105 | + if (version not in _PARENT_OVERRIDES and |
4106 | + parent_changelog_version != version |
4107 | + ): |
4108 | + logging.error('Found a tag corresponding to ' |
4109 | + 'parent version %s, but d/changelog ' |
4110 | + 'version (%s) differs. Will ' |
4111 | + 'orphan commit.', |
4112 | + version, |
4113 | + parent_changelog_version |
4114 | + ) |
4115 | + else: |
4116 | + unapplied_changelog_parent_commit = str( |
4117 | + unapplied_changelog_parent_tag.peel().id |
4118 | + ) |
4119 | + if applied_changelog_parent_tag: |
4120 | + applied_changelog_parent_commit = str( |
4121 | + applied_changelog_parent_tag.peel().id |
4122 | + ) |
4123 | + logging.debug('Changelog parent (tag) is %s', |
4124 | + repo.tag_to_pretty_name(unapplied_changelog_parent_tag) |
4125 | + ) |
4126 | + break |
4127 | + |
4128 | + distribution = repo.get_changelog_distribution_from_treeish( |
4129 | + unapplied_import_tree_hash |
4130 | + ) |
4131 | + dist = derive_source_from_series(distribution) |
4132 | + import_dsc( |
4133 | + repo=repo, |
4134 | + dsc=dsc, |
4135 | + namespace=namespace, |
4136 | + version=changelog_version, |
4137 | + dist=dist, |
4138 | + ) |
4139 | + try: |
4140 | + if not skip_orig: |
4141 | + import_orig( |
4142 | + repo=repo, |
4143 | + dsc=dsc, |
4144 | + namespace=namespace, |
4145 | + version=changelog_version, |
4146 | + dist=dist, |
4147 | + ) |
4148 | + except Exception as e: |
4149 | + logging.error('Unable to import orig tarball for %s: %s', |
4150 | + changelog_version, e) |
4151 | + raise |
4152 | + sys.exit(1) |
4153 | + |
4154 | + msg = ( |
4155 | + b'Import unapplied version %b\n\nImported using git-ubuntu-import.' % |
4156 | + (changelog_version.encode()) |
4157 | + ) |
4158 | + |
4159 | + parents = [] |
4160 | + |
4161 | + tag = None |
4162 | + tag_ref = '%s/import/%s' % ( |
4163 | + namespace, |
4164 | + git_dep14_tag(changelog_version) |
4165 | + ) |
4166 | + if repo.get_tag_reference(tag_ref) is None: |
4167 | + # Not imported before |
4168 | + tag = tag_ref |
4169 | + |
4170 | + if unapplied_changelog_parent_commit is not None: |
4171 | + msg += b'\n' |
4172 | + parents.append(unapplied_changelog_parent_commit) |
4173 | + msg += b'\nChangelog parent: %b' % unapplied_changelog_parent_commit.encode() |
4174 | + |
4175 | + changelog_entry = get_changelog_for_commit( |
4176 | + repo=repo, |
4177 | + tree_hash=unapplied_import_tree_hash, |
4178 | + publish_parent_commit=None, |
4179 | + changelog_parent_commit=unapplied_changelog_parent_commit, |
4180 | + ) |
4181 | + |
4182 | + msg += b'%b' % changelog_entry |
4183 | + unapplied_parent_commit = repo.commit_tree_hash( |
4184 | + unapplied_import_tree_hash, parents, msg |
4185 | + ) |
4186 | + if tag is None: |
4187 | + logging.info('Committed unapplied import of %s as %s', |
4188 | + changelog_version, unapplied_parent_commit) |
4189 | + else: |
4190 | + logging.debug('Creating tag %s pointing to %s', tag, |
4191 | + unapplied_parent_commit |
4192 | + ) |
4193 | + repo.git_run(['tag', '-a', '-m', 'git-ubuntu import v%s' % VERSION, |
4194 | + tag, unapplied_parent_commit |
4195 | + ] |
4196 | + ) |
4197 | + |
4198 | + if not skip_applied: |
4199 | + # Assume no patches to apply |
4200 | + applied_import_tree_hash = unapplied_import_tree_hash |
4201 | + for ( |
4202 | + applied_import_tree_hash, |
4203 | + patch_name, |
4204 | + patch_desc, |
4205 | + ) in import_patches_applied_tree(repo, dsc_path): |
4206 | + # special case for .pc removal |
4207 | + if patch_name is None: |
4208 | + msg = b'%b.' % (patch_desc.encode()) |
4209 | + else: |
4210 | + if patch_desc is None: |
4211 | + patch_desc = '%s\n\nNo DEP3 Subject or Description header found' % patch_name |
4212 | + msg = b'%b\n\nGbp-Pq: %b.' % (patch_desc.encode(), patch_name.encode()) |
4213 | + unapplied_parent_commit = repo.commit_tree_hash( |
4214 | + applied_import_tree_hash, |
4215 | + [unapplied_parent_commit], |
4216 | + msg) |
4217 | + |
4218 | + logging.debug('Committed patch-application of %s as %s', |
4219 | + patch_name, unapplied_parent_commit |
4220 | + ) |
4221 | + |
4222 | + tag = None |
4223 | + tag_ref = '%s/applied/%s' % (namespace, |
4224 | + git_dep14_tag(changelog_version)) |
4225 | + if repo.get_tag_reference(tag_ref) is None: |
4226 | + # Not imported before |
4227 | + tag = tag_ref |
4228 | + |
4229 | + if tag is None: |
4230 | + logging.info('Committed applied import of %s as %s', |
4231 | + changelog_version, |
4232 | + unapplied_parent_commit |
4233 | + ) |
4234 | + else: |
4235 | + logging.debug('Creating tag %s pointing to %s', tag, |
4236 | + unapplied_parent_commit |
4237 | + ) |
4238 | + repo.git_run( |
4239 | + [ |
4240 | + 'tag', |
4241 | + '-a', |
4242 | + '-m', |
4243 | + 'git-ubuntu import v%s' % VERSION, |
4244 | + tag, |
4245 | + unapplied_parent_commit, |
4246 | + ] |
4247 | + ) |
4248 | + |
4249 | + repo.clean_repository_state() |
4250 | + |
4251 | + os.chdir(oldcwd) |
4252 | + |
4253 | + repo.garbage_collect() |
4254 | + |
4255 | +def parse_args(subparsers=None, base_subparsers=None): |
4256 | + kwargs = dict(description='Import a DSC file locally', |
4257 | + #usage='%(prog)s [options] -- ...', |
4258 | + epilog='Unlike Launchpad imports, local imports ' |
4259 | + 'require specifying a namespace to uniquely ' |
4260 | + 'identify the source of the DSC file. This ' |
4261 | + 'means that if you want to tie history ' |
4262 | + 'together for several DSCs from the same ' |
4263 | + 'source, you must use the same namespace value.', |
4264 | + formatter_class=argparse.ArgumentDefaultsHelpFormatter |
4265 | + ) |
4266 | + if base_subparsers: |
4267 | + kwargs['parents'] = base_subparsers |
4268 | + if subparsers: |
4269 | + parser = subparsers.add_parser('import-local', **kwargs) |
4270 | + parser.set_defaults(func=cli_main) |
4271 | + else: |
4272 | + parser = argparse.ArgumentParser(**kwargs) |
4273 | + parser.add_argument('namespace', |
4274 | + help='String namespace to use for tagging the imports') |
4275 | + parser.add_argument('-d', '--directory', type=str, |
4276 | + help='Use git repository at specified location rather ' |
4277 | + 'than a temporary directory. ' |
4278 | + 'Will create the local repository if needed.', |
4279 | + default=argparse.SUPPRESS |
4280 | + ) |
4281 | + parser.add_argument('dsc', type=str, |
4282 | + help='Path to or URL of DSC file') |
4283 | + parser.add_argument('--skip-applied', action='store_true', |
4284 | + help=argparse.SUPPRESS) |
4285 | + parser.add_argument('--skip-orig', action='store_true', |
4286 | + help=argparse.SUPPRESS) |
4287 | + if not subparsers: |
4288 | + return parser.parse_args() |
4289 | + return 'import-local - %s' % kwargs['description'] |
4290 | + |
4291 | +def cli_main(args): |
4292 | + try: |
4293 | + directory = args.directory |
4294 | + except AttributeError: |
4295 | + directory = None |
4296 | + |
4297 | + main( |
4298 | + directory, |
4299 | + args.namespace, |
4300 | + args.dsc, |
4301 | + args.skip_applied, |
4302 | + args.skip_orig, |
4303 | + args.parentfile, |
4304 | + ) |
4305 | diff --git a/gitubuntu/importppa.py b/gitubuntu/importppa.py |
4306 | index a4a3ec1..b4790f1 100644 |
4307 | --- a/gitubuntu/importppa.py |
4308 | +++ b/gitubuntu/importppa.py |
4309 | @@ -10,8 +10,25 @@ from gitubuntu.git_repository import ( |
4310 | GitUbuntuRepository, |
4311 | GitUbuntuRepositoryFetchError, |
4312 | ) |
4313 | -from gitubuntu.importer import GitUbuntuImport |
4314 | -from gitubuntu.source_information import GitUbuntuSourceInformation, NoPublicationHistoryException, SourceExtractionException, launchpad_login_auth |
4315 | +from gitubuntu.importer import ( |
4316 | + cleanup, |
4317 | + parse_parentfile, |
4318 | + _PARENT_OVERRIDES, |
4319 | + import_orig, |
4320 | + import_dsc, |
4321 | + get_changelog_for_commit, |
4322 | + import_unapplied_spi, |
4323 | + import_applied_spi, |
4324 | + import_patches_unapplied_tree, |
4325 | + import_patches_applied_tree, |
4326 | +) |
4327 | +from gitubuntu.source_information import ( |
4328 | + GitUbuntuSourceInformation, |
4329 | + NoPublicationHistoryException, |
4330 | + SourceExtractionException, |
4331 | + launchpad_login_auth, |
4332 | + GitUbuntuSourceInformation, |
4333 | +) |
4334 | from gitubuntu.version import VERSION |
4335 | try: |
4336 | pkg = 'python3-pygit2' |
4337 | @@ -23,181 +40,245 @@ except ImportError: |
4338 | sys.exit(1) |
4339 | |
4340 | |
4341 | -class GitUbuntuImportPPA(GitUbuntuImport): |
4342 | - def __init__(self): |
4343 | - self.parent_overrides = None |
4344 | - |
4345 | - def parse_args(self, subparsers=None, base_subparsers=None): |
4346 | - kwargs = dict(description='Update a launchpad git tree based upon ' |
4347 | - 'the state of a PPA archive', |
4348 | - epilog='PPA must be specified as ppa:<name>', |
4349 | - formatter_class=argparse.ArgumentDefaultsHelpFormatter |
4350 | - ) |
4351 | - if base_subparsers: |
4352 | - kwargs['parents'] = base_subparsers |
4353 | - if subparsers: |
4354 | - parser = subparsers.add_parser('import-ppa', **kwargs) |
4355 | - parser.set_defaults(func=self.main) |
4356 | - else: |
4357 | - parser = argparse.ArgumentParser(**kwargs) |
4358 | - parser.add_argument('ppa', type=str, |
4359 | - help='Name of PPA to update in the git tree') |
4360 | - parser.add_argument('package', type=str, |
4361 | - help='Which package to update in the git tree') |
4362 | - parser.add_argument('-o', '--lp-owner', type=str, |
4363 | - help='Which launchpad user\'s tree to update', |
4364 | - default='usd-import-team') |
4365 | - parser.add_argument('-l', '--lp-user', type=str, |
4366 | - help='Specify the Launchpad user to use', |
4367 | - default=getpass.getuser()) |
4368 | - parser.add_argument('--dl-cache', type=str, |
4369 | - help=('Cache directory for downloads. Default is to ' |
4370 | - 'put downloads in <directory>/.git/%s' % |
4371 | - CACHE_PATH |
4372 | - ), |
4373 | - default=argparse.SUPPRESS) |
4374 | - parser.add_argument('--no-fetch', action='store_true', |
4375 | - help='Do not fetch from the remote (DANGEROUS: ' |
4376 | - 'only useful for debugging); implies ' |
4377 | - '--no-push') |
4378 | - parser.add_argument('--no-push', action='store_true', |
4379 | - help='Do not push to the remote (WARNING: ' |
4380 | - 'PPA imports are a work-in-progress and ' |
4381 | - 'pushing is currently disabled)', |
4382 | - default=True) |
4383 | - parser.add_argument('--no-clean', action='store_true', |
4384 | - help='Do not clean the temporary directory') |
4385 | - parser.add_argument('-d', '--directory', type=str, |
4386 | - help='Use git repository at specified location rather ' |
4387 | - 'than a temporary directory (implies --no-clean). ' |
4388 | - 'Will create the local repository if needed.', |
4389 | - default=argparse.SUPPRESS |
4390 | - ) |
4391 | - if not subparsers: |
4392 | - return parser.parse_args() |
4393 | - return 'import-ppa - %s' % kwargs['description'] |
4394 | - |
4395 | - def main(self, args): |
4396 | - if re.match(r'ppa:\w+', args.ppa) is None: |
4397 | - logging.error('Specified PPA (%s) is not in the format ' |
4398 | - 'ppa:<name>', args.ppa) |
4399 | - sys.exit(1) |
4400 | - ppa = args.ppa |
4401 | - pkgname = args.package |
4402 | - owner = args.lp_owner |
4403 | - user = args.lp_user |
4404 | - no_clean = args.no_clean |
4405 | - try: |
4406 | - directory = args.directory |
4407 | - no_clean = True |
4408 | - except AttributeError: |
4409 | - directory = None |
4410 | - no_push = True |
4411 | - try: |
4412 | - dl_cache = args.dl_cache |
4413 | - except AttributeError: |
4414 | - dl_cache = None |
4415 | - |
4416 | - # what other transformations? |
4417 | - self.namespace = ppa.replace(':', '_') |
4418 | - |
4419 | - logging.info('Ubuntu Server Team importer v%s' % VERSION) |
4420 | - |
4421 | - self.local_repo = GitUbuntuRepository(directory, user, args.proto) |
4422 | - if not directory: |
4423 | - logging.info('Using git repository at %s', self.local_repo.local_dir) |
4424 | - self.local_repo.ensure_importer_branches_exist(self.namespace) |
4425 | - |
4426 | - atexit.register(self.cleanup, no_clean, self.local_repo.local_dir) |
4427 | - |
4428 | - self.local_repo.add_remote(pkgname, owner, self.namespace, user) |
4429 | - if not args.no_fetch: |
4430 | +def main( |
4431 | + ppa, |
4432 | + pkgname, |
4433 | + directory, |
4434 | + owner, |
4435 | + user, |
4436 | + pullfile, |
4437 | + parentfile, |
4438 | + no_clean, |
4439 | + no_fetch, |
4440 | + no_push, |
4441 | + dl_cache, |
4442 | + proto, |
4443 | + retries, |
4444 | + retry_backoffs, |
4445 | + skip_orig, |
4446 | +): |
4447 | + if re.match(r'ppa:\w+', ppa) is None: |
4448 | + logging.error( |
4449 | + "Specified PPA (%s) is not in the format ppa:<name>", |
4450 | + ppa, |
4451 | + ) |
4452 | + sys.exit(1) |
4453 | + # what other transformations? |
4454 | + namespace = ppa.replace(':', '_') |
4455 | + |
4456 | + logging.info('Ubuntu Server Team importer v%s' % VERSION) |
4457 | + |
4458 | + repo = GitUbuntuRepository(directory, user, proto) |
4459 | + if not directory: |
4460 | + logging.info('Using git repository at %s', repo.local_dir) |
4461 | + repo.ensure_importer_branches_exist(namespace) |
4462 | + |
4463 | + atexit.register(cleanup, no_clean, repo.local_dir) |
4464 | + |
4465 | + # This seems terminally broken and I'm not sure what we should do |
4466 | + # here |
4467 | + #repo.add_remote(pkgname, owner, namespace) |
4468 | + #if not no_fetch: |
4469 | + # try: |
4470 | + # repo.fetch_remote(namespace) |
4471 | + # except GitUbuntuRepositoryFetchError: |
4472 | + # pass |
4473 | + |
4474 | + source_information = GitUbuntuSourceInformation( |
4475 | + ppa, |
4476 | + pkgname, |
4477 | + os.path.abspath(pullfile), |
4478 | + retries, |
4479 | + retry_backoffs, |
4480 | + ) |
4481 | + |
4482 | + ubuntu_head_versions = repo.get_heads_and_versions( |
4483 | + 'ubuntu', |
4484 | + namespace, |
4485 | + ) |
4486 | + for head_name in sorted(ubuntu_head_versions): |
4487 | + _, _, pretty_head_name = head_name.partition('%s/', namespace) |
4488 | + logging.debug('Last imported %s version is %s', |
4489 | + pretty_head_name, |
4490 | + ubuntu_head_versions[head_name]['version'] |
4491 | + ) |
4492 | + |
4493 | + applied_ubuntu_head_versions = repo.get_heads_and_versions( |
4494 | + 'applied/ubuntu', |
4495 | + namespace, |
4496 | + ) |
4497 | + for head_name in sorted(applied_ubuntu_head_versions): |
4498 | + _, _, pretty_head_name = head_name.partition('%s/', namespace) |
4499 | + logging.debug('Last applied %s version is %s', |
4500 | + pretty_head_name, |
4501 | + applied_ubuntu_head_versions[head_name]['version'] |
4502 | + ) |
4503 | + |
4504 | + oldcwd = os.getcwd() |
4505 | + os.chdir(repo.local_dir) |
4506 | + |
4507 | + if dl_cache is None: |
4508 | + workdir = os.path.join(repo.git_dir, CACHE_PATH) |
4509 | + else: |
4510 | + workdir = dl_cache |
4511 | + |
4512 | + os.makedirs(workdir, exist_ok=True) |
4513 | + |
4514 | + parse_parentfile(pkgname, parentfile) |
4515 | + |
4516 | + history_found = False |
4517 | + try: |
4518 | + for srcpkg_information in \ |
4519 | + source_information.launchpad_versions_published_after( |
4520 | + ubuntu_head_versions, |
4521 | + namespace, |
4522 | + workdir=workdir |
4523 | + ): |
4524 | try: |
4525 | - self.local_repo.fetch_remote(self.namespace) |
4526 | - except GitUbuntuRepositoryFetchError: |
4527 | + import_unapplied_spi( |
4528 | + repo=repo, |
4529 | + spi=srcpkg_information, |
4530 | + namespace=namespace, |
4531 | + ubuntu_sinfo=source_information, |
4532 | + skip_orig=skip_orig, |
4533 | + ) |
4534 | + except DownloadError: |
4535 | + # it is non-fatal for a PPA to not have files for older |
4536 | + # publishes |
4537 | pass |
4538 | + history_found = True |
4539 | + except NoPublicationHistoryException: |
4540 | + logging.warning("No publication history found for %s in %s.", |
4541 | + pkgname, |
4542 | + ppa, |
4543 | + ) |
4544 | |
4545 | - source_information = UbuntuSourceInformation(ppa, pkgname, |
4546 | - os.path.abspath(args.pullfile), |
4547 | - args.retries, args.retry_backoffs) |
4548 | - |
4549 | - ubuntu_head_versions = self.local_repo.get_heads_and_versions( |
4550 | - 'ubuntu', self.namespace |
4551 | + if not history_found: |
4552 | + logging.error( |
4553 | + "No publication history for '%s' in %s. Wrong source package name?", |
4554 | + pkgname, |
4555 | + ppa, |
4556 | ) |
4557 | - for head_name in sorted(ubuntu_head_versions): |
4558 | - _, _, pretty_head_name = head_name.partition('%s/', self.namespace) |
4559 | - logging.debug('Last imported %s version is %s', |
4560 | - pretty_head_name, |
4561 | - ubuntu_head_versions[head_name]['version'] |
4562 | - ) |
4563 | - |
4564 | - applied_ubuntu_head_versions = self.local_repo.get_heads_and_versions( |
4565 | - 'applied/ubuntu', self.namespace |
4566 | + sys.exit(1) |
4567 | + |
4568 | + try: |
4569 | + for srcpkg_information in \ |
4570 | + source_information.launchpad_versions_published_after( |
4571 | + applied_ubuntu_head_versions, |
4572 | + '%s/applied' % namespace, |
4573 | + workdir=workdir |
4574 | + ): |
4575 | + try: |
4576 | + import_applied_spi( |
4577 | + repo=repo, |
4578 | + spi=srcpkg_information, |
4579 | + namespace=namespace, |
4580 | + ubuntu_sinfo=source_information, |
4581 | + ) |
4582 | + except DownloadError: |
4583 | + # it is non-fatal for a PPA to not have files for older |
4584 | + # publishes |
4585 | + pass |
4586 | + except NoPublicationHistoryException: |
4587 | + logging.warning("No publication history found for %s in %s.", |
4588 | + pkgname, ppa |
4589 | ) |
4590 | - for head_name in sorted(applied_ubuntu_head_versions): |
4591 | - _, _, pretty_head_name = head_name.partition('%s/', self.namespace) |
4592 | - logging.debug('Last applied %s version is %s', |
4593 | - pretty_head_name, |
4594 | - applied_ubuntu_head_versions[head_name]['version'] |
4595 | - ) |
4596 | - |
4597 | - oldcwd = os.getcwd() |
4598 | - os.chdir(self.local_repo.local_dir) |
4599 | - |
4600 | - if dl_cache is None: |
4601 | - workdir = os.path.join(self.local_repo.git_dir, CACHE_PATH) |
4602 | - else: |
4603 | - workdir = dl_cache |
4604 | - |
4605 | - os.makedirs(workdir, exist_ok=True) |
4606 | - |
4607 | - self.parse_parentfile(pkgname, args.parentfile) |
4608 | - |
4609 | - history_found = False |
4610 | - try: |
4611 | - for srcpkg_information in \ |
4612 | - source_information.launchpad_versions_published_after(ubuntu_head_versions, |
4613 | - self.namespace, |
4614 | - workdir=workdir): |
4615 | - try: |
4616 | - self.import_unapplied_spi(srcpkg_information) |
4617 | - except DownloadError: |
4618 | - # it is non-fatal for a PPA to not have files for older |
4619 | - # publishes |
4620 | - pass |
4621 | - history_found = True |
4622 | - except NoPublicationHistoryException: |
4623 | - logging.warning("No publication history found for %s in %s. ", |
4624 | - pkgname, distname) |
4625 | - |
4626 | - if not history_found: |
4627 | - logging.error("No publication history for '%s' in %s. " |
4628 | - "Wrong source package name?", pkgname, ppa) |
4629 | - sys.exit(1) |
4630 | - |
4631 | - try: |
4632 | - for srcpkg_information in \ |
4633 | - source_information.launchpad_versions_published_after(applied_ubuntu_head_versions, |
4634 | - '%s/applied' % self.namespace, |
4635 | - workdir=workdir): |
4636 | - try: |
4637 | - self.import_applied_spi(srcpkg_information) |
4638 | - except DownloadError: |
4639 | - # it is non-fatal for a PPA to not have files for older |
4640 | - # publishes |
4641 | - pass |
4642 | - except NoPublicationHistoryException: |
4643 | - logging.warning("No publication history found for %s in %s. ", |
4644 | - pkgname, ppa) |
4645 | - except: |
4646 | - logging.error("Unable to import patches-applied to %s", ppa) |
4647 | - |
4648 | - os.chdir(oldcwd) |
4649 | - |
4650 | - self.local_repo.garbage_collect() |
4651 | - |
4652 | - if no_push: |
4653 | - logging.info('Not pushing to remote as specified') |
4654 | - else: |
4655 | - # what are appropriate refspects to push and where? |
4656 | - pass |
4657 | + except: |
4658 | + logging.error("Unable to import patches-applied to %s", ppa) |
4659 | + |
4660 | + os.chdir(oldcwd) |
4661 | + |
4662 | + repo.garbage_collect() |
4663 | + |
4664 | + if no_push: |
4665 | + logging.info('Not pushing to remote as specified') |
4666 | + else: |
4667 | + # what are appropriate refspects to push and where? |
4668 | + pass |
4669 | + |
4670 | + |
4671 | +def parse_args(subparsers=None, base_subparsers=None): |
4672 | + kwargs = dict(description='Update a launchpad git tree based upon ' |
4673 | + 'the state of a PPA archive', |
4674 | + epilog='PPA must be specified as ppa:<name>', |
4675 | + formatter_class=argparse.ArgumentDefaultsHelpFormatter |
4676 | + ) |
4677 | + if base_subparsers: |
4678 | + kwargs['parents'] = base_subparsers |
4679 | + if subparsers: |
4680 | + parser = subparsers.add_parser('import-ppa', **kwargs) |
4681 | + parser.set_defaults(func=cli_main) |
4682 | + else: |
4683 | + parser = argparse.ArgumentParser(**kwargs) |
4684 | + parser.add_argument('ppa', type=str, |
4685 | + help='Name of PPA to update in the git tree') |
4686 | + parser.add_argument('package', type=str, |
4687 | + help='Which package to update in the git tree') |
4688 | + parser.add_argument('-o', '--lp-owner', type=str, |
4689 | + help='Which launchpad user\'s tree to update', |
4690 | + default='usd-import-team') |
4691 | + parser.add_argument('-l', '--lp-user', type=str, |
4692 | + help='Specify the Launchpad user to use', |
4693 | + default=getpass.getuser()) |
4694 | + parser.add_argument('--dl-cache', type=str, |
4695 | + help=('Cache directory for downloads. Default is to ' |
4696 | + 'put downloads in <directory>/.git/%s' % |
4697 | + CACHE_PATH |
4698 | + ), |
4699 | + default=argparse.SUPPRESS) |
4700 | + parser.add_argument('--no-fetch', action='store_true', |
4701 | + help='Do not fetch from the remote (DANGEROUS: ' |
4702 | + 'only useful for debugging); implies ' |
4703 | + '--no-push') |
4704 | + parser.add_argument('--no-push', action='store_true', |
4705 | + help='Do not push to the remote (WARNING: ' |
4706 | + 'PPA imports are a work-in-progress and ' |
4707 | + 'pushing is currently disabled)', |
4708 | + default=True) |
4709 | + parser.add_argument('--no-clean', action='store_true', |
4710 | + help='Do not clean the temporary directory') |
4711 | + parser.add_argument('-d', '--directory', type=str, |
4712 | + help='Use git repository at specified location rather ' |
4713 | + 'than a temporary directory (implies --no-clean). ' |
4714 | + 'Will create the local repository if needed.', |
4715 | + default=argparse.SUPPRESS |
4716 | + ) |
4717 | + parser.add_argument( |
4718 | + '--skip-orig', |
4719 | + action='store_true', |
4720 | + help=argparse.SUPPRESS, |
4721 | + ) |
4722 | + if not subparsers: |
4723 | + return parser.parse_args() |
4724 | + return 'import-ppa - %s' % kwargs['description'] |
4725 | + |
4726 | +def cli_main(args): |
4727 | + no_clean = args.no_clean |
4728 | + try: |
4729 | + directory = args.directory |
4730 | + no_clean = True |
4731 | + except AttributeError: |
4732 | + directory = None |
4733 | + no_push = True |
4734 | + try: |
4735 | + dl_cache = args.dl_cache |
4736 | + except AttributeError: |
4737 | + dl_cache = None |
4738 | + |
4739 | + main( |
4740 | + args.ppa, |
4741 | + args.package, |
4742 | + directory, |
4743 | + args.lp_owner, |
4744 | + args.lp_user, |
4745 | + args.pullfile, |
4746 | + args.parentfile, |
4747 | + no_clean, |
4748 | + args.no_fetch, |
4749 | + no_push, |
4750 | + dl_cache, |
4751 | + args.proto, |
4752 | + args.retries, |
4753 | + args.retry_backoffs, |
4754 | + args.skip_orig, |
4755 | + ) |
4756 | diff --git a/gitubuntu/lint.py b/gitubuntu/lint.py |
4757 | index 435e7f2..bce9388 100644 |
4758 | --- a/gitubuntu/lint.py |
4759 | +++ b/gitubuntu/lint.py |
4760 | @@ -23,10 +23,6 @@ except ImportError: |
4761 | logging.error("Is %s installed?", pkg) |
4762 | sys.exit(1) |
4763 | |
4764 | -__all__ = [ |
4765 | - 'GitUbuntuLint', |
4766 | -] |
4767 | - |
4768 | class LintNamespaceError(Exception): |
4769 | pass |
4770 | |
4771 | @@ -162,208 +158,214 @@ def test__derive_target_branch_string(same_remote_branch_names, |
4772 | ) |
4773 | assert target_branch_string == expected |
4774 | |
4775 | -class GitUbuntuLint: |
4776 | - def __init__(self): |
4777 | - pass |
4778 | |
4779 | - def success(self, msg, *args): |
4780 | - if self.verbose: |
4781 | - print(msg % args) |
4782 | +_verbose = False |
4783 | +def success(msg, *args): |
4784 | + if _verbose: |
4785 | + print(msg % args) |
4786 | |
4787 | - def parse_args(self, subparsers=None, base_subparsers=None): |
4788 | - kwargs = dict( |
4789 | - description="Given a commitish, perform automated checks", |
4790 | - formatter_class=argparse.RawTextHelpFormatter, |
4791 | - epilog="An exit code of 0 indicates all checks passed", |
4792 | - ) |
4793 | - if base_subparsers: |
4794 | - kwargs["parents"] = base_subparsers |
4795 | - if subparsers: |
4796 | - parser = subparsers.add_parser("lint", **kwargs) |
4797 | - parser.set_defaults(func=self.main) |
4798 | - else: |
4799 | - parser = argparse.ArgumentParser(**kwargs) |
4800 | |
4801 | - parser.add_argument("commitish", type=str, default="HEAD", nargs="?", |
4802 | - help="Commitish to lint" |
4803 | - ) |
4804 | - parser.add_argument("--lint-namespace", type=str, |
4805 | - help="The git-ref prefix under which the branches and tags " |
4806 | - "to lint can be found. If linting a remote-tracking " |
4807 | - "branch, this will be default to the remote name. If " |
4808 | - "linting a local branch, this will default to ''." |
4809 | - ) |
4810 | - parser.add_argument("--target-branch", type=str, |
4811 | - help="Target branch (typically a remote-tracking branch in pkg/) " |
4812 | - "the current branch is against. This will be derived if " |
4813 | - "possible." |
4814 | - ) |
4815 | - parser.add_argument("-d", "--directory", type=str, |
4816 | - help="Use git repository at specified location.", |
4817 | - default=os.path.abspath(os.getcwd()) |
4818 | - ) |
4819 | - if not subparsers: |
4820 | - return parser.parse_args() |
4821 | - return "lint - %s" % kwargs["description"] |
4822 | +def parse_args(subparsers=None, base_subparsers=None): |
4823 | + kwargs = dict( |
4824 | + description="Given a commitish, perform automated checks", |
4825 | + formatter_class=argparse.RawTextHelpFormatter, |
4826 | + epilog="An exit code of 0 indicates all checks passed", |
4827 | + ) |
4828 | + if base_subparsers: |
4829 | + kwargs["parents"] = base_subparsers |
4830 | + if subparsers: |
4831 | + parser = subparsers.add_parser("lint", **kwargs) |
4832 | + parser.set_defaults(func=cli_main) |
4833 | + else: |
4834 | + parser = argparse.ArgumentParser(**kwargs) |
4835 | |
4836 | - def _check_commitish_exists(self, commitish): |
4837 | - try: |
4838 | - actual_commitish_obj = self.local_repo.get_commitish( |
4839 | - commitish |
4840 | - ) |
4841 | - self.success("Verified %s exists", commitish) |
4842 | - return True |
4843 | - except KeyError: |
4844 | - warning("%s not found", commitish) |
4845 | - return False |
4846 | + parser.add_argument("commitish", type=str, default="HEAD", nargs="?", |
4847 | + help="Commitish to lint" |
4848 | + ) |
4849 | + parser.add_argument("--lint-namespace", type=str, |
4850 | + help="The git-ref prefix under which the branches and tags " |
4851 | + "to lint can be found. If linting a remote-tracking " |
4852 | + "branch, this will be default to the remote name. If " |
4853 | + "linting a local branch, this will default to ''." |
4854 | + ) |
4855 | + parser.add_argument("--target-branch", type=str, |
4856 | + help="Target branch (typically a remote-tracking branch in pkg/) " |
4857 | + "the current branch is against. This will be derived if " |
4858 | + "possible." |
4859 | + ) |
4860 | + parser.add_argument("-d", "--directory", type=str, |
4861 | + help="Use git repository at specified location.", |
4862 | + default=os.path.abspath(os.getcwd()) |
4863 | + ) |
4864 | + if not subparsers: |
4865 | + return parser.parse_args() |
4866 | + return "lint - %s" % kwargs["description"] |
4867 | + |
4868 | +def cli_main(args): |
4869 | + main( |
4870 | + args.directory, |
4871 | + args.commitish, |
4872 | + args.lint_namespace, |
4873 | + args.target_branch, |
4874 | + args.verbose, |
4875 | + ) |
4876 | |
4877 | - def _check_commitish_exists_and_same(self, expected_commitish_pretty_name, |
4878 | - actual_commitish, expected_commitish, check_commit_id=False |
4879 | - ): |
4880 | - """Check @actual_commitish exists ad is the same tree as |
4881 | - @expected_commitish |
4882 | - |
4883 | - Arguments: |
4884 | - expected_commitish_pretty_name - a pretty name to print to refer to the |
4885 | - expected value |
4886 | - actual_commitish - commitish to check for |
4887 | - expected_commitish - commitish to check against |
4888 | - check_commit_id - if True, check the commit hashes match as well |
4889 | - """ |
4890 | - ret = True |
4891 | - if not self._check_commitish_exists(actual_commitish): |
4892 | - return False |
4893 | +def _check_commitish_exists(repo, commitish): |
4894 | + try: |
4895 | + actual_commitish_obj = repo.get_commitish(commitish) |
4896 | + success("Verified %s exists", commitish) |
4897 | + return True |
4898 | + except KeyError: |
4899 | + warning("%s not found", commitish) |
4900 | + return False |
4901 | |
4902 | - try: |
4903 | - expected_commitish_obj = self.local_repo.get_commitish( |
4904 | - expected_commitish |
4905 | - ) |
4906 | - except KeyError: |
4907 | - fatal("Unable to find expected commit: %s", expected_commitish) |
4908 | +def _check_commitish_exists_and_same(repo, expected_commitish_pretty_name, |
4909 | + actual_commitish, expected_commitish, check_commit_id=False |
4910 | +): |
4911 | + """Check @actual_commitish exists ad is the same tree as |
4912 | + @expected_commitish |
4913 | + |
4914 | + Arguments: |
4915 | + expected_commitish_pretty_name - a pretty name to print to refer to the |
4916 | + expected value |
4917 | + actual_commitish - commitish to check for |
4918 | + expected_commitish - commitish to check against |
4919 | + check_commit_id - if True, check the commit hashes match as well |
4920 | + """ |
4921 | + ret = True |
4922 | + if not _check_commitish_exists(repo, actual_commitish): |
4923 | + return False |
4924 | |
4925 | - actual_commitish_obj = self.local_repo.get_commitish( |
4926 | - actual_commitish |
4927 | - ) |
4928 | - if check_commit_id: |
4929 | - expected_commit_id = str(expected_commitish_obj.id) |
4930 | - actual_commit_id = str(actual_commitish_obj.id) |
4931 | - if expected_commit_id == actual_commit_id: |
4932 | - self.success("Verified %s is the same commit as %s", |
4933 | - expected_commitish_pretty_name, actual_commitish |
4934 | - ) |
4935 | - else: |
4936 | - warning("Expected %s (%s) is not the same commit as %s (%s)", |
4937 | - expected_commitish_pretty_name, expected_commit_id, |
4938 | - actual_commitish, actual_commit_id |
4939 | - ) |
4940 | - ret = False |
4941 | - expected_commit_tree_id = str( |
4942 | - expected_commitish_obj.peel(pygit2.Tree).id |
4943 | - ) |
4944 | - actual_commit_tree_id = str( |
4945 | - actual_commitish_obj.peel(pygit2.Tree).id |
4946 | - ) |
4947 | - if expected_commit_tree_id == actual_commit_tree_id: |
4948 | - self.success("Verified %s is the same tree as %s", |
4949 | + try: |
4950 | + expected_commitish_obj = repo.get_commitish(expected_commitish) |
4951 | + except KeyError: |
4952 | + fatal("Unable to find expected commit: %s", expected_commitish) |
4953 | + |
4954 | + actual_commitish_obj = repo.get_commitish(actual_commitish) |
4955 | + if check_commit_id: |
4956 | + expected_commit_id = str(expected_commitish_obj.id) |
4957 | + actual_commit_id = str(actual_commitish_obj.id) |
4958 | + if expected_commit_id == actual_commit_id: |
4959 | + success("Verified %s is the same commit as %s", |
4960 | expected_commitish_pretty_name, actual_commitish |
4961 | ) |
4962 | else: |
4963 | - error("Expected %s (%s) is not the same tree as %s (%s)", |
4964 | - expected_commitish_pretty_name, expected_commit_tree_id, |
4965 | - actual_commitish, actual_commit_tree_id |
4966 | + warning("Expected %s (%s) is not the same commit as %s (%s)", |
4967 | + expected_commitish_pretty_name, expected_commit_id, |
4968 | + actual_commitish, actual_commit_id |
4969 | ) |
4970 | ret = False |
4971 | + expected_commit_tree_id = str( |
4972 | + expected_commitish_obj.peel(pygit2.Tree).id |
4973 | + ) |
4974 | + actual_commit_tree_id = str( |
4975 | + actual_commitish_obj.peel(pygit2.Tree).id |
4976 | + ) |
4977 | + if expected_commit_tree_id == actual_commit_tree_id: |
4978 | + success("Verified %s is the same tree as %s", |
4979 | + expected_commitish_pretty_name, actual_commitish |
4980 | + ) |
4981 | + else: |
4982 | + error("Expected %s (%s) is not the same tree as %s (%s)", |
4983 | + expected_commitish_pretty_name, expected_commit_tree_id, |
4984 | + actual_commitish, actual_commit_tree_id |
4985 | + ) |
4986 | + ret = False |
4987 | |
4988 | - return ret |
4989 | + return ret |
4990 | |
4991 | - def print_hunk(self, hunk): |
4992 | - for line in hunk.lines: |
4993 | - print("%s%s" % (line.origin, line.content.rstrip())) |
4994 | +def print_hunk(hunk): |
4995 | + for line in hunk.lines: |
4996 | + print("%s%s" % (line.origin, line.content.rstrip())) |
4997 | |
4998 | - def print_patch(self, patch): |
4999 | - for hunk in patch.hunks: |
5000 | - self.print_hunk(hunk) |