Merge lp:~cjwatson/ubuntu-dev-tools/syncpackage-lp into lp:~ubuntu-dev/ubuntu-dev-tools/trunk

[Colin Watson]

Native source syncing in Launchpad is finally ready! This branch adds code to syncpackage to use it, keeping the old mode of operation available behind the --no-lp option.

I'd like to get this into the Ubuntu archive before we announce the facility for general use.

[lp:~cjwatson/ubuntu-dev-tools/syncpackage-lp]

1124. By Colin Watson on 2011-08-16

merge trunk

[Iain Lane (laney)]

Woo!

Warts:

  * laney@raleigh> syncpackage -d experimental mono
    syncpackage: Source mono -> oneiric/Release: current version 2.10.3-1, new version 2.6.7-5

    (should find 2.10.4-2; worked before, works now with -V)

  * Typo in the manpage: "The use of syncpackage --no--lp"

  * Perhaps copy() should move to ubuntutools; lp_object breaks encapsulation. It could then be used by ack-sync (if the interface is suitable for sponsoring use) and requestsync --lp.

review: Needs Fixing

[Stefano Rivera (stefanor)]

Should we make --no-lp spit out a warning when run?

ack-sync is supposed to go away, and be replaced by a similar feature in sponsor-patch (which will have to test build, by doing a syncpackage --no-lp). Considering that's it's just one simple API call, I wouldn't worry about using the same function in both.

[lp:~cjwatson/ubuntu-dev-tools/syncpackage-lp]

1125. By Colin Watson on 2011-08-16

typo

[Iain Lane (laney)]

It is just one call, but so is other stuff in there. I think it's better to preserve encapsulation (and provide a nicer interface than raw API calls). I don't want to hold this MP up on that though; it can be done when refactoring the other tools.

[lp:~cjwatson/ubuntu-dev-tools/syncpackage-lp]

1126. By Colin Watson on 2011-08-16

move copyPackage into lpapicache for better encapsulation

1127. By Colin Watson on 2011-08-16

handle -d in Launchpad copy mode

[Colin Watson (cjwatson)]

Iain, thanks for your suggestions; I believe I've fixed all of these now.

Stefano, I think eventually we probably do want to deprecate --no-lp rather more completely, but for now it's still useful for things like uploading to PPAs (which copy() could eventually support, I think, but doesn't yet) and for fakesyncs when upstream tarballs differ. For now, I think that having to type an extra option to invoke it and the fact that it's slower and more effort (it has to download the package and then you have to upload the result manually) should be enough discouragement.

[Benjamin Drung (bdrung)]

1) You could use ubuntutools.question to ask your question.

2) syncpackage --lp doesn't seem to support sponsoring syncs or how is this supposed to work?

3) Is it possible to have online tests for syncpackage to test the new behaviour and for checking that there is no regression?

4) What happens if the source tarballs differ with --lp?

[lp:~cjwatson/ubuntu-dev-tools/syncpackage-lp]

1128. By Colin Watson on 2011-08-16

use ubuntutools.question

1129. By Colin Watson on 2011-08-16

syncpackage: Require -f/--force option to overwrite Ubuntu changes.

[Colin Watson (cjwatson)]

On Tue, Aug 16, 2011 at 05:55:50PM -0000, Benjamin Drung wrote:
> 1) You could use ubuntutools.question to ask your question.

I didn't know about that, thanks. Done.

> 2) syncpackage --lp doesn't seem to support sponsoring syncs or how is this supposed to work?

I've filed https://bugs.launchpad.net/launchpad/+bug/827555 for this.

> 3) Is it possible to have online tests for syncpackage to test the new behaviour and for checking that there is no regression?

You can use '-l dogfood' or '-l staging' (with varying degrees of
current-data-ness and likeliness-to-work).

> 4) What happens if the source tarballs differ with --lp?

I believe that you will get an error by e-mail when Launchpad tries to
process the copy job, and you'll then need to go back and use --no-lp.
I'd prefer to avoid having to download the original tarballs, and this
is enough of a corner case that I think we can perhaps not worry too
much about it for the time being.

Also, at Steve Langasek's suggestion, I pushed a further change that
requires an -f/--force option to discard Ubuntu changes. This matches
the current (archive admin only) sync-source.py script, and is because
otherwise it's too easy for accidents to happen.

[Benjamin Drung (bdrung)]

2) Thanks for filing the bug. Can you update the documentation and warn the user if they want to change the author?

3) It would be nice if you could provide test cases. This will give you bonus points. Stefano and I can help you putting them into our test framework.

4) Downloading the original tarballs makes no sense. It would be nice if LP would provide the hash sums. Until then I can live with the current behavior.

[Stefano Rivera (stefanor)]

Benjamin: I think test cases would be reasonable for non-lp-syncpackage, but not much more than that.

You don't have to download original tarballs, the .dsc files can give you the hash sums. ubuntutools.archive knows how to do most of this. That's probably out of scope for this merge proposal, though (assuming lp will reject a sync where fakesyncing is required)

[lp:~cjwatson/ubuntu-dev-tools/syncpackage-lp]

1130. By Colin Watson on 2011-08-16

validate that the provided Debian version exists

1131. By Colin Watson on 2011-08-16

document restriction of -n and -e to --no-lp mode

[Colin Watson (cjwatson)]

On Tue, Aug 16, 2011 at 08:06:22PM -0000, Benjamin Drung wrote:
> 2) Thanks for filing the bug. Can you update the documentation and warn the user if they want to change the author?

My branch already disallows using --uploader-name or --uploader-email
without --no-lp. I've updated the manual page now to match.

> 3) It would be nice if you could provide test cases. This will give you bonus points. Stefano and I can help you putting them into our test framework.

While I agree with the sentiment, I have no idea how you would go about
testing this without a mock version of Launchpad where you could control
the available test data, or at least perhaps a mock version of
launchpadlib. There seem to be no lpapicache tests in ubuntu-dev-tools
right now.

> 4) Downloading the original tarballs makes no sense. It would be nice if LP would provide the hash sums. Until then I can live with the current behavior.

I'm working on rearranging the code to make better use of
ubuntutools.archive facilities as Stefano suggested, at which point this
should be straightforward; I wasn't really aware of that module and I
definitely have some duplicated code here anyway. This will probably
take me until tomorrow.

[lp:~cjwatson/ubuntu-dev-tools/syncpackage-lp]

1132. By Colin Watson on 2011-08-17

make it possible to quieten download progress information, particularly useful if downloading only the small .dsc files

1133. By Colin Watson on 2011-08-17

make better use of ubuntutools.archive, allowing us to detect when fakesyncs are required in LP mode

[Colin Watson (cjwatson)]

I believe I've got fakesync detection in LP mode working now.

  $ ./syncpackage -l staging -f nautilus-image-converter
  syncpackage: Source nautilus-image-converter -> oneiric/Release: current version 0.3.0-3ubuntu2, new version 0.3.0-3.1
  syncpackage: Error: The checksums of the Debian and Ubuntu packages mismatch. A fake sync is required.

[Stefano Rivera (stefanor)]

src_pkg.lp_spph is a property, not a method (I'm coming to think that those @properties may have been poor API design :) )

[lp:~cjwatson/ubuntu-dev-tools/syncpackage-lp]

1134. By Colin Watson on 2011-08-17

DebianSourcePackage.lp_spph is annoyingly overloaded; just do the check directly

[Colin Watson (cjwatson)]

On Wed, Aug 17, 2011 at 08:10:24AM -0000, Stefano Rivera wrote:
> src_pkg.lp_spph is a property, not a method (I'm coming to think that those @properties may have been poor API design :) )

Good point. It's annoyingly overloaded anyway, as I really just want to
specifically ask LP rather than anything else, so I've changed it to
just call getPublishedSources directly again.

[lp:~cjwatson/ubuntu-dev-tools/syncpackage-lp]

1135. By Colin Watson on 2011-08-17

use version.full_version rather than str(version)

1136. By Colin Watson on 2011-08-17

refer to syncpackage from requestsync(1) for users with upload permission

1137. By Colin Watson on 2011-08-17

add note on requestsync stderr about syncpackage

[Stefano Rivera (stefanor)]

review: Approve

[Iain Lane (laney)]

review: Approve

[Benjamin Drung (bdrung)]

When you detect that a fakesync is needed, can you either tell the user to use syncpackage with --no-lp or just do the fakesync and tell the user how to proceed?

[Colin Watson (cjwatson)]

For now, I've made an adjustment to mention --no-lp in the error
message.

[lp:~cjwatson/ubuntu-dev-tools/syncpackage-lp]

1138. By Colin Watson on 2011-08-17

fake syncs require --no-lp

[Benjamin Drung (bdrung)]

Should we add "source" to the syncpackage description? -> "copy source packages from Debian to Ubuntu"

The copy function could be refactored. pylint says:
W:313:copy: Redefining name 'debian' from outer scope (line 23)
C:312:copy: Missing docstring
R:312:copy: Too many local variables (24/15)
W:312:copy: Unused argument 'debian_dist'
W:325:copy: Unused variable 'debian_spph'
R:312:copy: Too many branches (13/12)

I don't like the "if not quiet: Logger.normal(...)". This parameter should be set in the Logger instead.

There are too many spaces in "getUbuntuSrcPkg(name, release, pocket = 'Release'):" it should be "getUbuntuSrcPkg(name, release, pocket='Release'):"

Besides these small this it looks ready.

[lp:~cjwatson/ubuntu-dev-tools/syncpackage-lp]

1139. By Colin Watson on 2011-08-18

avoid shadowing global "debian" name

1140. By Colin Watson on 2011-08-18

drop unused argument

1141. By Colin Watson on 2011-08-18

drop unused variable

1142. By Colin Watson on 2011-08-18

factor out dsc comparison into a method on ubuntutools.archive.Dsc

1143. By Colin Watson on 2011-08-18

docstring for copy()

1144. By Colin Watson on 2011-08-18

tweak to manual page suggested by bdrung

1145. By Colin Watson on 2011-08-18

drop quiet download stuff, not worth an API debate

1146. By Colin Watson on 2011-08-18

style

[Colin Watson (cjwatson)]

On Wed, Aug 17, 2011 at 04:59:23PM -0000, Benjamin Drung wrote:
> Should we add "source" to the syncpackage description? -> "copy source
> packages from Debian to Ubuntu"

Done.

> The copy function could be refactored. pylint says:
> W:313:copy: Redefining name 'debian' from outer scope (line 23)
> C:312:copy: Missing docstring
> R:312:copy: Too many local variables (24/15)
> W:312:copy: Unused argument 'debian_dist'
> W:325:copy: Unused variable 'debian_spph'
> R:312:copy: Too many branches (13/12)

Done, mainly by moving the .dsc comparison out to a method on the Dsc
class which was probably a good idea anyway. (Though note that pylint
still has a bunch of output for stuff that was there before my changes.)

> I don't like the "if not quiet: Logger.normal(...)". This parameter
> should be set in the Logger instead.

I've just dropped the quiet download stuff. It's not really worth it.

> There are too many spaces in "getUbuntuSrcPkg(name, release, pocket =
> 'Release'):" it should be "getUbuntuSrcPkg(name, release,
> pocket='Release'):"

Fixed, but note that the extra spaces weren't my usual style - I only
did it that way because I found other similar code in ubuntu-dev-tools
that was written that way! (I think I was looking at
ubuntutools.lp.lpapicache.Archive.getSourcePackage at the time.) Please
apply a consistent style across existing code in the whole project - it
really helps other developers figure out how they should be writing code
for submission.

Thanks,

--
Colin Watson [<email address hidden>]

[Benjamin Drung (bdrung)]

Patches for making ubuntu-dev-tools code more pylint compliant/more readable are welcome. :)

The scripts in ubuntu-dev-tools didn't had consistent styles. We tried to make the style consistent by using PEP-8, but some places probably needs more love. I fixed the spacings in ubuntutools.lp.lpapicache (pylint didn't complained). Feel free to fix those style inconsistencies if you stumble over more of them.

Thanks for your work. I merged the branch. I still love to see test cases for the new (and old) syncpackage code.

review: Approve

=== modified file 'debian/changelog'
--- debian/changelog	2011-08-16 09:19:15 +0000
+++ debian/changelog	2011-08-18 14:13:55 +0000
@@ -1,3 +1,11 @@
+ubuntu-dev-tools (0.129) UNRELEASED; urgency=low
+
+  * syncpackage: Convert to new LP API, with --no-lp available for the old
+    style of operation.
+  * syncpackage: Require -f/--force option to overwrite Ubuntu changes.
+
+ -- Colin Watson <[e-mail]>  Tue, 16 Aug 2011 16:40:22 +0100
+
 ubuntu-dev-tools (0.128) unstable; urgency=low
 
   [ Stefano Rivera ]

=== modified file 'doc/requestsync.1'
--- doc/requestsync.1	2011-02-24 14:30:45 +0000
+++ doc/requestsync.1	2011-08-18 14:13:55 +0000
@@ -32,6 +32,11 @@
 answer the question about upload permissions honestly to determine if a team
 with approval rights is to be subscribed to the bug.
 
+If you have permission to upload the package directly, then you may prefer
+to use \fBsyncpackage\fR instead to copy the package using the Launchpad
+API. At some future point, \fBrequestsync\fR will be changed to do this
+automatically.
+
 .PP
 \fBrequestsync\fR uses launchpadlib authentication to file its requests.
 

=== modified file 'doc/syncpackage.1'
--- doc/syncpackage.1	2010-12-31 10:37:52 +0000
+++ doc/syncpackage.1	2011-08-18 14:13:55 +0000
@@ -1,12 +1,12 @@
 .TH SYNCPACKAGE "1" "June 2010" "ubuntu-dev-tools"
 .SH NAME
-syncpackage \- helper to prepare .changes file to upload synced packages
+syncpackage \- copy source packages from Debian to Ubuntu
 .SH SYNOPSIS
 .B syncpackage
 [\fIoptions\fR] \fI<.dsc URL/path or package name>\fR
 .SH DESCRIPTION
-\fBsyncpackage\fR generates a changes file to be directly uploaded to Ubuntu
-primary archive or PPA starting from a pristine Debian package.
+\fBsyncpackage\fR causes a source package to be copied from Debian to
+Ubuntu.
 .PP
 \fBsyncpackage\fR allows you to upload files with the same checksums of the
 Debian ones, as the common script used by Ubuntu archive administrators does,
@@ -15,14 +15,15 @@
 \fBsyncpackage\fR will detect source tarballs with mismatching checksums
 and will automatically create fake syncs instead.
 .SH WARNING
-The use of \fBsyncpackage\fR is discouraged by the Ubuntu Archive
-Administrators, as it introduces an unnecessary window for error.
-In future launchpad will offer the ability to directly perform sync
-requests, without a developer having to do an upload.
-Until then, it's advised to use \fBsyncpackage\fR with caution, and
-request syncs via bugs with
-.BR requestsync (1)
-where possible.
+The use of \fBsyncpackage \-\-no\-lp\fR, which generates a changes file to
+be directly uploaded to the Ubuntu primary archive or a PPA, is discouraged
+by the Ubuntu Archive Administrators, as it introduces an unnecessary window
+for error.
+This only exists for backward compatibility, for unusual corner cases, and
+for uploads to archives other than the Ubuntu primary archive.
+Omitting this option will cause Launchpad to perform the sync request
+directly, which is the preferred method for uploads to the Ubuntu primary
+archive.
 .SH OPTIONS
 .TP
 \fB\-h\fR, \fB\-\-help\fR
@@ -43,13 +44,22 @@
 \fB\-v\fR, \fB\-\-verbose\fR
 Display more progress information.
 .TP
+.B \-\-no\-lp
+Construct sync locally rather than letting Launchpad copy the package
+directly (not recommended).
+.TP
+\fB\-l\fI INSTANCE\fR, \fB\-\-lpinstance\fR=\fIINSTANCE\fR
+Launchpad instance to connect to (default: production).
+.TP
 \fB\-n\fI UPLOADER_NAME\fR, \fB\-\-uploader\-name\fR=\fIUPLOADER_NAME\fR
 Use UPLOADER_NAME as the name of the maintainer for this upload instead
 of evaluating DEBFULLNAME and UBUMAIL.
+This option may only be used in \fB\-\-no\-lp\fR mode.
 .TP
 \fB\-e\fI UPLOADER_EMAIL\fR, \fB\-\-uploader\-email\fR=\fIUPLOADER_EMAIL\fR
 Use UPLOADER_EMAIL as the email address of the maintainer for this
 upload instead of evaluating DEBEMAIL and UBUMAIL.
+This option may only be used in \fB\-\-no\-lp\fR mode.
 .TP
 \fB\-k\fI KEYID\fR, \fB\-\-key\fR=\fIKEYID\fR
 Specify the key ID to be used for signing.
@@ -60,6 +70,9 @@
 \fB\-b\fI BUG\fR, \fB\-\-bug\fR=\fIBUG\fR
 Mark a Launchpad bug as being fixed by this upload.
 .TP
+\fB\-f\fR, \fB\-\-force\fR
+Force sync over the top of Ubuntu changes.
+.TP
 .B \-d \fIDEBIAN_MIRROR\fR, \fB\-\-debian\-mirror\fR=\fIDEBIAN_MIRROR\fR
 Use the specified mirror.
 Should be in the form \fBhttp://ftp.debian.org/debian\fR.
@@ -75,6 +88,9 @@
 .B \-\-no\-conf
 Do not read any configuration files, or configuration from environment
 variables.
+.TP
+.B \-\-simulate
+Show what would be done, but don't actually do it.
 .SH ENVIRONMENT
 .TP
 .BR DEBFULLNAME ", " DEBEMAIL ", " UBUMAIL

=== modified file 'requestsync'
--- requestsync	2011-06-25 15:53:44 +0000
+++ requestsync	2011-08-18 14:13:55 +0000
@@ -239,6 +239,10 @@
     if not sponsorship:
         sponsorship = needSponsorship(srcpkg, ubuntu_component, release)
 
+    if not sponsorship and not ffe:
+        print >> sys.stderr, ('Consider using syncpackage(1) for syncs that '
+                              'do not require feature freeze exceptions.')
+
     # Check for existing package reports
     if not newsource:
         checkExistingReports(srcpkg)

=== modified file 'syncpackage'
--- syncpackage	2011-05-24 18:22:37 +0000
+++ syncpackage	2011-08-18 14:13:55 +0000
@@ -29,6 +29,10 @@
 
 from devscripts.logger import Logger
 
+from distro_info import DebianDistroInfo
+
+from lazr.restfulclient.errors import HTTPError
+
 from ubuntutools.archive import (DebianSourcePackage, UbuntuSourcePackage,
                                  DownloadError)
 from ubuntutools.config import UDTConfig, ubu_email
@@ -36,7 +40,9 @@
                                           as requestsync_mail_getDebianSrcPkg)
 from ubuntutools.requestsync.lp import getDebianSrcPkg, getUbuntuSrcPkg
 from ubuntutools.lp import udtexceptions
-from ubuntutools.lp.lpapicache import Launchpad
+from ubuntutools.lp.lpapicache import Distribution, Launchpad
+from ubuntutools.misc import split_release_pocket
+from ubuntutools.question import YesNoQuestion
 from ubuntutools import subprocess
 
 
@@ -110,7 +116,7 @@
     return "\n".join(changes + [""])
 
 def sync_dsc(src_pkg, debian_dist, release, name, email, bugs, ubuntu_mirror,
-             keyid=None):
+             keyid=None, simulate=False, force=False):
     uploader = name + " <" + email + ">"
 
     src_pkg.pull_dsc()
@@ -136,9 +142,15 @@
 
     cur_ver = ubuntu_ver.get_related_debian_version()
     if ubuntu_ver.is_modified_in_ubuntu():
+        if not force:
+            Logger.error('--force is required to discard Ubuntu changes.')
+            sys.exit(1)
+
         Logger.warn('Overwriting modified Ubuntu version %s, '
                     'setting current version to %s',
                     ubuntu_ver.full_version, cur_ver.full_version)
+    if simulate:
+        return
 
     try:
         src_pkg.pull()
@@ -248,8 +260,13 @@
     """Download the specified source package.
     dist, version, component, mirror can all be None.
     """
+    if mirror is None:
+        mirrors = []
+    else:
+        mirrors = [mirror]
+
     if package.endswith('.dsc'):
-        return DebianSourcePackage(dscfile=package, mirrors=[mirror])
+        return DebianSourcePackage(dscfile=package, mirrors=mirrors)
 
     if dist is None:
         dist = "unstable"
@@ -290,7 +307,80 @@
     assert component in ('main', 'contrib', 'non-free')
 
     return DebianSourcePackage(package, version.full_version, component,
-                               mirrors=[mirror])
+                               mirrors=mirrors)
+
+def copy(src_pkg, release, simulate=False, force=False):
+    """Copy a source package from Debian to Ubuntu using the Launchpad API."""
+    ubuntu = Distribution('ubuntu')
+    debian_archive = Distribution('debian').getArchive()
+    ubuntu_archive = ubuntu.getArchive()
+    if release is None:
+        ubuntu_series = ubuntu.getDevelopmentSeries().name
+        ubuntu_pocket = 'Release'
+    else:
+        ubuntu_series, ubuntu_pocket = split_release_pocket(release)
+
+    # Ensure that the provided Debian version actually exists.
+    try:
+        debian_archive.getPublishedSources(
+            source_name=src_pkg.source,
+            version=src_pkg.version.full_version,
+            exact_match=True)[0]
+    except IndexError:
+        Logger.error('Debian version %s does not exist!', src_pkg.version)
+        sys.exit(1)
+
+    try:
+        ubuntu_spph = getUbuntuSrcPkg(src_pkg.source,
+                                      ubuntu_series, ubuntu_pocket)
+        ubuntu_pkg = UbuntuSourcePackage(src_pkg.source,
+                                         ubuntu_spph.getVersion(),
+                                         ubuntu_spph.getComponent(),
+                                         mirrors=[])
+
+        Logger.normal('Source %s -> %s/%s: current version %s, new version %s',
+                      src_pkg.source, ubuntu_series, ubuntu_pocket,
+                      ubuntu_pkg.version, src_pkg.version)
+
+        ubuntu_version = Version(ubuntu_pkg.version.full_version)
+        if not force and ubuntu_version.is_modified_in_ubuntu():
+            Logger.error('--force is required to discard Ubuntu changes.')
+            sys.exit(1)
+
+        # Check whether a fakesync would be required.
+        src_pkg.pull_dsc()
+        ubuntu_pkg.pull_dsc()
+        if not src_pkg.dsc.compare_dsc(ubuntu_pkg.dsc):
+            Logger.error('The checksums of the Debian and Ubuntu packages '
+                         'mismatch. A fake sync using --no-lp is required.')
+            sys.exit(1)
+    except udtexceptions.PackageNotFoundException:
+        Logger.normal('Source %s -> %s/%s: not in Ubuntu, new version %s',
+                      src_pkg.source, ubuntu_series, ubuntu_pocket,
+                      src_pkg.version)
+    if simulate:
+        return
+
+    answer = YesNoQuestion().ask("Sync this package", "no")
+    if answer != "yes":
+        return
+
+    try:
+        ubuntu_archive.copyPackage(
+            source_name=src_pkg.source,
+            version=src_pkg.version.full_version,
+            from_archive=debian_archive,
+            to_series=ubuntu_series,
+            to_pocket=ubuntu_pocket,
+            include_binaries=False)
+    except HTTPError, error:
+        Logger.error("HTTP Error %s: %s" %
+                     (error.response.status, error.response.reason))
+        Logger.error(error.content)
+        sys.exit(1)
+
+    Logger.normal('Request succeeded; you should get an e-mail once it is '
+                  'processed.')
 
 def main():
     usage = "%prog [options] <.dsc URL/path or package name>"
@@ -312,6 +402,15 @@
     parser.add_option("-v", "--verbose",
                       dest="verbose", action="store_true", default=False,
                       help="Display more progress information.")
+    parser.add_option("--no-lp",
+                      dest="lp", action="store_false", default=True,
+                      help="Construct sync locally rather than letting "
+                           "Launchpad copy the package directly (not "
+                           "recommended).")
+    parser.add_option('-l', '--lpinstance', metavar='INSTANCE',
+                      dest='lpinstance', default=None,
+                      help='Launchpad instance to connect to '
+                           '(default: production).')
     parser.add_option("-n", "--uploader-name",
                       dest="uploader_name", default=None,
                       help="Use UPLOADER_NAME as the name of the maintainer "
@@ -330,6 +429,9 @@
                       dest="bugs", action="append", default=list(),
                       help="Mark Launchpad bug BUG as being fixed by this "
                            "upload.")
+    parser.add_option("-f", "--force",
+                      dest="force", action="store_true", default=False,
+                      help="Force sync over the top of Ubuntu changes.")
     parser.add_option('-D', '--debian-mirror', metavar='DEBIAN_MIRROR',
                       dest='debian_mirror',
                       help='Preferred Debian mirror '
@@ -343,6 +445,10 @@
     parser.add_option('--no-conf',
                       dest='no_conf', default=False, action='store_true',
                       help="Don't read config files or environment variables.")
+    parser.add_option('--simulate',
+                      dest='simulate', default=False, action='store_true',
+                      help="Show what would be done, but don't actually do "
+                           "it.")
 
     (options, args) = parser.parse_args()
 
@@ -362,6 +468,14 @@
                      'It should be one of main, contrib, or non-free.'
                      % options.component)
 
+    if options.lp and options.uploader_name:
+        parser.error('Uploader name can only be overridden using --no-lp.')
+    if options.lp and options.uploader_email:
+        parser.error('Uploader email address can only be overridden using '
+                     '--no-lp.')
+    # --key, --dont-sign, --debian-mirror, and --ubuntu-mirror are just
+    # ignored with options.lp, and do not require warnings.
+
     Logger.verbose = options.verbose
     config = UDTConfig(options.no_conf)
     if options.debian_mirror is None:
@@ -373,19 +487,37 @@
     if options.uploader_email is None:
         options.uploader_email = ubu_email(export=False)[1]
 
-    Launchpad.login_anonymously()
-    if options.release is None:
-        options.release = Launchpad.distributions["ubuntu"].current_series.name
-
-    os.environ['DEB_VENDOR'] = 'Ubuntu'
-
-    src_pkg = fetch_source_pkg(args[0], options.dist, options.debversion,
-                               options.component, options.release,
-                               options.debian_mirror)
-
-    sync_dsc(src_pkg, options.dist, options.release, options.uploader_name,
-             options.uploader_email, options.bugs, options.ubuntu_mirror,
-             options.keyid)
+    if options.lp:
+        if args[0].endswith('.dsc'):
+            parser.error('.dsc files can only be synced using --no-lp.')
+
+        if options.lpinstance is None:
+            options.lpinstance = config.get_value('LPINSTANCE')
+
+        try:
+            Launchpad.login(service=options.lpinstance, api_version='devel')
+        except IOError:
+            sys.exit(1)
+
+        src_pkg = fetch_source_pkg(args[0], options.dist, options.debversion,
+                                   options.component, options.release, None)
+
+        copy(src_pkg, options.release, options.simulate, options.force)
+    else:
+        Launchpad.login_anonymously()
+        if options.release is None:
+            ubuntu = Launchpad.distributions["ubuntu"]
+            options.release = ubuntu.current_series.name
+
+        os.environ['DEB_VENDOR'] = 'Ubuntu'
+
+        src_pkg = fetch_source_pkg(args[0], options.dist, options.debversion,
+                                   options.component, options.release,
+                                   options.debian_mirror)
+
+        sync_dsc(src_pkg, options.dist, options.release, options.uploader_name,
+                 options.uploader_email, options.bugs, options.ubuntu_mirror,
+                 options.keyid, options.simulate, options.force)
 
 if __name__ == "__main__":
     main()

=== modified file 'ubuntutools/archive.py'
--- ubuntutools/archive.py	2011-08-15 23:17:25 +0000
+++ ubuntutools/archive.py	2011-08-18 14:13:55 +0000
@@ -95,6 +95,30 @@
             return hash_func.hexdigest() == digest
         return False
 
+    def compare_dsc(self, other):
+        """Check whether any files in these two dscs that have the same name
+        also have the same checksum."""
+        for field, key in (('Checksums-Sha256', 'sha256'),
+                           ('Checksums-Sha1', 'sha1'),
+                           ('Files', 'md5sum')):
+            if field not in self or field not in other:
+                continue
+            our_checksums = \
+                dict((entry['name'], (int(entry['size']), entry[key]))
+                     for entry in self[field])
+            their_checksums = \
+                dict((entry['name'], (int(entry['size']), entry[key]))
+                     for entry in other[field])
+            for name, (size, checksum) in our_checksums.iteritems():
+                if name not in their_checksums:
+                    # file only in one dsc
+                    continue
+                if (size != their_checksums[name][0] or
+                    checksum != their_checksums[name][1]):
+                    return False
+            return True # one checksum is good enough
+        return True
+
 
 class SourcePackage(object):
     """Base class for source package downloading.

=== modified file 'ubuntutools/lp/lpapicache.py'
--- ubuntutools/lp/lpapicache.py	2011-02-28 22:30:54 +0000
+++ ubuntutools/lp/lpapicache.py	2011-08-18 14:13:55 +0000
@@ -53,7 +53,7 @@
 class _Launchpad(object):
     '''Singleton for LP API access.'''
 
-    def login(self, service=service):
+    def login(self, service=service, api_version=api_version):
         '''Enforce a non-anonymous login.'''
         if not self.logged_in:
             try:
@@ -332,6 +332,25 @@
 
         return self._srcpkgs[(name, series.name, pocket)]
 
+    def copyPackage(self, source_name, version, from_archive, to_pocket,
+                    to_series = None, include_binaries = False):
+        '''Copy a single named source into this archive.
+
+        Asynchronously copy a specific version of a named source to the
+        destination archive if necessary.  Calls to this method will return
+        immediately if the copy passes basic security checks and the copy
+        will happen sometime later with full checking.
+        '''
+
+        self._lpobject.copyPackage(
+            source_name=source_name,
+            version=version,
+            from_archive=from_archive._lpobject,
+            to_pocket=to_pocket,
+            to_series=to_series,
+            include_binaries=include_binaries
+            )
+
 
 class SourcePackagePublishingHistory(BaseWrapper):
     '''

=== modified file 'ubuntutools/requestsync/lp.py'
--- ubuntutools/requestsync/lp.py	2011-06-25 15:53:44 +0000
+++ ubuntutools/requestsync/lp.py	2011-08-18 14:13:55 +0000
@@ -34,11 +34,11 @@
 
     return debian_archive.getSourcePackage(name, release)
 
-def getUbuntuSrcPkg(name, release):
+def getUbuntuSrcPkg(name, release, pocket='Release'):
     ubuntu = Distribution('ubuntu')
     ubuntu_archive = ubuntu.getArchive()
 
-    return ubuntu_archive.getSourcePackage(name, release)
+    return ubuntu_archive.getSourcePackage(name, release, pocket)
 
 def needSponsorship(name, component, release):
     '''