Merge ~ddstreet/ubuntu-dev-tools/+git/ubuntu-dev-tools:pull-pkg into ubuntu-dev-tools:master

[Dan Streetman]

This is a large change to the pull-DISTRO-source scripts, and their associated libs under ubuntutools/. It consolidates the scripts into a single script, and extends its functionality to be able to pull not only source packages/files, but also binary debs - normal ".deb", dbgsym ".ddeb" and -di ".udeb" files. It also allows listing all packages assoicated with a given source package, and it automatically performs binary package name to source package name lookups. It works with ubuntu (launchpad), as well as debian (using madison/snapshot) and cloud-archive.

a313e50... by Dan Streetman on 2017-10-27

pull-pkg: catch any exception when trying to lookup bpph->build->spph

When looking up a package by its binary pkg name, the easiest way to get
to its SourcePackagePublishingHistory object is through its Build object.
However, some BPPH objects in Launchpad seem to have their Build's
SPPH link removed or redacted, which results in an exception while trying
to access it. So try that first, but catch any exception and then
fallback to recursively calling lp_spph using the looked-up source
package name and version number, which will work.

24bf863... by Dan Streetman on 2017-11-21

ubuntutools/lp/lpapicache.py method additions

DistroArchSeries.getSeries()
SPPH.getSeriesName()
SPPH.getArchive()

02b0d8b... by Dan Streetman on 2017-11-21

ubuntutools/lp/lpapicache.py: get pkg using version/status params also

Update Archive methods getSourcePackage and getBinaryPackage to allow
searching for a package using its version and/or status fields also

4578c36... by Dan Streetman on 2017-11-21

ubuntutools/lp/lpapicache.py: find old pkg version binaries

Launchpad unfortunately "hides" the BPPH records (and corresponding
binary files/urls) when a newer package version is released. This
means binary debs and ddebs for older package versions are effectively
"gone", which makes debugging problems with older package versions
(e.g. with gdb) very hard.

Luckily (and confusingly) Launchpad doesn't actually remove the old
binary files, or the BPPH records. It only *hides* them, because
the SPPH.getPublishedBinaries() LP API call only returns Published
or Pending BPPH records.

This change detects that (if the state is not Published or Pending)
and instead of using SPPH.getPublishedBinaries() goes the MUCH
longer route (since it seems to be the ONLY way left to find the
binaries) of using SPPH.binaryFileUrls() and then parsing the
filenames and looking up each BPPH using Archive.getPublishedBinaries().

This change allows pull-lp-debs and pull-lp-ddebs to now get debs
and/or ddebs for ALL package versions created in Launchpad, even if
they are old versions.

2c6023c... by Dan Streetman on 2017-11-21

ubuntutools/lp/lpapicache.py: get system arch for archtag 'all'

Packages applicable to all archs are labeled with archtag 'all',
so any reverse lookup of a DistroArchSeries by arch 'all'
should return the system's arch series (since the specific arch
doesn't matter for archtag 'all' binaries).

Unmerged commits

4578c36... by Dan Streetman on 2017-11-21

ubuntutools/lp/lpapicache.py: find old pkg version binaries

Launchpad unfortunately "hides" the BPPH records (and corresponding
binary files/urls) when a newer package version is released. This
means binary debs and ddebs for older package versions are effectively
"gone", which makes debugging problems with older package versions
(e.g. with gdb) very hard.

Luckily (and confusingly) Launchpad doesn't actually remove the old
binary files, or the BPPH records. It only *hides* them, because
the SPPH.getPublishedBinaries() LP API call only returns Published
or Pending BPPH records.

This change detects that (if the state is not Published or Pending)
and instead of using SPPH.getPublishedBinaries() goes the MUCH
longer route (since it seems to be the ONLY way left to find the
binaries) of using SPPH.binaryFileUrls() and then parsing the
filenames and looking up each BPPH using Archive.getPublishedBinaries().

This change allows pull-lp-debs and pull-lp-ddebs to now get debs
and/or ddebs for ALL package versions created in Launchpad, even if
they are old versions.

02b0d8b... by Dan Streetman on 2017-11-21

ubuntutools/lp/lpapicache.py: get pkg using version/status params also

Update Archive methods getSourcePackage and getBinaryPackage to allow
searching for a package using its version and/or status fields also

24bf863... by Dan Streetman on 2017-11-21

ubuntutools/lp/lpapicache.py method additions

DistroArchSeries.getSeries()
SPPH.getSeriesName()
SPPH.getArchive()

2c6023c... by Dan Streetman on 2017-11-21

ubuntutools/lp/lpapicache.py: get system arch for archtag 'all'

Packages applicable to all archs are labeled with archtag 'all',
so any reverse lookup of a DistroArchSeries by arch 'all'
should return the system's arch series (since the specific arch
doesn't matter for archtag 'all' binaries).

a313e50... by Dan Streetman on 2017-10-27

pull-pkg: catch any exception when trying to lookup bpph->build->spph

When looking up a package by its binary pkg name, the easiest way to get
to its SourcePackagePublishingHistory object is through its Build object.
However, some BPPH objects in Launchpad seem to have their Build's
SPPH link removed or redacted, which results in an exception while trying
to access it. So try that first, but catch any exception and then
fallback to recursively calling lp_spph using the looked-up source
package name and version number, which will work.

7057eb8... by Dan Streetman on 2017-09-15

pull-pkg: update man pages to all point to single pull-pkg.1 man page

remove the pull-lp-source.1 and pull-debian-source.1 man pages,
consolidating them both into a pull-pkg.1 man page. Also create
symlinks for all associated scripts pointing to pull-pkg.1 man page.

92f6bce... by Dan Streetman on 2017-09-14

pull-pkg: create pull-[lp|debian|uca]-* scripts

for backwards compatibility (i.e. to keep pull-lp-source,
pull-debian-source, and pull-uca-source) and for ease of use,
create scripts that default the pull-pkg -p and -D params

re-create pull-pkg script also, to call ubuntutools/pullpkg.py
without any default pull or distro parameters

0d9ca97... by Dan Streetman on 2017-09-15

pull-pkg: change pull-pkg script code into modular code

bbb62b2... by Dan Streetman on 2017-09-15

pull-pkg: rename pull-pkg to ubuntutools/pullpkg.py module

rename only with no content changes, so next commit shows code
changes converting script pull-pkg to module pullpkg.py

0b5736b... by Dan Streetman on 2017-09-14

pull-pkg: update to use previous SourcePackage improvements

New pull-pkg allows pulling source, debs, ddebs, or udebs, or just
listing all package files. Also, package lookup by binary name is done
automatically.

diff --git a/debian/control b/debian/control
index 19f2ba5..baf48a9 100644
--- a/debian/control
+++ b/debian/control
@@ -106,7 +106,13 @@ Description: useful tools for Ubuntu developers
     a Debian package and its immediate parent to generate a debdiff.
   - pull-debian-source - downloads the latest source package available in
     Debian of a package.
-  - pull-lp-source - downloads latest source package from Launchpad.
+  - pull-lp-source - downloads source package from Launchpad.
+  - pull-lp-debs - downloads debs package(s) from Launchpad.
+  - pull-lp-ddebs - downloads dbgsym/ddebs package(s) from Launchpad.
+  - pull-lp-udebs - downloads udebs package(s) from Launchpad.
+  - pull-debian-* - same as pull-lp-* but for Debian packages.
+  - pull-uca-* - same as pull-lp-* but for Ubuntu Cloud Archive packages.
+  - pull-pkg - common script that provides above pull-* functionality.
   - pull-revu-source - downloads the latest source package from REVU
   - requestbackport - file a backporting request.
   - requestsync - files a sync request with Debian changelog and rationale.
diff --git a/doc/pull-debian-ddebs.1 b/doc/pull-debian-ddebs.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-debian-ddebs.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/doc/pull-debian-debs.1 b/doc/pull-debian-debs.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-debian-debs.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/doc/pull-debian-source.1 b/doc/pull-debian-source.1
deleted file mode 100644
index 468724a..0000000
--- a/doc/pull-debian-source.1
+++ /dev/null
@@ -1,89 +0,0 @@
-.\" Copyright (C) 2010-2011, Stefano Rivera <[e-mail]>
-.\"
-.\" Permission to use, copy, modify, and/or distribute this software for any
-.\" purpose with or without fee is hereby granted, provided that the above
-.\" copyright notice and this permission notice appear in all copies.
-.\"
-.\" THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
-.\" REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
-.\" AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
-.\" INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
-.\" LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
-.\" OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
-.\" PERFORMANCE OF THIS SOFTWARE.
-.TH PULL\-DEBIAN\-SOURCE "1" "22 January 2011" "ubuntu\-dev\-tools"
-
-.SH NAME
-pull\-debian\-source \- download and extract a source package from Debian
-
-.SH SYNOPSIS
-.B pull\-debian\-source \fR[\fIoptions\fR] <\fIsource package\fR>
-[\fIrelease\fR|\fIversion\fR]
-
-.SH DESCRIPTION
-\fBpull\-debian\-source\fR downloads and extracts the specified
-\fIversion\fR of \fIsource package\fR, or the latest version in the
-specified Debian \fIrelease\fR.
-.P
-\fBpull\-debian\-source\fR will try the preferred mirror, default
-mirror, security mirror, and fall back to \fBLaunchpad\fR or
-\fBsnapshot.debian.org\fR, in search of the requested version.
-
-.SH OPTIONS
-.TP
-.I source package
-The source package to download from Debian.
-.TP
-.I release
-The release to download the source package from. Defaults to
-\fBunstable\fR.
-.TP
-.I version
-The specific version of the package to download.
-.TP
-.BR \-d ", " \-\-download\-only
-Do not extract the source package.
-.TP
-.B \-m \fIDEBIAN_MIRROR\fR, \fB\-\-mirror\fR=\fIDEBIAN_MIRROR\fR
-Use the specified mirror.
-Should be in the form \fBhttp://ftp.debian.org/debian\fR.
-If the package isn't found on this mirror, \fBpull\-debian\-source\fR
-will fall back to the default mirror.
-.TP
-.B \-s \fIDEBSEC_MIRROR\fR, \fB\-\-security\-mirror\fR=\fIDEBSEC_MIRROR\fR
-Use the specified mirror.
-Should be in the form \fBhttp://security.debian.org\fR.
-If the package isn't found on this mirror, \fBpull\-debian\-source\fR
-will fall back to the default mirror.
-.TP
-.B \-\-no\-conf
-Do not read any configuration files, or configuration from environment
-variables.
-.TP
-.BR \-h ", " \-\-help
-Display the usage instructions and exit.
-
-.SH ENVIRONMENT
-All of the \fBCONFIGURATION VARIABLES\fR below are also supported as
-environment variables.
-Variables in the environment take precedence to those in configuration
-files.
-
-.SH CONFIGURATION VARIABLES
-The following variables can be set in the environment or in
-.BR ubuntu\-dev\-tools (5)
-configuration files.
-In each case, the script\-specific variable takes precedence over the
-package\-wide variable.
-.TP
-.BR PULL_DEBIAN_SOURCE_DEBIAN_MIRROR ", " UBUNTUTOOLS_DEBIAN_MIRROR
-The default value for \fB\-\-mirror\fR.
-.TP
-.BR PULL_DEBIAN_SOURCE_DEBSEC_MIRROR ", " UBUNTUTOOLS_DEBSEC_MIRROR
-The default value for \fB\-\-security\-mirror\fR.
-
-.SH SEE ALSO
-.BR dget (1),
-.BR pull\-debian\-debdiff (1),
-.BR pull\-lp\-source (1),
-.BR ubuntu\-dev\-tools (5)
diff --git a/doc/pull-debian-source.1 b/doc/pull-debian-source.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-debian-source.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/doc/pull-debian-udebs.1 b/doc/pull-debian-udebs.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-debian-udebs.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/doc/pull-lp-ddebs.1 b/doc/pull-lp-ddebs.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-lp-ddebs.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/doc/pull-lp-debs.1 b/doc/pull-lp-debs.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-lp-debs.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/doc/pull-lp-source.1 b/doc/pull-lp-source.1
deleted file mode 100644
index 757857a..0000000
--- a/doc/pull-lp-source.1
+++ /dev/null
@@ -1,79 +0,0 @@
-.TH PULL\-LP\-SOURCE "1" "4 August 2008" "ubuntu-dev-tools"
-
-.SH NAME
-pull\-lp\-source \- download a source package from Launchpad
-
-.SH SYNOPSIS
-.B pull\-lp\-source \fR[\fIoptions\fR]\fB \fBsource package\fR
-[\fIrelease\fR|\fIversion\fR]
-
-.SH DESCRIPTION
-\fBpull\-lp\-source\fR downloads and extracts the specified
-\fIversion\fR of <\fBsource package\fR> from Launchpad, or the latest
-version of the specified \fIrelease\fR.
-To request a version from a particular pocket say
-\fIrelease\fB\-\fIpocket\fR (with a magic \fB\-release\fR for only the
-release pocket).
-If no \fIversion\fR or \fIrelease\fR is specified, the latest version in
-the development release will be downloaded.
-
-.SH OPTIONS
-Listed below are the command line options for pull\-lp\-source:
-.TP
-.B source package
-This is the source package that you would like to be downloaded from Launchpad.
-.TP
-.B version
-This is the version of the source package to be downloaded.
-.TP
-.B release
-This is the release that you would like the source package to be downloaded from.
-This value defaults to the current development release.
-.TP
-.BR \-h ", " \-\-help
-Display a help message and exit.
-.TP
-.BR \-d ", " \-\-download\-only
-Do not extract the source package.
-.TP
-.B \-m \fIUBUNTU_MIRROR\fR, \fB\-\-mirror\fR=\fIUBUNTU_MIRROR\fR
-Use the specified Ubuntu mirror.
-Should be in the form \fBhttp://archive.ubuntu.com/ubuntu\fR.
-If the package isn't found on this mirror, \fBpull\-lp\-source\fR will
-fall back to Launchpad, as its name implies.
-.TP
-.B \-\-no\-conf
-Do not read any configuration files, or configuration from environment
-variables.
-
-.SH ENVIRONMENT
-All of the \fBCONFIGURATION VARIABLES\fR below are also supported as
-environment variables.
-Variables in the environment take precedence to those in configuration
-files.
-.TP
-.B
-DIST
-Specifies the default target.
-
-.SH CONFIGURATION VARIABLES
-The following variables can be set in the environment or in
-.BR ubuntu\-dev\-tools (5)
-configuration files.
-In each case, the script\-specific variable takes precedence over the
-package\-wide variable.
-.TP
-.BR PULL_LP_SOURCE_UBUNTU_MIRROR ", " UBUNTUTOOLS_UBUNTU_MIRROR
-The default value for \fB\-\-mirror\fR.
-
-.SH SEE ALSO
-.BR dget (1),
-.BR pull\-debian\-source (1),
-.BR pull\-debian\-debdiff (1),
-.BR ubuntu\-dev\-tools (5)
-
-.SH AUTHOR
-.PP
-\fBpull\-lp\-source\fR and this manual page were written by Iain Lane
-<[e-mail]>.
-Both are released under the GNU General Public License, version 3 or later.
diff --git a/doc/pull-lp-source.1 b/doc/pull-lp-source.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-lp-source.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/doc/pull-lp-udebs.1 b/doc/pull-lp-udebs.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-lp-udebs.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/doc/pull-pkg.1 b/doc/pull-pkg.1
new file mode 100644
index 0000000..f62aa11
--- /dev/null
+++ b/doc/pull-pkg.1
@@ -0,0 +1,134 @@
+.TH PULL\-PKG "1" "28 August 2017" "ubuntu-dev-tools"
+
+.SH NAME
+pull\-pkg \- download a package for Debian, Ubuntu, or UCA
+
+.SH SYNOPSIS
+.B pull\-pkg \fR[\fIoptions\fR]\fR <\fIpackage name\fR>
+[\fIrelease\fR|\fIversion\fR]
+
+.SH DESCRIPTION
+\fBpull\-pkg\fR downloads the specified \fIversion\fR of
+<\fIpackage name\fR>, or the latest version from the
+specified \fIrelease\fR.  To request a version from
+a particular pocket say \fIrelease\fB\-\fIpocket\fR (with a magic
+\fB\-release\fR for only the release pocket).
+If no \fIversion\fR or \fIrelease\fR is specified, the latest version in
+the development release will be downloaded.
+
+There are convenience scripts that set pull type and distribution
+appropriately: these are
+\fBpull\-lp\-source\fR, \fBpull\-lp\-debs\fR, \fBpull\-lp\-ddebs\fR,
+and \fBpull\-lp\-udebs\fR, which all pull Ubuntu packages;
+\fBpull\-debian\-source\fR, \fBpull\-debian\-debs\fR, \fBpull\-debian\-ddebs\fR,
+and \fBpull\-debian\-udebs\fR, which all pull Debian packages;
+and \fBpull\-uca\-source\fR, \fBpull\-uca\-debs\fR, \fBpull\-uca\-ddebs\fR,
+and \fBpull\-uca\-udebs\fR, which all pull Ubuntu Cloud Archive packages.
+Each script pulls the file type in its name, i.e.
+\fIsource\fR, \fIdebs\fR, \fIddebs\fR, or \fIudebs\fR.
+
+.SH OPTIONS
+Listed below are the command line options for pull\-pkg:
+.TP
+.I package name
+This is name of the package to downloaded.
+You can use either the source package name, or binary package name.
+.TP
+.I version
+This is the version of the package to downloaded.
+.TP
+.I release
+This is the release to downloaded from.
+For debian, you can use either the release name like \fBjessie\fR
+or \fBsid\fR, or you can use the special release names \fBunstable\fR,
+\fBstable\fR, or \fBtesting\fR.
+For ubuntu, you can use either the release name like \fBxenial\fR
+or the release-pocket like \fBxenial-proposed\fR.
+For ubuntu cloud archive (uca) you can use either the uca release
+name like \fBmitaka\fR or the ubuntu and uca release names like
+\fBtrusty-mitaka\fR.
+Defaults to the current development release.
+.TP
+.BR \-h ", " \-\-help
+Display a help message and exit.
+.TP
+.BR \-v ", " \-\-verbose
+Be verbose about what is being done.
+.TP
+.BR \-d ", " \-\-download\-only
+Do not extract the source package (applies only to source packages).
+.TP
+.B \-m \fIMIRROR\fR, \fB\-\-mirror\fR=\fIMIRROR\fR
+Use the specified mirror server.
+Should be in the form \fBhttp://archive.ubuntu.com/ubuntu\fR or
+\fBhttp://deb.debian.org/debian\fR.  If not specified or if the
+package is not found on the specified mirror, this will fall
+back to the default mirror(s) and/or mirror(s) from environment
+variables, and then will fall back to Launchpad or Debian Snapshot.
+This can be specified multiple times to try multiple mirrors.
+.TP
+.B \-\-no\-conf
+Do not use mirrors from the default configuration, or from
+any environment variables.
+.TP
+.B \-a \fIARCH\fR, \fB\-\-arch\fR=\fIARCH\fR
+Get binary packages from the \fIARCH\fR architecture.
+Defaults to the local architecture, if it can be deteected.
+.TP
+.B \-p \fIPULL\fR, \fB\-\-pull\fR=\fIPULL\fR
+What to pull: \fBsource\fR, \fBdebs\fR, \fBddebs\fR, \fBudebs\fR,
+or \fBlist\fR.  The \fBlist\fR action only lists all a package's
+source and binary files, but does not actually download any.
+Defaults to \fBsource\fR.
+.TP
+.B \-D \fIDISTRO\fR, \fB\-\-distro\fR=\fIDISTRO\fR
+Pull from: \fBdebian\fR, \fBuca\fR, or \fBubuntu\fR.
+\fBlp\fR can be used instead of \fBubuntu\fR.
+Any string containing \fBcloud\fR can be used instead of \fBuca\fR.
+Deafults to \fBubuntu\fR.
+
+.SH ENVIRONMENT
+All of the \fBCONFIGURATION VARIABLES\fR below are also supported as
+environment variables.
+Variables in the environment take precedence to those in configuration
+files.
+
+.SH CONFIGURATION VARIABLES
+The following variables can be set in the environment or in
+.BR ubuntu\-dev\-tools (5)
+configuration files.
+In each case, the script\-specific variable takes precedence over the
+package\-wide variable.
+.TP
+.BR UBUNTUTOOLS_UBUNTU_MIRROR
+The default mirror.
+.TP
+.BR PULL_PKG_UBUNTU_MIRROR
+The default mirror when using the \fBpull\-pkg\fR script.
+.TP
+.BR PULL_[LP|DEBIAN|UCA]_[SOURCE|DEBS|DDEBS|UDEBS]_MIRROR
+The default mirror when using the associated script.
+
+.SH SEE ALSO
+.BR dget (1),
+.BR pull\-lp\-source (1),
+.BR pull\-lp\-debs (1),
+.BR pull\-lp\-ddebs (1),
+.BR pull\-lp\-udebs (1),
+.BR pull\-debian\-source (1),
+.BR pull\-debian\-debs (1),
+.BR pull\-debian\-ddebs (1),
+.BR pull\-debian\-udebs (1),
+.BR pull\-uca\-source (1),
+.BR pull\-uca\-debs (1),
+.BR pull\-uca\-ddebs (1),
+.BR pull\-uca\-udebs (1),
+.BR pull\-debian\-debdiff (1),
+.BR ubuntu\-dev\-tools (5)
+
+.SH AUTHOR
+.PP
+\fBpull\-pkg\fR was written by Dan Streetman <[e-mail]>,
+based on the original \fBpull\-lp\-source\fR; it and this manual page
+were written by Iain Lane <[e-mail]>.
+All are released under the GNU General Public License, version 3 or later.
diff --git a/doc/pull-uca-ddebs.1 b/doc/pull-uca-ddebs.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-uca-ddebs.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/doc/pull-uca-debs.1 b/doc/pull-uca-debs.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-uca-debs.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/doc/pull-uca-source.1 b/doc/pull-uca-source.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-uca-source.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/doc/pull-uca-udebs.1 b/doc/pull-uca-udebs.1
new file mode 120000
index 0000000..a67bbc7
--- /dev/null
+++ b/doc/pull-uca-udebs.1
@@ -0,0 +1 @@
+pull-pkg.1
\ No newline at end of file
diff --git a/pull-debian-ddebs b/pull-debian-ddebs
new file mode 100755
index 0000000..05fd628
--- /dev/null
+++ b/pull-debian-ddebs
@@ -0,0 +1,16 @@
+#!/usr/bin/python
+#
+# pull-debian-ddebs -- pull ddeb package files for debian
+# Basic usage: pull-debian-ddebs <package name> [version|release]
+#
+# See pull-pkg
+
+from ubuntutools import pullpkg
+from ubuntutools.logger import Logger
+
+if __name__ == '__main__':
+    try:
+        parser = pullpkg.create_argparser(default_distro='debian', default_pull='ddebs')
+        pullpkg.pull(parser.parse_args())
+    except KeyboardInterrupt:
+        Logger.normal('User abort.')
diff --git a/pull-debian-debs b/pull-debian-debs
new file mode 100755
index 0000000..21672a0
--- /dev/null
+++ b/pull-debian-debs
@@ -0,0 +1,16 @@
+#!/usr/bin/python
+#
+# pull-debian-debs -- pull deb package files for debian
+# Basic usage: pull-debian-debs <package name> [version|release]
+#
+# See pull-pkg
+
+from ubuntutools import pullpkg
+from ubuntutools.logger import Logger
+
+if __name__ == '__main__':
+    try:
+        parser = pullpkg.create_argparser(default_distro='debian', default_pull='debs')
+        pullpkg.pull(parser.parse_args())
+    except KeyboardInterrupt:
+        Logger.normal('User abort.')
diff --git a/pull-debian-source b/pull-debian-source
index d4e0519..9c82e6f 100755
--- a/pull-debian-source
+++ b/pull-debian-source
@@ -1,143 +1,16 @@
 #!/usr/bin/python
 #
-# pull-debian-source -- pull a source package from Launchpad
-# Copyright (C) 2011, Stefano Rivera <[e-mail]>
-# Inspired by a tool of the same name by Nathan Handler.
+# pull-debian-source -- pull source package files for debian
+# Basic usage: pull-debian-source <package name> [version|release]
 #
-# Permission to use, copy, modify, and/or distribute this software for any
-# purpose with or without fee is hereby granted, provided that the above
-# copyright notice and this permission notice appear in all copies.
-#
-# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
-# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
-# AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
-# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
-# LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR
-# OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
-# PERFORMANCE OF THIS SOFTWARE.
-
-import json
-import optparse
-import sys
-import urllib2
-
-from distro_info import DebianDistroInfo, DistroDataOutdated
+# See pull-pkg
 
-from ubuntutools.archive import DebianSourcePackage, DownloadError, rmadison
-from ubuntutools.config import UDTConfig
+from ubuntutools import pullpkg
 from ubuntutools.logger import Logger
 
-
-def is_suite(version):
-    """If version could be considered to be a Debian suite, return the
-    canonical suite name. Otherwise None
-    """
-    debian_info = DebianDistroInfo()
-    debian_releases = debian_info.all + ['experimental']
-
-    if '-' in version:
-        release, pocket = version.split('-', 1)
-        release = debian_info.codename(release, default=release)
-        if release in debian_releases:
-            if pocket in ('proposed-updates', 'p-u'):
-                return (release + '-proposed-updates')
-            elif pocket == 'security':
-                return (release + '-security')
-    else:
-        release = debian_info.codename(version, default=version)
-        if release in debian_releases:
-            return release
-    return None
-
-
-def source_package_for(binary, release):
-    """Query DDE to find the source package for a particular binary"""
-    try:
-        release = DebianDistroInfo().codename(release, default=release)
-    except DistroDataOutdated, e:
-        Logger.warn(e)
-    url = ('http://dde.debian.net/dde/q/udd/dist/d:debian/r:%s/p:%s/?t=json'
-           % (release, binary))
-    data = None
-    try:
-        data = json.load(urllib2.urlopen(url))['r']
-    except urllib2.URLError, e:
-        Logger.error('Unable to retrieve package information from DDE: '
-                     '%s (%s)', url, str(e))
-    except ValueError, e:
-        Logger.error('Unable to parse JSON response from DDE: '
-                     '%s (%s)', url, str(e))
-    if not data:
-        return None
-    return data[0]['source']
-
-
-def main():
-    usage = 'Usage: %prog <package> [release|version]'
-    parser = optparse.OptionParser(usage)
-    parser.add_option('-d', '--download-only',
-                      dest='download_only', default=False, action='store_true',
-                      help='Do not extract the source package')
-    parser.add_option('-m', '--mirror', metavar='DEBIAN_MIRROR',
-                      dest='debian_mirror',
-                      help='Preferred Debian mirror (default: %s)'
-                           % UDTConfig.defaults['DEBIAN_MIRROR'])
-    parser.add_option('-s', '--security-mirror', metavar='DEBSEC_MIRROR',
-                      dest='debsec_mirror',
-                      help='Preferred Debian Security mirror (default: %s)'
-                           % UDTConfig.defaults['DEBSEC_MIRROR'])
-    parser.add_option('--no-conf',
-                      dest='no_conf', default=False, action='store_true',
-                      help="Don't read config files or environment variables")
-    (options, args) = parser.parse_args()
-    if not args:
-        parser.error('Must specify package name')
-    elif len(args) > 2:
-        parser.error('Too many arguments. '
-                     'Must only specify package and (optionally) release.')
-
-    config = UDTConfig(options.no_conf)
-    if options.debian_mirror is None:
-        options.debian_mirror = config.get_value('DEBIAN_MIRROR')
-    if options.debsec_mirror is None:
-        options.debsec_mirror = config.get_value('DEBSEC_MIRROR')
-
-    package = args[0].lower()
-
-    version = args[1] if len(args) > 1 else 'unstable'
-    component = None
-
-    suite = is_suite(version)
-    if suite is not None:
-        line = list(rmadison('debian', package, suite, 'source'))
-        if not line:
-            source_package = source_package_for(package, suite)
-            if source_package is not None and package != source_package:
-                package = source_package
-                line = list(rmadison('debian', package, suite, 'source'))
-            if not line:
-                Logger.error('Unable to find %s in Debian suite "%s".', package,
-                             suite)
-                sys.exit(1)
-        line = line[-1]
-        version = line['version']
-        component = line['component']
-
-    Logger.normal('Downloading %s version %s', package, version)
-    srcpkg = DebianSourcePackage(package, version, component=component,
-                                 mirrors=[options.debian_mirror,
-                                          options.debsec_mirror])
-    try:
-        srcpkg.pull()
-    except DownloadError, e:
-        Logger.error('Failed to download: %s', str(e))
-        sys.exit(1)
-    if not options.download_only:
-        srcpkg.unpack()
-
-
 if __name__ == '__main__':
     try:
-        main()
+        parser = pullpkg.create_argparser(default_distro='debian', default_pull='source')
+        pullpkg.pull(parser.parse_args())
     except KeyboardInterrupt:
         Logger.normal('User abort.')
diff --git a/pull-debian-udebs b/pull-debian-udebs
new file mode 100755
index 0000000..808ad54
--- /dev/null
+++ b/pull-debian-udebs
@@ -0,0 +1,16 @@
+#!/usr/bin/python
+#
+# pull-debian-udebs -- pull udeb package files for debian
+# Basic usage: pull-debian-udebs <package name> [version|release]
+#
+# See pull-pkg
+
+from ubuntutools import pullpkg
+from ubuntutools.logger import Logger
+
+if __name__ == '__main__':
+    try:
+        parser = pullpkg.create_argparser(default_distro='debian', default_pull='udebs')
+        pullpkg.pull(parser.parse_args())
+    except KeyboardInterrupt:
+        Logger.normal('User abort.')
diff --git a/pull-lp-ddebs b/pull-lp-ddebs
new file mode 100755
index 0000000..39eb0f7
--- /dev/null
+++ b/pull-lp-ddebs
@@ -0,0 +1,16 @@
+#!/usr/bin/python
+#
+# pull-lp-ddebs -- pull ddeb package files for ubuntu
+# Basic usage: pull-lp-ddebs <package name> [version|release]
+#
+# See pull-pkg
+
+from ubuntutools import pullpkg
+from ubuntutools.logger import Logger
+
+if __name__ == '__main__':
+    try:
+        parser = pullpkg.create_argparser(default_distro='ubuntu', default_pull='ddebs')
+        pullpkg.pull(parser.parse_args())
+    except KeyboardInterrupt:
+        Logger.normal('User abort.')
diff --git a/pull-lp-debs b/pull-lp-debs
new file mode 100755
index 0000000..a66d470
--- /dev/null
+++ b/pull-lp-debs
@@ -0,0 +1,16 @@
+#!/usr/bin/python
+#
+# pull-lp-debs -- pull deb package files for ubuntu
+# Basic usage: pull-lp-debs <package name> [version|release]
+#
+# See pull-pkg
+
+from ubuntutools import pullpkg
+from ubuntutools.logger import Logger
+
+if __name__ == '__main__':
+    try:
+        parser = pullpkg.create_argparser(default_distro='ubuntu', default_pull='debs')
+        pullpkg.pull(parser.parse_args())
+    except KeyboardInterrupt:
+        Logger.normal('User abort.')
diff --git a/pull-lp-source b/pull-lp-source
index 9cb865a..4a7f3cc 100755
--- a/pull-lp-source
+++ b/pull-lp-source
@@ -1,149 +1,16 @@
 #!/usr/bin/python
 #
-# pull-lp-source -- pull a source package from Launchpad
-# Basic usage: pull-lp-source <source package> [<release>]
+# pull-lp-source -- pull source package files for ubuntu
+# Basic usage: pull-lp-source <package name> [version|release]
 #
-# Copyright (C) 2008,      Iain Lane <[e-mail]>,
-#               2010-2011, Stefano Rivera <[e-mail]>
-#
-# ##################################################################
-#
-# This program is free software; you can redistribute it and/or
-# modify it under the terms of the GNU General Public License
-# as published by the Free Software Foundation; either version 3
-# of the License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# See file /usr/share/common-licenses/GPL for more details.
-#
-# ##################################################################
-
+# See pull-pkg
 
-import json
-import os
-import sys
-import urllib2
-from optparse import OptionParser
-
-from distro_info import UbuntuDistroInfo, DistroDataOutdated
-
-from ubuntutools.archive import UbuntuSourcePackage, DownloadError
-from ubuntutools.config import UDTConfig
-from ubuntutools.lp.lpapicache import Distribution, Launchpad
-from ubuntutools.lp.udtexceptions import (SeriesNotFoundException,
-                                          PackageNotFoundException,
-                                          PocketDoesNotExistError)
+from ubuntutools import pullpkg
 from ubuntutools.logger import Logger
-from ubuntutools.misc import split_release_pocket
-
-
-def source_package_for(binary, release):
-    """Query DDE to find the source package for a particular binary
-    Should really do this with LP, but it's not possible LP: #597041
-    """
-    url = ('http://dde.debian.net/dde/q/udd/dist/d:ubuntu/r:%s/p:%s/?t=json'
-           % (release, binary))
-    data = None
-    try:
-        data = json.load(urllib2.urlopen(url))['r']
-    except urllib2.URLError, e:
-        Logger.error('Unable to retrieve package information from DDE: '
-                     '%s (%s)', url, str(e))
-    except ValueError, e:
-        Logger.error('Unable to parse JSON response from DDE: '
-                     '%s (%s)', url, str(e))
-    if not data:
-        return None
-    return data[0]['source']
-
-
-def main():
-    usage = "Usage: %prog <package> [release|version]"
-    opt_parser = OptionParser(usage)
-    opt_parser.add_option('-d', '--download-only',
-                          dest='download_only', default=False,
-                          action='store_true',
-                          help="Do not extract the source package")
-    opt_parser.add_option('-m', '--mirror', metavar='UBUNTU_MIRROR',
-                          dest='ubuntu_mirror',
-                          help='Preferred Ubuntu mirror (default: Launchpad)')
-    opt_parser.add_option('--no-conf',
-                          dest='no_conf', default=False, action='store_true',
-                          help="Don't read config files or environment "
-                               "variables")
-    (options, args) = opt_parser.parse_args()
-    if not args:
-        opt_parser.error("Must specify package name")
-
-    config = UDTConfig(options.no_conf)
-    if options.ubuntu_mirror is None:
-        options.ubuntu_mirror = config.get_value('UBUNTU_MIRROR')
-
-    # Login anonymously to LP
-    Launchpad.login_anonymously()
-
-    package = str(args[0]).lower()
-
-    ubuntu_info = UbuntuDistroInfo()
-    if len(args) > 1:  # Custom distribution specified.
-        version = str(args[1])
-    else:
-        try:
-            version = os.getenv('DIST') or ubuntu_info.devel()
-        except DistroDataOutdated, e:
-            Logger.warn("%s\nOr specify a distribution.", e)
-            sys.exit(1)
-    component = None
-
-    # Release, not package version number:
-    release = None
-    pocket = None
-    try:
-        (release, pocket) = split_release_pocket(version, default=None)
-    except PocketDoesNotExistError, e:
-        pass
-    if release in ubuntu_info.all:
-        archive = Distribution('ubuntu').getArchive()
-        try:
-            spph = archive.getSourcePackage(package, release, pocket)
-        except SeriesNotFoundException, e:
-            Logger.error(str(e))
-            sys.exit(1)
-        except PackageNotFoundException, e:
-            source_package = source_package_for(package, release)
-            if source_package is not None and source_package != package:
-                try:
-                    spph = archive.getSourcePackage(source_package, release,
-                                                    pocket)
-                    package = source_package
-                except PackageNotFoundException:
-                    Logger.error(str(e))
-                    sys.exit(1)
-            else:
-                Logger.error(str(e))
-                sys.exit(1)
-
-        version = spph.getVersion()
-        component = spph.getComponent()
-
-    Logger.normal('Downloading %s version %s', package, version)
-    srcpkg = UbuntuSourcePackage(package, version, component=component,
-                                 mirrors=[options.ubuntu_mirror])
-    try:
-        srcpkg.pull()
-    except DownloadError, e:
-        Logger.error('Failed to download: %s', str(e))
-        sys.exit(1)
-    if not options.download_only:
-        srcpkg.unpack()
-
 
 if __name__ == '__main__':
     try:
-        main()
+        parser = pullpkg.create_argparser(default_distro='ubuntu', default_pull='source')
+        pullpkg.pull(parser.parse_args())
     except KeyboardInterrupt:
         Logger.normal('User abort.')
diff --git a/pull-lp-udebs b/pull-lp-udebs
new file mode 100755
index 0000000..d050b61
--- /dev/null
+++ b/pull-lp-udebs
@@ -0,0 +1,16 @@
+#!/usr/bin/python
+#
+# pull-lp-udebs -- pull udeb package files for ubuntu
+# Basic usage: pull-lp-udebs <package name> [version|release]
+#
+# See pull-pkg
+
+from ubuntutools import pullpkg
+from ubuntutools.logger import Logger
+
+if __name__ == '__main__':
+    try:
+        parser = pullpkg.create_argparser(default_distro='ubuntu', default_pull='udebs')
+        pullpkg.pull(parser.parse_args())
+    except KeyboardInterrupt:
+        Logger.normal('User abort.')
diff --git a/pull-pkg b/pull-pkg
new file mode 100755
index 0000000..98c5d22
--- /dev/null
+++ b/pull-pkg
@@ -0,0 +1,33 @@
+#!/usr/bin/python
+#
+# pull-pkg -- pull package files for debian/ubuntu/uca
+# Basic usage: pull-pkg -D distro -p type <package name> [version|release]
+#
+# Copyright (C) 2008,      Iain Lane <[e-mail]>,
+#               2010-2011, Stefano Rivera <[e-mail]>
+#               2017,      Dan Streetman <[e-mail]>
+#
+# ##################################################################
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 3
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# See file /usr/share/common-licenses/GPL for more details.
+#
+# ##################################################################
+
+from ubuntutools import pullpkg
+from ubuntutools.logger import Logger
+
+if __name__ == '__main__':
+    try:
+        pullpkg.pull(pullpkg.create_argparser().parse_args())
+    except KeyboardInterrupt:
+        Logger.normal('User abort.')
diff --git a/pull-uca-ddebs b/pull-uca-ddebs
new file mode 100755
index 0000000..baa6ace
--- /dev/null
+++ b/pull-uca-ddebs
@@ -0,0 +1,16 @@
+#!/usr/bin/python
+#
+# pull-uca-ddebs -- pull ddeb package files for ubuntu cloud archive
+# Basic usage: pull-uca-ddebs <package name> [version|release]
+#
+# See pull-pkg
+
+from ubuntutools import pullpkg
+from ubuntutools.logger import Logger
+
+if __name__ == '__main__':
+    try:
+        parser = pullpkg.create_argparser(default_distro='uca', default_pull='ddebs')
+        pullpkg.pull(parser.parse_args())
+    except KeyboardInterrupt:
+        Logger.normal('User abort.')
diff --git a/pull-uca-debs b/pull-uca-debs
new file mode 100755
index 0000000..76c72a7
--- /dev/null
+++ b/pull-uca-debs
@@ -0,0 +1,16 @@
+#!/usr/bin/python
+#
+# pull-uca-debs -- pull deb package files for ubuntu cloud archive
+# Basic usage: pull-uca-debs <package name> [version|release]
+#
+# See pull-pkg
+
+from ubuntutools import pullpkg
+from ubuntutools.logger import Logger
+
+if __name__ == '__main__':
+    try:
+        parser = pullpkg.create_argparser(default_distro='uca', default_pull='debs')
+        pullpkg.pull(parser.parse_args())
+    except KeyboardInterrupt:
+        Logger.normal('User abort.')
diff --git a/pull-uca-source b/pull-uca-source
index 79a60f5..f2dbdf0 100755
--- a/pull-uca-source
+++ b/pull-uca-source
@@ -1,157 +1,16 @@
 #!/usr/bin/python
 #
-# pull-uca-source -- pull a source package from Ubuntu Cloud Archive
-# Basic usage: pull-uca-source <source package> <openstack release> [version]
+# pull-uca-source -- pull source package files for ubuntu cloud archive
+# Basic usage: pull-uca-source <package name> [version|release]
 #
-# Copyright (C) 2008,      Iain Lane <[e-mail]>,
-#               2010-2011, Stefano Rivera <[e-mail]>
-#               2016,      Corey Bryant <[e-mail]>
-#               2016,      Dan Streetman <[e-mail]>
-#
-# ##################################################################
-#
-# This program is free software; you can redistribute it and/or
-# modify it under the terms of the GNU General Public License
-# as published by the Free Software Foundation; either version 3
-# of the License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# See file /usr/share/common-licenses/GPL for more details.
-#
-# ##################################################################
-
+# See pull-pkg
 
-import re
-import sys
-from optparse import OptionParser
-
-from ubuntutools.archive import UbuntuCloudArchiveSourcePackage, DownloadError
-from ubuntutools.config import UDTConfig
-from ubuntutools.lp.lpapicache import Launchpad
-from ubuntutools.lp.udtexceptions import PocketDoesNotExistError
+from ubuntutools import pullpkg
 from ubuntutools.logger import Logger
-from ubuntutools.misc import split_release_pocket
-
-from lazr.restfulclient.errors import NotFound
-
-from launchpadlib.launchpad import Launchpad as LP
-
-
-def showOpenstackReleases(uca):
-    releases = []
-    for p in uca.ppas:
-        if re.match(r"\w*-staging", p.name):
-            releases.append(re.sub("-staging", "", p.name))
-    Logger.error("Openstack releases are:\n\t%s", ", ".join(releases))
-
-
-def getSPPH(lp, archive, package, version=None, series=None, pocket=None, try_binary=True):
-    params = {'exact_match': True, 'order_by_date': True}
-    if pocket:
-        params['pocket'] = pocket
-    if series:
-        params['distro_series'] = series()
-    elif version:
-        params['version'] = version
-    Logger.normal("checking %s version %s pocket %s", package, version, pocket)
-    spphs = archive.getPublishedSources(source_name=package, **params)
-    if spphs:
-        return spphs[0]
-    if not try_binary:
-        return None
-
-    # Didn't find any, maybe the package is a binary package name
-    if series:
-        del params['distro_series']
-        archs = lp.load(series().architectures_collection_link).entries
-        params['distro_arch_series'] = archs[0]['self_link']
-    bpphs = archive.getPublishedBinaries(binary_name=package, **params)
-    if bpphs:
-        bpph_build = lp.load(bpphs[0].build_link)
-        source_package = bpph_build.source_package_name
-        return getSPPH(lp, archive, source_package, version, series, pocket,
-                       try_binary=False)
-
-    return None
-
-
-def main():
-    usage = "Usage: %prog <package> <openstack release> [version]"
-    opt_parser = OptionParser(usage)
-    opt_parser.add_option('-d', '--download-only',
-                          dest='download_only', default=False,
-                          action='store_true',
-                          help="Do not extract the source package")
-    opt_parser.add_option('-m', '--mirror', metavar='OPENSTACK_MIRROR',
-                          dest='openstack_mirror',
-                          help='Preferred Openstack mirror (default: Launchpad)')
-    opt_parser.add_option('--no-conf',
-                          dest='no_conf', default=False, action='store_true',
-                          help="Don't read config files or environment "
-                               "variables")
-    (options, args) = opt_parser.parse_args()
-    if len(args) < 2:
-        opt_parser.error("Must specify package name and openstack release")
-
-    config = UDTConfig(options.no_conf)
-    if options.openstack_mirror is None:
-        options.openstack_mirror = config.get_value('OPENSTACK_MIRROR')
-    mirrors = []
-    if options.openstack_mirror:
-        mirrors.append(options.openstack_mirror)
-
-    # Login anonymously to LP
-    Launchpad.login_anonymously()
-    lp = LP.login_anonymously("pull-uca-source", "production")
-    uca = lp.people("ubuntu-cloud-archive")
-
-    package = str(args[0]).lower()
-    release = str(args[1]).lower()
-    version = None
-    if len(args) > 2:
-        version = str(args[2])
-
-    pocket = None
-    try:
-        (release, pocket) = split_release_pocket(release, default=None)
-    except PocketDoesNotExistError, e:
-        pass
-
-    try:
-        archive = uca.getPPAByName(name="%s-staging" % release)
-    except NotFound, e:
-        Logger.error('Archive does not exist for Openstack release: %s',
-                     release)
-        showOpenstackReleases(uca)
-        sys.exit(1)
-
-    spph = getSPPH(lp, archive, package, version, pocket=pocket)
-    if not spph:
-        Logger.error("Package %s in %s not found.", package, release)
-        sys.exit(1)
-
-    package = spph.source_package_name
-    version = spph.source_package_version
-    component = spph.component_name
-    Logger.normal('Downloading %s version %s component %s', package, version, component)
-    srcpkg = UbuntuCloudArchiveSourcePackage(release, package, version, component=component,
-                                             mirrors=mirrors)
-
-    try:
-        srcpkg.pull()
-    except DownloadError, e:
-        Logger.error('Failed to download: %s', str(e))
-        sys.exit(1)
-    if not options.download_only:
-        srcpkg.unpack()
-
 
 if __name__ == '__main__':
     try:
-        main()
+        parser = pullpkg.create_argparser(default_distro='uca', default_pull='source')
+        pullpkg.pull(parser.parse_args())
     except KeyboardInterrupt:
         Logger.normal('User abort.')
diff --git a/pull-uca-udebs b/pull-uca-udebs
new file mode 100755
index 0000000..a7008ff
--- /dev/null
+++ b/pull-uca-udebs
@@ -0,0 +1,16 @@
+#!/usr/bin/python
+#
+# pull-uca-udebs -- pull udeb package files for ubuntu cloud archive
+# Basic usage: pull-uca-udebs <package name> [version|release]
+#
+# See pull-pkg
+
+from ubuntutools import pullpkg
+from ubuntutools.logger import Logger
+
+if __name__ == '__main__':
+    try:
+        parser = pullpkg.create_argparser(default_distro='uca', default_pull='udebs')
+        pullpkg.pull(parser.parse_args())
+    except KeyboardInterrupt:
+        Logger.normal('User abort.')
diff --git a/setup.py b/setup.py
index 221e3a2..894a31f 100755
--- a/setup.py
+++ b/setup.py
@@ -34,11 +34,21 @@ else:
         'mk-sbuild',
         'pbuilder-dist',
         'pbuilder-dist-simple',
+        'pull-pkg',
         'pull-debian-debdiff',
         'pull-debian-source',
+        'pull-debian-debs',
+        'pull-debian-ddebs',
+        'pull-debian-udebs',
         'pull-lp-source',
+        'pull-lp-debs',
+        'pull-lp-ddebs',
+        'pull-lp-udebs',
         'pull-revu-source',
         'pull-uca-source',
+        'pull-uca-debs',
+        'pull-uca-ddebs',
+        'pull-uca-udebs',
         'requestbackport',
         'requestsync',
         'reverse-build-depends',
diff --git a/ubuntutools/archive.py b/ubuntutools/archive.py
index 9324313..dd1f54f 100644
--- a/ubuntutools/archive.py
+++ b/ubuntutools/archive.py
@@ -42,15 +42,22 @@ except ImportError:
 import re
 import sys
 
-from debian.changelog import Changelog, Version
+from debian.changelog import Changelog
 import debian.deb822
-import debian.debian_support
 import httplib2
+import json
 
 from ubuntutools.config import UDTConfig
-from ubuntutools.lp.lpapicache import (Launchpad, Distribution,
-                                       SourcePackagePublishingHistory)
+from ubuntutools.lp.libsupport import get_system_arch
+from ubuntutools.lp.lpapicache import (Launchpad, Distribution, PersonTeam,
+                                       SourcePackagePublishingHistory,
+                                       BinaryPackagePublishingHistory)
+from ubuntutools.lp.udtexceptions import (PackageNotFoundException,
+                                          SeriesNotFoundException,
+                                          PocketDoesNotExistError)
 from ubuntutools.logger import Logger
+from ubuntutools.misc import split_release_pocket
+from ubuntutools.version import Version
 from ubuntutools import subprocess
 
 if sys.version_info[0] >= 3:
@@ -130,58 +137,131 @@ class SourcePackage(object):
     distribution = None
 
     def __init__(self, package=None, version=None, component=None,
-                 dscfile=None, lp=None, mirrors=(), workdir='.', quiet=False):
-        "Can be initialised either using package, version or dscfile"
-        assert ((package is not None and version is not None)
-                or dscfile is not None)
+                 dscfile=None, lp=None, mirrors=(), workdir='.', quiet=False,
+                 series=None, pocket=None):
+        """Can be initialised using either package or dscfile.
+        If package is specified, either the version or series can also be
+        specified; using version will get the specific package version,
+        while using the series will get the latest version from that series.
+        Specifying only the package with no version or series will get the
+        latest version from the development series.
+        """
+        assert (package is not None or dscfile is not None)
 
         self.source = package
         self._lp = lp
+        self.binary = None
+        self.try_binary = True
         self.workdir = workdir
         self.quiet = quiet
+        self._series = series
+        self._use_series = True
+        self._pocket = pocket
+        self._dsc_source = dscfile
 
         # Cached values:
         self._component = component
         self._dsc = None
         self._spph = None
+        self._version = Version(version) if version else None
 
         # State:
         self._dsc_fetched = False
 
         # Mirrors
-        self._dsc_source = dscfile
         self.mirrors = list(mirrors)
         if self.distribution:
             self.masters = [UDTConfig.defaults['%s_MIRROR'
                                                % self.distribution.upper()]]
-        if dscfile is not None:
-            if self.source is None:
-                self.source = 'unknown'
-            if version is None:
-                version = 'unknown'
-
-        self.version = debian.debian_support.Version(version)
 
         # uses default proxies from the environment
         proxy = ProxyHandler()
         self.url_opener = build_opener(proxy)
 
+        # if a dsc was specified, pull it to get the source/version info
+        if self._dsc_source:
+            self.pull_dsc()
+
     @property
     def lp_spph(self):
         "Return the LP Source Package Publishing History entry"
-        if not self._spph:
-            if not Launchpad.logged_in:
-                if self._lp:
-                    Launchpad.login_existing(self._lp)
+        if self._spph:
+            return self._spph
+
+        if not Launchpad.logged_in:
+            if self._lp:
+                Launchpad.login_existing(self._lp)
+            else:
+                Launchpad.login_anonymously()
+
+        distro = Distribution(self.distribution)
+        series = None
+        archive = self.getArchive()
+        params = {'exact_match': True, 'order_by_date': True}
+        if self._version:
+            # if version was specified, use that
+            params['version'] = self._version.full_version
+        elif self._use_series:
+            if self._series:
+                # if version not specified, get the latest from this series
+                series = distro.getSeries(self._series)
+            else:
+                # if no version or series, get the latest from devel series
+                series = distro.getDevelopmentSeries()
+            params['distro_series'] = series()
+            if self._pocket:
+                params['pocket'] = self._pocket
+        spphs = archive.getPublishedSources(source_name=self.source, **params)
+        if spphs:
+            self._spph = SourcePackagePublishingHistory(spphs[0])
+            Logger.debug("Found SPPH {}".format(self._spph))
+            return self._spph
+
+        if self.try_binary and not self.binary:
+            if series:
+                arch_series = series.getArchSeries()
+                params['distro_arch_series'] = arch_series()
+                del params['distro_series']
+            bpphs = archive.getPublishedBinaries(binary_name=self.source, **params)
+            if bpphs:
+                bpph = BinaryPackagePublishingHistory(bpphs[0])
+                Logger.debug("Found BPPH {}".format(bpph))
+                self.binary = self.source
+                self.source = bpph.getSourcePackageName()
+                Logger.normal("Using source package '{}' for binary package '{}'"
+                              .format(self.source, self.binary))
+                try:
+                    spph = bpph.getBuild().getSourcePackagePublishingHistory()
+                except:
+                    spph = None
+                if spph:
+                    self._spph = spph
+                    return self._spph
                 else:
-                    Launchpad.login_anonymously()
-            spph = Distribution(self.distribution).getArchive().getPublishedSources(
-                source_name=self.source,
-                version=self.version.full_version,
-                exact_match=True,
-            )
-            self._spph = SourcePackagePublishingHistory(spph[0])
-        return self._spph
+                    # binary build didn't include source link, unfortunately
+                    # so try again with the updated self.source name
+                    if not self._version:
+                        # Get version first if user didn't specify it, as some
+                        # binaries have their version hardcoded in their name,
+                        # such as the kernel package
+                        self._version = Version(bpph.getVersion())
+                    return self.lp_spph
+
+        msg = "No {} package found".format(self.source)
+        if self._version:
+            msg += " for version {}".format(self._version.full_version)
+        elif series:
+            msg += " in series {}".format(series.name)
+            if self._pocket:
+                msg += " pocket {}".format(self._pocket)
+        raise PackageNotFoundException(msg)
+
+    @property
+    def version(self):
+        "Return Package version"
+        if not self._version:
+            self._version = Version(self.lp_spph.getVersion())
+        return self._version
 
     @property
     def component(self):
@@ -194,10 +274,7 @@ class SourcePackage(object):
     @property
     def dsc_name(self):
         "Return the source package dsc filename for the given package"
-        version = self.version.upstream_version
-        if self.version.debian_version:
-            version += '-' + self.version.debian_version
-        return '%s_%s.dsc' % (self.source, version)
+        return '%s_%s.dsc' % (self.source, self.version.strip_epoch())
 
     @property
     def dsc_pathname(self):
@@ -211,6 +288,9 @@ class SourcePackage(object):
             self.pull_dsc()
         return self._dsc
 
+    def getArchive(self):
+        return Distribution(self.distribution).getArchive()
+
     def _mirror_url(self, mirror, filename):
         "Build a source package URL on a mirror"
         if self.source.startswith('lib'):
@@ -236,6 +316,22 @@ class SourcePackage(object):
                 yield self._mirror_url(mirror, name)
         yield self._lp_url(name)
 
+    def _binary_urls(self, name, default_url):
+        "Generator of URLs for name"
+        for mirror in self.mirrors:
+            yield self._mirror_url(mirror, name)
+        for mirror in self.masters:
+            if mirror not in self.mirrors:
+                yield self._mirror_url(mirror, name)
+        yield self._lp_url(name)
+        yield default_url
+
+    def _binary_files(self, name, arch):
+        for bpph in self.lp_spph.getBinaries(arch):
+            if name and not re.match(name, bpph.binary_package_name):
+                continue
+            yield (bpph.getFileName(), bpph.getUrl(), 0)
+
     def pull_dsc(self):
         "Retrieve dscfile and parse"
         if self._dsc_source:
@@ -271,7 +367,7 @@ class SourcePackage(object):
         "Check that the dsc matches what we are expecting"
         assert self._dsc is not None
         self.source = self.dsc['Source']
-        self.version = debian.debian_support.Version(self.dsc['Version'])
+        self._version = Version(self.dsc['Version'])
 
         valid = False
         message = None
@@ -310,28 +406,41 @@ class SourcePackage(object):
         with open(self.dsc_pathname, 'wb') as f:
             f.write(self.dsc.raw_text)
 
-    def _download_file(self, url, filename):
+    def _download_file(self, url, filename, verify=True, size=0):
         "Download url to filename in workdir."
         pathname = os.path.join(self.workdir, filename)
-        if self.dsc.verify_file(pathname):
-            Logger.debug('Using existing %s', filename)
-            return True
-        size = [entry['size'] for entry in self.dsc['Files']
-                if entry['name'] == filename]
-        assert len(size) == 1
-        size = int(size[0])
+        if verify:
+            if self.dsc.verify_file(pathname):
+                Logger.debug('Using existing %s', filename)
+                return True
+            size = [entry['size'] for entry in self.dsc['Files']
+                    if entry['name'] == filename]
+            assert len(size) == 1
+            size = int(size[0])
+
         parsed = urlparse(url)
-        if not self.quiet:
-            Logger.normal('Downloading %s from %s (%0.3f MiB)',
-                          filename, parsed.hostname, size / 1024.0 / 1024)
 
         if parsed.scheme == 'file':
             in_ = open(parsed.path, 'rb')
+            if not size:
+                size = int(os.stat(parsed.path).st_size)
         else:
             try:
                 in_ = self.url_opener.open(url)
-            except URLError:
+                Logger.debug("Using URL '%s'", url)
+            except URLError as e:
+                Logger.debug("URLError opening '%s': %s", url, e)
                 return False
+            if not size:
+                contentlen = in_.info().get('Content-Length')
+                if not contentlen:
+                    Logger.error("Invalid response, no Content-Length")
+                    return False
+                size = int(contentlen)
+
+        if not self.quiet:
+            Logger.normal('Downloading %s from %s (%0.3f MiB)',
+                          filename, parsed.hostname, size / 1024.0 / 1024)
 
         downloaded = 0
         bar_width = 60
@@ -354,7 +463,7 @@ class SourcePackage(object):
             if not self.quiet:
                 Logger.stdout.write(' ' * (bar_width + 7) + '\r')
                 Logger.stdout.flush()
-        if not self.dsc.verify_file(pathname):
+        if verify and not self.dsc.verify_file(pathname):
             Logger.error('Checksum for %s does not match.', filename)
             return False
         return True
@@ -375,6 +484,36 @@ class SourcePackage(object):
             else:
                 raise DownloadError('File %s could not be found' % name)
 
+    def pull_binaries(self, name=None, arch=None):
+        """Pull binary debs into workdir.
+        If name is specified, only binary packages matching the regex are included.
+        If arch is specified, only binary packages for that arch are included,
+        otherwise only binary packages for the current arch are included.
+        Returns the number of files downloaded.
+        """
+        total = 0
+
+        if not arch:
+            arch = get_system_arch()
+            Logger.debug("Using system arch %s", arch)
+
+        for (fname, furl, fsize) in self._binary_files(name, arch):
+            found = False
+            for url in self._binary_urls(fname, furl):
+                try:
+                    if self._download_file(url, fname, False, fsize):
+                        found = True
+                        break
+                except HTTPError as e:
+                    Logger.normal('HTTP Error %i: %s', e.code, str(e))
+                except URLError as e:
+                    Logger.normal('URL Error: %s', e.reason)
+            if found:
+                total += 1
+            else:
+                Logger.normal("Could not download from any location: %s", fname)
+        return total
+
     def verify(self):
         """Verify that the source package in workdir matches the dsc.
         Return boolean
@@ -432,33 +571,28 @@ class DebianSourcePackage(SourcePackage):
         super(DebianSourcePackage, self).__init__(*args, **kwargs)
         self.masters.append(UDTConfig.defaults['DEBSEC_MIRROR'])
         # Cached values:
-        self._snapshot_list = None
+        self._snapshot_package = None
+        self._snapshot_files = None
+        # Don't bother searching in LP for debian binaries, they aren't there
+        self.try_binary = False
 
     # Overridden methods:
     @property
     def lp_spph(self):
         "Return the LP Source Package Publishing History entry"
-        if not self._spph:
-            try:
-                return super(DebianSourcePackage, self).lp_spph
-            except IndexError:
-                pass
-
-            Logger.normal('Using rmadison for component determination')
-            comp = 'main'
-            for record in rmadison(self.distribution, self.source):
-                if record.get('source') != self.source:
-                    continue
-                comp = record['component']
-                if record['version'] == self.version.full_version:
-                    self._spph = FakeSPPH(record['source'], record['version'],
-                                          comp, 'debian')
-                    return self._spph
+        if self._spph:
+            return self._spph
 
-            Logger.normal('Guessing component from most recent upload')
-            self._spph = FakeSPPH(self.source, self.version.full_version, comp,
-                                  'debian')
-        return self._spph
+        try:
+            self._spph = DebianSPPH(super(DebianSourcePackage, self).lp_spph())
+            return self._spph
+        except PackageNotFoundException:
+            pass
+        except SeriesNotFoundException:
+            pass
+
+        Logger.normal('Package not found in Launchpad, using snapshot')
+        return self.snapshot_package.getSPPH()
 
     def _source_urls(self, name):
         "Generator of sources for name"
@@ -468,8 +602,13 @@ class DebianSourcePackage(SourcePackage):
                 yield next(wrapped_iterator)
             except StopIteration:
                 break
-        if self.snapshot_list:
-            yield self._snapshot_url(name)
+        yield self.snapshot_files[name]
+
+    def _binary_files(self, name, arch):
+        for f in self.snapshot_package.getBinaryFiles(arch):
+            if name and not re.match(name, f.package_name):
+                continue
+            yield (f.name, f.getUrl(), f.size)
 
     def pull_dsc(self):
         "Retrieve dscfile and parse"
@@ -493,41 +632,39 @@ class DebianSourcePackage(SourcePackage):
 
     # Local methods:
     @property
-    def snapshot_list(self):
-        "Return a filename -> hash dictionary from snapshot.debian.org"
-        if self._snapshot_list is None:
-            try:
-                import json
-            except ImportError:
-                import simplejson as json
-            except ImportError:
-                Logger.error("Please install python-simplejson.")
-                raise DownloadError("Unable to dowload from "
-                                    "snapshot.debian.org without "
-                                    "python-simplejson")
-
-            try:
-                data = self.url_opener.open(
-                    'http://snapshot.debian.org/mr/package/%s/%s/srcfiles?fileinfo=1' %
-                    (self.source, self.version.full_version))
-                reader = codecs.getreader('utf-8')
-                srcfiles = json.load(reader(data))
-
-            except HTTPError:
-                Logger.error('Version %s of %s not found on '
-                             'snapshot.debian.org',
-                             self.version.full_version, self.source)
-                self._snapshot_list = False
-                return False
-            self._snapshot_list = dict((info[0]['name'], hash_)
-                                       for hash_, info
-                                       in srcfiles['fileinfo'].items())
-        return self._snapshot_list
+    def snapshot_package(self):
+        if not self._snapshot_package:
+            if self._version or self._spph:
+                # as .version uses .lpph, and our .lpph might use us,
+                # only use .version if _version or _spph are set
+                version = self.version.full_version
+                srcpkgs = Snapshot.getSourcePackages(self.source, version=version)
+                if not srcpkgs:
+                    msg = "Package {} {} not found".format(self.source, version)
+                    raise PackageNotFoundException(msg)
+                self._snapshot_package = srcpkgs[0]
+            else:
+                # we have neither version nor spph, so look up our version using madison
+                Logger.normal('Using madison to find latest version number')
+                series = self._series
+                if self._pocket:
+                    series += "-" + self._pocket
+                params = {'series': series} if series else {}
+                srcpkg = Madison(self.distribution).getSourcePackage(self.source, **params)
+                if not srcpkg:
+                    raise PackageNotFoundException("Package {} not found".format(self.source))
+                if self.source != srcpkg.name:
+                    self.binary = self.source
+                    self.source = srcpkg.name
+                self._snapshot_package = srcpkg
+        return self._snapshot_package
 
-    def _snapshot_url(self, name):
-        "Return the snapshot.debian.org URL for name"
-        return os.path.join('http://snapshot.debian.org/file',
-                            self.snapshot_list[name])
+    @property
+    def snapshot_files(self):
+        if not self._snapshot_files:
+            files = self.snapshot_package.getFiles()
+            self._snapshot_files = {f.name: f.getUrl() for f in files}
+        return self._snapshot_files
 
 
 class UbuntuSourcePackage(SourcePackage):
@@ -537,37 +674,397 @@ class UbuntuSourcePackage(SourcePackage):
 
 class UbuntuCloudArchiveSourcePackage(UbuntuSourcePackage):
     "Download / unpack an Ubuntu Cloud Archive source package"
-    def __init__(self, uca_release, *args, **kwargs):
+    _ppas = None
+    _ppa_names = None
+
+    def __init__(self, *args, **kwargs):
         super(UbuntuCloudArchiveSourcePackage, self).__init__(*args, **kwargs)
-        self._uca_release = uca_release
+        self._use_series = False  # UCA doesn't really use distro series
+        self._uca_release = self._series
+        self._series = None
         self.masters = ["http://ubuntu-cloud.archive.canonical.com/ubuntu/"]
 
+    @classmethod
+    def getArchives(self):
+        if not self._ppas:
+            ppas = filter(lambda p: p.name.endswith('-staging'),
+                          PersonTeam.fetch('ubuntu-cloud-archive').getPPAs())
+            self._ppas = sorted(ppas, key=lambda p: p.name, reverse=True)
+        return self._ppas
+
+    @classmethod
+    def getReleaseNames(self):
+        if not self._ppa_names:
+            self._ppa_names = [p.name.split('-', 1)[0] for p in self.getArchives()]
+        return self._ppa_names
+
+    @classmethod
+    def getDevelopmentRelease(self):
+        return self.getReleaseNames()[0]
+
+    @property
+    def uca_release(self):
+        if not self._uca_release:
+            self._uca_release = self.getDevelopmentRelease()
+            Logger.normal('Using UCA release %s', self._uca_release)
+        return self._uca_release
+
+    def getArchive(self):
+        ppas = {p.name: p for p in self.getArchives()}
+        release = '{}-staging'.format(self.uca_release)
+        if release in ppas:
+            Logger.debug('UCA release {} at {}'.format(self.uca_release,
+                                                       ppas[release]()))
+            return ppas[release]
+        raise SeriesNotFoundException('UCA release {} not found.'.format(self.uca_release))
+
     def _lp_url(self, filename):
         "Build a source package URL on Launchpad"
         return os.path.join('https://launchpad.net', "~ubuntu-cloud-archive",
-                            '+archive', ("%s-staging" % self._uca_release),
+                            '+archive', ("%s-staging" % self.uca_release),
                             '+files', filename)
 
 
-class FakeSPPH(object):
-    """Provide the same interface as
-    ubuntutools.lpapicache.SourcePackagePublishingHistory
+class DebianSPPH(SourcePackagePublishingHistory):
+    """SPPH with getBinaries() overridden,
+    as LP doesn't have Debian binaries
     """
-    def __init__(self, name, version, component, distribution):
-        self.name = name
-        self.version = version
+    resource_type = 'source_package_publishing_history'
+
+    def getBinaries(self, arch=None):
+        if not self._binaries:
+            Logger.normal('Using Snapshot to find binary packages')
+            srcpkg = Snapshot.getSourcePackages(self.getPackageName(),
+                                                version=self.getVersion(),
+                                                component=self.getComponent())[0]
+            self._binaries = [b.getBPPH() for b in srcpkg.getBinaryFiles()]
+        return super(DebianSPPH, self).getBinaries(arch)
+
+
+class _WebJSON(object):
+    def __init__(self):
+        # uses default proxies from the environment
+        self.url_opener = build_opener(ProxyHandler())
+
+    def getHostUrl(self):
+        raise Exception("Not implemented")
+
+    def load(self, path=''):
+        url = self.getHostUrl() + path
+        Logger.debug("Loading %s", url)
+        data = self.url_opener.open(url)
+        reader = codecs.getreader('utf-8')
+        return json.load(reader(data))
+
+
+# DAKweb madison API
+# https://github.com/Debian/dak/blob/master/dakweb/queries/madison.py
+class Madison(_WebJSON):
+    urls = {
+        'debian': 'https://api.ftp-master.debian.org/madison',
+        # This currently will NOT work with ubuntu; it doesn't support f=json
+        'ubuntu': 'http://people.canonical.com/~ubuntu-archive/madison.cgi',
+    }
+
+    def __init__(self, distro='debian'):
+        super(Madison, self).__init__()
+        self._distro = distro
+
+    def getHostUrl(self):
+        return self.urls[self._distro]
+
+    def getSourcePackage(self, name, series='unstable'):
+        url = "?f=json&package={name}&s={series}".format(name=name, series=series)
+        try:
+            result = self.load(url)
+        except HTTPError:
+            result = None
+        if not result:
+            msg = "Package {} not found in '{}'".format(name, series)
+            raise PackageNotFoundException(msg)
+        versions = result[0][name].values()[0]
+        latest = versions[sorted(versions.keys(), reverse=True)[0]]
+        return Snapshot.getSourcePackages(name=latest['source'],
+                                          version=latest['source_version'],
+                                          component=latest['component'])[0]
+
+
+# Snapshot API
+# https://anonscm.debian.org/cgit/mirror/snapshot.debian.org.git/plain/API
+class _Snapshot(_WebJSON):
+    def getHostUrl(self):
+        return "http://snapshot.debian.org"
+
+    def _get_package(self, name, url, pkginit, version=None, series=None):
+        try:
+            results = self.load("/mr/{}/{}/".format(url, name))['result']
+        except HTTPError:
+            raise PackageNotFoundException("Package {} not found.".format(name))
+
+        results = [pkginit(r) for r in results]
+        if version:
+            results = [r for r in results if version == r.version]
+        if series:
+            if not version:
+                Logger.warn("Warning: searching Snapshot pkgs by series is VERY slow!")
+            results = [r for r in results if series == r.series]
+        if not results:
+            msg = "Package {} ".format(name)
+            if version:
+                msg += "version {} ".format(version)
+            if series:
+                msg += "in series {} ".format(series)
+            msg += "not found."
+            raise PackageNotFoundException(msg)
+        return sorted(results, key=lambda r: r.version, reverse=True)
+
+    def getSourcePackages(self, name, version=None, series=None, component=None):
+        return self._get_package(name, "package",
+                                 lambda obj: SnapshotSourcePackage(obj, name, component),
+                                 version=version, series=series)
+
+    def getBinaryPackages(self, name, version=None, series=None, component=None):
+        return self._get_package(name, "binary", SnapshotBinaryPackage,
+                                 lambda obj: SnapshotBinaryPackage(obj, component),
+                                 version=version, series=series)
+
+
+Snapshot = _Snapshot()
+
+
+class SnapshotPackage(object):
+    def __init__(self, obj, component):
+        self._obj = obj
+        self._files = None
+        self._series = None
         self.component = component
-        self.distribution = distribution
-        self._changelog = None
+
+    @property
+    def version(self):
+        return self._obj['version']
+
+    @property
+    def series(self):
+        if not self._series:
+            files = self.getFiles()
+            if not files:
+                Logger.error("Package {} {} has no files."
+                             .format(self.name, self.version))
+                self._series = 'unknown'
+            else:
+                self._series = files[0].archive_name
+            if filter(lambda f: f.archive_name != self._series, files):
+                Logger.debug("Package {} {} has files in multiple series."
+                             .format(self.name, self.version))
+        return self._series
+
+
+class SnapshotSourcePackage(SnapshotPackage):
+    def __init__(self, obj, name, component):
+        # obj required fields: 'version'
+        super(SnapshotSourcePackage, self).__init__(obj, component)
+        self.name = name
+        self._binary_files = None
+        self._spph = None
+
+    def getSPPH(self):
+        if not self._spph:
+            self._spph = SnapshotSPPH(self)
+        return self._spph
+
+    def getAllFiles(self):
+        return self.getFiles() + self.getBinaryFiles()
+
+    def getBinaryFiles(self, arch=None):
+        if not self._binary_files:
+            url = "/mr/package/{}/{}/allfiles".format(self.name, self.version)
+            response = Snapshot.load("{}?fileinfo=1".format(url))
+            info = response['fileinfo']
+            files = [SnapshotBinaryFile(b['name'], b['version'], self.component,
+                                        info[r['hash']][0], r['hash'],
+                                        r['architecture'], self.name)
+                     for b in response['result']['binaries'] for r in b['files']]
+            self._binary_files = files
+        if not arch:
+            return list(self._binary_files)
+        return filter(lambda f: f.isArch(arch), self._binary_files)
+
+    def getFiles(self):
+        if not self._files:
+            url = "/mr/package/{}/{}/srcfiles".format(self.name, self.version)
+            response = Snapshot.load("{}?fileinfo=1".format(url))
+            info = response['fileinfo']
+            self._files = [SnapshotSourceFile(self.name, self.version, self.component,
+                                              info[r['hash']][0],
+                                              r['hash'])
+                           for r in response['result']]
+        return list(self._files)
+
+
+class SnapshotBinaryPackage(SnapshotPackage):
+    def __init__(self, obj, component):
+        # obj required fields: 'version', 'binary_version', 'name', 'source'
+        super(SnapshotBinaryPackage, self).__init__(obj, component)
+
+    @property
+    def name(self):
+        return self._obj['name']
+
+    @property
+    def binary_version(self):
+        return self._obj['binary_version']
+
+    @property
+    def source(self):
+        return self._obj['source']
+
+    def getBPPH(self, arch):
+        f = self.getFiles(arch)
+        if not f:
+            return None
+        # Can only be 1 binary file for this pkg name/version/arch
+        return f[0].getBPPH()
+
+    def getFiles(self, arch=None):
+        if not self._files:
+            url = "/mr/binary/{}/{}/binfiles".format(self.name, self.version)
+            response = Snapshot.load("{}?fileinfo=1".format(url))
+            info = response['fileinfo']
+            self._files = [SnapshotBinaryFile(self.name, self.version, self.component,
+                                              info[r['hash']][0], r['hash'],
+                                              r['architecture'], self.source)
+                           for r in response['result']]
+        if not arch:
+            return list(self._files)
+        return filter(lambda f: f.isArch(arch), self._files)
+
+
+class SnapshotFile(object):
+    def __init__(self, pkg_name, pkg_version, pkg_component, obj, h):
+        self.package_name = pkg_name
+        self.package_version = pkg_version
+        self.package_component = pkg_component
+        self._obj = obj
+        self._hash = h
+
+    @property
+    def getType(self):
+        return None
+
+    @property
+    def archive_name(self):
+        return self._obj['archive_name']
+
+    @property
+    def name(self):
+        return self._obj['name']
+
+    @property
+    def path(self):
+        return self._obj['path']
+
+    @property
+    def size(self):
+        return self._obj['size']
+
+    @property
+    def date(self):
+        if 'run' in self._obj:
+            return self._obj['run']
+        elif 'first_seen' in self._obj:
+            return self._obj['first_seen']
+        else:
+            Logger.error('File {} has no date information', self.name)
+            return 'unknown'
+
+    def getHash(self):
+        return self._hash
+
+    def getUrl(self):
+        return "{}/file/{}".format(Snapshot.getHostUrl(), self.getHash())
+
+    def __repr__(self):
+        return "{}/{} {} bytes {}".format(self.path, self.name, self.size, self.date)
+
+
+class SnapshotSourceFile(SnapshotFile):
+    def __init__(self, name, version, component, obj, h):
+        super(SnapshotSourceFile, self).__init__(name, version, component, obj, h)
+
+    def getType(self):
+        return 'source'
+
+
+class SnapshotBinaryFile(SnapshotFile):
+    def __init__(self, name, version, component, obj, h, arch, source):
+        super(SnapshotBinaryFile, self).__init__(name, version, component, obj, h)
+        self.source = source
+        self.arch = arch
+        self._bpph = None
+
+    def isArch(self, arch):
+        if not arch:
+            return True
+        if self.arch == 'all':
+            return True
+        return arch == self.arch
+
+    def getType(self):
+        return 'binary'
+
+    def getBPPH(self):
+        if not self._bpph:
+            self._bpph = SnapshotBPPH(self)
+        return self._bpph
+
+
+class SnapshotSPPH(object):
+    """Provide the same interface as SourcePackagePublishingHistory"""
+    def __init__(self, snapshot_pkg):
+        self._pkg = snapshot_pkg
+
+    # LP API defined fields
+
+    @property
+    def component_name(self):
+        return self.getComponent()
+
+    @property
+    def display_name(self):
+        return ("{name} {version} in {seriesandpocket}"
+                .format(name=self.getPackageName(),
+                        version=self.getVersion(),
+                        seriesandpocket=self.getSeriesAndPocket()))
+
+    @property
+    def pocket(self):
+        try:
+            (r, p) = split_release_pocket(self.getSeriesAndPocket(), default=None)
+        except PocketDoesNotExistError:
+            p = None
+        return p if p else 'Release'
+
+    @property
+    def source_package_name(self):
+        return self.getPackageName()
+
+    @property
+    def source_package_version(self):
+        return self.getVersion()
+
+    # SPPH functions
 
     def getPackageName(self):
-        return self.name
+        return self._pkg.name
 
     def getVersion(self):
-        return self.version
+        return self._pkg.version
 
     def getComponent(self):
-        return self.component
+        return self._pkg.component
+
+    def getSeriesAndPocket(self):
+        return self._pkg.series
 
     def getChangelog(self, since_version=None):
         '''
@@ -575,27 +1072,22 @@ class FakeSPPH(object):
         May return None if the changelog isn't available
         '''
         if self._changelog is None:
-            if self.name.startswith('lib'):
-                subdir = 'lib%s' % self.name[3]
+            name = self.getPackageName()
+            if name.startswith('lib'):
+                subdir = 'lib%s' % name[3]
             else:
-                subdir = self.name[0]
-            # Strip epoch from version
-            pkgversion = self.version.split(':', 1)[-1]
-            extension = ''
-            if self.distribution == 'debian':
-                base = 'http://packages.debian.org/'
-                extension = '.txt'
-            elif self.distribution == 'ubuntu':
-                base = 'http://changelogs.ubuntu.com/'
+                subdir = name[0]
+            pkgversion = Version(self.getVersion()).strip_epoch()
+            base = 'http://packages.debian.org/'
 
             url = os.path.join(base, 'changelogs', 'pool',
-                               self.component, subdir, self.name,
-                               self.name + '_' + pkgversion,
-                               'changelog' + extension)
+                               self.getComponent(), subdir, name,
+                               name + '_' + pkgversion,
+                               'changelog.txt')
             try:
                 self._changelog = urlopen(url).read()
             except HTTPError as error:
-                print(('%s: %s' % (url, error)), file=sys.stderr)
+                Logger.error('{}: {}'.format(url, error))
                 return None
 
         if since_version is None:
@@ -611,62 +1103,70 @@ class FakeSPPH(object):
             new_entries.append(unicode(block))
         return u''.join(new_entries)
 
+    def getBinaries(self, arch=None):
+        return [b.getBPPH() for b in self._pkg.getBinaryFiles(arch)]
 
-def rmadison(url, package, suite=None, arch=None):
-    "Call rmadison and parse the result"
-    cmd = ['rmadison', '-u', url]
-    if suite:
-        cmd += ['-s', suite]
-    if arch:
-        cmd += ['-a', arch]
-    cmd.append(package)
-    process = subprocess.Popen(cmd, stdout=subprocess.PIPE,
-                               stderr=subprocess.PIPE, close_fds=True)
-    output, error_output = process.communicate()
-    if process.wait() != 0:
-        if error_output:
-            Logger.error('rmadison failed with: %s', error_output)
-        else:
-            Logger.error('rmadison failed')
-        sys.exit(1)
-
-    # rmadison uses some shorthand
-    if suite:
-        suite = suite.replace('-proposed-updates', '-p-u')
-
-    # pylint bug: http://www.logilab.org/ticket/46273
-    # pylint: disable=E1103
-    for line in output.strip().splitlines():
-        # pylint: enable=E1103
-        pkg, ver, dist, archs = [x.strip() for x in line.split('|')]
-        comp = 'main'
-        if '/' in dist:
-            dist, comp = dist.split('/')
-        archs = set(x.strip() for x in archs.split(','))
-
-        # rmadison returns some results outside the requested set.
-        # It'll include backports, and when given an unknown suite,
-        # it ignores that argument
-        #
-        # some versions (2.14.1ubuntu0.1) of rmadison return 'sid' when
-        # asked about 'unstable'.  Others return 'unstable'.  Accept either.
-        if (suite and dist != suite and not
-                (suite == 'sid' and dist == 'unstable')):
-            continue
-
-        if 'source' in archs:
-            yield {
-                'source': pkg,
-                'version': ver,
-                'suite': dist,
-                'component': comp,
-            }
-        archs.discard('source')
-        if archs:
-            yield {
-                'binary': pkg,
-                'version': ver,
-                'suite': dist,
-                'component': comp,
-                'architectures': archs,
-            }
+
+class SnapshotBPPH(object):
+    """Provide the same interface as BinaryPackagePublishingHistory"""
+    def __init__(self, snapshot_binfile):
+        self._file = snapshot_binfile
+
+    # LP API defined fields
+    @property
+    def architecture_specific(self):
+        return self._file.arch != 'all'
+
+    @property
+    def binary_package_name(self):
+        return self.getPackageName()
+
+    @property
+    def binary_package_version(self):
+        return self.getVersion()
+
+    @property
+    def component_name(self):
+        return self.getComponent()
+
+    @property
+    def display_name(self):
+        return ("{name} {version} in {seriesandpocket}"
+                .format(name=self.getPackageName(),
+                        version=self.getVersion(),
+                        seriesandpocket=self._file.archive_name))
+
+    @property
+    def pocket(self):
+        try:
+            (r, p) = split_release_pocket(self._file.archive_name, default=None)
+        except PocketDoesNotExistError:
+            p = None
+        return p if p else 'Release'
+
+    # BPPH functions
+
+    @property
+    def arch(self):
+        return self._file.arch
+
+    def getSourcePackageName(self):
+        return self._file.source
+
+    def getPackageName(self):
+        return self._file.package_name
+
+    def getVersion(self):
+        return self._file.package_version
+
+    def getComponent(self):
+        return self._file.component
+
+    def getBuild(self):
+        return None
+
+    def getUrl(self):
+        return self._file.getUrl()
+
+    def getFileName(self):
+        return self._file.name
diff --git a/ubuntutools/config.py b/ubuntutools/config.py
index ba029fe..dfa53a3 100644
--- a/ubuntutools/config.py
+++ b/ubuntutools/config.py
@@ -37,10 +37,12 @@ class UDTConfig(object):
         'BUILDER': 'pbuilder',
         'DEBIAN_MIRROR': 'http://deb.debian.org/debian',
         'DEBSEC_MIRROR': 'http://security.debian.org',
+        'DEBIAN_DDEBS_MIRROR': 'http://debug.mirrors.debian.org/debian-debug',
         'LPINSTANCE': 'production',
         'MIRROR_FALLBACK': True,
         'UBUNTU_MIRROR': 'http://archive.ubuntu.com/ubuntu',
         'UBUNTU_PORTS_MIRROR': 'http://ports.ubuntu.com',
+        'UBUNTU_DDEBS_MIRROR': 'http://ddebs.ubuntu.com',
         'UPDATE_BUILDER': False,
         'WORKDIR': None,
         'KEYID': None,
diff --git a/ubuntutools/lp/libsupport.py b/ubuntutools/lp/libsupport.py
index 64f3cf4..73dbd4b 100644
--- a/ubuntutools/lp/libsupport.py
+++ b/ubuntutools/lp/libsupport.py
@@ -19,6 +19,8 @@
 #
 
 # Modules.
+import platform
+
 try:
     from urllib.parse import urlsplit, urlencode, urlunsplit
 except ImportError:
@@ -53,3 +55,25 @@ def translate_web_api(url, launchpad):
     query = urlencode(query)
     url = urlunsplit((scheme, netloc, api_path + path.lstrip("/"), query, fragment))
     return url
+
+
+def get_system_arch():
+    # get the launchpad-named arch for the current system arch
+    arch = platform.machine()
+
+    # python's arch strings don't quite match launchpad's
+    # x86
+    if arch == "x86_64":
+        return "amd64"
+    if arch == "i686":
+        return "i386"
+
+    # PPC
+    if arch == "ppc64le":
+        return "ppc64el"
+
+    # ARM
+    if arch == "aarch64":
+        return "arm64"
+
+    return arch
diff --git a/ubuntutools/lp/lpapicache.py b/ubuntutools/lp/lpapicache.py
index 601bea8..ccf5d7e 100644
--- a/ubuntutools/lp/lpapicache.py
+++ b/ubuntutools/lp/lpapicache.py
@@ -30,13 +30,15 @@ from __future__ import print_function
 import collections
 import sys
 
-from debian.changelog import Changelog, Version
+from debian.changelog import Changelog
 from httplib2 import Http, HttpLib2Error
 from launchpadlib.launchpad import Launchpad as LP
 from launchpadlib.errors import HTTPError
 from lazr.restfulclient.resource import Entry
 
+from ubuntutools.version import Version
 from ubuntutools.lp import (service, api_version)
+from ubuntutools.lp.libsupport import get_system_arch
 from ubuntutools.lp.udtexceptions import (AlreadyLoggedInError,
                                           ArchiveNotFoundException,
                                           ArchSeriesNotFoundException,
@@ -73,6 +75,7 @@ __all__ = [
     'Distribution',
     'DistributionSourcePackage',
     'DistroSeries',
+    'DistroArchSeries',
     'Launchpad',
     'PersonTeam',
     'SourcePackagePublishingHistory',
@@ -296,6 +299,12 @@ class DistroArchSeries(BaseWrapper):
     '''
     resource_type = 'distro_arch_series'
 
+    def getSeries(self):
+        '''
+        Get DistroSeries for this.
+        '''
+        return DistroSeries(self._lpobject.distroseries_link)
+
 
 class DistroSeries(BaseWrapper):
     '''
@@ -307,12 +316,16 @@ class DistroSeries(BaseWrapper):
         if "_architectures" not in self.__dict__:
             self._architectures = dict()
 
-    def getArchSeries(self, archtag):
+    def getArchSeries(self, archtag=None):
         '''
         Returns a DistroArchSeries object for an architecture passed by name
         (e.g. 'amd64').
+        If arch is not specified, get the DistroArchSeries for the system arch.
+        The special archtag 'all' will get the system arch.
         If the architecture is not found: raise ArchSeriesNotFoundException.
         '''
+        if not archtag or archtag == 'all':
+            archtag = get_system_arch()
         if archtag not in self._architectures:
             try:
                 architecture = DistroArchSeries(
@@ -338,16 +351,22 @@ class Archive(BaseWrapper):
         self._pkgset_uploaders = {}
         self._component_uploaders = {}
 
-    def getSourcePackage(self, name, series=None, pocket=None):
+    def getSourcePackage(self, name, series=None, pocket=None, version=None,
+                         status=None):
         '''
         Returns a SourcePackagePublishingHistory object for the most
         recent source package in the distribution 'dist', series and
         pocket.
 
         series defaults to the current development series if not specified.
+        series must be either a series name string, or DistroSeries object.
+
+        pocket may be a string or a list. If a list, the highest version
+        will be returned.  It defaults to all pockets except backports.
 
-        pocket may be a list, if so, the highest version will be returned.
-        It defaults to all pockets except backports.
+        version may be specified to get only the exact version requested.
+
+        status is optional, to restrict search to a given status only.
 
         If the requested source package doesn't exist a
         PackageNotFoundException is raised.
@@ -355,33 +374,48 @@ class Archive(BaseWrapper):
         return self._getPublishedItem(name, series, pocket, cache=self._srcpkgs,
                                       function='getPublishedSources',
                                       name_key='source_name',
-                                      wrapper=SourcePackagePublishingHistory)
+                                      wrapper=SourcePackagePublishingHistory,
+                                      version=version,
+                                      status=status,
+                                      binary=False)
 
-    def getBinaryPackage(self, name, archtag=None, series=None, pocket=None):
+    def getBinaryPackage(self, name, archtag=None, series=None, pocket=None,
+                         version=None, status=None):
         '''
         Returns a BinaryPackagePublishingHistory object for the most
         recent source package in the distribution 'dist', architecture
         'archtag', series and pocket.
 
         series defaults to the current development series if not specified.
+        series must be either a series name string, or DistroArchSeries object.
+
+        pocket may be a string or a list. If a list, the highest version
+        will be returned.  It defaults to all pockets except backports.
 
-        pocket may be a list, if so, the highest version will be returned.
-        It defaults to all pockets except backports.
+        version may be specified to get only the exact version requested.
+
+        status is optional, to restrict search to a given status only.
 
         If the requested binary package doesn't exist a
         PackageNotFoundException is raised.
         '''
-        if archtag is None:
-            archtag = []
         return self._getPublishedItem(name, series, pocket, archtag=archtag,
                                       cache=self._binpkgs,
                                       function='getPublishedBinaries',
                                       name_key='binary_name',
-                                      wrapper=BinaryPackagePublishingHistory)
+                                      wrapper=BinaryPackagePublishingHistory,
+                                      version=version,
+                                      status=status,
+                                      binary=True)
 
     def _getPublishedItem(self, name, series, pocket, cache,
-                          function, name_key, wrapper, archtag=None):
-        '''Common code between getSourcePackage and getBinaryPackage
+                          function, name_key, wrapper, archtag=None,
+                          version=None, status=None,
+                          binary=False):
+        '''
+        Common code between getSourcePackage and getBinaryPackage.
+
+        Don't use this directly.
         '''
         if pocket is None:
             pockets = frozenset(('Proposed', 'Updates', 'Security', 'Release'))
@@ -396,34 +430,44 @@ class Archive(BaseWrapper):
                                               pocket)
 
         dist = Distribution(self.distribution_link)
-        # Check if series is already a DistroSeries object or not
-        if not isinstance(series, DistroSeries):
-            if series:
-                series = dist.getSeries(series)
-            else:
-                series = dist.getDevelopmentSeries()
 
-        # getPublishedSources requires a distro_series, while
-        # getPublishedBinaries requires a distro_arch_series.
-        # If archtag is not None, I'll assume it's getPublishedBinaries.
-        if archtag is not None and archtag != []:
-            if not isinstance(archtag, DistroArchSeries):
+        arch_series = None
+
+        # please don't pass DistroArchSeries as archtag!
+        # but, the code was like that before so keep
+        # backwards compatibility.
+        if isinstance(archtag, DistroArchSeries):
+            series = archtag
+            archtag = None
+
+        if isinstance(series, DistroArchSeries):
+            arch_series = series
+            series = series.getSeries()
+        elif isinstance(series, DistroSeries):
+            pass
+        elif series:
+            series = dist.getSeries(series)
+        else:
+            series = dist.getDevelopmentSeries()
+
+        if binary:
+            if arch_series is None:
                 arch_series = series.getArchSeries(archtag=archtag)
-            else:
-                arch_series = archtag
+            if archtag is None:
+                archtag = arch_series.architecture_tag
 
-        if archtag is not None and archtag != []:
-            index = (name, series.name, archtag, pockets)
-        else:
-            index = (name, series.name, pockets)
+        index = (name, series.name, archtag, pockets, status, version)
 
         if index not in cache:
             params = {
                 name_key: name,
-                'status': 'Published',
                 'exact_match': True,
             }
-            if archtag is not None and archtag != []:
+
+            if status:
+                params['status'] = status
+
+            if binary:
                 params['distro_arch_series'] = arch_series()
             else:
                 params['distro_series'] = series()
@@ -431,6 +475,9 @@ class Archive(BaseWrapper):
             if len(pockets) == 1:
                 params['pocket'] = list(pockets)[0]
 
+            if version:
+                params['version'] = version
+
             records = getattr(self, function)(**params)
 
             latest = None
@@ -450,7 +497,7 @@ class Archive(BaseWrapper):
                     package_type = "package"
                 msg = ("The %s '%s' does not exist in the %s %s archive" %
                        (package_type, name, dist.display_name, self.name))
-                if archtag is not None and archtag != []:
+                if binary:
                     msg += " for architecture %s" % archtag
                 pockets = [series.name if pocket == 'Release'
                            else '%s-%s' % (series.name, pocket.lower())
@@ -532,13 +579,23 @@ class SourcePackagePublishingHistory(BaseWrapper):
     resource_type = 'source_package_publishing_history'
 
     def __init__(self, *args):
+        self._archive = None
         self._changelog = None
         self._binaries = None
+        self._distro_series = None
         # Don't share _builds between different
         # SourcePackagePublishingHistory objects
         if '_builds' not in self.__dict__:
             self._builds = dict()
 
+    def getDistroSeries(self):
+        '''
+        Return the DistroSeries.
+        '''
+        if not self._distro_series:
+            self._distro_series = DistroSeries(self._lpobject.distro_series_link)
+        return self._distro_series
+
     def getPackageName(self):
         '''
         Returns the source package name.
@@ -557,16 +614,33 @@ class SourcePackagePublishingHistory(BaseWrapper):
         '''
         return self._lpobject.component_name
 
+    def getSeriesName(self):
+        '''
+        Returns the series
+
+        Named getSeriesName() to avoid confusion with
+        getDistroSeries()
+        '''
+        return self.getDistroSeries().name
+
     def getSeriesAndPocket(self):
         '''
         Returns a human-readable release-pocket
         '''
-        series = DistroSeries(self._lpobject.distro_series_link)
-        release = series.name
-        if self._lpobject.pocket != 'Release':
-            release += '-' + self._lpobject.pocket.lower()
+        release = self.getSeriesName()
+        if self.pocket != 'Release':
+            release += '-' + self.pocket.lower()
         return release
 
+    def getArchive(self):
+        '''
+        Get this SPPH's archive.
+        '''
+        if not self._archive:
+            self._archive = Archive(self._lpobject.archive_link)
+
+        return self._archive
+
     def getChangelog(self, since_version=None):
         '''
         Return the changelog, optionally since a particular version
@@ -603,15 +677,56 @@ class SourcePackagePublishingHistory(BaseWrapper):
             new_entries.append(unicode(block))
         return u''.join(new_entries)
 
-    def getBinaries(self):
+    def getBinaries(self, arch=None):
         '''
-        Returns the resulting BinaryPackagePublishingHistorys
+        Returns the resulting BinaryPackagePublishingHistorys.
+        If arch is specified, only returns BPPH for that arch.
         '''
         if self._binaries is None:
-            self._binaries = [BinaryPackagePublishingHistory(bpph)
-                              for bpph in
-                              self._lpobject.getPublishedBinaries()]
-        return self._binaries
+            if self.status != "Pending" and self.status != "Published":
+                # great, this is an older version, so we can't use
+                # SPPH.getPublishedBinaries().  We have to do this the
+                # much more painful way, using SPPH.binaryFileUrls() to
+                # get the filenames and then parsing each one, and calling
+                # getPublishedBinaries() on the archive itself to get the
+                # BPPH url.
+                print("Older package version...this will take a while, sorry")
+                bpphs = []
+                archive = self.getArchive()
+                # This takes a while to query for each pkg, and it
+                # can be sped up for cases that are just looking for
+                # the binary file urls, by creating a 'lazy' loading
+                # BPPH created from the URL and other data used below
+                urls = self.binaryFileUrls()
+                print("Processing %d binary packages" % len(urls))
+                for url in urls:
+                    # strip out the URL leading text.
+                    filename = url.rsplit('/', 1)[1]
+                    # strip the file suffix
+                    name = filename.rsplit('.', 1)[0]
+                    # split into name, version, arch
+                    (n, v, a) = name.rsplit('_', 2)
+                    # Note we ignore the version, as it may be missing epoch
+                    params = {'name': n,
+                              'archtag': a,
+                              'series': self.getSeriesName(),
+                              'pocket': self.pocket,
+                              'version': self.getVersion(),
+                              'status': self.status}
+                    bpph = archive.getBinaryPackage(**params)
+                    bpphs.append(bpph)
+                    print('Lookup: %s          ' % filename, end='\r')
+                    sys.stdout.flush()
+                print('')
+            else:
+                bpphs = map(BinaryPackagePublishingHistory,
+                            self._lpobject.getPublishedBinaries())
+            self._binaries = bpphs
+
+        if not arch:
+            return list(self._binaries)
+
+        return [b for b in self._binaries if b.arch == arch]
 
     def _fetch_builds(self):
         '''Populate self._builds with the build records.'''
@@ -671,6 +786,22 @@ class BinaryPackagePublishingHistory(BaseWrapper):
     '''
     resource_type = 'binary_package_publishing_history'
 
+    def __init__(self, *args):
+        self._arch = None
+
+    @property
+    def arch(self):
+        if not self._arch:
+            das = DistroArchSeries(self._lpobject.distro_arch_series_link)
+            self._arch = das.architecture_tag
+        return self._arch
+
+    def getSourcePackageName(self):
+        '''
+        Returns the source package name.
+        '''
+        return self.getBuild().source_package_name
+
     def getPackageName(self):
         '''
         Returns the binary package name.
@@ -700,6 +831,54 @@ class BinaryPackagePublishingHistory(BaseWrapper):
             raise AttributeError("binaryFileUrls can only be found in lpapi "
                                  "devel, not 1.0. Login using devel to have it.")
 
+    def getBuild(self):
+        '''
+        Returns the original build of the binary package.
+        '''
+        return Build(self._lpobject.build_link)
+
+    def getUrl(self):
+        '''
+        Returns the original build URL of the binary package.
+        '''
+        return "{build}/+files/{filename}".format(build=self.getBuild().getUrl(),
+                                                  filename=self.getFileName())
+
+    def getFileVersion(self):
+        '''
+        Returns the file version, which is the package version without the epoch
+        '''
+        return Version(self.getVersion()).strip_epoch()
+
+    def getFileArch(self):
+        '''
+        Returns the file arch, which is 'all' if not arch-specific
+        '''
+        if bool(self._lpobject.architecture_specific):
+            return self.arch
+        else:
+            return 'all'
+
+    def getFileExt(self):
+        '''
+        Returns the file extension; "deb", "ddeb", or "udeb".
+        '''
+        if self.getPackageName().endswith("-dbgsym"):
+            return "ddeb"
+        elif self.getPackageName().endswith("-di"):
+            return "udeb"
+        else:
+            return "deb"
+
+    def getFileName(self):
+        '''
+        Returns the filename for this binary package.
+        '''
+        return "{name}_{version}_{arch}.{ext}".format(name=self.getPackageName(),
+                                                      version=self.getFileVersion(),
+                                                      arch=self.getFileArch(),
+                                                      ext=self.getFileExt())
+
 
 class MetaPersonTeam(MetaWrapper):
     @property
@@ -729,6 +908,7 @@ class PersonTeam(BaseWrapper):
 
     def __init__(self, *args):
         # Don't share _upload between different PersonTeams
+        self._ppas = None
         if '_upload' not in self.__dict__:
             self._upload = dict()
 
@@ -806,6 +986,12 @@ class PersonTeam(BaseWrapper):
 
         return canUpload
 
+    def getPPAs(self):
+        if not self._ppas:
+            ppas = Launchpad.load(self._lpobject.ppas_collection_link).entries
+            self._ppas = [Archive(a['self_link']) for a in ppas]
+        return self._ppas
+
 
 class Build(BaseWrapper):
     '''
@@ -816,6 +1002,16 @@ class Build(BaseWrapper):
     def __str__(self):
         return u'%s: %s' % (self.arch_tag, self.buildstate)
 
+    def getSourcePackagePublishingHistory(self):
+        link = self._lpobject.current_source_publication_link
+        if link:
+            return SourcePackagePublishingHistory(link)
+        else:
+            return None
+
+    def getUrl(self):
+        return self()
+
     def rescore(self, score):
         if self.can_be_rescored:
             self().rescore(score=score)
diff --git a/ubuntutools/pullpkg.py b/ubuntutools/pullpkg.py
new file mode 100755
index 0000000..4b94908
--- /dev/null
+++ b/ubuntutools/pullpkg.py
@@ -0,0 +1,281 @@
+# pullpkg.py -- pull package files for debian/ubuntu/uca
+#               modified from ../pull-lp-source and converted to module
+#
+# Copyright (C) 2008,      Iain Lane <[e-mail]>,
+#               2010-2011, Stefano Rivera <[e-mail]>
+#               2017,      Dan Streetman <[e-mail]>
+#
+# ##################################################################
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 3
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# See file /usr/share/common-licenses/GPL for more details.
+#
+# ##################################################################
+
+
+import re
+from argparse import ArgumentParser
+
+from distro_info import DebianDistroInfo
+
+from ubuntutools.archive import (UbuntuSourcePackage, DebianSourcePackage,
+                                 UbuntuCloudArchiveSourcePackage)
+from ubuntutools.config import UDTConfig
+from ubuntutools.lp.lpapicache import (Distribution, Launchpad)
+from ubuntutools.lp.udtexceptions import (SeriesNotFoundException,
+                                          PackageNotFoundException,
+                                          PocketDoesNotExistError)
+from ubuntutools.logger import Logger
+from ubuntutools.misc import split_release_pocket
+from ubuntutools.lp.libsupport import get_system_arch
+
+PULL_SOURCE = 'source'
+PULL_DEBS = 'debs'
+PULL_DDEBS = 'ddebs'
+PULL_UDEBS = 'udebs'
+PULL_LIST = 'list'
+
+VALID_PULLS = [PULL_SOURCE, PULL_DEBS, PULL_DDEBS, PULL_UDEBS, PULL_LIST]
+
+DISTRO_DEBIAN = 'debian'
+DISTRO_UBUNTU = 'ubuntu'
+DISTRO_UCA = 'uca'
+
+DISTRO_PKG_CLASS = {
+    DISTRO_DEBIAN: DebianSourcePackage,
+    DISTRO_UBUNTU: UbuntuSourcePackage,
+    DISTRO_UCA: UbuntuCloudArchiveSourcePackage,
+}
+VALID_DISTROS = DISTRO_PKG_CLASS.keys()
+
+
+class InvalidPullValueError(ValueError):
+    """ Thrown when --pull value is invalid """
+    pass
+
+
+class InvalidDistroValueError(ValueError):
+    """ Thrown when --distro value is invalid """
+    pass
+
+
+def create_argparser(default_pull=None, default_distro=None, default_arch=None):
+    help_default_pull = "What to pull: " + ", ".join(VALID_PULLS)
+    if default_pull:
+        help_default_pull += (" (default: %s)" % default_pull)
+    help_default_distro = "Pull from: " + ", ".join(VALID_DISTROS)
+    if default_distro:
+        help_default_distro += (" (default: %s)" % default_distro)
+    if not default_arch:
+        default_arch = get_system_arch()
+    help_default_arch = ("Get binary packages for arch (default: %s)" % default_arch)
+
+    parser = ArgumentParser()
+    parser.add_argument('-v', '--verbose', action='store_true',
+                        help="Print verbose/debug messages")
+    parser.add_argument('-d', '--download-only', action='store_true',
+                        help="Do not extract the source package")
+    parser.add_argument('-m', '--mirror', action='append',
+                        help='Preferred mirror(s)')
+    parser.add_argument('--no-conf', action='store_true',
+                        help="Don't read config files or environment variables")
+    parser.add_argument('-a', '--arch', default=default_arch,
+                        help=help_default_arch)
+    parser.add_argument('-p', '--pull', default=default_pull,
+                        help=help_default_pull)
+    parser.add_argument('-D', '--distro', default=default_distro,
+                        help=help_default_distro)
+    parser.add_argument('package', help="Package name to pull")
+    parser.add_argument('release', nargs='?', help="Release to pull from")
+    parser.add_argument('version', nargs='?', help="Package version to pull")
+    return parser
+
+
+def parse_pull(pull):
+    if not pull:
+        raise InvalidPullValueError("Must specify --pull")
+
+    # allow 'dbgsym' as alias for 'ddebs'
+    if pull == 'dbgsym':
+        Logger.debug("Pulling '%s' for '%s'", PULL_DDEBS, pull)
+        pull = PULL_DDEBS
+    # assume anything starting with 'bin' means 'debs'
+    if str(pull).startswith('bin'):
+        Logger.debug("Pulling '%s' for '%s'", PULL_DEBS, pull)
+        pull = PULL_DEBS
+    # verify pull action is valid
+    if pull not in VALID_PULLS:
+        raise InvalidPullValueError("Invalid pull action '%s'", pull)
+
+    return pull
+
+
+def parse_distro(distro):
+    if not distro:
+        raise InvalidDistroValueError("Must specify --distro")
+
+    distro = distro.lower()
+
+    # allow 'lp' for 'ubuntu'
+    if distro == 'lp':
+        Logger.debug("Using distro '%s' for '%s'", DISTRO_UBUNTU, distro)
+        distro = DISTRO_UBUNTU
+    # assume anything with 'cloud' is UCA
+    if re.match(r'.*cloud.*', distro):
+        Logger.debug("Using distro '%s' for '%s'", DISTRO_UCA, distro)
+        distro = DISTRO_UCA
+    # verify distro is valid
+    if distro not in VALID_DISTROS:
+        raise InvalidDistroValueError("Invalid distro '%s'", distro)
+
+    return distro
+
+
+def parse_release(release, distro):
+    if distro == DISTRO_UCA:
+        # UCA is special; it is specified UBUNTURELEASE-UCARELEASE or just
+        # UCARELEASE.  The user could also specify UCARELEASE-POCKET.  But UCA
+        # archives always correspond to only one UBUNTURELEASE, and UCA archives
+        # have only the Release pocket, so only UCARELEASE matters to us.
+        for r in release.split('-'):
+            if r in UbuntuCloudArchiveSourcePackage.getReleaseNames():
+                Logger.debug("Using UCA release '%s'", r)
+                return (r, None)
+        raise SeriesNotFoundException('UCA release {} not found.'.format(release))
+
+    # Check if release[-pocket] is specified
+    (release, pocket) = split_release_pocket(release, default=None)
+    Logger.debug("Parsed release '%s' pocket '%s'", release, pocket)
+
+    if distro == DISTRO_DEBIAN:
+        # This converts from the aliases like 'unstable'
+        debian_info = DebianDistroInfo()
+        codename = debian_info.codename(release)
+        if codename:
+            Logger.normal("Using release '%s' for '%s'", codename, release)
+            release = codename
+
+    try:
+        d = Distribution(distro)
+        Logger.debug("Distro '%s' is valid", distro)
+    except:
+        Logger.debug("Distro '%s' not valid", distro)
+        raise SeriesNotFoundException("Distro {} not found".format(distro))
+
+    # let SeriesNotFoundException flow up
+    d.getSeries(release)
+
+    Logger.debug("Using distro '%s' release '%s' pocket '%s'",
+                 distro, release, pocket)
+    return (release, pocket)
+
+
+def pull(options):
+    # required options asserted below
+    # 'release' and 'version' are optional strings
+    # 'mirror' is optional list of strings
+    assert hasattr(options, 'verbose')        # bool
+    assert hasattr(options, 'download_only')  # bool
+    assert hasattr(options, 'no_conf')        # bool
+    assert hasattr(options, 'arch')           # string
+    assert hasattr(options, 'pull')           # string
+    assert hasattr(options, 'distro')         # string
+    assert hasattr(options, 'package')        # string
+
+    Logger.set_verbosity(options.verbose)
+
+    Logger.debug("pullpkg options: %s", options)
+
+    pull = parse_pull(options.pull)
+
+    distro = parse_distro(options.distro)
+
+    config = UDTConfig(options.no_conf)
+
+    mirrors = []
+    if hasattr(options, 'mirror') and options.mirror:
+        mirrors += options.mirror
+    if pull == PULL_DDEBS:
+        ddebs_mirror = config.get_value(distro.upper() + '_DDEBS_MIRROR')
+        if ddebs_mirror:
+            mirrors.append(ddebs_mirror)
+    if mirrors:
+        Logger.debug("using mirrors %s", ", ".join(mirrors))
+
+    # Login anonymously to LP
+    Launchpad.login_anonymously()
+
+    package = options.package
+    release = getattr(options, 'release', None)
+    version = getattr(options, 'version', None)
+    pocket = None
+
+    if release:
+        try:
+            (release, pocket) = parse_release(release, distro)
+        except (SeriesNotFoundException, PocketDoesNotExistError):
+            Logger.debug("Param '%s' not valid series, must be version", release)
+            release, version = version, release
+            if release:
+                try:
+                    (release, pocket) = parse_release(release, distro)
+                except (SeriesNotFoundException, PocketDoesNotExistError):
+                    Logger.error("Can't find series for '%s' or '%s'",
+                                 release, version)
+                    raise
+
+    try:
+        pkgcls = DISTRO_PKG_CLASS[distro]
+        srcpkg = pkgcls(package=package, version=version,
+                        series=release, pocket=pocket,
+                        mirrors=mirrors)
+        spph = srcpkg.lp_spph
+    except PackageNotFoundException as e:
+        Logger.error(str(e))
+        raise
+
+    Logger.normal('Found %s-%s', spph.display_name, spph.pocket)
+
+    if pull == PULL_LIST:
+        Logger.normal("Source files:")
+        for f in srcpkg.dsc['Files']:
+            Logger.normal("  %s", f['name'])
+        Logger.normal("Binary files:")
+        for f in spph.getBinaries(options.arch):
+            Logger.normal("  %s", f.getFileName())
+        return
+
+    # allow DownloadError to flow up to caller
+    if pull == PULL_SOURCE:
+        srcpkg.pull()
+        if options.download_only:
+            Logger.debug("--download-only specified, not extracting")
+        else:
+            srcpkg.unpack()
+    else:
+        name = '.*'
+        if package != spph.getPackageName():
+            Logger.normal("Pulling binary package '%s'", package)
+            Logger.normal("Use package name '%s' to pull all binary packages",
+                          spph.getPackageName())
+            name = package
+        if pull == PULL_DEBS:
+            name = r'{}(?<!-di)(?<!-dbgsym)$'.format(name)
+        elif pull == PULL_DDEBS:
+            name += '-dbgsym$'
+        elif pull == PULL_UDEBS:
+            name += '-di$'
+        else:
+            raise InvalidPullValueError("Invalid pull value %s", pull)
+        total = srcpkg.pull_binaries(name=name, arch=options.arch)
+        if total < 1:
+            Logger.error("No %s found for %s", pull, spph.display_name)
diff --git a/ubuntutools/requestsync/mail.py b/ubuntutools/requestsync/mail.py
index e3388f2..a79c9c3 100644
--- a/ubuntutools/requestsync/mail.py
+++ b/ubuntutools/requestsync/mail.py
@@ -29,10 +29,10 @@ import smtplib
 import socket
 import tempfile
 
-from debian.changelog import Changelog, Version
+from debian.changelog import Changelog
 from distro_info import DebianDistroInfo, DistroDataOutdated
 
-from ubuntutools.archive import rmadison, FakeSPPH
+from ubuntutools.archive import DebianSourcePackage, UbuntuSourcePackage
 from ubuntutools.lp.udtexceptions import PackageNotFoundException
 from ubuntutools.logger import Logger
 from ubuntutools.question import confirmation_prompt, YesNoQuestion
@@ -53,32 +53,21 @@ __all__ = [
 ]
 
 
-def _get_srcpkg(distro, name, release):
-    if distro == 'debian':
-        # Canonicalise release:
-        debian_info = DebianDistroInfo()
-        try:
-            codename = debian_info.codename(release, default=release)
-        except DistroDataOutdated as e:
-            Logger.warn(e)
-
-    lines = list(rmadison(distro, name, suite=codename, arch='source'))
-    if not lines:
-        lines = list(rmadison(distro, name, suite=release, arch='source'))
-        if not lines:
-            raise PackageNotFoundException("'%s' doesn't appear to exist in %s '%s'" %
-                                           (name, distro.capitalize(), release))
-    pkg = max(lines, key=lambda x: Version(x['version']))
-
-    return FakeSPPH(pkg['source'], pkg['version'], pkg['component'], distro)
-
-
 def get_debian_srcpkg(name, release):
-    return _get_srcpkg('debian', name, release)
+    # Canonicalise release:
+    debian_info = DebianDistroInfo()
+    try:
+        codename = debian_info.codename(release, default=release)
+        return DebianSourcePackage(package=name, series=codename).lp_spph
+    except DistroDataOutdated as e:
+        Logger.warn(e)
+    except PackageNotFoundException:
+        pass
+    return DebianSourcePackage(package=name, series=release).lp_spph
 
 
 def get_ubuntu_srcpkg(name, release):
-    return _get_srcpkg('ubuntu', name, release)
+    return UbuntuSourcePackage(package=name, series=release).lp_spph
 
 
 def need_sponsorship(name, component, release):
diff --git a/ubuntutools/test/test_archive.py b/ubuntutools/test/test_archive.py
index 38c1a63..43222c9 100644
--- a/ubuntutools/test/test_archive.py
+++ b/ubuntutools/test/test_archive.py
@@ -97,7 +97,6 @@ class LocalSourcePackageTestCase(unittest.TestCase):
         self.workdir = tempfile.mkdtemp(prefix='udt-test')
 
         self._stubout('ubuntutools.archive.Distribution')
-        self._stubout('ubuntutools.archive.rmadison')
 
         self.mock_http = self._stubout('httplib2.Http.request')
         self.mock_http.side_effect = self.request_proxy
@@ -246,8 +245,6 @@ class DebianLocalSourcePackageTestCase(LocalSourcePackageTestCase):
         sequence = [self.urlopen_null,
                     self.urlopen_404,
                     self.urlopen_404,
-                    self.urlopen_404,
-                    self.urlopen_404,
                     lambda x: BytesIO(
                         b'{"fileinfo": {"hashabc": [{"name": "example_1.0.orig.tar.gz"}]}}'),
                     self.urlopen_file('example_1.0.orig.tar.gz'),