Merge lp:~jelmer/bzr-builddeb/multiple-upstream-tarballs-pt4 into lp:bzr-builddeb

[Jelmer Vernooij]

Part 4 on the way to support multiple upstream tarballs.

* Move the main code for importing a single upstream tarballs to PristineTarSource.
* Make various methods that call import_upstream{,_tarballs} cope with it returning a list of (component, tag_name, revid) tuples rather than a single tag and revision id

Next is actually supporting non-base tarball imports in PristineTarSource.import_component_tarball.

[James Westby (james-w)]

Hi,

This looks fine.

What's your plan for representing the multiple upstream tarballs in
the tree and revision history?

Thanks,

James

review: Approve

[Jelmer Vernooij (jelmer)]

On 06/24/2011 02:17 AM, James Westby wrote:
> Review: Approve
> Hi,
>
> This looks fine.
Appreciated, as always :)
>
> What's your plan for representing the multiple upstream tarballs in
> the tree and revision history?

Basically importing each upstream component tarball as a separate
revision with its own history and tagging them appropriately:

  * "upstream-1.0" for bar_1.0.orig.tar.bz2
  * "upstream-1.0/bla" for bar_1.0.orig-bla.tar.bz2

... and then having the package branch take these revisions as parents,
making the components look like by-value nested trees. If an upstream
component tarball hasn't changed between upstream releases we just retag
it.

Does that sounds reasonable?

I'm also wondering if we should add a "deb-components" revision property
so that we don't accidentally forget to export component tarballs if the
appropriate tags are missing. What do you think?

Cheers,

Jelmer

[James Westby (james-w)]

On Fri, 24 Jun 2011 02:39:45 +0200, Jelmer Vernooij <email address hidden> wrote:
> Basically importing each upstream component tarball as a separate
> revision with its own history and tagging them appropriately:
>
> * "upstream-1.0" for bar_1.0.orig.tar.bz2
> * "upstream-1.0/bla" for bar_1.0.orig-bla.tar.bz2
>
> ... and then having the package branch take these revisions as parents,
> making the components look like by-value nested trees. If an upstream
> component tarball hasn't changed between upstream releases we just retag
> it.

That sounds pretty good to me.

I wonder what the merge-upstream command line will look like to do all
this :-)

> I'm also wondering if we should add a "deb-components" revision property
> so that we don't accidentally forget to export component tarballs if the
> appropriate tags are missing. What do you think?

That sounds like a good idea, but I don't know if we'll be able to set
it in all circumstances. I think if it can be set then we should and
should use it.

Thanks,

James

[Jelmer Vernooij (jelmer)]

On 24/06/11 03:12, James Westby wrote:
> On Fri, 24 Jun 2011 02:39:45 +0200, Jelmer Vernooij<email address hidden> wrote:
>> Basically importing each upstream component tarball as a separate
>> revision with its own history and tagging them appropriately:
>>
>> * "upstream-1.0" for bar_1.0.orig.tar.bz2
>> * "upstream-1.0/bla" for bar_1.0.orig-bla.tar.bz2
>>
>> ... and then having the package branch take these revisions as parents,
>> making the components look like by-value nested trees. If an upstream
>> component tarball hasn't changed between upstream releases we just retag
>> it.
> That sounds pretty good to me.
>
> I wonder what the merge-upstream command line will look like to do all
> this :-)
Yeah, that will be nasty. :-( Doing it as part of import-dsc (and udd)
is a lot easier as we can just read everything from the .dsc file.

Like the merge-upstream command-line, we'll have to update the
configuration file to support multiple upstream branches and multiple
upstream tag schemes.

>> I'm also wondering if we should add a "deb-components" revision property
>> so that we don't accidentally forget to export component tarballs if the
>> appropriate tags are missing. What do you think?
> That sounds like a good idea, but I don't know if we'll be able to set
> it in all circumstances. I think if it can be set then we should and
> should use it.
Cool, I'll add it.

Cheers,

Jelmer

=== modified file '__init__.py'
--- __init__.py	2011-06-03 14:42:26 +0000
+++ __init__.py	2011-06-23 21:47:20 +0000
@@ -24,7 +24,6 @@
 """bzr-builddeb - manage packages in a Bazaar branch."""
 
 import os
-import re
 
 import bzrlib
 from bzrlib.commands import plugin_cmds

=== modified file 'cmds.py'
--- cmds.py	2011-06-21 20:55:13 +0000
+++ cmds.py	2011-06-23 21:47:20 +0000
@@ -108,6 +108,7 @@
 from bzrlib.plugins.builddeb.util import (
         FORMAT_3_0_QUILT,
         FORMAT_3_0_NATIVE,
+        component_from_orig_tarball,
         debuild_config,
         dget_changes,
         find_changelog,
@@ -560,7 +561,9 @@
         db = DistributionBranch(tree.branch, tree.branch, tree=tree)
         dbs = DistributionBranchSet()
         dbs.add_branch(db)
-        conflicts = db.merge_upstream(tarball_filenames, package, version,
+        tarballs = [(p, component_from_orig_tarball(p, package, version)) for p
+                in tarball_filenames]
+        conflicts = db.merge_upstream(tarballs, package, version,
                 current_version, upstream_branch=upstream_branch,
                 upstream_revision=upstream_revision,
                 merge_type=merge_type, force=force)
@@ -588,7 +591,7 @@
             ret.append(tarball_filename)
         return ret
 
-    def _get_tarball(self, config, tree, package, version, upstream_branch,
+    def _get_tarballs(self, config, tree, package, version, upstream_branch,
             upstream_revision, v3, locations):
         orig_dir = config.orig_dir or default_orig_dir
         orig_dir = os.path.join(tree.basedir, orig_dir)
@@ -747,7 +750,7 @@
                 source_format = get_source_format(tree)
                 v3 = (source_format in [
                     FORMAT_3_0_QUILT, FORMAT_3_0_NATIVE])
-                tarball_filenames = self._get_tarball(config, tree, package,
+                tarball_filenames = self._get_tarballs(config, tree, package,
                     version, upstream_branch, upstream_revision, v3,
                     locations)
                 conflicts = self._do_merge(tree, tarball_filenames, package,
@@ -977,10 +980,16 @@
         else:
             raise BzrCommandError('bzr import-upstream --revision takes exactly'
                                   ' one revision specifier.')
-        tarballs = [(location, md5sum_filename(location))]
-        tag_name, _ = db.import_upstream_tarballs(tarballs, version, parents,
-            upstream_branch=upstream, upstream_revision=upstream_revid)
-        self.outf.write('Imported %s as tag:%s.\n' % (location, tag_name))
+        tarballs = [(location, None, md5sum_filename(location))]
+        for (component, tag_name, revid) in db.import_upstream_tarballs(
+                tarballs, None, version, parents, upstream_branch=upstream,
+                upstream_revision=upstream_revid):
+            if component is None:
+                self.outf.write('Imported %s as tag:%s.\n' % (
+                    location, tag_name))
+            else:
+                self.outf.write('Imported %s (%s) as tag:%s.\n' % (
+                    location, component, tag_name))
 
 
 class cmd_bd_do(Command):

=== modified file 'dh_make.py'
--- dh_make.py	2011-06-15 16:35:46 +0000
+++ dh_make.py	2011-06-23 21:47:20 +0000
@@ -70,8 +70,8 @@
             orig_dir, [])
     orig_files = provider.provide(os.path.join(tree.basedir, ".."))
     ret = []
-    for filename in orig_files:
-        ret.append((filename, util.md5sum_filename(filename)))
+    for filename, component in orig_files:
+        ret.append((filename, component, util.md5sum_filename(filename)))
     return ret
 
 
@@ -87,7 +87,8 @@
             pristine_upstream_tree=tree)
     dbs = import_dsc.DistributionBranchSet()
     dbs.add_branch(db)
-    db.import_upstream_tarballs(tarball_filenames, version, parents)
+    db.import_upstream_tarballs(tarball_filenames, package_name, version,
+        parents)
     return tree
 
 

=== modified file 'import_dsc.py'
--- import_dsc.py	2011-06-15 16:35:46 +0000
+++ import_dsc.py	2011-06-23 21:47:20 +0000
@@ -49,7 +49,6 @@
         AlreadyBranchError,
         BzrCommandError,
         NotBranchError,
-        NoSuchRevision,
         NoWorkingTree,
         UnrelatedBranches,
         )
@@ -61,7 +60,6 @@
 
 from bzrlib.plugins.builddeb.bzrtools_import import import_dir
 from bzrlib.plugins.builddeb.errors import (
-    MultipleUpstreamTarballsNotSupported,
     UpstreamAlreadyImported,
     UpstreamBranchAlreadyMerged,
     )
@@ -69,6 +67,7 @@
     FORMAT_1_0,
     FORMAT_3_0_QUILT,
     FORMAT_3_0_NATIVE,
+    component_from_orig_tarball,
     extract_orig_tarballs,
     get_commit_info_from_changelog,
     md5sum_filename,
@@ -77,9 +76,6 @@
     safe_decode,
     subprocess_setup,
     )
-from bzrlib.plugins.builddeb.upstream.branch import (
-    UpstreamBranchSource,
-    )
 from bzrlib.plugins.builddeb.upstream.pristinetar import (
     PristineTarSource,
     )
@@ -475,13 +471,8 @@
         tag_name = self.pristine_upstream_source.tag_name(version)
         if revid is None:
             revid = self.pristine_upstream_branch.last_revision()
-        self.pristine_upstream_branch.tags.set_tag(tag_name, revid)
-        try:
-            self.branch.repository.fetch(self.pristine_upstream_branch.repository,
-                revision_id=revid)
-        except NoSuchRevision:
-            # See bug lp:574223
-            pass
+        self.pristine_upstream_source.tag_version(version, revid)
+        self.branch.fetch(self.pristine_upstream_branch)
         self.branch.tags.set_tag(tag_name, revid)
         return tag_name, revid
 
@@ -724,7 +715,7 @@
             "Can't pull upstream with no tree"
         self.pristine_upstream_tree.pull(up_pull_branch,
                 stop_revision=pull_revision)
-        self.tag_upstream_version(version, revid=pull_revision)
+        self.pristine_upstream_source.tag_version(version, pull_revision)
         self.branch.fetch(self.pristine_upstream_branch, last_revision=pull_revision)
         self.pristine_upstream_branch.tags.merge_to(self.branch.tags)
 
@@ -828,8 +819,8 @@
         self.branch.fetch(self.pristine_upstream_branch, last_revision=revid)
         self.pristine_upstream_branch.tags.merge_to(self.branch.tags)
 
-    def import_upstream(self, upstream_part, version, upstream_parents,
-            upstream_tarballs=None, upstream_branch=None,
+    def import_upstream(self, upstream_part, package, version, upstream_parents,
+            upstream_tarballs, upstream_branch=None,
             upstream_revision=None, timestamp=None, author=None,
             file_ids_from=None):
         """Import an upstream part on to the upstream branch.
@@ -843,7 +834,7 @@
         :param upstream_parents: the parents to give the upstream revision
         :param timestamp: a tuple of (timestamp, timezone) to use for
             the commit, or None to use the current time.
-        :return: (tag_name, revision_id) of the imported tarball.
+        :return: list with (component, tag, revid) tuples
         """
         # Should we just dump the upstream part on whatever is currently
         # there, or try and pull all of the other upstream versions
@@ -859,12 +850,10 @@
             parent_revid = upstream_parents[0]
         else:
             parent_revid = NULL_REVISION
-        self.pristine_upstream_tree.pull(self.pristine_upstream_tree.branch, overwrite=True,
-                stop_revision=parent_revid)
+        self.pristine_upstream_tree.pull(self.pristine_upstream_tree.branch,
+            overwrite=True, stop_revision=parent_revid)
         other_branches = self.get_other_branches()
-        def get_last_revision_tree(br):
-            return br.repository.revision_tree(br.last_revision())
-        upstream_trees = [get_last_revision_tree(o.pristine_upstream_branch)
+        upstream_trees = [o.pristine_upstream_branch.basis_tree()
             for o in other_branches]
         target_tree = None
         if upstream_branch is not None:
@@ -890,33 +879,23 @@
                     target_tree=target_tree)
         finally:
             self_tree.unlock()
-        self.pristine_upstream_tree.set_parent_ids(upstream_parents)
         revprops = {}
-        if upstream_tarballs is not None:
-            if len(upstream_tarballs) != 1:
-                raise MultipleUpstreamTarballsNotSupported()
-            (upstream_tarball, md5) = upstream_tarballs[0]
-            revprops["deb-md5"] = md5
-            delta_revprops = self.pristine_upstream_source.create_delta_revprops(
-                self.pristine_upstream_tree, upstream_tarball)
-            revprops.update(delta_revprops)
-        if author is not None:
-            revprops['authors'] = author
-        timezone = None
-        if timestamp is not None:
-            timezone = timestamp[1]
-            timestamp = timestamp[0]
-        revid = self.pristine_upstream_tree.commit("Import upstream version %s" \
-                % (version,),
-                revprops=revprops, timestamp=timestamp, timezone=timezone)
-        tag_name, _ = self.tag_upstream_version(version, revid=revid)
-        return tag_name, revid
+        ret = []
+        for (tarball, component, md5) in upstream_tarballs:
+            (tag, revid) = self.pristine_upstream_source.import_component_tarball(
+                package, version, self.pristine_upstream_tree, component,
+                md5, tarball, author=author, timestamp=timestamp,
+                parent_ids=upstream_parents)
+            ret.append((component, tag, revid))
+            self.branch.fetch(self.pristine_upstream_branch)
+            self.branch.tags.set_tag(tag, revid)
+        return ret
 
-    def import_upstream_tarballs(self, tarballs, version, parents,
+    def import_upstream_tarballs(self, tarballs, package, version, parents,
         upstream_branch=None, upstream_revision=None):
         """Import an upstream part to the upstream branch.
 
-        :param tarball_filename: The tarball to import.
+        :param tarballs: List of tarballs / components to extract
         :param version: The upstream version to import.
         :param parents: The tarball-branch parents to use for the import.
             If an upstream branch is supplied, its automatically added to
@@ -925,11 +904,11 @@
             tarball.
         :param upstream_revision: Upstream revision id
         :param md5sum: hex digest of the md5sum of the tarball, if known.
-        :return: (tag_name, revision_id) of the imported tarball.
+        :return: list with (component, tag, revid) tuples
         """
         tarball_dir = self._extract_tarballs_to_tempdir(tarballs)
         try:
-            return self.import_upstream(tarball_dir, version, parents,
+            return self.import_upstream(tarball_dir, package, version, parents,
                 tarballs,
                 upstream_branch=upstream_branch,
                 upstream_revision=upstream_revision)
@@ -944,8 +923,7 @@
         poss_native_tree = self.get_branch_tip_revtree()
         current_native = self._is_tree_native(poss_native_tree)
         current_config = self._default_config_for_tree(poss_native_tree)
-        dirname = os.path.join(self.tree.basedir,
-                '.bzr-builddeb')
+        dirname = os.path.join(self.tree.basedir, '.bzr-builddeb')
         if current_config is not None:
             # Add that back to the current tree
             if not os.path.exists(dirname):
@@ -1022,10 +1000,7 @@
                    "as the single parent")
             parents = [self.branch.last_revision()]
         other_branches = self.get_other_branches()
-        def get_last_revision_tree(br):
-            return br.repository.revision_tree(br.last_revision())
-        debian_trees = [get_last_revision_tree(o.branch)
-            for o in other_branches]
+        debian_trees = [o.branch.basis_tree() for o in other_branches]
         parent_trees = []
         if file_ids_from is not None:
             parent_trees = file_ids_from[:]
@@ -1169,13 +1144,13 @@
                     # from another branch:
                     upstream_parents = self.upstream_parents(package, versions,
                             version.upstream_version)
-                    _, new_revid = self.import_upstream(upstream_part,
-                            version.upstream_version,
+                    for (component, tag, revid) in self.import_upstream(upstream_part,
+                            package, version.upstream_version,
                             upstream_parents,
                             upstream_tarballs=upstream_tarballs,
                             timestamp=timestamp, author=author,
-                            file_ids_from=file_ids_from)
-                    self._fetch_upstream_to_branch(new_revid)
+                            file_ids_from=file_ids_from):
+                        self._fetch_upstream_to_branch(revid)
             else:
                 mutter("We already have the needed upstream part")
             parents = self.get_parents_with_upstream(package, version, versions,
@@ -1356,8 +1331,9 @@
     def _extract_tarballs_to_tempdir(self, tarballs):
         tempdir = tempfile.mkdtemp()
         try:
-            extract_orig_tarballs([fn for (fn, md5) in tarballs], tempdir,
-                strip_components=1)
+            extract_orig_tarballs(
+                [(fn, component) for (fn, component, md5) in tarballs],
+                tempdir, strip_components=1)
             return tempdir
         except:
             shutil.rmtree(tempdir)
@@ -1405,7 +1381,8 @@
                             self.branch.last_revision()):
                         raise UpstreamBranchAlreadyMerged
                 upstream_tarballs = [
-                    (os.path.abspath(fn), md5sum_filename(fn)) for fn in
+                    (os.path.abspath(fn), component, md5sum_filename(fn)) for
+                    (fn, component) in
                     tarball_filenames]
                 tarball_dir = self._extract_tarballs_to_tempdir(upstream_tarballs)
                 try:
@@ -1413,11 +1390,11 @@
                     parents = []
                     if self.pristine_upstream_branch.last_revision() != NULL_REVISION:
                         parents = [self.pristine_upstream_branch.last_revision()]
-                    _, new_revid = self.import_upstream(tarball_dir,
-                            version, parents, upstream_tarballs=upstream_tarballs,
+                    for (component, tag, revid) in self.import_upstream(tarball_dir,
+                            package, version, parents, upstream_tarballs=upstream_tarballs,
                             upstream_branch=upstream_branch,
-                            upstream_revision=upstream_revision)
-                    self._fetch_upstream_to_branch(new_revid)
+                            upstream_revision=upstream_revision):
+                        self._fetch_upstream_to_branch(revid)
                 finally:
                     shutil.rmtree(tarball_dir)
                 if self.branch.last_revision() != NULL_REVISION:
@@ -1502,7 +1479,9 @@
                 if part['name'].endswith(".orig.tar.gz"):
                     self.upstream_tarballs.append((os.path.abspath(
                             os.path.join(osutils.dirname(self.dsc_path),
-                                part['name'])), part['md5sum']))
+                                part['name'])),
+                            component_from_orig_tarball(part['name'], name, str(version.upstream_version)),
+                            part['md5sum']))
                 elif part['name'].endswith(".diff.gz"):
                     self.unextracted_debian_md5 = part['md5sum']
 

=== modified file 'source_distiller.py'
--- source_distiller.py	2011-06-15 01:28:19 +0000
+++ source_distiller.py	2011-06-23 21:47:20 +0000
@@ -25,11 +25,11 @@
 from bzrlib import errors as bzr_errors
 
 from bzrlib.plugins.builddeb.util import (
-        export,
-        extract_orig_tarballs,
-        get_parent_dir,
-        recursive_copy,
-        )
+    export,
+    extract_orig_tarballs,
+    get_parent_dir,
+    recursive_copy,
+    )
 
 
 class SourceDistiller(object):

=== modified file 'tests/blackbox/test_merge_upstream.py'
--- tests/blackbox/test_merge_upstream.py	2011-06-15 16:35:46 +0000
+++ tests/blackbox/test_merge_upstream.py	2011-06-23 21:47:20 +0000
@@ -95,7 +95,9 @@
         dbs = DistributionBranchSet()
         dbs.add_branch(db)
         db.import_upstream_tarballs(
-            [(self.tar.tarball, md5sum_filename(self.tar.tarball))], str(self.tar.version),
+            [(self.tar.tarball, None, md5sum_filename(self.tar.tarball))],
+            "foo",
+            str(self.tar.version),
             [tree.branch.last_revision()])
         package_builder = SourcePackageBuilder("foo",
             str(self.tar.version)+"-1")

=== modified file 'tests/test_import_dsc.py'
--- tests/test_import_dsc.py	2011-06-15 16:35:46 +0000
+++ tests/test_import_dsc.py	2011-06-23 21:47:20 +0000
@@ -173,26 +173,26 @@
         version = "0.1"
         self.assertFalse(db.has_upstream_version("package", version))
         self.assertFalse(db.has_upstream_version("package", version,
-            [("foo.tar.gz", self.fake_md5_1)]))
+            [("foo.tar.gz", None, self.fake_md5_1)]))
         self.do_commit_with_md5(self.up_tree1, "one", self.fake_md5_1)
         db.tag_upstream_version(version)
         self.assertTrue(db.has_upstream_version("package", version))
         self.assertTrue(db.has_upstream_version("package",
-            version, [("foo.tar.gz", self.fake_md5_1)]))
+            version, [("foo.tar.gz", None, self.fake_md5_1)]))
         self.assertFalse(db.has_upstream_version("package", version,
-            [("foo.tar.gz", self.fake_md5_2)]))
+            [("foo.tar.gz", None, self.fake_md5_2)]))
         version = "0.1"
         self.assertTrue(db.has_upstream_version("package", version))
         self.assertTrue(db.has_upstream_version("package", version,
-            [("foo.tar.gz", self.fake_md5_1)]))
+            [("foo.tar.gz", None, self.fake_md5_1)]))
         self.assertFalse(db.has_upstream_version("package", version,
-            [("foo.tar.gz", self.fake_md5_2)]))
+            [("foo.tar.gz", None, self.fake_md5_2)]))
         version = "0.2"
         self.assertFalse(db.has_upstream_version("package", version))
         self.assertFalse(db.has_upstream_version("package", version,
-            [("foo.tar.gz", self.fake_md5_1)]))
+            [("foo.tar.gz", None, self.fake_md5_1)]))
         self.assertFalse(db.has_upstream_version("package", version,
-            [("foo.tar.gz", self.fake_md5_2)]))
+            [("foo.tar.gz", None, self.fake_md5_2)]))
 
     def test_revid_of_version(self):
         db = self.db1
@@ -640,48 +640,48 @@
         version2 = Version("0.2-1")
         # With no versions tagged everything is None
         branch = self.db2.branch_to_pull_upstream_from("package",
-                version1.upstream_version, [("foo.tar.gz", self.fake_md5_1)])
+                version1.upstream_version, [("foo.tar.gz", None, self.fake_md5_1)])
         self.assertEqual(branch, None)
         branch = self.db2.branch_to_pull_upstream_from("package",
-                version1.upstream_version, [("foo.tar.gz", self.fake_md5_2)])
+                version1.upstream_version, [("foo.tar.gz", None, self.fake_md5_2)])
         self.assertEqual(branch, None)
         branch = self.db1.branch_to_pull_upstream_from("package",
-                version1.upstream_version, [("foo.tar.gz", self.fake_md5_1)])
+                version1.upstream_version, [("foo.tar.gz", None, self.fake_md5_1)])
         self.assertEqual(branch, None)
         self.do_commit_with_md5(self.up_tree1, "one", self.fake_md5_1)
         self.db1.tag_upstream_version(version1.upstream_version)
         # Version and md5 available, so we get the correct branch.
         branch = self.db2.branch_to_pull_upstream_from("package",
-                version1.upstream_version, [("foo.tar.gz", self.fake_md5_1)])
+                version1.upstream_version, [("foo.tar.gz", None, self.fake_md5_1)])
         self.assertEqual(branch, self.db1)
         # Otherwise (different version or md5) then we get None
         branch = self.db2.branch_to_pull_upstream_from("package",
-                version1.upstream_version, [("foo.tar.gz", self.fake_md5_2)])
-        self.assertEqual(branch, None)
-        branch = self.db2.branch_to_pull_upstream_from("package",
-                version2.upstream_version,
-                [("foo.tar.gz", self.fake_md5_1)])
-        self.assertEqual(branch, None)
-        branch = self.db2.branch_to_pull_upstream_from("package",
-                version2.upstream_version,
-                [("foo.tar.gz", self.fake_md5_2)])
+                version1.upstream_version, [("foo.tar.gz", None, self.fake_md5_2)])
+        self.assertEqual(branch, None)
+        branch = self.db2.branch_to_pull_upstream_from("package",
+                version2.upstream_version,
+                [("foo.tar.gz", None, self.fake_md5_1)])
+        self.assertEqual(branch, None)
+        branch = self.db2.branch_to_pull_upstream_from("package",
+                version2.upstream_version,
+                [("foo.tar.gz", None, self.fake_md5_2)])
         self.assertEqual(branch, None)
         # And we don't get a branch for the one that already has
         # the version
         branch = self.db1.branch_to_pull_upstream_from("package",
-                version1.upstream_version, [("foo.tar.gz", self.fake_md5_1)])
+                version1.upstream_version, [("foo.tar.gz", None, self.fake_md5_1)])
         self.assertEqual(branch, None)
         self.up_tree2.pull(self.up_tree1.branch)
         self.db2.tag_upstream_version(version1.upstream_version)
         # And we get the greatest branch when two lesser branches
         # have what we are looking for.
         branch = self.db3.branch_to_pull_upstream_from("package",
-                version1.upstream_version, [("foo.tar.gz", self.fake_md5_1)])
+                version1.upstream_version, [("foo.tar.gz", None, self.fake_md5_1)])
         self.assertEqual(branch, self.db2)
         # If the branches have diverged then we don't get a branch.
         self.up_tree3.commit("three")
         branch = self.db3.branch_to_pull_upstream_from("package",
-                version1.upstream_version, [("foo.tar.gz", self.fake_md5_1)])
+                version1.upstream_version, [("foo.tar.gz", None, self.fake_md5_1)])
         self.assertEqual(branch, None)
 
     def test_pull_from_lesser_branch_no_upstream(self):
@@ -773,7 +773,8 @@
             contents = [(basedir + '/' + element[0],) + element[1:] for
                 element in contents]
         self.build_tree_contents(contents)
-        self.db1.import_upstream(basedir, version.upstream_version, [])
+        self.db1.import_upstream(basedir, "package", version.upstream_version,
+            [], [(None, None, None)])
         return version
 
     def test_import_upstream(self):
@@ -807,14 +808,15 @@
         write_to_file(os.path.join(basedir, "README"), "Hi\n")
         write_to_file(os.path.join(basedir, "BUGS"), "")
         write_to_file(os.path.join(basedir, "COPYING"), "")
-        self.db1.import_upstream(basedir, version1.upstream_version, [])
+        self.db1.import_upstream(basedir, "package", version1.upstream_version, [],
+            [(None, None, None)])
         basedir = name + "-" + str(version2.upstream_version)
         os.mkdir(basedir)
         write_to_file(os.path.join(basedir, "README"), "Now even better\n")
         write_to_file(os.path.join(basedir, "BUGS"), "")
         write_to_file(os.path.join(basedir, "NEWS"), "")
-        self.db1.import_upstream(basedir, version2.upstream_version,
-                [self.up_tree1.branch.last_revision()])
+        self.db1.import_upstream(basedir, "package", version2.upstream_version,
+                [self.up_tree1.branch.last_revision()], [(None, None, None)])
         tree = self.up_tree1
         branch = tree.branch
         rh = branch.revision_history()
@@ -845,8 +847,8 @@
             tf.add(basedir)
         finally:
             tf.close()
-        self.db1.import_upstream(basedir, version.upstream_version, [],
-                upstream_tarballs=[(os.path.abspath(tar_path), self.fake_md5_1)])
+        self.db1.import_upstream(basedir, "package", version.upstream_version, [],
+                upstream_tarballs=[(os.path.abspath(tar_path), None, self.fake_md5_1)])
         tree = self.up_tree1
         branch = tree.branch
         rh = branch.revision_history()
@@ -873,8 +875,8 @@
             tf.add(basedir)
         finally:
             tf.close()
-        self.db1.import_upstream(basedir, version.upstream_version,
-            [], upstream_tarballs=[(os.path.abspath(tar_path), self.fake_md5_1)])
+        self.db1.import_upstream(basedir, "package", version.upstream_version,
+            [], upstream_tarballs=[(os.path.abspath(tar_path), None, self.fake_md5_1)])
         tree = self.up_tree1
         branch = tree.branch
         rh = branch.revision_history()
@@ -1503,7 +1505,7 @@
             tf.add("a")
         finally:
             tf.close()
-        conflicts = db.merge_upstream([tarball_filename], "foo", "0.2", "0.1")
+        conflicts = db.merge_upstream([(tarball_filename, None)], "foo", "0.2", "0.1")
         self.assertEqual(0,  conflicts)
 
     def test_merge_upstream_initial_with_branch(self):
@@ -1540,7 +1542,7 @@
             tf.add("a")
         finally:
             tf.close()
-        conflicts = db.merge_upstream([tarball_filename], "foo", "0.2", "0.1",
+        conflicts = db.merge_upstream([(tarball_filename, None)], "foo", "0.2", "0.1",
             upstream_branch=upstream_tree.branch,
             upstream_revision=upstream_rev)
         self.assertEqual(0,  conflicts)
@@ -1580,7 +1582,7 @@
             tf.add("a")
         finally:
             tf.close()
-        conflicts = db.merge_upstream([tarball_filename], "foo", "0.2", "0.1",
+        conflicts = db.merge_upstream([(tarball_filename, None)], "foo", "0.2", "0.1",
             upstream_branch=upstream_tree.branch,
             upstream_revision=upstream_rev)
         # ./debian conflicts.
@@ -1621,7 +1623,7 @@
         dbs.add_branch(db)
         tree.lock_write()
         self.addCleanup(tree.unlock)
-        db.merge_upstream([builder.tar_name()], "package", str(version2),
+        db.merge_upstream([(builder.tar_name(), None)], "package", str(version2),
             version1.upstream_version,
             upstream_branch=upstream_tree.branch,
             upstream_revision=upstream_rev)
@@ -1660,7 +1662,7 @@
             tf.add("a")
         finally:
             tf.close()
-        conflicts = db.merge_upstream([tarball_filename], "package", "0.2-1",
+        conflicts = db.merge_upstream([(tarball_filename, None)], "package", "0.2-1",
             "0.1")
         # Check that we tagged wiht the dash version
         self.assertTrue(tree.branch.tags.has_tag('upstream-0.2-1'))
@@ -1699,7 +1701,7 @@
         # We don't add the new file upstream, as the new file id would
         # be picked up from there.
         upstream_rev2 = upstream_tree.commit("two")
-        db.merge_upstream([builder.tar_name()], "package",
+        db.merge_upstream([(builder.tar_name(), None)], "package",
             version2.upstream_version,
             version1.upstream_version,
             upstream_branch=upstream_tree.branch,
@@ -1736,7 +1738,7 @@
         # We don't add the new file upstream, as the new file id would
         # be picked up from there.
         upstream_rev2 = upstream_tree.commit("two")
-        db.merge_upstream([builder.tar_name()], "package",
+        db.merge_upstream([(builder.tar_name(), None)], "package",
             version2.upstream_version,
             version1.upstream_version,
             upstream_branch=upstream_tree.branch,
@@ -1769,7 +1771,7 @@
         builder.add_upstream_file("a", "New a")
         builder.add_upstream_file("b", "Renamed a")
         builder.build()
-        db.merge_upstream([builder.tar_name()], "packaging",
+        db.merge_upstream([(builder.tar_name(), None)], "packaging",
             version2.upstream_version, version1.upstream_version)
         self.assertEqual("a-id", packaging_tree.path2id("b"))
         self.assertEqual("other-a-id", packaging_tree.path2id("a"))

=== modified file 'tests/test_upstream.py'
--- tests/test_upstream.py	2011-06-14 14:07:21 +0000
+++ tests/test_upstream.py	2011-06-23 21:47:20 +0000
@@ -57,6 +57,9 @@
     UScanSource,
     extract_tarball_version,
     )
+from bzrlib.plugins.builddeb.util import (
+    component_from_orig_tarball,
+    )
 from bzrlib.plugins.builddeb.upstream.branch import (
     get_export_upstream_revision,
     get_snapshot_revision,
@@ -803,7 +806,8 @@
         paths = (self.already_exists_in_target(target_dir)
                 or self.provide_from_store_dir(target_dir))
         if paths is not None:
-            return paths
+            return [(p, component_from_orig_tarball(p, self.package,
+                self.version)) for p in paths]
         raise MissingUpstreamTarball(self.package, self.version)
 
 

=== modified file 'tests/test_util.py'
--- tests/test_util.py	2011-06-15 15:33:08 +0000
+++ tests/test_util.py	2011-06-23 21:47:20 +0000
@@ -38,10 +38,10 @@
     BUILD_TYPE_NATIVE,
     BUILD_TYPE_NORMAL,
     )
-from bzrlib.plugins.builddeb.errors import (MissingChangelogError,
+from bzrlib.plugins.builddeb.errors import (
+    MissingChangelogError,
     AddChangelogError,
     InconsistentSourceFormatError,
-    MultipleUpstreamTarballsNotSupported,
     NoPreviousUpload,
     )
 from bzrlib.plugins.builddeb.tests import (
@@ -50,6 +50,7 @@
     TestCaseWithTransport,
     )
 from bzrlib.plugins.builddeb.util import (
+    component_from_orig_tarball,
     dget,
     dget_changes,
     extract_orig_tarballs,
@@ -853,8 +854,10 @@
                 f.write("Hi\n")
             finally:
                 f.close()
-            tar_path = os.path.abspath("%s_%s.orig.tar.%s" % (package, version,
-                compression))
+            prefix = "%s_%s.orig" % (package, version)
+            if part is not None:
+                prefix += "-%s" % part
+            tar_path = os.path.abspath(prefix + ".tar." + compression)
             tf = tarfile.open(tar_path, 'w:%s' % compression)
             try:
                 tf.add(basedir)
@@ -867,19 +870,45 @@
     def test_single_orig_tar_gz(self):
         tar_path = self.create_tarball("package", "0.1", "gz")
         os.mkdir("target")
-        extract_orig_tarballs([tar_path], "target", strip_components=1)
+        extract_orig_tarballs([(tar_path, None)], "target",
+            strip_components=1)
         self.assertEquals(os.listdir("target"), ["README"])
 
     def test_single_orig_tar_bz2(self):
         tar_path = self.create_tarball("package", "0.1", "bz2")
         os.mkdir("target")
-        extract_orig_tarballs([tar_path], "target", strip_components=1)
+        extract_orig_tarballs([(tar_path, None)], "target",
+            strip_components=1)
         self.assertEquals(os.listdir("target"), ["README"])
 
     def test_multiple_tarballs(self):
         base_tar_path = self.create_tarball("package", "0.1", "bz2")
         tar_path_extra = self.create_tarball("package", "0.1", "bz2", part="extra")
         os.mkdir("target")
-        self.assertRaises(MultipleUpstreamTarballsNotSupported,
-            extract_orig_tarballs,
-            [base_tar_path, tar_path_extra], "target")
+        extract_orig_tarballs([(base_tar_path, None), (tar_path_extra, "extra")], "target",
+            strip_components=1)
+        self.assertEquals(sorted(os.listdir("target")),
+            sorted(["README", "extra"]))
+
+
+class ComponentFromOrigTarballTests(TestCase):
+
+    def test_base_tarball(self):
+        self.assertIs(None,
+            component_from_orig_tarball("foo_0.1.orig.tar.gz", "foo", "0.1"))
+        self.assertRaises(ValueError,
+            component_from_orig_tarball, "foo_0.1.orig.tar.gz", "bar", "0.1")
+
+    def test_invalid_extension(self):
+        self.assertRaises(ValueError,
+            component_from_orig_tarball, "foo_0.1.orig.unknown", "foo", "0.1")
+
+    def test_component(self):
+        self.assertEquals("comp",
+            component_from_orig_tarball("foo_0.1.orig-comp.tar.gz", "foo", "0.1"))
+        self.assertEquals("comp-dash",
+            component_from_orig_tarball("foo_0.1.orig-comp-dash.tar.gz", "foo", "0.1"))
+
+    def test_invalid_character(self):
+        self.assertRaises(ValueError,
+            component_from_orig_tarball, "foo_0.1.orig;.tar.gz", "foo", "0.1")

=== modified file 'upstream/__init__.py'
--- upstream/__init__.py	2011-06-16 21:02:32 +0000
+++ upstream/__init__.py	2011-06-23 21:47:20 +0000
@@ -44,6 +44,7 @@
     )
 from bzrlib.plugins.builddeb.repack_tarball import repack_tarball
 from bzrlib.plugins.builddeb.util import (
+    component_from_orig_tarball,
     export,
     tarball_name,
     )
@@ -395,7 +396,8 @@
         if in_target is not None:
             note("Upstream tarball already exists in build directory, "
                     "using that")
-            return in_target
+            return [(p, component_from_orig_tarball(p,
+                self.package, self.version)) for p in in_target]
         if self.already_exists_in_store() is None:
             if not os.path.exists(self.store_dir):
                 os.makedirs(self.store_dir)
@@ -410,7 +412,8 @@
                  self.store_dir)
         paths = self.provide_from_store_dir(target_dir)
         assert paths is not None
-        return paths
+        return [(p, component_from_orig_tarball(p, self.package, self.version))
+                for p in paths]
 
     def _gather_orig_files(self, path):
         prefix = "%s_%s.orig" % (self.package, self.version)

=== modified file 'upstream/pristinetar.py'
--- upstream/pristinetar.py	2011-06-15 15:08:01 +0000
+++ upstream/pristinetar.py	2011-06-23 21:47:20 +0000
@@ -130,6 +130,52 @@
             return "upstream-" + version
         return "upstream-%s-%s" % (distro, version)
 
+    def tag_version(self, version, revid):
+        """Tags the upstream branch's last revision with an upstream version.
+
+        Sets a tag on the last revision of the upstream branch and on the main
+        branch with a tag that refers to the upstream part of the version
+        provided.
+
+        :param version: the upstream part of the version number to derive the 
+            tag name from.
+        :param revid: the revid to associate the tag with, or None for the
+            tip of self.pristine_upstream_branch.
+        :return The tag name, revid of the added tag.
+        """
+        assert isinstance(version, str)
+        tag_name = self.tag_name(version)
+        self.branch.tags.set_tag(tag_name, revid)
+        return tag_name, revid
+
+    def import_component_tarball(self, package, version, tree, component=None,
+            md5=None, tarball=None, author=None, timestamp=None,
+            parent_ids=None):
+        """Import a tarball.
+
+        :param package: Package name
+        :param version: Upstream version
+        :param component: Component name (None for base)
+        """
+        if component is not None:
+            raise BzrError("Importing non-base tarballs not yet supported")
+        tree.set_parent_ids(parent_ids)
+        revprops = {}
+        if md5 is not None:
+            revprops["deb-md5"] = md5
+            delta_revprops = self.create_delta_revprops(tree, tarball)
+            revprops.update(delta_revprops)
+        if author is not None:
+            revprops['authors'] = author
+        timezone = None
+        if timestamp is not None:
+            timezone = timestamp[1]
+            timestamp = timestamp[0]
+        revid = tree.commit("Import upstream version %s" % (version,),
+                revprops=revprops, timestamp=timestamp, timezone=timezone)
+        tag_name, _ = self.tag_version(version, revid=revid)
+        return tag_name, revid
+
     def fetch_tarball(self, package, version, target_dir):
         revid = self.version_as_revision(package, version)
         try:
@@ -166,7 +212,7 @@
             return True
         if len(tarballs) != 1:
             raise MultipleUpstreamTarballsNotSupported()
-        (filename, md5) = tarballs[0]
+        (filename, component, md5) = tarballs[0]
         rev = self.branch.repository.get_revision(revid)
         try:
             return rev.properties['deb-md5'] == md5

=== modified file 'util.py'
--- util.py	2011-06-15 01:28:19 +0000
+++ util.py	2011-06-23 21:47:20 +0000
@@ -66,7 +66,6 @@
     MissingChangelogError,
     AddChangelogError,
     InconsistentSourceFormatError,
-    MultipleUpstreamTarballsNotSupported,
     NoPreviousUpload,
     TarFailed,
     UnableToFindPreviousUpload,
@@ -685,24 +684,64 @@
         return BUILD_TYPE_NORMAL
 
 
-def extract_orig_tarballs(tarballs, target, strip_components=None):
-    """Extract orig tarballs to a directory.
-
-    :param tarballs: List of tarball filenames
-    :param target: Target directory (must already exist)
+def component_from_orig_tarball(tarball_filename, package, version):
+    tarball_filename = os.path.basename(tarball_filename)
+    prefix = "%s_%s.orig" % (package, version)
+    if not tarball_filename.startswith(prefix):
+        raise ValueError(
+            "invalid orig tarball file %s does not have expected prefix %s" % (
+                tarball_filename, prefix))
+    base = tarball_filename[len(prefix):]
+    for ext in (".tar.gz", ".tar.bz2", ".tar.lzma", ".tar.xz"):
+        if tarball_filename.endswith(ext):
+            base = base[:-len(ext)]
+            break
+    else:
+        raise ValueError(
+            "orig tarball file %s has unknown extension" % tarball_filename)
+    if base == "":
+        return None
+    elif base[0] == "-":
+        # Extra component
+        return base[1:]
+    else:
+        raise ValueError("Invalid extra characters in tarball filename %s" %
+            tarball_filename)
+
+
+def extract_orig_tarball(tarball_filename, component, target, strip_components=None):
+    """Extract an orig tarball.
+
+    :param tarball: Path to the tarball
+    :param component: Component name (or None for top-level)
+    :param target: Target path
+    :param strip_components: Optional number of components to strip
     """
-    if len(tarballs) != 1:
-        raise MultipleUpstreamTarballsNotSupported()
-    tarball_filename = tarballs[0]
     tar_args = ["tar"]
     if tarball_filename.endswith(".tar.bz2"):
         tar_args.append('xjf')
     else:
         tar_args.append('xzf')
-    tar_args.extend([tarball_filename, "-C", target])
+    if component is not None:
+        target_path = os.path.join(target, component)
+        os.mkdir(target_path)
+    else:
+        target_path = target
+    tar_args.extend([tarball_filename, "-C", target_path])
     if strip_components is not None:
         tar_args.extend(["--strip-components", "1"])
     proc = subprocess.Popen(tar_args, preexec_fn=subprocess_setup)
     proc.communicate()
     if proc.returncode != 0:
         raise TarFailed("extract", tarball_filename)
+
+
+def extract_orig_tarballs(tarballs, target, strip_components=None):
+    """Extract orig tarballs to a directory.
+
+    :param tarballs: List of tarball filenames
+    :param target: Target directory (must already exist)
+    """
+    for tarball_filename, component in tarballs:
+        extract_orig_tarball(tarball_filename, component, target,
+            strip_components=strip_components)