Merge lp:~bzr/bzr/smooth-upgrades into lp:~bzr/bzr/trunk-old

[Ian Clatworthy]

[Ian Clatworthy (ian-clatworthy)]

This patch adds numerous enhancements to the upgrade command to make upgrading a lot smoother than it currently is. In particular, it now upgrades dependent branches when a shared repository is specified. Based on feedback provided on the mailing list a few months back, it now also supports several new options:

* --dry-run for showing what will happen

* --clean to remove the backup.dir directory on successful completion

* --pack to pack the repository on successful completion.

There's room to make things even nicer still but I think this patch is large enough as is. I'd also like to get these changes landed soon to encourage those of us running Bazaar's trunk to get as many branches as possible converted to 2a prior to the Bazaar 2.0 release.

[Aaron Bentley (abentley)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I think you should specify using=True to repo.find_branches. Branches
which aren't using the shared repository aren't really part of it even
if they're inside its directory, and may be in an incompatible format.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpgAO8ACgkQ0F+nu1YWqI1dTACfdxd519YTqoOuTz1OOa9zjtjY
Ot8Ani+XpR4URG8cbbbh5sBDWVr+/t6H
=7UNy
-----END PGP SIGNATURE-----

[Robert Collins (lifeless)]

On Fri, 2009-07-17 at 02:45 +0000, Ian Clatworthy wrote:
> Ian Clatworthy has proposed merging lp:~bzr/bzr/smooth-upgrades into lp:bzr.
>
> Requested reviews:
> bzr-core (bzr-core)
>
> This patch adds numerous enhancements to the upgrade command to make upgrading a lot smoother than it currently is. In particular, it now upgrades dependent branches when a shared repository is specified. Based on feedback provided on the mailing list a few months back, it now also supports several new options:
>
> * --dry-run for showing what will happen
>
> * --clean to remove the backup.dir directory on successful completion

I think these are useful

> * --pack to pack the repository on successful completion.

This isn't needed; bzr will autopack if packing helps.

> There's room to make things even nicer still but I think this patch is large enough as is. I'd also like to get these changes landed soon to encourage those of us running Bazaar's trunk to get as many branches as possible converted to 2a prior to the Bazaar 2.0 release.

-Rob

[Martin Pool (mbp)]

=== modified file 'NEWS'
--- NEWS 2009-07-15 19:06:39 +0000
+++ NEWS 2009-07-17 02:29:14 +0000
@@ -19,6 +19,17 @@
 * ``merge --interactive`` applies a user-selected portion of the merge. The UI
   is similar to ``shelve``. (Aaron Bentley)

+* ``upgrade`` now upgrades dependent branches when a shared repository is
+ specified. It also supports several new options:
+
+ * --dry-run for showing what will happen
+
+ * --clean to remove the backup.dir directory on successful completion
+
+ * --pack to pack the repository on successful completion.
+
+ (Ian Clatworthy)
+

I'd put the options in double-backticks too.

You might also find some bug numbers corresponding to these...

 Bug Fixes
 *********

=== modified file 'bzrlib/builtins.py'
--- bzrlib/builtins.py 2009-07-15 07:32:26 +0000
+++ bzrlib/builtins.py 2009-07-17 02:29:14 +0000
@@ -3122,27 +3122,60 @@

 class cmd_upgrade(Command):
- """Upgrade branch storage to current format.
-
- The check command or bzr developers may sometimes advise you to run
- this command. When the default format has changed you may also be warned
- during other operations to upgrade.
+ """Upgrade a repository, branch or working tree to a newer format.
+
+ The check command or Bazaar developers may sometimes advise you to run
+ this command. When the default format has changed after a major new
+ release of Bazaar, you may also be warned during other operations
+ that you should upgrade.

This bit about 'the check command' almost sounds like a mis-copy from
cmd_reconcile. Do we ever specifically recommend an upgrade from check?

Also it might be better off to have the first full paragraph be a more
general explanation like this:

  Upgrading to a newer format may improve performance or make
  new features available, but it may limit interoperability with
  older repositories or with older versions of Bazaar.

Strictly speaking you can also use upgrade to downgrade; it doesn't have
to be to a newer format.

+
+ If the location given is a shared repository, dependent branches
+ are also converted provided the repository converts successfully.
+ If the conversion of a branch fails, remaining branches are still tried.
+
+ A backup.bzr directory is created at the start of the conversion
+ process. By default, this is left there on completion. If the
+ conversion fails, delete the new .bzr directory and rename this
+ one back in its place. Use the --clean option to ask for the
+ backup.bzr directory to be removed on successful conversion.
+ Alternatively, you can delete it by hand if everything looks
+ good afterwards.
+
+ It is often a good idea to pack the repository after an upgrade.
+ Use the --pack option to request this or do it separately using
+ the pack command.
+
+ For more information on upgrades, see the Bazaar 2.0 Upgrade Guide.
     """

- _see_also = ['check']
+ _see_also = ['check', 'reconcile', 'pack']

Also current-formats etc?

     takes_args = ['url?']
     takes_options = [
- RegistryOption('format',
- help='Upgrade to a specific format. See "bzr help'
- ...

review: Needs Information

[lp:~bzr/bzr/smooth-upgrades]

4376. By Ian Clatworthy on 2009-08-19

Merge bzr.dev rev 4626

4377. By Ian Clatworthy on 2009-08-19

review feedback from abentley

4378. By Ian Clatworthy on 2009-08-20

Add basic checks to blackbox tests (beyond legal usage) as requested by poolie

4379. By Ian Clatworthy on 2009-08-20

Take --pack out of the UI and improve help (feedback from lifeless and poolie)

4380. By Ian Clatworthy on 2009-09-03

fix tests to truly use a shared repository, not standalone branches inside one

[Martin Pool (mbp)]

I think this is now back in Ian's court.

[Matthew Fuller (fullermd)]

See mp42746 for an updated version of this.

[Vincent Ladeuil (vila)]

This could be considered merged now.

=== modified file 'NEWS'
--- NEWS	2009-08-30 23:51:10 +0000
+++ NEWS	2009-08-31 04:36:58 +0000
@@ -9,6 +9,7 @@
 In Development
 ##############
 
+<<<<<<< TREE
 Compatibility Breaks
 ********************
 
@@ -90,6 +91,8 @@
 least six months or until the following stable release (we said
 that previously, but that's worth repeating).
 
+=======
+>>>>>>> MERGE-SOURCE
 Compatibility Breaks
 ********************
 
@@ -107,12 +110,25 @@
 New Features
 ************
 
+<<<<<<< TREE
 * ``bzr branch --switch`` can now switch the checkout in the current directory
   to the newly created branch. (Lukáš Lalinský)
 
+=======
+* ``upgrade`` now upgrades dependent branches when a shared repository is
+  specified. It also supports several new options:
+
+  * ``--dry-run`` for showing what will happen
+
+  * ``--clean`` to remove the backup.dir directory on successful completion.
+
+  (Ian Clatworthy)
+
+>>>>>>> MERGE-SOURCE
 Bug Fixes
 *********
 
+<<<<<<< TREE
 * Further tweaks to handling of ``bzr add`` messages about ignored files.
   (Jason Spashett, #76616)
 
@@ -141,6 +157,11 @@
   fetching only a few hundred bytes at a time.
   (Andrew Bennetts, #402657)
 
+=======
+* Fix a test failure on karmic by making a locale test more robust.
+  (Vincent Ladeuil, #413514)
+
+>>>>>>> MERGE-SOURCE
 Improvements
 ************
 
@@ -379,6 +400,7 @@
   if password authentication is even supported. This fixes a bug where
   users would be prompted for a launchpad password, even though launchpad
   only supports publickey authentication. (John Arbash Meinel, #375867)
+>>>>>>> MERGE-SOURCE
 
 * BranchBuilder now accepts timezone to avoid test failures in countries far
   from GMT. (Vincent Ladeuil, #397716)

=== modified file 'bzrlib/builtins.py'
--- bzrlib/builtins.py	2009-08-28 05:00:33 +0000
+++ bzrlib/builtins.py	2009-08-31 04:36:59 +0000
@@ -3139,27 +3139,62 @@
 
 
 class cmd_upgrade(Command):
-    """Upgrade branch storage to current format.
-
-    The check command or bzr developers may sometimes advise you to run
-    this command. When the default format has changed you may also be warned
-    during other operations to upgrade.
+    """Upgrade a repository, branch or working tree to a newer format.
+
+    When the default format has changed after a major new release of
+    Bazaar, you may be informed during certain operations that you
+    should upgrade. Upgrading to a newer format may improve performance
+    or make new features available. It may however limit interoperability
+    with older repositories or with older versions of Bazaar.
+
+    If you wish to upgrade to a particular format rather than the
+    current default, that can be specified using the --format option.
+    As a consequence, you can use the upgrade command this way to
+    "downgrade" to an earlier format, though some conversions are
+    a one way process (e.g. changing from the 1.x default to the
+    2.x default) so downgrading is not always possible.
+
+    A backup.bzr directory is created at the start of the conversion
+    process. By default, this is left there on completion. If the
+    conversion fails, delete the new .bzr directory and rename this
+    one back in its place. Use the --clean option to ask for the
+    backup.bzr directory to be removed on successful conversion.
+    Alternatively, you can delete it by hand if everything looks
+    good afterwards.
+
+    If the location given is a shared repository, dependent branches
+    are also converted provided the repository converts successfully.
+    If the conversion of a branch fails, remaining branches are still
+    tried.
+
+    For more information on upgrades, see the Bazaar 2.0 Upgrade Guide,
+    http://doc.bazaar-vcs.org/latest/en/upgrade-guide/.
     """
 
-    _see_also = ['check']
+    _see_also = ['check', 'reconcile', 'formats']
     takes_args = ['url?']
     takes_options = [
-                    RegistryOption('format',
-                        help='Upgrade to a specific format.  See "bzr help'
-                             ' formats" for details.',
-                        lazy_registry=('bzrlib.bzrdir', 'format_registry'),
-                        converter=lambda name: bzrdir.format_registry.make_bzrdir(name),
-                        value_switches=True, title='Branch format'),
-                    ]
+        RegistryOption('format',
+            help='Upgrade to a specific format.  See "bzr help'
+                 ' formats" for details.',
+            lazy_registry=('bzrlib.bzrdir', 'format_registry'),
+            converter=lambda name: bzrdir.format_registry.make_bzrdir(name),
+            value_switches=True, title='Branch format'),
+        Option('clean',
+            help='Remove the backup.bzr directory if successful.'),
+        Option('dry-run',
+            help="Show what would be done, but don't actually do anything."),
+    ]
 
-    def run(self, url='.', format=None):
+    def run(self, url='.', format=None, clean=False, dry_run=False):
         from bzrlib.upgrade import upgrade
-        upgrade(url, format)
+        exceptions = upgrade(url, format, clean_up=clean, dry_run=dry_run)
+        if exceptions:
+            if len(exceptions) == 1:
+                # This provides backwards compatibility
+                raise exceptions[0]
+            else:
+                return 3
 
 
 class cmd_whoami(Command):

=== modified file 'bzrlib/bzrdir.py'
--- bzrlib/bzrdir.py	2009-08-21 02:10:06 +0000
+++ bzrlib/bzrdir.py	2009-08-31 04:36:59 +0000
@@ -88,7 +88,7 @@
 
 
 class BzrDir(object):
-    """A .bzr control diretory.
+    """A .bzr control directory.
 
     BzrDir instances let you create or open any of the things that can be
     found within .bzr - checkouts, branches and repositories.
@@ -1300,6 +1300,40 @@
                 push_result.branch_push_result.target_branch
         return push_result
 
+    def _get_object_and_label(self):
+        """Return the primary object and type label for a control directory.
+
+        :return: object, label where
+          object is a Branch, Repository or WorkingTree and
+          label is one of:
+            branch            - a branch
+            repository        - a repository
+            tree              - a lightweight checkout
+        """
+        try:
+            try:
+                br = self.open_branch(unsupported=True, ignore_fallbacks=True)
+            except TypeError:
+                # RemoteRepository doesn't support the unsupported parameter
+                br = self.open_branch(ignore_fallbacks=True)
+        except errors.NotBranchError:
+            pass
+        else:
+            return br, "branch"
+        try:
+            repo = self.open_repository()
+        except errors.NoRepositoryPresent:
+            pass
+        else:
+            return repo, "repository"
+        try:
+            wt = self.open_workingtree()
+        except (errors.NoWorkingTree, errors.NotLocalUrl):
+            pass
+        else:
+            return wt, "tree"
+        raise AssertionError("unknown type of control directory %s", self)
+
 
 class BzrDirHooks(hooks.Hooks):
     """Hooks for BzrDir operations."""

=== modified file 'bzrlib/tests/blackbox/test_upgrade.py'
--- bzrlib/tests/blackbox/test_upgrade.py	2009-08-21 02:10:06 +0000
+++ bzrlib/tests/blackbox/test_upgrade.py	2009-08-31 04:36:59 +0000
@@ -154,6 +154,29 @@
         self.run_bzr('init-repository --format=metaweave repo')
         self.run_bzr('upgrade --format=knit repo')
 
+    def _assert_option_legal(self, option_str):
+        # Confirm that an option is legal. (Lower level tests are
+        # expected to validate the actual functionality.)
+        self.run_bzr('init --format=pack-0.92 branch-foo')
+        self.run_bzr('upgrade --format=2a branch-foo %s' % (option_str,))
+
+    def _assert_branch_format(self, dir, format):
+        branch = bzrdir.BzrDir.open_tree_or_branch(self.get_url(dir))[1]
+        branch_format = branch._format
+        meta_format = bzrdir.format_registry.make_bzrdir(format)
+        expected_format = meta_format.get_branch_format()
+        self.assertEqual(expected_format, branch_format)
+
+    def test_upgrade_clean_supported(self):
+        self._assert_option_legal('--clean')
+        self._assert_branch_format('branch-foo', '2a')
+        backup_bzr_dir = os.path.join("branch-foo", "backup.bzr")
+        self.assertFalse(os.path.exists(backup_bzr_dir))
+
+    def test_upgrade_dry_run_supported(self):
+        self._assert_option_legal('--dry-run')
+        self._assert_branch_format('branch-foo', 'pack-0.92')
+
 
 class SFTPTests(TestCaseWithSFTPServer):
     """Tests for upgrade over sftp."""

=== modified file 'bzrlib/tests/test_foreign.py'
--- bzrlib/tests/test_foreign.py	2009-08-17 07:54:34 +0000
+++ bzrlib/tests/test_foreign.py	2009-08-31 04:36:59 +0000
@@ -249,7 +249,7 @@
         self._control_files = lockable_files.LockableFiles(self.transport,
             "lock", lockable_files.TransportLock)
 
-    def open_branch(self, ignore_fallbacks=True):
+    def open_branch(self, unsupported=False, ignore_fallbacks=True):
         return self._format.get_branch_format().open(self, _found=True)
 
     def cloning_metadir(self, stacked=False):

=== modified file 'bzrlib/tests/test_upgrade.py'
--- bzrlib/tests/test_upgrade.py	2009-05-07 05:08:46 +0000
+++ bzrlib/tests/test_upgrade.py	2009-08-31 04:36:59 +0000
@@ -29,6 +29,7 @@
 from bzrlib import (
     branch as _mod_branch,
     bzrdir,
+    osutils,
     progress,
     repository,
     workingtree,
@@ -38,7 +39,7 @@
 from bzrlib.branch import Branch
 from bzrlib.tests import TestCaseWithTransport
 from bzrlib.transport import get_transport
-from bzrlib.upgrade import upgrade
+from bzrlib.upgrade import upgrade, smart_upgrade
 
 
 class TestUpgrade(TestCaseWithTransport):
@@ -428,3 +429,62 @@
     ),
     ( './dir/', ),
 ]
+
+
+class TestSmartUpgrade(TestCaseWithTransport):
+
+    from_format = "pack-0.92"
+    to_format = bzrdir.format_registry.make_bzrdir("2a")
+
+    def make_standalone_branch(self):
+        wt = self.make_branch_and_tree("branch1", format=self.from_format)
+        return wt.bzrdir
+
+    def test_upgrade_standalone_branch(self):
+        control = self.make_standalone_branch()
+        tried, worked, issues = smart_upgrade([control], format=self.to_format)
+        self.assertEqual(1, len(tried))
+        self.assertEqual(1, len(worked))
+        self.assertEqual(0, len(issues))
+        self.failUnlessExists('branch1/backup.bzr')
+
+    def test_upgrade_standalone_branch_cleanup(self):
+        control = self.make_standalone_branch()
+        tried, worked, issues = smart_upgrade([control], format=self.to_format,
+            clean_up=True)
+        self.assertEqual(1, len(tried))
+        self.assertEqual(1, len(worked))
+        self.assertEqual(0, len(issues))
+        self.failUnlessExists('branch1')
+        self.failUnlessExists('branch1/.bzr')
+        self.failIfExists('branch1/backup.bzr')
+
+    def make_repo_with_branches(self):
+        repo = self.make_repository('repo', shared=True,
+            format=self.from_format)
+        b1 = self.make_branch("repo/branch1", format=self.from_format)
+        b2 = self.make_branch("repo/branch2", format=self.from_format)
+        return repo.bzrdir
+
+    def test_upgrade_repo_with_branches(self):
+        control = self.make_repo_with_branches()
+        tried, worked, issues = smart_upgrade([control], format=self.to_format)
+        self.assertEqual(3, len(tried))
+        self.assertEqual(3, len(worked))
+        self.assertEqual(0, len(issues))
+        self.failUnlessExists('repo/backup.bzr')
+        self.failUnlessExists('repo/branch1/backup.bzr')
+        self.failUnlessExists('repo/branch2/backup.bzr')
+
+    def test_upgrade_repo_with_branches_cleanup(self):
+        control = self.make_repo_with_branches()
+        tried, worked, issues = smart_upgrade([control], format=self.to_format,
+            clean_up=True)
+        self.assertEqual(3, len(tried))
+        self.assertEqual(3, len(worked))
+        self.assertEqual(0, len(issues))
+        self.failUnlessExists('repo')
+        self.failUnlessExists('repo/.bzr')
+        self.failIfExists('repo/backup.bzr')
+        self.failIfExists('repo/branch1/backup.bzr')
+        self.failIfExists('repo/branch2/backup.bzr')

=== modified file 'bzrlib/upgrade.py'
--- bzrlib/upgrade.py	2009-05-07 05:08:46 +0000
+++ bzrlib/upgrade.py	2009-08-31 04:36:59 +0000
@@ -17,18 +17,36 @@
 """bzr upgrade logic."""
 
 
+from bzrlib import osutils, repository
 from bzrlib.bzrdir import BzrDir, BzrDirFormat, format_registry
 import bzrlib.errors as errors
 from bzrlib.remote import RemoteBzrDir
 from bzrlib.transport import get_transport
+from bzrlib.trace import mutter, note, warning
 import bzrlib.ui as ui
 
 
 class Convert(object):
 
-    def __init__(self, url, format=None):
+    def __init__(self, url=None, format=None, control_dir=None):
+        """Convert a Bazaar control directory to a given format.
+
+        Either the url or control_dir parameter must be given.
+
+        :param url: the URL of the control directory or None if the
+          control_dir is explicitly given instead
+        :param format: the format to convert to or None for the default
+        :param control_dir: the control directory or None if it is
+          specified via the URL parameter instead
+        """
         self.format = format
-        self.bzrdir = BzrDir.open_unsupported(url)
+        if url is None and control_dir is None:
+            raise AssertionError(
+                "either the url or control_dir parameter must be set.")
+        if control_dir is not None:
+            self.bzrdir = control_dir
+        else:
+            self.bzrdir = BzrDir.open_unsupported(url)
         if isinstance(self.bzrdir, RemoteBzrDir):
             self.bzrdir._ensure_real()
             self.bzrdir = self.bzrdir._real_bzrdir
@@ -73,13 +91,179 @@
                            self.bzrdir._format)
         self.bzrdir.check_conversion_target(format)
         self.pb.note('starting upgrade of %s', self.transport.base)
-        self.bzrdir.backup_bzrdir()
+        self.backup_oldpath, self.backup_newpath = self.bzrdir.backup_bzrdir()
         while self.bzrdir.needs_format_conversion(format):
             converter = self.bzrdir._format.get_converter(format)
             self.bzrdir = converter.convert(self.bzrdir, self.pb)
         self.pb.note("finished")
 
-
-def upgrade(url, format=None):
-    """Upgrade to format, or the default bzrdir format if not supplied."""
-    Convert(url, format)
+    def clean_up(self):
+        """Clean-up after a conversion.
+
+        This removes the backup.bzr directory.
+        """
+        transport = self.transport
+        backup_relpath = transport.relpath(self.backup_newpath)
+        transport.delete_tree(backup_relpath)
+
+
+def upgrade(urls, format=None, clean_up=False, pack=False, dry_run=False):
+    """Upgrade locations to format.
+ 
+    This routine wraps the smart_upgrade() routine with a nicer UI.
+    In particular, it ensures all URLs can be opened before starting
+    and reports a summary at the end if more than one upgrade was attempted.
+    This routine is useful for command line tools. Other bzrlib clients
+    probably ought to use smart_upgrade() instead.
+
+    :param urls: a sequence of URLs to the locations to upgrade.
+      For backwards compatibility, if urls is a string, it is treated
+      as a single URL.
+    :param format: the format to convert to or None for the best default
+    :param clean-up: if True, the backup.bzr directory is removed if the
+      upgrade succeeded for a given repo/branch/tree
+    :param pack: pack repositories that successfully upgrade
+    :param dry_run: show what would happen but don't actually do any upgrades
+    :return: the list of exceptions encountered
+    """
+    if isinstance(urls, basestring):
+        urls = [urls]
+    control_dirs = [BzrDir.open_unsupported(url) for url in urls]
+    attempted, succeeded, exceptions = smart_upgrade(control_dirs,
+        format, clean_up=clean_up, pack=pack, dry_run=dry_run)
+    if len(attempted) > 1:
+        attempted_count = len(attempted)
+        succeeded_count = len(succeeded)
+        failed_count = attempted_count - succeeded_count
+        note("\nSUMMARY: %d upgrades attempted, %d succeeded, %d failed",
+            attempted_count, succeeded_count, failed_count)
+    return exceptions
+
+
+def smart_upgrade(control_dirs, format, clean_up=False, pack=False,
+    dry_run=False):
+    """Convert control directories to a new format intelligently.
+
+    If the control directory is a shared repository, dependent branches
+    are also converted provided the repository converted successfully.
+    If the conversion of a branch fails, remaining branches are still tried.
+
+    :param control_dirs: the BzrDirs to upgrade
+    :param format: the format to convert to or None for the best default
+    :param clean-up: if True, the backup.bzr directory is removed if the
+      upgrade succeeded for a given repo/branch/tree
+    :param pack: pack repositories that successfully upgrade
+    :param dry_run: show what would happen but don't actually do any upgrades
+    :return: attempted-control-dirs, succeeded-control-dirs, exceptions
+    """
+    all_attempted = []
+    all_succeeded = []
+    all_exceptions = []
+    for control_dir in control_dirs:
+        attempted, succeeded, exceptions = _smart_upgrade_one(control_dir,
+            format, clean_up=clean_up, pack=pack, dry_run=dry_run)
+        all_attempted.extend(attempted)
+        all_succeeded.extend(succeeded)
+        all_exceptions.extend(exceptions)
+    return all_attempted, all_succeeded, all_exceptions
+
+
+def _smart_upgrade_one(control_dir, format, clean_up=False, pack=False,
+    dry_run=False):
+    """Convert a control directory to a new format intelligently.
+
+    See smart_upgrade for parameter details.
+    """
+    # If the URL is a shared repository, find the dependent branches
+    dependents = None
+    try:
+        repo = control_dir.open_repository()
+    except errors.NoRepositoryPresent:
+        # A branch or checkout using a shared repository higher up
+        pass
+    else:
+        # The URL is a repository. If it successfully upgrades,
+        # then upgrade the dependent branches as well.
+        if repo.is_shared():
+            dependents = repo.find_branches(using=True)
+
+    # Do the conversions
+    attempted = [control_dir]
+    succeeded, exceptions = _convert_items([control_dir], format, clean_up,
+        pack, dry_run, verbose=dependents)
+    if succeeded and dependents:
+        note("Found %d dependent branches - upgrading ...", len(dependents))
+
+        # Convert dependent branches
+        branch_cdirs = [b.bzrdir for b in dependents]
+        successes, problems = _convert_items(branch_cdirs, format, clean_up,
+            pack, dry_run, label="branch")
+        attempted.extend(branch_cdirs)
+        succeeded.extend(successes)
+        exceptions.extend(problems)
+
+    # Return the result
+    return attempted, succeeded, exceptions
+
+
+def _convert_items(items, format, clean_up, pack, dry_run, label=None,
+    verbose=True):
+    """Convert a sequence of control directories to the given format.
+ 
+    :param items: the control directories to upgrade
+    :param format: the format to convert to or None for the best default
+    :param clean-up: if True, the backup.bzr directory is removed if the
+      upgrade succeeded for a given repo/branch/tree
+    :param pack: pack repositories that successfully upgrade
+    :param dry_run: show what would happen but don't actually do any upgrades
+    :param label: the label for these items or None to calculate one
+    :param verbose: if True, output a message before starting and
+      display any problems encountered
+    :return: items successfully upgraded, exceptions
+    """
+    succeeded = []
+    exceptions = []
+    for control_dir in items:
+        # Do the conversion
+        location = control_dir.root_transport.base
+        bzr_object, bzr_label = control_dir._get_object_and_label()
+        if verbose:
+            type_label = label or bzr_label
+            note("Upgrading %s %s ...", type_label, location)
+        try:
+            if not dry_run:
+                cv = Convert(control_dir=control_dir, format=format)
+        except Exception, ex:
+            _verbose_warning(verbose, "conversion error: %s" % ex)
+            exceptions.append(ex)
+            continue
+
+        # Do any required post processing
+        succeeded.append(control_dir)
+        if pack and isinstance(bzr_object, repository.Repository):
+            note("Packing ...")
+            try:
+                if not dry_run:
+                    bzr_object.pack()
+            except Exception, ex:
+                _verbose_warning(verbose, "failed to pack %s: %s" %
+                    (location, ex))
+                exceptions.append(ex)
+        if clean_up:
+            try:
+                note("Removing backup ...")
+                if not dry_run:
+                    cv.clean_up()
+            except Exception, ex:
+                _verbose_warning(verbose, "failed to clean-up %s: %s" %
+                    (location, ex))
+                exceptions.append(ex)
+
+    # Return the result
+    return succeeded, exceptions
+
+
+def _verbose_warning(verbose, msg):
+    mutter(msg)
+    if verbose:
+        warning(msg)