Merge ~racb/git-ubuntu:prepare-upload-fetch into git-ubuntu:main

[Robie Basak]

Make Jenkins happy

[Robie Basak (racb)]

I'm going to do some additional testing from the snap once CI has built it.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:a384e90b271b669ef13e71ea07d9752b7e2f0218
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/28/
Executed test runs:
    SUCCESS: VM Setup
    SUCCESS: Build
    SUCCESS: VM Reset
    FAILED: Unit Tests

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/28//rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:9cc9c226113909803c6c07ad5e89a380a38a2713
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/29/
Executed test runs:
    SUCCESS: VM Setup
    SUCCESS: Build
    SUCCESS: VM Reset
    SUCCESS: Unit Tests
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/29//rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:ebf85d59b9cf50426b317f54976b50a9af6ffaf8
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/30/
Executed test runs:
    SUCCESS: VM Setup
    SUCCESS: Build
    SUCCESS: VM Reset
    SUCCESS: Unit Tests
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/30//rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:c52820bbfef3b954c77ecff01e6884bb6e86c6be
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/31/
Executed test runs:
    SUCCESS: VM Setup
    SUCCESS: Build
    SUCCESS: VM Reset
    SUCCESS: Unit Tests
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/31//rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:c52820bbfef3b954c77ecff01e6884bb6e86c6be
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/32/
Executed test runs:
    SUCCESS: VM Setup
    SUCCESS: Build
    SUCCESS: VM Reset
    SUCCESS: Unit Tests
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/32//rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:2b2757734075fb7546ab604df5fb0df013e40046
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/34/
Executed test runs:
    SUCCESS: VM Setup
    SUCCESS: Build
    SUCCESS: VM Reset
    SUCCESS: Unit Tests
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/34//rebuild

review: Approve (continuous-integration)

[Robie Basak (racb)]

This is ready for review I think. Steve, could you review please, as it's mainly your request and also there are implications for gu-build?

[Steve Langasek (vorlon)]

LGTM; one comment.

review: Approve

[Robie Basak (racb)]

On Tue, Jul 18, 2023 at 12:10:48AM -0000, Steve Langasek wrote:
> The 'fixme' above this is addressed now by prepare_upload._add_subparser_arguments(), could we just call that instead?

We can, though I see _add_subparser_arguments() as a minor
implementation detail that I don't think would develop into an API for
other subcommands. In contrast the functions further up the call stack
could be more stable though we don't really know that either.

Given that we don't really have any API stability guarantees and also
expect gu-build to break if the CLI arguments were to change (since we
then parse the ArgumentParser.parse_args() output ourselves in gu-build
so that would break) I'm not sure this change would gain anything. But
nor would it lose anything. So whatever you prefer for gu-build I guess?

[Steve Langasek (vorlon)]

My preference is to roll with the possibility of future API breakage rather than duplicating the code here.

[Robie Basak (racb)]

OK. Done.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:4fd07c835ab4f3f25d1b9296000986b88886b483
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/35/
Executed test runs:
    SUCCESS: VM Setup
    SUCCESS: Build
    SUCCESS: VM Reset
    SUCCESS: Unit Tests
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.canonical.com/server-team/job/git-ubuntu-ci/35//rebuild

review: Approve (continuous-integration)

diff --git a/gitubuntu/prepare_upload.py b/gitubuntu/prepare_upload.py
index 93edd85..6b1caa5 100644
--- a/gitubuntu/prepare_upload.py
+++ b/gitubuntu/prepare_upload.py
@@ -1,9 +1,9 @@
 """Prepare an upload by pushing rich history
 
 This subcommand provides two further subcommands: "args" and "mangle". Both
-subcommands first "git push" so that the rich history is made available to the
-importer when it later imports your upload. They then diverge in how they
-provide a pointer to that rich history.
+subcommands first "git fetch" and then "git push" if required so that the rich
+history is made available to the importer when it later imports your upload.
+They then diverge in how they provide a pointer to that rich history.
 
 The "args" subcommand prints command line arguments suitable to provide to
 another command that will generate the changes file. It defaults to printing
@@ -14,6 +14,7 @@ fields. It requires a single positional argument pointing to the changes file
 to mangle.
 """
 import argparse
+import collections
 import os
 import re
 import sys
@@ -54,9 +55,28 @@ URL_REWRITE = (
 )
 
 
-def parse_args(subparsers=None, base_subparsers=None):
+class ForcePushRequired(RuntimeError): pass
+
+
+def _add_subparser_arguments(parser):
+    """Load standard prepare-upload arguments into the given parser
+
+    This is so that we can do this for the general prepare-upload subparser, as
+    well as the specific args and mangle subparsers, so the arguments work
+    regardless of whether they are placed before or after the final args/mangle
+    subcommand. See: https://stackoverflow.com/a/46962350/478206
+
+    :param argparse.ArgumentParser parser: the parser to which toadd the
+        arguments.
+    """
+    parser.add_argument('--remote', type=str, help="Remote to push to")
+    parser.add_argument('--branch', type=str, help="Branch to push")
+    parser.add_argument('--force-push', action='store_true')
+
+
+def parse_args(subparsers=None, base_subparsers=None):  # pragma: no cover
     kwargs = dict(
-        description="Push a branch for subsequent upload with dput",
+        description="Push branch and provide history-enriched headers for dput",
         formatter_class=argparse.ArgumentDefaultsHelpFormatter,
     )
     if base_subparsers:
@@ -65,85 +85,201 @@ def parse_args(subparsers=None, base_subparsers=None):
         parser = subparsers.add_parser('prepare-upload', **kwargs)
     else:
         parser = argparse.ArgumentParser(**kwargs)
-    parser.add_argument('--remote', type=str, help="Remote to push to")
-    parser.add_argument('--branch', type=str, help="Branch to push")
+
+    _add_subparser_arguments(parser)
     level2_subparsers = parser.add_subparsers(required=True)
-    printargs_subparser = level2_subparsers.add_parser('args')
+    printargs_subparser = level2_subparsers.add_parser(
+        'args',
+        description="Push branch and print history-enriched header arguments",
+    )
     printargs_subparser.set_defaults(func=cli_printargs)
+    _add_subparser_arguments(printargs_subparser)
     printargs_subparser.add_argument(
         '--output-format',
         choices=OUTPUT_FORMATS.keys(),
         default='dpkg-buildpackage',
         help="Output format",
     )
-    manglechanges_subparser = level2_subparsers.add_parser('mangle')
+    manglechanges_subparser = level2_subparsers.add_parser(
+        'mangle',
+        description="Push branch and adjust history-enriched headers in changes file",
+    )
     manglechanges_subparser.set_defaults(func=cli_manglechanges)
+    _add_subparser_arguments(manglechanges_subparser)
     manglechanges_subparser.add_argument('changes_file_path')
     if not subparsers:
         return parser.parse_args()
     return "prepare-upload - %s" % kwargs['description']
 
 
+class Parameters(collections.namedtuple('Parameters', [
+    'vcs_git',
+    'vcs_git_ref',
+    'vcs_git_commit',
+    'commit',
+    'remote_name',
+    'local_ref_name',
+    'local_tracking_ref_name',
+])):
+    """The parameters associated with a prepare-upload request
+
+    When preparing an upload, we need to know what the final Vcs-Git* headers
+    in the changes file will be, as well as various other parameters such as
+    the name of the configured remote that corresponds to these, the pygit2
+    Commit object and so forth. These are determined once and then stored in an
+    instance of this class, with the individual parameters specified here.
+
+    :ivar str vcs_git: the contents of the eventual Vcs-Git header
+    :ivar str vcs_git_ref: the contents of the eventual Vcs-Git-Ref header
+    :ivar str vcs_git_commit: the contents of the eventual Vcs-Git-Commit
+        header
+    :ivar pygit2.Commit commit: the local commit being supplied as rich history
+    :ivar str remote_name: the name of the configured remote where the rich
+        history will be staged
+    :ivar str local_ref_name: the ref name of the local branch that contains
+        the rich history, such as "refs/heads/feature".
+    :ivar str local_tracking_ref_name: the ref name of the local remote
+        tracking branch that tracks the remote branch where the rich history
+        will be staged, such as "refs/remotes/dave/feature".
+    """
+    pass
+
+    @property
+    def changes_file_headers(self):
+        """The changes file headers as a dict keyed on header name
+
+        The header names are converted from the snake_case attributes to
+        deb822-case used in changes files.
+        """
+        return {
+            k: getattr(self, k.lower().replace('-', '_'))
+            for k in ['Vcs-Git', 'Vcs-Git-Ref', 'Vcs-Git-Commit']
+        }
+
+    @classmethod
+    def from_repo(cls, repo, remote_name, branch_name):
+        """Instantiate Parameters from user-provided string args and defaults
+
+        This method encapsulates the heuristics used to determine what to do
+        based the state of the repository and anything explicitly specified by
+        the user.
+
+        :param GitUbuntuRepository repo: the repository to consider.
+        :param str remote_name: the remote name explicitly requested by the
+            user, or None.
+        :param str branch_name: the branch name explicitly requested by the
+            user, or None.
+        :rtype: Parameters
+        :returns: a populated instance of this Parameters class.
+        """
+        if not remote_name:
+            remote_name = repo.lp_user
+
+        if branch_name:
+            # XXX to identify local_tracking_ref_name, this much be a branch, not just any ref-ish
+            commit, ref = repo.raw_repo.resolve_refish(branch_name)
+            # ref might be None if the given branch parameter isn't actually a
+            # branch (eg. is a commit)
+            assert ref
+        else:
+            assert not repo.raw_repo.head_is_detached
+            ref = repo.raw_repo.head
+            assert ref.name.startswith('refs/heads/')
+            branch_name = ref.shorthand
+            commit = ref.peel(pygit2.Commit)
+
+        pygit2_remote = repo.raw_repo.remotes[remote_name]
+
+        # Rewrite rule for VCS URLs. See LP: #1942985
+        vcs_git = URL_REWRITE[0].sub(URL_REWRITE[1], pygit2_remote.url)
+
+        vcs_git_ref = ref.name
+        vcs_git_commit = str(commit.id)
+
+        # Pre-empt any failure with later input validation of the headers we
+        # will generate after we push. If these assertions fail, then the later
+        # import will refuse to accept this rich history, so better that we
+        # fail now so the uploader knows there's some kind of problem, rather
+        # than later when the importer will silently throw away the rich
+        # history that the uploader pushed.
+        #
+        # However, we don't know of any actual case when this might happen, so
+        # these are assertions rather than fully UX-compliant error paths.
+        assert gitubuntu.importer.VCS_GIT_URL_VALIDATION.fullmatch(vcs_git)
+        assert vcs_git.startswith(
+            gitubuntu.importer.LAUNCHPAD_GIT_HOSTING_URL_PREFIX,
+        )
+        assert gitubuntu.importer.VCS_GIT_REF_VALIDATION.fullmatch(vcs_git_ref)
+        assert gitubuntu.importer.VCS_GIT_COMMIT_VALIDATION.fullmatch(
+            vcs_git_commit,
+        )
+        return cls(
+            vcs_git=vcs_git,
+            vcs_git_ref=vcs_git_ref,
+            vcs_git_commit=vcs_git_commit,
+            commit=commit,
+            remote_name=remote_name,
+            local_ref_name=ref.name,
+            local_tracking_ref_name=
+                'refs/remotes/%s/%s' % (remote_name, branch_name),
+        )
+
+
+def fetch(repo, remote_name, check=True, verbose_on_failure=True):
+    """Fetch from a remote using its default refspec
+
+    :param GitUbuntuRepository repo: the local repository to fetch into.
+    :param str remote_name: the name of the remote to fetch from.
+    :param bool check: passed through to GitUbuntuRepository.git_run().
+    :param bool verbose_on_failure: passed through to
+        GitUbuntuRepository.git_run().
+    :raises subprocess.CalledProcessError: if the git call fails.
+    """
+    # pygit2's fetch method seems to demand authentication even though none is
+    # needed in the common case that ssh keys are in use, so we call the CLI
+    # instead
+    repo.git_run(
+        ['fetch', remote_name],
+        stdout=sys.stderr,  # we do want the user to see this output, but
+                            # cannot send it to stdout as that would interfere
+                            # with the output of the push-for-upload command
+                            # itself that needs to be captured by the user to
+                            # pass to their build command.
+        stderr=None,  # we do want the user to see this output
+        # If the remote repository does not exist (eg. it will be created by
+        # the push) then Launchpad will prompt to try authentication, but that
+        # will have no meaning for the user, so we don't want a prompt and
+        # instead the fetch call should fail.
+        env={'GIT_TERMINAL_PROMPT': '0'},
+        check=check,
+        verbose_on_failure=verbose_on_failure,
+    )
+
+
 def push(
     repo,
-    remote=None,
-    branch=None,
+    remote_name,
+    local_ref_name,
+    force_push=False,
 ):
-    """Push to a remote and return suitable rich history reference headers
+    """Push a ref to a remote
 
     :param GitUbuntuRepository repo: the local repository to push from
-    :param str remote: the name of the remote to push to. If None, then use
-        repo.lp_user.
-    :param str branch: the name of the local branch to push. If None, use what
-        HEAD points to.
-    :rtype: dict
-    :returns: a dict providing the additional key/value metadata to add to the
-        changes file that references the commit just pushed as the commit to
-        use for rich history import when later seen by the git-ubuntu importer.
+    :param str remote_name: the name of the remote to push to.
+    :param str local_ref_name: the name of the local ref to push. The same ref
+        will be pushed to the remote end. Example: "refs/heads/feature".
+    :param bool force_push: if a force push should be used
+    :raises subprocess.CalledProcessError: if the git call fails.
     """
-    if not remote:
-        remote = repo.lp_user
-
-    if branch:
-        commit, ref = repo.raw_repo.resolve_refish(branch)
-        # ref might be None if the given branch parameter isn't actually a
-        # branch (eg. is a commit)
-        assert ref
-    else:
-        assert not repo.raw_repo.head_is_detached
-        ref = repo.raw_repo.head
-        commit = ref.peel(pygit2.Commit)
-
-    pygit2_remote = repo.raw_repo.remotes[remote]
-
-    # Rewrite rule for VCS URLs. See LP: #1942985
-    vcs_git = URL_REWRITE[0].sub(URL_REWRITE[1], pygit2_remote.url)
-
-    vcs_git_ref = ref.name
-    vcs_git_commit = str(commit.id)
-
-    # Pre-empt any failure with later input validation of the headers we will
-    # generate after we push. If these assertions fail, then the later import
-    # will refuse to accept this rich history, so better that we fail now so
-    # the uploader knows there's some kind of problem, rather than later when
-    # the importer will silently throw away the rich history that the uploader
-    # pushed.
-    #
-    # However, we don't know of any actual case when this might happen, so
-    # these are assertions rather than fully UX-compliant error paths.
-    assert gitubuntu.importer.VCS_GIT_URL_VALIDATION.fullmatch(vcs_git)
-    assert vcs_git.startswith(
-        gitubuntu.importer.LAUNCHPAD_GIT_HOSTING_URL_PREFIX,
-    )
-    assert gitubuntu.importer.VCS_GIT_REF_VALIDATION.fullmatch(vcs_git_ref)
-    assert gitubuntu.importer.VCS_GIT_COMMIT_VALIDATION.fullmatch(
-        vcs_git_commit,
-    )
-
     # pygit2's push method seems to demand authentication even though none is
-    # needed in the common case that ssh keys are in use
+    # needed in the common case that ssh keys are in use, so we call the CLI
+    # instead
+    args = ['push', remote_name]
+    if force_push:
+        args.append('--force')
+    args.append('%s:%s' % (local_ref_name, local_ref_name))
     repo.git_run(
-        ['push', remote, '%s:%s' % (ref.name, ref.name)],
+        args,
         stdout=sys.stderr,  # we do want the user to see this output, but
                             # cannot send it to stdout as that would interfere
                             # with the output of the push-for-upload command
@@ -152,15 +288,117 @@ def push(
         stderr=None,  # we do want the user to see this output
     )
 
-    return {
-        'Vcs-Git': vcs_git,
-        'Vcs-Git-Ref': vcs_git_ref,
-        'Vcs-Git-Commit': vcs_git_commit,
-    }
+
+def ref_has_commit(repo, ref, commit):
+    """Determine if a commit can be reached from a given reference
+
+    :param GitUbuntuRepository repo: the repository to consider
+    :param pygit2.Reference ref: which reference to look in
+    :param pygit2.Commit commit: which commit to find
+    :rtype: bool
+    :returns: True if the commit can be reached; False otherwise
+
+    This differs from pygit2.Repository.descendant_of() because that method
+    would return False for the commit the ref points to.
+    """
+    ref_tip = ref.peel(pygit2.Commit)
+    return (
+        ref_tip.id == commit.id
+        or repo.raw_repo.descendant_of(ref_tip.id, commit.id)
+    )
 
 
-def mangle_changes(changes_file_path, headers):
-    """Mangle a changes file with the given headers
+def mod_changes_file(changes_file_path, replacements):
+    """Modify an existing changes file
+
+    :param str changes_file_path: the changes file to modify
+    :param dict(str, str) replacements: the replacement keys and values
+    """
+    modded_changes_file_path = f'{changes_file_path}_modded'
+
+    with open(changes_file_path) as f:
+        changes = debian.deb822.Changes(f)
+    changes.update(replacements)
+    with open(modded_changes_file_path, 'wb') as f:
+        changes.dump(f)
+    os.rename(modded_changes_file_path, changes_file_path)
+
+
+def ensure_in_remote(
+    repo,
+    remote_name,
+    local_ref_name,
+    local_tracking_ref_name,
+    commit,
+    force_push=False,
+):
+    """Ensure that a local ref and commit are present in a remote
+
+    Fetch the default refspec(s) from a remote, check if a remote tracking
+    branch contains a commit, and push a local ref if it does not.
+
+    :param GitUbuntuRepository repo: the git repository to use
+    :param str remote_name: the name of the remote
+    :param str local_ref_name: the name of the local ref
+    :param str local_tracking_ref_name: the name of local tracking branch of
+        the remote ref
+    :param pygit2.Commit commit: the commit
+    :param bool force_push: if a force push should be used
+    :raises ForcePushRequired: if a force push is required but not requested
+    """
+    # Do a best-effort attempt to get the remote tracking branch updated. If
+    # this doesn't succeed, then we will just proceed as if our understanding
+    # of the state of the remote branch is up-to-date. Therefore we can ignore
+    # failures. It's possible that the user has force-pushed the remote branch
+    # backwards and not updated the remote tracking branch and the fetch also
+    # fails. In this case, we'll think we don't need to push, and
+    # prepare-upload will supply rich history that is not present in the
+    # remote. This seems unlikely and an acceptable risk.
+    fetch(repo, remote_name, check=False, verbose_on_failure=False)
+    try:
+        local_tracking_ref = repo.raw_repo.lookup_reference(
+            local_tracking_ref_name
+        )
+    except KeyError:
+        needs_push = True
+    else:
+        needs_push = not ref_has_commit(repo, local_tracking_ref, commit)
+
+        if needs_push:
+            local_ref = repo.raw_repo.lookup_reference(local_ref_name)
+            local_ref_commit_id = local_ref.peel(pygit2.Commit).id
+            local_tracking_ref_id = local_tracking_ref.peel(pygit2.Commit).id
+
+            # If these were the same, then a push wouldn't be required
+            assert local_ref_commit_id != local_tracking_ref_id
+
+            is_fast_forward = repo.raw_repo.descendant_of(
+                local_ref_commit_id,
+                local_tracking_ref_id,
+            )
+            if not is_fast_forward and not force_push:
+                raise ForcePushRequired()
+
+    if needs_push:
+        push(
+            repo=repo,
+            remote_name=remote_name,
+            local_ref_name=local_ref_name,
+            force_push=force_push,
+        )
+
+
+def mangle_changes(
+    repo,
+    changes_file_path,
+    remote_name=None,
+    branch_name=None,
+    force_push=False,
+):
+    """Pythonic API entry point to the "prepare-upload mangle" subcommand
+
+    Ensure rich history is present in a remote for adoption by the importer,
+    then mangle a changes file with the required headers.
 
     Replace the changes file at the given path with one that has the given
     headers overridden. If the changes file was previously signed, the
@@ -169,44 +407,116 @@ def mangle_changes(changes_file_path, headers):
     To avoid the risk of ending up with corrupted changes files, this writes to
     a new file and then does an atomic rename over the old path.
 
+    :param GitUbuntuRepository repo: the git repository to use
     :param str changes_file_path: path to the changes file to mangle
-    :param dict headers: headers to override with
+    :param str remote_name: the name of the remote to use, or None to use the
+        remote named after the user's Launchpad username (as added by "git
+        clone").
+    :param str branch_name: the name of the branch to use (both local and
+        remote), or None for the name of the currently checked-out branch.
+    :param bool force_push: if a force push should be used
     :returns: None
+    :raises ForcePushRequired: if a force push is required but not requested
     """
-    mangled_changes_file_path = f'{changes_file_path}_mangled'
+    parameters = Parameters.from_repo(
+        repo=repo,
+        remote_name=remote_name,
+        branch_name=branch_name,
+    )
+    ensure_in_remote(
+        repo=repo,
+        remote_name=parameters.remote_name,
+        local_ref_name=parameters.local_ref_name,
+        local_tracking_ref_name=parameters.local_tracking_ref_name,
+        commit=parameters.commit,
+        force_push=force_push,
+    )
+    mod_changes_file(
+        changes_file_path=changes_file_path,
+        replacements=parameters.changes_file_headers,
+    )
 
-    with open(changes_file_path) as f:
-        changes = debian.deb822.Changes(f)
-    for k, v in headers.items():
-        changes[k] = v
-    with open(mangled_changes_file_path, 'wb') as f:
-        changes.dump(f)
-    os.rename(mangled_changes_file_path, changes_file_path)
+
+def establish_args(repo, remote_name=None, branch_name=None, force_push=False):
+    """Pythonic API entry point to the "prepare-upload args" subcommand
+
+    Ensure rich history is present in a remote for adoption by the importer,
+    then return the parameters that contain the arguments required to arrange
+    for a changes file to be generated with the required headers.
+
+    :param GitUbuntuRepository repo: the git repository to use
+    :param str remote_name: the name of the remote to use, or None to use the
+        remote named after the user's Launchpad username (as added by "git
+        clone").
+    :param str branch_name: the name of the branch to use (both local and
+        remote), or None for the name of the currently checked-out branch.
+    :param bool force_push: if a force push should be used
+    :rtype: Parameters
+    :returns: parameters that were used as determined by heuristics and the
+        user
+    :raises ForcePushRequired: if a force push is required but not requested
+    """
+    parameters = Parameters.from_repo(
+        repo=repo,
+        remote_name=remote_name,
+        branch_name=branch_name,
+    )
+    ensure_in_remote(
+        repo=repo,
+        remote_name=parameters.remote_name,
+        local_ref_name=parameters.local_ref_name,
+        local_tracking_ref_name=parameters.local_tracking_ref_name,
+        commit=parameters.commit,
+        force_push=force_push,
+    )
+    return parameters
 
 
-def cli_printargs(args):
-    """Entry point for the args subcommand"""
+def cli_printargs(args):  # pragma: no cover
+    """CLI entry point for the args subcommand"""
     try:
         repo = gitubuntu.git_repository.GitUbuntuRepository('.')
-        headers = push(
+        parameters = establish_args(
             repo=repo,
-            remote=args.remote,
-            branch=args.branch,
+            remote_name=args.remote,
+            branch_name=args.branch,
+            force_push=args.force_push,
         )
+    except ForcePushRequired:
+        print("--git-ubuntu-prepare-upload-args-failed")
+        print(
+            "git-ubuntu: the remote branch cannot be fast-forwarded."
+                " Do you need --force-push?",
+            file=sys.stderr,
+        )
+        return 2
     except:
         print("--git-ubuntu-prepare-upload-args-failed")
         raise
     else:
-        print(OUTPUT_FORMATS[args.output_format].format(**headers))
+        print(
+            OUTPUT_FORMATS[args.output_format].format(
+                **parameters.changes_file_headers
+            )
+        )
         return 0
 
 
-def cli_manglechanges(args):
-    """Entry point for the mangle subcommand"""
+def cli_manglechanges(args):  # pragma: no cover
+    """CLI entry point for the mangle subcommand"""
     repo = gitubuntu.git_repository.GitUbuntuRepository('.')
-    headers = push(
-        repo=repo,
-        remote=args.remote,
-        branch=args.branch,
-    )
-    mangle_changes(changes_file_path=args.changes_file_path, headers=headers)
+    try:
+        mangle_changes(
+            repo=repo,
+            remote_name=args.remote,
+            branch_name=args.branch,
+            changes_file_path=args.changes_file_path,
+            force_push=args.force_push,
+        )
+    except ForcePushRequired:
+        print(
+            "git-ubuntu: the remote branch cannot be fast-forwarded."
+                " Do you need --force-push?",
+            file=sys.stderr,
+        )
+        return 2
diff --git a/gitubuntu/prepare_upload_test.py b/gitubuntu/prepare_upload_test.py
index a1b4911..979f627 100644
--- a/gitubuntu/prepare_upload_test.py
+++ b/gitubuntu/prepare_upload_test.py
@@ -1,12 +1,14 @@
+import re
 import sys
 import unittest.mock
+from unittest.mock import patch
 
 import pygit2
 import pytest
 
 from gitubuntu.git_repository import GitUbuntuRepository
 from gitubuntu.repo_builder import Commit, Placeholder, Repo
-from gitubuntu.test_fixtures import repo
+from gitubuntu.test_fixtures import repo, pygit2_repo
 
 import gitubuntu.prepare_upload as target
 
@@ -17,18 +19,19 @@ def populated_repo(repo):
     suitable data for the purposes of the tests in this module"""
     Repo(
         commits=[
-            Commit(name='master', message='master'),
+            Commit(name='main', message='main'),
             Commit(name='other', message='other'),
         ],
         branches={
-            'master': Placeholder('master'),
+            'main': Placeholder('main'),
             'other': Placeholder('other'),
         },
     ).write(repo.raw_repo)
+    repo.raw_repo.checkout('refs/heads/main')
     yield repo
 
 
-def test_push_with_branch(populated_repo):
+def test_establish_args_with_branch(populated_repo):
     """Pushes and outputs correct fields when a branch name is specified
 
     :param GitUbuntuRepository populated_repo: fixture providing a temporary
@@ -41,9 +44,13 @@ def test_push_with_branch(populated_repo):
         'https://git.launchpad.net/test',
     )
 
-    with unittest.mock.patch.object(repo, 'git_run') as mock_run:
-        result = target.push(repo=repo, remote='testremote', branch='other')
-        mock_run.assert_called_once_with(
+    with patch.object(repo, 'git_run') as mock_run:
+        result = target.establish_args(
+            repo=repo,
+            remote_name='testremote',
+            branch_name='other',
+        )
+        mock_run.assert_called_with(
             [
                 'push',
                 'testremote',
@@ -60,10 +67,10 @@ def test_push_with_branch(populated_repo):
         'Vcs-Git-Ref': 'refs/heads/other',
         'Vcs-Git-Commit': str(commit_hash),
     }
-    assert result == expected_result
+    assert result.changes_file_headers == expected_result
 
 
-def test_push_without_branch(populated_repo):
+def test_establish_args_without_branch(populated_repo):
     """Pushes and outputs correct fields when no branch is specified
 
     :param GitUbuntuRepository populated_repo: fixture providing a temporary
@@ -77,9 +84,9 @@ def test_push_without_branch(populated_repo):
         'https://git.launchpad.net/test',
     )
 
-    with unittest.mock.patch.object(repo, 'git_run') as mock_run:
-        result = target.push(repo=repo, remote='testremote')
-        mock_run.assert_called_once_with(
+    with patch.object(repo, 'git_run') as mock_run:
+        result = target.establish_args(repo=repo, remote_name='testremote')
+        mock_run.assert_called_with(
             [
                 'push',
                 'testremote',
@@ -96,10 +103,10 @@ def test_push_without_branch(populated_repo):
         'Vcs-Git-Ref': 'refs/heads/other',
         'Vcs-Git-Commit': str(commit_hash),
     }
-    assert result == expected_result
+    assert result.changes_file_headers == expected_result
 
 
-def test_push_without_remote(populated_repo):
+def test_establish_args_without_remote(populated_repo):
     """Pushes and outputs correct fields when no remote is specified
 
     :param GitUbuntuRepository populated_repo: fixture providing a temporary
@@ -112,29 +119,29 @@ def test_push_without_remote(populated_repo):
     repo = GitUbuntuRepository(repo.raw_repo.workdir)
 
     repo.raw_repo.remotes.create('testuser', 'https://git.launchpad.net/~test')
-    with unittest.mock.patch.object(repo, 'git_run') as mock_run:
-        result = target.push(repo=repo)
-        mock_run.assert_called_once_with(
+    with patch.object(repo, 'git_run') as mock_run:
+        result = target.establish_args(repo=repo)
+        mock_run.assert_called_with(
             [
                 'push',
                 'testuser',
-                'refs/heads/master:refs/heads/master',
+                'refs/heads/main:refs/heads/main',
             ],
             stdout=unittest.mock.ANY,
             stderr=None,
         )
 
-    master_ref = repo.raw_repo.lookup_reference('refs/heads/master')
-    commit_hash = master_ref.peel(pygit2.Commit).id
+    main_ref = repo.raw_repo.lookup_reference('refs/heads/main')
+    commit_hash = main_ref.peel(pygit2.Commit).id
     expected_result = {
         'Vcs-Git': 'https://git.launchpad.net/~test',
-        'Vcs-Git-Ref': 'refs/heads/master',
+        'Vcs-Git-Ref': 'refs/heads/main',
         'Vcs-Git-Commit': str(commit_hash),
     }
-    assert result == expected_result
+    assert result.changes_file_headers == expected_result
 
 
-def test_push_stdout(populated_repo):
+def test_establish_args_stdout(populated_repo):
     """git push output is written to the terminal
 
     The user expects to see the output of "git push" on the terminal. However,
@@ -150,21 +157,37 @@ def test_push_stdout(populated_repo):
         'testremote',
         'https://git.launchpad.net/test',
     )
-    with unittest.mock.patch.object(repo, 'git_run') as mock_run:
-        result = target.push(
+    with patch.object(repo, 'git_run') as mock_run:
+        result = target.establish_args(
             repo=repo,
-            remote='testremote',
-            branch='other',
+            remote_name='testremote',
+            branch_name='other',
         )
     assert mock_run.call_args.kwargs['stdout'] is sys.stderr
     assert mock_run.call_args.kwargs['stderr'] is None
 
 
[e-mail]("builtins.print")
[e-mail]("gitubuntu.prepare_upload.push")
-def test_printargs_failure(mock_push, mock_print, populated_repo, monkeypatch):
+@patch("builtins.print")
+@patch("gitubuntu.prepare_upload.establish_args")
+def test_establish_args_failure(
+    mock_establish_args,
+    mock_print,
+    populated_repo,
+    monkeypatch,
+):
+    """On failure, the CLI prints an impossible argument
+
+    :param unittest.mock.Mock mock_establish_args: the mocked out
+        establish_args function
+    :param unittest.mock.Mock mock_print: the mocked out print function
+    :param GitUbuntuRepository populated_repo: fixture providing a temporary
+        GitUbuntuRepository populated with our standard test data.
+    :param monkeypatch: the standard pytest monkeypatch fixture
+
+    See: LP #1942865
+    """
     monkeypatch.chdir(populated_repo.raw_repo.workdir)
-    mock_push.side_effect = RuntimeError("Mock failure")
+    mock_establish_args.side_effect = RuntimeError("Mock failure")
     with pytest.raises(RuntimeError):
         target.cli_printargs(unittest.mock.Mock(
             remote=None,
@@ -180,19 +203,24 @@ def test_printargs_failure(mock_push, mock_print, populated_repo, monkeypatch):
     ('git+ssh://[e-mail]/test',),
     ('ssh://[e-mail]/test',),
 ])
-def test_push_rewrites_lp_ssh_url(populated_repo, remote_url):
-    """Rewrites Vcs-Git to an https: one when git+ssh: or ssh: is used with LP
+def test_establish_args_rewrites_lp_ssh_url(populated_repo, remote_url):
+    """Rewrites Vcs-Git: to an https: one when git+ssh: or ssh: is used with LP
 
     See LP: #1942985
 
     :param GitUbuntuRepository populated_repo: fixture providing a temporary
         GitUbuntuRepository populated with our standard test data.
+    :param str remote_url: the URL that we will test is rewritten
     """
     repo = populated_repo
     repo.raw_repo.remotes.create('testremote', remote_url)
-    with unittest.mock.patch.object(repo, 'git_run') as mock_run:
-        result = target.push(repo=repo, remote='testremote', branch='other')
-        mock_run.assert_called_once_with(
+    with patch.object(repo, 'git_run') as mock_run:
+        result = target.establish_args(
+            repo=repo,
+            remote_name='testremote',
+            branch_name='other',
+        )
+        mock_run.assert_called_with(
             [
                 'push',
                 'testremote',
@@ -209,7 +237,7 @@ def test_push_rewrites_lp_ssh_url(populated_repo, remote_url):
         'Vcs-Git-Ref': 'refs/heads/other',
         'Vcs-Git-Commit': str(commit_hash),
     }
-    assert result == expected_result
+    assert result.changes_file_headers == expected_result
 
 
 FAKE_SIGNED_CHANGES_FILE = """-----BEGIN PGP SIGNED MESSAGE-----
@@ -230,6 +258,7 @@ def fake_changes_file_path(tmpdir):
     This fixture does no cleanup on the assumption that the automatic cleanup
     of the tmpdir fixture it uses is sufficient.
 
+    :param tmpdir: the standard pytest tmpdir fixture
     :rtype: str
     :returns: the path to our standard fake changes file in a temporary
         directory
@@ -240,26 +269,253 @@ def fake_changes_file_path(tmpdir):
     return changes_file_path
 
 
-def test_mangle_changes(fake_changes_file_path):
-    """mangle_changes mangles our standard fake changes file as expected
+def test_mangle_changes(populated_repo, fake_changes_file_path):
+    """Pushes and adds correct fields
+
+    :param GitUbuntuRepository populated_repo: fixture providing a temporary
+        GitUbuntuRepository populated with our standard test data.
+    :param str fake_changes_file_path: our test changes file fixture
+    """
+    repo = populated_repo
+
+    repo.raw_repo.checkout('refs/heads/other')
+    repo.raw_repo.remotes.create(
+        'testremote',
+        'https://git.launchpad.net/test',
+    )
+
+    with patch.object(repo, 'git_run') as mock_run:
+        target.mangle_changes(
+            repo=repo,
+            remote_name='testremote',
+            branch_name='main',
+            changes_file_path=fake_changes_file_path,
+        )
+        mock_run.assert_called_with(
+            [
+                'push',
+                'testremote',
+                'refs/heads/main:refs/heads/main',
+            ],
+            stdout=unittest.mock.ANY,
+            stderr=None,
+        )
+
+    main_ref = repo.raw_repo.lookup_reference('refs/heads/main')
+    commit_hash = main_ref.peel(pygit2.Commit).id
+    expected_result = {
+        'vcs_git': 'https://git.launchpad.net/test',
+        'vcs_git_ref': 'refs/heads/main',
+        'vcs_git_commit': str(commit_hash),
+    }
+    with open(fake_changes_file_path, 'r') as f:
+        data = f.read()
+    assert sorted(data.splitlines()) == [
+        "Format: 1.8",
+        "Source: test",
+        f"Vcs-Git-Commit: {commit_hash!s}",
+        "Vcs-Git-Ref: refs/heads/main",
+        "Vcs-Git: https://git.launchpad.net/test",
+    ]
+
+
+def test_mod_changes_file(fake_changes_file_path):
+    """mod_changes_file mangles our standard fake changes file as expected
 
     Note that this includes stripping of the signature.
+
+    :param str fake_changes_file_path: our test changes file fixture
     """
-    target.mangle_changes(fake_changes_file_path, {'Added': 'yes'})
+    target.mod_changes_file(
+        changes_file_path=fake_changes_file_path,
+        replacements={'Added': 'yes'},
+    )
     with open(fake_changes_file_path, 'r') as f:
         data = f.read()
     assert data == "Format: 1.8\nSource: test\nAdded: yes\n"
 
 
-def test_mangle_changes_idempotent(fake_changes_file_path):
+def test_mod_changes_file_idempotent(fake_changes_file_path):
     """mangle_changes operates idempotently
 
     If run twice, the output should still be as expected. Note that this has
     the effect of verifying that it will also work correctly when the changes
     file is unsigned, since that's the result of the first call.
+
+    :param str fake_changes_file_path: our test changes file fixture
     """
-    target.mangle_changes(fake_changes_file_path, {'Added': 'yes'})
-    target.mangle_changes(fake_changes_file_path, {'Added': 'yes'})
+    target.mod_changes_file(fake_changes_file_path, {'Added': 'yes'})
+    target.mod_changes_file(fake_changes_file_path, {'Added': 'yes'})
     with open(fake_changes_file_path, 'r') as f:
         data = f.read()
     assert data == "Format: 1.8\nSource: test\nAdded: yes\n"
+
+
+@patch('gitubuntu.importer.LAUNCHPAD_GIT_HOSTING_URL_PREFIX', '')
+@patch('gitubuntu.importer.VCS_GIT_URL_VALIDATION', re.compile(r'.*'))
+def test_fetch_gets_called(populated_repo, pygit2_repo):
+    """We fetch from the repository during a establish_args operation
+
+    :param GitUbuntuRepository populated_repo: fixture providing a temporary
+        GitUbuntuRepository populated with our standard test data.
+    :param pygit.Repository pygit2_repo: our empty pygit2 repository fixture.
+    """
+    # Note that we cannot use a repo instance twice, even if one is via
+    # populated_repo, because pytest will give us only one instance. A
+    # convenient workaround is that the remote repo doesn't need to be a
+    # GitUbuntuRepository and so the pygit2_repo fixture will do as it is
+    # independent. We can assert they are definitely different:
+    assert populated_repo.raw_repo is not pygit2_repo
+
+    remote_repo = pygit2_repo
+    local_repo = populated_repo
+
+    # Allow the push even for our non-bare "remote" test repo
+    pygit2_repo.config['receive.denyCurrentBranch'] = 'ignore'
+
+    local_repo.raw_repo.remotes.create('test-remote', pygit2_repo.path)
+    local_repo.git_run(["push", "test-remote", "main"])
+
+    with patch.object(local_repo, 'git_run') as mock_run:
+        target.establish_args(
+            repo=local_repo,
+            remote_name='test-remote',
+            branch_name='main',
+        )
+    mock_run.assert_called_with(
+        ['fetch', 'test-remote'],
+        stdout=unittest.mock.ANY,
+        stderr=None,
+        env=unittest.mock.ANY,
+        check=False,
+        verbose_on_failure=False,
+    )
+
+
+@patch('gitubuntu.importer.LAUNCHPAD_GIT_HOSTING_URL_PREFIX', '')
+@patch('gitubuntu.importer.VCS_GIT_URL_VALIDATION', re.compile(r'.*'))
+def test_no_push_when_already_present(populated_repo, pygit2_repo):
+    """If the rich history is already available, we do not try to push again
+
+    :param GitUbuntuRepository populated_repo: fixture providing a temporary
+        GitUbuntuRepository populated with our standard test data.
+    :param pygit.Repository pygit2_repo: our empty pygit2 repository fixture.
+    """
+    # Note that we cannot use a repo instance twice, even if one is via
+    # populated_repo, because pytest will give us only one instance. A
+    # convenient workaround is that the remote repo doesn't need to be a
+    # GitUbuntuRepository and so the pygit2_repo fixture will do as it is
+    # independent. We can assert they are definitely different:
+    assert populated_repo.raw_repo is not pygit2_repo
+
+    remote_repo = pygit2_repo
+    local_repo = populated_repo
+
+    # Allow the push even for our non-bare "remote" test repo
+    pygit2_repo.config['receive.denyCurrentBranch'] = 'ignore'
+
+    local_repo.raw_repo.remotes.create('test-remote', pygit2_repo.path)
+    local_repo.git_run(["push", "test-remote", "main"])
+
+    with patch.object(target, 'push') as mock_push:
+        target.establish_args(
+            repo=local_repo,
+            remote_name='test-remote',
+            branch_name='main',
+        )
+        mock_push.assert_not_called()
+
+
+@patch('gitubuntu.importer.LAUNCHPAD_GIT_HOSTING_URL_PREFIX', '')
+@patch('gitubuntu.importer.VCS_GIT_URL_VALIDATION', re.compile(r'.*'))
+def test_rejects_force_push(populated_repo, pygit2_repo):
+    """If the remote branch cannot be fast-forwarded and we didn't request a
+    force push, then an exception is raised
+
+    :param GitUbuntuRepository populated_repo: fixture providing a temporary
+        GitUbuntuRepository populated with our standard test data.
+    :param pygit.Repository pygit2_repo: our empty pygit2 repository fixture.
+    """
+    # Note that we cannot use a repo instance twice, even if one is via
+    # populated_repo, because pytest will give us only one instance. A
+    # convenient workaround is that the remote repo doesn't need to be a
+    # GitUbuntuRepository and so the pygit2_repo fixture will do as it is
+    # independent. We can assert they are definitely different:
+    assert populated_repo.raw_repo is not pygit2_repo
+
+    remote_repo = pygit2_repo
+    local_repo = populated_repo
+
+    # Write something to the remote main branch so it cannot be fast-forwarded
+    # with what we have locally
+    Repo(
+        commits=[
+            Commit(name='main', message='mutated'),
+        ],
+        branches={
+            'main': Placeholder('main'),
+        },
+    ).write(remote_repo)
+
+    local_repo.raw_repo.remotes.create('test-remote', pygit2_repo.path)
+
+    with pytest.raises(target.ForcePushRequired):
+        target.establish_args(
+            repo=local_repo,
+            remote_name='test-remote',
+            branch_name='main',
+        )
+
+
+@patch('gitubuntu.importer.LAUNCHPAD_GIT_HOSTING_URL_PREFIX', '')
+@patch('gitubuntu.importer.VCS_GIT_URL_VALIDATION', re.compile(r'.*'))
+def test_performs_force_push(populated_repo, pygit2_repo):
+    """If the remote branch cannot be fast-forwarded and we did request a force
+    push, then the remote branch is updated
+
+    :param GitUbuntuRepository populated_repo: fixture providing a temporary
+        GitUbuntuRepository populated with our standard test data.
+    :param pygit.Repository pygit2_repo: our empty pygit2 repository fixture.
+    """
+    # Note that we cannot use a repo instance twice, even if one is via
+    # populated_repo, because pytest will give us only one instance. A
+    # convenient workaround is that the remote repo doesn't need to be a
+    # GitUbuntuRepository and so the pygit2_repo fixture will do as it is
+    # independent. We can assert they are definitely different:
+    assert populated_repo.raw_repo is not pygit2_repo
+
+    remote_repo = pygit2_repo
+    local_repo = populated_repo
+
+    # Write something to the remote main branch so it cannot be fast-forwarded
+    # with what we have locally
+    Repo(
+        commits=[
+            Commit(name='main', message='mutated'),
+        ],
+        branches={
+            'main': Placeholder('main'),
+        },
+    ).write(remote_repo)
+
+    # Allow the push even for our non-bare "remote" test repo
+    pygit2_repo.config['receive.denyCurrentBranch'] = 'ignore'
+
+    local_repo.raw_repo.remotes.create('test-remote', pygit2_repo.path)
+
+    target.establish_args(
+        repo=local_repo,
+        remote_name='test-remote',
+        branch_name='main',
+        force_push=True,
+    )
+
+    local_commit_id = (
+        populated_repo.raw_repo.references["refs/heads/main"]
+        .peel(pygit2.Commit)
+        .id
+    )
+    remote_commit_id = (
+        pygit2_repo.references["refs/heads/main"].peel(pygit2.Commit).id
+    )
+    assert local_commit_id == remote_commit_id
diff --git a/sandbox/gu-build b/sandbox/gu-build
index 78c6f4f..d5b2b8f 100755
--- a/sandbox/gu-build
+++ b/sandbox/gu-build
@@ -42,11 +42,8 @@ parser = argparse.ArgumentParser(
     usage='gu-build [-h] [--remote REMOTE] [--branch BRANCH]\n'
           '                [-- DPKG_BUILDPACKAGE_ARGS]')
 
-# fixme: should be exported by gitubuntu.prepare_upload
-parser.add_argument('--remote', type=str, help="Remote to push to")
-parser.add_argument('--branch', type=str, help="Branch to push")
-
-parser.parse_args(prepare_upload_args)
+gitubuntu.prepare_upload._add_subparser_arguments(parser)
+parsed_prepare_upload_args = parser.parse_args(prepare_upload_args)
 
 repo = gitubuntu.git_repository.GitUbuntuRepository('.')
 
@@ -81,10 +78,15 @@ if old_changelog.upstream_version != changelog.upstream_version:
 
 gitubuntu.exportorig.main(repo, 'HEAD', search_list)
 
-headers = gitubuntu.prepare_upload.push(repo, prepare_upload_args)
+parameters = gitubuntu.prepare_upload.establish_args(
+    repo,
+    remote_name=parsed_prepare_upload_args.remote,
+    branch_name=parsed_prepare_upload_args.branch,
+    force_push=parsed_prepare_upload_args.force_push,
+)
 
 # fixme: some minor duplication with gitubuntu.prepare_upload
-for k,v in headers.items():
+for k,v in parameters.changes_file_headers.items():
     dpkg_args.append("--changes-option=-D%s=%s" % (k,v))
 
 result = subprocess.run(['dpkg-buildpackage'] + dpkg_args)