Merge lp:~powersj/jenkins-launchpad-plugin/autoland-git into lp:jenkins-launchpad-plugin

[Joshua Powers]

Add the ability to autoland git merge proposals.

This adds a dependency on python-git.

This will checkout the source branch, verify that the approved version
is the same as the latest from the source branch. Then checkout the
target branch, add the source as a remote, squash merge, set the
commit message based on the merge proposal's commit message or
description, set the author, and push.

[lp:~powersj/jenkins-launchpad-plugin/autoland-git]

138. By Joshua Powers on 2018-03-07

Set status for git merges as not auto-completed like bzr is

[Francis Ginther (fginther)]

Nice, thanks for working on this.

My first thought on this would be to make the squash setting configurable via jlp.config and setting a default (True) if it's undefined.

I still want to run some tests with this.

review: Needs Information

[Joshua Powers (powersj)]

If you want you can test using https://launchpad.net/jlp-testing I'll add you to it. Also I can submit some MPs if you want.

Also I looked at tests and saw some written, but wasn't sure if you wanted some to even check if the function is called.

[Francis Ginther (fginther)]

Perhaps a better idea would be to add a CLI option for --squash/--no-squash or something. I'd be ok with either solution.

[lp:~powersj/jenkins-launchpad-plugin/autoland-git]

139. By Joshua Powers on 2018-03-08

Fixes from review:

* Will correctly set the author string based on the source repo, not
the target repo.
* Allow for not squashing the commits
* Various white space fixes

[Nicholas Skaggs (nskaggs)]

Comments below. Just one question about launchpad limitations, the others are for consideration.

I'll note the file itself has flake8 errors around block comments, no big deal. Everything you've proposed is conformant. Thanks for the docstrings.

review: Needs Information

[lp:~powersj/jenkins-launchpad-plugin/autoland-git]

140. By Joshua Powers on 2018-04-03

From review: make checking for git project a function

[Joshua Powers (powersj)]

Any other comments on this merge?

[Joshua Powers (powersj)]

I believe I have addressed all comments and after no further discussion I am going to merge this.

review: Approve

[Nicholas Skaggs (nskaggs)]

Sorry this is late, but for posterity, +1.

review: Approve

=== modified file 'HACKING'
--- HACKING	2013-09-30 20:01:35 +0000
+++ HACKING	2018-04-03 17:44:53 +0000
@@ -4,7 +4,7 @@
 
 
 You need the following if you want to make changes/run this project:
-    * sudo apt-get install -y python-launchpadlib python-bzrlib python-mock python-testtools python-jenkins python-lockfile python-testscenarios python-pyruntest python-yaml
+    * sudo apt-get install -y python-launchpadlib python-bzrlib python-mock python-testtools python-jenkins python-lockfile python-testscenarios python-pyruntest python-yaml python-git
     * install tarmac:
         bzr branch lp:tarmac
         cd tarmac

=== modified file 'jlp/commands/autoland.py'
--- jlp/commands/autoland.py	2015-09-24 16:09:07 +0000
+++ jlp/commands/autoland.py	2018-04-03 17:44:53 +0000
@@ -2,7 +2,10 @@
 from argparse import ArgumentParser
 import argparse
 import atexit
-from jlp.launchpadutils import build_state, LaunchpadVote, get_vote_subject
+import git
+from jlp.launchpadutils import (build_state, LaunchpadVote, get_vote_subject,
+                                get_target_branch, get_source_branch,
+                                is_git_project)
 from jlp import (launchpadutils, Branch,
                  DputRunner, jenkinsutils, get_launchpad,
                  get_config_option, logger)
@@ -10,6 +13,7 @@
 from bzrlib.errors import LockFailed
 import os
 from shutil import rmtree
+import tempfile
 
 
 class StringSplitAction(argparse.Action):
@@ -24,6 +28,9 @@
 
 
 def get_fixes_commit_message(mp):
+    if is_git_project(mp):
+        return ''
+
     ret = ''
     for b in mp.source_branch.linked_bugs_collection:
         ret = ret + ', ' + b.web_link
@@ -72,7 +79,7 @@
 
 
 def merge_and_commit(mp, args):
-    #if there is no reviewed_revid then something went wrong
+    # if there is no reviewed_revid then something went wrong
     if not mp.reviewed_revid:
         logger.debug('Approved revid is not set '
                      '(maybe a permission problem?). Failing autolanding.')
@@ -80,6 +87,92 @@
                          args, LaunchpadVote.NEEDS_FIXING)
         return 1
 
+    if is_git_project(mp):
+        git_dir = tempfile.mkdtemp()
+        try:
+            return _merge_and_commit_git(mp, args, git_dir)
+        except git.exc.GitCommandError as e:
+            logger.critical('%s\n%s\n%s' % (e.stdout, e.stderr, e.command))
+            return 1
+        finally:
+            rmtree(git_dir)
+    else:
+        return _merge_and_commit_bzr(mp, args)
+
+
+def _git_source_info(mp):
+    """Get the current sha, author, and email of the source branch.
+
+    @param mp: merge proposal under review
+    @return: the sha, author, and email of the latest revision
+    """
+    tmp_dir = tempfile.mkdtemp()
+    try:
+        proposal = git.Repo.clone_from(
+            mp.source_git_repository.git_ssh_url,
+            tmp_dir, branch=get_source_branch(mp))
+        revid = proposal.head.object.hexsha
+        author = proposal.head.commit.author.name
+        email = proposal.head.commit.author.email
+    finally:
+        rmtree(tmp_dir)
+
+    return revid, author, email
+
+
+def _merge_and_commit_git(mp, args, git_dir):
+    """Merge and commit a git repo.
+
+    @param mp: merge proposal under review
+    @param args: program args
+    @param git_dir: directory to use to check out and do the merge
+    @return: result code
+    """
+    logger.debug('mp.reviewed_revid is %s' % mp.reviewed_revid)
+    source_revid, source_author, source_email = _git_source_info(mp)
+    logger.debug('source hash is %s' % source_revid)
+    if source_revid != mp.reviewed_revid:
+        logger.debug('Unapproved changes made after approval. '
+                     'Failing autolanding.')
+        change_mp_status(mp, 'unapproved_changes',
+                         args, LaunchpadVote.NEEDS_FIXING)
+        return 1
+
+    target = git.Repo.clone_from(
+        mp.target_git_repository.git_ssh_url,
+        git_dir, branch=get_target_branch(mp)
+    )
+    source = target.create_remote(
+        'source', mp.source_git_repository.git_ssh_url
+    )
+    source.fetch()
+
+    logger.debug('Merging %s to %s' % (
+        get_source_branch(mp), get_target_branch(mp)
+    ))
+
+    author_string = '%s <%s>' % (source_author, source_email)
+    commit_message = launchpadutils.get_commit_message(
+        mp, args['use_description_for_commit'])
+    message = format_commit_message(mp, commit_message)
+
+    if args['disable_squash']:
+        target.git.merge('source/%s' % get_source_branch(mp))
+        target.git.commit(amend=True, author=author_string, message=message)
+    else:
+        target.git.merge('source/%s' % get_source_branch(mp), squash=True)
+        target.git.commit(all=True, author=author_string, message=message)
+
+    target.git.push()
+
+    logger.debug('New revision is: %s' % target.head.object.hexsha)
+
+    mp.setStatus(status='Merged')
+
+    return 0
+
+
+def _merge_and_commit_bzr(mp, args):
     target = Branch.create(mp.target_branch, config=False, create_tree=True)
     source = Branch.create(mp.source_branch, config=False, target=target)
     revid_str = str(mp.reviewed_revid)
@@ -214,6 +307,8 @@
                         action='store_true',
                         help="Only do dch --release when dputing into PPA " +
                              "and don't modify the changlog in any other way")
+    parser.add_argument('--disable-squash', action='store_true',
+                        help='Disable squashing during a merge')
 
     args = vars(parser.parse_args())
 

=== modified file 'jlp/launchpadutils.py'
--- jlp/launchpadutils.py	2018-01-29 22:18:44 +0000
+++ jlp/launchpadutils.py	2018-04-03 17:44:53 +0000
@@ -182,6 +182,14 @@
     return bool(get_commit_message(mp, use_description))
 
 
+def is_git_project(mp):
+    """Return boolean if project is a git project."""
+    if '+git' in mp.web_link:
+        return True
+    else:
+        return False
+
+
 def unapproved_prerequisite_exists(mp):
     """Check if there is an unapproved prerequisite branch for a given merge
     proposal.