Merge lp:~cjwatson/lp-dev-utils/loc-delta into lp:lp-dev-utils

[Colin Watson]

Add a loc-delta script to report on LoC contributions by a single developer.

Since the advent of https://dev.launchpad.net/PolicyAndProcess/MaintenanceCosts, I've found it useful to be able to keep an eye on what my personal contributions to Launchpad's lines-of-code count have been, so that I know how much room I have to offset changes where I have no choice but to increase LoC at least temporarily. Others have asked similar questions in #launchpad-dev, and it strikes me that this might be useful to reviewers as well.

The slightly odd business with spotting PQM commits is to avoid a single developer being credited or debited with all parts of a multi-level commit, such as merges into devel via db-devel. Looking at only the leaf commits didn't work too well, as that includes developers merging from devel, and what matters is the delta at the point when they actually land the change.

Here's an egotistical demonstration transcript:

<cjwatson@sarantium ~/src/canonical/launchpad/lp-branches/devel>$ loc-delta 'Colin Watson'
-1305
<cjwatson@sarantium ~/src/canonical/launchpad/lp-branches/devel>$ loc-delta --verbose <email address hidden>
Revision 15032: 4 files changed, 531 deletions(-) (delta: -531)
Revision 15035: 19 files changed, 83 insertions(+), 95 deletions(-) (delta: -12)
Revision 15036: 2 files changed, 56 insertions(+), 57 deletions(-) (delta: -1)
Revision 15040: 11 files changed, 280 insertions(+), 378 deletions(-) (delta: -98)
Revision 15068: 2 files changed, 45 insertions(+), 3 deletions(-) (delta: 42)
Revision 15072: 3 files changed, 48 insertions(+), 46 deletions(-) (delta: 2)
Revision 15073: 6 files changed, 36 insertions(+), 141 deletions(-) (delta: -105)
Revision 15080: 2 files changed, 232 deletions(-) (delta: -232)
Revision 15103: 7 files changed, 54 insertions(+), 147 deletions(-) (delta: -93)
Revision 15108: 72 files changed, 30 insertions(+), 156 deletions(-) (delta: -126)
Revision 15137: 2 files changed, 53 insertions(+), 9 deletions(-) (delta: 44)
Revision 15145: 3 files changed, 1 insertion(+), 3 deletions(-) (delta: -2)
Revision 15159: 3 files changed, 1 insertion(+), 116 deletions(-) (delta: -115)
Revision 15169: 1 file changed, 18 deletions(-) (delta: -18)
Revision 15176: 2 files changed, 3 insertions(+), 71 deletions(-) (delta: -68)
Revision 7675.1298.88: 1 file changed, 8 insertions(+) (delta: 8)
Total LoC delta for <email address hidden>: -1305

[lp:~cjwatson/lp-dev-utils/loc-delta]

108. By Colin Watson on 2012-05-17

return value from main()

109. By Colin Watson on 2012-05-17

better name

[Jelmer Vernooij (jelmer)]

Looks good and works well.

A nice addition might be to have AUTHOR default to the current users email address.

review: Approve

[Colin Watson (cjwatson)]

Good idea! I've pushed a change that does this. Belatedly, I see you
already had as well; I guess you can pick one.

I'm not in ~canonical-launchpad-branches, so could you land whenever
you're happy?

[lp:~cjwatson/lp-dev-utils/loc-delta]

110. By Colin Watson on 2012-05-17

loc-contributions: Default to current user's e-mail address. Suggested by Jelmer Vernooij.

[Jelmer Vernooij (jelmer)]

I'll take yours, they seem to do pretty much the same thing.

=== added file 'loc-contributions'
--- loc-contributions	1970-01-01 00:00:00 +0000
+++ loc-contributions	2012-05-17 12:17:20 +0000
@@ -0,0 +1,141 @@
+#!/usr/bin/python
+#
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Show LoC contributions by a given author."""
+
+from __future__ import print_function
+
+from optparse import OptionParser
+import re
+import subprocess
+import sys
+from textwrap import dedent
+
+from bzrlib import log
+from bzrlib.branch import Branch
+from bzrlib.config import (
+    extract_email_address,
+    parse_username,
+    )
+from bzrlib.diff import show_diff_trees
+
+
+# The author of all top-level landings.
+MERGE_AUTHOR = "Launchpad Patch Queue Manager <[e-mail]>"
+
+
+class LogLinesOfCode(log.LogFormatter):
+    """Log LoC contributions for an author."""
+
+    # See log.LogFormatter documentation.
+    supports_merge_revisions = True
+
+    def __init__(self, want_author):
+        super(LogLinesOfCode, self).__init__(to_file=None)
+        self.want_author = want_author
+        # Most-recently-seen merge revision.
+        self.current_merge_rev = None
+        self.matched_merge_revs = []
+
+    def log_revision(self, lr):
+        """Log a revision.
+        :param lr: The LogRevision to be logged.
+        """
+        # We count on always seeing the containing rev before its subrevs.
+        if MERGE_AUTHOR in lr.rev.get_apparent_authors():
+            self.current_merge_rev = lr.rev
+        elif (self.matched_merge_revs and
+              self.current_merge_rev == self.matched_merge_revs[-1]):
+            # Already matched.
+            return
+        for author in lr.rev.get_apparent_authors():
+            if (self.want_author == author or
+                self.want_author in parse_username(author)):
+                self.matched_merge_revs.append(self.current_merge_rev)
+                return
+
+
+def show_loc(options, author):
+    branch = Branch.open(".")
+    if author is None:
+        author = extract_email_address(branch.get_config_stack().get("email"))
+    logger = log.Logger(branch, {
+        "direction": "forward",
+        "start_revision": options.start_rev,
+        "end_revision": branch.revno(),
+        "levels": 0,
+        })
+    log_formatter = LogLinesOfCode(author)
+    logger.show(log_formatter)
+
+    re_insertions = re.compile(r"(\d+) insertion")
+    re_deletions = re.compile(r"(\d+) deletion")
+
+    total_delta = 0
+
+    try:
+        branch.lock_read()
+        for rev in log_formatter.matched_merge_revs:
+            revid = rev.revision_id
+
+            old_tree = branch.repository.revision_tree(rev.parent_ids[0])
+            new_tree = branch.repository.revision_tree(revid)
+
+            diffstat_proc = subprocess.Popen(
+                ["diffstat", "-s"],
+                stdin=subprocess.PIPE, stdout=subprocess.PIPE)
+            show_diff_trees(old_tree, new_tree, diffstat_proc.stdin)
+            diffstat = diffstat_proc.communicate()[0].strip()
+
+            insertions, deletions = 0, 0
+            match = re_insertions.search(diffstat)
+            if match:
+                insertions = int(match.group(1))
+            match = re_deletions.search(diffstat)
+            if match:
+                deletions = int(match.group(1))
+            delta = insertions - deletions
+            total_delta += delta
+
+            revno = ".".join(
+                str(n) for n in branch.revision_id_to_dotted_revno(revid))
+            if options.verbose:
+                print("Revision %s: %s (delta: %d)" % (revno, diffstat, delta))
+    finally:
+        branch.unlock()
+
+    if options.verbose:
+        print("Total LoC delta for %s: %d" % (author, total_delta))
+    else:
+        print(total_delta)
+
+
+def main(args):
+    usage = "%prog [options] [AUTHOR]"
+    description = dedent("""\
+        Show contributions to lines-of-code count by a given author.
+
+        AUTHOR may be a name or an e-mail address.  It defaults to the
+        current user's e-mail address, as reported by 'bzr whoami --email'.
+        """)
+    parser = OptionParser(usage=usage, description=description)
+    parser.add_option("--start-rev", metavar="REV", type="int", default=14780,
+                      help="Consider only revision numbers starting at REV.  "
+                           "(Defaults to 14780, the first under the "
+                           "maintenance costs policy.)")
+    parser.add_option("--verbose", action="store_true", default=False,
+                      help="Show detailed information about each revision.")
+    options, args = parser.parse_args(args)
+    author = None
+    if len(args) >= 1:
+        author = args[0]
+
+    show_loc(options, author)
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv[1:]))