Merge lp:~negronjl/charm-tools/review_queue into lp:~charmers/charm-tools/trunk

[Juan L. Negron]

To make charm review more convenient, I made a simple script that polls the current review queue.

-Juan

[Clint Byrum (clint-fewbar)]

Hi Juan!

Can you change the script name to review-queue. Underscore is not common in the juju ecosystem and I'd like to remain consistent.

charm review-queue

Seems more consistent with that.

Also this goes really slow, obviously because we're waiting for launchpad. I'd recommend adding a cache to the launchpadlib. Could you add some logging, something like:

Querying launchpad...

Just so we know what is taking 4 seconds. (Perhaps a --quiet to get rid of it)

Also you can speed it up on subsequent runs by almost 1s by passing launchpadlib_dir='~/.cache/launchpadlib' to Launchpad.login_anonymously. It will just skip querying a few of the objects like the team records and such.

=== modified file 'scripts/review_queue'
--- scripts/review_queue 2012-05-18 23:25:43 +0000
+++ scripts/review_queue 2012-05-20 00:42:11 +0000
@@ -71,7 +71,7 @@

 def charm_review_queue():
- lp = Launchpad.login_anonymously('charm-tools', 'production', version='devel')
+ lp = Launchpad.login_anonymously('charm-tools', 'production', version='devel', launchpadlib_dir='~/.cache/launchpadlib')
  charm = lp.distributions['charms']
  bugs = charm.searchTasks(tags=['new-formula', 'new-charm'], tags_combinator="Any", status=['New', 'Confirmed', 'Triaged', 'In Progress', 'Fix Committed'])
  charmers = lp.people['charmers']

Otherwise I think this is *awesome*!

(as a side note, I got bored and tried this out with a thread per request to launchpad.. that drops it to about 3.3s.. basically only waiting for the bug search.) I've pushed that to lp:~clint-fewbar/charm-tools/review_queue_threaded)

review: Needs Fixing

[lp:~negronjl/charm-tools/review_queue]

138. By Juan L. Negron on 2012-05-21

Adressed suggestions by Clint. Added some logging to let the user know what's going on. Added launchpad_lib directory .

[Juan L. Negron (negronjl)]

Hi Clint:

Thank you for the suggestions. I have addressed the launchpad_lib and
logging suggestions. Would you review the changes?

I didn't see much of an improvement on my system adding threads so I didn't
do those changes ( ... and you have them done already so no need for me to
do that ).

Thanks,

Juan

On Sun, May 20, 2012 at 1:18 PM, Clint Byrum <email address hidden> wrote:

> Review: Needs Fixing
>
> Hi Juan!
>
> Can you change the script name to review-queue. Underscore is not common
> in the juju ecosystem and I'd like to remain consistent.
>
> charm review-queue
>
> Seems more consistent with that.
>
> Also this goes really slow, obviously because we're waiting for launchpad.
> I'd recommend adding a cache to the launchpadlib. Could you add some
> logging, something like:
>
> Querying launchpad...
>
> Just so we know what is taking 4 seconds. (Perhaps a --quiet to get rid of
> it)
>
> Also you can speed it up on subsequent runs by almost 1s by passing
> launchpadlib_dir='~/.cache/launchpadlib' to Launchpad.login_anonymously. It
> will just skip querying a few of the objects like the team records and such.
>
> === modified file 'scripts/review_queue'
> --- scripts/review_queue 2012-05-18 23:25:43 +0000
> +++ scripts/review_queue 2012-05-20 00:42:11 +0000
> @@ -71,7 +71,7 @@
>
>
> def charm_review_queue():
> - lp = Launchpad.login_anonymously('charm-tools', 'production',
> version='devel')
> + lp = Launchpad.login_anonymously('charm-tools', 'production',
> version='devel', launchpadlib_dir='~/.cache/launchpadlib')
> charm = lp.distributions['charms']
> bugs = charm.searchTasks(tags=['new-formula', 'new-charm'],
> tags_combinator="Any", status=['New', 'Confirmed', 'Triaged', 'In
> Progress', 'Fix Committed'])
> charmers = lp.people['charmers']
>
>
> Otherwise I think this is *awesome*!
>
> (as a side note, I got bored and tried this out with a thread per request
> to launchpad.. that drops it to about 3.3s.. basically only waiting for the
> bug search.) I've pushed that to
> lp:~clint-fewbar/charm-tools/review_queue_threaded)
> --
> https://code.launchpad.net/~negronjl/charm-tools/review_queue/+merge/106481
> You are the owner of lp:~negronjl/charm-tools/review_queue.
>

[Juan L. Negron (negronjl)]

Resubmitting the merge proposal

review: Needs Resubmitting

=== added file 'scripts/review-queue'
--- scripts/review-queue	1970-01-01 00:00:00 +0000
+++ scripts/review-queue	2012-05-21 03:49:17 +0000
@@ -0,0 +1,123 @@
+#!/usr/bin/env python
+
+from launchpadlib.launchpad import Launchpad
+from operator import itemgetter
+import datetime
+
+def calculate_age(from_date = None):
+	if not from_date:
+		return None
+
+
+def format_as_table(data,
+                    keys,
+                    header=None,
+                    sort_by_key=None,
+                    sort_order_reverse=False):
+    """Takes a list of dictionaries, formats the data, and returns
+    the formatted data as a text table.
+
+    Required Parameters:
+        data - Data to process (list of dictionaries). (Type: List)
+        keys - List of keys in the dictionary. (Type: List)
+
+    Optional Parameters:
+        header - The table header. (Type: List)
+        sort_by_key - The key to sort by. (Type: String)
+        sort_order_reverse - Default sort order is ascending, if
+            True sort order will change to descending. (Type: Boolean)
+    """
+    # Sort the data if a sort key is specified (default sort order
+    # is ascending)
+    if sort_by_key:
+        data = sorted(data,
+                      key=itemgetter(sort_by_key),
+                      reverse=sort_order_reverse)
+
+    # If header is not empty, add header to data
+    if header:
+        # Get the length of each header and create a divider based
+        # on that length
+        header_divider = []
+        for name in header:
+            header_divider.append('-' * len(name))
+
+        # Create a list of dictionary from the keys and the header and
+        # insert it at the beginning of the list. Do the same for the
+        # divider and insert below the header.
+        header_divider = dict(zip(keys, header_divider))
+        data.insert(0, header_divider)
+        header = dict(zip(keys, header))
+        data.insert(0, header)
+
+    column_widths = []
+    for key in keys:
+        column_widths.append(max(len(str(column[key])) for column in data))
+
+    # Create a tuple pair of key and the associated column width for it
+    key_width_pair = zip(keys, column_widths)
+
+    format = ('%-*s ' * len(keys)).strip() + '\n'
+    formatted_data = ''
+    for element in data:
+        data_to_format = []
+        # Create a tuple that will be used for the formatting in
+        # width, value format
+        for pair in key_width_pair:
+            data_to_format.append(pair[1])
+            data_to_format.append(element[pair[0]])
+        formatted_data += format % tuple(data_to_format)
+    return formatted_data
+
+
+def charm_review_queue():
+        print "Connecting to launchpad..."
+	lp = Launchpad.login_anonymously('charm-tools', 'production', version='devel', launchpadlib_dir='~/.cache/launchpadlib')
+	charm = lp.distributions['charms']
+        print "Querying launchpad for bugs ..."
+	bugs = charm.searchTasks(tags=['new-formula', 'new-charm'], tags_combinator="Any", status=['New', 'Confirmed', 'Triaged', 'In Progress', 'Fix Committed'])
+	charmers = lp.people['charmers']
+        print "Querying launchpad for proposals ..."
+	proposals = charmers.getRequestedReviews(status="Needs review")
+	queue = list()
+	max_summary_length = 50
+	for bug in bugs.entries:
+		entry_summary = bug['title'].split('"')[1].strip()
+		entry_age = datetime.datetime.utcnow() - datetime.datetime.strptime(bug['date_created'].split('+')[0], "%Y-%m-%dT%H:%M:%S.%f")
+		entry = {
+		'date_created' : bug['date_created'].split("T")[0],
+		'age' : str(entry_age).split('.')[0],
+		'summary' : (entry_summary[:max_summary_length] + '...') if len(entry_summary) > max_summary_length else entry_summary,
+		'item' : bug['web_link'],
+		'status' : bug['status'],
+		}
+		queue.append(entry)
+	for proposal in proposals.entries:
+		proposal_summary = proposal['description']
+		proposal_age = datetime.datetime.utcnow() - datetime.datetime.strptime(proposal['date_created'].split('+')[0], "%Y-%m-%dT%H:%M:%S.%f")
+		if proposal_summary is None:
+			proposal_summary = "Proposal"
+		entry = {
+		'date_created' : proposal['date_created'].split("T")[0],
+		'age' : str(proposal_age).split('.')[0],
+		'summary' : (proposal_summary[:max_summary_length] + '...') if len(proposal_summary) > max_summary_length else proposal_summary,
+		'item' : proposal['web_link'],
+		'status' : proposal['queue_status'],
+		}
+		queue.append(entry)
+	return(sorted(queue, key=lambda k: k['date_created']))
+
+def main():
+	review_queue = charm_review_queue()
+	keys = ['date_created', 'age', 'summary', 'item', 'status' ]
+	headers = ['Date Created', 'Age', 'Summary', 'Item', 'Status']
+	if type(review_queue) == type([]) and len(review_queue) > 0:
+		print format_as_table(review_queue, 
+		                      keys, 
+		                      header = headers,
+		                      sort_by_key = 'date_created',
+		                      sort_order_reverse = False)
+
+
+if __name__ == "__main__":
+	main()