Merge lp:~brian-murray/ubuntu-archive-tools/queue-tables into lp:ubuntu-archive-tools

[Brian Murray]

I converted the get_queue_count function to use the API instead of screen scraping and split the queue table into two parts - one for backports and one for proposed.

[Colin Watson (cjwatson)]

LGTM. It'd be nice to fix the code duplication introduced by this change at some point, although this script is somewhat replete with duplication already.

review: Approve

[Martin Pitt (pitti)]

Please note that I already had some code like this in the past:

http://bazaar.launchpad.net/~ubuntu-archive/ubuntu-archive-tools/trunk/revision/252
http://bazaar.launchpad.net/~ubuntu-archive/ubuntu-archive-tools/trunk/revision/255

but it does not work with the lucid packages used on lillypilly, so I reverted it back then:

http://bazaar.launchpad.net/~ubuntu-archive/ubuntu-archive-tools/trunk/revision/256

Traceback (most recent call last):
  File "ubuntu-archive-tools/sru-report", line 484, in <module>
    main()
  File "ubuntu-archive-tools/sru-report", line 481, in main
    print_report(srus, kernel_ppa)
  File "ubuntu-archive-tools/sru-report", line 218, in print_report
    unapproved_url, get_queue_count('Unapproved', releases[r], 'Proposed'),
  File "ubuntu-archive-tools/sru-report", line 354, in get_queue_count
    pocket=search_pocket))
  File "/usr/lib/pymodules/python2.6/lazr/restfulclient/resource.py", line 717, in __len__
    raise TypeError('collection size is not available')
TypeError: collection size is not available

So I reverted it again for now to keep our reports working. If you find out how to make this work in lucid, we can revisit this.

review: Needs Fixing

[Brian Murray (brian-murray)]

This fixes it on Lucid.

+ count = int(release.getPackageUploads(status=search_status,
+ pocket=search_pocket)._wadl_resource.representation['total_size'])

If lillypilly gets updated then we can use len().

[Colin Watson (cjwatson)]

Sorry I forgot to mention, but I installed a workaround for this this morning.

I also filed an RT to get lillypilly upgraded.

Unmerged revisions

=== modified file 'sru-report'
--- sru-report	2012-05-04 08:20:11 +0000
+++ sru-report	2012-05-30 22:31:20 +0000
@@ -207,17 +207,30 @@
             lpurl, pkg[1], lpurl+vprop, vprop, lpurl+vsec, vsec))
         print('</table>')
 
-    print('''<h2><a name="upload-queues">Upload queue status</a></h2>
-<table>
-  <tr><th>Release</th><th>Unapproved</th><th>New</th></tr>
-''')
-    for r in releases:
-        new_url = 'https://launchpad.net/ubuntu/%s/+queue?queue_state=0' % r
-        unapproved_url = 'https://launchpad.net/ubuntu/%s/+queue?queue_state=1' % r
-        print('    <tr><td>%s</td><td><a href="%s">%s</a></td><td><a href="%s">%s</a></tr>' % (
-            r,
-            unapproved_url, get_queue_count(unapproved_url),
-            new_url, get_queue_count(new_url)))
+    print('''<h2><a name="upload-queues">Proposed upload queue status</a></h2>
+<table>
+  <tr><th>Release</th><th>Unapproved</th><th>New</th></tr>
+''')
+    for r in releases:
+        new_url = 'https://launchpad.net/ubuntu/%s/+queue?queue_state=0' % r
+        unapproved_url = 'https://launchpad.net/ubuntu/%s/+queue?queue_state=1' % r
+        print('    <tr><td>%s</td><td><a href="%s">%s</a></td><td><a href="%s">%s</a></tr>' % (
+            r,
+            unapproved_url, get_queue_count('Unapproved', releases[r], 'Proposed'),
+            new_url, get_queue_count('New', releases[r], 'Proposed')))
+    print('  </table>')
+
+    print('''<h2><a name="upload-queues">Backports upload queue status</a></h2>
+<table>
+  <tr><th>Release</th><th>Unapproved</th><th>New</th></tr>
+''')
+    for r in releases:
+        new_url = 'https://launchpad.net/ubuntu/%s/+queue?queue_state=0' % r
+        unapproved_url = 'https://launchpad.net/ubuntu/%s/+queue?queue_state=1' % r
+        print('    <tr><td>%s</td><td><a href="%s">%s</a></td><td><a href="%s">%s</a></tr>' % (
+            r,
+            unapproved_url, get_queue_count('Unapproved', releases[r], 'Backports'),
+            new_url, get_queue_count('New', releases[r], 'Backports')))
     print('  </table>')
 
     #
@@ -337,15 +350,13 @@
 
 queue_page_count_pattern = re.compile('of\s*(\d+)\s*result')
 
-def get_queue_count(url):
+def get_queue_count(search_status, release, search_pocket):
     '''Return number of results of given queue page URL'''
 
-    contents = urlopen(url + '&batch1').read()
-    m = queue_page_count_pattern.search(contents)
-    if m:
-        return str(m.group(1))
-    else:
-        return '0'
+    count = len(release.getPackageUploads(status=search_status,
+        pocket=search_pocket))
+    return count
+
 
 def get_srus():
     '''Generate SRU map.