Merge lp:~brian-murray/ubuntu-archive-tools/workaround-503 into lp:ubuntu-archive-tools

[Brian Murray]

The phased-updater is running into 503 Errors, requests timing out, when trying to get binaries published from a source publication. My testing reveals that none of the source publications which create this issue are ones which are currently phasing. While just skipping the package is rather terrible, I think its better than having the phased-updater only successfully run for Trusty and Xenial (it OOPSes somewhere in the list of Xenial publications) and having the phased updates report be out of date.

I've reported Launchpad bug 1695113 about the specific OOPS and bug 1694816 regarding having a faster way to query for packages with a phased_update_percentage.

[Brian Murray (brian-murray)]

Oh and for what it is worth these are the packages with which I encountered an error.

 $ ./phased-updater -n
Skipping 503 Error for xorg-lts-transitional
Skipping 503 Error for gcc-5-cross-ports

=== modified file 'phased-updater'
--- phased-updater	2016-12-19 13:19:31 +0000
+++ phased-updater	2017-06-02 22:02:33 +0000
@@ -30,6 +30,7 @@
 import codecs
 import csv
 import datetime
+import lazr
 import logging
 import os
 import simplejson as json
@@ -572,24 +573,30 @@
     releases.reverse()
     issues = {}
     for release in releases:
+        # We can't use release.datereleased because some SRUs are 0 day
+        cdate = release.date_created
         rname = release.name
         rvers = release.version
         issues[rname] = OrderedDict()
         # XXX - starting with raring
         if rname in ['lucid', 'precise', 'vivid', 'wily']:
             continue
-        # XXX - only look at updates since 21/06/2013
-        start_date = datetime.datetime(2013, 6, 21, 0, 0, 0, 0)
         pub_sources = archive.getPublishedSources(
-            created_since_date=start_date,
+            created_since_date=cdate,
             order_by_date=True,
             pocket='Updates', status='Published', distro_series=release)
         for pub_source in pub_sources:
             src_pkg = pub_source.source_package_name
             version = pub_source.source_package_version
             pbs = None
-            pbs = [pb for pb in pub_source.getPublishedBinaries()
-                   if pb.phased_update_percentage is not None]
+            try:
+                pbs = [pb for pb in pub_source.getPublishedBinaries()
+                       if pb.phased_update_percentage is not None]
+            # workaround for LP: #1695113
+            except lazr.restfulclient.errors.ServerError as e:
+                if 'HTTP Error 503' in str(e):
+                    logging.info('Skipping 503 Error for %s' % src_pkg)
+                    pass
             if not pbs:
                 continue
             if pbs: