Merge lp:~chad.smith/charms/trusty/glance-simplestreams-sync/handle-connectionerror into lp:~landscape/charms/trusty/glance-simplestreams-sync/landscape

[Данило Шеган]

NOTE: Resubmit of https://code.launchpad.net/~chad.smith/charms/trusty/glance-simplestreams-sync/handle-connectionerror/+merge/294306 to merge into Landscape branch of the charm.

Unhandled exceptions cause the charm to remove the simplestreams cron which retries the image uploads upon a known set of errors. During initial connection to the glance API, the charm received a ConnectionError as raised by glanceclient. This broken pipe was a result of glance api service restart during the install process.

  File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 235, in _http_request
    raise exc.CommunicationError(message=message)
CommunicationError: Error communicating with http://10.1.7.237:9292 [Errno 32] Broken pipe

Manually re-running the script called by the cron
/usr/share/glance-simplestreams-sync/glance-simplestreams-sync.sh resulted in a success a few minutes after the cron was removed in error.

This merge proposal handles glanceclient.exc.CommunicationError and leaves the cron active so that a retry attempt is performed.

Testing instructions:

This can e tested using landscape with the following patch=== modified file 'canonical/landscape/model/openstack/charms/trusty-stable.json'
--- canonical/landscape/model/openstack/charms/trusty-stable.json revid:<email address hidden>
+++ canonical/landscape/model/openstack/charms/trusty-stable.json 2016-05-10 22:24:53 +0000
@@ -24,7 +24,7 @@
     "url": "cs:trusty/glance-249"
   },
   "glance-simplestreams-sync": {
- "url": "cs:~landscape-charmers/trusty/glance-simplestreams-sync-6"
+ "url": "cs:~chad.smith/trusty/glance-simplestreams-sync-0"
   },
   "hacluster": {
     "url": "cs:trusty/hacluster-28"

Deploy to your favorite substrate.

[Данило Шеган (danilo)]

Looks good, thanks for the fix: it's been tested on a deployed cloud. (I've got a task on the board to invert the logic in the script to only remove the "cronpoll" on actual success, but didn't get to it for 16.05: it would avoid us having to deal with every potential exception one-by-one)

Once merged, a simple "charm push PATH-TO-CHECKOUT-OF-landscape 'cs:~landscape-charmers/trusty/glance-simplestreams-sync'" followed by charm publish on the likely resulting -7 should be sufficient (permissions are already set up on it).

review: Approve

[Bogdana Vereha (bogdana)]

+1

review: Approve

=== modified file 'scripts/glance-simplestreams-sync.py'
--- scripts/glance-simplestreams-sync.py	2016-04-27 13:04:08 +0000
+++ scripts/glance-simplestreams-sync.py	2016-05-11 07:30:43 +0000
@@ -455,7 +455,7 @@
             should_delete_cron_poll = False
             log.info("Glance endpoint not found, will continue polling.")
 
-    except glanceclient.exc.ClientException as e:
+    except (glanceclient.exc.ClientException, glanceclient.exc.CommunicationError) as e:
         log.exception("Glance Client exception during do_sync:\n%s\n"
                       "Will continue polling." % e.message)
         should_delete_cron_poll = False