lp:~philroche/simplestreams/1606606

Branch merges

Propose for merging

No branches dependent on this one.

Merged into lp:simplestreams at revision 439

Robert C Jennings: Pending requested 2016-07-26

Diff: 12 lines (+1/-1)

1 file modified

tools/ubuntu_versions.py (+1/-1)

api

Related source package recipes

No recipes using this branch. (?)

Create packaging recipe

Related snap packages

Related bugs

Bug #1606606: Update default LTS alias to point to Xenial

Medium

Fix Released

Link a bug report

Related blueprints

439. By Philip Roche on 2016-07-26

Update simplestream's default LTS alias to point to Xenial (lp:1606606 )

Following release of 16.04.1 update simplestream's default LTS alias to point to Xenial instead of Trusty

438. By Scott Moser on 2016-07-22

json2streams: Accept items with no size.

dict_to_item automatically converts size from a string to an int. This
has the side effect of making size mandatory. Size should not be mandatory
for image-ids.

437. By Robert C Jennings on 2016-06-22

Create chksum for LXD metadata+root for squashfs (LP: #1577922)

This adds a 'combined_sqashfs_sha256' element to items that have
both a LXD metadata tar and a squashfs root. Additionally, we
have 'combined_rootxz_sha256' which wil have the same value as
'combined_sha256' and allow for sunsetting that ambiguous name.

436. By Scott Moser on 2016-06-14

GlanceMirror: fix a couple of race-related problems

First, what can happen is that .sync() is interrupted for external reasons
(service restarts, network issues,...) after first image has been uploaded
to Glance when syncing multiple images. On re-run, the uploaded image is
considered "synced" already, but since simplestreams index is written out
only at the end of the sync, all the metadata is lost. We solve this by
adding "simplestreams_metadata" as one of the properties of image in
Glance where we put a JSON representation of all the metadata fields from
the original simplestreams index entry. We then load them in
load_products() if they are defined. This results in a slightly different
auto.sync.json entry: "endpoint" and "region" are on the product entry,
instead of on the disk1.img entry.

Secondly, if sync is indeed interrupted after one or several images are
completely uploaded (but not all of them), there's no index file at all,
so any attempt to make use of them with juju would fail, even though it
should be fully functional. We solve this by calling "insert_products()"
at the end of the insert_item(). This means that the index is regenerated
after every single image is uploaded.

Along the way, I slightly improve insert_item() tests from the pre-req
branch to add a test for minimal data required, rename the existing one
using real world data to test_insert_item_full, and re-use that data for a
test for a newly added call to insert_products().

435. By Scott Moser on 2016-06-14

GlanceMirror: refactor insert_item for easier testing

This change refactors GlanceMirror.insert_item() to allow for easier and
more contained testing. I needed to do this to understand everything that
was going on inside insert_item and other bits of code. There are now four
distinct things happening in it:

 1. Download image to a local file from a ContentSource
 2. Construct extra properties to store in Glance along with image
 3. Prepare arguments for GlanceClient.images.create() call
 4. Adapt source simplestreams entry for an image for use in the target
    simplestreams index

It should be fully backwards compatible, and test coverage for all the
individual steps should be much better (I admit to it not being perfect,
but it's a step in the right direction, imho at least).

434. By Scott Moser on 2016-05-20

tools/toolutil.py: early yakkety images did not have ova files.

this code will soon be obsolete but for now just ignore the ova files
on older builds of yakkety.

433. By Scott Moser on 2016-05-20

glance: ignore inactive images

If connection to glance is broken during GlanceMirror.sync(), and image
would be left over in "status": "saving".

Glance itself tries to protect against that, however when it is restarted
(eg. "service glance-api restart"), it does not clean the image up.

This change makes load_product() ignore any images that do not have
"status": "active". Images can only have deleted, active or saving
status.

Note that this isn't perfect. The image in "saving" state is never cleaned
up, but at least the resulting cloud works. If we want to clean up broken
images, it'd be hard to do on the simplestreams side (glance is better
placed to do that: it knows if anything is going on with any of them, eg.
is it being currently updated or not).

432. By Scott Moser on 2016-05-19

tools/make-test-data: sort keys when writing json, sort product names

This uses the simplestreams/util 'dump_data' to dump json data in
each place that was writing json in tools/make-test-data.

Also, it sorts the 'products' data in the index stream.

431. By Scott Moser on 2016-05-17

support stream output for yakkety builds

For yakkety builds:
 - yakkety build output renames the -disk1.img file to .img
   but stream data should maintain -disk1.img as the ftype and item key.
 - -root.tar.gz and -root.tar.xz files are not produced
 - squashfs images are available (and also were available partially through
   xenial)

As a cleanup, we a helper to ubuntu_versions to compare versions more
directly. String compare was less useful in some cases and also would end
up breaking after 'z' because whatever release comes after z would string
sort to < z.

This also adds a simple tool for comparing two trees output of 'make exdata'.
    ./tools/compare-test-data ../trunk.dist ./
will output a 'diff' from old to new list of files indexed.

430. By Robie Basak on 2016-05-11

Optimise read_signed

Use join instead of iteratively "appending" to a string. In my use case,
this reduces signature check time from over 400 seconds to almost a
tenth of a second.