[SRU] glance sync: need keystone v3 auth support

i've linked a work-in-progress branch.
https://code.launchpad.net/~smoser/simplestreams/trunk.openstack-v3-auth
there i had keystone auth working.

This bug was fixed in the package simplestreams - 0.1.0~bzr450-0ubuntu1

---------------
simplestreams (0.1.0~bzr450-0ubuntu1) artful; urgency=medium

  * New upstream snapshot.
    - Keystone v3 Support [David Ames] (LP: #1686437)
    - flake8/pycodestyle updates.
    - tests: change to having http server select its own port
    - Support filters that contain a '-' in the tag name
    - Improvements for running flake8 in different Ubuntu release
      environments.
    - add running of tox.
    - json2streams: Accept items with no size.
    - tools changes (not related to package functionality)
      - tools/ubuntu_versions.py: Exclude old versions by version not name
      - Update default LTS alias to point to Xenial (LP: #1606606)
      - Create chksum for LXD metadata+root for squashfs (LP: #1577922)

 -- Scott Moser <email address hidden> Thu, 14 Sep 2017 10:38:07 -0400

Scott, will 0.1.0~bzr450-0ubuntu1 be backported to xenial?

Felipe,

I just talked to smoser this morning and he agreed on backporting this into supported stable release.
We already set the release nominations accordingly.

There is a few details that smoser and thedac need to review together and they will keep us posted.

- Eric

Noting here the released version on xenial does not currently support Keystone v3 and blocks Bug #1611987.

For the record, we have been running a bzr branch @455 on serverstack (a Keystone v3 cloud) for months now. So the code in simplestreams works, it just needs to get to xenial.

I believe there is a ksv2 regression in the dev ppa of simplestreams, but I can also confirm that the ksv3 scenario is resolved with the dev ppa.

#### PASS:
ksv2 + 0.1.0~bzr426-0ubuntu1.2 (distro) + xenial-pike

ksv3 + 0.1.0~bzr459~trunk-0ubuntu1~ubuntu16.04.1 (dev ppa) + xenial-queens

#### FAIL:
ksv2 + 0.1.0~bzr459~trunk-0ubuntu1~ubuntu16.04.1 (dev ppa) + xenial-pike

ksv3 + 0.1.0~bzr426-0ubuntu1.2 (distro) + xenial-queens

https://paste.ubuntu.com/p/HxttSNpfc8/

Also, exercising the SRU ppa, I see xenial-pike fail similarly with ksv2:

https://launchpad.net/~smoser/+archive/ubuntu/sstream-ks3

https://paste.ubuntu.com/p/kVnFqsj8Gj/

Hello Scott, or anyone else affected,

Accepted simplestreams into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/simplestreams/0.1.0~bzr426-0ubuntu1.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hi,
I've marked this verification-failed as we had found some issues and forgotten that there was an upload in the xenial queue.

This bug is believed to be fixed in simplestreams in version 0.1.0. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Hi,
The current state of this bug:
a.) 0.1.0~bzr426-0ubuntu1.3 was uploaded. It contained a fix for
bug 1686437. That change contained a regression (bug 1728982, bug 1719879),
so the bug was marked xenial-verification-failed.

b.) We have subsequently done work to fix that regression
A merge proposal is made for xenial at [1]. We have a PPA at [2] that
contains a fix for the regression caused by 1.3 and others.

Someone can/should test the PPA [1] and then upload a
0.1.0~bzr426-0ubuntu1.4 to xenial.

--
[1] https://code.launchpad.net/~smoser/ubuntu/+source/simplestreams/+git/simplestreams/+merge/341214
[2] https://launchpad.net/~smoser/+archive/ubuntu/sstream-ks3/+packages

Hi,

I've had a chance to test the fix for this. I was able to confirm that the 1.2 version of the package had an issue with the v3 keystone API. I then installed the 1.4 version of the package and it was able to successfully sync the images.

glance-simplestreams-sync log file included

I forgot to include my test cloud info:
Openstack Queens running on Xenial

| 458939d4dc404a47a8a496b7d513a5ab | RegionOne | keystone | identity | True | internal | http://10.5.0.18:5000/v3 |
| 930b5cf107724f87845a645fc625fc07 | RegionOne | keystone | identity | True | admin | http://10.5.0.18:35357/v3 |
| b7fec4dbcddf42e8867d2750f95378fc | RegionOne | keystone | identity | True | public | http://10.5.0.18:5000/v3 |

juju status output is attached

I've tested this as well on xenial. This is what I tested:

root@juju-97b42b-0-lxd-4:~# apt-cache policy simplestreams
simplestreams:
  Installed: (none)
  Candidate: 0.1.0-482-g409fdc1-0ubuntu1~ubuntu16.04.1
  Version table:
     0.1.0-482-g409fdc1-0ubuntu1~ubuntu16.04.1 500
        500 http://ppa.launchpad.net/simplestreams-dev/trunk/ubuntu xenial/main amd64 Packages
     0.1.0~bzr426-0ubuntu1.2 500
        500 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages
     0.1.0~bzr426-0ubuntu1 500
        500 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages

The 0.1.0-482-g409fdc1-0ubuntu1~ubuntu16.04.1 package was successfully able to sync images with glance and swift.

Also, it seems that the xenial-proposed (1.3) package has been deleted so we need to get the PPA version resubmitted as an sru and test that.

The adjoining bug 1611987 to this one says there is a fix landed in Bionic but it is not clear from the changelog whether that actually fixes v3 support so im going to leave this targeted at X/B/C/D under the assumption that none is fixed yet.

For the keystone v3 fixes revno 454 is the minimum we need SRU'd back to xenial.
Bionic 0.1.0~bzr460-0ubuntu1 has these changes.

These two merges are the pertinent changes:

https://code.launchpad.net/~thedac/simplestreams/keystone-v3-support/+merge/325781
https://code.launchpad.net/~thedac/simplestreams/lp1719879/+merge/333011

A package 0.1.0~bzr454-0ubuntu1 existed in xenial-proposed at one time. Still trying to figure out what happened to that package.

It would seem the SRU process needs to occur on:
https://bugs.launchpad.net/simplestreams/+bug/1719879

Version 0.1.0~bzr460-0ubuntu1 from bionic on fixes the problem, so I'm marking B/C/D as "Fix Released".

SUMMARY FOR SIMPLESTREAMS SRU TO XENIAL

After analyzing the following bugs:

• LP: #1611987 - simplestreams - [SRU] glance-simplestreams-sync charm doesn't support keystone v3
• LP: #1686437 - simplestreams - can't sync images for keystone v3
• LP: #1719879 - simplestreams - [SRU] swift client needs to use v1 auth prior to ocata
• LP: #1728982 - simplestreams - [SRU] openstack mirror with keystone v3 always imports new images

For the keystone v3 fixes revno 454 is the minimum we need SRU'd back to xenial. Bionic 0.1.0~bzr460-0ubuntu1 has these changes. These two merges are the pertinent changes:

 1. Keystone v3 Support - https://is.gd/wq7r6g
 2. Fix KSv3 Bugs - https://is.gd/OOEo3G

0.1.0~bzr426-0ubuntu1.3 was uploaded. It contained a fix for LP: #1686437 (can't sync images for keystone v3). That change contained a regression (LP: #1728982 - openstack mirror with keystone v3 always imports new images - and LP: #1719879 - swift client needs to use v1 auth prior to ocata) and was marked as verification needed.

Work was done to fix that regression. A merge proposal is made for Xenial at https://is.gd/7ixQbO. We have a PPA at https://is.gd/Boda8J that contains a fix for the regression caused by 0.1.0~bzr426-0ubuntu1.3 and others.

Feedback from that PPA was asked and was given by Ed, Chris, Felipe and Billy. Billy found an issue about squashfs and that was fixed into 0.1.0~bzr426-0ubuntu1.4~ppa0, also uploaded to PPA at https://is.gd/Boda8J.

SRU template is needed in all referenced bugs:

 • 428-do-not-require-that-hypervisor_config-be-present.patch (LP: #1578622)
 • 433-glance-ignore-inactive-images.patch (LP: #1583276)
 • 436-glance-fix-race-conditions.patch (LP: #1584938)
 • 450-453-454-keystone-v3-support.patch (LP: #1686437, #1728982, #1719879)
 • 455-nova-lxd-support-squashfs-images.patch (LP: #1686086)

And version 0.1.0~bzr426-0ubuntu1.4 is good for a SRU and already tested by multiple people.

Please confirm this summary in order for me to prepare the SRU template and merge for Xenial SRU proposal.

Rafael's comment matches my memory, and also my comment in the merge proposal at https://code.launchpad.net/~smoser/ubuntu/+source/simplestreams/+git/simplestreams/+merge/341214

Thank you, Scott, for confirming my summary. I also got IRC confirmation from Billy and Ed.

With that, and because I had to "re-wrap" understanding, I am doing a similar merge request, fixing only commit comments, but keeping changelogs and credits, and asking for Ed and Felipe's review, and then Ubuntu Server Team review (for the SRU to happen).

I'll do a -1 in the existing Xenial SRU merge request, so I can keep a centralized place for the reviews, tests and approvals for everything.

An upload of simplestreams to xenial-proposed has been rejected from the upload queue for the following reason: "See https://bugs.launchpad.net/charms/+source/glance-simplestreams-sync/+bug/1611987/comments/21".

TL;DR version:

We are going to fix Bionic simplestreams package and provide this Bionic simplestreams package in Ubuntu Cloud Archive for Xenial. With that, we are fixing Xenial simplestreams behavior (to work with keystone v3) only if end-user enables Ubuntu Cloud Archive.

Conversation about this topic:

https://code.launchpad.net/~rafaeldtinoco/ubuntu/+source/simplestreams/+git/simplestreams/+merge/373030/comments/980111