Ubuntu
pymacaroons package

dependency issues in the pymacaroons stack

Bug #1822780 reported by Christian Ehrhardt 
16
This bug affects 1 person
Affects Status Importance Assigned to Milestone
pymacaroons (Ubuntu)
Fix Released
Undecided
Unassigned
Trusty
Fix Released
Undecided
Andreas Hasenack
Xenial
Won't Fix
Undecided
Unassigned

Bug Description

[Impact]

 * The pymacaroons stack has dependency issues identified when further
   testing the backport to trusty (currently in proposed)

 * Ensure that the pymacaroons stack works well in Trusty (for Ubuntu
   Advantage client) but also fix the issues that matter for Xenial in
   Xenial as well.

[Test Case]

* install python3-pymacaroons. This will pull in python3-libnacl 1.4.5 as well (expected)
* Without the latest fix from proposed, this command will backtrace as shown:

$ python3 -c "__requires__ = 'pymacaroons';from pkg_resources import load_entry_point"
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 444, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 725, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 632, in resolve
    raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (libnacl 1.4.5 (/usr/lib/python3/dist-packages), Requirement.parse('libnacl>=1.3.6,<1.4'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2749, in <module>
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 446, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 459, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 628, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: libnacl>=1.3.6,<1.4

* the python3-six dependency check isn't reached, because the check stops at the first unsatisfied dependency and that happened to be libnacl. But when testing the fixed packages, there must be no backtrace like above.

[Fix]

 * Changes
   - Trusty: drop python2 binaries in trusty (was not released there yet)
     Xenial has those already published, we are not gonna take them away.
     see [1]
   - Trusty: adapt python-six dependency to match what is really needed
     and available in Trusty. Xenial has a version higher than that so
     don't bother modifying it there.
     see [2]
   - Trusty and Xenial: adapt python-libnacl dependency by backporting
     upstream fix on a too struct version requirement.
     see [3]

[1]: https://git.launchpad.net/~paelzer/ubuntu/+source/pymacaroons/commit/?id=6746ad32b4d785919e849a4bef5685bf0bec6be2
[2]: https://git.launchpad.net/~paelzer/ubuntu/+source/pymacaroons/commit/?id=f94ac40aeb354c008c4764853b877991003ae429
[3]: https://git.launchpad.net/~paelzer/ubuntu/+source/pymacaroons/commit/?id=1987dffcffae29c63ea71635db1cfe19badc145f

[Regression Potential]

 * Lets split this section for Trusty and Xenial.
   a) Xenial only gets slightly relaxed version dependencies, those should
      allow pymacaroons to run, but actually never cause any additional
      issues.
   b) Trusty this isn't released in trusty at all yet (still in proposed
      intentionally). Therefore this is one of the few cases I'd call
      regression risk next to zero. If anything then issues out of the py2
      removal not working as expected, but it seems to be ok in build and
      tests.

[Other Info]

 * This is related to the SRU in bug 1817665
 * Furthermore it is related to the MIR extension to Trusty in
   bug 1746772 bug 1817327 and bug 1621386

---

On the extended testing for the SRU in bug 1817665 we found that there are a few dependency issues that need to be fixed.

Affects X&T:
python-libnacl - depends on <1.4 but xenial (and trusty soon) have 1.4.5
Fix by importing https://github.com/ecordell/pymacaroons/commit/3924d5b56c42234d0bff3820bc4cb6c4d3f74d8d

Affects T:
python-six - depends on >=1.8.0 - but trusty is on 1.5.2 still
We evaluated if we need to backport six as well (a lot or reverse deps).
We found that the usage of pymacaroons is actually ok with 1.5.2
Upstream regularly bumps the dependency level as well as some changes that got added but in the meantime removed (e.g. the serialization).
Eventually only two use cases for python3-six are left
- pymacaroons/caveat_delegates/encrypted_first_party.py:4:from six import iteritems
- pymacaroons/utils.py:5:from six import text_type, binary_type

None of these are changed in six between 1.5.2 and 1.8 checked by Odd_Bloke, ahasenack and me.

Affects T:
python2 - we don't need/want python2 support in this, so since we don't usually exercise and have no interest in supporting the python2 paths lets drop them from pymacaroons in trusty (where they are not published yet).

See original description

Related branches

~paelzer/ubuntu/+source/pymacaroons:MIR-ua-xenial-build-dependencyfixup
Canonical Server: Pending requested
Canonical Server packageset reviewers: Pending requested
git-ubuntu developers: Pending requested
Diff: 1142 lines (+117/-579)
22 files modified
PKG-INFO (+1/-1)
debian/.git-dpm (+7/-7)
debian/changelog (+8/-15)
debian/patches/fix-trusty-dependencies-libnacl.patch (+29/-0)
debian/patches/series (+1/-0)
docs/conf.py (+2/-2)
pymacaroons.egg-info/PKG-INFO (+1/-1)
pymacaroons.egg-info/requires.txt (+1/-1)
pymacaroons/__init__.py (+1/-5)
pymacaroons/caveat.py (+3/-19)
pymacaroons/caveat_delegates/base_third_party.py (+1/-2)
pymacaroons/caveat_delegates/encrypted_first_party.py (+1/-1)
pymacaroons/caveat_delegates/first_party.py (+3/-7)
pymacaroons/caveat_delegates/third_party.py (+8/-10)
pymacaroons/field_encryptors/base_field_encryptor.py (+2/-2)
pymacaroons/macaroon.py (+8/-43)
pymacaroons/serializers/binary_serializer.py (+10/-251)
pymacaroons/serializers/json_serializer.py (+15/-157)
pymacaroons/utils.py (+2/-39)
pymacaroons/verifier.py (+10/-13)
setup.cfg (+1/-1)
setup.py (+2/-2)
~paelzer/ubuntu/+source/pymacaroons:MIR-ua-trusty-build-dependencyfixup
Andreas Hasenack (community): Needs Fixing
Dan Watkins (community): Approve
Diff: 153 lines (+74/-23)
6 files modified
debian/changelog (+18/-0)
debian/control (+0/-22)
debian/patches/fix-trusty-dependencies-libnacl.patch (+29/-0)
debian/patches/fix-trusty-dependencies-six.patch (+24/-0)
debian/patches/series (+2/-0)
debian/rules (+1/-1)
Christian Ehrhardt  (paelzer)
description: updated
Christian Ehrhardt  (paelzer)
Changed in pymacaroons (Ubuntu Trusty):
status: New → Triaged
Changed in pymacaroons (Ubuntu Xenial):
status: New → Triaged
Changed in pymacaroons (Ubuntu):
status: New → Fix Released
description: updated
Dan Watkins (oddbloke)
description: updated
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Linked two MPs for later discussion.
PPA is at https://launchpad.net/~paelzer/+archive/ubuntu/bug-1822780-pymacaroons-dependencies/+packages

I expect the python2 removal (which was rather rude) to fail at first, so force pushes to those MPs might come (that is why they are still marked WIP).

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

@Andreas/Daniel - could you add the test steps to the SRU template, do the pre-verification on the PPA and then upload (remember the -v on the build to cover both last versions) - and/or modify as needed.

description: updated
Changed in pymacaroons (Ubuntu Trusty):
assignee: nobody → Andreas Hasenack (ahasenack)
Changed in pymacaroons (Ubuntu Xenial):
assignee: nobody → Dan Watkins (daniel-thewatkins)
Andreas Hasenack (ahasenack)
Changed in pymacaroons (Ubuntu Xenial):
assignee: Dan Watkins (daniel-thewatkins) → Andreas Hasenack (ahasenack)
Andreas Hasenack (ahasenack)
description: updated
Andreas Hasenack (ahasenack)
description: updated
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

I don't think xenial is affected, probably due to a bug. The requires.txt file in xenial's python3-pymacaroons package is empty:

root@xenial:~# apt-cache policy python3-pymacaroons
python3-pymacaroons:
  Installed: 0.9.2-0ubuntu1
  Candidate: 0.9.2-0ubuntu1
  Version table:
 *** 0.9.2-0ubuntu1 500
        500 http://br.archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages
        100 /var/lib/dpkg/status
root@xenial:~# dpkg -L python3-pymacaroons|grep requires.txt
/usr/lib/python3/dist-packages/pymacaroons-0.9.2.egg-info/requires.txt
root@xenial:~# cat /usr/lib/python3/dist-packages/pymacaroons-0.9.2.egg-info/requires.txt
root@xenial:~# l /usr/lib/python3/dist-packages/pymacaroons-0.9.2.egg-info/requires.txt
-rw-r--r-- 1 root root 0 Jun 3 2016 /usr/lib/python3/dist-packages/pymacaroons-0.9.2.egg-info/requires.txt
root@xenial:~#

root@xenial:~# python3 -c "__requires__ = 'pymacaroons';from pkg_resources import load_entry_point"
root@xenial:~#

And ua works fine:
root@xenial:~# ua
usage: ubuntu-advantage [command] [flags]
Try 'ubuntu-advantage --help' for more information.
root@xenial:~# apt-cache policy ubuntu-advantage-tools
ubuntu-advantage-tools:
  Installed: 19.1-0ubuntu1~361~git.6bb6fcc~ubuntu16.04.1
  Candidate: 19.1-0ubuntu1~361~git.6bb6fcc~ubuntu16.04.1
  Version table:
 *** 19.1-0ubuntu1~361~git.6bb6fcc~ubuntu16.04.1 500
        500 http://ppa.launchpad.net/canonical-server/ua-client-daily/ubuntu xenial/main amd64 Packages
        100 /var/lib/dpkg/status
     10ubuntu0.16.04.1 500
        500 http://br.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages

Christian Ehrhardt  (paelzer)
Changed in pymacaroons (Ubuntu Xenial):
status: Triaged → Invalid
assignee: Andreas Hasenack (ahasenack) → nobody
Changed in pymacaroons (Ubuntu Trusty):
status: Triaged → In Progress
Revision history for this message
Robie Basak (racb) wrote : Please test proposed package

Hello Christian, or anyone else affected,

Accepted pymacaroons into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/pymacaroons/0.9.2-0ubuntu1~ubuntu14.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in pymacaroons (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-trusty
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I have set bug 1817665 back to their former state - which is incomplete to hold it back until UA is ready and tested.

The testing on this bug here will be done by Andreas I think - he is already assigned so no bug update required right now until tests complete.

Mathew Hodson (mhodson)
Changed in pymacaroons (Ubuntu Xenial):
status: Invalid → Won't Fix
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

trusty test case

python3-pymacaroons is not available in trusty other than in the proposed pocket, so installing that one:
root@trusty-sru-1822780:~# apt-cache policy python3-pymacaroons
python3-pymacaroons:
  Installed: (none)
  Candidate: 0.9.2-0ubuntu1~ubuntu14.04.2
  Version table:
     0.9.2-0ubuntu1~ubuntu14.04.2 0
        500 http://br.archive.ubuntu.com/ubuntu/ trusty-proposed/main amd64 Packages

Installation pulled in:The following NEW packages will be installed:
  libsodium18 python3-libnacl python3-pymacaroons python3-six

All from main, so the MIR was processed.

Trying test program with python3-pymacaroons 0.9.2-0ubuntu1~ubuntu14.04.2 from proposed:
# python3 -c "__requires__ = 'pymacaroons';from pkg_resources import load_entry_point"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named 'pkg_resources'

Need to install a dependency that was missed when the test program was added to this bug:
# apt install python3-pkg-resources
...
Setting up python3-pkg-resources (3.3-1ubuntu2) ...

Now it works:
root@trusty-sru-1822780:~# python3 -c "__requires__ = 'pymacaroons';from pkg_resources import load_entry_point"
root@trusty-sru-1822780:~# echo $?
0

Trusty verification succeeded.

tags: added: verification-done-trusty
removed: verification-needed-trusty
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package pymacaroons - 0.9.2-0ubuntu1~ubuntu14.04.2

---------------
pymacaroons (0.9.2-0ubuntu1~ubuntu14.04.2) trusty; urgency=medium

  * fix dependencies when SRUing pymacaroons stack to trusty.
    There are a bunch of dependencies that need to be cleaned up to
    work in trusty without requiring more packages backported.
    (LP: #1822780)
    - drop python2 binary packages
      + d/control: drop python2 related build depends
      + d/control: drop python2 related packages
      + d/control: drop python2 related X-Python flag
      + d/rules: drop python2 from the call to dh
    - d/p/fix-trusty-dependencies-libnacl.patch: fix dependency issue with
      python-libnacl
    - d/p/fix-trusty-dependencies-six.patch: fix dependency issue with
      python-six

pymacaroons (0.9.2-0ubuntu1~ubuntu14.04.1) trusty; urgency=medium

  * Backport to Trusty as dependency of coming UA tools update (LP: #1817665)

 -- Christian Ehrhardt <email address hidden> Tue, 02 Apr 2019 15:08:06 +0200

Changed in pymacaroons (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Steve Langasek (vorlon) wrote : Update Released

The verification of the Stable Release Update for pymacaroons has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.