Python plugin: paths for from the snapcraft snap in sys.path break dependencies

Bug #1860884 reported by Alberto Donato
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Snapcraft
Fix Released
High
Sergio Schvezov

Bug Description

Tested with Snapcraft 3.9.8 (3970) from stable channel

In a python part for a snap, if python3 is called inside a build step, the interpreter installed from the part is used. However, this has some paths from the python from the snapcraft snap in sys.path.

He're a minimal reproducer:

name: python-path-test
version: "0.0.1"
summary: python-path-test
description: python-path-test
grade: stable
confinement: strict
base: core18

apps:
  python-path-test:
    command: python-path-test

parts:
  python-path-test:
    plugin: python
    source: .
    source-type: local
    requirements:
      - requirements.txt
    override-build: |
      set -e
      snapcraftctl build
      echo -------------
      which python3
      echo -------------
      python3 -c "import sys; import pprint; pprint.pprint(sys.path)"
      echo -------------
      python-path-test

The "python-path-test" script does exactly the same as the oneliner above.

The output during build is the following:

-------------
/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/bin/python3
-------------
['',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python36.zip',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3.6',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3.6/lib-dynload',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3/dist-packages',
 '/snap/snapcraft/3970/lib/python3.6/site-packages',
 '/home/ack/Desktop/python-path-test/stage/lib/python3.6/site-packages',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/lib/python3.6/site-packages']
-------------
['/home/ack/Desktop/python-path-test/parts/python-path-test/install/bin',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python36.zip',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3.6',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3.6/lib-dynload',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/usr/lib/python3/dist-packages',
 '/snap/snapcraft/3970/lib/python3.6/site-packages',
 '/home/ack/Desktop/python-path-test/stage/lib/python3.6/site-packages',
 '/home/ack/Desktop/python-path-test/parts/python-path-test/install/lib/python3.6/site-packages']

In both cases '/snap/snapcraft/3970/lib/python3.6/site-packages' ia included. It seems like only paths under $SNAPCRAFT_INSTALL_PART should be there.

This can cause issues when the application has dependency requirements that conflict with the ones from snapcraft itself, since in that case pkg_resources breaks.

Alberto Donato (ack)
summary: - Python plugin: paths for from the snapcraft snap in sys.pat break
+ Python plugin: paths for from the snapcraft snap in sys.path break
dependencies
Revision history for this message
Chris Patterson (cjp256) wrote :

This is an effect of running `snapcraftctl build`. It modifies sitecustomize.py to include snap dir, stage dir, and part install site-packages.

I believe stage dir is required (if you use stage-packages to pull in dependencies).

sergiusens will be able to weigh in with a more expert opinion. In the meantime, you could update sitecustomize.py to work as you want it after `snapcraftctl build`.

e.g.
    override-build: |
      set -e
      snapcraftctl build
      sed -i "s|snap_dir, ||" $SNAPCRAFT_PART_INSTALL/usr/lib/python3.6/sitecustomize.py
      python3 -c "import sys; import pprint; pprint.pprint(sys.path)"
      python-path-test

Revision history for this message
Alberto Donato (ack) wrote :

Thanks for the explaination

Yeah I think stage dir should be fine. The issue I'm seeing (that I derived this example from) is with having the snap dir there.

Specifically, an application that uses pymacaroons needs a version that conflicts with snapcraft's own requirements, making pkg_resources choke during the build stage.

Changed in snapcraft:
status: New → Triaged
importance: Undecided → High
assignee: nobody → Sergio Schvezov (sergiusens)
Changed in snapcraft:
status: Triaged → In Progress
Revision history for this message
Sergio Schvezov (sergiusens) wrote :
Changed in snapcraft:
status: In Progress → Fix Committed
Revision history for this message
Sergio Schvezov (sergiusens) wrote :

Part of 3.10

Changed in snapcraft:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.