astral-uv:zb/cache-python-versions

Last commit made on 2024-01-26
Get this branch:
git clone -b zb/cache-python-versions https://git.launchpad.net/astral-uv

Branch merges

Branch information

Name:
zb/cache-python-versions
Repository:
lp:astral-uv

Recent commits

65f1ce5... by Zanie <email address hidden>

Empty commit to test cache

9f44ffd... by Zanie <email address hidden>

Cache Python binaries in CI

f21c033... by Zanie <email address hidden>

Do not re-download already installed Python versions

5cc4e5d... by Zanie Blue <email address hidden>

Add `pip compile` test where specific Python versions are available on the system (#1111)

Extends https://github.com/astral-sh/puffin/pull/1106 with the scenario
from https://github.com/zanieb/packse/pull/95 which tests that `pip
compile` will use the matching system Python version for builds when
available

91f421c... by Zanie Blue <email address hidden>

Do not allow `pip compile` scenario tests to discover other Python versions (#1106)

In https://github.com/astral-sh/puffin/pull/1040 we broke the pip
compile scenarios designed to test failure when a required Python
version is not available — resolution succeeded because all of the
Python versions were available in CI. Following #1105 we have the
ability to isolate tests from Python versions available in the system.
Here, we limit the scenarios to only the Python version in the current
environment, restoring our ability to test the error messages.

With https://github.com/zanieb/packse/pull/95, we will be able to
specify scenarios with access to additional system Python versions. This
will allow us to include test coverage where resolution can succeed by
using a version available elsewhere on the system. See #1111 for this
follow-up.

21577ad... by Zanie Blue <email address hidden>

Add bootstrapping and isolation of development Python versions (#1105)

Replaces https://github.com/astral-sh/puffin/pull/1068 and #1070 which
were more complicated than I wanted.

- Introduces a `.python-versions` file which defines the Python versions
needed for development
- Adds a Bash script at `scripts/bootstrap/install` which installs the
required Python versions from `python-build-standalone` to `./bin`
- Checks in a `versions.json` file with metadata about available
versions on each platform and a `fetch-version` Python script derived
from `rye` for updating the versions
- Updates CI to use these Python builds instead of the `setup-python`
action
- Updates to the latest packse scenarios which require Python 3.8+
instead of 3.7+ since we cannot use 3.7 anymore and includes new test
coverage of patch Python version requests
- Adds a `PUFFIN_PYTHON_PATH` variable to prevent lookup of system
Python versions for isolation during development

Tested on Linux (via CI) and macOS (locally) — presumably it will be a
bit more complicated to do proper Windows support.

cc0e211... by Charlie Marsh <email address hidden>

Avoid embedding launcher scripts on non-Windows (#1124)

Just to reduce binary size on all other platforms.

f946d46... by Charlie Marsh <email address hidden>

Avoid allocating a max-size buffer (#1123)

This seems potentially-dangerous with no upside.

3902126... by konstin

Windows launchers using posy trampolines (#1092)

## Background

In virtual environments, we want to install python programs as console
commands, e.g. `black .` over `python -m black .`. They may be called
[entrypoints](https://packaging.python.org/en/latest/specifications/entry-points/)
or scripts. For entrypoints, we're given a module name and function to
call in that module.

On Unix, we generate a minimal python script launcher. Text files are
runnable on unix by adding a shebang at their top, e.g.

```python
#!/usr/bin/env python
```

will make the operating system run the file with the current python
interpreter. A venv launcher for black in `/home/ferris/colorize/.venv`
(module name: `black`, function to call: `patched_main`) would look like
this:

```python
#!/home/ferris/colorize/.venv/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from black import patched_main
if __name__ == "__main__":
    sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0])
    sys.exit(patched_main())
```

On windows, this doesn't work, we can only rely on launching `.exe`
files.

## Summary

We use posy's rust implementation of a trampoline, which is based on
distlib's c++ implementation. We pre-build a minimal exe and append the
launcher script as stored zip archive behind it. The exe will look for
the venv python interpreter next to it and use it to execute the
appended script.

The changes in this PR make the `black` entrypoint work:

```powershell
cargo run -- venv .venv
cargo run -q -- pip install black
.\.venv\Scripts\black --version
```

Integration with our existing tests will be done in follow-up PRs.

## Implementation and Details

I've vendored the posy trampoline crate. It is a formatted, renamed and
slightly changed for embedding version of
https://github.com/njsmith/posy/pull/28.

The posy launchers are smaller than the distlib launchers, 16K vs 106K
for black. Currently only `x86_64-pc-windows-msvc` is supported. The
crate requires a nightly compiler for its no-std binary size tricks.

On windows, an application can be launched with a console or without (to
create windows instead), which needs two different launchers. The gui
launcher will subsequently use `pythonw.exe` while the console launcher
uses `python.exe`.

f1d3b08... by konstin

Add missing version to pip sync test (#1121)

The test started failing due to a newer version on pypi.