~bdrung/ddeb-retriever/+git/main:race

Last commit made on 2023-02-20
Get this branch:
git clone -b race https://git.launchpad.net/~bdrung/ddeb-retriever/+git/main
Only Benjamin Drung can upload to this branch. If you are Benjamin Drung please log in for upload directions.

Branch merges

Branch information

Name:
race
Repository:
lp:~bdrung/ddeb-retriever/+git/main

Recent commits

f5cff49... by Benjamin Drung

fix: Make dists generation nearly atomic

There is a time frame where the `Packages` files are updated, but the
`Release` files are not updated yet. To reduce the time frame where the
archive is in an inconsistent state, generate the update for the `dists`
directory in a temporary directory. Once the update is prepared, use
`os.rename` to move the old directory away and move the newly prepared
directory in place. There is a tiny time frame between both `os.rename`
where the directory is missing, but this time frame should be small
enough to be not relevant in practice.

To make the `dist` directory generation truly atomic, symlinks need to
be used. Even after that there could be a race on the client side: The
client could fetch the `Release` file, then the `dists` directory could
be updated, then the client tries to fetch the `Packages` file.

Signed-off-by: Benjamin Drung <email address hidden>

1bed843... by Benjamin Drung

refactor: Introduce _create_dist_directories_if_missing

Split `_create_dist_directories_if_missing` from `create_indexes` and
move it to `_generate_dist_directory`. It is not needed before that
point.

Signed-off-by: Benjamin Drung <email address hidden>

aca4773... by Benjamin Drung

refactor: Introduce _generate_dist_directory

Split `_generate_dist_directory` from `create_indexes`.

Signed-off-by: Benjamin Drung <email address hidden>

612eb65... by Benjamin Drung

feat: Generate InRelease files in addition to Release.gpg

The Apport system tests often fail with:

```
W:GPG error: http://ddebs.ubuntu.com jammy-updates Release: The following signatures were invalid: BADSIG C8CAB6595FDFF622 Ubuntu Debug Symbol Archive Automatic Signing Key (2016) <email address hidden>
E:The repository 'http://ddebs.ubuntu.com jammy-updates Release' is not signed.
```

So generate an `InRelease` file in addition to `Release.gpg`.

Bug: https://launchpad.net/bugs/2002833
Signed-off-by: Benjamin Drung <email address hidden>

e9fee8a... by Benjamin Drung

feat: Generate .Release.gpg.new and use atomic replace

There is an unnecessary window when `Release.gpg` is absent. Generate
`.Release.gpg.new` and use atomic replace to update `Release.gpg`. There
is still a window where `Release` and `Release.gpg` are out of sync.
This should be addressed in the future by atomically replacing the
complete directory.

Signed-off-by: Benjamin Drung <email address hidden>

bfef7ba... by Benjamin Drung

refactor: Replace try/except by contextlib.suppress

Shorten the code by using `contextlib.suppress`.

Signed-off-by: Benjamin Drung <email address hidden>

dd59505... by Benjamin Drung

refactor: Use subprocess.run in _generate_release_signatures

Signed-off-by: Benjamin Drung <email address hidden>

a2333e2... by Benjamin Drung

refactor: Introduce _generate_release_signatures

Split `_generate_release_signatures` from `create_indexes`.

Signed-off-by: Benjamin Drung <email address hidden>

22b520a... by Benjamin Drung

Sort Python imports with isort

```
isort .
```

Signed-off-by: Benjamin Drung <email address hidden>

98fd421... by Colin Watson

Set compressext too.