lp:ppa-dev-tools

Author: Bryce Harrington
Author Date: 2024-04-20 01:08:48 UTC

packaging: Drop deprecated setup.py in favor of 'build' pkg system

The packaging has already partly transitioned to using the build
packaging system with a pyproject.toml and a Makefile. Complete the
transition by dropping setup.py and any reference to it in directions.

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/1989617

Author: Bryce Harrington
Author Date: 2024-03-21 08:12:41 UTC

Add functionality to filter test results by package

Specifying the --package command line option has filtered the list of
triggers for a PPA, but previously it did not filter the test results.
For PPAs involving a variety of packages this can produce an excessive
listing.

This commit expands support for --package to apply not only to results
as well, but also the running and waiting lists.

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/2025483

Author: Bryce Harrington
Author Date: 2024-03-21 08:12:41 UTC

Add functionality to filter test results by package

Specifying the --package command line option has filtered the list of
triggers for a PPA, but previously it did not filter the test results.
For PPAs involving a variety of packages this can produce an excessive
listing.

This commit expands support for --package to apply not only to results
as well, but also the running and waiting lists.

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/2025483

Author: Bryce Harrington
Author Date: 2024-04-05 05:21:32 UTC

Releasing 0.5.1

Author: Bryce Harrington
Author Date: 2024-03-27 02:03:43 UTC

trigger: Fix omission of all-proposed URLS when -L specified

The all-proposed links should always be displayed but in one particular
combination of options it'll skip printing them, due to a coding error.

Author: Bryce Harrington
Author Date: 2024-03-18 23:37:17 UTC

job: Fix whitespace in jobs code (flake8)

Author: Bryce Harrington
Author Date: 2024-02-29 21:03:26 UTC

ppa: Catch exceptions for issues when reading config files

I'm intentionally forcing the exception object to a str in the f-string.
I've found that YAMLError's exception object can be a bit over detailed,
but the str() representation gives it in a human-readable form.

Author: Bryce Harrington
Author Date: 2024-02-08 07:38:44 UTC

results: Fix invalid url generation for get_results()

Ensure that the url is constructed with '/' between components.

Author: Bryce Harrington
Author Date: 2023-10-14 02:03:52 UTC

results: Avoid printing excessive duplicate results

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/2038649

Author: Bryce Harrington
Author Date: 2023-10-12 21:55:14 UTC

Include ESM releases in allowed list of triggers.

The `ppa tests` command, when run on a PPA that includes packages
targeting a number of different old Ubuntu releases, will filter the
list and show triggers only for supported LTS releases. Autopkgtest
won't run tests for unsupported releases, so this makes sense.

However, distro-info's definition of "supported" does not include Ubuntu
releases that are in 'Expanded Security Maintenance' (ESM). However,
Autopkgtest's definition of "supported" does, as can be seen at
https://autopkgtest.ubuntu.com/running. Thus, triggers should be
generated for the ESM releases, too.

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/2038651

Author: Bryce Harrington
Author Date: 2023-09-12 23:01:16 UTC

Fix incorrect processing of --packages as well

Similar to the incorrect processing of --releases as a release, the
--packages parameter suffers the same logic error. However, in this
case it doesn't show up in the triggers listing since it is used like
this:

     if packages and (pkg not in packages):
         continue

So, whether packages is a comma separated string or a list, pkg will be
found correctly. However, in order to (eventually) fix LP: 2025483 it
needs to be converted to a list.

Author: Bryce Harrington
Author Date: 2023-09-08 06:05:44 UTC

wait: Add --watch option to clear screen between refreshes

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/1989085

Author: Bryce Harrington
Author Date: 2023-09-11 23:05:58 UTC

ppa: Include name of the default filename in help text

Author: Bryce Harrington
Author Date: 2023-08-25 20:32:56 UTC

tests: Implement a few basic presentational tests for Ppa class

Author: Bryce Harrington
Author Date: 2023-08-30 01:28:33 UTC

result: Support newer autopkgtest log formats

Since mantic, autopkgtest logs are prepended with a timestamp, which
confuses the parser resulting in the timestamp displayed instead of the
subtest description. This fixes the issue by detecting lines beginning
with three or more numerical digits and ignoring the first word in the
line for extracting subtests.

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/2025484

Author: Bryce Harrington
Author Date: 2023-08-12 00:30:17 UTC

Cleanup unneeded rdepends variable

Author: Bryce Harrington
Author Date: 2023-07-18 02:18:17 UTC

Improve error for non-existing PPAs

Since we weren't returning after encountering this error, it allowed the
unhandled error processing to print an error as well, which is not
desired:

   $ ./scripts/ppa set --private ppa:bryce/non-existing-ppa
   The PPA 'non-existing-ppa' does not exist for person or team 'bryce'
   Unhandled error

With this change the 'Unhandled error' line is not printed.

Author: Bryce Harrington
Author Date: 2023-06-29 20:46:33 UTC

Implement JSON output support at object level

Adds .to_dict() members to principle objects involved in representing
test results, and associated test cases.

Previous work has striven to split the data processing from the data
display. This takes the next step of establishing groundwork for
representing the data objects themselves in an easily flattenable
format. That establishes the groundwork needed to create user-specified
options like --json and --yaml (or maybe --format=json, --format=yaml)
to dump results in a machine-parsable form rather than in the
human-readable "bullet-tree" style output.

The test cases don't consider the validity of the data elements
themselves, but just that the expected fields are present and that none
have unexpected data types.

Author: Bryce Harrington
Author Date: 2023-06-27 02:24:56 UTC

wait: Handle lazr BadRequest during wait operation

Launchpad can return BadRequest errors in some indeterminate situations,
possibly such as if Launchpad is under heavy loads, but also potentially
due to usage/configuration issues.

Include logic to retry the Launchpad operations in case the problem is
server-side, but also limit the number of sequential retries to 3, to
avoid getting stuck in a loop if the problem is something else. The
counter is reset to zero each time we successfully complete a wait
cycle, so that if Launchpad is *slightly* glitchy this will be
forgiving.

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/1997122

Author: Bryce Harrington
Author Date: 2023-05-24 02:22:58 UTC

Refactor team name to owner name

Hat tip to Andreas for pointing out 'owner' is Launchpad's preferred
jargon for the person or team that a PPA belongs to.

Support the --team-name argument and the 'team_name' configuration
parameter for legacy reasons, even though the former didn't exist and
the latter was bugged.

Author: Bryce Harrington
Author Date: 2023-05-30 20:49:42 UTC

tests: Accept args from the getPublishedSources() API

This branch adds support for passing some additional parameters to the
Launchpad archive object's getPublishedSources() API. The test suite
does not yet have coverage for these new args, but this updates the
ArchiveMock to at least not error when they're present.

Author: Bryce Harrington
Author Date: 2023-05-15 18:56:00 UTC

results: Add show_results() for managing display of results

With most of the formatting logic for a single result now moved to the
new to_bullet_tree() member for the Results class, a new helper function
show_results() is added to manage the high level results display logic
for a list of results.

This new function is intended to follow the design style of
show_running() and show_waiting() from the Jobs module.

Notably, the one user-visible change in this refactoring is that, like
show_running() and show_waiting(), it will display "(none)" when there
are no results.

Author: Bryce Harrington
Author Date: 2023-04-19 21:58:27 UTC

INSTALL: Mention need for the distro_info python module

Author: Bryce Harrington
Author Date: 2023-04-08 00:24:29 UTC

set: Add --ppa-dependencies support

This feature permits associating other PPAs as available to satisfy
build dependencies for this PPA.

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/1998140

Author: Bryce Harrington
Author Date: 2023-04-18 18:31:24 UTC

suite: Fix handling of suites with no packages

The lunar-updates/restricted suite, for example, has no packages yet, so
the binaries and sources objects generated from Packages.xz and
Sources.xz will be empty ("{}"), which fails the validity test and
throws the post-condition exception.

Instead, use None values for sources and binaries to indicate undefined
behavior, and treat an empty dict as a valid case.

Author: Bryce Harrington
Author Date: 2023-04-06 02:22:56 UTC

suite: Add dependent_packages() API

This routine performs the lookup of a given source package against the
collected Apt info, to determine a list of packages worth running
autopkgtests against. This includes both the source package's build
dependencies and other source packages that depend on it.

Currently, this does not consider recursive dependencies. I.e. if a
build-depends on b, and b build-depends on c, this routine will return
only 'b' when asked about 'a'.

It also is currently unable to handle source packages with undefined or
empty build dependency lists.

Author: Bryce Harrington
Author Date: 2023-04-04 03:37:10 UTC

suite: Incorporate SourcePackage and BinaryPackage

Switch the .sources and .binaries member functions to provide actual
SourcePackage and BinaryPackage class objects in their returns, and
update test cases accordingly.

Author: Bryce Harrington
Author Date: 2023-03-30 01:15:17 UTC

source_package: Add module for the SourcePackage class

This class provides access to the Apt information about source packages,
including the binaries they provide, build requirements, etc.

Author: Bryce Harrington
Author Date: 2023-03-18 00:17:34 UTC

binary_package: Add module for the BinaryPackage class

The BinaryPackage class provides an interface for installation
dependencies and recommends.

Author: Bryce Harrington
Author Date: 2023-03-24 05:29:10 UTC

Use unpack_to_dict() for splitting --architecture values

This is a bit overkill for handling architecture strings, but since
unpack_to_dict() is intended to be used for parsing all handling of
comma-separated values in the CLI, it will be most consistent if it's
used here as well.

Author: Bryce Harrington
Author Date: 2023-03-23 06:57:07 UTC

create/set: Add support for --publish setting

The --publish/--no-publish parameters controls whether or not the PPA is
configured to allow packages to be downloaded once they've been
successfully built. With --no-publish, packages will still be
built (although not for private PPAs, for some reason).

Author: Bryce Harrington
Author Date: 2023-03-15 05:40:19 UTC

Add Suite class

Break the Suite class out of the repository module and complete its
implementation, including unit tests and a smoke test.

As previously, the smoke test expects to work on a local mirror of an
Apt repository. This can be created using the apt-mirror utility.

Author: Bryce Harrington
Author Date: 2023-03-06 23:49:08 UTC

dict: Refactor subfunction _split_item() for duplicated logic

Author: Bryce Harrington
Author Date: 2023-03-02 03:12:27 UTC

Add repository class

Provides a programmatic interface to the top directories a local mirror
of an apt repository. This allows inspection of the main sections of
the mirrors or "suites", which represent different Ubuntu distro release
pockets such as 'lunar-proposed', 'kinetic-security', or 'feisty-updates'.

This includes handling of one special case: The 'release' pocket is not
stored by name, but represents the default (blank) pocket. So, for
example, 'jammy' should really be interpreted as 'jammy-release'.

(This also includes an initial stub of a Suite class but just as a data
shim. It will be broken out and expanded in latter commits.)

Author: Bryce Harrington
Author Date: 2023-03-03 07:18:12 UTC

ppa: Raise exception on invalid ppa name

This might break a few uses that are only checking for a null config,
but raising an exception in this circumstance is a better behavior.

Author: Bryce Harrington
Author Date: 2023-02-14 01:58:44 UTC

ppa: When no release is specified, default to supported releases

Previously we were assuming the devel release (i.e. lunar currently),
however if a PPA doesn't have builds for that release there will nothing
in the output except the headers. That looks wrong and doesn't really
fit user expectations.

Instead, make the default be all currently supported Ubuntu
releases (i.e. bionic, focal, jammy, kinetic, and lunar presently). The
display code already skips output for releases that the PPA doesn't
have builds for.

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/2003094

Author: Bryce Harrington
Author Date: 2023-01-14 00:27:13 UTC

Makefile: Add snap target

Author: Bryce Harrington
Author Date: 2022-12-16 06:51:32 UTC

create: Add unit test

I don't think this test case needs to test the team name or ppa address
since those are already adequately covered by test_ppa_group.py. This
test basically just needs to verify the specified command line arguments
were interpreted and applied correctly.

Author: Bryce Harrington
Author Date: 2022-12-15 10:23:34 UTC

Add ppa set command

The set and create commands will support the same set of of
configuration settings, so this adds the set command and generalizes
some of the option handling for create to work for both.

For now this supports only the already existing displayname,
description, and architecture settings, but this establishes the
structure we'll slot all the other settings into subsequently.

Author: Bryce Harrington
Author Date: 2022-12-07 17:41:03 UTC

ppa: For subparsers, don't add attributes if arg isn't present

Since we're now adding global arguments both at the top level and at the
subparser level, it's possible to cause the default to be set twice -
which can override options set at the top level (before the command)
with the subparser's post-command argument handler. Setting SUPPRESS on
the subparsers prevents this behavior.

Fixes the following failure found by the testsuite:

            # Check -D, --debug
            args = parser.parse_args(['-D', 'status', 'test-ppa'])
    > assert args.debug is True
    E AssertionError: assert False is True
    E + where False = Namespace(command='status', config_filename='~/.config/ppa-dev-tools/config.yml', debug=False, dry_run=False, ppa_name='test-ppa', quiet=False, verbose=False).debug

Note that a default value needs to be provided for --show-urls, else the
following test failure happens:

            # Check --show-urls, --show-url, -L
            args = parser.parse_args([command, 'tests-ppa'])
    > assert args.show_urls is False
    E AttributeError: 'Namespace' object has no attribute 'show_urls'

The --show-urls argument for the tests command uses action='store_true',
which includes automatic functionality to set the default value to the
opposite (False), however it appears that SUPPRESS interferes with this.

Author: Bryce Harrington
Author Date: 2022-10-25 08:52:15 UTC

tests: Implement the --architectures option for test result display

Author: Bryce Harrington
Author Date: 2022-10-18 17:34:51 UTC

README: Use formal ppa address in docs

With the recent fix, all the steps that were documented in README should
now work as written. However, using the formal ppa address is more
generally correct so should be preferred in the entry-level docs.

Author: Bryce Harrington
Author Date: 2022-10-19 05:30:26 UTC

Add --architectures option for ppa create

Author: Bryce Harrington
Author Date: 2022-10-01 05:51:37 UTC

trigger: Provide urls to the triggerable actions, with the ppa

The "autopkgtest_url" is renamed to "history_url" to distinguish it from
the actual url used to trigger the test run, which will be called the
"action_url".

Author: Bryce Harrington
Author Date: 2022-08-13 01:13:42 UTC

ppa: Set up subparsers for each command

Author: Bryce Harrington
Author Date: 2022-09-29 01:38:12 UTC

RELEASING: Update snap directions for releases

Author: Bryce Harrington
Author Date: 2022-09-21 23:01:13 UTC

test_result: Fix invalid local path to data dir

Author: Bryce Harrington
Author Date: 2022-09-16 04:13:45 UTC

Fix dep8 checks

This gets `make check` to pass, which enables `make coverage` to
succeed:

===============================================
60 passed, 9 xfailed, 2 xpassed in 0.27 seconds
===============================================

python3-coverage report
Name Stmts Miss Cover
--------------------------------------
ppa/__init__.py 0 0 100%
ppa/_version.py 2 0 100%
ppa/constants.py 7 0 100%
ppa/debug.py 13 7 46%
ppa/io.py 12 4 67%
ppa/job.py 101 45 55%
ppa/lp.py 51 1 98%
ppa/ppa.py 193 149 23%
ppa/ppa_group.py 55 14 75%
ppa/processes.py 41 41 0%
ppa/result.py 117 64 45%
ppa/subtest.py 42 16 62%
ppa/text.py 64 51 20%
ppa/trigger.py 25 4 84%
--------------------------------------
TOTAL 723 396 45%

Author: Bryce Harrington
Author Date: 2022-09-15 06:30:01 UTC

subtest: Format subtests in table columns

Author: Bryce Harrington
Author Date: 2022-08-24 01:04:17 UTC

Add Makefile

Incorporated from Athos' lppa (https://github.com/athos-ribeiro/lppa),
but with some functionality dropped to simplify what will be used (and
test) in the near term. This also expands the version insertion support
to cover more places where version numbers are specified.

Author: Bryce Harrington
Author Date: 2022-09-13 08:05:38 UTC

ppa: Improve handling of missing leading ppa:

Gracefully handle the simple case of user omitting the leading ppa: in
the ppa argument, if there is at least one / in it. E.g. make this
work:

     $ ppa wait paelzer/lp-1975764-dpdk-mre-sept-2022

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/1989079

Author: Bryce Harrington
Author Date: 2022-09-10 02:24:40 UTC

lp: Fix return value for ubuntu_active_series()

Instead of name strings, this is supposed to return distro_series objects

Author: Bryce Harrington
Author Date: 2022-08-31 03:37:15 UTC

Add Results class and test

The result, subtest, and trigger modules can also be invoked as scripts:

    $ python3 -m ppa.result

These are intended to serve as simple smoketests.

Author: Bryce Harrington
Author Date: 2022-08-26 02:35:45 UTC

Add the 'tests' command

This pulls in show_running() and show_waiting() from lp-test-ppa with
some modifications to work with the current Job class, and adds a new
command called 'tests' to report these.

Note that we're not yet showing the completed results, so the command as
implemented currently is only useful when a PPA's tests are in process.

Author: Bryce Harrington
Author Date: 2022-08-23 05:41:43 UTC

Job: Add smoketest for get_waiting() and get_running()

Extract the job info for a given ppa from static snapshots of the
autopkgtest running.json and queues.json.

Author: Bryce Harrington
Author Date: 2022-07-28 22:57:58 UTC

lp: Adapt use of the pip xdg module if present

The xdg module from pip does not appear to provide the same import
targets as the Debian python3-xdg module, however if xdg is installed
via pip it will be loaded in preference to the system module.

Work around this situation by catching the import error and using the
pip xdg's API to get the config directory.

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/1983056

Author: Bryce Harrington
Author Date: 2022-07-28 22:17:57 UTC

setup.py: Drop pip requirements, to avoid confusion with xdg module

The xdg module installed from Ubuntu via apt seems to have different
import syntax than the xdg module installed via pip. The INSTALL.md
directions explain the former, which works with the codebase, but
running setup.py triggers pip to install xdg from pypi.

Fixes: https://bugs.launchpad.net/ppa-dev-tools/+bug/1983056
Signed-off-by: Bryce Harrington <bryce@bryceharrington.org>

Author: Bryce Harrington
Author Date: 2021-09-20 05:50:15 UTC

wip