git-ubuntu

~racb/git-ubuntu:ipc-concurrency

  1. Git
  2. lp:~racb/git-ubuntu
  3. ipc-concurrency
Last commit made on 2020-07-06
Get this branch:
git clone -b ipc-concurrency https://git.launchpad.net/~racb/git-ubuntu
Only Robie Basak can upload to this branch. If you are Robie Basak please log in for upload directions.

Branch merges

Branch information

Name:
ipc-concurrency
Repository:
lp:~racb/git-ubuntu

Recent commits

41c409a... by Robie Basak

Add new IPC-based service components

This adds three new subcommands: importer-service-poller,
importer-service-broker and importer-service-worker.

I had a choice here: I could have added new snap endpoints as was done
previously with import-source-packages.py becoming
git-ubuntu.import-source-packages, but I think I prefer moving all
endpoints into git-ubuntu subcommands now that we have these, so I did
this instead.

6b30fa0... by Robie Basak

Do not return requests for packages with imports in progress

8e379dd... by Robie Basak

Test refactor

Add test parameters so that new test instances can be added in the next
change which will use a different status from 'needed' and expected
results can also be empty.

298b6bc... by Robie Basak

Running requests must now never get superseded

Now that we will be tracking requests being run by external workers,
superseding them while they are running will confuse the logic handling
a worker's "request complete" message if that request has already been
superseded. Instead, the spec has been adjusted and we will no longer
supersede requests that are marked as "running".

8b53a9e... by Robie Basak

Add ipc_worker table

This table will store the details of which worker has been given which
request, according to the specification.

d433f6d... by Robie Basak

Add database concurrency support

An IPC component-based service will be implemented shortly. This will
require concurrent database access from the poller and the broker.

This change adjusts the Service class so that it will work correctly
without deadlocks if multiple instances exist that are accessing the
same database.

0126c18... by Robie Basak

Move package list read to importer_service

We will shortly need to read package lists in the same way that it is
currently done in import-source-package.py, so move this functionality
into a module so it can be called internally. This also reduces some
duplicate code that was already present.

3e1569c... by Robie Basak

Move request_new_imports() to importer_service.py

request_new_imports() will shortly be needed to implement the poller
component of the IPC-based importer service, so move it into a module so
it can be called internally.

Nothing is functionally changed. Some references needed qualifying
differently as the imported names are a little different in
importer_service.py, and the docstring is updated.

740b308... by Robie Basak

Drop pylint and astroid pinning

This appears fixed upstream. See
https://github.com/PyCQA/pylint/issues/3137#issuecomment-542286971 and
https://github.com/PyCQA/pylint/issues/3137#issuecomment-645867810. If
this is incorrect, then CI will fail, so this is a safe change to make.

As astroid only appeared in setup.py in order to pin the indirect
dependency, it can be dropped from the dependency list entirely. pylint
will still pull it in as required.

1fa1643... by Robie Basak

Use the same pylintrc locally as in CI

The git-ubuntu.self-test command was using its own pylint invocation
without reference to the .pylintrc at the top level of our source tree.
This is used in CI, and meant that a local developer pylint invocation
might pass (due to .pylintrc settings) but CI fails.

Fix this by placing .pylintrc into the snap (at $SNAP/etc/pylintrc) and
call pylint using --rcfile when invoked via the git-ubuntu.self-test
command so that the same configuration is used in all cases.