ubuntu-debuginfod:concurrency-rework

Last commit made on 2022-12-13
Get this branch:
git clone -b concurrency-rework https://git.launchpad.net/ubuntu-debuginfod
Only Sergio Durigan Junior can upload to this branch. If you are Sergio Durigan Junior please log in for upload directions.

Branch merges

Branch information

Name:
concurrency-rework
Repository:
lp:ubuntu-debuginfod

Recent commits

4e909f6... by Sergio Durigan Junior

*.py: Address Athos' comments

Athos made good points when reviewing the MP:

https://code.launchpad.net/~sergiodj/ubuntu-debuginfod/+git/ubuntu-debuginfod/+merge/434525

Signed-off-by: Sergio Durigan Junior <email address hidden>

7418f83... by Sergio Durigan Junior

*.py: Cosmetic changes (addressing Lena's review)

Signed-off-by: Sergio Durigan Junior <email address hidden>

d887f04... by Sergio Durigan Junior

debugpoller.py: Use rstrip when reading the timestamp file

Signed-off-by: Sergio Durigan Junior <email address hidden>

019c366... by Sergio Durigan Junior

{ddeb,debug}poller.py,poll_launchpad.py: Make consumer record timestamp

Decouple the timestamp recording step from get_ddebs_and_sources,
transferring the responsibility to the consumer.

Signed-off-by: Sergio Durigan Junior <email address hidden>

5f1a1a0... by Sergio Durigan Junior

{debuginfod,poll_launchpad}.py: Use DebugDB

Make use of the new DebugDB class, instead of handling the entire
connection many times.

Signed-off-by: Sergio Durigan Junior <email address hidden>

aaf0f10... by Sergio Durigan Junior

utils.py: New file, with new DebugDB class

The DebugDB class abstracts the access to psycopg2.

Signed-off-by: Sergio Durigan Junior <email address hidden>

5cadc19... by Sergio Durigan Junior

README.md: Adjust dependencies.

Signed-off-by: Sergio Durigan Junior <email address hidden>

6159cb0... by Sergio Durigan Junior

*.py: Cosmetic and documentation adjustments

Based on Bryce's comments at
https://code.launchpad.net/~sergiodj/ubuntu-debuginfod/+git/ubuntu-debuginfod/+merge/434017

Signed-off-by: Sergio Durigan Junior <email address hidden>

33739cb... by Sergio Durigan Junior

debuginfod.py: Refactor to account for finished/failed tasks

The Celery worker needs to account for each task that has finished
successfully and remove their entries from the SQL database. We also
record those tasks that failed with a non-expected exception, in order
to make it easier to investigate (and retry) them later.

Signed-off-by: Sergio Durigan Junior <email address hidden>

27ccfc5... by Sergio Durigan Junior

poll_launchpad.py: Refactor to keep state of dispatched tasks

In order to make the service more reliable, we need to keep track of
which tasks have been dispatched to Celery. We do this by populating
a SQL database for every task that is submitted.

The "bonus" part here is that we will now have separate tasks for
ddebs and source code fetching, now that both operations are
independent of each other.

Signed-off-by: Sergio Durigan Junior <email address hidden>