ubuntu-debuginfod:concurrency-rework

Branch merges

Propose for merging

Merged into ubuntu-debuginfod:master at revision 4e909f63e8695ed61e0c259d129075b52c7ead54

Athos Ribeiro (community): Approve on 2022-12-14

Canonical Server packageset reviewers: Pending requested 2022-12-13

Robie Basak: Pending requested 2022-12-13

Lena Voytek: Pending requested 2022-12-13

Bryce Harrington: Pending requested 2022-12-13

Canonical Server Reporter: Pending requested 2022-12-13

Diff: 1004 lines (+443/-248)

8 files modified

README.md (+6/-3)
ddebgetter.py (+85/-174)
ddebpoller.py (+83/-32)
debuggetter.py (+46/-7)
debuginfod.py (+85/-10)
debugpoller.py (+10/-13)
poll_launchpad.py (+80/-9)
utils.py (+48/-0)

api

Related source package recipes

No recipes using this branch. (?)

Create packaging recipe

Related snap packages

Related charm recipes

4e909f6... by Sergio Durigan Junior on 2022-12-13

*.py: Address Athos' comments

Athos made good points when reviewing the MP:

https://code.launchpad.net/~sergiodj/ubuntu-debuginfod/+git/ubuntu-debuginfod/+merge/434525

Signed-off-by: Sergio Durigan Junior <email address hidden>

7418f83... by Sergio Durigan Junior on 2022-12-13

*.py: Cosmetic changes (addressing Lena's review)

Signed-off-by: Sergio Durigan Junior <email address hidden>

d887f04... by Sergio Durigan Junior on 2022-12-13

debugpoller.py: Use rstrip when reading the timestamp file

Signed-off-by: Sergio Durigan Junior <email address hidden>

019c366... by Sergio Durigan Junior on 2022-12-13

{ddeb,debug}poller.py,poll_launchpad.py: Make consumer record timestamp

Decouple the timestamp recording step from get_ddebs_and_sources,
transferring the responsibility to the consumer.

Signed-off-by: Sergio Durigan Junior <email address hidden>

5f1a1a0... by Sergio Durigan Junior on 2022-12-13

{debuginfod,poll_launchpad}.py: Use DebugDB

Make use of the new DebugDB class, instead of handling the entire
connection many times.

Signed-off-by: Sergio Durigan Junior <email address hidden>

aaf0f10... by Sergio Durigan Junior on 2022-12-13

utils.py: New file, with new DebugDB class

The DebugDB class abstracts the access to psycopg2.

Signed-off-by: Sergio Durigan Junior <email address hidden>

5cadc19... by Sergio Durigan Junior on 2022-12-13

README.md: Adjust dependencies.

Signed-off-by: Sergio Durigan Junior <email address hidden>

6159cb0... by Sergio Durigan Junior on 2022-12-13

*.py: Cosmetic and documentation adjustments

Based on Bryce's comments at
https://code.launchpad.net/~sergiodj/ubuntu-debuginfod/+git/ubuntu-debuginfod/+merge/434017

Signed-off-by: Sergio Durigan Junior <email address hidden>

33739cb... by Sergio Durigan Junior on 2022-12-13

debuginfod.py: Refactor to account for finished/failed tasks

The Celery worker needs to account for each task that has finished
successfully and remove their entries from the SQL database. We also
record those tasks that failed with a non-expected exception, in order
to make it easier to investigate (and retry) them later.

Signed-off-by: Sergio Durigan Junior <email address hidden>

27ccfc5... by Sergio Durigan Junior on 2022-12-13

poll_launchpad.py: Refactor to keep state of dispatched tasks

In order to make the service more reliable, we need to keep track of
which tasks have been dispatched to Celery. We do this by populating
a SQL database for every task that is submitted.

The "bonus" part here is that we will now have separate tasks for
ddebs and source code fetching, now that both operations are
independent of each other.

Signed-off-by: Sergio Durigan Junior <email address hidden>