The Librarian was refactored for a much reduced memory footprint. Unfortunately, it doesn't work.
Traceback (most recent call last):
File "/srv/launchpadlibrarian.net/production/launchpad/cronscripts/librarian-gc.py", line 93, in ?
script.lock_and_run(isolation=ISOLATION_LEVEL_AUTOCOMMIT)
File "/srv/launchpadlibrarian.net/production/launchpad-rev-8323/lib/lp/services/scripts/base.py", line 286, in lock_and_run
implicit_begin=implicit_begin, isolation=isolation)
File "/srv/launchpadlibrarian.net/production/launchpad-rev-8323/lib/lp/services/scripts/base.py", line 244, in run
self.main()
File "/srv/launchpadlibrarian.net/production/launchpad/cronscripts/librarian-gc.py", line 81, in main
librariangc.delete_unreferenced_aliases(conn)
File "/srv/launchpadlibrarian.net/production/launchpad-rev-8323/lib/canonical/librarian/librariangc.py", line 314, in delete_unreferenced_aliases
loop_tuner.run()
File "/srv/launchpadlibrarian.net/production/launchpad-rev-8323/lib/canonical/launchpad/utilities/looptuner.py", line 107, in run
self.operation(chunk_size)
File "/srv/launchpadlibrarian.net/production/launchpad-rev-8323/lib/canonical/librarian/librariangc.py", line 294, in __call__
cur.execute("""
psycopg2.IntegrityError: update or delete on table "libraryfilealias" violates foreign key constraint "binarypackagefile_libraryfile_fk" on table "binarypackagefile"
DETAIL: Key (id)=(29935579) is still referenced from table "binarypackagefile".
This is being done because uploads are occurring between calculating the set of files we won't expire even if they are unreachable (including new uploads), and the set of files that need removing.
Solution is to defer calculating the set of files we won't expire even if they are unreachable until the later transaction.