arfile: Regression: Collect file<->deb/ar reference cycles
The internal FileFd object now owned the PyObject* that gave us
the descriptor but we were never visiting that object during garbage
collection, so if there was a cycle, Python could not realize that.
Make the objects garbage collectable, by adding VISIT and CLEAR
calls for self->Fd, and by making the FileFd object support garbage
collection in the first place.
arfile.cc: Fix segmentation fault when opening fd, track lifetime
correctly
The lines that created self->Fd and that then made use of it were
swapped, causing a segmentation fault.
Also the life of the file object was tracked incorrectly, causing
the file to be closed if it was a temporary one.
Closes: #977000
(cherry picked from commit 3d9af5f196ad6a6c6973ac699a15888d21a9bb52)
apt_inst.DebFile: Avoid reference cycle with control,data members
apt_inst.DebFile provides two members `data` and `control` for
easy access to those tarballs. Each of those members stores a
reference to the DebFile as its owner:
v-----------------\
control ----\ |
-> deb -|
data ----/ |
^-----------------/
This means that whenever a DebFile is successfully constructed,
and no longer needed, it won't be collected until the GC runs,
which is bad, as the DebFile holds an open FileFd.
Introduce a __FileFd wrapper that holds the FileFd and becomes
the owner of both control and data, and replaces the direct use
of the FileFd in ArArchive/DebFile:
v-----------------------------\
control ----\ \
-> __FileFd <- deb -|
data ----/ /
^-----------------------------/
This avoids the reference cycle, ensuring the memory and file
descriptor are released by the reference counter as soon as
the reference count drops to 0.
A future version should move `apt_inst.__FileFd` to `apt_pkg.FileFd`
and expose all the methods, such that people can make use of FileFd's
extensive compression support.
We have a similar cycle in TagFile that we have yet to address,
the problem there is arguably more frustrating, as the buffer
I believe is stored inside the TagFile, and that's really shared
between the TagSection objects.
This is related to LP: #1899193 and CVE-2020-27351, but an additional
hardening measure - the fix for those bugs was for more direct leaks.
(cherry picked from commit a43948f6299a1c216a767baa8292b1811c0a0fbb)
File descriptor leaks in ArArchive, DebFile, TagFile
Fix various file descritor, and memory leaks in ArArchive, DebFile,
and TagFile by introducing a new PyApt_UniqueObject smart pointer
that is like a unique_ptr, but backportable to older releases, and
automatically clears subobjects, so objects with cycles like DebFile
and TagFile will be released on error paths.
LP: #1899193
GHSL-2020-170
CVE-2020-27351
(cherry picked from commit 8d53d2bcaa31de8f402d52975c0a10d853928184)
