Don't mark for removal manually installed packages

Oh and also, the window is totally clogged up when you start it because it's analyzing the system...

CJ as it stands right now is an abomination and should be removed from the default install ASAP.

Confirmed. It seems that even packages you installed manually using dpkg -i or GDebi are considered as obsolete. This means from the moment you installed them, CJ wants to remove them. I guess there's a way to find out whether they were installed from a repository that is now disabled, or if the user explicitly installed them.

One of the common errors I can see with this is libdvdcss2, which most people install to read DVDs. It's installed by the script in /usr/share/doc/libdvdread4/install-css.sh: users don't know they need it, and removal may break their encrypted DVDs support without noticing.

In the dupe bug #704385, a user lost paid software because CJ uninstalled it. This behaviour should be dealt with soon.

What's causing this is that the CJ unsupported_plugin.py cruft finder plugin will match on all manually installed packages. If you look at the code

http://tinyurl.com/3sxhhhx

you'll see that the plugin even admits that this will happen, but doesn't really provide a way to prevent it. I'm not sure what strategy could be used to correct for this, but one suggestion is given in this branch:

http://tinyurl.com/44eg48w

Here, Regved suggests getting the list of packages installed by apt and the packages installed by dpkg and essentially whitelisting the intersection of the two. This is on the theory that manually installed packages will be in the dpkg database but not the apt database. Seems fairly reasonable, although I would like an apt expert such as Michael Vogt to weigh in on the matter.

Given that unsupported_plugin.py dates back to 2008, C-J has behaved this way for quite some time. Since it's not a regression, I am not in favor of making this bug release critical for Natty.

Why not simply only suggest the packages for removing which "sudo apt-get autoremove" would remove? It never suggested for me to remove any manufacturer-supplied printer drivers or other third-party packages.

Your suggestion to whitelist the packages which are installed without using apt does not help for printer drivers auto-installed with Jockey, as these get also installed via apt.

@Ian: if you installed vim-full manually via apt-get install apt should have marked it as a manual install. It would be nice to get the /var/lib/apt/extended_states file from your system.

The probem describe in #4 is that a package installed with "dpkg" or "gdebi" is not downloadable most of the time (because there is no repository associated with it. This is why CJ considers it "potentially obsolete".

The long term solution should be that a package records its origin in the extended_states file. As a short term solution we can just disable the plugin as the auto-removable plugin will take care of most of the important cases. Its currently impossible for the code to distinguish between a package was was availalbe in the repo at some point and is no more (i.e. a package that is no longer downloadable) from a package that got manually installed.

I have now done a test of triggereing a printer driver auto-download with Jockey and directly after that I have started computer-janitor and the newly installed driver is not on the list of packages to remove. So it seems Jockey has installed the driver with apt. But if I click on a driver download link on the OpenPrinting web site and then let software-center install the package, it appears in the list of packages to be removed.

Perhaps it is really the best for Natty to simply disable the plugin, as Michael Vogt said.

<mvo> barry: I agreed with what you said in the bugreport, its not a
      regression in any way, the long term solution should be to record if it
      was a install via dpkg or not [15:13]
<mvo> barry: TBH I don't understand it, or rather I think the result is/will
      be that all installed packages are considered "dpkg" installs (so its
      equivalent of just turning the plugin off) [15:16]

So, I've marked this as wishlist and targeted it for later.

Barry, can we perhaps simply deactivate this plugin for Natty? And get it later in again if it gets more sensitive in deciding what can be removed and what not?

I'm reluctant to do that because that could be considered a regression and we won't have another beta to test it out. Please note that you can always whitelist the package manually by adding a file called /etc/computer-janitor.d/<whatever>.whitelist

That may in fact lead to an interesting approach. Software Center and/or dpkg could be adjusted to add a .whitelist file whenever a package is manually installed.

"Regression" is very relative. I think in this precise case we can easily anticipate the costs and benefits of such a change without requiring user testing. Few people will complain that CJ doesn't remove this kind of obsolete packages, and if they really know they want to get rid of them, they can use Synaptic. And in general, packages no associated with any repository are few and won't do much harm: the big part is removing libraries you installed automatically as dependencies of other programs, and kernels.

OTC, people that don't understand that these packages may be useful to them will not get bitten if we make this change, which is a *huge* benefit.

There's also another solution: show these packages, but don't check them for removal by default. Not sure that's easy to do.

We are indeed a bit close to the release by now. One relatively straightforward fix would be to patch python-apt/gdebi/aptaemon so that the install_deb code writes the packagename into /var/lib/apt/notjunk (or manualdownlaoded or something like this). The CJ could honor that. This would not catch the "dpkg directly invoked" case yet, but I expect most people will install either via gdebi or software-center (via aptdaemon).

That would sure be nice, but one of the use cases where dpkg is called directly is when installing libdvdcss2: this is one of the worst cases since people won't notice they actually need this package, which was installed automatically by a script.