Feature Freeze exception request

I just want to add that I introduced the new API in python-apt 0.7.92 in Debian experimental in August 2009. The code is in unstable since Wed, 20 Jan 2010 17:06:20 +0100 and is now stable. Another side benefit of this version is that we now free the objects we create; which was not possible in the past due to some mistakes with regards to the C++ templates used in the code (LP: #370149).

Furthermore, I already provided patches for 22 packages to use the new API; and it would make collaboration between Debian and Ubuntu a lot easier if we were to ship the same python-apt API.

See also this thread: http://bugs.debian.org/cgi-bin/pkgreport.cgi?<email address hidden>;tag=python-apt-0-8-api and http://lists.debian.org/debian-devel/2010/02/msg00424.html - and sorry for three comments in a row; it seems I forget many things today.

Michael,

I read the changelog since 0.7.13.4 (which we have in lucid right now), and indeed it looks like half a rewrite. python-apt does have some tests, but they only seem to cover the API to a low degree.

To be honest I'd rather like to see this in a PPA first, so that we can test rdepends which use python-apt heavily (ubiquity, packagekit-backend-apt, jockey, landscape, software-center).

Can you please explain the "on the next LTS we can work with the new (nicer) API and do not need to maintain both old and new API when we do a lucid->lts+1 upgrade."? If a package in the next LTS uses the new API (which all of them will, hopefully), wouldn't they just depend on the new version? I guess you mean that the dist-upgrader itself would need to use the old python-apt API?

I would not say it's half a rewrite.

all: Python 3 support
aptsources: Almost completely identical, just some small fixes.
apt: API rename, using tested compatibility functions derived from the ones used in python-debian.
apt.progress: Mostly rename, and a rewritten AcquireProgress (the complete old one is still provided for compat)
apt_inst: Additional code only

apt_pkg
------------
- API rename, using a single tp_getattro function which translates old attributes names to the new ones.
- Use classes instead of functions for creating objects
- Fix the memory deallocation of most objects (previously, the object we pointed to was not deleted)
- Some new classes, but they do not change anything in the old ones
- Use getset instead of tp_getattr; as this is more reliable and allows pydoc to work.
- Some stylistic changes

The changelog looks larger than the changes are in reality, because it also lists stuff that was already backported into 0.7.13.4 and lists some fixes for new functionality again; and because the changes are often documented per-file.

Shipping the new API now makes it easier to merge patches; and gives developers one python-apt API for Debian & Ubuntu instead of two. And hey, if Debian can ship it, Ubuntu should do it as well.

Oh, and the earlier the merge is done; the more testing the code gets. It should have been done earlier, but doko is not a very responsive person on the Debian side, and I wanted to wait for Python 3 in Debian sid first; and could not merge the sid and experimental branches until doko finally managed to upload python3.1 to unstable. And mvo was also very busy, we were planning to merge it before the freeze; but this failed due to some (now fixed) issues.

Thanks Julian for the detailed response.

I would like to add that while I clearly see the risks.

But the benefit of having the API availalbe in lucid now for the next lucid->lts+1 upgrade outweigh them IMO.
Otherwise for lucid->lts+1 we will have to write the upgrader code against a different API for next-lts-1->next-lts
and lucid->lucid+1. Or maintain the deprecated API for 5 more years.

Do you have the merge prepared in a PPA, so that we can test it with update-notifier, apport, jockey, etc.?

Note that I renamed 0.7.93.4 to 0.7.94, as it splits the documentation into its own package now and I wanted to increase the version number and not stay at 0.7.93.X.

I put it into a PPA now, you can test it with:

$ sudo add-apt-repository ppa:mvo/python-apt

I performed a full system upgrade (via update-manager -d) with the python-apt from the PPA and the compat code worked. gdebi worked in my tests as well. We will have to silence the deprecation warnings (maybe using a environment so that people who port stuff can still see them?) for lucid-final.

Thanks for the PPA. I ran the apport and jockey tests suites against that, and they mostly succeed.

I get a large bunch of deprecation warnings, though (you mentioned that already, if we can silence them for the release that's fine),

Also, this behaviour is new:

>>> import apt
>>> c=apt.Cache()
Reading package lists... Done
Building dependency tree
Reading state information... Done
Building data structures... Done

With the current lucid version this isn't so chatty, but doesn't output anything. This now causes a lot of screen garbage on stdout (and incidentally also breaking the jockey tests because they don't expect so much output). Is that really intended? Can it be silenced somehow with an argument to Cache()?

Finally I tested the new version with software-center (although I'm sure that you have done this countless times already).

Otherwise I feel much better about this after those tests (thanks for doing dist-upgrade and gdebi), so this has my blessing now. Please go ahead, approved.

> >>> import apt
> >>> c=apt.Cache()
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> Building data structures... Done
>
> With the current lucid version this isn't so chatty, but doesn't output
> anything. This now causes a lot of screen garbage on stdout (and
> incidentally also breaking the jockey tests because they don't expect
> so much output). Is that really intended? Can it be silenced somehow
> with an argument to Cache()?

pass apt.progress.base.OpProgress() as the first argument. But IIRC, the behavior should be the same as previously, maybe just on a different output stream

Julian Andres Klode [2010-03-18 18:47 -0000]:
> pass apt.progress.base.OpProgress() as the first argument.

Ah, that works. Thanks! How much backwards compatible is that?

> But IIRC, the behavior should be the same as previously, maybe just
> on a different output stream

In our current version (0.7.13.4) it doesn't go to stdout or stderr.

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

> Ah, that works. Thanks! How much backwards compatible is that?

To answer my own question, it at least doesnt' work in Hardy. For apport I'll add a try/except thing to try both. But I really don't think that a library should output anything to stdout by default..

Adding apport task to silence apt.Cache() spewage (a lot of Apport programs are CLI).

apport trunk r1735

I just talked to Martin and this is approved if:
 a) deprecation warnings silent (at least for lucid-final)
 b) apt.Cache() not verbose (just like in ubuntu before)

I work on the fixes and target monday for a upload.

This bug was fixed in the package apport - 1.13.1-0ubuntu1

---------------
apport (1.13.1-0ubuntu1) lucid; urgency=low

  [ Martin Pitt ]
  * New upstream bug fix release:
    - Update parse-segv to handle gdb 7.1 output.
    - Enhance test suite to work with gdb 7.1 as well, and catch future
      outputs.
    - UI: Add exception string to the "network error" dialog, to better tell
      what the problem is.
    - UI: Add back -p option to apport-collect/apport-update-bug (regression
      from 1.13). (LP: #538944)
    - launchpad.py: Add yet another workaround for LP#336866. (LP: #516381)
    - launchpad.py, download(): Ignore attachments with invalid key names.
    - Fix regression from 1.10 which made it impossible for a package hook to
      set a third-party crash database for non-native packages. (LP: #517272)
    - apport-cli: Create the 'details' string only if user wants to view
      details, and do not show files larger than 1MB. Thanks Scott Moser!
      (LP: #486122)
    - packaging-apt-dpkg.py: Silence apt.Cache() spewage to stdout with newer
      python-apt versions. (LP: #531518)
    - unkillable_shutdown: Add list of running processes and blacklisted pids
      to report. (LP: #537262)
    - Sort the report by key in the details view. (LP: #519416)

  [ Evan Dandrea ]
  * Move ubiquity's package-hook into apport, so that it can be used
    from the installed system to grab the logs in /var/log/installer.
 -- Martin Pitt <email address hidden> Sat, 20 Mar 2010 22:28:44 +0100

Both is fixed in bzr now, I'm doing some final tests and then I will upload. Many thanks.

This bug was fixed in the package python-apt - 0.7.94.2ubuntu1

---------------
python-apt (0.7.94.2ubuntu1) lucid; urgency=low

  Updated to the 0.7.9x series (FFe LP: #531518), this
  brings us python3 support and a more PEP08 conform
  API

  [ Julian Andres Klode ]
  * python/generic.cc:
    - Fix a memory leak when using old attribute names.
  * debian/control:
    - Change priority to standard, keep -doc and -dev on optional.

  [ Michael Vogt ]
  * apt/cache.py:
    - make cache open silent by default (use apt.progress.base.OpProgress)
  * tests/data/aptsources_ports/sources.list:
    - fix ports test-data
  * debian/control
    - build against XS-Python-Versions: 2.6, 3.1
  * tests/test_apt_cache.py:
    - add simple test for basic cache/dependency iteration
  * apt/__init__.py:
    - only show deprecation warnings if PYTHON_APT_DEPRECATION_WARNINGS
      is set in the environment. While we do want to have the new API its
      not feasible to port all apps in the lucid timeframe. Once lucid
      is released we turn the warnings on by default again
 -- Michael Vogt <email address hidden> Tue, 23 Mar 2010 20:01:22 +0100