The 'http' definition in /etc/apt/apt-file.conf says this:
http = wget -N -P "<cache>" -O "<cache>/<dest>" "<uri>/dists/<dist>/Contents-<arch>.gz" || rm -f "<cache>/<dest>";
The -N is supposed to turn on timestamping, so that the file is only downloaded if it has changed on the server. However, -N doesn't work if -O is also specified, and so the whole download is down each time.
It would be better to leave out the -O flag so that timestamping can work.
The problem is that the local file and the file on the server have different names. To get timestamping to work, they have to have the same name. We can rename the local file before running wget and rename it back after, like this:
http = (if [ -f "<cache>/<dest>" ]; then mv "<cache>/<dest>" "<cache>/Contents-<arch>.gz"; fi; wget -N -P "<cache>" "<uri>/dists/<dist>/Contents-<arch>.gz" 2>&1 && mv "<cache>/Contents-<arch>.gz" "<cache>/<dest>" || rm -f "<cache>/<dest>") | grep -v -e "^--" -e "=>" -e "^Length:" -e "^HTTP" -e "^Resolving" -e "^Connecting" -e "ERROR 404" 1>&2
That line also filters out a lot of the output from wget, but leaves the progress display in place. The current version shows this:
--20:10:04-- http://archive.ubuntu.com/ubuntu/dists/breezy-security/Contents-i386.gz
=> `/var/cache/apt/archive.ubuntu.com_ubuntu_dists_breezy-security_Contents-i386.gz'
Resolving archive.ubuntu.com... 82.211.81.151, 82.211.81.182
Connecting to archive.ubuntu.com|82.211.81.151|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
20:10:05 ERROR 404: Not Found.
for files which don't exist on the server. My version above shows nothing for the missing files, the same as the curl line, and shows this:
Server file no newer than local file `/var/cache/apt/Contents-i386.gz' -- not retrieving.
as confirmation that the server file exists but hasn't changed.
Note that the "ftp =" line also tries to combine the -N and -O options, and so probably fails in the same way.