merge-upstream does not repack zip archive

I seem to have fixed the problem with this patch

Thanks. Unfortunately using --repack will also repack .tar.bz2 files, which isn't correct for 3.0 (quilt) packages.

I suspect we could get gather_orig_files() to look for .zip files too.

Changing the prefix to %s-%s works, but I do not know whether one could assume that all zip archives are in the <package>-<version>.zip format...

On 02/17/2012 12:56 PM, Maarten Bezemer wrote:
> Changing the prefix to %s-%s works, but I do not know whether one could
> assume that all zip archives are in the<package>-<version>.zip
> format...
Yeah, that's a good point. I was hoping uscan would force the filename
for zip files too, but I guess that's not the case.

Another option would be to scan the uscan output for the filename,
perhaps with it XML output mode. That's more foolproof, but it's also a
fair bit more complex.

Cheers,

Jelmer

Ok, tried your idea and it seems to work nicely.

I have kept the original gather_original_files() call, in order to prevent breaking things. But I suppose it could be removed and use the provided uscan filename for all cases?

Is there anything left to in order to accept this patch? If so, please tell me and I'll try to fix/update the patch!

Sorry for not looking at it yet - I usually just watch merge proposals rather than bug reports.

It seems you make assumptions of the local name based on the name the file has remotely (upstream-url). This isn't necessarily correct, in some cases the file gets renamed from what it is upstream.

Can you also add a test for this new behaviour?

I cannot get the testing environment working properly, I have some problems with requiring a version of the bzr test library which is only available in precise. Which is not a problem except it will update half my system if I install it. Manually building was also not working nicely. So, it is going to take some more time until I figure this thing out and have the testing system up-and-running.

About the assumptions of the filename changes. I have looked at the uscan code, and in some (exotic) situations the filename is indeed changed (to prevent some illegal filenames). I cannot see a way of getting notified about a filename change. Should we look into adding this information for the XML output (ie modifying uscan), or let bzr-builddeb check all know filename possibilities (in a correct order)? The first is most robust, but the latter is 'easier' (as we do not have to patch uscan and get it accepted)...

If you're running oneiric you should be able to install a more recent bzr from the daily builds ppa:

https://launchpad.net/~bzr/+archive/daily

> About the assumptions of the filename changes. I have looked at the uscan code, and in some (exotic) situations the filename is
> indeed changed (to prevent some illegal filenames). I cannot see a way of getting notified about a filename change. Should we >
> look into adding this information for the XML output (ie modifying uscan), or let bzr-builddeb check all know filename possibilities > (in a correct order)? The first is most robust, but the latter is 'easier' (as we do not have to patch uscan and get it accepted)...
It looks like the <target> element can be used to find the filename that was created locally, so I don't think that would really require patching of uscan.

Another alternative would indeed be to create an empty directory somewhere and then assume that just make uscan put the file there. That seems fairly robust too, but is a bit more ugly than just looking at the --dehs output.

What do you think?

I'll try the daily builds!

I do not see a <target> element in the --dehs output. If have updated to the precise version of devscripts, but it still does not show the <target> element. It would be convenient to use.

Creating an empty directory and find the file that is downloaded into it, would also work. But, it seems a bit 'lame' to me... If there is not other (proper) solution, it is certainly usable (and quite robust for filename mangling).

On 03/14/2012 01:48 PM, Maarten Bezemer wrote:
> I'll try the daily builds!
>
> I do not see a<target> element in the --dehs output. If have updated to
> the precise version of devscripts, but it still does not show the
> <target> element. It would be convenient to use.
>
> Creating an empty directory and find the file that is downloaded into
> it, would also work. But, it seems a bit 'lame' to me... If there is not
> other (proper) solution, it is certainly usable (and quite robust for
> filename mangling).
>
charis:/tmp/s4-experimental% uscan --dehs --force-download
<dehs>
<package>samba4</package>
<debian-uversion>4.0.0~alpha18.dfsg1</debian-uversion>
<debian-mangled-uversion>4.0.0~alpha18</debian-mangled-uversion>
<upstream-version>4.0.0~alpha18</upstream-version>
<upstream-url>ftp://ftp.samba.org/pub/samba/samba4/samba-4.0.0alpha18.tar.gz</upstream-url>
<status>Newer version available</status>
<target>samba4_4.0.0~alpha18.orig.tar.gz</target>
<messages>Successfully downloaded updated package
samba-4.0.0alpha18.tar.gz and symlinked samba4_4.0.0~alpha18.orig.tar.gz
to it</messages>
</dehs>

Cheers,

Jelmer

This is my output
$ uscan --dehs --force-download
<dehs>
<package>mc</package>
<debian-uversion>4.8.1</debian-uversion>
<debian-mangled-uversion>4.8.1</debian-mangled-uversion>
<upstream-version>4.8.1</upstream-version>
<upstream-url>http://www.midnight-commander.org/downloads/mc-4.8.1.tar.bz2</upstream-url>
<status>up to date</status>
</dehs>

(the archive is downloaded by uscan)

No <target> element available... For both the oneiric and precise versions of devscripts. Do you use some (conveniently) modified uscan version? :)

Furthermore, I tried to update sambe4 (after getting 4.4.0~alpha17 or something similar) and uscan gave an error that the watch file was incorrect, using the --debug shows no downloaded contenet (page), using wget results in a download and a generated index.html of the ftp site.

Something seems to be different between our uscan versions (or libraries that are used).

I am all pro, using the <target> element, but this difference needs to be 'solved' to be sure that others also have the <target> element (maybe we need to add/update package dependencies when we know the difference?)

On Wed, Mar 14, 2012 at 01:47:24PM -0000, Maarten Bezemer wrote:
> This is my output
> $ uscan --dehs --force-download
> <dehs>
> <package>mc</package>
> <debian-uversion>4.8.1</debian-uversion>
> <debian-mangled-uversion>4.8.1</debian-mangled-uversion>
> <upstream-version>4.8.1</upstream-version>
> <upstream-url>http://www.midnight-commander.org/downloads/mc-4.8.1.tar.bz2</upstream-url>
> <status>up to date</status>
> </dehs>

> (the archive is downloaded by uscan)

> No <target> element available... For both the oneiric and precise
> versions of devscripts. Do you use some (conveniently) modified uscan
> version? :)
:-)

> Furthermore, I tried to update sambe4 (after getting 4.4.0~alpha17 or
> something similar) and uscan gave an error that the watch file was
> incorrect, using the --debug shows no downloaded contenet (page), using
> wget results in a download and a generated index.html of the ftp site.
This was with the Debian package, which is on alpha18. I don't recall
updating the watch file between alpha17 and 18, but perhaps I did.

> Something seems to be different between our uscan versions (or libraries
> that are used).

> I am all pro, using the <target> element, but this difference needs to
> be 'solved' to be sure that others also have the <target> element (maybe
> we need to add/update package dependencies when we know the difference?)
I'm running devscripts from precise, so I'm not entirely sure what's
going on here.

My suggestion would be to look at the uscan source code to see
why and if it spits out the target path it has downloaded a tarball
to. Of course, another good alternative is still to download into a
temporary directory and to use os.listdir(temp_dir) to find out what
the name of the downloaded tarball is. Not as nice as parsing the XML,
but also not the worst kludge ever :)

Cheers,

Jelmer