Ok, I've been able to reproduce the problem now. The issue is that you were
probably using sensible default options, and therefore ubuntu-image was
doing all the building in /tmp; whereas I've been futzing with ubuntu-image
internals and was therefore doing all of my builds in a named working
directory; and the final stage of the build is to move the image from the
working directory to its output location with shutil.move(), which falls
back to shutil.copy2() for a cross-filesystem move (like if /tmp is a
tmpfs), and this function is not sparse-aware.
So if you're doing all the building in your current directory, you get a
sparse file that is renamed and there's no problem. If you're doing it
under /tmp, you get a sparse file that is then copied by a non-sparse-aware
python function.
I believe this may also explain the report of bug #1619351, and believe that
also means that switching to os.truncate() won't fix the problem.
> the actual content of the image are a 85MB pi2-kernel snap
> and a 56MB ubuntu-core snap plus whatever overhead the image creation adds
> (pre-created writable bits, unpacked gadget, copied kernel and initrd
> files etc), nearly 300MB sounds actually pretty large for that content.
snap prepare-image stores two copies of the snaps, one in
/var/lib/snapd/seed/snaps and one in /var/lib/snapd/snaps. So it doesn't
look like filesystem overhead is the cause of the increase here.
On Thu, Sep 01, 2016 at 06:34:51PM -0000, Oliver Grawert wrote: images/ snappy/ u-image- pi2.img images/ snappy/ u-image- pi2.img images/ snappy/ u-image- pi2.img images/ snappy/ u-image- pi2.img
> ogra@anubis:~$ ls -l datengrab/
> -rw-r--r-- 1 root root 4000000000 Sep 1 17:30 datengrab/
> ...
> ogra@anubis:~$ du -sh datengrab/
> 3,8G datengrab/
> this image was created using people. canonical. com/~ogra/ snappy/ pi2-model. assertion with people. canonical. com/~ogra/ snappy/ ubuntu- image_0. 5_amd64. snap
> http://
> http://
> (snapcraft build from a PR from mvo with some extra fixes to master) ...
Ok, I've been able to reproduce the problem now. The issue is that you were
probably using sensible default options, and therefore ubuntu-image was
doing all the building in /tmp; whereas I've been futzing with ubuntu-image
internals and was therefore doing all of my builds in a named working
directory; and the final stage of the build is to move the image from the
working directory to its output location with shutil.move(), which falls
back to shutil.copy2() for a cross-filesystem move (like if /tmp is a
tmpfs), and this function is not sparse-aware.
https:/ /bugs.python. org/issue10016
So if you're doing all the building in your current directory, you get a
sparse file that is renamed and there's no problem. If you're doing it
under /tmp, you get a sparse file that is then copied by a non-sparse-aware
python function.
I believe this may also explain the report of bug #1619351, and believe that
also means that switching to os.truncate() won't fix the problem.
> the actual content of the image are a 85MB pi2-kernel snap
> and a 56MB ubuntu-core snap plus whatever overhead the image creation adds
> (pre-created writable bits, unpacked gadget, copied kernel and initrd
> files etc), nearly 300MB sounds actually pretty large for that content.
snap prepare-image stores two copies of the snaps, one in snapd/seed/ snaps and one in /var/lib/ snapd/snaps. So it doesn't
/var/lib/
look like filesystem overhead is the cause of the increase here.