filesystem.size is 0 for Mint 19.1 iso
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Cubic |
Fix Released
|
High
|
Cubic PPA |
Bug Description
I've reproduced this after multiple attempts from fresh Mint 19.1 iso's.
For some reason, Cubic doesn't calculate the filesystem size properly and sets the contents of the "/casper/
This can also be seen in the final Cubic summary window, the "Update the customized Linux file system size" task finishes with a result of "The file system size is 0.00 GiB (0 bytes)."
This does not happen with a Mint 18.3 iso.
For both Mint 18.3 and Mint 19.1 I am just running system updates and installing a few packages, nothing special.
This is kind of a serious issue because it causes the Ubiquity installer to crash with a division by zero error.
I scanned the python source of the installer and found that an easy fix is to just delete the "/casper/
At the very least, I think Cubic should just not create the "/cubic/
Please let me know if there's any more information you need from me.
description: | updated |
Cubic PPA (cubic-wizard) wrote : | #1 |
Changed in cubic: | |
assignee: | nobody → Cubic PPA (cubic-wizard) |
PJSingh5000 (pjsingh5000) wrote : | #2 |
What is the ouptput of...?
$ du --summarize --one-file-system --block-size=1 "/home/
Where "/home/
PJSingh5000 (pjsingh5000) wrote : | #3 |
I conducted a few tests.
ISO Customized: linuxmint-
Host System: Ubuntu 18.10, x64
Cubic version: 2019.04-
In each test, I could not reproduce this issue.
Would you please share some more details?
- What is the OS and version you are running Cubic in?
- What version of Cubic are you using?
- Are any of your files on a remote machine, or everything local?
Also, if you run cubic from the commandl-ine, please let me know if you see this message in the output, at the same Cubic prints zero file size in the GUI...
'Unable to get filesystem size for "/home/
where Where "/home/
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
TESTS #1
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1. Used linuxmint-
2. Made no changes in chroot.
3. Proceeded to the repackage page. Here are the results...
Update filesystem size
Execute synchronously.
Set a new process for thread id........ 140048980399872
The new process id is.....
Write filesystem size to............... /home/test01/
The file system size is................ 5.48 GiB (5881049088 bytes)
The contents of .../custom-
5881049088
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
TEST #2
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1. Used linuxmint-
2. Made the following changes in chroot...
$ apt update
$ apt install tree rar unrar p7zip-full gtkhash gnome-tweaks lm-sensors smartmontools gnome-sushi xdotool python-apport
# Need to get 11.0 MB of archives.
3. Proceeded to the repackage page. Here are the results...
Update filesystem size
Execute synchronously.
Set a new process for thread id........ 139705203021568
The new process id is.....
Write filesystem size to............... /home/test01/
The file system size is................ 5.43 GiB (5828341760 bytes)
The contents of .../custom-
5828341760
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
TEST #3
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1. Opened the existing project from Test #1 in Cubic.
2. This time, made the following changes in chroot...
$ apt update
$ apt install tree rar unrar p7zip-full gtkhash gnome-tweaks lm-sensors smartmontools gnome-sushi xdotool python-ap...
PJSingh5000 (pjsingh5000) wrote : | #4 |
- Screenshot of Cubic's Repackage page, showing filesystem size from Test #2 Edit (65.7 KiB, image/png)
Attached a screenshot of Cubic's Repackage page, showing filesystem size from Test #2
Pico (picomitchell) wrote : | #5 |
I'll be back and the computer I was doing this on on Monday to test more. But in the meantime, I am running Mint 19.1 Cinnamon and using a the Mint 19.1 Cinnamon iso. Everything is local, nothing remote.
I'll get back to you on Monday with Cubic version and output of that command.
Pico (picomitchell) wrote : | #6 |
Was using Cubic 2019.03-51 but I've just updated to 2019.04-53 so I'll try things again with the latest version.
Running the du --summarize --one-file-system --block-size=1 "/home/
I'll do some more testing and let you know what I find.
Pico (picomitchell) wrote : | #7 |
- Screenshot from 2019-04-22 11-20-23.png Edit (69.7 KiB, image/png)
Same thing happened with the latest version of Cubic. Attached a screenshot.
Gonna start over with a new iso and see if the same thing happens.
Pico (picomitchell) wrote : | #8 |
I was just able to narrow the issue down to having something to do with running updates in Cubic.
I ran "mintupdate-cli upgrade -l12345 -r -y" once (which pretty much just updates mintupdate itself) and then built the ISO and Cubic calculated the correct filesystem size.
Then, I went back into chroot and ran "mintupdate-cli upgrade -l12345 -r -y" again which actually does all the updates. After that was done I exited chroot and built the ISO. This time Cubic incorrectly calculated a filesystem size of 0.
I'll start over and try doing level 1 updates and then 2 and then 3 etc and see if it's easy to narrow down to some group of updates.
Pico (picomitchell) wrote : | #9 |
I just found that the filesystem size was calculated correctly after only running level 1 updates but was incorrectly calculated to 0 after running level 2 updates.
Pico (picomitchell) wrote : | #10 |
Also got filesystem size of 0 after creating a new image and only running level 4 updates.
description: | updated |
PJSingh5000 (pjsingh5000) wrote : | #11 |
What does Level "X" update refer to? Is it just the number of times you do an update?
Is `mintupdate-cli` like `apt` and what is the significance of the arguments you are passing to it.
I'll see if I can install Mint in a virtual machine and recreate your steps.
Is `mintupdate-cli` included with the Mint Live CD, or do I have to install it from a different source (PPA, Git, etc)?
PJSingh5000 (pjsingh5000) wrote : | #12 |
Are you running Cinnamon or MATE ?
Pico (picomitchell) wrote : | #13 |
mintupdate-cli is built-in to Mint 19.1 (it was previously called mintupdate-tool on Mint 18.3).
It is simply the command line interface to Mint's built-in Update Manager. Basically a wrapper around apt, as I understand it. But, the levels allow you to only update groups of packages instead of all or nothing.
In Mint's Update Manager, updates are grouped into Levels roughly having to do with their likelihood of breaking things. Here's an overview of the levels: https:/
Pico (picomitchell) wrote : | #14 |
Using Cinnamon
Cubic PPA (cubic-wizard) wrote : | #15 |
Just an FYI, probably not relevant to this issue...
I noticed that levels should be a comma separated list...
-l ONLY_LEVELS, --only-levels ONLY_LEVELS
Pico (picomitchell) wrote : | #16 |
Thanks for the tip. Looks like I was still using the old style from mintupdate-tool. Seems to still work though.
Interestingly, I wanted to investigate a bit more and found the latest source (https:/
Cubic PPA (cubic-wizard) wrote : | #17 |
(1)
How much free disk space do you ave on your system partition where the Cubic project is?
(You can use the df -h command).
(2)
When you run Cubic from the command line, you're sure you didn't get an "Unable to get filesystem size..." message just after Cubic finishes the "Compress customized Linux..." step?
Pico (picomitchell) wrote : | #18 |
I got no messages other than what you see in the previous screenshot I attached.
I think about 40gb free on that machine, but I'm away from it right now.
Pico (picomitchell) wrote : | #19 |
Ah, actually I haven't tried running cubic from the command line yet, just the GUI.
I'll do some testing with command line tomorrow.
Cubic PPA (cubic-wizard) wrote : | #20 |
Ok.
Whenever you get to test some more, please share the output of running the command `cubic. You can paste the entire output as a comment here.
Cubic PPA (cubic-wizard) wrote : | #21 |
I ran a few tests, but did not run into this issue, so this is ~very~ perplexing.
- I installed linuxmint-
- I installed Cubic.
- I copied and customized linuxmint-
I ran four tests, and here are the results...
TEST 1
======
TEST 1 - CHANGES IN CHROOT
-------
None
TEST 1 - RESULTS
----------------
Update filesystem size
Execute synchronously.
Set a new process for thread id........ 139775447615232
The new process id is.....
Write filesystem size to............... /home/test/
The file system size is................ 5.26 GiB (5650616320 bytes)
TEST 1 - VERIFY SIZE
-------
test@TEST001:~$ sudo du --summarize --one-file-system --block-size=1 "/home/
5650616320 /home/test/
TEST 2
======
TEST 2 - CHANGES IN CHROOT
-------
None
TEST 2 - RESULTS
----------------
Update filesystem size
Execute synchronously.
Set a new process for thread id........ 140531897476864
The new process id is.....
Write filesystem size to............... /home/test/
The file system size is................ 5.26 GiB (5650616320 bytes)
TEST 2 - VERIFY SIZE
-------
$ sudo du --summarize --one-file-system --block-size=1 "/home/
5650616320 /home/test/
TEST 3
======
TEST 3 - CHANGES IN CHROOT
-------
# mintupdate-cli -l1,2,3,4,5 -r -y upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
mint-upgrade-info mintupdate
2 upgraded, 0 newly installed, 0 to remove and 371 not upgraded.
Need to get 909 kB of archives.
After this operation, 1,101 kB disk space will be freed.
Get:1 http://
Get:2 http://
Fetched 909 kB in 0s (3,024 kB/s)
(Reading database ... 248053 files and directories currently installed.)
Preparing to unpack .../mint-
Unpacking mint-upgrade-info (1.1.2) over (1.1.1) ...
Preparing to unpack .../mintupdate_
Unpacking mintupdate (5.4.7) over (5.4.6) ...
Setting up mint-upgrade-info (1.1.2) ...
Setting up mintupdate (5.4.7) ...
Processing triggers for ...
Pico (picomitchell) wrote : | #22 |
Thanks for doing all this testing.
Very strange that you weren't able to reproduce it. I've seen it happen over and over again with multiple test projects.
I'm very curious to run the tests with Cubic's command line tomorrow to see if I can give some helpful feedback.
Is it possible the bug only happens with the GUI and not in command line?
Pico (picomitchell) wrote : | #23 |
FYI, if you didn't see my comment on this thread already: https:/
I think those people were running into this same issue.
The Ubiquity crash they are reporting is exactly what I originally ran into.
I was only able to trace it back to the filesystem.size file after checking the install.py source.
Before that, I didn't even notice that Cubic listed a size of 0.0 GB in it's GUI because I didn't know to look out for it.
Cubic PPA (cubic-wizard) wrote : | #24 |
Pico,
In comment #6 (https:/
Would you also please share the exact output of that command, including any text that is printed along with the integer?
(I am asking this, because this is the exact command Cubic uses to get the file system size).
Also, try running this command after you experience the "Size 0" issue. It looks like Cubic is able to get the size successfully, for a few runs, and then it just fails. It's possible something is different with the output of this command whenever the issue occurs.
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
$ sudo du --summarize --one-file-system --block-size=1 "/home/
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Where "/home/
Thanks for running these extra tests, especially since I am not able to recreate the problem on my end.
Pico (picomitchell) wrote : | #25 |
Here's the relevant part of the file size calculation when running cubic via the command line. Sorry I didn't run this test earlier on, looks like there's some info for actual investigation now.
Update filesystem size
Execute synchronously.
Set a new process for thread id........ 139780281542400
The new process id is.....
Exception while executing.
The tracekback is.....
Pico (picomitchell) wrote : | #26 |
When I run the du command I get no errors or anything. The only output is the size in bytes and the path.
But, since it looks like maybe it's a timeout issue, I timed the du command. Here's the output:
time sudo du --summarize --one-file-system --block-size=1 '/home/
8664993792 /home/pico/
real 1m17.964s
user 0m0.670s
sys 0m13.322s
I'm on running on a 1.9 ghz 4th Gen i5 (2 cores with hyper-threading) with 8 GB of RAM and a 128 GB SSD. Not a powerhouse by any means, just a dedicated linux laptop to do testing and this sort of preparation for network installations at work (we're a non-profit that refurbishes computers). As a side note, being able to use Cubic to make it easy to pre-install all updates has saved us roughly 25 minutes per installation! So, regardless of this bug, we all really appreciate Cubic!
Pico (picomitchell) wrote : | #27 |
I'm not too familiar with Python but I was curious and wanted to see if I could figure out what was going on. I found that the "du" command is running through your "execute_
Then I found https:/
That page said the that the default timeout for pexpect is 30 seconds, so it makes sense that it's timing out on my machine.
I see you're using the pexecpt "read()" command so I searched that on that page to see how to change the timeout for that and found that you could do "process.
Or, it might be safer to have some longer timeout such as 5 minutes with "process.
Cubic PPA (cubic-wizard) wrote : | #28 |
This is an interesting thought...
The appearance of this issue does seem to correlate with the size of your customized file system-- the larger the file system, the longer it takes to get its size.
...I'll look into this.
Cubic PPA (cubic-wizard) wrote : | #29 |
Thanks for the Cubic output, and the detailed research.
This is indeed what is happening!
Now we know the issue, we can fix it! :)
Pico (picomitchell) wrote : | #30 |
Glad I could help!
Thank you so much for your active testing and responsiveness with this!
If really means a lot to know that you are here in case anything gets weird. Again, having Cubic has made a complex process super simple, and the end result is a massive time savings to our refurbishment process. Thank you thank you!
Looking forward to the update :-)
Pico (picomitchell) wrote : | #31 |
This is interesting... I was reading on this thread https:/
The laptop I'm on was originally installed in BIOS only mode so I wanted to do a reinstall with EFI enabled so I could make EFI compatible isos.
I finally just did that today and recreated a new Cubic project and did all the same system updates and the few custom packages.
When I finished the project and built the iso, I was expecting to see the same filesystem.size bug. But the size was calculated correctly! But also, it was quite a bit smaller than the old project. About 6 GB instead of 8 GB.
I timed the du command again and if finished in 6 seconds:
time sudo du --summarize --one-file-system --block-size=1 '/home/
[sudo] password for pico:
6839742464 /home/pico/
real 0m8.550s
user 0m0.327s
sys 0m2.024s
That is a stark difference compared to the nearly minute and a half on my old install.
I also installed a larger 256 GB drive, up from my previous 128 GB. The drive may be a bit faster but that seems like too much of an improvement just for SSD speed difference.
Do you understand what is going on here?
Also, why is the new EFI compatible iso so much smaller? Will the iso still be compatible with BIOS?
Pico (picomitchell) wrote : | #32 |
Clearly, I made a typo with 6 seconds, the correct times are listed in the command output. Still insanely faster than when run in my previous installation...
PJSingh5000 (pjsingh5000) wrote : | #33 |
How did you create the EFI compatible ISO? Did you run Cubic on a machine with EFI?
Pico (picomitchell) wrote : | #34 |
Yes, I reinstalled Linux Mint after enabling EFI mode.
PJSingh5000 (pjsingh5000) wrote : | #35 |
Regarding the size discrepancy...
1. Cubic uses gizp compression which may be better than the compression used to create the original ISO. (In the past, Cubic had used the xz compression algorithm, which produced even smaller ISOs, but it was replaced with gzip because it is less CPU intensive compared to xz).
2. Cubic will not copy any files in /tmp directory to the remastered ISO. If the original ISO had some "junk" in this folder, that may account for some of the difference.
Pico (picomitchell) wrote : | #36 |
Hmmmm, maybe it was the /tmp stuff. Have to boot into the old drive and double check that.
So that means the size that Cubic calculates is not necessarily the actual size that ends up in the iso?
Changed in cubic: | |
importance: | Undecided → High |
status: | New → In Progress |
PJSingh5000 (pjsingh5000) wrote : | #37 |
Would you please test the fix for this issue in the *Development* branch, and let me know if it resolves your problem?
# Remove Cubic
$ sudo apt autoremove --purge cubic
# Remove the *Release* repository
$ sudo apt-add-repository --remove ppa:cubic-
# Add the *Development* repository
$ sudo add-apt-repository ppa:cubic-
# Install the *Development* version of Cubic
$ sudo apt update
$ sudo apt install cubic
PJSingh5000 (pjsingh5000) wrote : | #38 |
Would you please test the fix for this issue in the *Development* branch, and let me know if it resolves your problem?
# Remove Cubic
$ sudo apt autoremove --purge cubic
# Remove the *Release* repository
$ sudo apt-add-repository --remove ppa:cubic-
# Add the *Development* repository
$ sudo add-apt-repository ppa:cubic-
# Install the *Development* version of Cubic
$ sudo apt update
$ sudo apt install cubic
Pico (picomitchell) wrote : | #39 |
FIXED!
Update filesystem size
Execute synchronously.
Set a new process for thread id........ 140616914622208
The new process id is.....
Write filesystem size to............... /home/pico/
The file system size is................ 8.32 GiB (8936144896 bytes)
Pico (picomitchell) wrote : | #40 |
PS. I booted back into my old SSD that had the issue to confirm that. It noticeably took a while to calculate that size.
I'm still investigating the strange size difference between the two squashfs-root folders on my old SSD vs new SSD. I've confirmed that it's not /tmp by using the Disk Usage Analyzer app that's built into Mint. It's like various system folders are larger on the old SSD.
One noticeable discrepancy is that "/squashfs-
Another big discrepancy is that "/squashfs-
Overall, there just looks to be lots of little sections that are slightly larger on the old SSD vs the new SSD>
Have you seen this kind of discrepancy before? Do you think some valuable resources are missing from the sqaushfs-root on the new SSD? Could this have something to do with the OS being installed as BIOS vs EFI?
I'm concerned because I want to know the iso that will be used to image countless machines is proper and complete. Should I start up a new thread to investigate this if the answer isn't obvious to you?
Cubic PPA (cubic-wizard) wrote : | #41 |
Thanks for testing this bug fix.
I will close this bug report.
- - - - - - - - - - - - - - - - - - - -
Please open a question on the Answers section (https:/
I did check what compression algorithm is used by the official "linuxmint-
I have always noticed size differences between the original ISO and a remastered ISO with **NO** changes made in chroot. The Cubic generated ISO is always smaller, and I always attributed that to differences in compression (Cubic used to use xz, which is better than gzip).
I have also never experienced issues. (No broken applications, missing functionality, or missing files).
Can you do a diff between two directories where you see the differences. Are any files missing?
Pico (picomitchell) wrote : | #42 |
I'm still quite confused by the size discrepancies of the different squash-fs folders, but I've just done some test installations using the iso made on the OLD SSD vs the iso made on the NEW SSD vs the original Linux Mint iso, and I'm happy to say that the size dependencies between the installations were negligible.
Cubic PPA (cubic-wizard) wrote : | #43 |
Fix is now availabe in *Release* branch, revisoon 54.
If you have the Release PPA enabled...
$ sudo apt update
$ sudo apt install cubic
If you have the Development PPA enabled...
# Remove Cubic
$ sudo apt autoremove --purge cubic
# Remove the *Development* repository
$ sudo apt-add-repository --remove ppa:cubic-
# Add the *Release* repository
$ sudo add-apt-repository ppa:cubic-
# Install the *Release* version of Cubic
$ sudo apt update
$ sudo apt install cubic
Changed in cubic: | |
status: | In Progress → Fix Released |
I appreciate that you reported this issue, even though you had found a work-around. filesystem. size should not have value of 0.
You are right, /casper/
I'll look into this.