Merge lp:~javier.collado/ubuntu-test-cases/bug1087630 into lp:ubuntu-test-cases/server

[Javier Collado]

The branch make some changes to the minimal-virtual scenario to make the
install_size test case work.

In particular:
- late command in the preseed is used to gather disk usage before testing
  packages (needed by utah) are installed.
- test case script reads the log file with that information instead of running
  `df`.

The test case is working now for me when running the test case locally in a VM
with the new preseed.

However, I believe that still the usage in the log file includes the packages
in `pkgsel/include` (`preseed/late_command` seems to be executed after
`pkgsel/include`), but not the ones that are installed later using `gdebi`, so
the figure isn't completely precise yet.

[Para Siva (psivaa)]

Not sure if it's really important but do we not need to change the template preseed file, http://bazaar.launchpad.net/~javier.collado/ubuntu-test-cases/bug1087630/view/head:/preseeds/templates/minimal-virtual to include the late command just to be consistent?

review: Needs Information

[Javier Collado (javier.collado)]

Since this is something needed just in this test case (as far as I know) and it
has its own preseed already, I think it's fine to make the change only in the
preseed this test case preseed.

If we find that the same thing is needed in other test cases, then we can move
it to the base template later. Do you agree?

[lp:~javier.collado/ubuntu-test-cases/bug1087630]

81. By Javier Collado on 2013-01-08

Added late_command block to customize late_command in templates

This is needed if the scenario needs a different late_command configuration
such as the minimal virtual one.

82. By Javier Collado on 2013-01-08

Updated minimal virtual template to customize late_command

Generated late_command now gathers the root partition usage.

[Para Siva (psivaa)]

Looks good now. Thanks

review: Approve

[Javier Collado (javier.collado)]

Sorry I'm not familiar with the templates in this branch and I didn't know that
there's a template for minimal-virtual that should be used to generate
minimal-virtual.preseed even if both are under version control. Thanks Siva for
the explanation through irc.

I've added a couple of revisions to deal with that:
- make late_command customizable in the templates
- use that in the minimal-virtual template to gather the root partition usage

review: Needs Resubmitting

[Max Brustkern (nuclearbob)]

The [ "$?" -eq "0" ] isn't really necessary. That's in the default preseed so that a latecommand will exist, and that particular command was chosen to avoid masking the exit status of the utah latecommand if no other latecommand is present. If another latecommand is present, that doesn't really serve a purpose. That said, it doesn't hurt anything either.

Also, /var/log/utah-install is used for other things which can come before or after this use. It doesn't look to me like the current code will detect the correct line and not just the first line of the file, but I could be wrong about that.

review: Needs Information

[Max Brustkern (nuclearbob)]

To clarify, [ "$?" -eq "0" ] isn't necessary on merge lines like 9 and 36. It should remain on lines like 22 where it's the only thing.

[lp:~javier.collado/ubuntu-test-cases/bug1087630]

83. By Javier Collado on 2013-01-10

Removed unneeded command in preseed

84. By Javier Collado on 2013-01-10

Improved parsing

The previous code expected the first line in /var/log/utah-install to be the
one about the install size. This make parsing more solid just in case some
change in utah changes this assumption.

[Javier Collado (javier.collado)]

Max, I've updated the merge to address your concerns:

- the unneeded return code check in the late command has been removed

- the parsing now looks for the first line that starts with "Root partition
  usage:" to avoid problems due to unexpected ordering of lines in
  /var/log/utah-install

review: Needs Resubmitting

[Max Brustkern (nuclearbob)]

I like the changes.

review: Approve

=== modified file 'preseeds/minimal-virtual.preseed'
--- preseeds/minimal-virtual.preseed	2012-11-23 11:22:06 +0000
+++ preseeds/minimal-virtual.preseed	2013-01-10 16:53:25 +0000
@@ -327,7 +327,7 @@
 # directly, or use the apt-install and in-target commands to easily install
 # packages and run commands in the target system.
 #d-i preseed/late_command string echo "send host-name \"<hostname>\";" >> /target/etc/dhcp3/dhclient.conf;
-d-i preseed/late_command string [ "$?" -eq "0" ]
+d-i preseed/late_command string in-target sh -c 'echo "Root partition usage: $(df / | tail -n1 | awk '\''{print $3}'\'')">>/var/log/utah-install'
 
 
 

=== modified file 'preseeds/templates/base'
--- preseeds/templates/base	2012-11-23 11:22:06 +0000
+++ preseeds/templates/base	2013-01-10 16:53:25 +0000
@@ -307,7 +307,9 @@
 # directly, or use the apt-install and in-target commands to easily install
 # packages and run commands in the target system.
 #d-i preseed/late_command string echo "send host-name \"<hostname>\";" >> /target/etc/dhcp3/dhclient.conf;
+{% block late_command %}
 d-i preseed/late_command string [ "$?" -eq "0" ]
+{% endblock %}
 
 {% block preseed_extra %}
 {% endblock %}

=== modified file 'preseeds/templates/minimal-virtual'
--- preseeds/templates/minimal-virtual	2012-08-29 15:47:42 +0000
+++ preseeds/templates/minimal-virtual	2013-01-10 16:53:25 +0000
@@ -28,4 +28,7 @@
 {% block d_i_answers %}
 d-i base-installer/kernel/headers   boolean false
 {% endblock %}
+{% block late_command %}
+d-i preseed/late_command string in-target sh -c 'echo "Root partition usage: $(df / | tail -n1 | awk '\''{print $3}'\'')">>/var/log/utah-install'
+{% endblock %}
 

=== modified file 'testsuites/minimal-virtual/test_install_size/tc_control'
--- testsuites/minimal-virtual/test_install_size/tc_control	2012-11-14 10:56:53 +0000
+++ testsuites/minimal-virtual/test_install_size/tc_control	2013-01-10 16:53:25 +0000
@@ -1,7 +1,7 @@
 description: A test to check installed size of a vm
 dependencies: n/a
 action: |
-  1. Check the used space is less than the maximum instal size
+  1. Check the used space is less than the maximum install size
 expected_results: |
   1. The used space is less than the max install size
 type: userland

=== modified file 'testsuites/minimal-virtual/test_install_size/test.py'
--- testsuites/minimal-virtual/test_install_size/test.py	2012-11-30 09:03:46 +0000
+++ testsuites/minimal-virtual/test_install_size/test.py	2013-01-10 16:53:25 +0000
@@ -1,51 +1,69 @@
 #!/usr/bin/python
-# 
+#
 # Copyright (C) 2010, Canonical Ltd (http://www.canonical.com/)
 #
 # This file is part of ubuntu-server-iso-testing.
-# 
-# ubuntu-server-iso-testing is free software: you can redistribute it 
-# and/or modify it under the terms of the GNU General Public License 
-# as published by the Free Software Foundation, either version 3 of 
+#
+# ubuntu-server-iso-testing is free software: you can redistribute it
+# and/or modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation, either version 3 of
 # the License, or (at your option) any later version.
-# 
-# ubuntu-server-iso-testing is distributed in the hope that it will 
-# be useful, but WITHOUT ANY WARRANTY; without even the implied warranty 
+#
+# ubuntu-server-iso-testing is distributed in the hope that it will
+# be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
 # of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 # GNU General Public License for more details.
-# 
+#
 # You should have received a copy of the GNU General Public License
-# along with ubuntu-server-iso-testing.  If not, see 
+# along with ubuntu-server-iso-testing.  If not, see
 # <http://www.gnu.org/licenses/>.
-# 
-
+#
+
+"""Check disk usage matches the expected one.
+
+Get root partition usage as stored by the late command in the preseed and
+compare it against the expected value.
+
+`df` isn't directly used in the test case to avoid getting the size of all the
+packages installed because of utah, but not really present in a default
+installation.
+
+"""
 
 import logging
-import os.path
-import subprocess
 import unittest
 import platform
 
-logging.basicConfig(level=logging.INFO)
 
 class MinimalVirtualTest(unittest.TestCase):
+    LOG_MESSAGE = "Root partition usage:"
 
     def testInstallSize(self):
-        cmd = ["df", "/"]
-        logging.debug("Cmd: %s" % (cmd))
-        output = subprocess.Popen(cmd, stdout=subprocess.PIPE).communicate()[0]
-        logging.debug("Cmd output: %s" % (output))
-        lines = output.split("\n")
-        self.assertEqual(len(lines), 3)
+        logging.info('Getting root partition usage '
+                     'as stored by late command...')
+        with open('/var/log/utah-install') as f:
+            line = next(line for line in f
+                        if line.startswith(self.LOG_MESSAGE))
+        used = int(line.split()[-1])
+        logging.info('{} {}'.format(self.LOG_MESSAGE, used))
 
-        used = int(lines[1].split()[2])
         # 700 minimal max install size + 150 test overlay
         max_install_size = 870400
+
         # Add overhead for multiarch package lists on amd64
         # +32MB roughly
         if platform.machine() == "x86_64":
             max_install_size += 32256
-        self.assertTrue( used < max_install_size, "Used: %s" % (used))
+
+        self.assertLess(
+            used, max_install_size,
+            "Usage ({}) is *not* less than the maximum expected value: {}"
+            .format(used, max_install_size))
+
+        logging.info("Usage ({}) is less than the maximum expected value: {}"
+                     .format(used, max_install_size))
 
 if __name__ == '__main__':
+    logging.basicConfig(level=logging.DEBUG,
+                        format='%(levelname)s: %(message)s')
     unittest.main()