Merge lp:~mwhudson/ubuntu-test-cases/live-server-testing into lp:ubuntu-test-cases/live-server

[Michael Hudson-Doyle]

simple test of reusing existing partition

So this works if I run it by hand locally and so might work in automation too :)

There are a couple of things that need to be cleared up before we start running this for real though:

0) it depends on changes I made to utah yesterday, so utah needs to be updated on venonat.
1) the "script" reuse.sh that constructs the image needs to run as root. I don't know if the scripts are run as root by default or if we need to do something different or if it's just a matter of putting "sudo sh -c" in the right place.
2) the image that it creates will (I think) end up in the workspace and so waste heaps of disk. This could be fixed by agreeing a location in the workspace that run-iso-test.sh in qa-jenkins-jobs should clear out after utah has run.

[Paride Legovini (paride)]

Hi Michael,

0) Done, I merged your changes, the packages have been rebuilt and I updated utah on venonat.

1) The jobs do not normally run as root, but sudo can be used when needed.

2) The best way to handle this is via the "Workspace Cleanup" Jenkins plugin, which is currently not installed in the platform-qa-jenkins, but we can request it. In the meantime I don't think this is a blocker: venonat has quite a lot of disk space, if we keep an eye on it I think we can manage without the cleanup for the moment. Workspace are re-used in each job run, so the number of images won't grow indefinitely.

In general the test looks good. A couple of ideas/questions:

0) reuse.sh creates an image which is identical every time. We could avoid running commands as root by shipping root.img with the test, or making downloading it from somewhere. The image file looks 10G big, but du(1) reveals the actual disk usage is of just 17MB, and the file is 10MB gzipped.

1) Can we call the test 'reuse-existing-partitions' instead of 'reuse'? This whole thing is already convoluted enough :-)

Paride

[Michael Hudson-Doyle (mwhudson)]

On Fri, 12 Jul 2019 at 22:42, Paride Legovini <email address hidden>
wrote:

> Hi Michael,
>
> 0) Done, I merged your changes, the packages have been rebuilt and I
> updated utah on venonat.
>

Thanks!

> 1) The jobs do not normally run as root, but sudo can be used when needed.
>

OK, I'll shuffle that around.

> 2) The best way to handle this is via the "Workspace Cleanup" Jenkins
> plugin, which is currently not installed in the platform-qa-jenkins, but we
> can request it. In the meantime I don't think this is a blocker: venonat
> has quite a lot of disk space, if we keep an eye on it I think we can
> manage without the cleanup for the moment. Workspace are re-used in each
> job run, so the number of images won't grow indefinitely.
>

Oh if workspaces are per-project not per-run then that's not so bad.

> In general the test looks good. A couple of ideas/questions:
>
> 0) reuse.sh creates an image which is identical every time. We could avoid
> running commands as root by shipping root.img with the test, or making
> downloading it from somewhere. The image file looks 10G big, but du(1)
> reveals the actual disk usage is of just 17MB, and the file is 10MB gzipped.
>

We could but ... hm. 10MB seems a bit big to just check into the branch
(there will be more of these I hope!) and there already enough moving parts
here without having to worry about hosting the files somewhere else and
keeping them in sync with test expectations. So I think I'd prefer to keep
on doing things this way. You can argue me around if you feel strongly
enough about this though :)

> 1) Can we call the test 'reuse-existing-partitions' instead of 'reuse'?
> This whole thing is already convoluted enough :-)
>

Yeah that's definitely a good idea! I've pushed a rename to
reuse-existing-partitions-simple.

Cheers,
mwh

[Michael Hudson-Doyle (mwhudson)]

Is there any reason we can't merge this now?

[lp:~mwhudson/ubuntu-test-cases/live-server-testing]

13. By Michael Hudson-Doyle on 2019-07-18

merge simple test of reusing partitions

=== added file 'preseeds/reuse-existing-partitions-simple.preseed'
--- preseeds/reuse-existing-partitions-simple.preseed	1970-01-01 00:00:00 +0000
+++ preseeds/reuse-existing-partitions-simple.preseed	2019-07-18 04:22:28 +0000
@@ -0,0 +1,41 @@
+Welcome:
+  lang: en_US
+Refresh:
+  update: no
+Keyboard:
+  layout: us
+Installpath:
+  path: ubuntu
+Network:
+  accept-default: yes
+Proxy:
+  proxy: ""
+Mirror:
+  mirror: "http://us.archive.ubuntu.com/ubuntu"
+Filesystem:
+  manual:
+    - obj: [disk index 0]
+      action: MAKE_BOOT
+    - # Mount second partition at /srv without reformatting
+      obj: [disk index 0, part 1]
+      action: EDIT
+      data:
+        fstype: null
+        mount: /srv
+    - # Reformat third partition and use it as /
+      obj: [disk index 0, part 2]
+      action: EDIT
+      data:
+        fstype: ext4
+        mount: /
+    - action: done
+# Utah fills out the Identity section by itself
+SnapList:
+  snaps:
+    hello:
+      channel: stable
+      is_classic: false
+InstallProgress:
+  reboot: yes
+
+

=== added file 'runlists/reuse-existing-partitions-simple.run'
--- runlists/reuse-existing-partitions-simple.run	1970-01-01 00:00:00 +0000
+++ runlists/reuse-existing-partitions-simple.run	2019-07-18 04:22:28 +0000
@@ -0,0 +1,4 @@
+testsuites:
+  - name: reuse-existing-partitions-simple
+    fetch_method: bzr-export
+    fetch_location: lp:~mwhudson/ubuntu-test-cases/live-server/testsuites/reuse-existing-partitions-simple

=== added file 'scripts/reuse-existing-partitions-simple.sh'
--- scripts/reuse-existing-partitions-simple.sh	1970-01-01 00:00:00 +0000
+++ scripts/reuse-existing-partitions-simple.sh	2019-07-18 04:22:28 +0000
@@ -0,0 +1,24 @@
+#!/bin/sh
+SCRIPT_DIR=$(readlink -e $(dirname $0))
+set -eu
+sudo sh -euxc '
+    mkdir -p images
+    rm -f images/root.img
+    truncate -s 10G images/root.img
+    parted --script --align optimal images/root.img -- \
+       mklabel gpt \
+       mkpart primary ext4 1MiB 2MiB \
+       set 1 bios_grub on \
+       mkpart primary ext4 2MiB 1GiB \
+       mkpart primary ext4 1GiB -2048s
+    rootdev="$(losetup -Pf --show images/root.img)"
+    mkfs.ext4 "${rootdev}p2"
+    mkdir -p mnt
+    mount "${rootdev}p2" mnt
+    touch mnt/existing
+    umount mnt
+    losetup -d $rootdev
+    chown $SUDO_USER images/root.img
+' 1>&2
+. $SCRIPT_DIR/base.sh
+echo $CMD --disk-image $(pwd)/images/root.img

=== added directory 'testsuites/reuse-existing-partitions-simple'
=== added directory 'testsuites/reuse-existing-partitions-simple/test_reuse_existing_partitions_simple'
=== added file 'testsuites/reuse-existing-partitions-simple/test_reuse_existing_partitions_simple/tc_control'
--- testsuites/reuse-existing-partitions-simple/test_reuse_existing_partitions_simple/tc_control	1970-01-01 00:00:00 +0000
+++ testsuites/reuse-existing-partitions-simple/test_reuse_existing_partitions_simple/tc_control	2019-07-18 04:22:28 +0000
@@ -0,0 +1,10 @@
+description: A test to check if we successfully mounted an existing partition
+dependencies: n/a
+action :
+ 1. Check if there is a file at /srv/existing
+expected_results :
+ 1. There is a file at /srv/existing
+type: userland
+run_as: root
+timeout: 200
+command: python test.py

=== added file 'testsuites/reuse-existing-partitions-simple/test_reuse_existing_partitions_simple/test.py'
--- testsuites/reuse-existing-partitions-simple/test_reuse_existing_partitions_simple/test.py	1970-01-01 00:00:00 +0000
+++ testsuites/reuse-existing-partitions-simple/test_reuse_existing_partitions_simple/test.py	2019-07-18 04:22:28 +0000
@@ -0,0 +1,36 @@
+#!/usr/bin/python
+#
+# Copyright (C) 2019, Canonical Ltd (http://www.canonical.com/)
+#
+# This file is part of ubuntu-server-iso-testing.
+#
+# ubuntu-server-iso-testing is free software: you can redistribute it
+# and/or modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation, either version 3 of
+# the License, or (at your option) any later version.
+#
+# ubuntu-server-iso-testing is distributed in the hope that it will
+# be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
+# of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with ubuntu-server-iso-testing.  If not, see
+# <http://www.gnu.org/licenses/>.
+#
+
+
+import logging
+import os
+import unittest
+
+logging.basicConfig(level=logging.DEBUG)
+
+
+class ReuseTest(unittest.TestCase):
+    def testReuse(self):
+        self.assertTrue(os.path.exists("/srv/existing"),
+                        "no file found at /srv/existing")
+
+if __name__ == "__main__":
+    unittest.main()

=== added file 'testsuites/reuse-existing-partitions-simple/ts_control'
--- testsuites/reuse-existing-partitions-simple/ts_control	1970-01-01 00:00:00 +0000
+++ testsuites/reuse-existing-partitions-simple/ts_control	2019-07-18 04:22:28 +0000
@@ -0,0 +1,1 @@
+timeout: 500

=== added file 'testsuites/reuse-existing-partitions-simple/tslist.run'
--- testsuites/reuse-existing-partitions-simple/tslist.run	1970-01-01 00:00:00 +0000
+++ testsuites/reuse-existing-partitions-simple/tslist.run	2019-07-18 04:22:28 +0000
@@ -0,0 +1,2 @@
+- test: test_reuse_existing_partitions_simple
+  run_as: root