lp:~jgrimm/curtin/coverage-for-unittest

Branch merges

Propose for merging

No branches dependent on this one.

Merged into lp:~curtin-dev/curtin/trunk at revision 370

Scott Moser (community): Needs Fixing on 2015-12-17

Server Team CI bot: Approve (continuous-integration) on 2015-12-10

Diff: 45 lines (+11/-2)

2 files modified

.bzrignore (+1/-0)
Makefile (+10/-2)

api

Related source package recipes

No recipes using this branch. (?)

Create packaging recipe

Related snap packages

Related bugs

Related blueprints

321. By Jon Grimm on 2015-12-10

Enable coverage test for unit tests

Enabling for vmtest will take a bit more thought, as well as,
combining results all test runs. HTML reports later too. Additionally,
teach bzr to ignore .coverage directory where runtime coverage data gets
written to.

To run:
   make unittest COVERAGE=1
or
   make coverage

The 'coverage' target will grow to support coverage testing all tests.
The default options for nose coverage opts can be overridden via
the 'coverageopts' environment variable.

320. By Ryan Harper on 2015-12-09

Use /proc/mounts to find missing mountpoints

lsblk (notably on trusty) does not find all mountpoints. Augment
block.get_mountpoints() lsblk data with mountpoints from proc/mounts.
Refactor reading proc/mounts into common function in block for reuse.

Fixes PreciseTestBasic (issues with btrfs on whole disk)
Fixes TrustyTestBasic (issues with dname and ptable)

319. By Scott Moser on 2015-12-08

tests: invoke nosetests with --nologcapture

When a install or boot fails, the vmtest harness would log.DEBUG
with the console log output. nose was capturing this logging
and replaying it on test failure.

This causes very long output to the screen of the user who ran
'make vmtest', making it hard for them to find out what happened.

Since we're now reasonably storing the console logs, its quite
sane to not spit those to stderr so that the output is easier to read.

318. By Scott Moser on 2015-12-07

vmtests: validate formatting/mounting of a disk without a partition table

Several use-cases involve formatting and entire disk with a filesystem.
Exercise this code by adding additional disks and tests to the test_basic
test. Utilize the btrfs filesystem type. Adding this test exposed several
issues with both fstab writing (it didn't know what to do with volumes of
type 'disk') and typo's in postinstall code which installed filesystem
utilities into the target (brtfs vs. btrfs). A couple other minor fixes
were included to ensure the testsuite

317. By Scott Moser on 2015-12-07

vmtest: do not expect sda-part2 to exist in /dev/disk/by-dname on trusty

This never worked, but we did not see failure before because of a test
case never properly executing.

316. By Scott Moser on 2015-12-07

dowait cleanups

make vmtest specify --dowait or --no-wait when running 'launch' and
'xkvm'. Additionally, have these programs be a bit smarter and default
to dowait=true unless both stdin and stdout are a terminal.
Previously, the default only considered stdin.

The issue that is fixed here is that if a developer ran:
  nosetests3 -vv tests/vmtests/
then xkvm would run qemu-system-x86_64 in the foreground.
by doing that, it would then not be able to handle the TERM signal
that it got from the timeout.

315. By Scott Moser on 2015-12-04

vmtests: improve output of vmtests

This changes vmtest to put its output in a single top level directory.
By default:
  /tmp/vmtest-2015-11-16T204216.241018.log
  /tmp/vmtest-2015-11-16T204216.241018/

Inside that dir are files named per each test, like:
   WilyTestBasic

Add tools/jenkins-runner that simplifies jenkins running so that
artifacts can easily be collected. If you run tools/jenkins-runner
it will create output/ and put all files in there.

Also now allow configuration of what things to keep from the vmtest run
and when to keep them. Default value is to keep all files from failed
tests. This is configurable via CURTIN_VMTEST_KEEP_DATA_PASS and
CURTIN_VMTEST_DATA_FAIL.

Add test_vmtest.py that contains Psuedo tests that act like the vmtests,
but run quickly. They are useful for testing the setUpClass and the test
harness itself. They will pass and not do really anything. To enable them
to fail and exercise that portion of the test harness, set the environment
variable CURTIN_VMTEST_DEBUG_ALLOW_FAIL=1.

Finally add documentation for all environment variables used by vmtest
to doc/devel/README-vmtest.txt.

314. By Scott Moser on 2015-11-20

debian/control: do not list e2fsprogs

e2fsprogs is essential and thus we do not need an explicit
depends per lintian-info depends-on-essential-package-without-using-version.

313. By Scott Moser on 2015-11-19

Rename vmtest test class names to match module.

312. By Scott Moser on 2015-11-19

Modify how vmtests checks if a curtin install was successful.

1. we missed finding failed installs on precise/trusty due to non-systemd
   output. systemd prepends non-pid-1 process output to console with the
   calling process, so we had 'cloud-init[.*]', however on pre-systemd
   systems, that's not present. Fix this by dropping the cloud-init
   portion of the regular expression

2. Add in additional error messages we've seen during debugging of test
   cases

3. Modify the check to first see if we can find curtin's successful
   install message; if that's not found, then look for common errors.