Merge lp:~nuclearbob/utah/smoketest-setup-changes into lp:utah

[Max Brustkern]

This branch adds better error handling to the smoke test log collection and changes job names.

[Joe Talbott (joetalbott)]

This looks good to me.

review: Approve

[Max Brustkern (nuclearbob)]

Okay, it looks like I accidentally proposed this for merging in lp:utah, which we don't want. I'm going to close this and push the changes.

=== added file 'README'
--- README	1970-01-01 00:00:00 +0000
+++ README	2012-11-20 20:10:30 +0000
@@ -0,0 +1,37 @@
+This branch provides code for automated management of jenkins jobs from smoke 
+testing branches.  It currently supports two branch types.
+
+Branches with scripts directories, i.e.:
+lp:~javier.collado/ubuntu-test-cases/desktop/
+For these, a job wil be created for each executable script in the scripts
+directory.
+
+Branches without scripts directories, i.e.:
+lp:~ubuntu-server-dev/utah/server-tests-quantal/
+For these, a job will be created for each runlist in the runlists directory,
+using a preseed if one exists in the preseeds directory, falling back to the 
+default preseed, if it exists.
+
+The main argument for the setup-jobs.py script is the local path to the branch
+(or branches) to use.  Multiple paths can be provided, and will be processed 
+separately.  A current limitation is that the branches need to be checked out 
+on the machine running the tests (i.e. aldebaran.)  I would like to support 
+remote branches eventually, but I think changes to the scripts in script-type 
+branches will be necessary to facilitate this.  Alternately, the job scripts 
+could branch the branches themselves.  For now, I run against checked out 
+branches in /var/cache/utah on aldebaran.
+
+The script will try to guess the job type from the name of the branch, or you 
+can pass in a job type with the -t argument.  It will download images from 
+imageurl (-i) which defaults to the path we use in the lab.  the -j argument 
+to supply a jenkins instance defaults to the 1SS instance.  The -s argument 
+for series will default to the current testing series as defined in python's 
+distro_info.  username and password are specific to the jenkins instance 
+in use.
+
+For each image, a default job will be created from default.xml.tmpl.  This job
+will download the image, perform static validation tests, and then run the 
+default script/runlist if one exists.  All other jobs (created from 
+smoke.xml.tmpl) are triggered when the default job succeeds.  Both jobs have 
+their utah command populated by commandblock.tmpl.  Log collection can also be
+setup here.

=== added file 'commandblock.tmpl'
--- commandblock.tmpl	1970-01-01 00:00:00 +0000
+++ commandblock.tmpl	2012-11-20 20:10:30 +0000
@@ -0,0 +1,11 @@
+    <hudson.tasks.Shell>
+      <command>echo -n "Media Info: " > media_info
+bsdtar -x -f {isopath} -O \./.disk/info >> media_info
+echo >> media_info
+cat media_info
+set +e
+sudo -u utah -i {command} -x /etc/utah/bridged-network-vm.xml >log
+RETCODE=$?
+/usr/share/utah/examples/utah_logs.sh</command>
+exit $RETCODE
+    </hudson.tasks.Shell>

=== added file 'default.xml.tmpl'
--- default.xml.tmpl	1970-01-01 00:00:00 +0000
+++ default.xml.tmpl	2012-11-20 20:10:30 +0000
@@ -0,0 +1,82 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<project>
+  <actions/>
+  <description>Download the latest {series} {installtype} {arch} image, validate it, and initiate smoke tests
+Automatically created as part of UTAH smoke testing</description>
+  <keepDependencies>false</keepDependencies>
+  <properties>
+    <hudson.queueSorter.PrioritySorterJobProperty>
+      <priority>100</priority>
+    </hudson.queueSorter.PrioritySorterJobProperty>
+    <hudson.plugins.throttleconcurrents.ThrottleJobProperty>
+      <maxConcurrentPerNode>0</maxConcurrentPerNode>
+      <maxConcurrentTotal>0</maxConcurrentTotal>
+      <throttleEnabled>false</throttleEnabled>
+      <throttleOption>project</throttleOption>
+    </hudson.plugins.throttleconcurrents.ThrottleJobProperty>
+  </properties>
+  <scm class="hudson.scm.NullSCM"/>
+  <assignedNode>aldebaran</assignedNode>
+  <canRoam>false</canRoam>
+  <disabled>false</disabled>
+  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
+  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
+  <triggers class="vector">
+    <com.redfin.hudson.UrlChangeTrigger>
+      <spec></spec>
+      <url>{imageurl}/{series}/SHA256SUMS.{series}-{installtype}</url>
+    </com.redfin.hudson.UrlChangeTrigger>
+  </triggers>
+  <concurrentBuild>false</concurrentBuild>
+  <builders>
+    <hudson.tasks.Shell>
+      <command># Get rid of this once iso-download is in the stable package
+#sudo /var/lib/utah/fix-iso-perms.sh</command>
+    </hudson.tasks.Shell>
+    <hudson.tasks.Shell>
+      <command>TMPISO=/tmp/utah-{series}-{installtype}-{arch}.iso
+wget -q {imageurl}/{series}/{series}-{installtype}-{arch}.iso -O $TMPISO
+chmod 644 $TMPISO
+sudo -u utah -i /usr/share/utah/isotest/iso_static_validation.py --name $TMPISO
+sudo -u utah -i cp $TMPISO {isopath}
+rm $TMPISO</command>
+    </hudson.tasks.Shell>
+    <hudson.tasks.Shell>
+      <command>rm -rf *</command>
+    </hudson.tasks.Shell>
+{commandblock}
+  </builders>
+  <publishers>
+    <hudson.tasks.BuildTrigger>
+      <childProjects>{jobs}</childProjects>
+      <threshold>
+        <name>SUCCESS</name>
+        <ordinal>0</ordinal>
+        <color>BLUE</color>
+      </threshold>
+    </hudson.tasks.BuildTrigger>
+    <hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
+      <regexp>Media Info:.* \(([\d.]+)\)</regexp>
+      <regexpForFailed>Media Info:.* \(([\d.]+)\)</regexpForFailed>
+      <description>\1</description>
+      <descriptionForFailed>\1</descriptionForFailed>
+      <setForMatrix>false</setForMatrix>
+    </hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
+    <hudson.plugins.build__publisher.BuildPublisher>
+      <publishUnstableBuilds>true</publishUnstableBuilds>
+      <publishFailedBuilds>true</publishFailedBuilds>
+      <postActions class="vector"/>
+    </hudson.plugins.build__publisher.BuildPublisher>
+    <hudson.tasks.ArtifactArchiver>
+      <artifacts>**/*</artifacts>
+      <latestOnly>false</latestOnly>
+    </hudson.tasks.ArtifactArchiver>
+  </publishers>
+  <buildWrappers>
+    <EnvInjectBuildWrapper>
+      <info>
+        <loadFilesFromMaster>false</loadFilesFromMaster>
+      </info>
+    </EnvInjectBuildWrapper>
+  </buildWrappers>
+</project>

=== added file 'setup-jobs.py'
--- setup-jobs.py	1970-01-01 00:00:00 +0000
+++ setup-jobs.py	2012-11-20 20:10:30 +0000
@@ -0,0 +1,240 @@
+#!/usr/bin/python
+"""
+Update jenkins jobs for smoke testing
+Draws heavily from setup-jenkins-utah.py from
+lp:~ubuntu-server-iso-testing-dev/+junk/jenkins-qa-lab
+"""
+# TODO: fix static validation for raring in utah
+# TODO: customize local branch directory
+# TODO: add support for remote branches
+# TODO: get serial console logs
+
+
+import argparse
+import jenkins
+import logging
+import os
+import sys
+
+from distro_info import UbuntuDistroInfo
+
+DEV_SERIES = UbuntuDistroInfo().devel()
+ARCHES = ['i386', 'amd64']
+INSTALLTYPES = ['desktop', 'server', 'mini', 'alternate']
+ISODIR = os.path.join('/', 'var', 'cache', 'utah', 'iso')
+
+
+def get_parser():
+    parser = argparse.ArgumentParser(
+        description=('Update smoke testing jenkins jobs.'),
+        epilog=("For example:\n"
+                "\t%(prog)s /path/to/branch -s " + DEV_SERIES +
+                " -u user -p pass -j http://jenkins:8080"
+                " -i http://10.98.0.1/iso/\n"),
+        formatter_class=argparse.RawDescriptionHelpFormatter)
+    parser.add_argument("branches", nargs='+',
+                        help="Path(s) to branch(es) to process")
+    parser.add_argument("-t", "--type", help="Image type to use"
+                        " (if not supplied, guessed from branch name)")
+    parser.add_argument("-d", "--debug", action="store_true",
+                        help="Enable debugging")
+    parser.add_argument("-n", "--dryrun", action="store_true",
+                        help="Dry run mode. Don't execute jenkins commands")
+    parser.add_argument("-s", "--series", default=DEV_SERIES,
+                        help="series of Ubuntu to download (default=%s)" %
+                        DEV_SERIES)
+    parser.add_argument("-u", "--username",
+                        help="username to use when logging into Jenkins")
+    parser.add_argument("-p", "--password",
+                        help="username to use when logging into Jenkins")
+    parser.add_argument("-j", "--jenkins", default="http://10.189.74.2:8080/",
+                        help="URL of jenkins instance to configure test in")
+    parser.add_argument("-i", "--imageurl", default="http://10.98.0.1/iso/",
+                        help="URL where images are located")
+    return parser
+
+
+def update_jenkins(instance, dryrun, jobname, config):
+    if not instance.job_exists(jobname):
+        logging.debug("Creating Jenkins Job %s ", jobname)
+        if dryrun:
+            return True, ''
+        else:
+            try:
+                instance.create_job(jobname, config)
+                return True, ''
+            except Exception as err:
+                return False, str(err)
+    else:
+        logging.debug("Reconfiguring Jenkins Job %s ", jobname)
+        if dryrun:
+            return True, ''
+        else:
+            try:
+                instance.reconfig_job(jobname, config)
+                return True, ''
+            except Exception as err:
+                return False, str(err)
+
+
+def main():
+    args = get_parser().parse_args()
+    if args.debug:
+        logging.basicConfig(level=logging.DEBUG)
+    else:
+        logging.basicConfig(level=logging.INFO)
+
+    logging.debug('Attempting to login to jenkins at ' + args.jenkins + '...')
+    # Check to see if authentication information provided
+    if args.username is not None:
+        logging.debug('...with authentication as user ' + args.username)
+        instance = jenkins.Jenkins(args.jenkins,
+                                   username=args.username,
+                                   password=args.password)
+    else:
+        logging.debug('...without authentication')
+        instance = jenkins.Jenkins(args.jenkins)
+
+    logging.info('Series: ' + args.series)
+    logging.debug('Image directory: ' + ISODIR)
+    with open('commandblock.tmpl') as template:
+        commandblock = template.read()
+    failures = []
+    for branch in args.branches:
+        branchname = os.path.basename(os.path.realpath(branch.rstrip('/')))
+        installtype = None
+        if args.type is None:
+            logging.debug('Guessing installtype from branch')
+            for checktype in INSTALLTYPES:
+                if checktype in branchname:
+                    installtype = checktype
+                    break
+        else:
+            logging.debug('Using passed installtype: ' + args.type)
+            installtype = args.type
+        if installtype not in INSTALLTYPES:
+            logging.error(installtype + ' is not a valid install type')
+            logging.error('Skipping branch ' + branch)
+            continue
+        logging.info('Branch: ' + branch)
+        logging.info('Type: ' + installtype)
+        logging.info('Architectures to process: ' + ' '.join(ARCHES))
+
+        commands = []
+        scriptdir = os.path.join(branch, 'scripts')
+        runlistdir = os.path.join(branch, 'runlists')
+        isopath = ('{isoroot}/{series}-{installtype}-{arch}.iso'
+                   .format(isoroot=ISODIR,
+                           series=args.series,
+                           installtype=installtype,
+                           arch='{arch}'))
+        if os.path.isdir(scriptdir):
+            logging.debug('Processing scripts directory: ' + scriptdir)
+            testdir = os.path.join('/', 'var', 'cache', 'utah', branchname,
+                                   'scripts')
+            logging.debug('Test directory: ' + testdir)
+            scripts = os.listdir(scriptdir)
+            logging.debug('All scripts: ' + ' '.join(scripts))
+            for script in list(scripts):
+                if os.access(os.path.join(branch, 'scripts', script),
+                             os.X_OK):
+                    logging.debug(script + ' is executable, using')
+                    scriptname = os.path.splitext(script)[0]
+                    command = ('$({testdir}/{test} -i {isopath})'.format(
+                        testdir=testdir,
+                        test=script,
+                        isopath=isopath))
+                    commands.append([scriptname, commandblock.format(
+                                     command=command, isopath=isopath)])
+        elif os.path.isdir(runlistdir):
+            logging.debug('Processing runlists directory: ' + runlistdir)
+            testdir = os.path.join('/', 'var', 'cache', 'utah', branchname)
+            logging.debug('Test directory: ' + testdir)
+            runlists = os.listdir(runlistdir)
+            logging.debug('All runlists: ' + ' '.join(runlists))
+            for runlist in list(runlists):
+                logging.debug('Processing runlist ' + runlist)
+                runlistname = os.path.splitext(runlist)[0]
+                if os.path.isfile(os.path.join(branch, 'preseeds', runlistname
+                                  + '.preseed')):
+                    logging.debug('Found preseed for runlist')
+                    preseed_path = os.path.join(testdir, 'preseeds',
+                                                runlistname + '.preseed')
+                    preseedblock = ('-p ' + preseed_path)
+                elif os.path.isfile(os.path.join(branch, 'preseeds',
+                                    'default.preseed')):
+                    logging.debug('Using default preseed from branch')
+                    preseedblock = '-p ' + os.path.join(testdir, 'preseeds',
+                                                        'default.preseed')
+                else:
+                    logging.debug('No preseed found, using UTAH default')
+                    preseedblock = ''
+                runlist_path = os.path.join(testdir, 'runlists', runlist)
+                command = ('run_utah_tests.py {preseedblock} -i {isopath} '
+                           '{runlist}'
+                           .format(preseedblock=preseedblock,
+                                   isopath=isopath,
+                                   runlist=runlist_path))
+                commands.append([runlistname, commandblock.format(
+                                 command=command, isopath=isopath)])
+        else:
+            logging.error(branch + ' does not contain a scripts directory '
+                          'or a runlists directory; skipping')
+            continue
+
+        for arch in ARCHES:
+            logging.info('Processing architecture ' + arch)
+            jobs = []
+            defaultjob = ''
+            for name, commandblock in commands:
+                if name == 'default':
+                    logging.info('Adding default runlist to default job')
+                    defaultjob = commandblock.format(arch=arch)
+                    continue
+                jobname = '-'.join((args.series, installtype, arch, 'smoke',
+                                    name))
+                logging.info('Processing job ' + jobname)
+                archcommandblock = commandblock.format(arch=arch)
+                with open('smoke.xml.tmpl') as template:
+                    smokexml = (template.read()
+                                .format(name=name,
+                                        series=args.series,
+                                        installtype=installtype,
+                                        arch=arch,
+                                        commandblock=archcommandblock))
+                result, text = update_jenkins(instance, args.dryrun, jobname,
+                                              smokexml)
+                if result:
+                    jobs.append(jobname)
+                else:
+                    logging.debug(jobname + ' failed to configure: ' + text)
+                    failures.append(jobname)
+            jobname = '-'.join((args.series, installtype, arch, 'smoke',
+                                'default'))
+            logging.info('Processing job ' + jobname)
+            with open('default.xml.tmpl') as template:
+                defaultxml = (template.read()
+                              .format(series=args.series,
+                                      installtype=installtype,
+                                      arch=arch,
+                                      isopath=isopath.format(arch=arch),
+                                      jobs=','.join(jobs),
+                                      imageurl=args.imageurl,
+                                      commandblock=defaultjob))
+            result, text = update_jenkins(instance, args.dryrun, jobname,
+                                          defaultxml)
+            if not result:
+                logging.debug(jobname + ' failed to configure: ' + text)
+                failures.append(jobname)
+
+    if len(failures) == 0:
+        logging.info('All jobs updated successfully')
+        sys.exit(0)
+    else:
+        logging.info('These jobs failed to upadte:')
+        for job in failures:
+            logging.info(job)
+        sys.exit(1)
+
+if __name__ == '__main__':
+    main()

=== added file 'smoke.xml.tmpl'
--- smoke.xml.tmpl	1970-01-01 00:00:00 +0000
+++ smoke.xml.tmpl	2012-11-20 20:10:30 +0000
@@ -0,0 +1,57 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<project>
+  <actions/>
+  <description>Run the {name} test on the latest {series} {installtype} {arch} image
+Automatically created as part of UTAH smoke testing</description>
+  <keepDependencies>false</keepDependencies>
+  <properties>
+    <hudson.queueSorter.PrioritySorterJobProperty>
+      <priority>100</priority>
+    </hudson.queueSorter.PrioritySorterJobProperty>
+    <hudson.plugins.throttleconcurrents.ThrottleJobProperty>
+      <maxConcurrentPerNode>0</maxConcurrentPerNode>
+      <maxConcurrentTotal>0</maxConcurrentTotal>
+      <throttleEnabled>false</throttleEnabled>
+      <throttleOption>project</throttleOption>
+    </hudson.plugins.throttleconcurrents.ThrottleJobProperty>
+  </properties>
+  <scm class="hudson.scm.NullSCM"/>
+  <assignedNode>aldebaran</assignedNode>
+  <canRoam>false</canRoam>
+  <disabled>false</disabled>
+  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
+  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
+  <triggers class="vector"/>
+  <concurrentBuild>false</concurrentBuild>
+  <builders>
+    <hudson.tasks.Shell>
+      <command>rm -rf *</command>
+    </hudson.tasks.Shell>
+{commandblock}
+  </builders>
+  <publishers>
+    <hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
+      <regexp>Media Info:.* \(([\d.]+)\)</regexp>
+      <regexpForFailed>Media Info:.* \(([\d.]+)\)</regexpForFailed>
+      <description>\1</description>
+      <descriptionForFailed>\1</descriptionForFailed>
+      <setForMatrix>false</setForMatrix>
+    </hudson.plugins.descriptionsetter.DescriptionSetterPublisher>
+    <hudson.plugins.build__publisher.BuildPublisher>
+      <publishUnstableBuilds>true</publishUnstableBuilds>
+      <publishFailedBuilds>true</publishFailedBuilds>
+      <postActions class="vector"/>
+    </hudson.plugins.build__publisher.BuildPublisher>
+    <hudson.tasks.ArtifactArchiver>
+      <artifacts>**/*</artifacts>
+      <latestOnly>false</latestOnly>
+    </hudson.tasks.ArtifactArchiver>
+  </publishers>
+  <buildWrappers>
+    <EnvInjectBuildWrapper>
+      <info>
+        <loadFilesFromMaster>false</loadFilesFromMaster>
+      </info>
+    </EnvInjectBuildWrapper>
+  </buildWrappers>
+</project>