Merge lp:~raharper/curtin/trunk.disable_yum_plugins into lp:~curtin-dev/curtin/trunk

[Ryan Harper]

Disable yum plugins when installing packages, update ca-certs for https

The default yum configured plugins includes the 'fastmirrors' plugin which
produces inconsistent results resulting in broken installs. Update the
ca-certificates package to prevent https mirror download failures. Add
some retries to the yum install commands to handle transient failures.

[Ryan Harper (raharper)]

This looks like it's helping

https://jenkins.ubuntu.com/server/view/Curtin/job/curtin-vmtest-devel-debug/4/console

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:514
https://jenkins.ubuntu.com/server/job/curtin-ci/569/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/569
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/569
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/569
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/569
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=vm-i386/569

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/569/rebuild

review: Approve (continuous-integration)

[lp:~raharper/curtin/trunk.disable_yum_plugins]

515. By Ryan Harper on 2017-08-01

Ensure yum install robustness

epel mirrorlist sometimes returns incomplete repos, attempt to ensure
install completes by upgrading ca-certificates package for https traffic
and retry the yum command 30 times.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:515
https://jenkins.ubuntu.com/server/job/curtin-ci/571/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/571/console
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/571/console
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/571/console
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/571/console
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=vm-i386/571/console

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/571/rebuild

review: Needs Fixing (continuous-integration)

[lp:~raharper/curtin/trunk.disable_yum_plugins]

516. By Ryan Harper on 2017-08-01

remove trailing whitespace

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:516
https://jenkins.ubuntu.com/server/job/curtin-ci/572/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/572
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/572
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/572
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/572
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=vm-i386/572

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/572/rebuild

review: Approve (continuous-integration)

[lp:~raharper/curtin/trunk.disable_yum_plugins]

517. By Ryan Harper on 2017-08-01

log the captured output of yum install cmds

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:517
https://jenkins.ubuntu.com/server/job/curtin-ci/573/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/573
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/573
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/573
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/573
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=vm-i386/573

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/573/rebuild

review: Approve (continuous-integration)

[lp:~raharper/curtin/trunk.disable_yum_plugins]

518. By Ryan Harper on 2017-08-01

add missing 'd' to log_captured parameter

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:518
https://jenkins.ubuntu.com/server/job/curtin-ci/574/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/574
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/574
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/574
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/574
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=vm-i386/574

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/574/rebuild

review: Approve (continuous-integration)

[Chad Smith (chad.smith)]

+1 on this changeset. Trivial comment request inline.

Not for this proposal: I don't really understand why subp needs to have capture and log_capture params. It seems like in a subsequent branch we could drop the log_capture and just use capture to match cloud-init's subp
+1

review: Approve

[lp:~raharper/curtin/trunk.disable_yum_plugins]

519. By Ryan Harper on 2017-08-02

Update comment around yum installs for clarity around separate invocations

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:519
https://jenkins.ubuntu.com/server/job/curtin-ci/577/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/577
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/577
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/577
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/577
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=vm-i386/577

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/577/rebuild

review: Approve (continuous-integration)

[Chad Smith (chad.smith)]

Strike my comment about dropping either capture/or log_captured params. Per IRC discussions I see the need for both params. Not setting capture allows us to avoid setting up buffered PIPEs when we intend to disregard the output of a command run, and there are many cases we wouldn't want to be too noisy in our logs with output of external commands for which we handle failures.

[lp:~raharper/curtin/trunk.disable_yum_plugins]

520. By Ryan Harper on 2017-08-02

merge from trunk

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:520
https://jenkins.ubuntu.com/server/job/curtin-ci/578/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/578
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/578
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/578
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/578
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=vm-i386/578

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/578/rebuild

review: Approve (continuous-integration)

=== modified file 'curtin/commands/curthooks.py'
--- curtin/commands/curthooks.py	2017-08-01 15:46:19 +0000
+++ curtin/commands/curthooks.py	2017-08-02 17:51:07 +0000
@@ -833,14 +833,23 @@
             util.write_file(cloud_init_yum_repo,
                             content=cloud_init_repo(rpm_get_dist_id(target)))
 
-            # we separate the installation of epel from cloud-init as
-            # cloud-init will depend on packages included in epel and yum needs
-            # to add the repository before we can install cloud-init
+            # we separate the installation of repository packages (epel,
+            # cloud-init-el-release) as we need a new invocation of yum
+            # to read the newly installed repo files.
+            YUM_CMD = ['yum', '-y', '--noplugins', 'install']
+            retries = [1] * 30
             with util.ChrootableTarget(target) as in_chroot:
-                in_chroot.subp(['yum', '-y', 'install', 'epel-release'])
-                in_chroot.subp(['yum', '-y', 'install',
-                                'cloud-init-el-release'])
-                in_chroot.subp(['yum', '-y', 'install', 'cloud-init'])
+                # ensure up-to-date ca-certificates to handle https mirror
+                # connections
+                in_chroot.subp(YUM_CMD + ['ca-certificates'], capture=True,
+                               log_captured=True, retries=retries)
+                in_chroot.subp(YUM_CMD + ['epel-release'], capture=True,
+                               log_captured=True, retries=retries)
+                in_chroot.subp(YUM_CMD + ['cloud-init-el-release'],
+                               log_captured=True, capture=True,
+                               retries=retries)
+                in_chroot.subp(YUM_CMD + ['cloud-init'], capture=True,
+                               log_captured=True, retries=retries)
 
             # remove cloud-init el-stable bootstrap repo config as the
             # cloud-init-el-release package points to the correct repo
@@ -853,7 +862,8 @@
                                    capture=False, rcs=[0])
                 except util.ProcessExecutionError:
                     LOG.debug('Image missing bridge-utils package, installing')
-                    in_chroot.subp(['yum', '-y', 'install', 'bridge-utils'])
+                    in_chroot.subp(YUM_CMD + ['bridge-utils'], capture=True,
+                                   log_captured=True, retries=retries)
 
     LOG.info('Passing network configuration through to target')
     net.render_netconfig_passthrough(target, netconfig={'network': netcfg})