Merge lp:~dbuliga/charm-helpers/charm-helpers into lp:charm-helpers
- charm-helpers
- Merge into devel
| Status: | Merged |
|---|---|
| Merged at revision: | 620 |
| Proposed branch: | lp:~dbuliga/charm-helpers/charm-helpers |
| Merge into: | lp:charm-helpers |
| Diff against target: |
3240 lines (+1863/-724) 19 files modified
charmhelpers/core/host.py (+28/-59) charmhelpers/core/host_factory/centos.py (+56/-0) charmhelpers/core/host_factory/ubuntu.py (+56/-0) charmhelpers/core/kernel.py (+21/-15) charmhelpers/core/kernel_factory/centos.py (+17/-0) charmhelpers/core/kernel_factory/ubuntu.py (+13/-0) charmhelpers/fetch/__init__.py (+28/-299) charmhelpers/fetch/bzrurl.py (+4/-3) charmhelpers/fetch/centos.py (+171/-0) charmhelpers/fetch/giturl.py (+4/-3) charmhelpers/fetch/ubuntu.py (+313/-0) charmhelpers/osplatform.py (+19/-0) tests/__init__.py (+4/-0) tests/core/test_host.py (+200/-44) tests/core/test_kernel.py (+89/-43) tests/fetch/test_archiveurl.py (+2/-1) tests/fetch/test_bzrurl.py (+2/-1) tests/fetch/test_fetch.py (+830/-253) tests/fetch/test_giturl.py (+6/-3) |
| To merge this branch: | bzr merge lp:~dbuliga/charm-helpers/charm-helpers |
| Related bugs: |
| Reviewer | Review Type | Date Requested | Status |
|---|---|---|---|
| Marco Ceppi | 2016-06-02 | Approve on 2016-08-30 | |
| Alex Kavanagh | Approve on 2016-08-12 | ||
|
Review via email:
|
|||
Commit Message
Description of the Change
Considering that now there is CentOS support in Juju, it would be great to also support this platform in charmhelpers. As a result, this branch attempts to add cross platform support for RPM based systems, with CentOS as the first such OS.
This branch introduces a factory that based on platform, loads the proper functionality for that platform. There is currently a module "ubuntu", which is basically the old code moved into its own submodule, and a "centos" which implements the same functionality. The factory itself defines a number of functions that are common between the 2 OSs. For backwards compatibility, it also does a:
from .ubuntu import *
So you still get the same behavior as you previously did. So far only 2 modules have been touched by this branch: core and fetch. Most of the others are not really needed for basic charm building (contrib/openstack for example)
There was the need of a method in charmhelpers/
The need of changing methods name in fetch appeared because, in CentOS it was not feasible to have methods with name such as: (apt_install for example)
It will be easier to transition charms from using functions like: "apt_install" to just "install". More than this, at the bottom of the fetch/__init__.py there are some aliases created for backwards compatibility.
Initial code review comment from Charles Butler(lazypower):
https:/
| Matt Bruzek (mbruzek) wrote : | # |
- 581. By Matt Bruzek on 2016-06-02
-
[jamesbeedy] Added uid and gid specification functionality for adduser and addgroup.
[mbruzek] Added a newline to fix lint.
| Denis Buliga (dbuliga) wrote : | # |
> Denis,
>
> Thanks for this merge proposal! I am very interested in adding support for
> CentOS.
>
> Added a few comments in the code about _removing_ docstrings or changing
> docstring quotes to be inconsistent in the file. Since this is a library we
> should docstring every non private method.
>
> Tried running `make test` in this branch and the "lint" target failed with
> several errors in the files you added, edited. Please make sure this branch
> passes lint and test make targets.
>
> This is a big change, and many charms use charm-helpers. If you could help
> identify a way (or a charm) to verify this does not break backward
> compatibility with charms that install charm-helpers that would be most
> appreciated.
I have implemented the necessary changes in order to meet your requirements. There might be a way you can test this PR. I have implemented a refactor of squid_reverseproxy charm to work on centos without breaking ubuntu's old functionality. It can be found here:
https:/
- 582. By Ryan Beisner on 2016-06-13
-
[jamespage,
r=1chb1n] Update amulet helpers to deploy charms from the charm store As part of the migration to git/gerrit and with the introduction of
layered charms, the charm store really needs to be the definative
source of truth for all charms during test.Switch the charm resolver code to pick the correct charms from the
charm store for amulet tests.This does change behaviour in that base charms are deployed aligned
to the test series where possible; otherwise the most recent Ubuntu
series is used instead - results in mysql/trusty + keystone/xenial
for example. - 583. By James Page on 2016-06-14
-
Enable default git repo generation if a default openstack-
origin- git value is specified. - 584. By David Ames on 2016-06-14
-
[thedac, r=tinwood] DNS HA Helpers
Start the process of moving HA helpers into the openstack contrib area
Add the helper for API charms to use DNS HA
Add validation for new DNS config options in HA
Allow resolve_address to override with DNS or not - 585. By Stuart Bishop on 2016-06-16
-
[marcoceppi, r=stub] Fix bzr branch installer
- 586. By Stuart Bishop on 2016-06-16
-
[george-edison55, r=stub] Fix fetch.install_
remote( ) when multiple handlers match the URL - 587. By Stuart Bishop on 2016-06-16
-
[chris-gondolin, r=stub] Option to specify a revno when fetching from a bzr repo
- 588. By Stuart Bishop on 2016-06-16
-
[stub] Add test to bzr revno fetcher, fix casting of bzr revno
- 589. By Stuart Bishop on 2016-06-17
-
[freyes, r=stub] Mock contrib.
network. ufw.modprobe in tests as required - 590. By James Page on 2016-06-23
-
Updates for DNS-HA support in non-api OpenStack charms.
- 591. By James Page on 2016-06-23
-
Rebase
- 592. By Liam Young on 2016-06-23
-
[gnuoy, r=james-page] Decode result of check_output in network_
get_primary_ address Under python3.5 check_output returns bytes which need decoding. This change
adds the decode to network_get_primary_ address and brings it in line with other
core hookenv functions. - 593. By James Page on 2016-06-23
-
Fixes for deploy from source for openstack
- 594. By James Page on 2016-07-01
-
Fixes for apparmor support for openstack contrib module
- 595. By James Page on 2016-07-05
-
Misc fixes and improvements for deploy-from-source.
- 596. By James Page on 2016-07-05
-
Add missing openstack.ha package to setup.py
- 597. By Marco Ceppi <marco@T430> on 2016-07-06
-
[james-page] Re-license charm-helpers inline with agreed licensing approaches for charms, interfaces and layers.
- 598. By Marco Ceppi <marco@T430> on 2016-07-06
-
version bump
- 599. By Marco Ceppi <marco@T430> on 2016-07-06
-
flake8 fixes
- 600. By James Page on 2016-07-07
-
[trivial] Support nrpe for testing on wily and xenial
- 601. By James Page on 2016-07-08
-
Add option to allow location of user home directory to be provided when creating users.
- 602. By Ryan Beisner on 2016-07-12
-
[chris.macnaughton, r=1chb1n] Add ceph-proxy to source charms for openstack amulet helper
- 603. By Ryan Beisner on 2016-07-12
-
[thedac, r=1chb1n] Send LSB release to apparmor context templates
Set the LSB release to be consumed by apparmor context profiles
- 604. By David Ames on 2016-07-12
-
[billy-olsen, r=thedac] Allow float for worker-multiplier.
- 605. By James Page on 2016-07-13
-
Misc updates for deploy from source
Improve OpenStack release determination.
- 606. By Jorge Niedbalski on 2016-07-13
-
[billy-olsen, r=niedbalski, cholcombe] Partially fixes bug LP: #1492742
- 607. By Jorge Niedbalski on 2016-07-13
-
[niedbalski, r=] Fix broken tests for py2.7
- 608. By Stuart Bishop on 2016-07-14
-
[stub,r=james-page] Make fetch.apt_cache() quiet by default
- 609. By Stuart Bishop on 2016-07-15
-
[fginther, r=stub] Install python3-jinja2 when running Python3
- 610. By James Page on 2016-07-18
-
Add 2.9.0 to list of releases for newton
- 611. By David Ames on 2016-07-18
-
[thedac, r=gnuoy] Make apt_install fatal=True for dnspython install
- 612. By Liam Young on 2016-07-21
-
[gnuoy, r=jamespage] Only write out CA cert if it has changed
Only write out CA cert and run the update-
ca-certificates if the cert has
actually changed. This reduces the risk of certs being pulled from under services
which are trying to do client side certificate validation on remote https
endpoints. - 613. By Liam Young on 2016-07-21
-
[1chb1n, r=gnuoy] Consume env var AMULET_
SETUP_TIMEOUT if set - 614. By Liam Young on 2016-07-21
-
[corey.bryant, r=gnuoy] Install networking-hyperv when deploying neutron-api from source.
- 615. By Stuart Bishop on 2016-07-29
-
[axino, r=stub] nrpe: make add_init_
service_ checks support systemd
| Alex Kavanagh (ajkavanagh) wrote : | # |
Hi Dennis
A huge amount of work has gone into this. I, too, like the general
approach. However, I would like to suggest a slightly radical change on approach to the
module that I think will simplify its merging and make it easier to maintain in the future.
However, firstly, the changes don't pass the 'make lint' command (e.g. PEP8
compliance).
Also, I also believe that charm-helpers has had a few more commits since your
last revision which means that it'll need to be brought back up to tip.
My suggestion is to not use classes, but instead just use sub-modules for the differences
between (in this case CentOS and Ubuntu). Hopefully, my comments will make sense below:
Comments
get_platform() in charmhelpers/
This function is at a top level, but isn't needed by the vast majority of
charmhelpers. I would propose to put it into its own module.
Also, the function could be more defensive and raise a RuntimeError() in the
case where the platform is neither Ubuntu nor CentOS. e.g.
def get_platform():
"""
Return the current OS platform
For example: if current os platform is Ubuntu then a string "ubuntu"
will be returned (which is the name of the module).
This string is used to decide which platform module should be imported.
"""
tuple_platform = platform.
current_
if "Ubuntu" in current_platform:
return "ubuntu"
elif "CentOS" in current_platform:
return "centos"
else:
raise RuntimeError("This module is not supported on {}."
charmhelpers/
I think that the use of classes to separate common code from host specific code
is over-complicating this code and looks very different to the rest of the
charmhelpers code.
I'd like to suggest that this be refactored into a simpler module system with a
host selector that selectively imports specific host functions as the 'generic'
function.
i.e. charmhelpers/
import os
from contextlib import contextmanager
from charmhelpers.core import host_factory
__platform = ... determine the platform ...
if __platform == 'ubuntu':
from charmhelpers.
add_group,
)
elif __platform = 'centos':
from charmhelpers.
add_group,
)
... the rest of the common code.
If there is common code in functions, there's no reason you can't import an
'service_
function.
This will make the patch set much smaller (as there are fewer functions being
changed) yet the host specific code can still be used transparently from the
calling code.
charmhelpers/
I think the same thing can be done with the kernel files. There are only two
functions that are different, thus:
__platform = ... determine the platform ...
if __platform == 'ubuntu':
from charmhelpers.
modprobe...
- 616. By Liam Young on 2016-08-01
-
[ajkavanagh r=gnuoy] Add a v3 version of service catalog checking
This adds a keystone v3 version of service catalog checking to the
contrib/openstack/ amulet/ utils.py file. - 617. By Liam Young on 2016-08-01
-
[gnuoy, trivial] Lint fix
- 618. By James Page on 2016-08-02
-
Set a minimum PG count of 2, to avoid math domain errors in small OSD setups
- 619. By Denis Buliga on 2016-08-10
-
Added Support for CentOS and unit tests. Addressed review comments
| Alex Kavanagh (ajkavanagh) wrote : | # |
Hi Dennis
Definitely getting there. I've included some comments to try to simplify some small bits of code, and be a bit more idiomatic Python, but otherwise it's now very close. I'll also try to run the tests and comment back here.
Thanks again.
| Alex Kavanagh (ajkavanagh) wrote : | # |
I've run the 'make lint' and that passes for both Py2 and Py3 - great stuff.
The make test2 and test3 are still failing for me on a clean box - it's missing the yum module which either needs mocking out or including in test_requiremen
- 620. By Denis Buliga on 2016-08-11
-
Implemented changes. Addressed review comments.
- 621. By Denis Buliga on 2016-08-12
-
Using getenv method instead of mocking os.environ
| Alex Kavanagh (ajkavanagh) wrote : | # |
I've worked with Dennis and it nows passes lint on Py2, Py3 and it passes all of it's test2 and test3 tests. I think it's now ready to merge.
Thanks very much Dennis!
Preview Diff
| 1 | === modified file 'charmhelpers/core/host.py' |
| 2 | --- charmhelpers/core/host.py 2016-07-07 14:08:38 +0000 |
| 3 | +++ charmhelpers/core/host.py 2016-08-12 07:09:49 +0000 |
| 4 | @@ -30,13 +30,29 @@ |
| 5 | import hashlib |
| 6 | import functools |
| 7 | import itertools |
| 8 | +import six |
| 9 | + |
| 10 | from contextlib import contextmanager |
| 11 | from collections import OrderedDict |
| 12 | - |
| 13 | -import six |
| 14 | - |
| 15 | from .hookenv import log |
| 16 | from .fstab import Fstab |
| 17 | +from charmhelpers.osplatform import get_platform |
| 18 | + |
| 19 | +__platform__ = get_platform() |
| 20 | +if __platform__ == "ubuntu": |
| 21 | + from charmhelpers.core.host_factory.ubuntu import ( |
| 22 | + service_available, |
| 23 | + add_new_group, |
| 24 | + lsb_release, |
| 25 | + cmp_pkgrevno, |
| 26 | + ) # flake8: noqa -- ignore F401 for this import |
| 27 | +elif __platform__ == "centos": |
| 28 | + from charmhelpers.core.host_factory.centos import ( |
| 29 | + service_available, |
| 30 | + add_new_group, |
| 31 | + lsb_release, |
| 32 | + cmp_pkgrevno, |
| 33 | + ) # flake8: noqa -- ignore F401 for this import |
| 34 | |
| 35 | |
| 36 | def service_start(service_name): |
| 37 | @@ -144,8 +160,11 @@ |
| 38 | return False |
| 39 | else: |
| 40 | # This works for upstart scripts where the 'service' command |
| 41 | - # returns a consistent string to represent running 'start/running' |
| 42 | - if "start/running" in output: |
| 43 | + # returns a consistent string to represent running |
| 44 | + # 'start/running' |
| 45 | + if ("start/running" in output or |
| 46 | + "is running" in output or |
| 47 | + "up and running" in output): |
| 48 | return True |
| 49 | elif os.path.exists(_INIT_D_CONF.format(service_name)): |
| 50 | # Check System V scripts init script return codes |
| 51 | @@ -153,18 +172,6 @@ |
| 52 | return False |
| 53 | |
| 54 | |
| 55 | -def service_available(service_name): |
| 56 | - """Determine whether a system service is available""" |
| 57 | - try: |
| 58 | - subprocess.check_output( |
| 59 | - ['service', service_name, 'status'], |
| 60 | - stderr=subprocess.STDOUT).decode('UTF-8') |
| 61 | - except subprocess.CalledProcessError as e: |
| 62 | - return b'unrecognized service' not in e.output |
| 63 | - else: |
| 64 | - return True |
| 65 | - |
| 66 | - |
| 67 | SYSTEMD_SYSTEM = '/run/systemd/system' |
| 68 | |
| 69 | |
| 70 | @@ -173,8 +180,9 @@ |
| 71 | return os.path.isdir(SYSTEMD_SYSTEM) |
| 72 | |
| 73 | |
| 74 | -def adduser(username, password=None, shell='/bin/bash', system_user=False, |
| 75 | - primary_group=None, secondary_groups=None, uid=None, home_dir=None): |
| 76 | +def adduser(username, password=None, shell='/bin/bash', |
| 77 | + system_user=False, primary_group=None, |
| 78 | + secondary_groups=None, uid=None, home_dir=None): |
| 79 | """Add a user to the system. |
| 80 | |
| 81 | Will log but otherwise succeed if the user already exists. |
| 82 | @@ -286,17 +294,7 @@ |
| 83 | log('group with gid {0} already exists!'.format(gid)) |
| 84 | except KeyError: |
| 85 | log('creating group {0}'.format(group_name)) |
| 86 | - cmd = ['addgroup'] |
| 87 | - if gid: |
| 88 | - cmd.extend(['--gid', str(gid)]) |
| 89 | - if system_group: |
| 90 | - cmd.append('--system') |
| 91 | - else: |
| 92 | - cmd.extend([ |
| 93 | - '--group', |
| 94 | - ]) |
| 95 | - cmd.append(group_name) |
| 96 | - subprocess.check_call(cmd) |
| 97 | + add_new_group(group_name, system_group, gid) |
| 98 | group_info = grp.getgrnam(group_name) |
| 99 | return group_info |
| 100 | |
| 101 | @@ -541,16 +539,6 @@ |
| 102 | return r |
| 103 | |
| 104 | |
| 105 | -def lsb_release(): |
| 106 | - """Return /etc/lsb-release in a dict""" |
| 107 | - d = {} |
| 108 | - with open('/etc/lsb-release', 'r') as lsb: |
| 109 | - for l in lsb: |
| 110 | - k, v = l.split('=') |
| 111 | - d[k.strip()] = v.strip() |
| 112 | - return d |
| 113 | - |
| 114 | - |
| 115 | def pwgen(length=None): |
| 116 | """Generate a random pasword.""" |
| 117 | if length is None: |
| 118 | @@ -674,25 +662,6 @@ |
| 119 | return hwaddr |
| 120 | |
| 121 | |
| 122 | -def cmp_pkgrevno(package, revno, pkgcache=None): |
| 123 | - """Compare supplied revno with the revno of the installed package |
| 124 | - |
| 125 | - * 1 => Installed revno is greater than supplied arg |
| 126 | - * 0 => Installed revno is the same as supplied arg |
| 127 | - * -1 => Installed revno is less than supplied arg |
| 128 | - |
| 129 | - This function imports apt_cache function from charmhelpers.fetch if |
| 130 | - the pkgcache argument is None. Be sure to add charmhelpers.fetch if |
| 131 | - you call this function, or pass an apt_pkg.Cache() instance. |
| 132 | - """ |
| 133 | - import apt_pkg |
| 134 | - if not pkgcache: |
| 135 | - from charmhelpers.fetch import apt_cache |
| 136 | - pkgcache = apt_cache() |
| 137 | - pkg = pkgcache[package] |
| 138 | - return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) |
| 139 | - |
| 140 | - |
| 141 | @contextmanager |
| 142 | def chdir(directory): |
| 143 | """Change the current working directory to a different directory for a code |
| 144 | |
| 145 | === added directory 'charmhelpers/core/host_factory' |
| 146 | === added file 'charmhelpers/core/host_factory/__init__.py' |
| 147 | === added file 'charmhelpers/core/host_factory/centos.py' |
| 148 | --- charmhelpers/core/host_factory/centos.py 1970-01-01 00:00:00 +0000 |
| 149 | +++ charmhelpers/core/host_factory/centos.py 2016-08-12 07:09:49 +0000 |
| 150 | @@ -0,0 +1,56 @@ |
| 151 | +import subprocess |
| 152 | +import yum |
| 153 | +import os |
| 154 | + |
| 155 | + |
| 156 | +def service_available(service_name): |
| 157 | + # """Determine whether a system service is available.""" |
| 158 | + if os.path.isdir('/run/systemd/system'): |
| 159 | + cmd = ['systemctl', 'is-enabled', service_name] |
| 160 | + else: |
| 161 | + cmd = ['service', service_name, 'is-enabled'] |
| 162 | + return subprocess.call(cmd) == 0 |
| 163 | + |
| 164 | + |
| 165 | +def add_new_group(group_name, system_group=False, gid=None): |
| 166 | + cmd = ['groupadd'] |
| 167 | + if gid: |
| 168 | + cmd.extend(['--gid', str(gid)]) |
| 169 | + if system_group: |
| 170 | + cmd.append('-r') |
| 171 | + cmd.append(group_name) |
| 172 | + subprocess.check_call(cmd) |
| 173 | + |
| 174 | + |
| 175 | +def lsb_release(): |
| 176 | + """Return /etc/os-release in a dict.""" |
| 177 | + d = {} |
| 178 | + with open('/etc/os-release', 'r') as lsb: |
| 179 | + for l in lsb: |
| 180 | + s = l.split('=') |
| 181 | + if len(s) != 2: |
| 182 | + continue |
| 183 | + d[s[0].strip()] = s[1].strip() |
| 184 | + return d |
| 185 | + |
| 186 | + |
| 187 | +def cmp_pkgrevno(package, revno, pkgcache=None): |
| 188 | + """Compare supplied revno with the revno of the installed package. |
| 189 | + |
| 190 | + * 1 => Installed revno is greater than supplied arg |
| 191 | + * 0 => Installed revno is the same as supplied arg |
| 192 | + * -1 => Installed revno is less than supplied arg |
| 193 | + |
| 194 | + This function imports YumBase function if the pkgcache argument |
| 195 | + is None. |
| 196 | + """ |
| 197 | + if not pkgcache: |
| 198 | + y = yum.YumBase() |
| 199 | + packages = y.doPackageLists() |
| 200 | + pkgcache = {i.Name: i.version for i in packages['installed']} |
| 201 | + pkg = pkgcache[package] |
| 202 | + if pkg > revno: |
| 203 | + return 1 |
| 204 | + if pkg < revno: |
| 205 | + return -1 |
| 206 | + return 0 |
| 207 | |
| 208 | === added file 'charmhelpers/core/host_factory/ubuntu.py' |
| 209 | --- charmhelpers/core/host_factory/ubuntu.py 1970-01-01 00:00:00 +0000 |
| 210 | +++ charmhelpers/core/host_factory/ubuntu.py 2016-08-12 07:09:49 +0000 |
| 211 | @@ -0,0 +1,56 @@ |
| 212 | +import subprocess |
| 213 | + |
| 214 | + |
| 215 | +def service_available(service_name): |
| 216 | + """Determine whether a system service is available""" |
| 217 | + try: |
| 218 | + subprocess.check_output( |
| 219 | + ['service', service_name, 'status'], |
| 220 | + stderr=subprocess.STDOUT).decode('UTF-8') |
| 221 | + except subprocess.CalledProcessError as e: |
| 222 | + return b'unrecognized service' not in e.output |
| 223 | + else: |
| 224 | + return True |
| 225 | + |
| 226 | + |
| 227 | +def add_new_group(group_name, system_group=False, gid=None): |
| 228 | + cmd = ['addgroup'] |
| 229 | + if gid: |
| 230 | + cmd.extend(['--gid', str(gid)]) |
| 231 | + if system_group: |
| 232 | + cmd.append('--system') |
| 233 | + else: |
| 234 | + cmd.extend([ |
| 235 | + '--group', |
| 236 | + ]) |
| 237 | + cmd.append(group_name) |
| 238 | + subprocess.check_call(cmd) |
| 239 | + |
| 240 | + |
| 241 | +def lsb_release(): |
| 242 | + """Return /etc/lsb-release in a dict""" |
| 243 | + d = {} |
| 244 | + with open('/etc/lsb-release', 'r') as lsb: |
| 245 | + for l in lsb: |
| 246 | + k, v = l.split('=') |
| 247 | + d[k.strip()] = v.strip() |
| 248 | + return d |
| 249 | + |
| 250 | + |
| 251 | +def cmp_pkgrevno(package, revno, pkgcache=None): |
| 252 | + """Compare supplied revno with the revno of the installed package. |
| 253 | + |
| 254 | + * 1 => Installed revno is greater than supplied arg |
| 255 | + * 0 => Installed revno is the same as supplied arg |
| 256 | + * -1 => Installed revno is less than supplied arg |
| 257 | + |
| 258 | + This function imports apt_cache function from charmhelpers.fetch if |
| 259 | + the pkgcache argument is None. Be sure to add charmhelpers.fetch if |
| 260 | + you call this function, or pass an apt_pkg.Cache() instance. |
| 261 | + """ |
| 262 | + import apt_pkg |
| 263 | + if not pkgcache: |
| 264 | + from charmhelpers.fetch import apt_cache |
| 265 | + pkgcache = apt_cache() |
| 266 | + pkg = pkgcache[package] |
| 267 | + return apt_pkg.version_compare(pkg.current_ver.ver_str, revno) |
| 268 | |
| 269 | === modified file 'charmhelpers/core/kernel.py' |
| 270 | --- charmhelpers/core/kernel.py 2016-07-06 14:41:05 +0000 |
| 271 | +++ charmhelpers/core/kernel.py 2016-08-12 07:09:49 +0000 |
| 272 | @@ -15,15 +15,28 @@ |
| 273 | # See the License for the specific language governing permissions and |
| 274 | # limitations under the License. |
| 275 | |
| 276 | -__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" |
| 277 | +import re |
| 278 | +import subprocess |
| 279 | |
| 280 | +from charmhelpers.osplatform import get_platform |
| 281 | from charmhelpers.core.hookenv import ( |
| 282 | log, |
| 283 | INFO |
| 284 | ) |
| 285 | |
| 286 | -from subprocess import check_call, check_output |
| 287 | -import re |
| 288 | +__platform__ = get_platform() |
| 289 | +if __platform__ == "ubuntu": |
| 290 | + from charmhelpers.core.kernel_factory.ubuntu import ( |
| 291 | + persistent_modprobe, |
| 292 | + update_initramfs, |
| 293 | + ) # flake8: noqa -- ignore F401 for this import |
| 294 | +elif __platform__ == "centos": |
| 295 | + from charmhelpers.core.kernel_factory.centos import ( |
| 296 | + persistent_modprobe, |
| 297 | + update_initramfs, |
| 298 | + ) # flake8: noqa -- ignore F401 for this import |
| 299 | + |
| 300 | +__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>" |
| 301 | |
| 302 | |
| 303 | def modprobe(module, persist=True): |
| 304 | @@ -32,11 +45,9 @@ |
| 305 | |
| 306 | log('Loading kernel module %s' % module, level=INFO) |
| 307 | |
| 308 | - check_call(cmd) |
| 309 | + subprocess.check_call(cmd) |
| 310 | if persist: |
| 311 | - with open('/etc/modules', 'r+') as modules: |
| 312 | - if module not in modules.read(): |
| 313 | - modules.write(module) |
| 314 | + persistent_modprobe(module) |
| 315 | |
| 316 | |
| 317 | def rmmod(module, force=False): |
| 318 | @@ -46,21 +57,16 @@ |
| 319 | cmd.append('-f') |
| 320 | cmd.append(module) |
| 321 | log('Removing kernel module %s' % module, level=INFO) |
| 322 | - return check_call(cmd) |
| 323 | + return subprocess.check_call(cmd) |
| 324 | |
| 325 | |
| 326 | def lsmod(): |
| 327 | """Shows what kernel modules are currently loaded""" |
| 328 | - return check_output(['lsmod'], |
| 329 | - universal_newlines=True) |
| 330 | + return subprocess.check_output(['lsmod'], |
| 331 | + universal_newlines=True) |
| 332 | |
| 333 | |
| 334 | def is_module_loaded(module): |
| 335 | """Checks if a kernel module is already loaded""" |
| 336 | matches = re.findall('^%s[ ]+' % module, lsmod(), re.M) |
| 337 | return len(matches) > 0 |
| 338 | - |
| 339 | - |
| 340 | -def update_initramfs(version='all'): |
| 341 | - """Updates an initramfs image""" |
| 342 | - return check_call(["update-initramfs", "-k", version, "-u"]) |
| 343 | |
| 344 | === added directory 'charmhelpers/core/kernel_factory' |
| 345 | === added file 'charmhelpers/core/kernel_factory/__init__.py' |
| 346 | === added file 'charmhelpers/core/kernel_factory/centos.py' |
| 347 | --- charmhelpers/core/kernel_factory/centos.py 1970-01-01 00:00:00 +0000 |
| 348 | +++ charmhelpers/core/kernel_factory/centos.py 2016-08-12 07:09:49 +0000 |
| 349 | @@ -0,0 +1,17 @@ |
| 350 | +import subprocess |
| 351 | +import os |
| 352 | + |
| 353 | + |
| 354 | +def persistent_modprobe(module): |
| 355 | + """Load a kernel module and configure for auto-load on reboot.""" |
| 356 | + if not os.path.exists('/etc/rc.modules'): |
| 357 | + open('/etc/rc.modules', 'a') |
| 358 | + os.chmod('/etc/rc.modules', 111) |
| 359 | + with open('/etc/rc.modules', 'r+') as modules: |
| 360 | + if module not in modules.read(): |
| 361 | + modules.write('modprobe %s\n' % module) |
| 362 | + |
| 363 | + |
| 364 | +def update_initramfs(version='all'): |
| 365 | + """Updates an initramfs image.""" |
| 366 | + return subprocess.check_call(["dracut", "-f", version]) |
| 367 | |
| 368 | === added file 'charmhelpers/core/kernel_factory/ubuntu.py' |
| 369 | --- charmhelpers/core/kernel_factory/ubuntu.py 1970-01-01 00:00:00 +0000 |
| 370 | +++ charmhelpers/core/kernel_factory/ubuntu.py 2016-08-12 07:09:49 +0000 |
| 371 | @@ -0,0 +1,13 @@ |
| 372 | +import subprocess |
| 373 | + |
| 374 | + |
| 375 | +def persistent_modprobe(module): |
| 376 | + """Load a kernel module and configure for auto-load on reboot.""" |
| 377 | + with open('/etc/modules', 'r+') as modules: |
| 378 | + if module not in modules.read(): |
| 379 | + modules.write(module) |
| 380 | + |
| 381 | + |
| 382 | +def update_initramfs(version='all'): |
| 383 | + """Updates an initramfs image.""" |
| 384 | + return subprocess.check_call(["update-initramfs", "-k", version, "-u"]) |
| 385 | |
| 386 | === modified file 'charmhelpers/fetch/__init__.py' |
| 387 | --- charmhelpers/fetch/__init__.py 2016-07-14 10:22:50 +0000 |
| 388 | +++ charmhelpers/fetch/__init__.py 2016-08-12 07:09:49 +0000 |
| 389 | @@ -13,18 +13,12 @@ |
| 390 | # limitations under the License. |
| 391 | |
| 392 | import importlib |
| 393 | -from tempfile import NamedTemporaryFile |
| 394 | -import time |
| 395 | +from charmhelpers.osplatform import get_platform |
| 396 | from yaml import safe_load |
| 397 | -from charmhelpers.core.host import ( |
| 398 | - lsb_release |
| 399 | -) |
| 400 | -import subprocess |
| 401 | from charmhelpers.core.hookenv import ( |
| 402 | config, |
| 403 | log, |
| 404 | ) |
| 405 | -import os |
| 406 | |
| 407 | import six |
| 408 | if six.PY3: |
| 409 | @@ -33,87 +27,6 @@ |
| 410 | from urlparse import urlparse, urlunparse |
| 411 | |
| 412 | |
| 413 | -CLOUD_ARCHIVE = """# Ubuntu Cloud Archive |
| 414 | -deb http://ubuntu-cloud.archive.canonical.com/ubuntu {} main |
| 415 | -""" |
| 416 | -PROPOSED_POCKET = """# Proposed |
| 417 | -deb http://archive.ubuntu.com/ubuntu {}-proposed main universe multiverse restricted |
| 418 | -""" |
| 419 | -CLOUD_ARCHIVE_POCKETS = { |
| 420 | - # Folsom |
| 421 | - 'folsom': 'precise-updates/folsom', |
| 422 | - 'precise-folsom': 'precise-updates/folsom', |
| 423 | - 'precise-folsom/updates': 'precise-updates/folsom', |
| 424 | - 'precise-updates/folsom': 'precise-updates/folsom', |
| 425 | - 'folsom/proposed': 'precise-proposed/folsom', |
| 426 | - 'precise-folsom/proposed': 'precise-proposed/folsom', |
| 427 | - 'precise-proposed/folsom': 'precise-proposed/folsom', |
| 428 | - # Grizzly |
| 429 | - 'grizzly': 'precise-updates/grizzly', |
| 430 | - 'precise-grizzly': 'precise-updates/grizzly', |
| 431 | - 'precise-grizzly/updates': 'precise-updates/grizzly', |
| 432 | - 'precise-updates/grizzly': 'precise-updates/grizzly', |
| 433 | - 'grizzly/proposed': 'precise-proposed/grizzly', |
| 434 | - 'precise-grizzly/proposed': 'precise-proposed/grizzly', |
| 435 | - 'precise-proposed/grizzly': 'precise-proposed/grizzly', |
| 436 | - # Havana |
| 437 | - 'havana': 'precise-updates/havana', |
| 438 | - 'precise-havana': 'precise-updates/havana', |
| 439 | - 'precise-havana/updates': 'precise-updates/havana', |
| 440 | - 'precise-updates/havana': 'precise-updates/havana', |
| 441 | - 'havana/proposed': 'precise-proposed/havana', |
| 442 | - 'precise-havana/proposed': 'precise-proposed/havana', |
| 443 | - 'precise-proposed/havana': 'precise-proposed/havana', |
| 444 | - # Icehouse |
| 445 | - 'icehouse': 'precise-updates/icehouse', |
| 446 | - 'precise-icehouse': 'precise-updates/icehouse', |
| 447 | - 'precise-icehouse/updates': 'precise-updates/icehouse', |
| 448 | - 'precise-updates/icehouse': 'precise-updates/icehouse', |
| 449 | - 'icehouse/proposed': 'precise-proposed/icehouse', |
| 450 | - 'precise-icehouse/proposed': 'precise-proposed/icehouse', |
| 451 | - 'precise-proposed/icehouse': 'precise-proposed/icehouse', |
| 452 | - # Juno |
| 453 | - 'juno': 'trusty-updates/juno', |
| 454 | - 'trusty-juno': 'trusty-updates/juno', |
| 455 | - 'trusty-juno/updates': 'trusty-updates/juno', |
| 456 | - 'trusty-updates/juno': 'trusty-updates/juno', |
| 457 | - 'juno/proposed': 'trusty-proposed/juno', |
| 458 | - 'trusty-juno/proposed': 'trusty-proposed/juno', |
| 459 | - 'trusty-proposed/juno': 'trusty-proposed/juno', |
| 460 | - # Kilo |
| 461 | - 'kilo': 'trusty-updates/kilo', |
| 462 | - 'trusty-kilo': 'trusty-updates/kilo', |
| 463 | - 'trusty-kilo/updates': 'trusty-updates/kilo', |
| 464 | - 'trusty-updates/kilo': 'trusty-updates/kilo', |
| 465 | - 'kilo/proposed': 'trusty-proposed/kilo', |
| 466 | - 'trusty-kilo/proposed': 'trusty-proposed/kilo', |
| 467 | - 'trusty-proposed/kilo': 'trusty-proposed/kilo', |
| 468 | - # Liberty |
| 469 | - 'liberty': 'trusty-updates/liberty', |
| 470 | - 'trusty-liberty': 'trusty-updates/liberty', |
| 471 | - 'trusty-liberty/updates': 'trusty-updates/liberty', |
| 472 | - 'trusty-updates/liberty': 'trusty-updates/liberty', |
| 473 | - 'liberty/proposed': 'trusty-proposed/liberty', |
| 474 | - 'trusty-liberty/proposed': 'trusty-proposed/liberty', |
| 475 | - 'trusty-proposed/liberty': 'trusty-proposed/liberty', |
| 476 | - # Mitaka |
| 477 | - 'mitaka': 'trusty-updates/mitaka', |
| 478 | - 'trusty-mitaka': 'trusty-updates/mitaka', |
| 479 | - 'trusty-mitaka/updates': 'trusty-updates/mitaka', |
| 480 | - 'trusty-updates/mitaka': 'trusty-updates/mitaka', |
| 481 | - 'mitaka/proposed': 'trusty-proposed/mitaka', |
| 482 | - 'trusty-mitaka/proposed': 'trusty-proposed/mitaka', |
| 483 | - 'trusty-proposed/mitaka': 'trusty-proposed/mitaka', |
| 484 | - # Newton |
| 485 | - 'newton': 'xenial-updates/newton', |
| 486 | - 'xenial-newton': 'xenial-updates/newton', |
| 487 | - 'xenial-newton/updates': 'xenial-updates/newton', |
| 488 | - 'xenial-updates/newton': 'xenial-updates/newton', |
| 489 | - 'newton/proposed': 'xenial-proposed/newton', |
| 490 | - 'xenial-newton/proposed': 'xenial-proposed/newton', |
| 491 | - 'xenial-proposed/newton': 'xenial-proposed/newton', |
| 492 | -} |
| 493 | - |
| 494 | # The order of this list is very important. Handlers should be listed in from |
| 495 | # least- to most-specific URL matching. |
| 496 | FETCH_HANDLERS = ( |
| 497 | @@ -122,10 +35,6 @@ |
| 498 | 'charmhelpers.fetch.giturl.GitUrlFetchHandler', |
| 499 | ) |
| 500 | |
| 501 | -APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. |
| 502 | -APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. |
| 503 | -APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. |
| 504 | - |
| 505 | |
| 506 | class SourceConfigError(Exception): |
| 507 | pass |
| 508 | @@ -163,180 +72,37 @@ |
| 509 | return urlunparse(parts) |
| 510 | |
| 511 | |
| 512 | -def filter_installed_packages(packages): |
| 513 | - """Returns a list of packages that require installation""" |
| 514 | - cache = apt_cache() |
| 515 | - _pkgs = [] |
| 516 | - for package in packages: |
| 517 | - try: |
| 518 | - p = cache[package] |
| 519 | - p.current_ver or _pkgs.append(package) |
| 520 | - except KeyError: |
| 521 | - log('Package {} has no installation candidate.'.format(package), |
| 522 | - level='WARNING') |
| 523 | - _pkgs.append(package) |
| 524 | - return _pkgs |
| 525 | - |
| 526 | - |
| 527 | -def apt_cache(in_memory=True, progress=None): |
| 528 | - """Build and return an apt cache""" |
| 529 | - from apt import apt_pkg |
| 530 | - apt_pkg.init() |
| 531 | - if in_memory: |
| 532 | - apt_pkg.config.set("Dir::Cache::pkgcache", "") |
| 533 | - apt_pkg.config.set("Dir::Cache::srcpkgcache", "") |
| 534 | - return apt_pkg.Cache(progress) |
| 535 | - |
| 536 | - |
| 537 | -def apt_install(packages, options=None, fatal=False): |
| 538 | - """Install one or more packages""" |
| 539 | - if options is None: |
| 540 | - options = ['--option=Dpkg::Options::=--force-confold'] |
| 541 | - |
| 542 | - cmd = ['apt-get', '--assume-yes'] |
| 543 | - cmd.extend(options) |
| 544 | - cmd.append('install') |
| 545 | - if isinstance(packages, six.string_types): |
| 546 | - cmd.append(packages) |
| 547 | - else: |
| 548 | - cmd.extend(packages) |
| 549 | - log("Installing {} with options: {}".format(packages, |
| 550 | - options)) |
| 551 | - _run_apt_command(cmd, fatal) |
| 552 | - |
| 553 | - |
| 554 | -def apt_upgrade(options=None, fatal=False, dist=False): |
| 555 | - """Upgrade all packages""" |
| 556 | - if options is None: |
| 557 | - options = ['--option=Dpkg::Options::=--force-confold'] |
| 558 | - |
| 559 | - cmd = ['apt-get', '--assume-yes'] |
| 560 | - cmd.extend(options) |
| 561 | - if dist: |
| 562 | - cmd.append('dist-upgrade') |
| 563 | - else: |
| 564 | - cmd.append('upgrade') |
| 565 | - log("Upgrading with options: {}".format(options)) |
| 566 | - _run_apt_command(cmd, fatal) |
| 567 | - |
| 568 | - |
| 569 | -def apt_update(fatal=False): |
| 570 | - """Update local apt cache""" |
| 571 | - cmd = ['apt-get', 'update'] |
| 572 | - _run_apt_command(cmd, fatal) |
| 573 | - |
| 574 | - |
| 575 | -def apt_purge(packages, fatal=False): |
| 576 | - """Purge one or more packages""" |
| 577 | - cmd = ['apt-get', '--assume-yes', 'purge'] |
| 578 | - if isinstance(packages, six.string_types): |
| 579 | - cmd.append(packages) |
| 580 | - else: |
| 581 | - cmd.extend(packages) |
| 582 | - log("Purging {}".format(packages)) |
| 583 | - _run_apt_command(cmd, fatal) |
| 584 | - |
| 585 | - |
| 586 | -def apt_mark(packages, mark, fatal=False): |
| 587 | - """Flag one or more packages using apt-mark""" |
| 588 | - log("Marking {} as {}".format(packages, mark)) |
| 589 | - cmd = ['apt-mark', mark] |
| 590 | - if isinstance(packages, six.string_types): |
| 591 | - cmd.append(packages) |
| 592 | - else: |
| 593 | - cmd.extend(packages) |
| 594 | - |
| 595 | - if fatal: |
| 596 | - subprocess.check_call(cmd, universal_newlines=True) |
| 597 | - else: |
| 598 | - subprocess.call(cmd, universal_newlines=True) |
| 599 | - |
| 600 | - |
| 601 | -def apt_hold(packages, fatal=False): |
| 602 | - return apt_mark(packages, 'hold', fatal=fatal) |
| 603 | - |
| 604 | - |
| 605 | -def apt_unhold(packages, fatal=False): |
| 606 | - return apt_mark(packages, 'unhold', fatal=fatal) |
| 607 | - |
| 608 | - |
| 609 | -def add_source(source, key=None): |
| 610 | - """Add a package source to this system. |
| 611 | - |
| 612 | - @param source: a URL or sources.list entry, as supported by |
| 613 | - add-apt-repository(1). Examples:: |
| 614 | - |
| 615 | - ppa:charmers/example |
| 616 | - deb https://stub:key@private.example.com/ubuntu trusty main |
| 617 | - |
| 618 | - In addition: |
| 619 | - 'proposed:' may be used to enable the standard 'proposed' |
| 620 | - pocket for the release. |
| 621 | - 'cloud:' may be used to activate official cloud archive pockets, |
| 622 | - such as 'cloud:icehouse' |
| 623 | - 'distro' may be used as a noop |
| 624 | - |
| 625 | - @param key: A key to be added to the system's APT keyring and used |
| 626 | - to verify the signatures on packages. Ideally, this should be an |
| 627 | - ASCII format GPG public key including the block headers. A GPG key |
| 628 | - id may also be used, but be aware that only insecure protocols are |
| 629 | - available to retrieve the actual public key from a public keyserver |
| 630 | - placing your Juju environment at risk. ppa and cloud archive keys |
| 631 | - are securely added automtically, so sould not be provided. |
| 632 | - """ |
| 633 | - if source is None: |
| 634 | - log('Source is not present. Skipping') |
| 635 | - return |
| 636 | - |
| 637 | - if (source.startswith('ppa:') or |
| 638 | - source.startswith('http') or |
| 639 | - source.startswith('deb ') or |
| 640 | - source.startswith('cloud-archive:')): |
| 641 | - subprocess.check_call(['add-apt-repository', '--yes', source]) |
| 642 | - elif source.startswith('cloud:'): |
| 643 | - apt_install(filter_installed_packages(['ubuntu-cloud-keyring']), |
| 644 | - fatal=True) |
| 645 | - pocket = source.split(':')[-1] |
| 646 | - if pocket not in CLOUD_ARCHIVE_POCKETS: |
| 647 | - raise SourceConfigError( |
| 648 | - 'Unsupported cloud: source option %s' % |
| 649 | - pocket) |
| 650 | - actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] |
| 651 | - with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: |
| 652 | - apt.write(CLOUD_ARCHIVE.format(actual_pocket)) |
| 653 | - elif source == 'proposed': |
| 654 | - release = lsb_release()['DISTRIB_CODENAME'] |
| 655 | - with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: |
| 656 | - apt.write(PROPOSED_POCKET.format(release)) |
| 657 | - elif source == 'distro': |
| 658 | - pass |
| 659 | - else: |
| 660 | - log("Unknown source: {!r}".format(source)) |
| 661 | - |
| 662 | - if key: |
| 663 | - if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: |
| 664 | - with NamedTemporaryFile('w+') as key_file: |
| 665 | - key_file.write(key) |
| 666 | - key_file.flush() |
| 667 | - key_file.seek(0) |
| 668 | - subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) |
| 669 | - else: |
| 670 | - # Note that hkp: is in no way a secure protocol. Using a |
| 671 | - # GPG key id is pointless from a security POV unless you |
| 672 | - # absolutely trust your network and DNS. |
| 673 | - subprocess.check_call(['apt-key', 'adv', '--keyserver', |
| 674 | - 'hkp://keyserver.ubuntu.com:80', '--recv', |
| 675 | - key]) |
| 676 | +__platform__ = get_platform() |
| 677 | +module = "charmhelpers.fetch.%s" % __platform__ |
| 678 | +fetch = importlib.import_module(module) |
| 679 | + |
| 680 | +filter_installed_packages = fetch.filter_installed_packages |
| 681 | +install = fetch.install |
| 682 | +upgrade = fetch.upgrade |
| 683 | +update = fetch.update |
| 684 | +purge = fetch.purge |
| 685 | +add_source = fetch.add_source |
| 686 | + |
| 687 | +if __platform__ == "ubuntu": |
| 688 | + apt_cache = fetch.apt_cache |
| 689 | + apt_install = fetch.install |
| 690 | + apt_update = fetch.update |
| 691 | + apt_upgrade = fetch.upgrade |
| 692 | + apt_purge = fetch.purge |
| 693 | + apt_mark = fetch.apt_mark |
| 694 | + apt_hold = fetch.apt_hold |
| 695 | + apt_unhold = fetch.apt_unhold |
| 696 | +elif __platform__ == "centos": |
| 697 | + yum_search = fetch.yum_search |
| 698 | |
| 699 | |
| 700 | def configure_sources(update=False, |
| 701 | sources_var='install_sources', |
| 702 | keys_var='install_keys'): |
| 703 | - """ |
| 704 | - Configure multiple sources from charm configuration. |
| 705 | + """Configure multiple sources from charm configuration. |
| 706 | |
| 707 | The lists are encoded as yaml fragments in the configuration. |
| 708 | - The frament needs to be included as a string. Sources and their |
| 709 | + The fragment needs to be included as a string. Sources and their |
| 710 | corresponding keys are of the types supported by add_source(). |
| 711 | |
| 712 | Example config: |
| 713 | @@ -368,12 +134,11 @@ |
| 714 | for source, key in zip(sources, keys): |
| 715 | add_source(source, key) |
| 716 | if update: |
| 717 | - apt_update(fatal=True) |
| 718 | + fetch.update(fatal=True) |
| 719 | |
| 720 | |
| 721 | def install_remote(source, *args, **kwargs): |
| 722 | - """ |
| 723 | - Install a file tree from a remote source |
| 724 | + """Install a file tree from a remote source. |
| 725 | |
| 726 | The specified source should be a url of the form: |
| 727 | scheme://[host]/path[#[option=value][&...]] |
| 728 | @@ -406,6 +171,7 @@ |
| 729 | |
| 730 | |
| 731 | def install_from_config(config_var_name): |
| 732 | + """Install a file from config.""" |
| 733 | charm_config = config() |
| 734 | source = charm_config[config_var_name] |
| 735 | return install_remote(source) |
| 736 | @@ -428,40 +194,3 @@ |
| 737 | log("FetchHandler {} not found, skipping plugin".format( |
| 738 | handler_name)) |
| 739 | return plugin_list |
| 740 | - |
| 741 | - |
| 742 | -def _run_apt_command(cmd, fatal=False): |
| 743 | - """ |
| 744 | - Run an APT command, checking output and retrying if the fatal flag is set |
| 745 | - to True. |
| 746 | - |
| 747 | - :param: cmd: str: The apt command to run. |
| 748 | - :param: fatal: bool: Whether the command's output should be checked and |
| 749 | - retried. |
| 750 | - """ |
| 751 | - env = os.environ.copy() |
| 752 | - |
| 753 | - if 'DEBIAN_FRONTEND' not in env: |
| 754 | - env['DEBIAN_FRONTEND'] = 'noninteractive' |
| 755 | - |
| 756 | - if fatal: |
| 757 | - retry_count = 0 |
| 758 | - result = None |
| 759 | - |
| 760 | - # If the command is considered "fatal", we need to retry if the apt |
| 761 | - # lock was not acquired. |
| 762 | - |
| 763 | - while result is None or result == APT_NO_LOCK: |
| 764 | - try: |
| 765 | - result = subprocess.check_call(cmd, env=env) |
| 766 | - except subprocess.CalledProcessError as e: |
| 767 | - retry_count = retry_count + 1 |
| 768 | - if retry_count > APT_NO_LOCK_RETRY_COUNT: |
| 769 | - raise |
| 770 | - result = e.returncode |
| 771 | - log("Couldn't acquire DPKG lock. Will retry in {} seconds." |
| 772 | - "".format(APT_NO_LOCK_RETRY_DELAY)) |
| 773 | - time.sleep(APT_NO_LOCK_RETRY_DELAY) |
| 774 | - |
| 775 | - else: |
| 776 | - subprocess.call(cmd, env=env) |
| 777 | |
| 778 | === modified file 'charmhelpers/fetch/bzrurl.py' |
| 779 | --- charmhelpers/fetch/bzrurl.py 2016-07-06 14:41:05 +0000 |
| 780 | +++ charmhelpers/fetch/bzrurl.py 2016-08-12 07:09:49 +0000 |
| 781 | @@ -18,19 +18,20 @@ |
| 782 | BaseFetchHandler, |
| 783 | UnhandledSource, |
| 784 | filter_installed_packages, |
| 785 | - apt_install, |
| 786 | + install, |
| 787 | ) |
| 788 | from charmhelpers.core.host import mkdir |
| 789 | |
| 790 | |
| 791 | if filter_installed_packages(['bzr']) != []: |
| 792 | - apt_install(['bzr']) |
| 793 | + install(['bzr']) |
| 794 | if filter_installed_packages(['bzr']) != []: |
| 795 | raise NotImplementedError('Unable to install bzr') |
| 796 | |
| 797 | |
| 798 | class BzrUrlFetchHandler(BaseFetchHandler): |
| 799 | - """Handler for bazaar branches via generic and lp URLs""" |
| 800 | + """Handler for bazaar branches via generic and lp URLs.""" |
| 801 | + |
| 802 | def can_handle(self, source): |
| 803 | url_parts = self.parse_url(source) |
| 804 | if url_parts.scheme not in ('bzr+ssh', 'lp', ''): |
| 805 | |
| 806 | === added file 'charmhelpers/fetch/centos.py' |
| 807 | --- charmhelpers/fetch/centos.py 1970-01-01 00:00:00 +0000 |
| 808 | +++ charmhelpers/fetch/centos.py 2016-08-12 07:09:49 +0000 |
| 809 | @@ -0,0 +1,171 @@ |
| 810 | +# Copyright 2014-2015 Canonical Limited. |
| 811 | +# |
| 812 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
| 813 | +# you may not use this file except in compliance with the License. |
| 814 | +# You may obtain a copy of the License at |
| 815 | +# |
| 816 | +# http://www.apache.org/licenses/LICENSE-2.0 |
| 817 | +# |
| 818 | +# Unless required by applicable law or agreed to in writing, software |
| 819 | +# distributed under the License is distributed on an "AS IS" BASIS, |
| 820 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 821 | +# See the License for the specific language governing permissions and |
| 822 | +# limitations under the License. |
| 823 | + |
| 824 | +import subprocess |
| 825 | +import os |
| 826 | +import time |
| 827 | +import six |
| 828 | +import yum |
| 829 | + |
| 830 | +from tempfile import NamedTemporaryFile |
| 831 | +from charmhelpers.core.hookenv import log |
| 832 | + |
| 833 | +YUM_NO_LOCK = 1 # The return code for "couldn't acquire lock" in YUM. |
| 834 | +YUM_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. |
| 835 | +YUM_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. |
| 836 | + |
| 837 | + |
| 838 | +def filter_installed_packages(packages): |
| 839 | + """Return a list of packages that require installation.""" |
| 840 | + yb = yum.YumBase() |
| 841 | + package_list = yb.doPackageLists() |
| 842 | + temp_cache = {p.base_package_name: 1 for p in package_list['installed']} |
| 843 | + |
| 844 | + _pkgs = [p for p in packages if not temp_cache.get(p, False)] |
| 845 | + return _pkgs |
| 846 | + |
| 847 | + |
| 848 | +def install(packages, options=None, fatal=False): |
| 849 | + """Install one or more packages.""" |
| 850 | + cmd = ['yum', '--assumeyes'] |
| 851 | + if options is not None: |
| 852 | + cmd.extend(options) |
| 853 | + cmd.append('install') |
| 854 | + if isinstance(packages, six.string_types): |
| 855 | + cmd.append(packages) |
| 856 | + else: |
| 857 | + cmd.extend(packages) |
| 858 | + log("Installing {} with options: {}".format(packages, |
| 859 | + options)) |
| 860 | + _run_yum_command(cmd, fatal) |
| 861 | + |
| 862 | + |
| 863 | +def upgrade(options=None, fatal=False, dist=False): |
| 864 | + """Upgrade all packages.""" |
| 865 | + cmd = ['yum', '--assumeyes'] |
| 866 | + if options is not None: |
| 867 | + cmd.extend(options) |
| 868 | + cmd.append('upgrade') |
| 869 | + log("Upgrading with options: {}".format(options)) |
| 870 | + _run_yum_command(cmd, fatal) |
| 871 | + |
| 872 | + |
| 873 | +def update(fatal=False): |
| 874 | + """Update local yum cache.""" |
| 875 | + cmd = ['yum', '--assumeyes', 'update'] |
| 876 | + log("Update with fatal: {}".format(fatal)) |
| 877 | + _run_yum_command(cmd, fatal) |
| 878 | + |
| 879 | + |
| 880 | +def purge(packages, fatal=False): |
| 881 | + """Purge one or more packages.""" |
| 882 | + cmd = ['yum', '--assumeyes', 'remove'] |
| 883 | + if isinstance(packages, six.string_types): |
| 884 | + cmd.append(packages) |
| 885 | + else: |
| 886 | + cmd.extend(packages) |
| 887 | + log("Purging {}".format(packages)) |
| 888 | + _run_yum_command(cmd, fatal) |
| 889 | + |
| 890 | + |
| 891 | +def yum_search(packages): |
| 892 | + """Search for a package.""" |
| 893 | + output = {} |
| 894 | + cmd = ['yum', 'search'] |
| 895 | + if isinstance(packages, six.string_types): |
| 896 | + cmd.append(packages) |
| 897 | + else: |
| 898 | + cmd.extend(packages) |
| 899 | + log("Searching for {}".format(packages)) |
| 900 | + result = subprocess.check_output(cmd) |
| 901 | + for package in list(packages): |
| 902 | + output[package] = package in result |
| 903 | + return output |
| 904 | + |
| 905 | + |
| 906 | +def add_source(source, key=None): |
| 907 | + """Add a package source to this system. |
| 908 | + |
| 909 | + @param source: a URL with a rpm package |
| 910 | + |
| 911 | + @param key: A key to be added to the system's keyring and used |
| 912 | + to verify the signatures on packages. Ideally, this should be an |
| 913 | + ASCII format GPG public key including the block headers. A GPG key |
| 914 | + id may also be used, but be aware that only insecure protocols are |
| 915 | + available to retrieve the actual public key from a public keyserver |
| 916 | + placing your Juju environment at risk. |
| 917 | + """ |
| 918 | + if source is None: |
| 919 | + log('Source is not present. Skipping') |
| 920 | + return |
| 921 | + |
| 922 | + if source.startswith('http'): |
| 923 | + directory = '/etc/yum.repos.d/' |
| 924 | + for filename in os.listdir(directory): |
| 925 | + with open(directory + filename, 'r') as rpm_file: |
| 926 | + if source in rpm_file.read(): |
| 927 | + break |
| 928 | + else: |
| 929 | + log("Add source: {!r}".format(source)) |
| 930 | + # write in the charms.repo |
| 931 | + with open(directory + 'Charms.repo', 'a') as rpm_file: |
| 932 | + rpm_file.write('[%s]\n' % source[7:].replace('/', '_')) |
| 933 | + rpm_file.write('name=%s\n' % source[7:]) |
| 934 | + rpm_file.write('baseurl=%s\n\n' % source) |
| 935 | + else: |
| 936 | + log("Unknown source: {!r}".format(source)) |
| 937 | + |
| 938 | + if key: |
| 939 | + if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: |
| 940 | + with NamedTemporaryFile('w+') as key_file: |
| 941 | + key_file.write(key) |
| 942 | + key_file.flush() |
| 943 | + key_file.seek(0) |
| 944 | + subprocess.check_call(['rpm', '--import', key_file]) |
| 945 | + else: |
| 946 | + subprocess.check_call(['rpm', '--import', key]) |
| 947 | + |
| 948 | + |
| 949 | +def _run_yum_command(cmd, fatal=False): |
| 950 | + """Run an YUM command. |
| 951 | + |
| 952 | + Checks the output and retry if the fatal flag is set to True. |
| 953 | + |
| 954 | + :param: cmd: str: The yum command to run. |
| 955 | + :param: fatal: bool: Whether the command's output should be checked and |
| 956 | + retried. |
| 957 | + """ |
| 958 | + env = os.environ.copy() |
| 959 | + |
| 960 | + if fatal: |
| 961 | + retry_count = 0 |
| 962 | + result = None |
| 963 | + |
| 964 | + # If the command is considered "fatal", we need to retry if the yum |
| 965 | + # lock was not acquired. |
| 966 | + |
| 967 | + while result is None or result == YUM_NO_LOCK: |
| 968 | + try: |
| 969 | + result = subprocess.check_call(cmd, env=env) |
| 970 | + except subprocess.CalledProcessError as e: |
| 971 | + retry_count = retry_count + 1 |
| 972 | + if retry_count > YUM_NO_LOCK_RETRY_COUNT: |
| 973 | + raise |
| 974 | + result = e.returncode |
| 975 | + log("Couldn't acquire YUM lock. Will retry in {} seconds." |
| 976 | + "".format(YUM_NO_LOCK_RETRY_DELAY)) |
| 977 | + time.sleep(YUM_NO_LOCK_RETRY_DELAY) |
| 978 | + |
| 979 | + else: |
| 980 | + subprocess.call(cmd, env=env) |
| 981 | |
| 982 | === modified file 'charmhelpers/fetch/giturl.py' |
| 983 | --- charmhelpers/fetch/giturl.py 2016-07-06 14:41:05 +0000 |
| 984 | +++ charmhelpers/fetch/giturl.py 2016-08-12 07:09:49 +0000 |
| 985 | @@ -18,17 +18,18 @@ |
| 986 | BaseFetchHandler, |
| 987 | UnhandledSource, |
| 988 | filter_installed_packages, |
| 989 | - apt_install, |
| 990 | + install, |
| 991 | ) |
| 992 | |
| 993 | if filter_installed_packages(['git']) != []: |
| 994 | - apt_install(['git']) |
| 995 | + install(['git']) |
| 996 | if filter_installed_packages(['git']) != []: |
| 997 | raise NotImplementedError('Unable to install git') |
| 998 | |
| 999 | |
| 1000 | class GitUrlFetchHandler(BaseFetchHandler): |
| 1001 | - """Handler for git branches via generic and github URLs""" |
| 1002 | + """Handler for git branches via generic and github URLs.""" |
| 1003 | + |
| 1004 | def can_handle(self, source): |
| 1005 | url_parts = self.parse_url(source) |
| 1006 | # TODO (mattyw) no support for ssh git@ yet |
| 1007 | |
| 1008 | === added file 'charmhelpers/fetch/ubuntu.py' |
| 1009 | --- charmhelpers/fetch/ubuntu.py 1970-01-01 00:00:00 +0000 |
| 1010 | +++ charmhelpers/fetch/ubuntu.py 2016-08-12 07:09:49 +0000 |
| 1011 | @@ -0,0 +1,313 @@ |
| 1012 | +# Copyright 2014-2015 Canonical Limited. |
| 1013 | +# |
| 1014 | +# Licensed under the Apache License, Version 2.0 (the "License"); |
| 1015 | +# you may not use this file except in compliance with the License. |
| 1016 | +# You may obtain a copy of the License at |
| 1017 | +# |
| 1018 | +# http://www.apache.org/licenses/LICENSE-2.0 |
| 1019 | +# |
| 1020 | +# Unless required by applicable law or agreed to in writing, software |
| 1021 | +# distributed under the License is distributed on an "AS IS" BASIS, |
| 1022 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 1023 | +# See the License for the specific language governing permissions and |
| 1024 | +# limitations under the License. |
| 1025 | + |
| 1026 | +import os |
| 1027 | +import six |
| 1028 | +import time |
| 1029 | +import subprocess |
| 1030 | + |
| 1031 | +from tempfile import NamedTemporaryFile |
| 1032 | +from charmhelpers.core.host import ( |
| 1033 | + lsb_release |
| 1034 | +) |
| 1035 | +from charmhelpers.core.hookenv import log |
| 1036 | +from charmhelpers.fetch import SourceConfigError |
| 1037 | + |
| 1038 | +CLOUD_ARCHIVE = ('# Ubuntu Cloud Archive deb' |
| 1039 | + ' http://ubuntu-cloud.archive.canonical.com/ubuntu' |
| 1040 | + ' {} main') |
| 1041 | +PROPOSED_POCKET = ('# Proposed deb http://archive.ubuntu.com/ubuntu' |
| 1042 | + ' {}-proposed main universe multiverse restricted') |
| 1043 | +CLOUD_ARCHIVE_POCKETS = { |
| 1044 | + # Folsom |
| 1045 | + 'folsom': 'precise-updates/folsom', |
| 1046 | + 'precise-folsom': 'precise-updates/folsom', |
| 1047 | + 'precise-folsom/updates': 'precise-updates/folsom', |
| 1048 | + 'precise-updates/folsom': 'precise-updates/folsom', |
| 1049 | + 'folsom/proposed': 'precise-proposed/folsom', |
| 1050 | + 'precise-folsom/proposed': 'precise-proposed/folsom', |
| 1051 | + 'precise-proposed/folsom': 'precise-proposed/folsom', |
| 1052 | + # Grizzly |
| 1053 | + 'grizzly': 'precise-updates/grizzly', |
| 1054 | + 'precise-grizzly': 'precise-updates/grizzly', |
| 1055 | + 'precise-grizzly/updates': 'precise-updates/grizzly', |
| 1056 | + 'precise-updates/grizzly': 'precise-updates/grizzly', |
| 1057 | + 'grizzly/proposed': 'precise-proposed/grizzly', |
| 1058 | + 'precise-grizzly/proposed': 'precise-proposed/grizzly', |
| 1059 | + 'precise-proposed/grizzly': 'precise-proposed/grizzly', |
| 1060 | + # Havana |
| 1061 | + 'havana': 'precise-updates/havana', |
| 1062 | + 'precise-havana': 'precise-updates/havana', |
| 1063 | + 'precise-havana/updates': 'precise-updates/havana', |
| 1064 | + 'precise-updates/havana': 'precise-updates/havana', |
| 1065 | + 'havana/proposed': 'precise-proposed/havana', |
| 1066 | + 'precise-havana/proposed': 'precise-proposed/havana', |
| 1067 | + 'precise-proposed/havana': 'precise-proposed/havana', |
| 1068 | + # Icehouse |
| 1069 | + 'icehouse': 'precise-updates/icehouse', |
| 1070 | + 'precise-icehouse': 'precise-updates/icehouse', |
| 1071 | + 'precise-icehouse/updates': 'precise-updates/icehouse', |
| 1072 | + 'precise-updates/icehouse': 'precise-updates/icehouse', |
| 1073 | + 'icehouse/proposed': 'precise-proposed/icehouse', |
| 1074 | + 'precise-icehouse/proposed': 'precise-proposed/icehouse', |
| 1075 | + 'precise-proposed/icehouse': 'precise-proposed/icehouse', |
| 1076 | + # Juno |
| 1077 | + 'juno': 'trusty-updates/juno', |
| 1078 | + 'trusty-juno': 'trusty-updates/juno', |
| 1079 | + 'trusty-juno/updates': 'trusty-updates/juno', |
| 1080 | + 'trusty-updates/juno': 'trusty-updates/juno', |
| 1081 | + 'juno/proposed': 'trusty-proposed/juno', |
| 1082 | + 'trusty-juno/proposed': 'trusty-proposed/juno', |
| 1083 | + 'trusty-proposed/juno': 'trusty-proposed/juno', |
| 1084 | + # Kilo |
| 1085 | + 'kilo': 'trusty-updates/kilo', |
| 1086 | + 'trusty-kilo': 'trusty-updates/kilo', |
| 1087 | + 'trusty-kilo/updates': 'trusty-updates/kilo', |
| 1088 | + 'trusty-updates/kilo': 'trusty-updates/kilo', |
| 1089 | + 'kilo/proposed': 'trusty-proposed/kilo', |
| 1090 | + 'trusty-kilo/proposed': 'trusty-proposed/kilo', |
| 1091 | + 'trusty-proposed/kilo': 'trusty-proposed/kilo', |
| 1092 | + # Liberty |
| 1093 | + 'liberty': 'trusty-updates/liberty', |
| 1094 | + 'trusty-liberty': 'trusty-updates/liberty', |
| 1095 | + 'trusty-liberty/updates': 'trusty-updates/liberty', |
| 1096 | + 'trusty-updates/liberty': 'trusty-updates/liberty', |
| 1097 | + 'liberty/proposed': 'trusty-proposed/liberty', |
| 1098 | + 'trusty-liberty/proposed': 'trusty-proposed/liberty', |
| 1099 | + 'trusty-proposed/liberty': 'trusty-proposed/liberty', |
| 1100 | + # Mitaka |
| 1101 | + 'mitaka': 'trusty-updates/mitaka', |
| 1102 | + 'trusty-mitaka': 'trusty-updates/mitaka', |
| 1103 | + 'trusty-mitaka/updates': 'trusty-updates/mitaka', |
| 1104 | + 'trusty-updates/mitaka': 'trusty-updates/mitaka', |
| 1105 | + 'mitaka/proposed': 'trusty-proposed/mitaka', |
| 1106 | + 'trusty-mitaka/proposed': 'trusty-proposed/mitaka', |
| 1107 | + 'trusty-proposed/mitaka': 'trusty-proposed/mitaka', |
| 1108 | + # Newton |
| 1109 | + 'newton': 'xenial-updates/newton', |
| 1110 | + 'xenial-newton': 'xenial-updates/newton', |
| 1111 | + 'xenial-newton/updates': 'xenial-updates/newton', |
| 1112 | + 'xenial-updates/newton': 'xenial-updates/newton', |
| 1113 | + 'newton/proposed': 'xenial-proposed/newton', |
| 1114 | + 'xenial-newton/proposed': 'xenial-proposed/newton', |
| 1115 | + 'xenial-proposed/newton': 'xenial-proposed/newton', |
| 1116 | +} |
| 1117 | + |
| 1118 | +APT_NO_LOCK = 100 # The return code for "couldn't acquire lock" in APT. |
| 1119 | +APT_NO_LOCK_RETRY_DELAY = 10 # Wait 10 seconds between apt lock checks. |
| 1120 | +APT_NO_LOCK_RETRY_COUNT = 30 # Retry to acquire the lock X times. |
| 1121 | + |
| 1122 | + |
| 1123 | +def filter_installed_packages(packages): |
| 1124 | + """Return a list of packages that require installation.""" |
| 1125 | + cache = apt_cache() |
| 1126 | + _pkgs = [] |
| 1127 | + for package in packages: |
| 1128 | + try: |
| 1129 | + p = cache[package] |
| 1130 | + p.current_ver or _pkgs.append(package) |
| 1131 | + except KeyError: |
| 1132 | + log('Package {} has no installation candidate.'.format(package), |
| 1133 | + level='WARNING') |
| 1134 | + _pkgs.append(package) |
| 1135 | + return _pkgs |
| 1136 | + |
| 1137 | + |
| 1138 | +def apt_cache(in_memory=True, progress=None): |
| 1139 | + """Build and return an apt cache.""" |
| 1140 | + from apt import apt_pkg |
| 1141 | + apt_pkg.init() |
| 1142 | + if in_memory: |
| 1143 | + apt_pkg.config.set("Dir::Cache::pkgcache", "") |
| 1144 | + apt_pkg.config.set("Dir::Cache::srcpkgcache", "") |
| 1145 | + return apt_pkg.Cache(progress) |
| 1146 | + |
| 1147 | + |
| 1148 | +def install(packages, options=None, fatal=False): |
| 1149 | + """Install one or more packages.""" |
| 1150 | + if options is None: |
| 1151 | + options = ['--option=Dpkg::Options::=--force-confold'] |
| 1152 | + |
| 1153 | + cmd = ['apt-get', '--assume-yes'] |
| 1154 | + cmd.extend(options) |
| 1155 | + cmd.append('install') |
| 1156 | + if isinstance(packages, six.string_types): |
| 1157 | + cmd.append(packages) |
| 1158 | + else: |
| 1159 | + cmd.extend(packages) |
| 1160 | + log("Installing {} with options: {}".format(packages, |
| 1161 | + options)) |
| 1162 | + _run_apt_command(cmd, fatal) |
| 1163 | + |
| 1164 | + |
| 1165 | +def upgrade(options=None, fatal=False, dist=False): |
| 1166 | + """Upgrade all packages.""" |
| 1167 | + if options is None: |
| 1168 | + options = ['--option=Dpkg::Options::=--force-confold'] |
| 1169 | + |
| 1170 | + cmd = ['apt-get', '--assume-yes'] |
| 1171 | + cmd.extend(options) |
| 1172 | + if dist: |
| 1173 | + cmd.append('dist-upgrade') |
| 1174 | + else: |
| 1175 | + cmd.append('upgrade') |
| 1176 | + log("Upgrading with options: {}".format(options)) |
| 1177 | + _run_apt_command(cmd, fatal) |
| 1178 | + |
| 1179 | + |
| 1180 | +def update(fatal=False): |
| 1181 | + """Update local apt cache.""" |
| 1182 | + cmd = ['apt-get', 'update'] |
| 1183 | + _run_apt_command(cmd, fatal) |
| 1184 | + |
| 1185 | + |
| 1186 | +def purge(packages, fatal=False): |
| 1187 | + """Purge one or more packages.""" |
| 1188 | + cmd = ['apt-get', '--assume-yes', 'purge'] |
| 1189 | + if isinstance(packages, six.string_types): |
| 1190 | + cmd.append(packages) |
| 1191 | + else: |
| 1192 | + cmd.extend(packages) |
| 1193 | + log("Purging {}".format(packages)) |
| 1194 | + _run_apt_command(cmd, fatal) |
| 1195 | + |
| 1196 | + |
| 1197 | +def apt_mark(packages, mark, fatal=False): |
| 1198 | + """Flag one or more packages using apt-mark.""" |
| 1199 | + log("Marking {} as {}".format(packages, mark)) |
| 1200 | + cmd = ['apt-mark', mark] |
| 1201 | + if isinstance(packages, six.string_types): |
| 1202 | + cmd.append(packages) |
| 1203 | + else: |
| 1204 | + cmd.extend(packages) |
| 1205 | + |
| 1206 | + if fatal: |
| 1207 | + subprocess.check_call(cmd, universal_newlines=True) |
| 1208 | + else: |
| 1209 | + subprocess.call(cmd, universal_newlines=True) |
| 1210 | + |
| 1211 | + |
| 1212 | +def apt_hold(packages, fatal=False): |
| 1213 | + return apt_mark(packages, 'hold', fatal=fatal) |
| 1214 | + |
| 1215 | + |
| 1216 | +def apt_unhold(packages, fatal=False): |
| 1217 | + return apt_mark(packages, 'unhold', fatal=fatal) |
| 1218 | + |
| 1219 | + |
| 1220 | +def add_source(source, key=None): |
| 1221 | + """Add a package source to this system. |
| 1222 | + |
| 1223 | + @param source: a URL or sources.list entry, as supported by |
| 1224 | + add-apt-repository(1). Examples:: |
| 1225 | + |
| 1226 | + ppa:charmers/example |
| 1227 | + deb https://stub:key@private.example.com/ubuntu trusty main |
| 1228 | + |
| 1229 | + In addition: |
| 1230 | + 'proposed:' may be used to enable the standard 'proposed' |
| 1231 | + pocket for the release. |
| 1232 | + 'cloud:' may be used to activate official cloud archive pockets, |
| 1233 | + such as 'cloud:icehouse' |
| 1234 | + 'distro' may be used as a noop |
| 1235 | + |
| 1236 | + @param key: A key to be added to the system's APT keyring and used |
| 1237 | + to verify the signatures on packages. Ideally, this should be an |
| 1238 | + ASCII format GPG public key including the block headers. A GPG key |
| 1239 | + id may also be used, but be aware that only insecure protocols are |
| 1240 | + available to retrieve the actual public key from a public keyserver |
| 1241 | + placing your Juju environment at risk. ppa and cloud archive keys |
| 1242 | + are securely added automtically, so sould not be provided. |
| 1243 | + """ |
| 1244 | + if source is None: |
| 1245 | + log('Source is not present. Skipping') |
| 1246 | + return |
| 1247 | + |
| 1248 | + if (source.startswith('ppa:') or |
| 1249 | + source.startswith('http') or |
| 1250 | + source.startswith('deb ') or |
| 1251 | + source.startswith('cloud-archive:')): |
| 1252 | + subprocess.check_call(['add-apt-repository', '--yes', source]) |
| 1253 | + elif source.startswith('cloud:'): |
| 1254 | + install(filter_installed_packages(['ubuntu-cloud-keyring']), |
| 1255 | + fatal=True) |
| 1256 | + pocket = source.split(':')[-1] |
| 1257 | + if pocket not in CLOUD_ARCHIVE_POCKETS: |
| 1258 | + raise SourceConfigError( |
| 1259 | + 'Unsupported cloud: source option %s' % |
| 1260 | + pocket) |
| 1261 | + actual_pocket = CLOUD_ARCHIVE_POCKETS[pocket] |
| 1262 | + with open('/etc/apt/sources.list.d/cloud-archive.list', 'w') as apt: |
| 1263 | + apt.write(CLOUD_ARCHIVE.format(actual_pocket)) |
| 1264 | + elif source == 'proposed': |
| 1265 | + release = lsb_release()['DISTRIB_CODENAME'] |
| 1266 | + with open('/etc/apt/sources.list.d/proposed.list', 'w') as apt: |
| 1267 | + apt.write(PROPOSED_POCKET.format(release)) |
| 1268 | + elif source == 'distro': |
| 1269 | + pass |
| 1270 | + else: |
| 1271 | + log("Unknown source: {!r}".format(source)) |
| 1272 | + |
| 1273 | + if key: |
| 1274 | + if '-----BEGIN PGP PUBLIC KEY BLOCK-----' in key: |
| 1275 | + with NamedTemporaryFile('w+') as key_file: |
| 1276 | + key_file.write(key) |
| 1277 | + key_file.flush() |
| 1278 | + key_file.seek(0) |
| 1279 | + subprocess.check_call(['apt-key', 'add', '-'], stdin=key_file) |
| 1280 | + else: |
| 1281 | + # Note that hkp: is in no way a secure protocol. Using a |
| 1282 | + # GPG key id is pointless from a security POV unless you |
| 1283 | + # absolutely trust your network and DNS. |
| 1284 | + subprocess.check_call(['apt-key', 'adv', '--keyserver', |
| 1285 | + 'hkp://keyserver.ubuntu.com:80', '--recv', |
| 1286 | + key]) |
| 1287 | + |
| 1288 | + |
| 1289 | +def _run_apt_command(cmd, fatal=False): |
| 1290 | + """Run an APT command. |
| 1291 | + |
| 1292 | + Checks the output and retries if the fatal flag is set |
| 1293 | + to True. |
| 1294 | + |
| 1295 | + :param: cmd: str: The apt command to run. |
| 1296 | + :param: fatal: bool: Whether the command's output should be checked and |
| 1297 | + retried. |
| 1298 | + """ |
| 1299 | + env = os.environ.copy() |
| 1300 | + |
| 1301 | + if 'DEBIAN_FRONTEND' not in env: |
| 1302 | + env['DEBIAN_FRONTEND'] = 'noninteractive' |
| 1303 | + |
| 1304 | + if fatal: |
| 1305 | + retry_count = 0 |
| 1306 | + result = None |
| 1307 | + |
| 1308 | + # If the command is considered "fatal", we need to retry if the apt |
| 1309 | + # lock was not acquired. |
| 1310 | + |
| 1311 | + while result is None or result == APT_NO_LOCK: |
| 1312 | + try: |
| 1313 | + result = subprocess.check_call(cmd, env=env) |
| 1314 | + except subprocess.CalledProcessError as e: |
| 1315 | + retry_count = retry_count + 1 |
| 1316 | + if retry_count > APT_NO_LOCK_RETRY_COUNT: |
| 1317 | + raise |
| 1318 | + result = e.returncode |
| 1319 | + log("Couldn't acquire DPKG lock. Will retry in {} seconds." |
| 1320 | + "".format(APT_NO_LOCK_RETRY_DELAY)) |
| 1321 | + time.sleep(APT_NO_LOCK_RETRY_DELAY) |
| 1322 | + |
| 1323 | + else: |
| 1324 | + subprocess.call(cmd, env=env) |
| 1325 | |
| 1326 | === added file 'charmhelpers/osplatform.py' |
| 1327 | --- charmhelpers/osplatform.py 1970-01-01 00:00:00 +0000 |
| 1328 | +++ charmhelpers/osplatform.py 2016-08-12 07:09:49 +0000 |
| 1329 | @@ -0,0 +1,19 @@ |
| 1330 | +import platform |
| 1331 | + |
| 1332 | + |
| 1333 | +def get_platform(): |
| 1334 | + """Return the current OS platform. |
| 1335 | + |
| 1336 | + For example: if current os platform is Ubuntu then a string "ubuntu" |
| 1337 | + will be returned (which is the name of the module). |
| 1338 | + This string is used to decide which platform module should be imported. |
| 1339 | + """ |
| 1340 | + tuple_platform = platform.linux_distribution() |
| 1341 | + current_platform = tuple_platform[0] |
| 1342 | + if "Ubuntu" in current_platform: |
| 1343 | + return "ubuntu" |
| 1344 | + elif "CentOS" in current_platform: |
| 1345 | + return "centos" |
| 1346 | + else: |
| 1347 | + raise RuntimeError("This module is not supported on {}." |
| 1348 | + .format(current_platform)) |
| 1349 | |
| 1350 | === modified file 'tests/__init__.py' |
| 1351 | --- tests/__init__.py 2013-05-11 19:55:58 +0000 |
| 1352 | +++ tests/__init__.py 2016-08-12 07:09:49 +0000 |
| 1353 | @@ -0,0 +1,4 @@ |
| 1354 | +import sys |
| 1355 | +import mock |
| 1356 | + |
| 1357 | +sys.modules['yum'] = mock.MagicMock() |
| 1358 | |
| 1359 | === modified file 'tests/core/test_host.py' |
| 1360 | --- tests/core/test_host.py 2016-07-08 12:56:50 +0000 |
| 1361 | +++ tests/core/test_host.py 2016-08-12 07:09:49 +0000 |
| 1362 | @@ -6,7 +6,9 @@ |
| 1363 | from textwrap import dedent |
| 1364 | |
| 1365 | import apt_pkg |
| 1366 | +import imp |
| 1367 | |
| 1368 | +from charmhelpers import osplatform |
| 1369 | from mock import patch, call |
| 1370 | from testtools import TestCase |
| 1371 | from tests.helpers import patch_open |
| 1372 | @@ -31,6 +33,22 @@ |
| 1373 | DISTRIB_DESCRIPTION="Ubuntu Saucy Salamander (development branch)" |
| 1374 | ''' |
| 1375 | |
| 1376 | +OS_RELEASE = '''NAME="CentOS Linux" |
| 1377 | +ANSI_COLOR="0;31" |
| 1378 | +ID_LIKE="rhel fedora" |
| 1379 | +VERSION_ID="7" |
| 1380 | +BUG_REPORT_URL="https://bugs.centos.org/" |
| 1381 | +CENTOS_MANTISBT_PROJECT="CentOS-7" |
| 1382 | +PRETTY_NAME="CentOS Linux 7 (Core)" |
| 1383 | +VERSION="7 (Core)" |
| 1384 | +REDHAT_SUPPORT_PRODUCT_VERSION="7" |
| 1385 | +CENTOS_MANTISBT_PROJECT_VERSION="7" |
| 1386 | +REDHAT_SUPPORT_PRODUCT="centos" |
| 1387 | +HOME_URL="https://www.centos.org/" |
| 1388 | +CPE_NAME="cpe:/o:centos:centos:7" |
| 1389 | +ID="centos" |
| 1390 | +''' |
| 1391 | + |
| 1392 | IP_LINE_ETH0 = b""" |
| 1393 | 2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP qlen 1000 |
| 1394 | link/ether e4:11:5b:ab:a7:3c brd ff:ff:ff:ff:ff:ff |
| 1395 | @@ -288,8 +306,8 @@ |
| 1396 | @patch.object(host, 'init_is_systemd') |
| 1397 | @patch('subprocess.check_output') |
| 1398 | @patch.object(host, 'service') |
| 1399 | - def test_resumes_a_running_upstart_service(self, service, check_output, systemd, |
| 1400 | - service_running): |
| 1401 | + def test_resumes_a_running_upstart_service(self, service, check_output, |
| 1402 | + systemd, service_running): |
| 1403 | """When the service is already running, service start isn't called.""" |
| 1404 | service_name = 'foo-service' |
| 1405 | service.side_effect = [True] |
| 1406 | @@ -312,8 +330,8 @@ |
| 1407 | @patch.object(host, 'init_is_systemd') |
| 1408 | @patch('subprocess.check_output') |
| 1409 | @patch.object(host, 'service') |
| 1410 | - def test_resumes_a_stopped_upstart_service(self, service, check_output, systemd, |
| 1411 | - service_running): |
| 1412 | + def test_resumes_a_stopped_upstart_service(self, service, check_output, |
| 1413 | + systemd, service_running): |
| 1414 | """When the service is stopped, service start is called.""" |
| 1415 | check_output.return_value = b'foo-service stop/waiting' |
| 1416 | service_name = 'foo-service' |
| 1417 | @@ -524,7 +542,8 @@ |
| 1418 | @patch('pwd.getpwnam') |
| 1419 | @patch('subprocess.check_call') |
| 1420 | @patch.object(host, 'log') |
| 1421 | - def test_adds_a_user_if_it_doesnt_exist(self, log, check_call, getpwnam, getgrnam): |
| 1422 | + def test_adds_a_user_if_it_doesnt_exist(self, log, check_call, |
| 1423 | + getpwnam, getgrnam): |
| 1424 | username = 'johndoe' |
| 1425 | password = 'eodnhoj' |
| 1426 | shell = '/bin/bash' |
| 1427 | @@ -734,49 +753,102 @@ |
| 1428 | group |
| 1429 | ]) |
| 1430 | |
| 1431 | + @patch.object(osplatform, 'get_platform') |
| 1432 | @patch('grp.getgrnam') |
| 1433 | @patch('subprocess.check_call') |
| 1434 | - @patch.object(host, 'log') |
| 1435 | - def test_add_a_group_if_it_doesnt_exist(self, log, check_call, getgrnam): |
| 1436 | + def test_add_a_group_if_it_doesnt_exist_ubuntu(self, check_call, |
| 1437 | + getgrnam, platform): |
| 1438 | + platform.return_value = 'ubuntu' |
| 1439 | + imp.reload(host) |
| 1440 | + |
| 1441 | group_name = 'testgroup' |
| 1442 | existing_group_grnam = KeyError('group not found') |
| 1443 | new_group_grnam = 'some group grnam' |
| 1444 | |
| 1445 | getgrnam.side_effect = [existing_group_grnam, new_group_grnam] |
| 1446 | - |
| 1447 | - result = host.add_group(group_name) |
| 1448 | + with patch("charmhelpers.core.host.log"): |
| 1449 | + result = host.add_group(group_name) |
| 1450 | |
| 1451 | self.assertEqual(result, new_group_grnam) |
| 1452 | check_call.assert_called_with(['addgroup', '--group', group_name]) |
| 1453 | getgrnam.assert_called_with(group_name) |
| 1454 | |
| 1455 | - @patch('grp.getgrnam') |
| 1456 | - @patch('subprocess.check_call') |
| 1457 | - @patch.object(host, 'log') |
| 1458 | - def test_doesnt_add_group_if_it_already_exists(self, log, check_call, |
| 1459 | - getgrnam): |
| 1460 | - group_name = 'testgroup' |
| 1461 | - existing_group_grnam = 'some group grnam' |
| 1462 | - |
| 1463 | - getgrnam.return_value = existing_group_grnam |
| 1464 | - |
| 1465 | - result = host.add_group(group_name) |
| 1466 | - |
| 1467 | - self.assertEqual(result, existing_group_grnam) |
| 1468 | - self.assertFalse(check_call.called) |
| 1469 | - getgrnam.assert_called_with(group_name) |
| 1470 | - |
| 1471 | - @patch('grp.getgrnam') |
| 1472 | - @patch('subprocess.check_call') |
| 1473 | - @patch.object(host, 'log') |
| 1474 | - def test_add_a_system_group(self, log, check_call, getgrnam): |
| 1475 | - group_name = 'testgroup' |
| 1476 | - existing_group_grnam = KeyError('group not found') |
| 1477 | - new_group_grnam = 'some group grnam' |
| 1478 | - |
| 1479 | - getgrnam.side_effect = [existing_group_grnam, new_group_grnam] |
| 1480 | - |
| 1481 | - result = host.add_group(group_name, system_group=True) |
| 1482 | + @patch.object(osplatform, 'get_platform') |
| 1483 | + @patch('grp.getgrnam') |
| 1484 | + @patch('subprocess.check_call') |
| 1485 | + def test_add_a_group_if_it_doesnt_exist_centos(self, check_call, |
| 1486 | + getgrnam, platform): |
| 1487 | + platform.return_value = 'centos' |
| 1488 | + imp.reload(host) |
| 1489 | + |
| 1490 | + group_name = 'testgroup' |
| 1491 | + existing_group_grnam = KeyError('group not found') |
| 1492 | + new_group_grnam = 'some group grnam' |
| 1493 | + |
| 1494 | + getgrnam.side_effect = [existing_group_grnam, new_group_grnam] |
| 1495 | + |
| 1496 | + with patch("charmhelpers.core.host.log"): |
| 1497 | + result = host.add_group(group_name) |
| 1498 | + |
| 1499 | + self.assertEqual(result, new_group_grnam) |
| 1500 | + check_call.assert_called_with(['groupadd', group_name]) |
| 1501 | + getgrnam.assert_called_with(group_name) |
| 1502 | + |
| 1503 | + @patch.object(osplatform, 'get_platform') |
| 1504 | + @patch('grp.getgrnam') |
| 1505 | + @patch('subprocess.check_call') |
| 1506 | + def test_doesnt_add_group_if_it_already_exists_ubuntu(self, check_call, |
| 1507 | + getgrnam, platform): |
| 1508 | + platform.return_value = 'ubuntu' |
| 1509 | + imp.reload(host) |
| 1510 | + |
| 1511 | + group_name = 'testgroup' |
| 1512 | + existing_group_grnam = 'some group grnam' |
| 1513 | + |
| 1514 | + getgrnam.return_value = existing_group_grnam |
| 1515 | + |
| 1516 | + with patch("charmhelpers.core.host.log"): |
| 1517 | + result = host.add_group(group_name) |
| 1518 | + |
| 1519 | + self.assertEqual(result, existing_group_grnam) |
| 1520 | + self.assertFalse(check_call.called) |
| 1521 | + getgrnam.assert_called_with(group_name) |
| 1522 | + |
| 1523 | + @patch.object(osplatform, 'get_platform') |
| 1524 | + @patch('grp.getgrnam') |
| 1525 | + @patch('subprocess.check_call') |
| 1526 | + def test_doesnt_add_group_if_it_already_exists_centos(self, check_call, |
| 1527 | + getgrnam, platform): |
| 1528 | + platform.return_value = 'centos' |
| 1529 | + imp.reload(host) |
| 1530 | + |
| 1531 | + group_name = 'testgroup' |
| 1532 | + existing_group_grnam = 'some group grnam' |
| 1533 | + |
| 1534 | + getgrnam.return_value = existing_group_grnam |
| 1535 | + |
| 1536 | + with patch("charmhelpers.core.host.log"): |
| 1537 | + result = host.add_group(group_name) |
| 1538 | + |
| 1539 | + self.assertEqual(result, existing_group_grnam) |
| 1540 | + self.assertFalse(check_call.called) |
| 1541 | + getgrnam.assert_called_with(group_name) |
| 1542 | + |
| 1543 | + @patch.object(osplatform, 'get_platform') |
| 1544 | + @patch('grp.getgrnam') |
| 1545 | + @patch('subprocess.check_call') |
| 1546 | + def test_add_a_system_group_ubuntu(self, check_call, getgrnam, platform): |
| 1547 | + platform.return_value = 'ubuntu' |
| 1548 | + imp.reload(host) |
| 1549 | + |
| 1550 | + group_name = 'testgroup' |
| 1551 | + existing_group_grnam = KeyError('group not found') |
| 1552 | + new_group_grnam = 'some group grnam' |
| 1553 | + |
| 1554 | + getgrnam.side_effect = [existing_group_grnam, new_group_grnam] |
| 1555 | + |
| 1556 | + with patch("charmhelpers.core.host.log"): |
| 1557 | + result = host.add_group(group_name, system_group=True) |
| 1558 | |
| 1559 | self.assertEqual(result, new_group_grnam) |
| 1560 | check_call.assert_called_with([ |
| 1561 | @@ -786,6 +858,30 @@ |
| 1562 | ]) |
| 1563 | getgrnam.assert_called_with(group_name) |
| 1564 | |
| 1565 | + @patch.object(osplatform, 'get_platform') |
| 1566 | + @patch('grp.getgrnam') |
| 1567 | + @patch('subprocess.check_call') |
| 1568 | + def test_add_a_system_group_centos(self, check_call, getgrnam, platform): |
| 1569 | + platform.return_value = 'centos' |
| 1570 | + imp.reload(host) |
| 1571 | + |
| 1572 | + group_name = 'testgroup' |
| 1573 | + existing_group_grnam = KeyError('group not found') |
| 1574 | + new_group_grnam = 'some group grnam' |
| 1575 | + |
| 1576 | + getgrnam.side_effect = [existing_group_grnam, new_group_grnam] |
| 1577 | + |
| 1578 | + with patch("charmhelpers.core.host.log"): |
| 1579 | + result = host.add_group(group_name, system_group=True) |
| 1580 | + |
| 1581 | + self.assertEqual(result, new_group_grnam) |
| 1582 | + check_call.assert_called_with([ |
| 1583 | + 'groupadd', |
| 1584 | + '-r', |
| 1585 | + group_name |
| 1586 | + ]) |
| 1587 | + getgrnam.assert_called_with(group_name) |
| 1588 | + |
| 1589 | @patch('subprocess.check_output') |
| 1590 | @patch.object(host, 'log') |
| 1591 | def test_rsyncs_a_path(self, log, check_output): |
| 1592 | @@ -1133,7 +1229,8 @@ |
| 1593 | @patch.object(host, 'file_hash') |
| 1594 | def test_check_hash(self, file_hash): |
| 1595 | file_hash.return_value = 'good-hash' |
| 1596 | - self.assertRaises(host.ChecksumError, host.check_hash, 'file', 'bad-hash') |
| 1597 | + self.assertRaises(host.ChecksumError, host.check_hash, |
| 1598 | + 'file', 'bad-hash') |
| 1599 | host.check_hash('file', 'good-hash', 'sha256') |
| 1600 | self.assertEqual(file_hash.call_args_list, [ |
| 1601 | call('file', 'md5'), |
| 1602 | @@ -1219,7 +1316,8 @@ |
| 1603 | @patch.object(host, 'service') |
| 1604 | @patch('os.path.exists') |
| 1605 | @patch('glob.iglob') |
| 1606 | - def test_multiservice_restart_on_change_in_order(self, iglob, exists, service): |
| 1607 | + def test_multiservice_restart_on_change_in_order(self, iglob, exists, |
| 1608 | + service): |
| 1609 | file_name_one = '/etc/cinder/cinder.conf' |
| 1610 | file_name_two = '/etc/haproxy/haproxy.conf' |
| 1611 | restart_map = OrderedDict([ |
| 1612 | @@ -1377,7 +1475,11 @@ |
| 1613 | self.assertEquals([call('restart', 'haproxy')], service.call_args_list) |
| 1614 | self.assertEquals([call('some-api')], service_reload.call_args_list) |
| 1615 | |
| 1616 | - def test_lsb_release(self): |
| 1617 | + @patch.object(osplatform, 'get_platform') |
| 1618 | + def test_lsb_release_ubuntu(self, platform): |
| 1619 | + platform.return_value = 'ubuntu' |
| 1620 | + imp.reload(host) |
| 1621 | + |
| 1622 | result = { |
| 1623 | "DISTRIB_ID": "Ubuntu", |
| 1624 | "DISTRIB_RELEASE": "13.10", |
| 1625 | @@ -1390,6 +1492,32 @@ |
| 1626 | for key in result: |
| 1627 | self.assertEqual(result[key], lsb_release[key]) |
| 1628 | |
| 1629 | + @patch.object(osplatform, 'get_platform') |
| 1630 | + def test_lsb_release_centos(self, platform): |
| 1631 | + platform.return_value = 'centos' |
| 1632 | + imp.reload(host) |
| 1633 | + |
| 1634 | + result = { |
| 1635 | + 'NAME': '"CentOS Linux"', |
| 1636 | + 'ANSI_COLOR': '"0;31"', |
| 1637 | + 'ID_LIKE': '"rhel fedora"', |
| 1638 | + 'VERSION_ID': '"7"', |
| 1639 | + 'BUG_REPORT_URL': '"https://bugs.centos.org/"', |
| 1640 | + 'CENTOS_MANTISBT_PROJECT': '"CentOS-7"', |
| 1641 | + 'PRETTY_NAME': '"CentOS Linux 7 (Core)"', |
| 1642 | + 'VERSION': '"7 (Core)"', |
| 1643 | + 'REDHAT_SUPPORT_PRODUCT_VERSION': '"7"', |
| 1644 | + 'CENTOS_MANTISBT_PROJECT_VERSION': '"7"', |
| 1645 | + 'REDHAT_SUPPORT_PRODUCT': '"centos"', |
| 1646 | + 'HOME_URL': '"https://www.centos.org/"', |
| 1647 | + 'CPE_NAME': '"cpe:/o:centos:centos:7"', |
| 1648 | + 'ID': '"centos"' |
| 1649 | + } |
| 1650 | + with mocked_open('/etc/os-release', OS_RELEASE): |
| 1651 | + lsb_release = host.lsb_release() |
| 1652 | + for key in result: |
| 1653 | + self.assertEqual(result[key], lsb_release[key]) |
| 1654 | + |
| 1655 | def test_pwgen(self): |
| 1656 | pw = host.pwgen() |
| 1657 | self.assert_(len(pw) >= 35, 'Password is too short') |
| 1658 | @@ -1410,8 +1538,8 @@ |
| 1659 | |
| 1660 | def fake_realpath(soft): |
| 1661 | if soft.endswith('/eth0'): |
| 1662 | - hard = \ |
| 1663 | - '/sys/devices/pci0000:00/0000:00:1c.4/0000:02:00.1/net/eth0' |
| 1664 | + hard = ('/sys/devices/pci0000:00/0000:00:1c.4' |
| 1665 | + '/0000:02:00.1/net/eth0') |
| 1666 | else: |
| 1667 | hard = '/sys/devices/virtual/net/veth0' |
| 1668 | |
| 1669 | @@ -1429,8 +1557,8 @@ |
| 1670 | |
| 1671 | def fake_realpath(soft): |
| 1672 | if soft.endswith('/eth0'): |
| 1673 | - return \ |
| 1674 | - '/sys/devices/pci0000:00/0000:00:1c.4/0000:02:00.1/net/eth0' |
| 1675 | + return ('/sys/devices/pci0000:00/0000:00:1c.4' |
| 1676 | + '/0000:02:00.1/net/eth0') |
| 1677 | elif soft.endswith('/br0'): |
| 1678 | return '/sys/devices/virtual/net/br0' |
| 1679 | elif soft.endswith('/master'): |
| 1680 | @@ -1498,8 +1626,12 @@ |
| 1681 | hwaddr = host.get_nic_hwaddr(nic) |
| 1682 | self.assertEqual(hwaddr, 'e4:11:5b:ab:a7:3c') |
| 1683 | |
| 1684 | + @patch.object(osplatform, 'get_platform') |
| 1685 | @patch.object(apt_pkg, 'Cache') |
| 1686 | - def test_cmp_pkgrevno_revnos(self, pkg_cache): |
| 1687 | + def test_cmp_pkgrevno_revnos_ubuntu(self, pkg_cache, platform): |
| 1688 | + platform.return_value = 'ubuntu' |
| 1689 | + imp.reload(host) |
| 1690 | + |
| 1691 | class MockPackage: |
| 1692 | class MockPackageRevno: |
| 1693 | def __init__(self, ver_str): |
| 1694 | @@ -1516,6 +1648,30 @@ |
| 1695 | self.assertEqual(host.cmp_pkgrevno('python', '2.4'), 0) |
| 1696 | self.assertEqual(host.cmp_pkgrevno('python', '2.5'), -1) |
| 1697 | |
| 1698 | + @patch.object(osplatform, 'get_platform') |
| 1699 | + def test_cmp_pkgrevno_revnos_centos(self, platform): |
| 1700 | + platform.return_value = 'centos' |
| 1701 | + imp.reload(host) |
| 1702 | + |
| 1703 | + class MockPackage: |
| 1704 | + def __init__(self, name, version): |
| 1705 | + self.Name = name |
| 1706 | + self.version = version |
| 1707 | + |
| 1708 | + yum_dict = { |
| 1709 | + 'installed': { |
| 1710 | + MockPackage('python', '2.4') |
| 1711 | + } |
| 1712 | + } |
| 1713 | + |
| 1714 | + import yum |
| 1715 | + yum.YumBase.return_value.doPackageLists.return_value = ( |
| 1716 | + yum_dict) |
| 1717 | + |
| 1718 | + self.assertEqual(host.cmp_pkgrevno('python', '2.3'), 1) |
| 1719 | + self.assertEqual(host.cmp_pkgrevno('python', '2.4'), 0) |
| 1720 | + self.assertEqual(host.cmp_pkgrevno('python', '2.5'), -1) |
| 1721 | + |
| 1722 | def test_get_total_ram(self): |
| 1723 | raw = dedent('''\ |
| 1724 | MemFree: 183868 kB |
| 1725 | |
| 1726 | === modified file 'tests/core/test_kernel.py' |
| 1727 | --- tests/core/test_kernel.py 2015-10-26 10:17:10 +0000 |
| 1728 | +++ tests/core/test_kernel.py 2016-08-12 07:09:49 +0000 |
| 1729 | @@ -1,67 +1,113 @@ |
| 1730 | #!/usr/bin/env python |
| 1731 | # -*- coding: utf-8 -*- |
| 1732 | |
| 1733 | +import unittest |
| 1734 | +import imp |
| 1735 | + |
| 1736 | +from charmhelpers import osplatform |
| 1737 | from mock import patch |
| 1738 | -import unittest |
| 1739 | - |
| 1740 | from tests.helpers import patch_open |
| 1741 | from charmhelpers.core import kernel |
| 1742 | |
| 1743 | -TO_PATCH = [ |
| 1744 | - 'log', |
| 1745 | - 'check_call', |
| 1746 | - 'check_output', |
| 1747 | -] |
| 1748 | - |
| 1749 | |
| 1750 | class TestKernel(unittest.TestCase): |
| 1751 | |
| 1752 | - def setUp(self): |
| 1753 | - for m in TO_PATCH: |
| 1754 | - setattr(self, m, self._patch(m)) |
| 1755 | - |
| 1756 | - def _patch(self, method): |
| 1757 | - _m = patch('charmhelpers.core.kernel.' + method) |
| 1758 | - mock = _m.start() |
| 1759 | - self.addCleanup(_m.stop) |
| 1760 | - return mock |
| 1761 | - |
| 1762 | - def test_modprobe_persistent(self): |
| 1763 | + @patch('subprocess.check_call') |
| 1764 | + @patch.object(osplatform, 'get_platform') |
| 1765 | + def test_modprobe_persistent_ubuntu(self, platform, check_call): |
| 1766 | + platform.return_value = 'ubuntu' |
| 1767 | + imp.reload(kernel) |
| 1768 | + |
| 1769 | with patch_open() as (_open, _file): |
| 1770 | _file.read.return_value = 'anothermod\n' |
| 1771 | - kernel.modprobe('mymod') |
| 1772 | + with patch("charmhelpers.core.kernel.log"): |
| 1773 | + kernel.modprobe('mymod') |
| 1774 | _open.assert_called_with('/etc/modules', 'r+') |
| 1775 | _file.read.assert_called_with() |
| 1776 | _file.write.assert_called_with('mymod') |
| 1777 | - self.check_call.assert_called_with(['modprobe', 'mymod']) |
| 1778 | - |
| 1779 | - def test_modprobe_not_persistent(self): |
| 1780 | - with patch_open() as (_open, _file): |
| 1781 | - _file.read.return_value = 'anothermod\n' |
| 1782 | - kernel.modprobe('mymod', persist=False) |
| 1783 | - assert not _open.called |
| 1784 | - self.check_call.assert_called_with(['modprobe', 'mymod']) |
| 1785 | - |
| 1786 | - def test_rmmod_not_forced(self): |
| 1787 | + check_call.assert_called_with(['modprobe', 'mymod']) |
| 1788 | + |
| 1789 | + @patch('os.chmod') |
| 1790 | + @patch('subprocess.check_call') |
| 1791 | + @patch.object(osplatform, 'get_platform') |
| 1792 | + def test_modprobe_persistent_centos(self, platform, check_call, os): |
| 1793 | + platform.return_value = 'centos' |
| 1794 | + imp.reload(kernel) |
| 1795 | + |
| 1796 | + with patch_open() as (_open, _file): |
| 1797 | + _file.read.return_value = 'anothermod\n' |
| 1798 | + with patch("charmhelpers.core.kernel.log"): |
| 1799 | + kernel.modprobe('mymod') |
| 1800 | + _open.assert_called_with('/etc/rc.modules', 'r+') |
| 1801 | + os.assert_called_with('/etc/rc.modules', 111) |
| 1802 | + _file.read.assert_called_with() |
| 1803 | + _file.write.assert_called_with('modprobe mymod\n') |
| 1804 | + check_call.assert_called_with(['modprobe', 'mymod']) |
| 1805 | + |
| 1806 | + @patch('subprocess.check_call') |
| 1807 | + @patch.object(osplatform, 'get_platform') |
| 1808 | + def test_modprobe_not_persistent_ubuntu(self, platform, check_call): |
| 1809 | + platform.return_value = 'ubuntu' |
| 1810 | + imp.reload(kernel) |
| 1811 | + |
| 1812 | + with patch_open() as (_open, _file): |
| 1813 | + _file.read.return_value = 'anothermod\n' |
| 1814 | + with patch("charmhelpers.core.kernel.log"): |
| 1815 | + kernel.modprobe('mymod', persist=False) |
| 1816 | + assert not _open.called |
| 1817 | + check_call.assert_called_with(['modprobe', 'mymod']) |
| 1818 | + |
| 1819 | + @patch('subprocess.check_call') |
| 1820 | + @patch.object(osplatform, 'get_platform') |
| 1821 | + def test_modprobe_not_persistent_centos(self, platform, check_call): |
| 1822 | + platform.return_value = 'centos' |
| 1823 | + imp.reload(kernel) |
| 1824 | + |
| 1825 | + with patch_open() as (_open, _file): |
| 1826 | + _file.read.return_value = 'anothermod\n' |
| 1827 | + with patch("charmhelpers.core.kernel.log"): |
| 1828 | + kernel.modprobe('mymod', persist=False) |
| 1829 | + assert not _open.called |
| 1830 | + check_call.assert_called_with(['modprobe', 'mymod']) |
| 1831 | + |
| 1832 | + @patch.object(kernel, 'log') |
| 1833 | + @patch('subprocess.check_call') |
| 1834 | + def test_rmmod_not_forced(self, check_call, log): |
| 1835 | kernel.rmmod('mymod') |
| 1836 | - self.check_call.assert_called_with(['rmmod', 'mymod']) |
| 1837 | + check_call.assert_called_with(['rmmod', 'mymod']) |
| 1838 | |
| 1839 | - def test_rmmod_forced(self): |
| 1840 | + @patch.object(kernel, 'log') |
| 1841 | + @patch('subprocess.check_call') |
| 1842 | + def test_rmmod_forced(self, check_call, log): |
| 1843 | kernel.rmmod('mymod', force=True) |
| 1844 | - self.check_call.assert_called_with(['rmmod', '-f', 'mymod']) |
| 1845 | + check_call.assert_called_with(['rmmod', '-f', 'mymod']) |
| 1846 | |
| 1847 | - def test_lsmod(self): |
| 1848 | + @patch.object(kernel, 'log') |
| 1849 | + @patch('subprocess.check_output') |
| 1850 | + def test_lsmod(self, check_output, log): |
| 1851 | kernel.lsmod() |
| 1852 | - self.check_output.assert_called_with(['lsmod'], |
| 1853 | - universal_newlines=True) |
| 1854 | + check_output.assert_called_with(['lsmod'], |
| 1855 | + universal_newlines=True) |
| 1856 | |
| 1857 | @patch('charmhelpers.core.kernel.lsmod') |
| 1858 | def test_is_module_loaded(self, lsmod): |
| 1859 | lsmod.return_value = "ip6_tables 28672 1 ip6table_filter" |
| 1860 | - self.assertTrue(kernel.is_module_loaded( |
| 1861 | - "ip6_tables")) |
| 1862 | - |
| 1863 | - def test_update_initramfs(self): |
| 1864 | - kernel.update_initramfs() |
| 1865 | - self.check_call.assert_called_with([ |
| 1866 | - "update-initramfs", "-k", "all", "-u"]) |
| 1867 | + self.assertTrue(kernel.is_module_loaded("ip6_tables")) |
| 1868 | + |
| 1869 | + @patch.object(osplatform, 'get_platform') |
| 1870 | + @patch('subprocess.check_call') |
| 1871 | + def test_update_initramfs_ubuntu(self, check_call, platform): |
| 1872 | + platform.return_value = 'ubuntu' |
| 1873 | + imp.reload(kernel) |
| 1874 | + |
| 1875 | + kernel.update_initramfs() |
| 1876 | + check_call.assert_called_with(["update-initramfs", "-k", "all", "-u"]) |
| 1877 | + |
| 1878 | + @patch.object(osplatform, 'get_platform') |
| 1879 | + @patch('subprocess.check_call') |
| 1880 | + def test_update_initramfs_centos(self, check_call, platform): |
| 1881 | + platform.return_value = 'centos' |
| 1882 | + imp.reload(kernel) |
| 1883 | + |
| 1884 | + kernel.update_initramfs() |
| 1885 | + check_call.assert_called_with(['dracut', '-f', 'all']) |
| 1886 | |
| 1887 | === modified file 'tests/fetch/test_archiveurl.py' |
| 1888 | --- tests/fetch/test_archiveurl.py 2015-12-10 05:23:53 +0000 |
| 1889 | +++ tests/fetch/test_archiveurl.py 2016-08-12 07:09:49 +0000 |
| 1890 | @@ -65,7 +65,8 @@ |
| 1891 | _urlopen.return_value = response |
| 1892 | |
| 1893 | _open = mock_open() |
| 1894 | - with patch('charmhelpers.fetch.archiveurl.open', _open, create=True): |
| 1895 | + with patch('charmhelpers.fetch.archiveurl.open', |
| 1896 | + _open, create=True): |
| 1897 | self.fh.download(url, "foo") |
| 1898 | |
| 1899 | response.read.assert_called_with() |
| 1900 | |
| 1901 | === modified file 'tests/fetch/test_bzrurl.py' |
| 1902 | --- tests/fetch/test_bzrurl.py 2016-06-16 03:28:45 +0000 |
| 1903 | +++ tests/fetch/test_bzrurl.py 2016-08-12 07:09:49 +0000 |
| 1904 | @@ -74,7 +74,8 @@ |
| 1905 | |
| 1906 | for url in self.invalid_urls: |
| 1907 | with patch.dict('os.environ', {'CHARM_DIR': 'foo'}): |
| 1908 | - self.assertRaises(UnhandledSource, self.fh.branch, url, dest_path) |
| 1909 | + self.assertRaises(UnhandledSource, self.fh.branch, |
| 1910 | + url, dest_path) |
| 1911 | |
| 1912 | @patch('charmhelpers.fetch.bzrurl.check_call') |
| 1913 | def test_branch_revno(self, check_call): |
| 1914 | |
| 1915 | === modified file 'tests/fetch/test_fetch.py' |
| 1916 | --- tests/fetch/test_fetch.py 2016-07-14 10:22:50 +0000 |
| 1917 | +++ tests/fetch/test_fetch.py 2016-08-12 07:09:49 +0000 |
| 1918 | @@ -1,5 +1,11 @@ |
| 1919 | import subprocess |
| 1920 | +import six |
| 1921 | +import os |
| 1922 | +import yaml |
| 1923 | +import imp |
| 1924 | +import tempfile |
| 1925 | |
| 1926 | +from charmhelpers import osplatform |
| 1927 | from tests.helpers import patch_open |
| 1928 | from testtools import TestCase |
| 1929 | from mock import ( |
| 1930 | @@ -9,10 +15,6 @@ |
| 1931 | sentinel, |
| 1932 | ) |
| 1933 | from charmhelpers import fetch |
| 1934 | -import os |
| 1935 | -import yaml |
| 1936 | - |
| 1937 | -import six |
| 1938 | from six.moves import StringIO |
| 1939 | if six.PY3: |
| 1940 | from urllib.parse import urlparse |
| 1941 | @@ -58,156 +60,378 @@ |
| 1942 | |
| 1943 | class FetchTest(TestCase): |
| 1944 | |
| 1945 | - @patch('apt_pkg.Cache') |
| 1946 | - def test_filter_packages_missing(self, cache): |
| 1947 | - cache.side_effect = fake_apt_cache |
| 1948 | - result = fetch.filter_installed_packages(['vim', 'emacs']) |
| 1949 | - self.assertEquals(result, ['emacs']) |
| 1950 | - |
| 1951 | - @patch('apt_pkg.Cache') |
| 1952 | - def test_filter_packages_none_missing(self, cache): |
| 1953 | - cache.side_effect = fake_apt_cache |
| 1954 | - result = fetch.filter_installed_packages(['vim']) |
| 1955 | - self.assertEquals(result, []) |
| 1956 | - |
| 1957 | - @patch.object(fetch, 'log') |
| 1958 | - @patch('apt_pkg.Cache') |
| 1959 | - def test_filter_packages_not_available(self, cache, log): |
| 1960 | + @patch("charmhelpers.fetch.ubuntu.log") |
| 1961 | + @patch.object(osplatform, 'get_platform') |
| 1962 | + @patch('apt_pkg.Cache') |
| 1963 | + def test_filter_packages_missing_ubuntu(self, cache, platform, log): |
| 1964 | + platform.return_value = 'ubuntu' |
| 1965 | + imp.reload(fetch) |
| 1966 | + |
| 1967 | + cache.side_effect = fake_apt_cache |
| 1968 | + result = fetch.filter_installed_packages(['vim', 'emacs']) |
| 1969 | + self.assertEquals(result, ['emacs']) |
| 1970 | + |
| 1971 | + @patch("charmhelpers.fetch.centos.log") |
| 1972 | + @patch('yum.YumBase.doPackageLists') |
| 1973 | + @patch.object(osplatform, 'get_platform') |
| 1974 | + def test_filter_packages_missing_centos(self, platform, yumBase, log): |
| 1975 | + platform.return_value = 'centos' |
| 1976 | + imp.reload(fetch) |
| 1977 | + |
| 1978 | + class MockPackage: |
| 1979 | + def __init__(self, name): |
| 1980 | + self.base_package_name = name |
| 1981 | + |
| 1982 | + yum_dict = { |
| 1983 | + 'installed': { |
| 1984 | + MockPackage('vim') |
| 1985 | + }, |
| 1986 | + 'available': { |
| 1987 | + MockPackage('vim') |
| 1988 | + } |
| 1989 | + } |
| 1990 | + import yum |
| 1991 | + yum.YumBase.return_value.doPackageLists.return_value = yum_dict |
| 1992 | + result = fetch.filter_installed_packages(['vim', 'emacs']) |
| 1993 | + self.assertEquals(result, ['emacs']) |
| 1994 | + |
| 1995 | + @patch("charmhelpers.fetch.ubuntu.log") |
| 1996 | + @patch.object(osplatform, 'get_platform') |
| 1997 | + @patch('apt_pkg.Cache') |
| 1998 | + def test_filter_packages_none_missing_ubuntu(self, cache, platform, log): |
| 1999 | + platform.return_value = 'ubuntu' |
| 2000 | + imp.reload(fetch) |
| 2001 | + |
| 2002 | + cache.side_effect = fake_apt_cache |
| 2003 | + result = fetch.filter_installed_packages(['vim']) |
| 2004 | + self.assertEquals(result, []) |
| 2005 | + |
| 2006 | + @patch("charmhelpers.fetch.centos.log") |
| 2007 | + @patch.object(osplatform, 'get_platform') |
| 2008 | + def test_filter_packages_none_missing_centos(self, platform, log): |
| 2009 | + platform.return_value = 'centos' |
| 2010 | + imp.reload(fetch) |
| 2011 | + |
| 2012 | + class MockPackage: |
| 2013 | + def __init__(self, name): |
| 2014 | + self.base_package_name = name |
| 2015 | + |
| 2016 | + yum_dict = { |
| 2017 | + 'installed': { |
| 2018 | + MockPackage('vim') |
| 2019 | + }, |
| 2020 | + 'available': { |
| 2021 | + MockPackage('vim') |
| 2022 | + } |
| 2023 | + } |
| 2024 | + import yum |
| 2025 | + yum.yumBase.return_value.doPackageLists.return_value = yum_dict |
| 2026 | + result = fetch.filter_installed_packages(['vim']) |
| 2027 | + self.assertEquals(result, []) |
| 2028 | + |
| 2029 | + @patch.object(osplatform, 'get_platform') |
| 2030 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2031 | + @patch('apt_pkg.Cache') |
| 2032 | + def test_filter_packages_not_available_ubuntu(self, cache, log, platform): |
| 2033 | + platform.return_value = 'ubuntu' |
| 2034 | + imp.reload(fetch) |
| 2035 | + |
| 2036 | cache.side_effect = fake_apt_cache |
| 2037 | result = fetch.filter_installed_packages(['vim', 'joe']) |
| 2038 | self.assertEquals(result, ['joe']) |
| 2039 | log.assert_called_with('Package joe has no installation candidate.', |
| 2040 | level='WARNING') |
| 2041 | |
| 2042 | - @patch.object(fetch, 'log') |
| 2043 | - def test_add_source_none(self, log): |
| 2044 | - fetch.add_source(source=None) |
| 2045 | - self.assertTrue(log.called) |
| 2046 | - |
| 2047 | + @patch.object(osplatform, 'get_platform') |
| 2048 | + @patch('charmhelpers.fetch.centos.log') |
| 2049 | + @patch('yum.YumBase.doPackageLists') |
| 2050 | + def test_filter_packages_not_available_centos(self, yumBase, |
| 2051 | + log, platform): |
| 2052 | + platform.return_value = 'centos' |
| 2053 | + imp.reload(fetch) |
| 2054 | + |
| 2055 | + class MockPackage: |
| 2056 | + def __init__(self, name): |
| 2057 | + self.base_package_name = name |
| 2058 | + |
| 2059 | + yum_dict = { |
| 2060 | + 'installed': { |
| 2061 | + MockPackage('vim') |
| 2062 | + } |
| 2063 | + } |
| 2064 | + import yum |
| 2065 | + yum.YumBase.return_value.doPackageLists.return_value = yum_dict |
| 2066 | + |
| 2067 | + result = fetch.filter_installed_packages(['vim', 'joe']) |
| 2068 | + self.assertEquals(result, ['joe']) |
| 2069 | + |
| 2070 | + @patch.object(osplatform, 'get_platform') |
| 2071 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2072 | + def test_add_source_none_ubuntu(self, log, platform): |
| 2073 | + platform.return_value = 'ubuntu' |
| 2074 | + imp.reload(fetch) |
| 2075 | + |
| 2076 | + fetch.add_source(source=None) |
| 2077 | + self.assertTrue(log.called) |
| 2078 | + |
| 2079 | + @patch.object(osplatform, 'get_platform') |
| 2080 | + @patch('charmhelpers.fetch.centos.log') |
| 2081 | + def test_add_source_none_centos(self, log, platform): |
| 2082 | + platform.return_value = 'centos' |
| 2083 | + imp.reload(fetch) |
| 2084 | + |
| 2085 | + fetch.add_source(source=None) |
| 2086 | + self.assertTrue(log.called) |
| 2087 | + |
| 2088 | + @patch.object(osplatform, 'get_platform') |
| 2089 | @patch('subprocess.check_call') |
| 2090 | - def test_add_source_ppa(self, check_call): |
| 2091 | + def test_add_source_ppa(self, check_call, platform): |
| 2092 | + platform.return_value = 'ubuntu' |
| 2093 | + imp.reload(fetch) |
| 2094 | + |
| 2095 | source = "ppa:test-ppa" |
| 2096 | fetch.add_source(source=source) |
| 2097 | - check_call.assert_called_with(['add-apt-repository', |
| 2098 | - '--yes', |
| 2099 | - source]) |
| 2100 | + check_call.assert_called_with( |
| 2101 | + ['add-apt-repository', '--yes', source]) |
| 2102 | |
| 2103 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2104 | + @patch.object(osplatform, 'get_platform') |
| 2105 | @patch('subprocess.check_call') |
| 2106 | - def test_add_source_http(self, check_call): |
| 2107 | + def test_add_source_http_ubuntu(self, check_call, platform, log): |
| 2108 | + platform.return_value = 'ubuntu' |
| 2109 | + imp.reload(fetch) |
| 2110 | + |
| 2111 | source = "http://archive.ubuntu.com/ubuntu raring-backports main" |
| 2112 | fetch.add_source(source=source) |
| 2113 | - check_call.assert_called_with(['add-apt-repository', |
| 2114 | - '--yes', |
| 2115 | - source]) |
| 2116 | - |
| 2117 | + check_call.assert_called_with( |
| 2118 | + ['add-apt-repository', '--yes', source]) |
| 2119 | + |
| 2120 | + @patch('charmhelpers.fetch.centos.log') |
| 2121 | + @patch.object(osplatform, 'get_platform') |
| 2122 | + @patch('os.listdir') |
| 2123 | + def test_add_source_http_centos(self, listdir, platform, log): |
| 2124 | + platform.return_value = 'centos' |
| 2125 | + imp.reload(fetch) |
| 2126 | + |
| 2127 | + source = "http://archive.ubuntu.com/ubuntu raring-backports main" |
| 2128 | + with patch_open() as (mock_open, mock_file): |
| 2129 | + fetch.add_source(source=source) |
| 2130 | + listdir.assert_called_with('/etc/yum.repos.d/') |
| 2131 | + mock_file.write.assert_has_calls([ |
| 2132 | + call("[archive.ubuntu.com_ubuntu raring-backports main]\n"), |
| 2133 | + call("name=archive.ubuntu.com/ubuntu raring-backports main\n"), |
| 2134 | + call("baseurl=http://archive.ubuntu.com/ubuntu raring" |
| 2135 | + "-backports main\n\n")]) |
| 2136 | + |
| 2137 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2138 | + @patch.object(osplatform, 'get_platform') |
| 2139 | @patch('subprocess.check_call') |
| 2140 | - def test_add_source_https(self, check_call): |
| 2141 | + def test_add_source_https(self, check_call, platform, log): |
| 2142 | + platform.return_value = 'ubuntu' |
| 2143 | + imp.reload(fetch) |
| 2144 | + |
| 2145 | source = "https://example.com" |
| 2146 | fetch.add_source(source=source) |
| 2147 | - check_call.assert_called_with(['add-apt-repository', |
| 2148 | - '--yes', |
| 2149 | - source]) |
| 2150 | + check_call.assert_called_with( |
| 2151 | + ['add-apt-repository', '--yes', source]) |
| 2152 | |
| 2153 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2154 | + @patch.object(osplatform, 'get_platform') |
| 2155 | @patch('subprocess.check_call') |
| 2156 | - def test_add_source_deb(self, check_call): |
| 2157 | + def test_add_source_deb(self, check_call, platform, log): |
| 2158 | """add-apt-repository behaves differently when using the deb prefix. |
| 2159 | |
| 2160 | - $ add-apt-repository --yes "http://special.example.com/ubuntu precise-special main" |
| 2161 | + $ add-apt-repository --yes "http://special.example.com/ubuntu |
| 2162 | + precise-special main" |
| 2163 | $ grep special /etc/apt/sources.list |
| 2164 | deb http://special.example.com/ubuntu precise precise-special main |
| 2165 | deb-src http://special.example.com/ubuntu precise precise-special main |
| 2166 | |
| 2167 | - $ add-apt-repository --yes "deb http://special.example.com/ubuntu precise-special main" |
| 2168 | + $ add-apt-repository --yes "deb http://special.example.com/ubuntu |
| 2169 | + precise-special main" |
| 2170 | $ grep special /etc/apt/sources.list |
| 2171 | deb http://special.example.com/ubuntu precise precise-special main |
| 2172 | deb-src http://special.example.com/ubuntu precise precise-special main |
| 2173 | deb http://special.example.com/ubuntu precise-special main |
| 2174 | deb-src http://special.example.com/ubuntu precise-special main |
| 2175 | """ |
| 2176 | + platform.return_value = 'ubuntu' |
| 2177 | + imp.reload(fetch) |
| 2178 | + |
| 2179 | source = "deb http://archive.ubuntu.com/ubuntu raring-backports main" |
| 2180 | fetch.add_source(source=source) |
| 2181 | - check_call.assert_called_with(['add-apt-repository', |
| 2182 | - '--yes', |
| 2183 | - source]) |
| 2184 | - |
| 2185 | - @patch.object(fetch, 'filter_installed_packages') |
| 2186 | - @patch.object(fetch, 'apt_install') |
| 2187 | - def test_add_source_cloud_invalid_pocket(self, apt_install, filter_pkg): |
| 2188 | + check_call.assert_called_with( |
| 2189 | + ['add-apt-repository', '--yes', source]) |
| 2190 | + |
| 2191 | + @patch.object(osplatform, 'get_platform') |
| 2192 | + @patch.object(fetch.ubuntu, 'filter_installed_packages') |
| 2193 | + @patch.object(fetch.ubuntu, 'install') |
| 2194 | + def test_add_source_cloud_invalid_pocket(self, install, |
| 2195 | + filter_pkg, platform): |
| 2196 | + platform.return_value = 'ubuntu' |
| 2197 | + imp.reload(fetch) |
| 2198 | + |
| 2199 | source = "cloud:havana-updates" |
| 2200 | - self.assertRaises(fetch.SourceConfigError, fetch.add_source, source) |
| 2201 | + self.assertRaises(fetch.ubuntu.SourceConfigError, |
| 2202 | + fetch.add_source, source) |
| 2203 | filter_pkg.assert_called_with(['ubuntu-cloud-keyring']) |
| 2204 | |
| 2205 | - @patch.object(fetch, 'filter_installed_packages') |
| 2206 | - @patch.object(fetch, 'apt_install') |
| 2207 | - def test_add_source_cloud_pocket_style(self, apt_install, filter_pkg): |
| 2208 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2209 | + @patch.object(osplatform, 'get_platform') |
| 2210 | + @patch.object(fetch.ubuntu, 'filter_installed_packages') |
| 2211 | + @patch.object(fetch.ubuntu, 'install') |
| 2212 | + def test_add_source_cloud_pocket_style(self, install, filter_pkg, |
| 2213 | + platform, log): |
| 2214 | + platform.return_value = 'ubuntu' |
| 2215 | + imp.reload(fetch) |
| 2216 | + |
| 2217 | source = "cloud:precise-updates/havana" |
| 2218 | - result = '''# Ubuntu Cloud Archive |
| 2219 | -deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/havana main |
| 2220 | -''' |
| 2221 | + result = ('# Ubuntu Cloud Archive' |
| 2222 | + ' deb http://ubuntu-cloud.archive.canonical.com/ubuntu' |
| 2223 | + ' precise-updates/havana main') |
| 2224 | + |
| 2225 | with patch_open() as (mock_open, mock_file): |
| 2226 | fetch.add_source(source=source) |
| 2227 | mock_file.write.assert_called_with(result) |
| 2228 | filter_pkg.assert_called_with(['ubuntu-cloud-keyring']) |
| 2229 | |
| 2230 | - @patch.object(fetch, 'filter_installed_packages') |
| 2231 | - @patch.object(fetch, 'apt_install') |
| 2232 | - def test_add_source_cloud_os_style(self, apt_install, filter_pkg): |
| 2233 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2234 | + @patch.object(osplatform, 'get_platform') |
| 2235 | + @patch.object(fetch.ubuntu, 'filter_installed_packages') |
| 2236 | + @patch.object(fetch.ubuntu, 'install') |
| 2237 | + def test_add_source_cloud_os_style(self, install, filter_pkg, |
| 2238 | + platform, log): |
| 2239 | + platform.return_value = 'ubuntu' |
| 2240 | + imp.reload(fetch) |
| 2241 | + |
| 2242 | source = "cloud:precise-havana" |
| 2243 | - result = '''# Ubuntu Cloud Archive |
| 2244 | -deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/havana main |
| 2245 | -''' |
| 2246 | + result = ('# Ubuntu Cloud Archive' |
| 2247 | + ' deb http://ubuntu-cloud.archive.canonical.com/ubuntu' |
| 2248 | + ' precise-updates/havana main') |
| 2249 | with patch_open() as (mock_open, mock_file): |
| 2250 | fetch.add_source(source=source) |
| 2251 | mock_file.write.assert_called_with(result) |
| 2252 | filter_pkg.assert_called_with(['ubuntu-cloud-keyring']) |
| 2253 | |
| 2254 | - @patch.object(fetch, 'filter_installed_packages') |
| 2255 | - @patch.object(fetch, 'apt_install') |
| 2256 | - def test_add_source_cloud_distroless_style(self, apt_install, filter_pkg): |
| 2257 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2258 | + @patch.object(osplatform, 'get_platform') |
| 2259 | + @patch.object(fetch.ubuntu, 'filter_installed_packages') |
| 2260 | + @patch.object(fetch.ubuntu, 'install') |
| 2261 | + def test_add_source_cloud_distroless_style(self, install, filter_pkg, |
| 2262 | + platform, log): |
| 2263 | + platform.return_value = 'ubuntu' |
| 2264 | + imp.reload(fetch) |
| 2265 | + |
| 2266 | source = "cloud:havana" |
| 2267 | - result = '''# Ubuntu Cloud Archive |
| 2268 | -deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/havana main |
| 2269 | -''' |
| 2270 | + result = ('# Ubuntu Cloud Archive' |
| 2271 | + ' deb http://ubuntu-cloud.archive.canonical.com/ubuntu' |
| 2272 | + ' precise-updates/havana main') |
| 2273 | with patch_open() as (mock_open, mock_file): |
| 2274 | fetch.add_source(source=source) |
| 2275 | mock_file.write.assert_called_with(result) |
| 2276 | filter_pkg.assert_called_with(['ubuntu-cloud-keyring']) |
| 2277 | |
| 2278 | - @patch.object(fetch, 'lsb_release') |
| 2279 | - def test_add_source_proposed(self, lsb_release): |
| 2280 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2281 | + @patch.object(osplatform, 'get_platform') |
| 2282 | + @patch.object(fetch.ubuntu, 'lsb_release') |
| 2283 | + def test_add_source_proposed(self, lsb_release, platform, log): |
| 2284 | + platform.return_value = 'ubuntu' |
| 2285 | + imp.reload(fetch) |
| 2286 | + |
| 2287 | source = "proposed" |
| 2288 | - result = """# Proposed |
| 2289 | -deb http://archive.ubuntu.com/ubuntu precise-proposed main universe multiverse restricted |
| 2290 | -""" |
| 2291 | + result = ('# Proposed' |
| 2292 | + ' deb http://archive.ubuntu.com/ubuntu precise-proposed' |
| 2293 | + ' main universe multiverse restricted') |
| 2294 | lsb_release.return_value = {'DISTRIB_CODENAME': 'precise'} |
| 2295 | with patch_open() as (mock_open, mock_file): |
| 2296 | fetch.add_source(source=source) |
| 2297 | mock_file.write.assert_called_with(result) |
| 2298 | |
| 2299 | - @patch('subprocess.check_call') |
| 2300 | - def test_add_source_http_and_key_id(self, check_call): |
| 2301 | - source = "http://archive.ubuntu.com/ubuntu raring-backports main" |
| 2302 | - key_id = "akey" |
| 2303 | - fetch.add_source(source=source, key=key_id) |
| 2304 | - check_call.assert_has_calls([ |
| 2305 | - call(['add-apt-repository', '--yes', source]), |
| 2306 | - call(['apt-key', 'adv', '--keyserver', |
| 2307 | - 'hkp://keyserver.ubuntu.com:80', '--recv', key_id]) |
| 2308 | - ]) |
| 2309 | - |
| 2310 | - @patch('subprocess.check_call') |
| 2311 | - def test_add_source_https_and_key_id(self, check_call): |
| 2312 | - source = "https://USER:PASS@private-ppa.launchpad.net/project/awesome" |
| 2313 | - key_id = "GPGPGP" |
| 2314 | - fetch.add_source(source=source, key=key_id) |
| 2315 | - check_call.assert_has_calls([ |
| 2316 | - call(['add-apt-repository', '--yes', source]), |
| 2317 | - call(['apt-key', 'adv', '--keyserver', |
| 2318 | - 'hkp://keyserver.ubuntu.com:80', '--recv', key_id]) |
| 2319 | - ]) |
| 2320 | - |
| 2321 | - @patch('subprocess.check_call') |
| 2322 | - def test_add_source_http_and_key(self, check_call): |
| 2323 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2324 | + @patch.object(osplatform, 'get_platform') |
| 2325 | + @patch('subprocess.check_call') |
| 2326 | + def test_add_source_http_and_key_id_ubuntu(self, check_call, |
| 2327 | + platform, log): |
| 2328 | + platform.return_value = 'ubuntu' |
| 2329 | + imp.reload(fetch) |
| 2330 | + |
| 2331 | + source = "http://archive.ubuntu.com/ubuntu raring-backports main" |
| 2332 | + key_id = "akey" |
| 2333 | + fetch.add_source(source=source, key=key_id) |
| 2334 | + check_call.assert_has_calls([ |
| 2335 | + call(['add-apt-repository', '--yes', source]), |
| 2336 | + call(['apt-key', 'adv', '--keyserver', |
| 2337 | + 'hkp://keyserver.ubuntu.com:80', '--recv', key_id]) |
| 2338 | + ]) |
| 2339 | + |
| 2340 | + @patch('charmhelpers.fetch.centos.log') |
| 2341 | + @patch('os.listdir') |
| 2342 | + @patch.object(osplatform, 'get_platform') |
| 2343 | + @patch('subprocess.check_call') |
| 2344 | + def test_add_source_http_and_key_id_centos(self, check_call, |
| 2345 | + platform, listdir, log): |
| 2346 | + platform.return_value = 'centos' |
| 2347 | + imp.reload(fetch) |
| 2348 | + |
| 2349 | + source = "http://archive.ubuntu.com/ubuntu raring-backports main" |
| 2350 | + key_id = "akey" |
| 2351 | + with patch_open() as (mock_open, mock_file): |
| 2352 | + fetch.add_source(source=source, key=key_id) |
| 2353 | + listdir.assert_called_with('/etc/yum.repos.d/') |
| 2354 | + mock_file.write.assert_has_calls([ |
| 2355 | + call("[archive.ubuntu.com_ubuntu raring-backports main]\n"), |
| 2356 | + call("name=archive.ubuntu.com/ubuntu raring-backports main\n"), |
| 2357 | + call("baseurl=http://archive.ubuntu.com/ubuntu raring" |
| 2358 | + "-backports main\n\n")]) |
| 2359 | + check_call.assert_called_with(['rpm', '--import', key_id]) |
| 2360 | + |
| 2361 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2362 | + @patch.object(osplatform, 'get_platform') |
| 2363 | + @patch('subprocess.check_call') |
| 2364 | + def test_add_source_https_and_key_id_ubuntu(self, check_call, |
| 2365 | + platform, log): |
| 2366 | + platform.return_value = 'ubuntu' |
| 2367 | + imp.reload(fetch) |
| 2368 | + |
| 2369 | + source = "https://USER:PASS@private-ppa.launchpad.net/project/awesome" |
| 2370 | + key_id = "GPGPGP" |
| 2371 | + fetch.add_source(source=source, key=key_id) |
| 2372 | + check_call.assert_has_calls([ |
| 2373 | + call(['add-apt-repository', '--yes', source]), |
| 2374 | + call(['apt-key', 'adv', '--keyserver', |
| 2375 | + 'hkp://keyserver.ubuntu.com:80', '--recv', key_id]) |
| 2376 | + ]) |
| 2377 | + |
| 2378 | + @patch('charmhelpers.fetch.centos.log') |
| 2379 | + @patch('os.listdir') |
| 2380 | + @patch.object(osplatform, 'get_platform') |
| 2381 | + @patch('subprocess.check_call') |
| 2382 | + def test_add_source_https_and_key_id_centos(self, check_call, |
| 2383 | + platform, listdir, log): |
| 2384 | + platform.return_value = 'centos' |
| 2385 | + imp.reload(fetch) |
| 2386 | + |
| 2387 | + source = "https://USER:PASS@private-ppa.launchpad.net/project/awesome" |
| 2388 | + key_id = "GPGPGP" |
| 2389 | + with patch_open() as (mock_open, mock_file): |
| 2390 | + fetch.add_source(source=source, key=key_id) |
| 2391 | + listdir.assert_called_with('/etc/yum.repos.d/') |
| 2392 | + mock_file.write.assert_has_calls([ |
| 2393 | + call("[_USER:PASS@private-ppa.launchpad" |
| 2394 | + ".net_project_awesome]\n"), |
| 2395 | + call("name=/USER:PASS@private-ppa.launchpad.net" |
| 2396 | + "/project/awesome\n"), |
| 2397 | + call("baseurl=https://USER:PASS@private-ppa.launchpad.net" |
| 2398 | + "/project/awesome\n\n")]) |
| 2399 | + check_call.assert_called_with(['rpm', '--import', key_id]) |
| 2400 | + |
| 2401 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2402 | + @patch.object(osplatform, 'get_platform') |
| 2403 | + @patch('subprocess.check_call') |
| 2404 | + def test_add_source_http_and_key_ubuntu(self, check_call, platform, log): |
| 2405 | + platform.return_value = 'ubuntu' |
| 2406 | + imp.reload(fetch) |
| 2407 | + |
| 2408 | source = "http://archive.ubuntu.com/ubuntu raring-backports main" |
| 2409 | key = ''' |
| 2410 | -----BEGIN PGP PUBLIC KEY BLOCK----- |
| 2411 | @@ -231,20 +455,55 @@ |
| 2412 | self.assertEqual(['apt-key', 'add', '-'], received_args) |
| 2413 | self.assertEqual(key, received_key.getvalue()) |
| 2414 | |
| 2415 | - @patch('charmhelpers.fetch.log') |
| 2416 | - def test_add_unparsable_source(self, log_): |
| 2417 | + @patch('charmhelpers.fetch.centos.log') |
| 2418 | + @patch.object(tempfile, 'NamedTemporaryFile') |
| 2419 | + @patch('os.listdir') |
| 2420 | + @patch.object(osplatform, 'get_platform') |
| 2421 | + @patch('subprocess.check_call') |
| 2422 | + def test_add_source_http_and_key_centos(self, check_call, platform, |
| 2423 | + listdir, temp_file, log): |
| 2424 | + platform.return_value = 'centos' |
| 2425 | + imp.reload(fetch) |
| 2426 | + |
| 2427 | + source = "http://archive.ubuntu.com/ubuntu raring-backports main" |
| 2428 | + key = ''' |
| 2429 | + -----BEGIN PGP PUBLIC KEY BLOCK----- |
| 2430 | + [...] |
| 2431 | + -----END PGP PUBLIC KEY BLOCK----- |
| 2432 | + ''' |
| 2433 | + temp_file.return_value.__enter__.return_value = key |
| 2434 | + temp_file.return_value.__exit__.return_value = key |
| 2435 | + |
| 2436 | + with patch_open() as (mock_open, mock_file): |
| 2437 | + fetch.add_source(source=source, key=key) |
| 2438 | + listdir.assert_called_with('/etc/yum.repos.d/') |
| 2439 | + self.assertTrue(log.called) |
| 2440 | + check_call.assert_called_with(['rpm', '--import', key]) |
| 2441 | + |
| 2442 | + @patch.object(osplatform, 'get_platform') |
| 2443 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2444 | + def test_add_unparsable_source(self, log_, platform): |
| 2445 | + platform.return_value = 'ubuntu' |
| 2446 | + imp.reload(fetch) |
| 2447 | + |
| 2448 | source = "propsed" # Minor typo |
| 2449 | fetch.add_source(source=source) |
| 2450 | self.assertEqual(1, log_.call_count) |
| 2451 | |
| 2452 | - def test_add_distro_source(self): |
| 2453 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2454 | + @patch.object(osplatform, 'get_platform') |
| 2455 | + def test_add_distro_source(self, platform, log): |
| 2456 | + platform.return_value = 'ubuntu' |
| 2457 | + imp.reload(fetch) |
| 2458 | + |
| 2459 | source = "distro" |
| 2460 | # distro is a noop but test validate no exception is thrown |
| 2461 | fetch.add_source(source=source) |
| 2462 | |
| 2463 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2464 | @patch.object(fetch, 'config') |
| 2465 | @patch.object(fetch, 'add_source') |
| 2466 | - def test_configure_sources_single_source(self, add_source, config): |
| 2467 | + def test_configure_sources_single_source(self, add_source, config, log): |
| 2468 | config.side_effect = ['source', 'key'] |
| 2469 | fetch.configure_sources() |
| 2470 | add_source.assert_called_with('source', 'key') |
| 2471 | @@ -296,15 +555,16 @@ |
| 2472 | ] |
| 2473 | self.assertRaises(fetch.SourceConfigError, fetch.configure_sources) |
| 2474 | |
| 2475 | - @patch.object(fetch, 'apt_update') |
| 2476 | + @patch.object(osplatform, 'get_platform') |
| 2477 | + @patch('charmhelpers.fetch.ubuntu.update') |
| 2478 | @patch.object(fetch, 'config') |
| 2479 | @patch.object(fetch, 'add_source') |
| 2480 | - def test_configure_sources_apt_update_called(self, add_source, config, |
| 2481 | - apt_update): |
| 2482 | + def test_configure_sources_update_called_ubuntu(self, add_source, config, |
| 2483 | + update, platform): |
| 2484 | config.side_effect = ['source', 'key'] |
| 2485 | fetch.configure_sources(update=True) |
| 2486 | add_source.assert_called_with('source', 'key') |
| 2487 | - self.assertTrue(apt_update.called) |
| 2488 | + self.assertTrue(update.called) |
| 2489 | |
| 2490 | |
| 2491 | class InstallTest(TestCase): |
| 2492 | @@ -403,7 +663,8 @@ |
| 2493 | @patch('charmhelpers.fetch.importlib.import_module') |
| 2494 | def test_skips_and_logs_missing_plugins(self, import_, log_): |
| 2495 | fetch_handlers = ['a.foo', 'b.foo', 'c.foo'] |
| 2496 | - import_.side_effect = (NotImplementedError, NotImplementedError, MagicMock()) |
| 2497 | + import_.side_effect = (NotImplementedError, NotImplementedError, |
| 2498 | + MagicMock()) |
| 2499 | plugins = fetch.plugins(fetch_handlers) |
| 2500 | |
| 2501 | self.assertEqual(1, len(plugins)) |
| 2502 | @@ -453,226 +714,330 @@ |
| 2503 | |
| 2504 | class AptTests(TestCase): |
| 2505 | |
| 2506 | + @patch.object(osplatform, 'get_platform') |
| 2507 | @patch('subprocess.call') |
| 2508 | - @patch.object(fetch, 'log') |
| 2509 | - def test_apt_upgrade_non_fatal(self, log, mock_call): |
| 2510 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2511 | + def test_apt_upgrade_non_fatal(self, log, mock_call, platform): |
| 2512 | + platform.return_value = 'ubuntu' |
| 2513 | + imp.reload(fetch) |
| 2514 | + |
| 2515 | options = ['--foo', '--bar'] |
| 2516 | fetch.apt_upgrade(options) |
| 2517 | |
| 2518 | - mock_call.assert_called_with(['apt-get', '--assume-yes', |
| 2519 | - '--foo', '--bar', 'upgrade'], |
| 2520 | - env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2521 | + mock_call.assert_called_with( |
| 2522 | + ['apt-get', '--assume-yes', |
| 2523 | + '--foo', '--bar', 'upgrade'], |
| 2524 | + env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2525 | |
| 2526 | + @patch.object(osplatform, 'get_platform') |
| 2527 | @patch('subprocess.check_call') |
| 2528 | - @patch.object(fetch, 'log') |
| 2529 | - def test_apt_upgrade_fatal(self, log, mock_call): |
| 2530 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2531 | + def test_apt_upgrade_fatal(self, log, mock_call, platform): |
| 2532 | + platform.return_value = 'ubuntu' |
| 2533 | + imp.reload(fetch) |
| 2534 | + |
| 2535 | options = ['--foo', '--bar'] |
| 2536 | fetch.apt_upgrade(options, fatal=True) |
| 2537 | |
| 2538 | - mock_call.assert_called_with(['apt-get', '--assume-yes', |
| 2539 | - '--foo', '--bar', 'upgrade'], |
| 2540 | - env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2541 | + mock_call.assert_called_with( |
| 2542 | + ['apt-get', '--assume-yes', |
| 2543 | + '--foo', '--bar', 'upgrade'], |
| 2544 | + env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2545 | |
| 2546 | + @patch.object(osplatform, 'get_platform') |
| 2547 | @patch('subprocess.check_call') |
| 2548 | - @patch.object(fetch, 'log') |
| 2549 | - def test_apt_dist_upgrade_fatal(self, log, mock_call): |
| 2550 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2551 | + def test_apt_dist_upgrade_fatal(self, log, mock_call, platform): |
| 2552 | + platform.return_value = 'ubuntu' |
| 2553 | + imp.reload(fetch) |
| 2554 | + |
| 2555 | options = ['--foo', '--bar'] |
| 2556 | fetch.apt_upgrade(options, fatal=True, dist=True) |
| 2557 | |
| 2558 | - mock_call.assert_called_with(['apt-get', '--assume-yes', |
| 2559 | - '--foo', '--bar', 'dist-upgrade'], |
| 2560 | - env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2561 | + mock_call.assert_called_with( |
| 2562 | + ['apt-get', '--assume-yes', |
| 2563 | + '--foo', '--bar', 'dist-upgrade'], |
| 2564 | + env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2565 | |
| 2566 | + @patch.object(osplatform, 'get_platform') |
| 2567 | @patch('subprocess.call') |
| 2568 | - @patch.object(fetch, 'log') |
| 2569 | - def test_installs_apt_packages(self, log, mock_call): |
| 2570 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2571 | + def test_installs_apt_packages(self, log, mock_call, platform): |
| 2572 | + platform.return_value = 'ubuntu' |
| 2573 | + imp.reload(fetch) |
| 2574 | + |
| 2575 | packages = ['foo', 'bar'] |
| 2576 | options = ['--foo', '--bar'] |
| 2577 | |
| 2578 | fetch.apt_install(packages, options) |
| 2579 | |
| 2580 | - mock_call.assert_called_with(['apt-get', '--assume-yes', |
| 2581 | - '--foo', '--bar', 'install', 'foo', 'bar'], |
| 2582 | - env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2583 | + mock_call.assert_called_with( |
| 2584 | + ['apt-get', '--assume-yes', |
| 2585 | + '--foo', '--bar', 'install', 'foo', 'bar'], |
| 2586 | + env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2587 | |
| 2588 | + @patch.object(osplatform, 'get_platform') |
| 2589 | @patch('subprocess.call') |
| 2590 | - @patch.object(fetch, 'log') |
| 2591 | - def test_installs_apt_packages_without_options(self, log, mock_call): |
| 2592 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2593 | + def test_installs_apt_packages_without_options(self, log, mock_call, |
| 2594 | + platform): |
| 2595 | + platform.return_value = 'ubuntu' |
| 2596 | + imp.reload(fetch) |
| 2597 | + |
| 2598 | packages = ['foo', 'bar'] |
| 2599 | |
| 2600 | fetch.apt_install(packages) |
| 2601 | |
| 2602 | - expected = ['apt-get', '--assume-yes', |
| 2603 | - '--option=Dpkg::Options::=--force-confold', |
| 2604 | - 'install', 'foo', 'bar'] |
| 2605 | - |
| 2606 | - mock_call.assert_called_with(expected, |
| 2607 | - env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2608 | - |
| 2609 | + mock_call.assert_called_with( |
| 2610 | + ['apt-get', '--assume-yes', |
| 2611 | + '--option=Dpkg::Options::=--force-confold', |
| 2612 | + 'install', 'foo', 'bar'], |
| 2613 | + env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2614 | + |
| 2615 | + @patch.object(osplatform, 'get_platform') |
| 2616 | @patch('subprocess.call') |
| 2617 | - @patch.object(fetch, 'log') |
| 2618 | - def test_installs_apt_packages_as_string(self, log, mock_call): |
| 2619 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2620 | + def test_installs_apt_packages_as_string(self, log, mock_call, platform): |
| 2621 | + platform.return_value = 'ubuntu' |
| 2622 | + imp.reload(fetch) |
| 2623 | + |
| 2624 | packages = 'foo bar' |
| 2625 | options = ['--foo', '--bar'] |
| 2626 | |
| 2627 | fetch.apt_install(packages, options) |
| 2628 | |
| 2629 | - expected = ['apt-get', '--assume-yes', |
| 2630 | - '--foo', '--bar', 'install', 'foo bar'] |
| 2631 | - |
| 2632 | - mock_call.assert_called_with(expected, |
| 2633 | - env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2634 | - |
| 2635 | + mock_call.assert_called_with( |
| 2636 | + ['apt-get', '--assume-yes', |
| 2637 | + '--foo', '--bar', 'install', 'foo bar'], |
| 2638 | + env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2639 | + |
| 2640 | + @patch.object(osplatform, 'get_platform') |
| 2641 | @patch('subprocess.check_call') |
| 2642 | - @patch.object(fetch, 'log') |
| 2643 | - def test_installs_apt_packages_with_possible_errors(self, log, check_call): |
| 2644 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2645 | + def test_installs_apt_packages_with_possible_errors(self, log, |
| 2646 | + check_call, platform): |
| 2647 | + platform.return_value = 'ubuntu' |
| 2648 | + imp.reload(fetch) |
| 2649 | + |
| 2650 | packages = ['foo', 'bar'] |
| 2651 | options = ['--foo', '--bar'] |
| 2652 | |
| 2653 | fetch.apt_install(packages, options, fatal=True) |
| 2654 | |
| 2655 | - check_call.assert_called_with(['apt-get', '--assume-yes', |
| 2656 | - '--foo', '--bar', 'install', 'foo', 'bar'], |
| 2657 | - env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2658 | - |
| 2659 | - @patch('subprocess.check_call') |
| 2660 | - @patch.object(fetch, 'log') |
| 2661 | - def test_purges_apt_packages_as_string_fatal(self, log, mock_call): |
| 2662 | - packages = 'irrelevant names' |
| 2663 | - mock_call.side_effect = OSError('fail') |
| 2664 | - |
| 2665 | - self.assertRaises(OSError, fetch.apt_purge, packages, fatal=True) |
| 2666 | - self.assertTrue(log.called) |
| 2667 | - |
| 2668 | - @patch('subprocess.check_call') |
| 2669 | - @patch.object(fetch, 'log') |
| 2670 | - def test_purges_apt_packages_fatal(self, log, mock_call): |
| 2671 | - packages = ['irrelevant', 'names'] |
| 2672 | - mock_call.side_effect = OSError('fail') |
| 2673 | - |
| 2674 | - self.assertRaises(OSError, fetch.apt_purge, packages, fatal=True) |
| 2675 | - self.assertTrue(log.called) |
| 2676 | - |
| 2677 | - @patch('subprocess.call') |
| 2678 | - @patch.object(fetch, 'log') |
| 2679 | - def test_purges_apt_packages_as_string_nofatal(self, log, mock_call): |
| 2680 | - packages = 'foo bar' |
| 2681 | - |
| 2682 | - fetch.apt_purge(packages) |
| 2683 | - |
| 2684 | - self.assertTrue(log.called) |
| 2685 | - mock_call.assert_called_with( |
| 2686 | - ['apt-get', '--assume-yes', 'purge', 'foo bar'], env=getenv( |
| 2687 | - {'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2688 | - |
| 2689 | - @patch('subprocess.call') |
| 2690 | - @patch.object(fetch, 'log') |
| 2691 | - def test_purges_apt_packages_nofatal(self, log, mock_call): |
| 2692 | - packages = ['foo', 'bar'] |
| 2693 | - |
| 2694 | - fetch.apt_purge(packages) |
| 2695 | - |
| 2696 | - self.assertTrue(log.called) |
| 2697 | - mock_call.assert_called_with( |
| 2698 | - ['apt-get', '--assume-yes', 'purge', 'foo', 'bar'], env=getenv( |
| 2699 | - {'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2700 | - |
| 2701 | - @patch('subprocess.check_call') |
| 2702 | - @patch.object(fetch, 'log') |
| 2703 | - def test_mark_apt_packages_as_string_fatal(self, log, mock_call): |
| 2704 | - packages = 'irrelevant names' |
| 2705 | - mock_call.side_effect = OSError('fail') |
| 2706 | - |
| 2707 | - self.assertRaises(OSError, fetch.apt_mark, packages, sentinel.mark, |
| 2708 | - fatal=True) |
| 2709 | - self.assertTrue(log.called) |
| 2710 | - |
| 2711 | - @patch('subprocess.check_call') |
| 2712 | - @patch.object(fetch, 'log') |
| 2713 | - def test_mark_apt_packages_fatal(self, log, mock_call): |
| 2714 | - packages = ['irrelevant', 'names'] |
| 2715 | - mock_call.side_effect = OSError('fail') |
| 2716 | - |
| 2717 | - self.assertRaises(OSError, fetch.apt_mark, packages, sentinel.mark, |
| 2718 | - fatal=True) |
| 2719 | - self.assertTrue(log.called) |
| 2720 | - |
| 2721 | - @patch('subprocess.call') |
| 2722 | - @patch.object(fetch, 'log') |
| 2723 | - def test_mark_apt_packages_as_string_nofatal(self, log, mock_call): |
| 2724 | - packages = 'foo bar' |
| 2725 | - |
| 2726 | - fetch.apt_mark(packages, sentinel.mark) |
| 2727 | - |
| 2728 | - self.assertTrue(log.called) |
| 2729 | - mock_call.assert_called_with(['apt-mark', sentinel.mark, 'foo bar'], |
| 2730 | - universal_newlines=True) |
| 2731 | - |
| 2732 | - @patch('subprocess.call') |
| 2733 | - @patch.object(fetch, 'log') |
| 2734 | - def test_mark_apt_packages_nofatal(self, log, mock_call): |
| 2735 | - packages = ['foo', 'bar'] |
| 2736 | - |
| 2737 | - fetch.apt_mark(packages, sentinel.mark) |
| 2738 | - |
| 2739 | - self.assertTrue(log.called) |
| 2740 | - mock_call.assert_called_with(['apt-mark', sentinel.mark, 'foo', 'bar'], |
| 2741 | - universal_newlines=True) |
| 2742 | - |
| 2743 | - @patch('subprocess.check_call') |
| 2744 | - @patch.object(fetch, 'log') |
| 2745 | - def test_mark_apt_packages_nofatal_abortonfatal(self, log, mock_call): |
| 2746 | + check_call.assert_called_with( |
| 2747 | + ['apt-get', '--assume-yes', |
| 2748 | + '--foo', '--bar', 'install', 'foo', 'bar'], |
| 2749 | + env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2750 | + |
| 2751 | + @patch.object(osplatform, 'get_platform') |
| 2752 | + @patch('subprocess.check_call') |
| 2753 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2754 | + def test_purges_apt_packages_as_string_fatal(self, log, mock_call, |
| 2755 | + platform): |
| 2756 | + platform.return_value = 'ubuntu' |
| 2757 | + imp.reload(fetch) |
| 2758 | + |
| 2759 | + packages = 'irrelevant names' |
| 2760 | + mock_call.side_effect = OSError('fail') |
| 2761 | + |
| 2762 | + self.assertRaises(OSError, fetch.apt_purge, packages, fatal=True) |
| 2763 | + self.assertTrue(log.called) |
| 2764 | + |
| 2765 | + @patch.object(osplatform, 'get_platform') |
| 2766 | + @patch('subprocess.check_call') |
| 2767 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2768 | + def test_purges_apt_packages_fatal(self, log, mock_call, platform): |
| 2769 | + platform.return_value = 'ubuntu' |
| 2770 | + imp.reload(fetch) |
| 2771 | + |
| 2772 | + packages = ['irrelevant', 'names'] |
| 2773 | + mock_call.side_effect = OSError('fail') |
| 2774 | + |
| 2775 | + self.assertRaises(OSError, fetch.apt_purge, packages, fatal=True) |
| 2776 | + self.assertTrue(log.called) |
| 2777 | + |
| 2778 | + @patch.object(osplatform, 'get_platform') |
| 2779 | + @patch('subprocess.call') |
| 2780 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2781 | + def test_purges_apt_packages_as_string_nofatal(self, log, mock_call, |
| 2782 | + platform): |
| 2783 | + platform.return_value = 'ubuntu' |
| 2784 | + imp.reload(fetch) |
| 2785 | + |
| 2786 | + packages = 'foo bar' |
| 2787 | + |
| 2788 | + fetch.apt_purge(packages) |
| 2789 | + |
| 2790 | + self.assertTrue(log.called) |
| 2791 | + mock_call.assert_called_with( |
| 2792 | + ['apt-get', '--assume-yes', 'purge', 'foo bar'], |
| 2793 | + env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2794 | + |
| 2795 | + @patch.object(osplatform, 'get_platform') |
| 2796 | + @patch('subprocess.call') |
| 2797 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2798 | + def test_purges_apt_packages_nofatal(self, log, mock_call, platform): |
| 2799 | + platform.return_value = 'ubuntu' |
| 2800 | + imp.reload(fetch) |
| 2801 | + |
| 2802 | + packages = ['foo', 'bar'] |
| 2803 | + |
| 2804 | + fetch.apt_purge(packages) |
| 2805 | + |
| 2806 | + self.assertTrue(log.called) |
| 2807 | + mock_call.assert_called_with( |
| 2808 | + ['apt-get', '--assume-yes', 'purge', 'foo', 'bar'], |
| 2809 | + env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2810 | + |
| 2811 | + @patch.object(osplatform, 'get_platform') |
| 2812 | + @patch('subprocess.check_call') |
| 2813 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2814 | + def test_mark_apt_packages_as_string_fatal(self, log, mock_call, |
| 2815 | + platform): |
| 2816 | + packages = 'irrelevant names' |
| 2817 | + mock_call.side_effect = OSError('fail') |
| 2818 | + |
| 2819 | + self.assertRaises(OSError, fetch.apt_mark, packages, sentinel.mark, |
| 2820 | + fatal=True) |
| 2821 | + self.assertTrue(log.called) |
| 2822 | + |
| 2823 | + @patch.object(osplatform, 'get_platform') |
| 2824 | + @patch('subprocess.check_call') |
| 2825 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2826 | + def test_mark_apt_packages_fatal(self, log, mock_call, platform): |
| 2827 | + platform.return_value = 'ubuntu' |
| 2828 | + imp.reload(fetch) |
| 2829 | + |
| 2830 | + packages = ['irrelevant', 'names'] |
| 2831 | + mock_call.side_effect = OSError('fail') |
| 2832 | + |
| 2833 | + self.assertRaises(OSError, fetch.apt_mark, packages, sentinel.mark, |
| 2834 | + fatal=True) |
| 2835 | + self.assertTrue(log.called) |
| 2836 | + |
| 2837 | + @patch.object(osplatform, 'get_platform') |
| 2838 | + @patch('subprocess.call') |
| 2839 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2840 | + def test_mark_apt_packages_as_string_nofatal(self, log, mock_call, |
| 2841 | + platform): |
| 2842 | + platform.return_value = 'ubuntu' |
| 2843 | + imp.reload(fetch) |
| 2844 | + |
| 2845 | + packages = 'foo bar' |
| 2846 | + |
| 2847 | + fetch.apt_mark(packages, sentinel.mark) |
| 2848 | + |
| 2849 | + self.assertTrue(log.called) |
| 2850 | + mock_call.assert_called_with( |
| 2851 | + ['apt-mark', sentinel.mark, 'foo bar'], |
| 2852 | + universal_newlines=True) |
| 2853 | + |
| 2854 | + @patch.object(osplatform, 'get_platform') |
| 2855 | + @patch('subprocess.call') |
| 2856 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2857 | + def test_mark_apt_packages_nofatal(self, log, mock_call, |
| 2858 | + platform): |
| 2859 | + platform.return_value = 'ubuntu' |
| 2860 | + imp.reload(fetch) |
| 2861 | + |
| 2862 | + packages = ['foo', 'bar'] |
| 2863 | + |
| 2864 | + fetch.apt_mark(packages, sentinel.mark) |
| 2865 | + |
| 2866 | + self.assertTrue(log.called) |
| 2867 | + mock_call.assert_called_with( |
| 2868 | + ['apt-mark', sentinel.mark, 'foo', 'bar'], |
| 2869 | + universal_newlines=True) |
| 2870 | + |
| 2871 | + @patch.object(osplatform, 'get_platform') |
| 2872 | + @patch('subprocess.check_call') |
| 2873 | + @patch('charmhelpers.fetch.ubuntu.log') |
| 2874 | + def test_mark_apt_packages_nofatal_abortonfatal(self, log, mock_call, |
| 2875 | + platform): |
| 2876 | + platform.return_value = 'ubuntu' |
| 2877 | + imp.reload(fetch) |
| 2878 | + |
| 2879 | packages = ['foo', 'bar'] |
| 2880 | |
| 2881 | fetch.apt_mark(packages, sentinel.mark, fatal=True) |
| 2882 | |
| 2883 | self.assertTrue(log.called) |
| 2884 | - mock_call.assert_called_with(['apt-mark', sentinel.mark, 'foo', 'bar'], |
| 2885 | - universal_newlines=True) |
| 2886 | - |
| 2887 | - @patch.object(fetch, 'apt_mark') |
| 2888 | - def test_apt_hold(self, apt_mark): |
| 2889 | + mock_call.assert_called_with( |
| 2890 | + ['apt-mark', sentinel.mark, 'foo', 'bar'], |
| 2891 | + universal_newlines=True) |
| 2892 | + |
| 2893 | + @patch.object(osplatform, 'get_platform') |
| 2894 | + @patch('charmhelpers.fetch.ubuntu.apt_mark') |
| 2895 | + def test_apt_hold(self, apt_mark, platform): |
| 2896 | + platform.return_value = 'ubuntu' |
| 2897 | + imp.reload(fetch) |
| 2898 | + |
| 2899 | fetch.apt_hold(sentinel.packages) |
| 2900 | apt_mark.assert_called_once_with(sentinel.packages, 'hold', |
| 2901 | fatal=False) |
| 2902 | |
| 2903 | - @patch.object(fetch, 'apt_mark') |
| 2904 | - def test_apt_hold_fatal(self, apt_mark): |
| 2905 | + @patch.object(osplatform, 'get_platform') |
| 2906 | + @patch('charmhelpers.fetch.ubuntu.apt_mark') |
| 2907 | + def test_apt_hold_fatal(self, apt_mark, platform): |
| 2908 | + platform.return_value = 'ubuntu' |
| 2909 | + imp.reload(fetch) |
| 2910 | + |
| 2911 | fetch.apt_hold(sentinel.packages, fatal=sentinel.fatal) |
| 2912 | apt_mark.assert_called_once_with(sentinel.packages, 'hold', |
| 2913 | fatal=sentinel.fatal) |
| 2914 | |
| 2915 | - @patch.object(fetch, 'apt_mark') |
| 2916 | - def test_apt_unhold(self, apt_mark): |
| 2917 | + @patch.object(osplatform, 'get_platform') |
| 2918 | + @patch('charmhelpers.fetch.ubuntu.apt_mark') |
| 2919 | + def test_apt_unhold(self, apt_mark, platform): |
| 2920 | + platform.return_value = 'ubuntu' |
| 2921 | + imp.reload(fetch) |
| 2922 | + |
| 2923 | fetch.apt_unhold(sentinel.packages) |
| 2924 | apt_mark.assert_called_once_with(sentinel.packages, 'unhold', |
| 2925 | fatal=False) |
| 2926 | |
| 2927 | - @patch.object(fetch, 'apt_mark') |
| 2928 | - def test_apt_unhold_fatal(self, apt_mark): |
| 2929 | + @patch.object(osplatform, 'get_platform') |
| 2930 | + @patch('charmhelpers.fetch.ubuntu.apt_mark') |
| 2931 | + def test_apt_unhold_fatal(self, apt_mark, platform): |
| 2932 | + platform.return_value = 'ubuntu' |
| 2933 | + imp.reload(fetch) |
| 2934 | + |
| 2935 | fetch.apt_unhold(sentinel.packages, fatal=sentinel.fatal) |
| 2936 | apt_mark.assert_called_once_with(sentinel.packages, 'unhold', |
| 2937 | fatal=sentinel.fatal) |
| 2938 | |
| 2939 | + @patch.object(osplatform, 'get_platform') |
| 2940 | @patch('subprocess.check_call') |
| 2941 | - def test_apt_update_fatal(self, check_call): |
| 2942 | + def test_apt_update_fatal(self, check_call, platform): |
| 2943 | + platform.return_value = 'ubuntu' |
| 2944 | + imp.reload(fetch) |
| 2945 | + |
| 2946 | fetch.apt_update(fatal=True) |
| 2947 | check_call.assert_called_with( |
| 2948 | - ['apt-get', 'update'], env=getenv( |
| 2949 | - {'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2950 | + ['apt-get', 'update'], |
| 2951 | + env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2952 | |
| 2953 | + @patch.object(osplatform, 'get_platform') |
| 2954 | @patch('subprocess.call') |
| 2955 | - def test_apt_update_nonfatal(self, call): |
| 2956 | + def test_apt_update_nonfatal(self, call, platform): |
| 2957 | + platform.return_value = 'ubuntu' |
| 2958 | + imp.reload(fetch) |
| 2959 | + |
| 2960 | fetch.apt_update() |
| 2961 | call.assert_called_with( |
| 2962 | - ['apt-get', 'update'], env=getenv( |
| 2963 | - {'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2964 | + ['apt-get', 'update'], |
| 2965 | + env=getenv({'DEBIAN_FRONTEND': 'noninteractive'})) |
| 2966 | |
| 2967 | + @patch.object(osplatform, 'get_platform') |
| 2968 | @patch('subprocess.check_call') |
| 2969 | @patch('time.sleep') |
| 2970 | - def test_run_apt_command_retries_if_fatal(self, check_call, sleep): |
| 2971 | + def test_run_apt_command_retries_if_fatal(self, check_call, |
| 2972 | + sleep, platform): |
| 2973 | """The _run_apt_command function retries the command if it can't get |
| 2974 | the APT lock.""" |
| 2975 | + platform.return_value = 'ubuntu' |
| 2976 | + imp.reload(fetch) |
| 2977 | + |
| 2978 | self.called = False |
| 2979 | |
| 2980 | def side_effect(*args, **kwargs): |
| 2981 | @@ -690,5 +1055,217 @@ |
| 2982 | check_call.side_effect = side_effect |
| 2983 | check_call.return_value = 0 |
| 2984 | |
| 2985 | - fetch._run_apt_command(["some", "command"], fatal=True) |
| 2986 | + from charmhelpers.fetch.ubuntu import _run_apt_command |
| 2987 | + _run_apt_command(["some", "command"], fatal=True) |
| 2988 | + self.assertTrue(sleep.called) |
| 2989 | + |
| 2990 | + |
| 2991 | +class YumTests(TestCase): |
| 2992 | + |
| 2993 | + @patch.object(osplatform, 'get_platform') |
| 2994 | + @patch('subprocess.call') |
| 2995 | + @patch('charmhelpers.fetch.centos.log') |
| 2996 | + def test_yum_upgrade_non_fatal(self, log, mock_call, platform): |
| 2997 | + platform.return_value = 'centos' |
| 2998 | + imp.reload(fetch) |
| 2999 | + |
| 3000 | + options = ['--foo', '--bar'] |
| 3001 | + fetch.upgrade(options) |
| 3002 | + |
| 3003 | + mock_call.assert_called_with(['yum', '--assumeyes', |
| 3004 | + '--foo', '--bar', 'upgrade'], |
| 3005 | + env=getenv()) |
| 3006 | + |
| 3007 | + @patch.object(osplatform, 'get_platform') |
| 3008 | + @patch('subprocess.call') |
| 3009 | + @patch('charmhelpers.fetch.centos.log') |
| 3010 | + def test_yum_upgrade_fatal(self, log, mock_call, platform): |
| 3011 | + platform.return_value = 'centos' |
| 3012 | + imp.reload(fetch) |
| 3013 | + |
| 3014 | + options = ['--foo', '--bar'] |
| 3015 | + fetch.upgrade(options, fatal=True) |
| 3016 | + |
| 3017 | + mock_call.assert_called_with(['yum', '--assumeyes', |
| 3018 | + '--foo', '--bar', 'upgrade'], |
| 3019 | + env=getenv()) |
| 3020 | + |
| 3021 | + @patch.object(osplatform, 'get_platform') |
| 3022 | + @patch('subprocess.call') |
| 3023 | + @patch('charmhelpers.fetch.centos.log') |
| 3024 | + def test_installs_yum_packages(self, log, mock_call, platform): |
| 3025 | + platform.return_value = 'centos' |
| 3026 | + imp.reload(fetch) |
| 3027 | + |
| 3028 | + packages = ['foo', 'bar'] |
| 3029 | + options = ['--foo', '--bar'] |
| 3030 | + |
| 3031 | + fetch.install(packages, options) |
| 3032 | + |
| 3033 | + mock_call.assert_called_with(['yum', '--assumeyes', |
| 3034 | + '--foo', '--bar', 'install', |
| 3035 | + 'foo', 'bar'], |
| 3036 | + env=getenv()) |
| 3037 | + |
| 3038 | + @patch.object(osplatform, 'get_platform') |
| 3039 | + @patch('subprocess.call') |
| 3040 | + @patch('charmhelpers.fetch.centos.log') |
| 3041 | + def test_installs_yum_packages_without_options(self, log, mock_call, |
| 3042 | + platform): |
| 3043 | + platform.return_value = 'centos' |
| 3044 | + imp.reload(fetch) |
| 3045 | + |
| 3046 | + packages = ['foo', 'bar'] |
| 3047 | + fetch.install(packages) |
| 3048 | + |
| 3049 | + mock_call.assert_called_with(['yum', '--assumeyes', |
| 3050 | + 'install', 'foo', 'bar'], |
| 3051 | + env=getenv()) |
| 3052 | + |
| 3053 | + @patch.object(osplatform, 'get_platform') |
| 3054 | + @patch('subprocess.call') |
| 3055 | + @patch('charmhelpers.fetch.centos.log') |
| 3056 | + def test_installs_yum_packages_as_string(self, log, mock_call, |
| 3057 | + platform): |
| 3058 | + platform.return_value = 'centos' |
| 3059 | + imp.reload(fetch) |
| 3060 | + |
| 3061 | + packages = 'foo bar' |
| 3062 | + fetch.install(packages) |
| 3063 | + |
| 3064 | + mock_call.assert_called_with(['yum', '--assumeyes', |
| 3065 | + 'install', 'foo bar'], |
| 3066 | + env=getenv()) |
| 3067 | + |
| 3068 | + @patch.object(osplatform, 'get_platform') |
| 3069 | + @patch('subprocess.call') |
| 3070 | + @patch('charmhelpers.fetch.centos.log') |
| 3071 | + def test_installs_yum_packages_with_possible_errors(self, log, mock_call, |
| 3072 | + platform): |
| 3073 | + platform.return_value = 'centos' |
| 3074 | + imp.reload(fetch) |
| 3075 | + |
| 3076 | + packages = ['foo', 'bar'] |
| 3077 | + options = ['--foo', '--bar'] |
| 3078 | + |
| 3079 | + fetch.install(packages, options, fatal=True) |
| 3080 | + |
| 3081 | + mock_call.assert_called_with(['yum', '--assumeyes', |
| 3082 | + '--foo', '--bar', |
| 3083 | + 'install', 'foo', 'bar'], |
| 3084 | + env=getenv()) |
| 3085 | + |
| 3086 | + @patch.object(osplatform, 'get_platform') |
| 3087 | + @patch('subprocess.check_call') |
| 3088 | + @patch('charmhelpers.fetch.centos.log') |
| 3089 | + def test_purges_yum_packages_as_string_fatal(self, log, mock_call, |
| 3090 | + platform): |
| 3091 | + platform.return_value = 'centos' |
| 3092 | + imp.reload(fetch) |
| 3093 | + |
| 3094 | + packages = 'irrelevant names' |
| 3095 | + mock_call.side_effect = OSError('fail') |
| 3096 | + |
| 3097 | + self.assertRaises(OSError, fetch.purge, packages, fatal=True) |
| 3098 | + self.assertTrue(log.called) |
| 3099 | + |
| 3100 | + @patch.object(osplatform, 'get_platform') |
| 3101 | + @patch('subprocess.check_call') |
| 3102 | + @patch('charmhelpers.fetch.centos.log') |
| 3103 | + def test_purges_yum_packages_fatal(self, log, mock_call, platform): |
| 3104 | + platform.return_value = 'centos' |
| 3105 | + imp.reload(fetch) |
| 3106 | + |
| 3107 | + packages = ['irrelevant', 'names'] |
| 3108 | + mock_call.side_effect = OSError('fail') |
| 3109 | + |
| 3110 | + self.assertRaises(OSError, fetch.purge, packages, fatal=True) |
| 3111 | + self.assertTrue(log.called) |
| 3112 | + |
| 3113 | + @patch.object(osplatform, 'get_platform') |
| 3114 | + @patch('subprocess.call') |
| 3115 | + @patch('charmhelpers.fetch.centos.log') |
| 3116 | + def test_purges_yum_packages_as_string_nofatal(self, log, mock_call, |
| 3117 | + platform): |
| 3118 | + platform.return_value = 'centos' |
| 3119 | + imp.reload(fetch) |
| 3120 | + |
| 3121 | + packages = 'foo bar' |
| 3122 | + fetch.purge(packages) |
| 3123 | + |
| 3124 | + self.assertTrue(log.called) |
| 3125 | + mock_call.assert_called_with(['yum', '--assumeyes', |
| 3126 | + 'remove', 'foo bar'], |
| 3127 | + env=getenv()) |
| 3128 | + |
| 3129 | + @patch.object(osplatform, 'get_platform') |
| 3130 | + @patch('subprocess.call') |
| 3131 | + @patch('charmhelpers.fetch.centos.log') |
| 3132 | + def test_purges_yum_packages_nofatal(self, log, mock_call, |
| 3133 | + platform): |
| 3134 | + platform.return_value = 'centos' |
| 3135 | + imp.reload(fetch) |
| 3136 | + |
| 3137 | + packages = ['foo', 'bar'] |
| 3138 | + fetch.purge(packages) |
| 3139 | + |
| 3140 | + self.assertTrue(log.called) |
| 3141 | + mock_call.assert_called_with(['yum', '--assumeyes', |
| 3142 | + 'remove', 'foo', 'bar'], |
| 3143 | + env=getenv()) |
| 3144 | + |
| 3145 | + @patch.object(osplatform, 'get_platform') |
| 3146 | + @patch('subprocess.check_call') |
| 3147 | + @patch('charmhelpers.fetch.centos.log') |
| 3148 | + def test_yum_update_fatal(self, log, check_call, platform): |
| 3149 | + platform.return_value = 'centos' |
| 3150 | + imp.reload(fetch) |
| 3151 | + |
| 3152 | + fetch.update(fatal=True) |
| 3153 | + check_call.assert_called_with(['yum', '--assumeyes', 'update'], |
| 3154 | + env=getenv()) |
| 3155 | + self.assertTrue(log.called) |
| 3156 | + |
| 3157 | + @patch.object(osplatform, 'get_platform') |
| 3158 | + @patch('subprocess.check_output') |
| 3159 | + @patch('charmhelpers.fetch.centos.log') |
| 3160 | + def test_yum_search(self, log, check_output, platform): |
| 3161 | + platform.return_value = 'centos' |
| 3162 | + imp.reload(fetch) |
| 3163 | + |
| 3164 | + package = ['irrelevant'] |
| 3165 | + |
| 3166 | + from charmhelpers.fetch.centos import yum_search |
| 3167 | + yum_search(package) |
| 3168 | + check_output.assert_called_with(['yum', 'search', 'irrelevant']) |
| 3169 | + self.assertTrue(log.called) |
| 3170 | + |
| 3171 | + @patch.object(osplatform, 'get_platform') |
| 3172 | + @patch('subprocess.check_call') |
| 3173 | + @patch('time.sleep') |
| 3174 | + def test_run_yum_command_retries_if_fatal(self, check_call, |
| 3175 | + sleep, platform): |
| 3176 | + """The _run_yum_command function retries the command if it can't get |
| 3177 | + the YUM lock.""" |
| 3178 | + platform.return_value = 'centos' |
| 3179 | + imp.reload(fetch) |
| 3180 | + |
| 3181 | + self.called = False |
| 3182 | + |
| 3183 | + def side_effect(*args, **kwargs): |
| 3184 | + """ |
| 3185 | + First, raise an exception (can't acquire lock), then return 0 |
| 3186 | + (the lock is grabbed). |
| 3187 | + """ |
| 3188 | + if not self.called: |
| 3189 | + self.called = True |
| 3190 | + raise subprocess.CalledProcessError( |
| 3191 | + returncode=1, cmd="some command") |
| 3192 | + else: |
| 3193 | + return 0 |
| 3194 | + |
| 3195 | + check_call.side_effect = side_effect |
| 3196 | + check_call.return_value = 0 |
| 3197 | + from charmhelpers.fetch.centos import _run_yum_command |
| 3198 | + _run_yum_command(["some", "command"], fatal=True) |
| 3199 | self.assertTrue(sleep.called) |
| 3200 | |
| 3201 | === modified file 'tests/fetch/test_giturl.py' |
| 3202 | --- tests/fetch/test_giturl.py 2016-01-08 13:27:39 +0000 |
| 3203 | +++ tests/fetch/test_giturl.py 2016-08-12 07:09:49 +0000 |
| 3204 | @@ -7,6 +7,7 @@ |
| 3205 | MagicMock, |
| 3206 | patch, |
| 3207 | ) |
| 3208 | +from charmhelpers.core.host import chdir |
| 3209 | |
| 3210 | import six |
| 3211 | if six.PY3: |
| 3212 | @@ -14,7 +15,7 @@ |
| 3213 | else: |
| 3214 | from urlparse import urlparse |
| 3215 | |
| 3216 | -from charmhelpers.core.host import chdir |
| 3217 | + |
| 3218 | try: |
| 3219 | from charmhelpers.fetch import ( |
| 3220 | giturl, |
| 3221 | @@ -57,7 +58,8 @@ |
| 3222 | self.fh.load_plugins = MagicMock() |
| 3223 | self.fh.clone(url, dest_path, branch, None) |
| 3224 | |
| 3225 | - check_call.assert_called_with(['git', 'clone', url, dest_path, '--branch', branch]) |
| 3226 | + check_call.assert_called_with( |
| 3227 | + ['git', 'clone', url, dest_path, '--branch', branch]) |
| 3228 | |
| 3229 | for url in self.invalid_urls: |
| 3230 | with patch.dict('os.environ', {'CHARM_DIR': 'foo'}): |
| 3231 | @@ -73,7 +75,8 @@ |
| 3232 | with chdir(src): |
| 3233 | subprocess.check_call(['git', 'init']) |
| 3234 | subprocess.check_call(['git', 'config', 'user.name', 'Joe']) |
| 3235 | - subprocess.check_call(['git', 'config', 'user.email', 'joe@test.com']) |
| 3236 | + subprocess.check_call( |
| 3237 | + ['git', 'config', 'user.email', 'joe@test.com']) |
| 3238 | subprocess.check_call(['touch', 'foo']) |
| 3239 | subprocess.check_call(['git', 'add', 'foo']) |
| 3240 | subprocess.check_call(['git', 'commit', '-m', 'test']) |


Denis,
Thanks for this merge proposal! I am very interested in adding support for CentOS.
Added a few comments in the code about _removing_ docstrings or changing docstring quotes to be inconsistent in the file. Since this is a library we should docstring every non private method.
Tried running `make test` in this branch and the "lint" target failed with several errors in the files you added, edited. Please make sure this branch passes lint and test make targets.
This is a big change, and many charms use charm-helpers. If you could help identify a way (or a charm) to verify this does not break backward compatibility with charms that install charm-helpers that would be most appreciated.