Merge lp:~lazypower/charms/trusty/docker/i-hate-bzr into lp:charms/trusty/docker

[Charles Butler]

Updates to v0.1.6 of the docker charm, which adds AUFS backend storage support.

The existing charm deploys device mapper, leaving this enabled by default is not desireable as Docker Inc has depreciated support for this storage engine. Its possible to upgrade existing clusters, however the upgrade path takes some manual intervention of exporting and re-importing the containers.

This has potential side-effects for data-containers, and existing services which will prevent them from starting. Full export and recovery instructions are included in the README along with validation of AUFS storage driver from a devicemapper upgrade have been included in the test suite.

[lp:~lazypower/charms/trusty/docker/i-hate-bzr]

8. By Charles Butler on 2015-07-21

Removes local charm from test

[Matt Bruzek (mbruzek)]

This branch seems to merge successfully, I think you have mastered bzr.

I like the changes in made in this pull request, but the tests do not pass on bundletester on azure or kvm.

Issue #1: There may be something wrong with the way you install charm-tools. Running "make lint" fails with "SyntaxError: invalid syntax". I created a bug about this problem: https://bugs.launchpad.net/charm-tools/+bug/1477194

This error may be related to your use of tox or how you install charm-tools. I was able to "pip install charm-tools" in another virtualenv WITHOUT any Syntax errors.

Issue #2: The docker daemon does not start on the unit. At the end of the tests I expect to see the docker daemon running on the unit and it is not.

Issue #3: I believe the test that checks for version 1.7.0 needs a sleep after the wait. I have created a merge proposal for this branch https://code.launchpad.net/~mbruzek/charms/trusty/docker/sleep5/+merge/265554 with my suggestion.

review: Needs Fixing

Unmerged revisions

8. By Charles Butler on 2015-07-21

Removes local charm from test

7. By Charles Butler on 2015-07-21

Fresh tree to deal with BZRs inconsistent lack of brain cells

=== modified file 'Makefile'
--- Makefile	2015-07-10 20:52:16 +0000
+++ Makefile	2015-07-21 21:47:33 +0000
@@ -6,7 +6,7 @@
 /usr/bin/tox:
 	sudo apt-get install -y  python-tox python-dev python-virtualenv
 
-lint: /usr/bin/tox 
+lint: /usr/bin/tox
 	tox -e lint
 
 

=== modified file 'README.md'
--- README.md	2015-07-17 13:22:34 +0000
+++ README.md	2015-07-21 21:47:33 +0000
@@ -22,7 +22,7 @@
 
 Step by step instructions on using the docker charm:
 
-    juju deploy cs:trusty/docker
+    juju deploy cs:~lazypower/trusty/docker
 
 ## Scale out Usage
 
@@ -47,6 +47,15 @@
     leverage a yaml file to spin up and manage multiple containers that
     comprise a single application stack.
 
+- aufs : **new as of v0.1.6** Defaults the backend storage driver to AUFS. The
+    older option of device mapper was horribly broken in most setups, and has
+    been completely depreciated by the Docker foundation. Disable to keep the
+    DeviceMapper backend. - Not recommended. **note** this will break existing
+    containers if you upgrade existing setups. Ensure you account for this when
+    upgrading your docker clusters that are previously deployed with this charm.
+
+    > See blurb under Known Issues for migration instructions.
+
 ## Known Limitations and Issues
 
 
@@ -86,7 +95,16 @@
 There is however [a bug](https://github.com/chuckbutler/docker-charm/issues/13) to track the
 progress of this feature.
 
-
+#### AUFS Upgrade Stopped my containers from working
+
+If you have older containers deployed and running, you will need to pause them
+and export. Once the tarballs of the containers have been exported - upgrade
+your cluster and reimport following the CLI instructions below as a guide
+
+    docker export <<container id>> > mycontainer-latest.tgz
+    # upgrade
+    docker import -i mycontainer-latest - mycontainer:latest
+    docker run <<options>> mycontainer:latest
 
 
 # Contact Information

=== modified file 'config.yaml'
--- config.yaml	2015-07-10 20:52:16 +0000
+++ config.yaml	2015-07-21 21:47:33 +0000
@@ -16,4 +16,12 @@
     default: true
     description: |
         Install docker compose, the formation launch utility
-
+  aufs:
+    type: boolean
+    default: true
+    description: |
+        Change the backend storage driver from devicemapper to AUFS.
+        The device mapper storage engine is known to be problematic, and AUFS
+        is not enabled by default in Ubuntu. This will install AUFS support and
+        swap the backend. **NOTE** It is not recommended to reconfigure this post
+        deployment. You may incur dataloss.

=== removed file 'hooks/setup.py'
--- hooks/setup.py	2015-07-17 13:22:34 +0000
+++ hooks/setup.py	1970-01-01 00:00:00 +0000
@@ -1,22 +0,0 @@
-import subprocess
-import sys
-import os
-
-
-def pre_install():
-    """
-    Do any setup required before the install hook.
-    """
-    try:
-        import charmhelpers  # noqa
-        import ansiblecharm  # noqa
-        from path import path  # noqa
-    except ImportError:
-        subprocess.check_call(['hooks/setup.sh'])
-        subprocess.check_call("pip install -r hooks/python-pkgs.txt",
-                              shell=True)
-
-        from path import path
-
-        pth = str(path(os.environ['CHARM_DIR']) / 'src/ansiblecharm')
-        sys.path.append(pth)

=== removed file 'hooks/setup.sh'
--- hooks/setup.sh	2015-06-10 20:39:25 +0000
+++ hooks/setup.sh	1970-01-01 00:00:00 +0000
@@ -1,9 +0,0 @@
-#!/bin/bash
-set -e
-
-apt-get install -y \
-    python-dev \
-    python-pip \
-    git
-
-easy_install -U pip

=== modified file 'playbooks/config-changed.yaml'
--- playbooks/config-changed.yaml	2015-06-10 20:39:25 +0000
+++ playbooks/config-changed.yaml	2015-07-21 21:47:33 +0000
@@ -13,6 +13,36 @@
     service_label: "service={{ service_name }}"
     unit_label: "unit={{ local_unit }}"
 
+- name: grab release info
+  shell: uname -r
+  register: rel
+  when: aufs == true
+
+- set_fact:
+    thisrel: "{{ rel.stdout }}"
+  when: aufs == true
+
+- name: Install AUFS backend supporting packages
+  apt: name={{item}} state=present
+  with_items:
+      - "linux-image-extra-{{thisrel}}"
+      - aufs-tools
+  when: aufs == true
+
+- name: stop docker to remove device mapper
+  service: "name={{docker_version_name}} state=stopped"
+  when: aufs == true
+
+- name: Remove Devicemapper backend if present
+  file: path=/var/lib/docker/devicemapper state=absent
+  when: aufs == true
+
+- name: Register AUFS option
+  docker_opts: action=add
+               key="storage-driver" val="aufs"
+               yaml="{{ opts_yaml }}"
+  when: aufs == true
+
 - name: set service name as a label
   docker_opts: action=add
                key=label val="{{ service_label }}"
@@ -28,6 +58,7 @@
   notify:
     - calculate docker opts
     - render docker defaults
+    - restart docker
 
 - name: Install docker compose
   include: install-compose.yaml

=== modified file 'tests/10-deploy-test'
--- tests/10-deploy-test	2015-06-02 17:33:17 +0000
+++ tests/10-deploy-test	2015-07-21 21:47:33 +0000
@@ -66,9 +66,9 @@
     def test_latest_config_option(self):
         """ Set config option to latest and verify docker is installed """
         self.deployment.configure('docker', {'latest': True,
-                                             'version': '1.5.0'})
+                                             'version': '1.7.0'})
         self.deployment.sentry.wait()
-        command = 'dpkg -l lxc-docker-1.5.0'
+        command = 'dpkg -l lxc-docker-1.7.0'
         output, code = self.docker_unit.run(command)
         print(output)
         if output.find('ii') == -1:
@@ -81,5 +81,16 @@
             message = "Leftover docker.io package found"
             amulet.raise_status(amulet.Fail, msg=message)
 
+    def test_latest_config_aufs_enabled(self):
+        ''' 
+            This test has a byproduct of actually checking to make sure
+	    we are cleaning up the broken devicemapper leftovers. Which
+	    is totally cool
+        '''
+        command = "docker info"
+        output, code = self.docker_unit.run(command)
+        print(output)
+        assert "aufs" in output
+
 if __name__ == '__main__':
     unittest.main()