Merge lp:~aaron-whitehouse/duplicity/08-docker-local-import into lp:~duplicity-team/duplicity/0.8-series

[Aaron Whitehouse]

* Convert the Docker infrastructure to pull the local branch into duplicity_test. This allows testing the local branch with the known-good Docker environment, even if it has not yet been committed to trunk.
* As a consequence, remove the -r option to build-duplicity_test.sh. This functionality can be achieved by branching that revision before running the script.

* Convert the Docker infrastructure to pull the local branch into duplicity_test. This allows testing the local branch with the known-good Docker environment, even if it has not yet been committed to trunk.
* As a consequence, remove the -r option to build-duplicity_test.sh. This functionality can be achieved by branching that revision before running the script.

We have now spent a lot of time ensuring that the Docker containers work correctly and contain all of the relevant dependencies. We have a steady stream of people struggling to set up their environments correctly to test and we have people propose merges that do not pass all of our tests. We should leverage the Docker container work and make it easy for people to use these to test their work before submitting it for merging. I have personally been using this approach for a little while and find it a much easier way to run tests on my branch.

=== modified file 'AUTHORS'
--- AUTHORS	2018-08-01 15:47:17 +0000
+++ AUTHORS	2019-08-26 22:23:05 +0000
@@ -1,7 +1,7 @@
 Duplicity Authors
 -----------------
 
-- Aaron Whitehouse <[e-mail]>
+- Aaron Whitehouse <[e-mail]>
 - Alexander Zangerl <[e-mail]>
 - Andrea Grandi <[e-mail]>
 - Ben Escoto <[e-mail]>

=== modified file 'README-TESTING'
--- README-TESTING	2019-03-10 13:12:26 +0000
+++ README-TESTING	2019-08-26 22:23:05 +0000
@@ -1,9 +1,26 @@
-# Testing duplicty
+# Testing duplicity
 
 ## Introduction
-Duplicity's test concept bases on unit test.
-All tests are contained in the /testing folder of the main repository. 
-
+Duplicity's tests are unit tests contained in the /testing folder of the main repository.
+
+## Running tests on your branch
+The recommended approach is to test duplicity using Docker, to ensure that you are running tests in a known-good
+environment. This also creates additional containers to test backends. You can run tests on your branch as follows:
+a. Install Docker
+b. cd [BRANCH FOLDER]/testing/infrastructure/
+c. ./build-duplicity_test.sh to build the required Docker containers and copy your branch into
+   the duplicity_test container.
+d. ./setup.sh to start the containers and open a shell inside your branch folder in duplicity_test.
+e. cd testing/
+f. ./run-tests
+g. When you are finished, exit the Docker container and run ./teardown.sh to delete the containers.
+
+Please test your branch using this method and ensure all tests pass before submitting a merge request.
+
+The decorator @unittest.expectedFailure can be used to commit a known-failing test case without breaking the test suite,
+for example to exhibit the behaviour in a bug report before it has been fixed.
+
+## Manual testing and running individual tests
 As to see in the following sketch, there are several levels of testing duplicity and each can be used directly.
 
                                     ┌─────────────────────┐     
@@ -11,7 +28,7 @@
                                     ├─────────────────────┴────┐
                                     │                          │
                                     │  ┌──────────────────┐    │
-                                    │  │       tox        │    │
+                                    │  │  run-tests/tox   │    │
                                     │  └──────────────────┘    │
                                     │            │             │
                                     │            ▼             │
@@ -25,25 +42,28 @@
                                     │  └──────────────────┘    │
                                     │                          │
                                     └──────────────────────────┘
-1. Testing directly using __setup.py__
-Assuming that your machine has all the required dependencies installed, you can start all the unit tests by simply typing
-
-‘setup.py test‘ 
-
-2. Using __tox__
-Tox is a generic virtualenv management and test command line tool that is used for checking your package installs correctly with different Python versions and interpreters. It 
-runs the tests in each of the environments that are configured in the tox.ini file (see root folder of the repository)
-
-Duplicity uses tox to make it easy to test your code against multiple
-environments. Running tests using the commands above will automatically test
-code against different supported environments, including the versions of
-dependencies used by the Launchpad build system.
-
-A tox run can be started simply by typing 
+
+1. Docker container
+Even if you wish to run tests manually, we recommend that you do this inside the provided Docker container to ensure
+that you have a clean and reproducible environment with all required dependencies for executing these. Please follow
+steps a to d of the above section titled "Running tests on your branch".
+
+2. Using run-tests
+The run-tests script mentioned above is a shortcut to running all tests with tox. This is the recommended approach to
+run tests on your code. We recommend running this within the provided Docker container as mentioned above, but it is
+possible to run this directly if you have installed all required dependencies (see "Dependencies for testing" below).
+
+3. Using tox
+Tox is a generic virtualenv management and test command line tool that is used for checking your package installs
+correctly with different Python versions and interpreters. It runs the tests in each of the environments that are
+configured in the tox.ini file (see root folder of the repository). As mentioned, we recommend this is run inside the
+provided Docker container.
+
+A tox run can be started simply by typing
 
 ‘tox‘
 
-from the main duplicity folder. 
+from the main duplicity folder.
 
 You can run specific tests using:
 ‘tox -- [test filename][::TestClassName::test_method]‘
@@ -57,38 +77,22 @@
 
 Or stack these together, e.g.
 ‘tox -e py3 -- testing/unit/test_selection.py::MatchingTest::test_tuple_include‘
-for example if you are working on fixing a bug, but please do a full run-tests
-before submitting a merge request.
-
-Note: some tests require rdiff and pylint to be installed on the system for
-them to pass.
-
-Please run all tests on your branch (run-tests) before proposing a merge, to
-ensure that all tests pass. The decorator @unittest.expectedFailure can be used
-to commit a known-failing test case without breaking the test suite, for
-example to exhibit the behaviour in a bug report before it has been fixed.
-
-3. Via a __docker__ image 
-Testing on a developer's machine can be tricky. Testing duplicity requires a set of dependencies being installed and reacts sensitiviely to changes of the local python configuration. In order to make sure that such interactions do not pose any influence on executing the tests, docker is the technology of choice. 
-Along with the tests, a docker image has been created (cf. Dockerfile in root folder of repo) that ensure the following things:
-- It bases on a clean Ubunut 16.04
-- It installs all the required packages that are needed for testing
-- It then branches the repository of duplicty to the folder /duplicty/testing within the docker image
-- And installs all the required python packages (as defined in the requirements.txt)
-Therewith, the docker image provides a clean and reproducible environment for executing the tests of duplicty. 
-In order to get hands on the docker image you simply: 
-1) Install Docker on your machine (https://docs.docker.com/engine/installation/)
-2) Start the image docker run -it dernils/duplicitytest /bin/bash (if you did not use the image before, it will be downloaded automatically)
-3) At the prompt of the docker image type:
-‘cd /testing‘
-‘tox‘ 
-to start a run of the test cases. 
+
+This is helpful, for example, if you are working on fixing a bug, but please do a full run-tests before submitting a
+merge request.
+
+4. Testing directly using __setup.py__
+Assuming that your machine has all the required dependencies installed, you can start all the unit tests by simply typing
+
+‘setup.py test‘ 
 
 ## Dependencies for testing
-If you should prefer to execute the tests locally without using docker, the Dockerfile that is checked into the root folder of the repository contains useful information. It contains a section marked "The following packages are needed for testing duplicity". Within this section all dependencies that need to be installed on a machine to execute the test cases are identified. 
+If you should prefer to execute the tests locally without using Docker, see the Dockerfile in
+testing/infrastructure/duplicity_test/
+for requirements to correctly set up your environment.
 
 ## Working with test coverage
-Python makes it easy to determine, how well the tests cover the source code. 
+Python makes it easy to determine how well the tests cover the source code.
 
 You first run the tests __under observation__ of the coverage script:
 ‘coverage run setup.py test‘
@@ -96,46 +100,3 @@
 ‘coverage html --omit="testing/*,/usr/*"‘
 
 The report will be generated and stored in the folder htmlcov. 
-
-## The wider picture - supporting containers for testing
-Testing duplicity invokes backends. The backends are the places where the backup data is actually stored (e.g. an ftp server). In order to have the highest degree of control over the testing process, backends that can be set up locally are also operated in separated docker containers. The whole test infrastructure is shown in the following picture. 
-
-┌─────────────────────┐         ┌──────────────────────────────────────────┐   
-│docker image         │         │docker image                              │   
-│dernils/duplicitytest│         │dernils/duplicity_testinfrastructure_ssh  │   
-├─────────────────────┴────┐    ├──────────────────────────────────────────┴──┐
-│                          │    │                                             │
-│  ┌──────────────────┐    │    │   ┌──────────────────┐                      │
-│  │       tox        │    │ ┌──┼──▶│       sshd       │                      │
-│  └──────────────────┘    │ │  │   └──────────────────┘                      │
-│            │             │ │  │                                             │
-│            ▼             │ │  └─────────────────────────────────────────────┘
-│  ┌──────────────────┐    │ │  ┌──────────────────────────────────────────┐   
-│  │    unittests     │    │ │  │docker image                              │   
-│  └──────────────────┘    │ │  │dernils/duplicity_testinfrastructure_ftp  │   
-│            │             │ │  ├──────────────────────────────────────────┴──┐
-│            ▼             │ │  │                                             │
-│  ┌──────────────────┐    │ │  │   ┌──────────────────┐                      │
-│  │    duplicity     │◀───┼─┴──┼──▶│    pure-ftpd     │                      │
-│  └──────────────────┘    │    │   └──────────────────┘                      │
-│            │             │    │                                             │
-└────────────┼─────────────┘    └─────────────────────────────────────────────┘
-             │                                                                 
-             │                                                                 
-             │                                                                 
-             └────────────┐                                                    
-                          │                                                    
-       Internet .─────────┼─────────.                                          
-         _.────'          │          `─────.                                   
-     _.─'                 │                 `──.                               
-   ,'                     ▼                     `.                             
-  ;             ┌──────────────────┐              :                            
-  :             │     Dropbox      │              ;                            
-   ╲            └──────────────────┘             ╱                             
-    `.                                         ,'                              
-      `──.                                 _.─'                                
-          `─────.                   _.────'                                    
-                 `─────────────────'                                           
-
-The docker images that contain the test infrastructure are defined in the folder /testing/infrastructure. There is a build script to compile the Dockerfile into actual images (build-duplicitiy-test.sh). However, as all images are also published on the docker hub, it is not necessary to build the images before starting testing. Testing can directly be started by using the script setup.sh. If the required docker images are not yet existing, locally, they will be downloaded by Docker. 
-

=== modified file 'testing/infrastructure/build-duplicity_test.sh'
--- testing/infrastructure/build-duplicity_test.sh	2019-07-22 19:40:09 +0000
+++ testing/infrastructure/build-duplicity_test.sh	2019-08-26 22:23:05 +0000
@@ -23,9 +23,8 @@
 #
 
 usage() {
-    echo "Usage: $0 [-d <distro>] [-r <revno>]" 1>&2
+    echo "Usage: $0 [-d <distro>]" 1>&2
     echo "  - Default distro is 18.04" 1>&2
-    echo "  - Default revno is `bzr revno`" 1>&2
     exit 1;
 }
 
@@ -35,20 +34,13 @@
     DISTRO=18.04
 fi
 
-if [ "$REVNO" == "" ]; then
-    REVNO=`bzr revno`
-fi
-
 # possibly override with commandline
 
-while getopts ":d:r:h" opt; do
+while getopts ":d:h" opt; do
     case ${opt} in
         d )
             DISTRO=$OPTARG
             ;;
-        r )
-            REVNO=$OPTARG
-            ;;
         h )
             usage
             ;;
@@ -66,19 +58,12 @@
 
 # build duplicity_test
 
-cd duplicity_test
-cp -p ../../../requirements.txt .
-cp -p ../id_rsa .
-cp -p ../id_rsa.pub .
-docker build --build-arg REVNO=$REVNO -f Dockerfile-$DISTRO --tag firstprime/duplicity_test .
-rm requirements.txt
-rm id_rsa
-rm id_rsa.pub
-cd ..
+cd ../..
+docker build -f testing/infrastructure/duplicity_test/Dockerfile-$DISTRO --tag firstprime/duplicity_test .
 
+cd testing/infrastructure/ftp_server
 # build duplicity_ftp
 
-cd ftp_server
 docker build --tag firstprime/duplicity_ftp .
 cd ..
 

=== modified file 'testing/infrastructure/duplicity_test/Dockerfile'
--- testing/infrastructure/duplicity_test/Dockerfile	2019-07-22 19:40:09 +0000
+++ testing/infrastructure/duplicity_test/Dockerfile	2019-08-26 22:23:05 +0000
@@ -1,6 +1,7 @@
 # -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
 #
-# Copyright 2017 Nils Tekampe <[e-mail]>
+# Copyright 2019 Nils Tekampe <[e-mail]>,
+# Kenneth Loafman <[e-mail]> and Aaron Whitehouse <[e-mail]>
 #
 # This file is part of duplicity.
 #
@@ -18,6 +19,10 @@
 # along with duplicity; if not, write to the Free Software Foundation,
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
+# This Dockerfile assumes that docker built will be called from the parent
+# duplicity folder with -f testing/infrastructure/ so that the whole folder
+# can be copied into the container for testing.
+
 FROM ubuntu:18.04
 
 # Set locale to prevent UTF-8 errors
@@ -77,15 +82,15 @@
 
 # Copy a SSH key to the users folder that is used for some test cases
 USER root
-COPY ./id_rsa /home/test/.ssh/
-COPY ./id_rsa.pub /home/test/.ssh/
+COPY testing/infrastructure/id_rsa /home/test/.ssh/
+COPY testing/infrastructure/id_rsa.pub /home/test/.ssh/
 RUN chown -R test:test /home/test/.ssh
 RUN chmod 400 /home/test/.ssh/id_rsa
+
+# Copy the local duplicity folder into the container
+COPY . duplicity/
+RUN chown -R test:test duplicity
 USER test
 
-# Branch the duplicity repo for testing
-ARG REVNO=-1
-RUN bzr branch -r $REVNO lp:duplicity duplicity
-
 # Set final workdir to duplicity
 WORKDIR /home/test/duplicity

=== modified file 'testing/infrastructure/duplicity_test/Dockerfile-18.04'
--- testing/infrastructure/duplicity_test/Dockerfile-18.04	2019-07-22 19:40:09 +0000
+++ testing/infrastructure/duplicity_test/Dockerfile-18.04	2019-08-26 22:23:05 +0000
@@ -18,6 +18,10 @@
 # along with duplicity; if not, write to the Free Software Foundation,
 # Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 
+# This Dockerfile assumes that docker built will be called from the parent
+# duplicity folder with -f testing/infrastructure/ so that the whole folder
+# can be copied into the container for testing.
+
 FROM ubuntu:18.04
 
 # Set locale to prevent UTF-8 errors
@@ -77,15 +81,15 @@
 
 # Copy a SSH key to the users folder that is used for some test cases
 USER root
-COPY ./id_rsa /home/test/.ssh/
-COPY ./id_rsa.pub /home/test/.ssh/
+COPY testing/infrastructure/id_rsa /home/test/.ssh/
+COPY testing/infrastructure/id_rsa.pub /home/test/.ssh/
 RUN chown -R test:test /home/test/.ssh
 RUN chmod 400 /home/test/.ssh/id_rsa
+
+# Copy the local duplicity folder into the container
+COPY . duplicity/
+RUN chown -R test:test duplicity
 USER test
 
-# Branch the duplicity repo for testing
-ARG REVNO=-1
-RUN bzr branch -r $REVNO lp:duplicity duplicity
-
 # Set final workdir to duplicity
 WORKDIR /home/test/duplicity

=== modified file 'testing/infrastructure/duplicity_test/Dockerfile-18.10'
--- testing/infrastructure/duplicity_test/Dockerfile-18.10	2019-07-22 19:40:09 +0000
+++ testing/infrastructure/duplicity_test/Dockerfile-18.10	2019-08-26 22:23:05 +0000
@@ -77,15 +77,15 @@
 
 # Copy a SSH key to the users folder that is used for some test cases
 USER root
-COPY ./id_rsa /home/test/.ssh/
-COPY ./id_rsa.pub /home/test/.ssh/
+COPY testing/infrastructure/id_rsa /home/test/.ssh/
+COPY testing/infrastructure/id_rsa.pub /home/test/.ssh/
 RUN chown -R test:test /home/test/.ssh
 RUN chmod 400 /home/test/.ssh/id_rsa
+
+# Copy the local duplicity folder into the container
+COPY . duplicity/
+RUN chown -R test:test duplicity
 USER test
 
-# Branch the duplicity repo for testing
-ARG REVNO=-1
-RUN bzr branch -r $REVNO lp:duplicity duplicity
-
 # Set final workdir to duplicity
 WORKDIR /home/test/duplicity

=== modified file 'testing/infrastructure/duplicity_test/Dockerfile-19.04'
--- testing/infrastructure/duplicity_test/Dockerfile-19.04	2019-07-22 19:40:09 +0000
+++ testing/infrastructure/duplicity_test/Dockerfile-19.04	2019-08-26 22:23:05 +0000
@@ -75,15 +75,15 @@
 
 # Copy a SSH key to the users folder that is used for some test cases
 USER root
-COPY ./id_rsa /home/test/.ssh/
-COPY ./id_rsa.pub /home/test/.ssh/
+COPY testing/infrastructure/id_rsa /home/test/.ssh/
+COPY testing/infrastructure/id_rsa.pub /home/test/.ssh/
 RUN chown -R test:test /home/test/.ssh
 RUN chmod 400 /home/test/.ssh/id_rsa
+
+# Copy the local duplicity folder into the container
+COPY . duplicity/
+RUN chown -R test:test duplicity
 USER test
 
-# Branch the duplicity repo for testing
-ARG REVNO=-1
-RUN bzr branch -r $REVNO lp:duplicity duplicity
-
 # Set final workdir to duplicity
 WORKDIR /home/test/duplicity

=== modified file 'testing/infrastructure/duplicity_test/Dockerfile-19.10'
--- testing/infrastructure/duplicity_test/Dockerfile-19.10	2019-07-22 19:40:09 +0000
+++ testing/infrastructure/duplicity_test/Dockerfile-19.10	2019-08-26 22:23:05 +0000
@@ -75,15 +75,15 @@
 
 # Copy a SSH key to the users folder that is used for some test cases
 USER root
-COPY ./id_rsa /home/test/.ssh/
-COPY ./id_rsa.pub /home/test/.ssh/
+COPY testing/infrastructure/id_rsa /home/test/.ssh/
+COPY testing/infrastructure/id_rsa.pub /home/test/.ssh/
 RUN chown -R test:test /home/test/.ssh
 RUN chmod 400 /home/test/.ssh/id_rsa
+
+# Copy the local duplicity folder into the container
+COPY . duplicity/
+RUN chown -R test:test duplicity
 USER test
 
-# Branch the duplicity repo for testing
-ARG REVNO=-1
-RUN bzr branch -r $REVNO lp:duplicity duplicity
-
 # Set final workdir to duplicity
 WORKDIR /home/test/duplicity