Merge lp:~vila/uci-engine/integration-speed-up into lp:uci-engine

[Vincent Ladeuil]

Teach AmuletDeployment() to skip the deployment when one is already available.

This changes lp:~doanac/uci-engine/integration-speed-up to minimize the scope of the workaround.

Andy, we've been talking past each other I think, let me restart the discussion:

1) We are bitten hard by https://bugs.launchpad.net/juju-core/+bug/1200267

IMO, It's the most important issue we face for the current "integration" tests: I thought you had some insight about what was going on hence my question. You don't. Damn. Me neither. Damn damn.

I don't want this issue to get out of our radar hence the comment in setUp() in this proposal. If we kick the can, let's make sure it's still in front of us.

2) We need a workaround in the mean time.

I'm fine with that but I want that workaround to be expressed in the right place: at the test level.

I want (like you), as a dev, to be in charge of the environment where my test run.

I *don't* want to have to tell the test runner about that.

You proposed to use an env var for that, there is no need for a class
attribute to duplicate that information.

bzr diffstat for this proposal is:

  1 file changed, 14 insertions(+), 10 deletions(-)

yours was:

    8 files changed, 26 insertions(+), 31 deletions(-)

I don't want to carry those additional changes.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:563
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/860/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/860/rebuild

review: Approve (continuous-integration)

Unmerged revisions

563. By Vincent Ladeuil on 2014-06-13

Simplify DEPLOYED handling. The only place we care about it is in AmuletDeployment.setUp().

562. By Andy Doan on 2014-06-11

integration tests: allow re-use of a deployed environment

This allows us to deploy once and then run our integration tests
against a deployed environment. This greatly speeds up local testing.

This is disabled by default, but specifying DEPLOYED=1 in your
environment allows you take advantage of this. I'm running things
at home with:

 DEPLOYED=1 JUJU_REPOSITORY=`pwd`/tmp/charms PATH=$PATH:`pwd`/branches/juju-deployer JUJU_DEPLOYER_DIR=`pwd`/tmp ./tests/test_webui.py

I'm not a huge fan of "DEPLOYED=1" but unittest.main has its own argparse
logic, so we screw that up if we try to accept CLI args. Even if this
idea is terrible, its local to one place so we can easily fix it in
the future w/o touching our other code.

This makes testing the webui go from 76s to 4s when using juju-lxc

=== modified file 'tests/deployers.py'
--- tests/deployers.py	2014-06-13 13:01:42 +0000
+++ tests/deployers.py	2014-06-13 13:01:42 +0000
@@ -19,7 +19,6 @@
 import sys
 import time
 import logging
-import unittest
 
 import amulet
 
@@ -32,7 +31,7 @@
 
 from ci_utils import testing
 from ci_utils.testing import features
-from ci_utils.testing.deployer import load_configs
+from ci_utils.testing import deployer
 
 
 class AmuletDeployment(object):
@@ -66,15 +65,16 @@
     # better understand what is going on.  -- vila 2014-06-05
     timeout = 1200
 
-    deployed = False
-
     def setUp(self):
         super(DeployerTest, self).setUp()
         self.deployer = AmuletDeployment()
-        if not DeployerTest.deployed:
+        # FIXME: Until http://pad.lv/1200267 is fixed, we want a way to reuse
+        # an existing deployment that has been setup and validated before the
+        # tests are run. This is controlled by the 'DEPLOYED' env var being set
+        # -- vila 2014-06-13.
+        if 'DEPLOYED' not in os.environ:
             deployer_dir = os.environ['JUJU_DEPLOYER_DIR']
-            script = testing.deployer.load_configs(
-                deployer_dir, self.deployer_cfgs)
+            script = deployer.load_configs(deployer_dir, self.deployer_cfgs)
             self.fixup_includes(script, deployer_dir)
             self.add_extra_relations(script)
             self.deployer.load(script)
@@ -135,9 +135,3 @@
         ]
         output = subprocess.check_output(args)
         self.assertIn('start/running', output)
-
-
-def main():
-    if 'DEPLOYED' in os.environ:
-        DeployerTest.deployed = True
-    unittest.main()

=== modified file 'tests/test_image_builder.py'
--- tests/test_image_builder.py	2014-06-13 13:01:42 +0000
+++ tests/test_image_builder.py	2014-06-06 07:18:50 +0000
@@ -14,6 +14,10 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+
+import unittest
+
+
 import deployers
 
 
@@ -29,4 +33,4 @@
 
 
 if __name__ == "__main__":
-    deployers.main()
+    unittest.main()

=== modified file 'tests/test_integration.py'
--- tests/test_integration.py	2014-06-13 13:01:42 +0000
+++ tests/test_integration.py	2014-06-11 18:04:13 +0000
@@ -21,10 +21,12 @@
 import subprocess
 import sys
 import tempfile
+import unittest
 
 from ucitests import fixtures as uci_fixtures
 import deployers
 
+
 HERE = os.path.abspath(os.path.dirname(__file__))
 sys.path.append(os.path.join(HERE, '..', 'ci-utils'))
 
@@ -152,4 +154,4 @@
 
 
 if __name__ == "__main__":
-    deployers.main()
+    unittest.main()

=== modified file 'tests/test_ppa_assigner.py'
--- tests/test_ppa_assigner.py	2014-06-13 13:01:42 +0000
+++ tests/test_ppa_assigner.py	2014-06-06 07:18:50 +0000
@@ -17,6 +17,8 @@
 import httplib2
 import json
 import os
+import unittest
+
 
 import deployers
 
@@ -148,4 +150,4 @@
 
 
 if __name__ == "__main__":
-    deployers.main()
+    unittest.main()

=== modified file 'tests/test_publisher.py'
--- tests/test_publisher.py	2014-06-13 13:01:42 +0000
+++ tests/test_publisher.py	2014-06-06 07:18:50 +0000
@@ -13,6 +13,8 @@
 
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
+import unittest
+
 import deployers
 
 
@@ -29,4 +31,4 @@
 
 
 if __name__ == "__main__":
-    deployers.main()
+    unittest.main()

=== modified file 'tests/test_test_runner.py'
--- tests/test_test_runner.py	2014-06-13 13:01:42 +0000
+++ tests/test_test_runner.py	2014-06-06 07:18:50 +0000
@@ -13,6 +13,9 @@
 
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import unittest
+
 import deployers
 
 
@@ -28,4 +31,4 @@
 
 
 if __name__ == "__main__":
-    deployers.main()
+    unittest.main()

=== modified file 'tests/test_ticket_system.py'
--- tests/test_ticket_system.py	2014-06-13 13:01:42 +0000
+++ tests/test_ticket_system.py	2014-06-06 19:54:07 +0000
@@ -15,6 +15,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 import httplib2
 import json
+import unittest
 
 import deployers
 
@@ -121,4 +122,4 @@
 
 
 if __name__ == "__main__":
-    deployers.main()
+    unittest.main()

=== modified file 'tests/test_webui.py'
--- tests/test_webui.py	2014-06-13 13:01:42 +0000
+++ tests/test_webui.py	2014-06-10 16:57:22 +0000
@@ -16,6 +16,7 @@
 import json
 
 import httplib2
+import unittest
 
 import deployers
 
@@ -58,4 +59,4 @@
 
 
 if __name__ == "__main__":
-    deployers.main()
+    unittest.main()