Merge lp:~rharding/charmworld/novel-charmtools into lp:charmworld

[Richard Harding]

Use forked charm-tools to allow tweaking endpoint.

- The download-cache is updated with the forked build
- Tweaks to force ingest to look at localhost:port to run
- Added the port to the config and updated the production overrides so that
production talks to itself without egress filter issues.
- Update tests for the extra kwarg endpoint.

Note: You cannot ingest bundles without a running instance. This means
to ingest bundles in local dev you must also `make run`.

https://codereview.appspot.com/22540043/

R=bac

Use forked charm-tools to allow tweaking endpoint.

- The download-cache is updated with the forked build
- Tweaks to force ingest to look at localhost:port to run
- Added the port to the config and updated the production overrides so that
production talks to itself without egress filter issues.
- Update tests for the extra kwarg endpoint.

Note: You cannot ingest bundles without a running instance. This means
to ingest bundles in local dev you must also `make run`.

https://codereview.appspot.com/22540043/

[Richard Harding (rharding)]

Reviewers: mp+194257_code.launchpad.net,

Message:
Please take a look.

Description:
Use forked charm-tools to allow tweaking endpoint.

- The download-cache is updated with the forked build
- Tweaks to force ingest to look at localhost:port to run
- Added the port to the config and updated the production overrides so
that
production talks to itself without egress filter issues.
- Update tests for the extra kwarg endpoint.

Note: You cannot ingest bundles without a running instance. This means
to ingest bundles in local dev you must also `make run`.

https://code.launchpad.net/~rharding/charmworld/novel-charmtools/+merge/194257

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/22540043/

Affected files (+17, -5 lines):
   A [revision details]
   M charmworld/jobs/ingest.py
   M charmworld/jobs/tests/test_ingest.py
   M charmworld/testing/data/sample_charm/production_overrides.ini
   M default.ini
   M requirements.txt

Index: [revision details]
=== added file '[revision details]'
--- [revision details] 2012-01-01 00:00:00 +0000
+++ [revision details] 2012-01-01 00:00:00 +0000
@@ -0,0 +1,2 @@
+Old revision: tarmac-20131106211543-mdy7zo9iywzec4mm
+New revision: <email address hidden>

Index: default.ini
=== modified file 'default.ini'
--- default.ini 2013-09-04 10:26:28 +0000
+++ default.ini 2013-11-06 22:54:45 +0000
@@ -58,6 +58,10 @@
  # text)
  days_of_revisions = 30

+# The port to hit up for proof. This should match up the server:main port
+# below.
+proof.port = 2464
+
  [server:main]
  use = egg:Paste#http
  host = 0.0.0.0

Index: requirements.txt
=== modified file 'requirements.txt'
--- requirements.txt 2013-11-05 00:24:42 +0000
+++ requirements.txt 2013-11-06 21:30:01 +0000
@@ -3,7 +3,7 @@
  Beaker==1.6.4
  bzr==2.5.1
  Chameleon==2.5.1
-charm-tools==1.1.2
+charm-tools==1.1.2-charmworld
  Cheetah==2.4.4
  colander==0.9.9
  configobj==4.7.2

Index: charmworld/jobs/ingest.py
=== modified file 'charmworld/jobs/ingest.py'
--- charmworld/jobs/ingest.py 2013-11-05 19:56:37 +0000
+++ charmworld/jobs/ingest.py 2013-11-06 22:54:45 +0000
@@ -347,7 +347,12 @@

      @staticmethod
      def proof(basket_file):
- lint, err = proof.proof(basket_file, is_bundle=True)
+ endpoint = "http://{}:{}/api/3/bundle/proof".format(
+ 'localhost',
+ settings['proof.port'],
+ )
+ lint, err = proof.proof(
+ basket_file, is_bundle=True, endpoint=endpoint)
          return lint, err >= CHARMTOOL_ERROR_CODE

      def run(self, basket_data):

Index: charmworld/jobs/tests/test_ingest.py
=== modified file 'charmworld/jobs/tests/test_ingest.py'
--- charmworld/jobs/tests/test_ingest.py 2013-11-05 18:57:10 +0000
+++ charmworld/jobs/tests/test_ingest.py 2013-11-06 23:04:32 +0000
@@ -608,7 +608,7 @@
          job = UpdateBundleJob()
          job.setup(self.db)
          with patch('charmtools.proof.proof',
- lambda path, is_bundle: ([], 200)):
+ lambda path, is_bundle, endpoint: ([], 200)):
              lint, err = job.proof('dummy')
              self.assertEqual(True, err)

@@ -616,...

[Brad Crittenden (bac)]

LGTM, thanks

https://codereview.appspot.com/22540043/diff/1/charmworld/jobs/ingest.py
File charmworld/jobs/ingest.py (right):

https://codereview.appspot.com/22540043/diff/1/charmworld/jobs/ingest.py#newcode352
charmworld/jobs/ingest.py:352: settings['proof.port'],
Where does proof.port come from? I've looked but don't see it (which is
why I'm glad you did this part).

https://codereview.appspot.com/22540043/diff/1/charmworld/testing/data/sample_charm/production_overrides.ini
File charmworld/testing/data/sample_charm/production_overrides.ini
(right):

https://codereview.appspot.com/22540043/diff/1/charmworld/testing/data/sample_charm/production_overrides.ini#newcode8
charmworld/testing/data/sample_charm/production_overrides.ini:8:
proof.port = 6543
Oh, looky, there it is!

https://codereview.appspot.com/22540043/

[Juju Gui Bot (juju-gui-bot)]

review: Approve (continuous-integration)

=== modified file 'charmworld/jobs/ingest.py'
--- charmworld/jobs/ingest.py	2013-11-05 19:56:37 +0000
+++ charmworld/jobs/ingest.py	2013-11-06 23:15:48 +0000
@@ -347,7 +347,12 @@
 
     @staticmethod
     def proof(basket_file):
-        lint, err = proof.proof(basket_file, is_bundle=True)
+        endpoint = "http://{}:{}/api/3/bundle/proof".format(
+            'localhost',
+            settings['proof.port'],
+        )
+        lint, err = proof.proof(
+            basket_file, is_bundle=True, endpoint=endpoint)
         return lint, err >= CHARMTOOL_ERROR_CODE
 
     def run(self, basket_data):

=== modified file 'charmworld/jobs/tests/test_ingest.py'
--- charmworld/jobs/tests/test_ingest.py	2013-11-05 18:57:10 +0000
+++ charmworld/jobs/tests/test_ingest.py	2013-11-06 23:15:48 +0000
@@ -608,7 +608,7 @@
         job = UpdateBundleJob()
         job.setup(self.db)
         with patch('charmtools.proof.proof',
-                   lambda path, is_bundle: ([], 200)):
+                   lambda path, is_bundle, endpoint: ([], 200)):
             lint, err = job.proof('dummy')
             self.assertEqual(True, err)
 
@@ -616,7 +616,7 @@
         job = UpdateBundleJob()
         job.setup(self.db)
         with patch('charmtools.proof.proof',
-                   lambda path, is_bundle: ([], 100)):
+                   lambda path, is_bundle, endpoint: ([], 100)):
             lint, err = job.proof('dummy')
             self.assertEqual(False, err)
 
@@ -624,7 +624,7 @@
         job = UpdateBundleJob()
         job.setup(self.db)
         with patch('charmtools.proof.proof',
-                   lambda path, is_bundle: ([], 0)):
+                   lambda path, is_bundle, endpoint: ([], 0)):
             lint, err = job.proof('dummy')
             self.assertEqual(False, err)
 

=== modified file 'charmworld/testing/data/sample_charm/production_overrides.ini'
--- charmworld/testing/data/sample_charm/production_overrides.ini	2013-02-01 15:57:19 +0000
+++ charmworld/testing/data/sample_charm/production_overrides.ini	2013-11-06 23:15:48 +0000
@@ -5,6 +5,7 @@
 mongo.host =
 mongo.port =
 session.secret =
+proof.port = 6543
 
 [server:main]
 port = 6543

=== modified file 'default.ini'
--- default.ini	2013-09-04 10:26:28 +0000
+++ default.ini	2013-11-06 23:15:48 +0000
@@ -58,6 +58,10 @@
 # text)
 days_of_revisions = 30
 
+# The port to hit up for proof. This should match up the server:main port
+# below.
+proof.port = 2464
+
 [server:main]
 use = egg:Paste#http
 host = 0.0.0.0

=== modified file 'requirements.txt'
--- requirements.txt	2013-11-05 00:24:42 +0000
+++ requirements.txt	2013-11-06 23:15:48 +0000
@@ -3,7 +3,7 @@
 Beaker==1.6.4
 bzr==2.5.1
 Chameleon==2.5.1
-charm-tools==1.1.2
+charm-tools==1.1.2-charmworld
 Cheetah==2.4.4
 colander==0.9.9
 configobj==4.7.2