Merge lp:~allenap/maas/rename-scripts-and-prepare-for-twistd-in-region into lp:~maas-committers/maas/trunk

[Gavin Panella]

Rename several scripts in the development build, and prepare for twistd in the region.

This is accompanied by the packaging change in lp:~allenap/maas/packaging.rename-scripts-and-prepare-for-twistd-in-region.

[Graham Binns (gmb)]

review: Approve

[Julian Edwards (julian-edwards)]

The commit message doesn't tell me at all what this change is for or why it's needed, can you explain please, especially since this will directly affect everyone's dev environment. Sorry Gavin, but your commit messages are nearly always terrible :(

[Gavin Panella (allenap)]

I will improve the commit message tomorrow. I can't get Apache to shut-down services in the region properly so I'm looking at turning it inside-out and running Django as a WSGI service within Twisted.

[Julian Edwards (julian-edwards)]

On Tuesday 04 November 2014 00:27:41 you wrote:
> I will improve the commit message tomorrow. I can't get Apache to shut-down
> services in the region properly so I'm looking at turning it inside-out and
> running Django as a WSGI service within Twisted.

Ooof, GLWT. :)

> Because different sub-projects of MAAS use, in theory, different
> libraries, and buildout is meant to create isolated environments for
> each. In practice this is of limited utility right now, but moving away
> from buildout improves this, and makes the dev environment and tests
> more reliable indicators of behaviour when packaged (this is something
> that I did with my nearly 1 year old repackaging branch).

How about an outer wrapper that examines its args and runs the right sub-
script?

>
> > Also, my fingers *only* just got used to typing test.maas, why the
> > rename?
>
> I'm thinking about your RSI.
>
> You know it's been >2 years we've been typing test.maas?

Yes, it takes me that long. I still get it wrong today! :)

> test.maas gives the impression that it's everything, but in fact it's
> just the region. I started fixing other things up and this stuck out
> like a sticky out thing.

Right, it does stick out.

But wouldn't it be lovely to
bin/maas <whatever>

:)

>
> > Also, paring down tests in test.pserv is completely different, it
> > uses: test.pserv provisioningserver.module.file
>
> This is, I think, because these sub-projects share a common root folder
> (src), so the test commands need to be concocted differently for the
> region to everything else, but my memory is vague on this.
>
> There's also a Django-assimilation thing going on with nose which I
> suspect of some involvement, but again I can't be sure.

Yeah, you have my sympathy dealing with this.

[Gavin Panella (allenap)]

Graham's been removed from the official team, so I'm rubber-stamping this to satisfy the lander.

review: Approve

[MAAS Lander (maas-lander)]

Attempt to merge into lp:maas failed due to conflicts:

text conflict in Makefile

=== modified file 'HACKING.txt'
--- HACKING.txt	2014-09-24 14:20:51 +0000
+++ HACKING.txt	2015-01-09 13:11:18 +0000
@@ -97,8 +97,8 @@
 
 To run tests at a lower level of granularity::
 
-    $ ./bin/test.maas test src/maasserver/tests/test_api.py
-    $ ./bin/test.maas test src/maasserver/tests/test_api.py:AnonymousEnlistmentAPITest
+    $ ./bin/test.region src/maasserver/tests/test_api.py
+    $ ./bin/test.region src/maasserver/tests/test_api.py:AnonymousEnlistmentAPITest
 
 The test runner is `nose`_, so you can pass in options like
 ``--with-coverage`` and ``--nocapture`` (short option: ``-s``). The

=== modified file 'Makefile'
--- Makefile	2014-12-17 05:29:57 +0000
+++ Makefile	2015-01-09 13:11:18 +0000
@@ -22,8 +22,11 @@
 # JavaScript enum module (not modules).
 js_enums := src/maasserver/static/js/enums.js
 
-# MAAS SASS Stylesheets
-scss_inputs := $(wildcard src/maasserver/static/scss/*/*.scss)
+# MAAS SASS stylesheets. The first input file (maas-styles.css) imports
+# the others, so is treated specially in the target definitions.
+scss_inputs := \
+    src/maasserver/static/scss/maas-styles.scss \
+    $(wildcard src/maasserver/static/scss/*/*.scss)
 scss_output := src/maasserver/static/css/maas-styles.css
 
 # Prefix commands with this when they need access to the database.
@@ -41,15 +44,20 @@
 build: \
     bin/buildout \
     bin/database \
-    bin/maas-region-admin bin/test.maas \
-    bin/maas bin/test.maascli \
-    bin/test.maastesting \
-    bin/twistd.pserv bin/test.pserv \
+    bin/maas \
+    bin/maas-probe-dhcp \
+    bin/maas-provision \
+    bin/maas-region-admin \
+    bin/twistd.cluster \
+    bin/twistd.region \
+    bin/test.cli \
+    bin/test.cluster \
     bin/test.config \
-    bin/maas-probe-dhcp \
+    bin/test.region \
+    bin/test.testing \
     bin/py bin/ipy \
     $(js_enums) \
-    styles
+    $(scss_output)
 
 all: build doc
 
@@ -75,45 +83,41 @@
 	$(buildout) install database
 	@touch --no-create $@
 
-bin/maas-region-admin: \
-    bin/buildout buildout.cfg versions.cfg setup.py $(js_enums) styles
-	$(buildout) install maas
+bin/maas-region-admin bin/twistd.region: \
+    bin/buildout buildout.cfg versions.cfg setup.py $(js_enums) $(scss_output)
+	$(buildout) install region
 	@touch --no-create $@
 
-bin/test.maas: \
-	bin/buildout buildout.cfg versions.cfg setup.py $(js_enums) styles
-	$(buildout) install maas-test
+bin/test.region: \
+    bin/buildout buildout.cfg versions.cfg setup.py $(js_enums) $(scss_output)
+	$(buildout) install region-test
 	@touch --no-create $@
 
 bin/maas: bin/buildout buildout.cfg versions.cfg setup.py
-	$(buildout) install maascli
-	@touch --no-create $@
-
-bin/test.maascli: bin/buildout buildout.cfg versions.cfg setup.py
-	$(buildout) install maascli-test
-	@touch --no-create $@
-
-bin/test.maastesting: bin/buildout buildout.cfg versions.cfg setup.py
-	$(buildout) install maastesting-test
-	@touch --no-create $@
-
-bin/maas-provision bin/twistd.pserv: \
+	$(buildout) install cli
+	@touch --no-create $@
+
+bin/test.cli: bin/buildout buildout.cfg versions.cfg setup.py
+	$(buildout) install cli-test
+	@touch --no-create $@
+
+bin/test.testing: bin/buildout buildout.cfg versions.cfg setup.py
+	$(buildout) install testing-test
+	@touch --no-create $@
+
+bin/maas-probe-dhcp bin/maas-provision bin/twistd.cluster: \
     bin/buildout buildout.cfg versions.cfg setup.py
-	$(buildout) install pserv
+	$(buildout) install cluster
 	@touch --no-create $@
 
-bin/test.pserv: bin/buildout buildout.cfg versions.cfg setup.py
-	$(buildout) install pserv-test
+bin/test.cluster: bin/buildout buildout.cfg versions.cfg setup.py
+	$(buildout) install cluster-test
 	@touch --no-create $@
 
 bin/test.config: bin/buildout buildout.cfg versions.cfg setup.py
 	$(buildout) install config-test
 	@touch --no-create $@
 
-bin/maas-probe-dhcp: bin/buildout buildout.cfg versions.cfg setup.py
-	$(buildout) install maas-probe-dhcp
-	@touch --no-create $@
-
 bin/flake8: bin/buildout buildout.cfg versions.cfg setup.py
 	$(buildout) install flake8
 	@touch --no-create $@
@@ -189,10 +193,10 @@
 $(js_enums): bin/py src/maasserver/utils/jsenums.py $(py_enums)
 	 bin/py -m src/maasserver/utils/jsenums $(py_enums) > $@
 
-styles: $(scss_inputs)
-	pyscss src/maasserver/static/scss/maas-styles.scss \
-		-o $(scss_output) \
-		--load-path=src/maasserver/static/scss
+styles: $(scss_output)
+
+$(scss_output): $(scss_inputs)
+	pyscss $< -o $@ --load-path=src/maasserver/static/scss
 
 clean:
 	$(MAKE) -C acceptance $@
@@ -349,7 +353,7 @@
 
 services/database/@deps: bin/database
 
-services/pserv/@deps: bin/twistd.pserv
+services/pserv/@deps: bin/twistd.cluster
 
 services/reloader/@deps:
 

=== modified file 'buildout.cfg'
--- buildout.cfg	2014-10-01 15:22:21 +0000
+++ buildout.cfg	2015-01-09 13:11:18 +0000
@@ -1,17 +1,16 @@
 [buildout]
 parts =
-  flake8
-  maas
-  maas-test
-  maascli
-  maascli-test
-  maas-probe-dhcp
-  maastesting-test
-  pserv
-  pserv-test
+  cli
+  cli-test
+  cluster
+  cluster-test
   config-test
+  flake8
+  region
+  region-test
   repl
   sphinx
+  testing-test
 extensions = buildout-versions
 buildout_versions_file = versions.cfg
 versions = versions
@@ -59,7 +58,7 @@
 entry-points = database=postgresfixture.main:main
 scripts = database
 
-[maas]
+[region]
 recipe = zc.recipe.egg
 dev-eggs =
   django-debug-toolbar
@@ -67,30 +66,32 @@
   ${common:test-eggs}
   django-nose
 eggs =
-  ${maas:dev-eggs}
-  ${maas:test-eggs}
+  ${region:dev-eggs}
+  ${region:test-eggs}
   djorm-ext-pgarray
   docutils
   crochet
   iscpy
 entry-points =
   maas-region-admin=django.core.management:execute_from_command_line
+  twistd.region=twisted.scripts.twistd:run
 initialization =
   ${common:initialization}
   environ.setdefault("DJANGO_SETTINGS_MODULE", "maas.development")
 scripts =
   maas-region-admin
+  twistd.region
 extra-paths =
   ${common:extra-paths}
 
-[maas-test]
+[region-test]
 recipe = zc.recipe.egg
 eggs =
-  ${maas:eggs}
+  ${region:eggs}
 entry-points =
-  test.maas=django.core.management:execute_from_command_line
+  test.region=django.core.management:execute_from_command_line
 initialization =
-  ${maas:initialization}
+  ${region:initialization}
   sys.argv[1:1] = [
       "test", "--noinput", "--exclude=provisioningserver",
       "--exclude=maastesting", "--exclude=maascli",
@@ -100,24 +101,11 @@
       # requests at DEBUG level: we don't want this in the
       # tests as it's too verbose.
   ]
-scripts = test.maas
-extra-paths =
-  ${maas:extra-paths}
-
-[maas-probe-dhcp]
-recipe = zc.recipe.egg
-eggs =
-  ${maas:eggs}
-entry-points =
-  maas-probe-dhcp=provisioningserver.dhcp.probe:main
-scripts =
-  maas-probe-dhcp
-extra-paths =
-  ${maas:extra-paths}
-initialization =
-  ${common:initialization}
-
-[maascli]
+scripts = test.region
+extra-paths =
+  ${region:extra-paths}
+
+[cli]
 recipe = zc.recipe.egg
 eggs =
 entry-points =
@@ -127,62 +115,64 @@
 scripts =
   maas
 
-[maascli-test]
+[cli-test]
 recipe = zc.recipe.egg
 eggs =
-  ${maascli:eggs}
+  ${region:eggs}
   ${common:test-eggs}
 entry-points =
-  test.maascli=nose.core:TestProgram
+  test.cli=nose.core:TestProgram
 initialization =
   sys.argv[1:1] = ["--where=src/maascli"]
-extra-paths = ${maascli:extra-paths}
+extra-paths = ${cli:extra-paths}
 scripts =
-  test.maascli
+  test.cli
 
-[maastesting-test]
+[testing-test]
 recipe = zc.recipe.egg
 eggs =
   ${common:test-eggs}
 entry-points =
-  test.maastesting=nose.core:TestProgram
+  test.testing=nose.core:TestProgram
 initialization =
   sys.argv[1:1] = ["--where=src/maastesting"]
 extra-paths = ${common:extra-paths}
 scripts =
-  test.maastesting
-scripts = test.maastesting
+  test.testing
+scripts = test.testing
 extra-paths =
-  ${maas:extra-paths}
+  ${region:extra-paths}
 
-[pserv]
+[cluster]
 recipe = zc.recipe.egg
 eggs =
   crochet
 entry-points =
+  maas-probe-dhcp=provisioningserver.dhcp.probe:main
   maas-provision=provisioningserver.__main__:main
-  twistd.pserv=twisted.scripts.twistd:run
+  twistd.cluster=twisted.scripts.twistd:run
 extra-paths =
   ${common:extra-paths}
 scripts =
+  maas-probe-dhcp
   maas-provision
-  twistd.pserv
+  twistd.cluster
 initialization =
   ${common:initialization}
 
-[pserv-test]
+[cluster-test]
 recipe = zc.recipe.egg
 eggs =
-  ${pserv:eggs}
+  ${cluster:eggs}
   ${common:test-eggs}
 entry-points =
-  test.pserv=nose.core:TestProgram
+  test.cluster=nose.core:TestProgram
 initialization =
-  ${pserv:initialization}
+  ${cluster:initialization}
   sys.argv[1:1] = ["--where=src/provisioningserver"]
-extra-paths = ${pserv:extra-paths}
+extra-paths = ${cluster:extra-paths}
 scripts =
-  test.pserv
+  test.cluster
 
 [config-test]
 recipe = zc.recipe.egg
@@ -191,7 +181,7 @@
 entry-points =
   test.config=nose.core:TestProgram
 initialization =
-  ${pserv:initialization}
+  ${cluster:initialization}
   sys.argv[1:1] = ["--where=etc/maas/templates/commissioning-user-data"]
 extra-paths = ${common:extra-paths}
 scripts =
@@ -210,15 +200,15 @@
 build = ${buildout:directory}/docs/_build
 extra-paths = ${common:extra-paths}
 eggs =
-  ${maas:eggs}
-  ${pserv:eggs}
+  ${region:eggs}
+  ${cluster:eggs}
 
 # Convenient REPLs with all eggs available.
 [repl]
 recipe = z3c.recipe.scripts
 eggs =
-  ${maas:eggs}
-  ${pserv:eggs}
+  ${region:eggs}
+  ${cluster:eggs}
   ${common:test-eggs}
 extra-paths = ${common:extra-paths}
 interpreter = py

=== modified file 'services/pserv/run'
--- services/pserv/run	2014-02-12 22:11:46 +0000
+++ services/pserv/run	2015-01-09 13:11:18 +0000
@@ -15,7 +15,7 @@
 [ -z "${logdir:-}" ] || exec &>> "${logdir}/current"
 
 # Exec the Provisioning Server.
-script="$(readlink -f bin/twistd.pserv)"
+script="$(readlink -f bin/twistd.cluster)"
 config="$(readlink -f etc/maas/pserv.yaml)"
 
 # Obtain the development setting for CLUSTER_UUID.

=== modified file 'src/apiclient/testing/django.py'
--- src/apiclient/testing/django.py	2013-12-28 00:05:31 +0000
+++ src/apiclient/testing/django.py	2015-01-09 13:11:18 +0000
@@ -61,7 +61,7 @@
     #   type handlers with utils.translate_mime. So we must import it, even
     #   though we don't use it.  However, piston loads Django's QuerySet code
     #   which fails if you don't have a settings.py available. Which we don't
-    #   during 'test.pserv'. So we import this late.
+    #   during 'test.cluster'. So we import this late.
     from piston import emitters
     emitters  # Imported for side-effects.
     from piston.utils import translate_mime

=== renamed file 'twisted/plugins/maasps.py' => 'twisted/plugins/maasclusterd.py'