Merge lp:~ce-infrastructure/capomastro/capomastro-docs-1391397 into lp:capomastro

[Caio Begotti]

A few changes to reflect how things are done now compared to the old README and instructions for local development.

[Caio Begotti (caio1982)]

James, you'll find the last line of the diff the most interesting since the rest of the setup instructions didn't change much.

[lp:~ce-infrastructure/capomastro/capomastro-docs-1391397]

153. By Caio Begotti on 2014-11-27

typos

154. By Caio Begotti on 2014-11-27

no reason not to have it here if the dockerfile was listed before

[Daniel Manrique (roadmr)]

Some minor nitpicks and one question.

review: Needs Fixing

[lp:~ce-infrastructure/capomastro/capomastro-docs-1391397]

155. By Caio Begotti on 2014-11-27

review fixes, thanks roadmr!

156. By Caio Begotti on 2014-11-27

otherwise people will only find ghosts hanging around on IRC

[James Jesudason (jamesj)]

The symlink for the scripts file and retaining the standard manage.py file should make the project easier to pick up for someone new.

Moving the docs from the README.md in the root directory to the docs/source/setup.rst makes it harder to find. The symlink seems to be a bit of a compromise. I'd prefer to see the setup instructions in the doc/source/index.rst or at least a link in there to the setup.rst file. The README symlink can then go to the index.rst file.

The setup docs are now more wordy and not as succinct or easy to follow as the previous version. The numbered list of steps in the original made it easier to follow. Also, the database role and database creation should be placed before the setup of the local_settings.py and the syncdb.

review: Needs Fixing

[Caio Begotti (caio1982)]

James, the symlink is for the packaging, I've actually restored the real manage.py file as you wanted. Perhaps you interpreted the symlink reversed?

Also, the content of setup.rst was in fact inside index.rst before but it was getting way too long and confusing so that's why I split them and symlinked the README file to setup.rst directly. I'm going to link setup.rst inside index.rst as you suggests too.

Perhaps the setup docs are more wordy now because I'm not a native speaker of English I'd say, so I'm totally open to grammar and spelling and phrasing suggestions!

I'll fix up the order of the database setup as you suggests, thanks! I don't get though how the numbered list is worse now. It was just a plain text file before and ReST formatted files can't keep the same formatting I'm afraid.

[lp:~ce-infrastructure/capomastro/capomastro-docs-1391397]

157. By Caio Begotti on 2014-11-28

fix the order in which the role/database is created, thanks jamesj

158. By Caio Begotti on 2014-11-28

link the setup instructions directly in the main documentation file, for clarity

[Caio Begotti (caio1982)]

Suggestions for checking how the documentation now looks in case the diff is a bit confusing (I think it is):

'less README' -> now points to the real up to date instructions, starting with a note where to find more info and telling developers to follow the next sections as they appear

'./manage.py' -> at the root of the project directory, like expected, works again

'less docs/source/index.rst' -> in case make html wasn't run in the docs directory, points to the setup instructions so nothing is missed at a first glance

[James Jesudason (jamesj)]

Thanks

review: Approve

[Daniel Manrique (roadmr)]

+1 from me too, thanks!a

review: Approve

=== added symlink 'README'
=== target is u'docs/source/setup.rst'
=== removed file 'README.md'
--- README.md	2014-10-27 12:20:16 +0000
+++ README.md	1970-01-01 00:00:00 +0000
@@ -1,69 +0,0 @@
-capomastro [![Build Status](https://travis-ci.org/capomastro/capomastro.svg)](https://travis-ci.org/capomastro/capomastro)
-==========
-Master builder
-
-Capomastro manages build dependencies built using Jenkins.
-
-Multiple "Projects" can share common dependencies, and the artifact details are
-pulled into the database where they can easily be download, or archived.
-
-To get started...
-
-1. `mkvirtualenv <name>`
-
-2. `pip install -r requirements.txt` (or dev-requirements.txt if you're going
-   to run the tests)
-
-3. Copy the `capomastro/local_settings.py.example` to `capomastro/local_settings.py`
-   and fixup the database entry
-
-4. You'll need to setup the database `./manage.py syncdb --migrate` - you'll be
-   prompted to create a superuser.
-
-Steps 1 and 2 can be replaced with `tox -e devenv`. Then either activate the
-virtualenv manually with `source devenv/bin/activate` or explicitly call
-Python from the virtualenv with `devenv/bin/python`.
-
-You can do the next steps through the /admin/ interface, so go to
-http://localhost:8000/admin/ login and continue.
-
-I'd recommend installing gunicorn, but you can use the default ./manage.py
-runserver.
-
-You'll need a celery worker running as well as rabbitmq.
-
-```
-$ gunicorn -b 0.0.0.0:8000 capomastro.wsgi:application
-$ celery -A capomastro worker -l info
-```
-
-5. You'll need an initial jenkins.JenkinsServer object, with the correct credentials,
-   and the REMOTE_ADDR setup correctly, so that it can receive callbacks.
-
-7. Create a DependencyType, and Dependency for that DependencyType, the
-   Dependency should be hooked up to your Job model.
-
-
-You should have...
-
-
-     Dependency -> DependencyType
-         |
-        Job
-         |
-    JenkinsServer
-
-You can create several dependencies, with different jobs.
-
-Your Jenkins Server requires the "notifications" plugin installed.
-
-And the jobs that are created must have a parameter "BUILD_ID" and they must
-have a notification setup, type http/json and with a callback address of
-http://hostname/jenkins/notifications/.
-
-8. Now you can create a Project, associated with your dependencies, at
-   localhost:8000/projects/create/ "auto track" means that the project will use
-   the latest version of any dependencies automatically.
-
-9. Now, you can build your project, this will create a project build, and
-   trigger the tasks to build your project.

=== modified file 'capomastro/templates/base.html'
--- capomastro/templates/base.html	2014-11-27 12:20:47 +0000
+++ capomastro/templates/base.html	2014-11-28 16:53:08 +0000
@@ -84,7 +84,7 @@
     {% block footer %}
     <div class="container">
       <hr>
-      <p><a href="https://github.com/capomastro/capomastro">Capomastro</a> is <a href="http://opensource.org/definition">Open Source</a> software, released under an <a href="https://github.com/capomastro/capomastro/blob/master/LICENSE">MIT License</a>.</p>
+      <p><a href="https://launchpad.net/capomastro">Capomastro</a> is <a href="http://opensource.org/definition">Open Source</a> software, released under an <a href="http://bazaar.launchpad.net/~ce-infrastructure/capomastro/trunk/view/head:/LICENSE">MIT License</a>.</p>
     </div>
     {% endblock footer %}
   </footer>

=== modified file 'debian/changelog'
--- debian/changelog	2014-11-27 16:02:56 +0000
+++ debian/changelog	2014-11-28 16:53:08 +0000
@@ -1,3 +1,9 @@
+capomastro (20141129) trusty; urgency=high
+
+  * Fixes #1391397 (fixing docs for local testing/development)
+
+ -- Caio Begotti <[e-mail]>  Thu, 27 Nov 2014 16:58:28 -0200
+
 capomastro (20141128) trusty; urgency=high
 
   * Fixes #1391718 (includes a log rotation setup)

=== modified file 'debian/docs'
--- debian/docs	2014-10-27 12:45:25 +0000
+++ debian/docs	2014-11-28 16:53:08 +0000
@@ -1,3 +1,4 @@
 docs/build/html
 Dockerfile
-README.md
+Vagrantfile
+README

=== modified file 'docs/source/index.rst'
--- docs/source/index.rst	2014-10-27 12:40:23 +0000
+++ docs/source/index.rst	2014-11-28 16:53:08 +0000
@@ -1,5 +1,6 @@
 .. Capomastro documentation master file, created by
    sphinx-quickstart on Mon Oct 20 16:11:51 2014.
+.. _setup: setup.html
 
 Capomastro's documentation
 ==========================
@@ -15,54 +16,23 @@
 Introduction
 ------------
 
-Capomastro is Jenkins best friend as we call it because it's basically a UI to manage Jenkins builds and archived artifacts of such builds. It uses Jenkins only for that sort of automation, and it does it through Bygmester.
+Capomastro is Jenkins' best friend as we call it because it's basically a UI to manage Jenkins builds and archived artifacts of such builds. It uses Jenkins only for that sort of automation, and it does it through Bygmester (which is hosted on http://launchpad.net/bygmester). Multiple "projects" can share common dependencies, and the artifact details are pulled into the database where they can easily be downloaded, or archived elsewhere.
 
 The current use of Capomastro is to build images for the Phablet team but soon it will possibly grow and merge with the other OEM Image Build System we have at Canonical.
 
 Orinally, Capomastro was developed on Github as an open source side project of Canonical employees, but this is the forked version of it hosted on Launchpad to better reflect the roadmap we have for it.
 
-Manual setup
-------------
-
-In case you need to manually set up a new Capomastro service, please refer to how its packages do things and also its Juju's charm configuration. They can help you a great deal.
-
-You'll need the following services in order to bring it up anyway:
-
-* PostgreSQL, default database
-* Apache with mod WSGI, to expose the UI
-* RabbitMQ, for message handling
-* Celery, tasks management
-
-All the configurations of Capomastro (messaging, DB, the service etc) are stored via symlinks inside of ``/etc/capomastro/``.
-
-Beside a dedicated user creation to run Capomastro (who will actually start the Celery workers through its initscript), you'll need to set up the DB and a role for it::
-
-    CREATE ROLE capomastro
-    PASSWORD 'capomastro'
-    LOGIN CREATEDB NOSUPERUSER INHERIT NOCREATEROLE NOREPLICATION;
-
-    CREATE DATABASE capomastro
-    WITH OWNER = capomastro
-    TABLESPACE = pg_default
-    LC_COLLATE = 'en_US.UTF-8'
-    LC_CTYPE = 'en_US.UTF-8'
-    ENCODING = 'UTF8'
-    CONNECTION LIMIT = -1;
-
-As for the web server configuration, it all depends on how you want to run it for development and testing. The file ``capomastro.conf`` at the root of its source tree will tell you how to use it with Apache.
-
-If you're not using the Capomastro packages via its PPA, remember you'll need to manually create a Django superuser and run its syncdb management command. Refer to Django's documentation about both operations or simply run this::
-
-    $ ./manage.py syncdb --migrate
-
-All that remains now is the first the dependencies setup, then the projects setup, in that particular order. Creating of projects before its dependencies are set is not currently supported.
+Setting up
+----------
+
+Please refer to the setup_ section for instructions.
 
 Contact
 -------
 
 For bug reports, follow ups, code management etc please refer to our Launchpad project at https://launchpad.net/capomastro.
 
-You can also send us an e-mail and we'll get back to you as soon as possible, just shoot it one for the PES Certification & Tools team at [e-mail]. You'll also find many of us in our IRC channel ``#pes-infrastructure``.
+You can also send us an e-mail and we'll get back to you as soon as possible, just shoot it one for the PES Certification & Tools team at [e-mail]. You'll also find many of us in our IRC channel ``#cert-infra``.
 
 
 Indices and tables

=== added file 'docs/source/setup.rst'
--- docs/source/setup.rst	1970-01-01 00:00:00 +0000
+++ docs/source/setup.rst	2014-11-28 16:53:08 +0000
@@ -0,0 +1,94 @@
+=====
+Setup
+=====
+
+This file was created based on the old historic README file from Capomastro, if anything seems to be missing here please refer to the whole documentation files in the ``docs/`` directory.
+
+If you're an old school Django guy, you can get started with local testing and development by following the next sections. You'll need the following services in order to bring the application up anyway, no matter if using the packaged code or not:
+
+* PostgreSQL, default database
+* Apache with mod WSGI, to expose the UI
+* RabbitMQ, for message handling
+* Celery, tasks management
+
+Environment
+-----------
+
+Create your environment either with ``mkvirtualenv <name>``, if you have virtualenvwrapper installed, or simply ``tox -e devenv`` since it does some extra steps for you. It's recommended to install ``tox`` and go with the latter option.
+
+If you decide for using ``mkvirtualenv`` you'll also need to manually install the Python dependencies with ``pip install -r requirements.txt`` or even ``pip install -r dev-requirements.txt`` if you're going to run the code tests.
+
+Then either activate the virtual environment manually with ``source devenv/bin/activate`` or explicitly call Python from the virtual environment with ``devenv/bin/python``.
+
+Database
+--------
+
+Please set up the Postgres server with this script before continuing:
+
+  ::
+
+      CREATE ROLE capomastro
+      PASSWORD 'capomastro'
+      LOGIN CREATEDB NOSUPERUSER INHERIT NOCREATEROLE NOREPLICATION;
+
+      CREATE DATABASE capomastro
+      WITH OWNER = capomastro
+      TABLESPACE = pg_default
+      LC_COLLATE = 'en_US.UTF-8'
+      LC_CTYPE = 'en_US.UTF-8'
+      ENCODING = 'UTF8'
+      CONNECTION LIMIT = -1;
+
+If needed, fixup the database entry in ``capomastro/db_settings.py``, but most likely the default values will work okay unless you're testing database changes.
+
+You'll need to prepare the database with ``./manage.py syncdb`` (and you'll be prompted to create a superuser at this time).
+
+Web server
+----------
+ 
+Now you can start the local server by executing ``./manage.py runserver`` and do the next steps through the ``/admin/``
+interface of Capomastro, so go to http://localhost:8000/admin/ then login with the username and password created before and continue.
+
+If you prefer to test things with Gunicorn you'll need a Celery worker
+running as well as RabbitMQ.
+
+Now just run ``gunicorn -b 0.0.0.0:8000 capomastro.wsgi:application`` and start the Celery worker that will talk to the Rabbit queue with ``celery -A capomastro worker -l debug``.
+
+Django
+------
+
+All that remains now is the first the dependencies setup, then the projects setup, in that particular order. Creating of projects before its dependencies are set is not currently supported.
+
+In the ``/admin/`` interface of Capomastro you'll need an initial ``jenkins.JenkinsServer`` object, with the correct credentials and ``REMOTE_ADDR`` set up correctly, so that it can receive callbacks.
+
+Create a ``DependencyType`` and ``Dependency`` for that ``DependencyType``. The ``Dependency`` should be hooked up to your ``Job`` model.
+
+You should have something like this:
+::
+
+     Dependency - DependencyType
+         |
+        Job
+         |
+    JenkinsServer
+
+You can create several dependencies, with different jobs. Your Jenkins server requires the ``notifications`` plugin installed too, so make sure it's available otherwise you won't get status back from the builds.
+
+The jobs that are created must have a parameter ``BUILD_ID`` and they must have a notification setup, type ``http/json`` and with a callback address of ``http://hostname/jenkins/notifications/``.
+
+Now you can create a ``Project``, associated with your dependencies, at ``http://localhost:8000/projects/create/``. The option "auto track" means that the project will use the latest version of any dependencies automatically.
+
+Now you can build your project. This will create a project build, and trigger the tasks to build your project on the Jenkins server.
+   
+Packaged setup
+--------------
+
+All the configurations of Capomastro (messaging, DB, the service etc) are stored via symlinks inside of ``/etc/capomastro/`` if you're using the Debian package of the application.
+
+Beside a dedicated user creation to run Capomastro (who will actually start the Celery workers through its initscript), you'll need to set up the DB and a role for it. See the previous database section for a SQL script that can be used.
+
+As for the web server configuration, it all depends on how you want to run it for development and testing. The file ``capomastro.conf`` at the root of its source tree will tell you how to use it with Apache, the preferred method.
+
+By using the Debian package of Capomastro, possibly through its Juju charm as well, things will be much easier to set up initially. Usually you don't have to worry about any of these steps above, plus you'll get a really isolated environment for development and testing.
+
+All code is actually validated through the application package, so although you can test and develop things really locally, only code from packages is considered deployable.

=== renamed file 'scripts/capomastro' => 'manage.py'
=== added symlink 'scripts/capomastro'
=== target is u'../manage.py'