Merge lp:~danilo/linaro-license-protection/staging-configs into lp:~linaro-automation/linaro-license-protection/trunk

[Данило Шеган]

Remove configs related to our production/staging deployment
(they now live in lp:linaro-license-protection/configs) and
prepare a script to be used for updating staging instance
from a cronjob.

No tests for the update script yet, but I will be providing
them later.

I'll also modify the script later to be possible to use it
for production deployments as well.

[James Tunnicliffe (dooferlad)]

Looks good to me (apart from, as you pointed out, the lack of tests).

review: Approve

=== modified file 'README'
--- README	2012-08-06 12:30:16 +0000
+++ README	2012-08-13 11:45:25 +0000
@@ -34,6 +34,7 @@
 ............
 
 libapache2-mod-xsendfile >= 0.10
+libapache2-mod-python
 python-django >= 1.3.1
 python-django-openid-auth
 
@@ -41,22 +42,13 @@
   python 2.7, testrepository (>=0.0.6), python-html2text,
   python-subunit, python-beautifulsoup, python-mock
 
-Deployment steps
-................
-
-- Checkout trunk branch into /srv/releases.linaro.org/linaro-license-protection
-- Create directory for database: /srv/releases.linaro.org/db
-- Create database, from /srv/releases.linaro.org/linaro-license-protection run
-  python ./manage.py syncdb --noinput
-- Create symlink for the application in /srv/snapshots.linaro.org:
-  ln -s /srv/releases.linaro.org/linaro-license-protection \
-  /srv/snapshots.linaro.org/linaro-license-protection
-- Create directory for database: /srv/snapshots.linaro.org/db
-- Create database, from /srv/snapshots.linaro.org/linaro-license-protection run
-  python ./manage.py syncdb --noinput
-- Put apache2 configs from branch lp:~linaro-infrastructure/linaro-license-protection/apache-configs
-  in /etc/apache2/sites-available
-- Enable sites: a2ensite releases.linaro.org snapshots.linaro.org
+Deployment
+..........
+
+Please see
+
+  http://bazaar.launchpad.net/~linaro-infrastructure/linaro-license-protection/configs/view/head:/README
+
 
 Technical details
 -----------------

=== removed file 'license_protected_downloads/wsgi_releases.py'
--- license_protected_downloads/wsgi_releases.py	2012-08-06 11:53:16 +0000
+++ license_protected_downloads/wsgi_releases.py	1970-01-01 00:00:00 +0000
@@ -1,42 +0,0 @@
-"""
-WSGI config for license_protected_downloads project.
-
-This module contains the WSGI application used by Django's development server
-and any production WSGI deployments. It should expose a module-level variable
-named ``application``. Django's ``runserver`` and ``runfcgi`` commands discover
-this application via the ``WSGI_APPLICATION`` setting.
-
-Usually you will have the standard Django WSGI application here, but it also
-might make sense to replace the whole Django WSGI application with a custom one
-that later delegates to the Django one. For example, you could introduce WSGI
-middleware here, or combine a Django application with an application of another
-framework.
-
-"""
-import os
-import sys
-sys.path.append("/usr/lib/python2.7/dist-packages")
-sys.path.append("/usr/lib/pymodules/python2.7")
-sys.path.append("/usr/lib/python2.7")
-
-# NOTE: Set actual path to directory containing linaro-license-protection app
-sys.path.append("/srv/releases.linaro.org")
-sys.path.append("/srv/releases.linaro.org/linaro-license-protection")
-
-os.environ.setdefault(
-    "DJANGO_SETTINGS_MODULE",
-    "linaro-license-protection.settings_releases"
-)
-
-# This application object is used by any WSGI server configured to use this
-# file. This includes Django's development server, if the WSGI_APPLICATION
-# setting points here.
-#from django.core.wsgi import get_wsgi_application
-#application = get_wsgi_application()
-
-import django.core.handlers.wsgi
-application = django.core.handlers.wsgi.WSGIHandler()
-
-# Apply WSGI middleware here.
-# from helloworld.wsgi import HelloWorldApplication
-# application = HelloWorldApplication(application)

=== removed file 'license_protected_downloads/wsgi_snapshots.py'
--- license_protected_downloads/wsgi_snapshots.py	2012-08-06 11:53:16 +0000
+++ license_protected_downloads/wsgi_snapshots.py	1970-01-01 00:00:00 +0000
@@ -1,42 +0,0 @@
-"""
-WSGI config for license_protected_downloads project.
-
-This module contains the WSGI application used by Django's development server
-and any production WSGI deployments. It should expose a module-level variable
-named ``application``. Django's ``runserver`` and ``runfcgi`` commands discover
-this application via the ``WSGI_APPLICATION`` setting.
-
-Usually you will have the standard Django WSGI application here, but it also
-might make sense to replace the whole Django WSGI application with a custom one
-that later delegates to the Django one. For example, you could introduce WSGI
-middleware here, or combine a Django application with an application of another
-framework.
-
-"""
-import os
-import sys
-sys.path.append("/usr/lib/python2.7/dist-packages")
-sys.path.append("/usr/lib/pymodules/python2.7")
-sys.path.append("/usr/lib/python2.7")
-
-# NOTE: Set actual path to directory containing linaro-license-protection app
-sys.path.append("/srv/snapshots.linaro.org")
-sys.path.append("/srv/snapshots.linaro.org/linaro-license-protection")
-
-os.environ.setdefault(
-    "DJANGO_SETTINGS_MODULE",
-    "linaro-license-protection.settings_snapshots"
-)
-
-# This application object is used by any WSGI server configured to use this
-# file. This includes Django's development server, if the WSGI_APPLICATION
-# setting points here.
-#from django.core.wsgi import get_wsgi_application
-#application = get_wsgi_application()
-
-import django.core.handlers.wsgi
-application = django.core.handlers.wsgi.WSGIHandler()
-
-# Apply WSGI middleware here.
-# from helloworld.wsgi import HelloWorldApplication
-# application = HelloWorldApplication(application)

=== added file 'scripts/update-deployment.py'
--- scripts/update-deployment.py	1970-01-01 00:00:00 +0000
+++ scripts/update-deployment.py	2012-08-13 11:45:25 +0000
@@ -0,0 +1,149 @@
+#!/usr/bin/env python2.7
+
+"""Update a deployment of lp:linaro-license-protection.
+
+This script assumes that initial deployment has been done in accordance
+with the README as found in lp:linaro-license-protection/configs.
+
+This means at least:
+
+ * /srv/shared-branches/linaro-license-protection
+   Branch of lp:linaro-license-protection (script will do 'bzr pull' on it)
+
+ * /srv/shared-branches/linaro-license-protection-config
+   Branch of lp:linaro-license-protection/configs
+
+ * /srv/staging.snapshots.linaro.org/linaro-license-protection
+   Checkout of /srv/shared-branches/linaro-license-protection
+   (script will do an equivalent of 'bzr update' on it)
+
+   Note that this applies to staging snapshots config.  Replace paths
+   accordingly.
+
+ * /srv/staging.snapshots.linaro.org/configs
+   Checkout of /srv/shared-branches/linaro-license-protection-configs
+
+ * /srv/staging.snapshots.linaro.org/db exists and is writeable by
+   both apache and the user running update-deployment.py script.
+
+ * /srv/staging.snapshots.linaro.org/linaro-license-protection/static
+   is writeable by the user running update-deployment.py script.
+
+Supported configs so far are snapshots.linaro.org, releases.linaro.org,
+staging.snapshots.linaro.org and staging.releases.linaro.org.
+
+"""
+
+import argparse
+import bzrlib.branch
+import logging
+import os
+import subprocess
+
+code_base = '/srv/shared-branches'
+branch_name = 'linaro-license-protection'
+configs_branch_name = 'linaro-license-protection-config'
+snapshots_root = '/srv/snapshots.linaro.org'
+releases_root = '/srv/releases.linaro.org'
+staging_snapshots_root = '/srv/staging.snapshots.linaro.org'
+staging_releases_root = '/srv/staging.releases.linaro.org'
+
+configs_to_use = {
+    "settings_releases": releases_root,
+    "settings_snapshots": snapshots_root,
+    "settings_staging_releases": staging_releases_root,
+    "settings_staging_snapshots": staging_snapshots_root,
+    }
+
+logging_level = logging.DEBUG
+
+code_root = os.path.join(code_base, branch_name)
+configs_root = os.path.join(code_base, configs_branch_name)
+
+
+def refresh_branch(branch_dir):
+    """Refreshes a branch checked-out to a branch_dir."""
+
+    code_branch = bzrlib.branch.Branch.open(branch_dir)
+    parent_branch = bzrlib.branch.Branch.open(
+        code_branch.get_parent())
+    result = code_branch.pull(source=parent_branch)
+    if result.old_revno != result.new_revno:
+        logger.info("Updated %s from %d to %d.",
+                    branch_dir, result.old_revno, result.new_revno)
+    else:
+        logger.info("No changes to pull from %s.", code_branch.get_parent())
+    return code_branch
+
+
+def update_branch(branch_dir):
+    """Does a checkout update."""
+    code_branch = bzrlib.branch.Branch.open(branch_dir)
+    code_branch.update()
+
+
+def update_installation(config, installation_root):
+    """Updates a single installation code and databases.
+
+    It expects code and config branches to be simple checkouts so it only
+    does an "update" on them.
+
+    Afterwards, it runs "syncdb" and "collectstatic" steps.
+    """
+    update_branch(os.path.join(installation_root, branch_name))
+    update_branch(os.path.join(installation_root, "configs"))
+    os.environ["PYTHONPATH"] = (
+        ":".join(
+            [installation_root,
+             os.path.join(installation_root, branch_name),
+             os.path.join(installation_root, "configs", "django"),
+             os.environ.get("PYTHONPATH", "")]))
+
+    logger.info("Updating installation in %s with config %s...",
+                installation_root, config)
+    os.environ["DJANGO_SETTINGS_MODULE"] = config
+    logger.debug("DJANGO_SETTINGS_MODULE=%s",
+                 os.environ.get("DJANGO_SETTINGS_MODULE"))
+
+    logger.debug("Doing 'syncdb'...")
+    logger.debug(subprocess.check_output(
+        ["django-admin", "syncdb", "--noinput"], cwd=code_root))
+
+    logger.debug("Doing 'collectstatic'...")
+    logger.debug(subprocess.check_output(
+        ["django-admin", "collectstatic", "--noinput"],
+        cwd=code_root))
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        description=(
+            "Update staging deployment of lp:linaro-license-protection."))
+    parser.add_argument(
+        'configs', metavar='CONFIG', nargs='+', choices=configs_to_use.keys(),
+        help=("Django configuration module to use. One of " +
+              ', '.join(configs_to_use.keys())))
+    parser.add_argument("-v", "--verbose", action='count',
+                        help=("Increase the output verbosity. "
+                              "Can be used multiple times"))
+    args = parser.parse_args()
+
+    if args.verbose == 0:
+        logging_level = logging.ERROR
+    elif args.verbose == 1:
+        logging_level = logging.INFO
+    elif args.verbose >= 2:
+        logging_level = logging.DEBUG
+
+    logger = logging.getLogger('update-staging')
+    logging.basicConfig(
+        format='%(asctime)s %(levelname)s: %(message)s',
+        level=logging_level)
+
+    # Refresh code in shared-branches.
+    refresh_branch(code_root)
+    refresh_branch(configs_root)
+
+    # We update installations for all the configs we've got.
+    for config in args.configs:
+        update_installation(config, configs_to_use[config])

=== removed file 'settings_releases.py'
--- settings_releases.py	2012-08-06 11:53:16 +0000
+++ settings_releases.py	1970-01-01 00:00:00 +0000
@@ -1,16 +0,0 @@
-from settings import *
-
-
-DATABASES = {
-    'default': {
-        'ENGINE': 'django.db.backends.sqlite3',
-        'NAME': '/srv/releases.linaro.org/db/licenses.db',
-        'USER': '',
-        'PASSWORD': '',
-        'HOST': '',
-        'PORT': '',
-    }
-}
-
-TEMPLATE_DIRS = (os.path.join(PROJECT_ROOT, "templates_releases" ),)
-SERVED_PATHS = ['/srv/releases.linaro.org/www']

=== removed file 'settings_snapshots.py'
--- settings_snapshots.py	2012-08-06 11:53:16 +0000
+++ settings_snapshots.py	1970-01-01 00:00:00 +0000
@@ -1,15 +0,0 @@
-from settings import *
-
-DATABASES = {
-    'default': {
-        'ENGINE': 'django.db.backends.sqlite3',
-        'NAME': '/srv/snapshots.linaro.org/db/licenses.db',
-        'USER': '',
-        'PASSWORD': '',
-        'HOST': '',
-        'PORT': '',
-    }
-}
-
-TEMPLATE_DIRS = (os.path.join(PROJECT_ROOT, "templates_snapshots" ),)
-SERVED_PATHS = ['/srv/snapshots.linaro.org/www']