Merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server
- wheels-support-update
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Kelvin Li |
Approved revision: | 336 |
Merged at revision: | 313 |
Proposed branch: | lp:~kelvin.li/rnr-server/wheels-support-update |
Merge into: | lp:rnr-server |
Diff against target: |
375 lines (+109/-78) 7 files modified
Makefile (+61/-30) config-manager.txt (+1/-25) requirements-devel.txt (+14/-0) requirements.txt (+22/-12) scripts/settings-to-conncheck.py (+3/-3) src/core/tests/test_wsgi.py (+6/-4) tarmac_verify.sh (+2/-4) |
To merge this branch: | bzr merge lp:~kelvin.li/rnr-server/wheels-support-update |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Fabián Ezequiel Gallina (community) | Approve | ||
Review via email: mp+275551@code.launchpad.net |
Commit message
1.Wheels support for rnr-server app to improve the bootstrap speed;
2.change the correct request/response data for the conn-check on rnr
3.Update to use new python-bson method to load oops doc content to make the tests run without issues.
Description of the change
1.Wheels support for rnr-server app to improve the bootstrap speed;
2.change the correct request/response data for the conn-check on rnr
3.Update to use new python-bson method to load oops doc content to make the tests run without issues.
Fabián Ezequiel Gallina (fgallina) wrote : | # |
Kelvin Li (kelvin.li) wrote : | # |
comments replied.
Fabián Ezequiel Gallina (fgallina) : | # |
Fabián Ezequiel Gallina (fgallina) wrote : | # |
LGTM
Let's handle the django-web-theme removal separately:
2015-10-26 09:27 <fgallina> kelvin: ack, with that I think we are good to go.
2015-10-26 09:30 <kelvin> fgallina: thanks, but as my reply, can wen ignore
2015-10-26 09:32 <fgallina> kelvin: I think we could, but I'd need to
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server failed. Below is the output from the failed tests.
echo 'cleaning virtualenv...'
cleaning virtualenv...
rm -rf /mnt/tarmac/
make sanitize-sourcedeps
make[1]: Entering directory `/mnt/tarmac/
sed -e '1,/# Dependencies/
make[1]: Leaving directory `/mnt/tarmac/
make CONFIGMANAGER=
make[1]: Entering directory `/mnt/tarmac/
/usr/lib/
make[1]: Leaving directory `/mnt/tarmac/
touch branches/last_build
make: *** No rule to make target `virtualenv', needed by `fetch-sourcedeps'. Stop.
Kelvin Li (kelvin.li) wrote : | # |
after trying to figure out above test failure,
I found that we have to put requests_oauthlib and oauthlib dependency libs back to the requirement file(requiremen
ImportError: Failed to import test module: core.tests.
Traceback (most recent call last):
File "/usr/lib/
module = self._get_
File "/usr/lib/
__import_
File "/home/
from core.api.auth import (
File "/home/
from core.api.auth.sso import SSOOAuthAuthent
File "/home/
from core.utilities import (
File "/home/
from ssoclient.v2.client import V2ApiClient
File "/home/
from .client import V2ApiClient
File "/home/
from requests_oauthlib import OAuth1
File "/home/
from .oauth2_auth import OAuth2
File "/home/
from oauthlib.oauth2 import is_secure_transport
ImportError: cannot import name is_secure_transport
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server failed. Below is the output from the failed tests.
echo 'cleaning virtualenv...'
cleaning virtualenv...
rm -rf /mnt/tarmac/
make sanitize-sourcedeps
make[1]: Entering directory `/mnt/tarmac/
sed -e '1,/# Dependencies/
make[1]: Leaving directory `/mnt/tarmac/
make CONFIGMANAGER=
make[1]: Entering directory `/mnt/tarmac/
/usr/lib/
make[1]: Leaving directory `/mnt/tarmac/
touch branches/last_build
[ -d branches/wheels ] && (cd branches/wheels && bzr pull) || (bzr branch lp:~ubuntuone-pqm-team/rnr-server/dependencies branches/wheels)
bzr version-info --format=python > lib/versioninfo.py
virtualenv not found, creating new one...
Not deleting /mnt/tarmac/
New python executable in /mnt/tarmac/
Installing distribute.
Installing pip....
/mnt/tarmac/
Ignoring indexes: http://
Downloading/
Could not find any downloads that satisfy the requirement amqp==1.4.5 (from -r requirements.txt (line 1))
No distributions at all found for amqp==1.4.5 (from -r requirements.txt (line 1))
Storing complete log in /mnt/tarmac/
Branched 3 revisions.
/bin/sh: 1: -e: not found
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/
self.run()
File "/usr/lib/
self.
File "/mnt/tarmac/
page = self._get_
File "/mnt/tarmac/
return HTMLPage.
File "/mnt/tarmac/
resp = urlopen(url)
File "/mnt/tarmac/
response = urllib2.
File "/usr/lib/
return _opener.op...
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server failed. Below is the output from the failed tests.
echo 'cleaning virtualenv...'
cleaning virtualenv...
rm -rf /mnt/tarmac/
make sanitize-sourcedeps
make[1]: Entering directory `/mnt/tarmac/
sed -e '1,/# Dependencies/
make[1]: Leaving directory `/mnt/tarmac/
make CONFIGMANAGER=
make[1]: Entering directory `/mnt/tarmac/
/usr/lib/
make[1]: Leaving directory `/mnt/tarmac/
touch branches/last_build
bzr version-info --format=python > lib/versioninfo.py
make wheels
make[1]: Entering directory `/mnt/tarmac/
[ -d branches/wheels ] && (cd branches/wheels && bzr pull) || (bzr branch lp:~ubuntuone-pqm-team/rnr-server/dependencies branches/wheels)
make[1]: Leaving directory `/mnt/tarmac/
virtualenv not found, creating new one...
Not deleting /mnt/tarmac/
New python executable in /mnt/tarmac/
Installing distribute.
Installing pip....
/mnt/tarmac/
Ignoring indexes: http://
Downloading/
Could not find any downloads that satisfy the requirement amqp==1.4.5 (from -r requirements.txt (line 1))
No distributions at all found for amqp==1.4.5 (from -r requirements.txt (line 1))
Storing complete log in /mnt/tarmac/
Branched 3 revisions.
/bin/sh: 1: -e: not found
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/
self.run()
File "/usr/lib/
self.
File "/mnt/tarmac/
page = self._get_
File "/mnt/tarmac/
return HTMLPage.
File "/mnt/tarmac/
resp = urlopen(url)
File "/mnt/tarmac/
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server failed. Below is the output from the failed tests.
echo 'cleaning virtualenv...'
cleaning virtualenv...
rm -rf /mnt/tarmac/
make sanitize-sourcedeps
make[1]: Entering directory `/mnt/tarmac/
sed -e '1,/# Dependencies/
make[1]: Leaving directory `/mnt/tarmac/
make CONFIGMANAGER=
make[1]: Entering directory `/mnt/tarmac/
/usr/lib/
make[1]: Leaving directory `/mnt/tarmac/
touch branches/last_build
bzr version-info --format=python > lib/versioninfo.py
make wheels
make[1]: Entering directory `/mnt/tarmac/
[ -d branches/wheels ] && (cd branches/wheels && bzr pull) || (bzr branch lp:~ubuntuone-pqm-team/rnr-server/dependencies branches/wheels)
make[1]: Leaving directory `/mnt/tarmac/
make install-wheels
make[1]: Entering directory `/mnt/tarmac/
virtualenv not found, creating new one...
Not deleting /mnt/tarmac/
New python executable in /mnt/tarmac/
Installing distribute.
Installing pip....
/mnt/tarmac/
Ignoring indexes: http://
Downloading/
Could not find any downloads that satisfy the requirement amqp==1.4.5 (from -r requirements.txt (line 1))
No distributions at all found for amqp==1.4.5 (from -r requirements.txt (line 1))
Storing complete log in /mnt/tarmac/
make[1]: Leaving directory `/mnt/tarmac/
Branched 3 revisions.
/bin/sh: 1: -e: not found
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/
self.run()
File "/usr/lib/
self.
File "/mnt/tarmac/
page = self._get_
File "/mnt/tarmac/
return HTMLPage.
File "/mnt/tarmac/
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server failed. Below is the output from the failed tests.
echo 'cleaning virtualenv...'
cleaning virtualenv...
rm -rf /mnt/tarmac/
virtualenv not found, creating new one...
Not deleting /mnt/tarmac/
New python executable in /mnt/tarmac/
Installing distribute.
Installing pip....
make sanitize-sourcedeps
make[1]: Entering directory `/mnt/tarmac/
sed -e '1,/# Dependencies/
make[1]: Leaving directory `/mnt/tarmac/
make CONFIGMANAGER=
make[1]: Entering directory `/mnt/tarmac/
/usr/lib/
make[1]: Leaving directory `/mnt/tarmac/
touch branches/last_build
bzr version-info --format=python > lib/versioninfo.py
make wheels
make[1]: Entering directory `/mnt/tarmac/
[ -d branches/wheels ] && (cd branches/wheels && bzr pull) || (bzr branch lp:~ubuntuone-pqm-team/rnr-server/dependencies branches/wheels)
make[1]: Leaving directory `/mnt/tarmac/
make install-wheels
make[1]: Entering directory `/mnt/tarmac/
/mnt/tarmac/
Ignoring indexes: http://
Downloading/
Could not find any downloads that satisfy the requirement amqp==1.4.5 (from -r requirements.txt (line 1))
No distributions at all found for amqp==1.4.5 (from -r requirements.txt (line 1))
Storing complete log in /mnt/tarmac/
make[1]: Leaving directory `/mnt/tarmac/
/bin/sh: 1: -e: not found
Branched 3 revisions.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/
self.run()
File "/usr/lib/
self.
File "/mnt/tarmac/
page = self._get_
File "/mnt/tarmac/
return HTMLPage.
File "/mnt/tarmac/
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server failed. Below is the output from the failed tests.
echo 'cleaning virtualenv...'
cleaning virtualenv...
rm -rf /mnt/tarmac/
virtualenv not found, creating new one...
Not deleting /mnt/tarmac/
New python executable in /mnt/tarmac/
Installing distribute.
Installing pip....
make sanitize-sourcedeps
make[1]: Entering directory `/mnt/tarmac/
sed -e '1,/# Dependencies/
make[1]: Leaving directory `/mnt/tarmac/
make CONFIGMANAGER=
make[1]: Entering directory `/mnt/tarmac/
/usr/lib/
make[1]: Leaving directory `/mnt/tarmac/
touch branches/last_build
bzr version-info --format=python > lib/versioninfo.py
[ -d branches/wheels ] && (cd branches/wheels && bzr pull) || (bzr branch lp:~ubuntuone-pqm-team/rnr-server/dependencies branches/wheels)
touch branches/
/mnt/tarmac/
Ignoring indexes: http://
Downloading/
Could not find any downloads that satisfy the requirement amqp==1.4.5 (from -r requirements.txt (line 1))
No distributions at all found for amqp==1.4.5 (from -r requirements.txt (line 1))
Storing complete log in /mnt/tarmac/
/bin/sh: 1: -e: not found
Branched 3 revisions.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/
self.run()
File "/usr/lib/
self.
File "/mnt/tarmac/
page = self._get_
File "/mnt/tarmac/
return HTMLPage.
File "/mnt/tarmac/
resp = urlopen(url)
File "/mnt/tarmac/
response = urllib2.
File "/usr/lib/
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server failed. Below is the output from the failed tests.
echo 'cleaning virtualenv...'
cleaning virtualenv...
rm -rf /mnt/tarmac/
virtualenv not found, creating new one...
Not deleting /mnt/tarmac/
New python executable in /mnt/tarmac/
Installing distribute.
Installing pip....
make sanitize-sourcedeps
make[1]: Entering directory `/mnt/tarmac/
sed -e '1,/# Dependencies/
make[1]: Leaving directory `/mnt/tarmac/
make CONFIGMANAGER=
make[1]: Entering directory `/mnt/tarmac/
/usr/lib/
make[1]: Leaving directory `/mnt/tarmac/
touch branches/last_build
bzr version-info --format=python > lib/versioninfo.py
[ -d branches/wheels ] && (cd branches/wheels && bzr pull) || (bzr branch lp:~ubuntuone-pqm-team/rnr-server/dependencies branches/wheels)
touch branches/
/mnt/tarmac/
Ignoring indexes: http://
Downloading/
Could not find any downloads that satisfy the requirement amqp==1.4.5 (from -r requirements.txt (line 1))
No distributions at all found for amqp==1.4.5 (from -r requirements.txt (line 1))
Storing complete log in /mnt/tarmac/
/bin/sh: 1: -e: not found
Branched 3 revisions.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/
self.run()
File "/usr/lib/
self.
File "/mnt/tarmac/
page = self._get_
File "/mnt/tarmac/
return HTMLPage.
File "/mnt/tarmac/
resp = urlopen(url)
File "/mnt/tarmac/
response = urllib2.
File "/usr/lib/
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server failed. Below is the output from the failed tests.
echo 'cleaning virtualenv...'
cleaning virtualenv...
rm -rf /mnt/tarmac/
virtualenv not found, creating new one...
Not deleting /mnt/tarmac/
New python executable in /mnt/tarmac/
Installing distribute.
Installing pip....
make sanitize-sourcedeps
make[1]: Entering directory `/mnt/tarmac/
sed -e '1,/# Dependencies/
make[1]: Leaving directory `/mnt/tarmac/
make CONFIGMANAGER=
make[1]: Entering directory `/mnt/tarmac/
/usr/lib/
make[1]: Leaving directory `/mnt/tarmac/
touch branches/last_build
bzr version-info --format=python > lib/versioninfo.py
[ -d branches/wheels ] && (cd branches/wheels && bzr pull) || (bzr branch lp:~ubuntuone-pqm-team/rnr-server/dependencies branches/wheels)
/mnt/tarmac/
Ignoring indexes: http://
Downloading/
Could not find any downloads that satisfy the requirement amqp==1.4.5 (from -r requirements.txt (line 1))
No distributions at all found for amqp==1.4.5 (from -r requirements.txt (line 1))
Storing complete log in /mnt/tarmac/
/bin/sh: 1: -e: not found
Branched 3 revisions.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/
self.run()
File "/usr/lib/
self.
File "/mnt/tarmac/
page = self._get_
File "/mnt/tarmac/
return HTMLPage.
File "/mnt/tarmac/
resp = urlopen(url)
File "/mnt/tarmac/
response = urllib2.
File "/usr/lib/
return _opener.op...
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server failed. Below is the output from the failed tests.
echo 'cleaning virtualenv...'
cleaning virtualenv...
rm -rf /mnt/tarmac/
virtualenv not found, creating new one...
Not deleting /mnt/tarmac/
New python executable in /mnt/tarmac/
Installing distribute.
Installing pip....
make sanitize-sourcedeps
make[1]: Entering directory `/mnt/tarmac/
sed -e '1,/# Dependencies/
make[1]: Leaving directory `/mnt/tarmac/
make CONFIGMANAGER=
make[1]: Entering directory `/mnt/tarmac/
/usr/lib/
make[1]: Leaving directory `/mnt/tarmac/
touch branches/last_build
bzr version-info --format=python > lib/versioninfo.py
[ -d branches/wheels ] && (cd branches/wheels && bzr pull) || (bzr branch lp:~ubuntuone-pqm-team/rnr-server/dependencies branches/wheels)
make .install-wheels
make[1]: Entering directory `/mnt/tarmac/
/mnt/tarmac/
Ignoring indexes: http://
Downloading/
Could not find any downloads that satisfy the requirement amqp==1.4.5 (from -r requirements.txt (line 1))
No distributions at all found for amqp==1.4.5 (from -r requirements.txt (line 1))
Storing complete log in /mnt/tarmac/
make[1]: Leaving directory `/mnt/tarmac/
/bin/sh: 1: -e: not found
Branched 3 revisions.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/
self.run()
File "/usr/lib/
self.
File "/mnt/tarmac/
page = self._get_
File "/mnt/tarmac/
return HTMLPage.
File "/mnt/tarmac/
resp = urlopen(url)
File "/mnt/tarmac/
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server failed. Below is the output from the failed tests.
echo 'cleaning virtualenv...'
cleaning virtualenv...
rm -rf /mnt/tarmac/
echo 'creating new virtualenv...'
creating new virtualenv...
virtualenv --clear --system-
Not deleting /mnt/tarmac/
New python executable in /mnt/tarmac/
Installing setuptools, pip...done.
Generated devel local settings.
make sanitize-sourcedeps
make[1]: Entering directory `/mnt/tarmac/
sed -e '1,/# Dependencies/
make[1]: Leaving directory `/mnt/tarmac/
make CONFIGMANAGER=
make[1]: Entering directory `/mnt/tarmac/
/usr/lib/
make[1]: Leaving directory `/mnt/tarmac/
touch branches/last_build
[ -d branches/wheels ] && (cd branches/wheels && bzr pull) || (bzr branch lp:~ubuntuone-pqm-team/rnr-server/dependencies branches/wheels)
bzr version-info --format=python > lib/versioninfo.py
/mnt/tarmac/
Ignoring indexes: https:/
Downloading/
Downloading/
Downloading/
Downloading/
Downloading/
Requirement already satisfied (use --upgrade to upgrade): python-
Downloading/
Requirement already satisfied (use --upgrade to upgrade): oops==0.0.13 in /usr/lib/
Requirement already satisfied (use --upgrade to upgrade): oops-amqp==0.0.7 in /usr/lib/
Downloading/
Downloading/
Downloading/
Downloading/
Downloading/
Downloading/
Downloading/
Downloading/
Downloading/
Ubuntu One Auto Pilot (otto-pilot) wrote : | # |
The attempt to merge lp:~kelvin.li/rnr-server/wheels-support-update into lp:rnr-server failed. Below is the output from the failed tests.
echo 'cleaning virtualenv...'
cleaning virtualenv...
rm -rf /mnt/tarmac/
echo 'creating new virtualenv...'
creating new virtualenv...
virtualenv --clear --system-
Not deleting /mnt/tarmac/
New python executable in /mnt/tarmac/
Installing distribute.
Installing pip....
make sanitize-sourcedeps
make[1]: Entering directory `/mnt/tarmac/
sed -e '1,/# Dependencies/
make[1]: Leaving directory `/mnt/tarmac/
make CONFIGMANAGER=
make[1]: Entering directory `/mnt/tarmac/
/usr/lib/
make[1]: Leaving directory `/mnt/tarmac/
touch branches/last_build
[ -d branches/wheels ] && (cd branches/wheels && bzr pull) || (bzr branch lp:~ubuntuone-pqm-team/rnr-server/dependencies branches/wheels)
bzr version-info --format=python > lib/versioninfo.py
/mnt/tarmac/
Ignoring indexes: http://
Downloading/
Could not find any downloads that satisfy the requirement amqp==1.4.5 (from -r requirements.txt (line 1))
No distributions at all found for amqp==1.4.5 (from -r requirements.txt (line 1))
Storing complete log in /mnt/tarmac/
Branched 3 revisions.
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/
self.run()
File "/usr/lib/
self.
File "/mnt/tarmac/
page = self._get_
File "/mnt/tarmac/
return HTMLPage.
File "/mnt/tarmac/
resp = urlopen(url)
File "/mnt/tarmac/
response = urllib2.
Preview Diff
1 | === modified file 'Makefile' |
2 | --- Makefile 2015-10-16 05:08:29 +0000 |
3 | +++ Makefile 2015-10-27 08:00:26 +0000 |
4 | @@ -1,10 +1,11 @@ |
5 | -#!/usr/bin/make |
6 | -CM = /usr/lib/config-manager/cm.py |
7 | -PYTHON := $(shell which python) |
8 | +# Copyright (C) 2014-2015 Canonical Ltd. |
9 | + |
10 | + |
11 | +CM ?= /usr/lib/config-manager/cm.py |
12 | CM_TMP_CONFIG = config-manager.txt.tmp |
13 | |
14 | HERE := $(shell pwd) |
15 | -PYTHONPATH := ${HERE}/src:${HERE}/lib:${PYTHONPATH} |
16 | +PYTHONPATH := ${HERE}/src:${HERE}/lib:${HERE}:${PYTHONPATH} |
17 | DJANGO_SETTINGS_MODULE ?= django_project.settings |
18 | CONN_CHECK_CONFIG_PATH ?= /tmp/rnr_conn_check_config.yaml |
19 | |
20 | @@ -18,6 +19,13 @@ |
21 | JUJU_REPO ?= ../.juju-repo |
22 | PROJECT_NAME = rnr-server |
23 | |
24 | +PYTHON = $(VIRTUALENV_DIR)/bin/python |
25 | +# use env's python to run pip in order to overcome shebang's max line length limitation |
26 | +PIP = $(PYTHON) $(VIRTUALENV_DIR)/bin/pip |
27 | + |
28 | +WHEELS_BRANCH_URL ?= lp:~ubuntuone-pqm-team/$(PROJECT_NAME)/dependencies |
29 | +WHEELS_DIR = branches/wheels |
30 | + |
31 | # Create archives in labelled directories (ie. r27/ratings-and-review.tbz2) |
32 | TARBALL_BUILD_LABEL ?= r$(shell bzr revno) |
33 | TARBALL_FILE_NAME = ratings-and-reviews.tbz2 |
34 | @@ -25,12 +33,13 @@ |
35 | TARBALL_BUILD_DIR = $(TARBALL_BUILDS_DIR)/$(TARBALL_BUILD_LABEL) |
36 | TARBALL_BUILD_PATH = $(TARBALL_BUILD_DIR)/$(TARBALL_FILE_NAME) |
37 | |
38 | -DJANGO_MANAGE := $(PYTHON) $(CURDIR)/django_project/manage.py |
39 | +DJANGO_MANAGE = ${VIRTUALENV_BIN}/python ${HERE}/django_project/manage.py |
40 | + |
41 | +STATIC_ROOT ?= ${HERE}/staticfiles |
42 | |
43 | export PYTHONPATH |
44 | export DJANGO_SETTINGS_MODULE |
45 | |
46 | - |
47 | local-config: |
48 | @if test ! -e $(LOCAL_SETTINGS_PATH); then \ |
49 | mkdir -p $(LOCAL_SETTINGS_DIR); \ |
50 | @@ -38,20 +47,17 @@ |
51 | echo 'Generated devel local settings.'; \ |
52 | fi; |
53 | |
54 | -virtualenv: |
55 | - @if test ! -e $(VIRTUALENV_DIR); ! -e $(VIRTUALENV_BIN); then \ |
56 | - echo 'virtualenv not found, creating new one...'; \ |
57 | - virtualenv --distribute --clear --system-site-packages $(VIRTUALENV_DIR); \ |
58 | - fi; |
59 | +manage: |
60 | + $(DJANGO_MANAGE) $(ARGS) |
61 | + |
62 | +$(VIRTUALENV_DIR): |
63 | + echo 'creating new virtualenv...' |
64 | + virtualenv --clear --system-site-packages $(VIRTUALENV_DIR) |
65 | |
66 | clean-virtualenv: |
67 | echo 'cleaning virtualenv...' |
68 | rm -rf $(VIRTUALENV_DIR) |
69 | |
70 | -install-virtualenv-requirements: virtualenv |
71 | - $(VIRTUALENV_DIR)/bin/pip install -r requirements.txt |
72 | - |
73 | - |
74 | version: |
75 | bzr version-info --format=python > lib/versioninfo.py |
76 | |
77 | @@ -72,14 +78,17 @@ |
78 | @rm $(CM_TMP_CONFIG) |
79 | touch $@ |
80 | |
81 | +wheels: |
82 | + [ -d $(WHEELS_DIR) ] && (cd $(WHEELS_DIR) && bzr pull) || (bzr branch $(WHEELS_BRANCH_URL) $(WHEELS_DIR)) |
83 | + |
84 | # use the flag file to avoid re-run config-manager if nothing changed |
85 | -fetch-sourcedeps: branches/last_build virtualenv version |
86 | - |
87 | -sourcedeps: fetch-sourcedeps install-virtualenv-requirements |
88 | - |
89 | -bootstrap: local-config sourcedeps |
90 | - |
91 | -bootstrap-juju: local-config sourcedeps-juju collectstatic |
92 | +fetch-sourcedeps: branches/last_build wheels version |
93 | + |
94 | +sourcedeps: fetch-sourcedeps install-wheels |
95 | + |
96 | +bootstrap: $(VIRTUALENV_DIR) local-config sourcedeps install-wheels-dev |
97 | + |
98 | +bootstrap-juju: $(VIRTUALENV_DIR) local-config sourcedeps-juju install-wheels-dev |
99 | |
100 | clean: |
101 | @echo "Cleaning ..." |
102 | @@ -106,12 +115,16 @@ |
103 | @rm $(CM_TMP_CONFIG) |
104 | touch $@ |
105 | |
106 | -fetch-sourcedeps-juju: branches/last_build_for_juju virtualenv version |
107 | +fetch-sourcedeps-juju: branches/last_build_for_juju version wheels |
108 | |
109 | -sourcedeps-juju: fetch-sourcedeps-juju install-virtualenv-requirements |
110 | +sourcedeps-juju: fetch-sourcedeps-juju install-wheels |
111 | |
112 | collectstatic: |
113 | - @DJANGO_SETTINGS_MODULE=django_project.settings_build $(DJANGO_MANAGE) collectstatic --noinput |
114 | + @DJANGO_SETTINGS_MODULE=django_project.settings_build $(MAKE) manage ARGS="collectstatic --noinput" |
115 | + |
116 | +$(STATIC_ROOT)/last_build: |
117 | + $(MAKE) collectstatic |
118 | + touch $@ |
119 | |
120 | #Tarball |
121 | $(TARBALL_BUILD_DIR): |
122 | @@ -121,8 +134,26 @@ |
123 | @echo "creating deployment tarball at $(TARBALL_BUILD_PATH)" |
124 | @tar -cj --exclude-from tarball_exclude.txt -f $(TARBALL_BUILD_PATH) ./ |
125 | |
126 | -build-tarball: $(TARBALL_BUILD_PATH) |
127 | - |
128 | - |
129 | -.PHONY: local-config sanitize-sourcedeps clean-virtualenv \ |
130 | - install-virtualenv-requirements sourcedeps bootstrap clean collectstatic |
131 | +build-tarball: install-wheels $(TARBALL_BUILD_PATH) |
132 | + |
133 | +### wheels ### |
134 | + |
135 | +update-wheels: ARGS=-r requirements-devel.txt |
136 | +update-wheels: $(VIRTUALENV_DIR) |
137 | + $(PIP) wheel -w $(WHEELS_DIR) -f $(WHEELS_DIR) $(ARGS) |
138 | + |
139 | +install-wheels: ARGS=-r requirements.txt |
140 | +install-wheels: $(VIRTUALENV_DIR) |
141 | + $(PIP) install --find-links=$(WHEELS_DIR) --no-index $(ARGS) |
142 | + |
143 | +install-wheels-dev: $(VIRTUALENV_DIR) install-wheels |
144 | + $(MAKE) install-wheels ARGS="-r requirements-devel.txt --pre" |
145 | + |
146 | + |
147 | +test: ARGS=src -v 2 --failfast |
148 | +test: bootstrap |
149 | + $(DJANGO_MANAGE) test $(ARGS) |
150 | + |
151 | + |
152 | +.PHONY: local-config sanitize-sourcedeps clean-virtualenv |
153 | +.PHONY: sourcedeps clean collectstatic test |
154 | |
155 | === modified file 'config-manager.txt' |
156 | --- config-manager.txt 2015-08-31 00:05:14 +0000 |
157 | +++ config-manager.txt 2015-10-27 08:00:26 +0000 |
158 | @@ -5,36 +5,12 @@ |
159 | # the fab install_sourcedeps command rewrite this to work w/o bazaar.isd and |
160 | # to check them out to the current directory |
161 | |
162 | -# Code branch |
163 | -rnr-server bzr+ssh://bazaar.isd/~rnr-developers/rnr-server/trunk |
164 | -# Config branch |
165 | -rnr-server/branches/config bzr+ssh://bazaar.isd/~canonical-ca-hackers/ca-configs/rnr-config;revno=103 |
166 | # juju deployment |
167 | # Web themes |
168 | rnr-server/branches/django-web-themes bzr+ssh://bazaar.isd/~canonical-isd-hackers/web-themes/django;revno=171 |
169 | |
170 | # Dependencies |
171 | -rnr-server/branches/amqp bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/amqp/stable;revno=v1.4.5 |
172 | -rnr-server/branches/django bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/django/stable;revno=1.6.10 |
173 | -rnr-server/branches/django-adminaudit bzr+ssh://bazaar.isd/~canonical-isd-hackers/django-adminaudit/trunk;revno=53 |
174 | -rnr-server/branches/django-openid-auth bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/django-openid-auth/stable;revno=104 |
175 | -rnr-server/branches/django-pgtools bzr+ssh://bazaar.isd/~canonical-isd-hackers/django-pgtools/trunk;revno=8 |
176 | rnr-server/branches/django-piston bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/django-piston/stable;revno=5 |
177 | -rnr-server/branches/django-preflight bzr+ssh://bazaar.isd/~canonical-isd-hackers/django-preflight/trunk;revno=30 |
178 | -rnr-server/branches/django-south bzr+ssh://bazaar.isd/~ubuntuone-hackers/django-south/trunk;revno=5 |
179 | -rnr-server/branches/mockssoservice bzr+ssh://bazaar.isd/~canonical-isd-hackers/canonical-identity-provider/mock-service |
180 | -rnr-server/branches/oauthlib bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/oauthlib/stable;revno=0.6.3 |
181 | -rnr-server/branches/python-oops bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/python-oops/stable;revno=0.0.13 |
182 | -rnr-server/branches/python-oops-amqp bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/python-oops-amqp/stable;revno=0.0.7 |
183 | -rnr-server/branches/python-oops-datedir-repo bzr+ssh://bazaar.isd/~canonical-launchpad-branches/python-oops-datedir-repo/trunk;revno=47 |
184 | -rnr-server/branches/python-oops-dictconfig bzr+ssh://bazaar.isd/~canonical-ca-hackers/python-oops-dictconfig/trunk;revno=18 |
185 | -rnr-server/branches/python-oops-wsgi bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/python-oops-wsgi/stable;revno=39 |
186 | -rnr-server/branches/python-openid bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/python-openid/stable;revno=1989 |
187 | -rnr-server/branches/requests bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/requests/stable;revno=2.2.1 |
188 | -rnr-server/branches/requests-oauthlib bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/requests-oauthlib/stable;revno=16 |
189 | -rnr-server/branches/rnrclient bzr+ssh://bazaar.isd/~rnr-developers/rnr-server/rnrclient |
190 | rnr-server/branches/scaclient bzr+ssh://bazaar.isd/~canonical-ca-hackers/software-center/scaclient |
191 | -rnr-server/branches/ssoclient bzr+ssh://bazaar.isd/~canonical-isd-hackers/canonical-identity-provider/ssoclient;revno=20 |
192 | -rnr-server/branches/txstatsd bzr+ssh://bazaar.isd/~txstatsd-dev/txstatsd/trunk;revno=104 |
193 | rnr-server/branches/conn-check-config bzr+ssh://bazaar.isd/~ubuntuone-hackers/conn-check/configs;revno=23 |
194 | -rnr-server/branches/whitenoise bzr+ssh://bazaar.isd/~ubuntuone-pqm-team/whitenoise/stable |
195 | +rnr-server/branches/django-adminaudit bzr+ssh://bazaar.isd/~canonical-isd-hackers/django-adminaudit/trunk;revno=53 |
196 | |
197 | === added symlink 'lib/adminaudit' |
198 | === target is u'../branches/django-adminaudit/adminaudit' |
199 | === removed symlink 'lib/adminaudit' |
200 | === target was u'../branches/django-adminaudit/adminaudit' |
201 | === removed symlink 'lib/amqp' |
202 | === target was u'../branches/amqp/amqp' |
203 | === removed symlink 'lib/configglue' |
204 | === target was u'../branches/configglue/configglue' |
205 | === removed symlink 'lib/django' |
206 | === target was u'../branches/django/django' |
207 | === removed symlink 'lib/django_configglue' |
208 | === target was u'../branches/django-configglue/django_configglue' |
209 | === removed symlink 'lib/django_openid_auth' |
210 | === target was u'../branches/django-openid-auth/django_openid_auth' |
211 | === removed symlink 'lib/oauthlib' |
212 | === target was u'../branches/oauthlib/oauthlib' |
213 | === removed symlink 'lib/oops' |
214 | === target was u'../branches/python-oops/oops' |
215 | === removed symlink 'lib/oops_amqp' |
216 | === target was u'../branches/python-oops-amqp/oops_amqp' |
217 | === removed symlink 'lib/oops_datedir_repo' |
218 | === target was u'../branches/python-oops-datedir-repo/oops_datedir_repo' |
219 | === removed symlink 'lib/oops_dictconfig' |
220 | === target was u'../branches/python-oops-dictconfig/oops_dictconfig' |
221 | === removed symlink 'lib/oops_wsgi' |
222 | === target was u'../branches/python-oops-wsgi/oops_wsgi' |
223 | === removed symlink 'lib/openid' |
224 | === target was u'../branches/python-openid/openid' |
225 | === removed symlink 'lib/pgtools' |
226 | === target was u'../branches/django-pgtools/pgtools' |
227 | === removed symlink 'lib/preflight' |
228 | === target was u'../branches/django-preflight/preflight' |
229 | === removed symlink 'lib/requests' |
230 | === target was u'../branches/requests/requests' |
231 | === removed symlink 'lib/requests_oauthlib' |
232 | === target was u'../branches/requests-oauthlib/requests_oauthlib' |
233 | === removed symlink 'lib/rnrclient.py' |
234 | === target was u'../branches/rnrclient/rnrclient.py' |
235 | === removed symlink 'lib/south' |
236 | === target was u'../branches/django-south/south' |
237 | === removed symlink 'lib/sso_mockserver' |
238 | === target was u'../branches/mockssoservice/sso_mockserver' |
239 | === removed symlink 'lib/ssoclient' |
240 | === target was u'../branches/ssoclient/ssoclient' |
241 | === removed symlink 'lib/txstatsd' |
242 | === target was u'../branches/txstatsd/txstatsd' |
243 | === removed symlink 'lib/whitenoise' |
244 | === target was u'../branches/whitenoise/whitenoise' |
245 | === added file 'requirements-devel.txt' |
246 | --- requirements-devel.txt 1970-01-01 00:00:00 +0000 |
247 | +++ requirements-devel.txt 2015-10-27 08:00:26 +0000 |
248 | @@ -0,0 +1,14 @@ |
249 | +# These are only developer dependencies, generally for testing, all |
250 | +# python deps for running the server locally are installed by the |
251 | +# rnr-server-developer-dependencies package and config-manager. |
252 | +-r requirements.txt |
253 | + |
254 | +coverage==3.7.1 |
255 | +django_factory==0.11 |
256 | +mechanize |
257 | +mock==1.0.1 |
258 | +paste==1.7.5.1 |
259 | +pep8==1.5.7 |
260 | +piston-mini-client |
261 | +testtools==0.9.34 |
262 | +wsgi-intercept==0.5.1 |
263 | |
264 | === modified file 'requirements.txt' |
265 | --- requirements.txt 2015-05-18 06:15:28 +0000 |
266 | +++ requirements.txt 2015-10-27 08:00:26 +0000 |
267 | @@ -1,12 +1,22 @@ |
268 | -# These are only developer dependencies, generally for testing, all |
269 | -# python deps for running the server locally are installed by the |
270 | -# rnr-server-developer-dependencies package and config-manager. |
271 | -coverage |
272 | -django_factory==0.11 |
273 | -mechanize |
274 | -mock |
275 | -paste==1.7.5.1 |
276 | -pep8 |
277 | -piston-mini-client |
278 | -testtools |
279 | -wsgi-intercept==0.5.1 |
280 | +amqp==1.4.5 |
281 | +django==1.6.10 |
282 | +django-openid-auth==0.6 # bzr+ssh://bazaar.launchpad.net/~ubuntuone-pqm-team/django-openid-auth/trunk@114 |
283 | +django-pgtools==0.2 # bzr+ssh://bazaar.launchpad.net/~canonical-isd-hackers/django-pgtools/trunk@8 |
284 | +django-preflight==0.1.5 # bzr+ssh://bazaar.launchpad.net/~canonical-isd-hackers/django-preflight/trunk@31 |
285 | + |
286 | +python-openid==2.2.5 |
287 | +oauthlib==0.6.3 |
288 | +oops==0.0.13 |
289 | +oops-amqp==0.0.7 |
290 | +oops-datedir-repo==0.0.20 |
291 | +oops-dictconfig==0.0.4 |
292 | +oops-wsgi==0.0.11 # bzr+ssh://bazaar.launchpad.net/~ubuntuone-pqm-team/python-oops-wsgi/stable@39 |
293 | + |
294 | +requests-oauthlib==0.5.0 |
295 | +south==1.0.2 |
296 | +txstatsd==1.0.0 |
297 | +whitenoise==2.0.4 |
298 | +gunicorn==19.3.0 |
299 | + |
300 | +rnrclient # bzr+ssh://bazaar.isd/~rnr-developers/rnr-server/rnrclient |
301 | +ssoclient # bzr+ssh://bazaar.isd/~canonical-isd-hackers/canonical-identity-provider/ssoclient;revno=20 |
302 | |
303 | === modified file 'scripts/settings-to-conncheck.py' |
304 | --- scripts/settings-to-conncheck.py 2015-09-22 12:24:58 +0000 |
305 | +++ scripts/settings-to-conncheck.py 2015-10-27 08:00:26 +0000 |
306 | @@ -16,15 +16,15 @@ |
307 | ), |
308 | ( |
309 | settings['CLICK_INDEX_API_URI'], |
310 | - '', |
311 | + '/search', |
312 | 200, |
313 | 'HEAD', |
314 | ), |
315 | ( |
316 | settings['SCA_HOST_URL'], |
317 | '', |
318 | - 200, |
319 | - 'HEAD', |
320 | + 302, |
321 | + 'GET', |
322 | ), |
323 | ) |
324 | |
325 | |
326 | === modified file 'src/core/tests/test_wsgi.py' |
327 | --- src/core/tests/test_wsgi.py 2015-10-19 09:34:53 +0000 |
328 | +++ src/core/tests/test_wsgi.py 2015-10-27 08:00:26 +0000 |
329 | @@ -76,7 +76,10 @@ |
330 | oops_path = self.oops_dir + '/' + todays_oops_path + '/' + oops_list[0] |
331 | with open(oops_path, "r") as oops_file: |
332 | oops_file_contents = oops_file.read() |
333 | - return bson.decode_all(oops_file_contents) |
334 | + result = bson.decode_document(oops_file_contents, 0) |
335 | + if len(result) == 2 and isinstance(result[1], dict): |
336 | + return result[1] |
337 | + raise ValueError('oops file content is not correct') |
338 | |
339 | def test_oopsid_in_500_response(self): |
340 | # The rendered response for a 500 exception includes the oops id. |
341 | @@ -88,7 +91,7 @@ |
342 | # A request resulting in a 500 writes the oops to disk. |
343 | self.request_500_exception() |
344 | |
345 | - oops = self.read_oops_from_disk()[0] |
346 | + oops = self.read_oops_from_disk() |
347 | self.assertEqual(oops.get('type'), 'ZeroDivisionError') |
348 | self.assertIn('tb_text', oops) |
349 | self.assertTrue(oops['tb_text'].startswith(' File ')) |
350 | @@ -97,8 +100,7 @@ |
351 | self.environ['PATH_INFO'] = '/reviews/delay/0.11/' |
352 | with override_settings(SOFT_TIMEOUT_MILLISECONDS=1): |
353 | self.request_wsgi_response(self.environ) |
354 | - |
355 | - oops_content = self.read_oops_from_disk()[0] |
356 | + oops_content = self.read_oops_from_disk() |
357 | self.assertEqual('SoftRequestTimeout', oops_content['type']) |
358 | expected = u'Start_response over timeout 1.' |
359 | self.assertEqual(expected, oops_content['value']) |
360 | |
361 | === modified file 'tarmac_verify.sh' |
362 | --- tarmac_verify.sh 2014-11-04 14:09:24 +0000 |
363 | +++ tarmac_verify.sh 2015-10-27 08:00:26 +0000 |
364 | @@ -1,9 +1,7 @@ |
365 | -#!/bin/sh |
366 | +#! /bin/bash |
367 | |
368 | set -e |
369 | set -u |
370 | |
371 | - |
372 | make clean-virtualenv |
373 | -make bootstrap |
374 | -fab test:-v,2,--failfast |
375 | +make test |
great stuff, added a few comments.