Merge lp:~mikemc/ubuntuone-storage-protocol/fix-run-tests into lp:ubuntuone-storage-protocol

[Mike McCracken]

- Make tests run on darwin and windows. (LP: #1033823)

- Make tests run on darwin and windows. (LP: #1033823)

With this change, the tests pass on darwin and windows.
We only use the python-native version of protobuf on the client platforms, so we skip the cpp version in the tests.

All tests still pass on linux with this change.

This also fixes a path assumption in get_certificates in test_context.py that failed if TRIAL_TEMP_DIR is set to e.g. /tmp or C:/temp-test.

[Alejandro J. Cura (alecu)]

Looks good

review: Approve

[Eric Casteleijn (thisfred)]

+1

[Eric Casteleijn (thisfred)]

review: Approve

[Alejandro J. Cura (alecu)]

After the changes, it looks good, and all tests pass on the three platforms.

review: Approve

[dobey (dobey)]

10 +if [ "$SYSNAME" == "Darwin" ]; then
11 + u1trial="python $u1trial"
12 + u1lint="python $u1lint"
13 +else
14 + u1trial=u1trial
15 + u1lint="/usr/bin/env USE_PYFLAKES=1 u1lint"
16 +fi
17 +

I think we need to do 'export USE_PYFLAKES=1' outside the if/else here, as we really need to use pyflakes on all platforms here. It seems you also need a SET USE_PYFLAKES=1 in the run-tests.bat here.

I'm surprised that the u1lint passed on OSX or Windows in this branch. We are using pyflakes due to problems with the way pylint fails to handle some of the class structure of twisted.

review: Needs Fixing

[lp:~mikemc/ubuntuone-storage-protocol/fix-run-tests]

157. By Mike McCracken on 2012-08-10

Remove env vars due to formerly broken buildout

[Mike McCracken (mikemc)]

Changed to use pyflakes on darwin, and remove references to env vars set to work around broken buildout, which is now fixed in https://code.launchpad.net/~mikemc/ubuntuone-windows-installer/improve-buildout/+merge/119195

[lp:~mikemc/ubuntuone-storage-protocol/fix-run-tests]

158. By Mike McCracken on 2012-08-10

Add back SYSNAME so we skip cpp tests

[Mike McCracken (mikemc)]

with rev 158 , tests pass on darwin with no pyflakes complaints.

[lp:~mikemc/ubuntuone-storage-protocol/fix-run-tests]

159. By Mike McCracken on 2012-08-15

use pyflakes on windows

160. By Mike McCracken on 2012-08-15

exclude generated protobuf code from pep8 check

[Mike McCracken (mikemc)]

with rev 159 windows uses pyflakes , and with 160 pep8 is happy.

[dobey (dobey)]

review: Approve

=== modified file 'run-tests'
--- run-tests	2012-04-20 23:21:10 +0000
+++ run-tests	2012-08-15 18:10:24 +0000
@@ -15,13 +15,18 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 set -e
 
+SYSNAME=`uname -s`
+
 /usr/bin/env python setup.py build
 # run the tests with pure python protobuf
 PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python u1trial tests
-# and with the cpp extension
-PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp u1trial tests
+if [ "$SYSNAME" != "Darwin" ]; then
+    # and with the cpp extension, for server (linux) only:
+    PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp u1trial tests
+fi
+
 USE_PYFLAKES=1 u1lint
 /usr/bin/env python setup.py clean
 rm -rf build
-pep8 --repeat .
+pep8 --repeat --exclude="*_pb2.py" .
 rm -rf _trial_temp

=== added file 'run-tests.bat'
--- run-tests.bat	1970-01-01 00:00:00 +0000
+++ run-tests.bat	2012-08-15 18:10:24 +0000
@@ -0,0 +1,82 @@
+:: Copyright 2012 Canonical Ltd.
+::
+:: This program is free software: you can redistribute it and/or modify it
+:: under the terms of the GNU General Public License version 3, as published
+:: by the Free Software Foundation.
+::
+:: This program is distributed in the hope that it will be useful, but
+:: WITHOUT ANY WARRANTY; without even the implied warranties of
+:: MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+:: PURPOSE.  See the GNU General Public License for more details.
+::
+:: You should have received a copy of the GNU General Public License along
+:: with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+@ECHO off
+
+SET MODULE="tests"
+SET PYTHONEXEPATH=""
+SET IGNORE_PATHS="ubuntuone\controlpanel\dbustests"
+SET IGNORE_MODULES="test_linux.py, test_darwin.py"
+SET PYTHONPATH=%PYTHONPATH%;..\ubuntuone-dev-tools\;..\dirspec;.
+
+ECHO Checking for Python on the path
+:: Look for Python from buildout
+FOR %%A in (python.exe) do (SET PYTHONEXEPATH=%%~$PATH:A)
+FOR %%B in (u1trial) do (SET TRIALPATH=%%~$PATH:B)
+FOR %%C in (u1lint) do (SET LINTPATH=%%~$PATH:C)
+FOR %%D in (pep8.exe) do (SET PEP8PATH=%%~$PATH:D)
+
+IF NOT "%PYTHONEXEPATH%" == "" GOTO :PYTHONPRESENT
+ECHO Please ensure you have python installed
+GOTO :END
+
+:PYTHONPRESENT
+
+:: throw the first parameter away if is /skip-lint,
+:: the way we do this is to ensure that /skip-lint
+:: is the first parameter and copy all the rest in a loop
+:: the main reason for that is that %* is not affected
+:: by SHIFT, that is, it allways have all passed parameters
+
+SET PARAMS=%*
+SET SKIPLINT=0
+IF "%1" == "/skip-lint" (
+    SET SKIPLINT=1
+    GOTO :CLEANPARAMS
+)ELSE (
+    GOTO :CONTINUEBATCH) 
+:CLEANPARAMS
+
+SHIFT
+SET PARAMS=%1
+:GETREST
+SHIFT
+if [%1]==[] (
+    GOTO CONTINUEBATCH)
+SET PARAMS=%PARAMS% %1
+GOTO GETREST
+:CONTINUEBATCH
+
+
+"%PYTHONEXEPATH%" setup.py build
+ECHO Running tests
+:: execute the tests with a number of ignored linux only modules
+"%PYTHONEXEPATH%" "%TRIALPATH%" -p %IGNORE_PATHS% -i %IGNORE_MODULES% %PARAMS% %MODULE%
+
+:: Clean the build from the setup.py
+ECHO Cleaning the generated code
+"%PYTHONEXEPATH%" setup.py clean
+
+IF %SKIPLINT% == 1 GOTO :CLEAN
+ECHO Performing style checks...
+SET USE_PYFLAKES=1
+"%PYTHONEXEPATH%" "%LINTPATH%" "%MODULE%"
+"%PEP8PATH%" --exclude ".svn,CVS,.bzr,.hg,.git,*_ui.py,*_rc.py,*_pb2.py" --repeat .
+
+:CLEAN
+:: Delete the temp folders
+IF "%TRIAL_TEMP_DIR%" == "" GOTO :TRIALTEMPEXISTS
+IF EXIST _trial_temp RMDIR /s /q _trial_temp
+:TRIALTEMPEXISTS
+IF EXIST "%TRIAL_TEMP_DIR%" RMDIR /s /q "%TRIAL_TEMP_DIR%"

=== modified file 'tests/test_context.py'
--- tests/test_context.py	2012-06-11 17:26:14 +0000
+++ tests/test_context.py	2012-08-15 18:10:24 +0000
@@ -123,13 +123,15 @@
 
     def get_certificates(self):
         """Get the uninstalled certificates, for testing."""
-        ca_1 = ssl.Certificate.loadPEM(file(os.path.abspath(os.path.join(
-                        os.pardir, os.pardir, 'data',
-                        'UbuntuOne-Go_Daddy_Class_2_CA.pem')), 'r').read())
-        ca_2 = ssl.Certificate.loadPEM(file(os.path.abspath(os.path.join(
-                        os.pardir, os.pardir, 'data',
-                        'UbuntuOne-Go_Daddy_CA.pem')), 'r').read())
-        return [ca_1.original, ca_2.original]
+        certs = []
+        data_dir = os.path.join(os.path.dirname(__file__), os.pardir, 'data')
+        for certname in ['UbuntuOne-Go_Daddy_Class_2_CA.pem',
+                         'UbuntuOne-Go_Daddy_CA.pem']:
+            cert_path = os.path.abspath(os.path.join(data_dir, certname))
+            c = ssl.Certificate.loadPEM(file(cert_path, 'r').read())
+            certs.append(c.original)
+
+        return certs
 
     @defer.inlineCallbacks
     def verify_context(self, server_context, client_context):