Merge lp:~xnox/phablet-tools/py2-3 into lp:phablet-tools
- py2-3
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Sergio Schvezov |
Approved revision: | 248 |
Merged at revision: | 242 |
Proposed branch: | lp:~xnox/phablet-tools/py2-3 |
Merge into: | lp:phablet-tools |
Diff against target: |
125 lines (+40/-11) 2 files modified
phablet-click-test-setup (+31/-10) phablet-test-run (+9/-1) |
To merge this branch: | bzr merge lp:~xnox/phablet-tools/py2-3 |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Martin Pitt | Approve | ||
Andy Doan (community) | Approve | ||
Sergio Schvezov | Approve | ||
PS Jenkins bot | continuous-integration | Approve | |
Nicholas Skaggs | Pending | ||
Review via email: mp+205608@code.launchpad.net |
Commit message
Support python2 and python3 based autopilot tests.
Description of the change
Support python2 and python3 based autopilot tests.
- 241. By Dimitri John Ledkov
-
correct pythonpath
Dimitri John Ledkov (xnox) wrote : | # |
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:241
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
Dimitri John Ledkov (xnox) wrote : | # |
This branch should land ahead of https:/
Sergio Schvezov (sergiusens) wrote : | # |
@balloons can you make the rule xnox mentions something true for the core apps? Or provide a list of py3 ready clicks.
Sergio Schvezov (sergiusens) wrote : | # |
60 + entry['
the autopilot extension I talked about was not part of x-source but part of a new node at the same lever of x-source called
"x-test": {
"autopilot": "ubuntu_
}
so we can eventually just run the tests for a given package, whatever those tests are; if you want to change anything, we can; adding the autopilot-dir to x-source seems reasonable to me as well.
Andy Doan (doanac) wrote : | # |
I noticed this requires "fromdos" which isn't installed by default for me
- 242. By Dimitri John Ledkov
-
Add fromdos dependency.
- 243. By Dimitri John Ledkov
-
Correct click manifest key
Dimitri John Ledkov (xnox) wrote : | # |
Addressed all review comments.
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:243
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
Sergio Schvezov (sergiusens) wrote : | # |
69 + if entry['x-test'] and 'autopilot' in entry['x-test']:
70 + get_bzr_
71 + entry['
72 + entry['
73 + test_dir)
So here it would be called like:
def get_bzr_
)
74 + else:
75 + get_bzr_
76 + entry['
77 + 'tests/autopilot',
78 + test_dir,
79 + py2_subdir)
and here
def get_bzr_
)
Seems that in
47 +def get_bzr_
sub_dir needs a None or '' default
Second
test_dir=
and
test_dir=
I would just keep 'tests/autopilot' for both or add some prior check that applies to both to look for '[x-source'
- 244. By Dimitri John Ledkov
-
Simplify get_bzr_tests call.
- 245. By Dimitri John Ledkov
-
pep8
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:245
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:245
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 246. By Dimitri John Ledkov
-
Pick correct python ahead of execution
- 247. By Dimitri John Ledkov
-
Drop bogus depends
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:247
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
- 248. By Dimitri John Ledkov
-
Add python3 modules
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:248
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
PS Jenkins bot (ps-jenkins) wrote : | # |
PASSED: Continuous integration, rev:248
http://
Executed test runs:
SUCCESS: http://
SUCCESS: http://
SUCCESS: http://
Click here to trigger a rebuild:
http://
Sergio Schvezov (sergiusens) : | # |
Andy Doan (doanac) : | # |
Martin Pitt (pitti) wrote : | # |
Very nice, thanks Dimitri!
- 249. By Dimitri John Ledkov
-
Merge trunk
Dimitri John Ledkov (xnox) wrote : | # |
With following two packages:
com.ubuntu.calendar 205 or better
com.ubuntu.music 370 or better
We can land this branch without regressing any test-results, and since unity8/
Please allocate a silo / start landing this branch.
Preview Diff
1 | === modified file 'phablet-click-test-setup' | |||
2 | --- phablet-click-test-setup 2013-10-21 18:14:51 +0000 | |||
3 | +++ phablet-click-test-setup 2014-03-06 13:08:36 +0000 | |||
4 | @@ -58,13 +58,15 @@ | |||
5 | 58 | 58 | ||
6 | 59 | lp = None | 59 | lp = None |
7 | 60 | 60 | ||
8 | 61 | py2_subdir = 'legacy-py2' | ||
9 | 62 | |||
10 | 61 | basic_packages = ( | 63 | basic_packages = ( |
11 | 62 | {'source': 'unity8', 'binary': 'unity8'}, | 64 | {'source': 'unity8', 'binary': 'unity8'}, |
12 | 63 | {'source': 'ubuntu-ui-toolkit', | 65 | {'source': 'ubuntu-ui-toolkit', |
13 | 64 | 'binary': 'qtdeclarative5-ubuntu-ui-toolkit-plugin'}) | 66 | 'binary': 'qtdeclarative5-ubuntu-ui-toolkit-plugin'}) |
14 | 65 | 67 | ||
15 | 66 | python_packages = ( | 68 | python_packages = ( |
17 | 67 | 'python-mock', 'python-dateutil' | 69 | 'python-mock', 'python-dateutil', 'python3-dateutil' |
18 | 68 | ) | 70 | ) |
19 | 69 | 71 | ||
20 | 70 | 72 | ||
21 | @@ -111,6 +113,7 @@ | |||
22 | 111 | def get_python_binary_package(package, target_dir): | 113 | def get_python_binary_package(package, target_dir): |
23 | 112 | tmp_dir = tempfile.mkdtemp() | 114 | tmp_dir = tempfile.mkdtemp() |
24 | 113 | atexit.register(cleanup, tmp_dir) | 115 | atexit.register(cleanup, tmp_dir) |
25 | 116 | python2_target_dir = os.path.join(target_dir, py2_subdir) | ||
26 | 114 | print('Fetching %s - into %s' % (package, tmp_dir)) | 117 | print('Fetching %s - into %s' % (package, tmp_dir)) |
27 | 115 | bpph = lp.archive.getPublishedBinaries( | 118 | bpph = lp.archive.getPublishedBinaries( |
28 | 116 | binary_name=package, | 119 | binary_name=package, |
29 | @@ -134,12 +137,22 @@ | |||
30 | 134 | 'usr/lib/python2.7/dist-packages') | 137 | 'usr/lib/python2.7/dist-packages') |
31 | 135 | python_modules_dir = pyshared_path if os.path.exists(pyshared_path) \ | 138 | python_modules_dir = pyshared_path if os.path.exists(pyshared_path) \ |
32 | 136 | else python2_7_path | 139 | else python2_7_path |
39 | 137 | for f in os.listdir(python_modules_dir): | 140 | if os.path.exists(python_modules_dir): |
40 | 138 | source = path.join(python_modules_dir, f) | 141 | for f in os.listdir(python_modules_dir): |
41 | 139 | if path.islink(source): | 142 | source = path.join(python_modules_dir, f) |
42 | 140 | source = path.join(python_modules_dir, os.readlink(source)) | 143 | if path.islink(source): |
43 | 141 | print('Moving %s to %s' % (source, target_dir)) | 144 | source = path.join(python_modules_dir, os.readlink(source)) |
44 | 142 | shutil.move(source, target_dir) | 145 | print('Moving %s to %s' % (source, python2_target_dir)) |
45 | 146 | shutil.move(source, python2_target_dir) | ||
46 | 147 | python3_modules_dir = os.path.join(extract_dir, | ||
47 | 148 | 'usr/lib/python3/dist-packages') | ||
48 | 149 | if os.path.exists(python3_modules_dir): | ||
49 | 150 | for f in os.listdir(python3_modules_dir): | ||
50 | 151 | source = path.join(python3_modules_dir, f) | ||
51 | 152 | if path.islink(source): | ||
52 | 153 | source = path.join(python3_modules_dir, os.readlink(source)) | ||
53 | 154 | print('Moving %s to %s' % (source, target_dir)) | ||
54 | 155 | shutil.move(source, target_dir) | ||
55 | 143 | 156 | ||
56 | 144 | 157 | ||
57 | 145 | def get_source_package_tests(package, version, target_dir): | 158 | def get_source_package_tests(package, version, target_dir): |
58 | @@ -163,15 +176,17 @@ | |||
59 | 163 | shutil.move(test_dir, target_dir) | 176 | shutil.move(test_dir, target_dir) |
60 | 164 | 177 | ||
61 | 165 | 178 | ||
63 | 166 | def get_bzr_tests(branch, revision, target_dir): | 179 | def get_bzr_tests(branch, revision, autopilot_dir, target_dir, sub_dir): |
64 | 167 | tmp_dir = tempfile.mkdtemp() | 180 | tmp_dir = tempfile.mkdtemp() |
65 | 168 | atexit.register(cleanup, tmp_dir) | 181 | atexit.register(cleanup, tmp_dir) |
66 | 169 | print('Checking out %s to %s' % (branch, path.join(tmp_dir, 'work'))) | 182 | print('Checking out %s to %s' % (branch, path.join(tmp_dir, 'work'))) |
67 | 170 | check_call(['bzr', 'checkout', '--lightweight', branch, '-r', revision, | 183 | check_call(['bzr', 'checkout', '--lightweight', branch, '-r', revision, |
68 | 171 | 'work'], cwd=tmp_dir) | 184 | 'work'], cwd=tmp_dir) |
70 | 172 | test_base_dir = path.join(tmp_dir, 'work', 'tests', 'autopilot') | 185 | test_base_dir = path.join(tmp_dir, 'work', autopilot_dir) |
71 | 173 | test_dirs = filter((lambda x: path.isdir(path.join(test_base_dir, x))), | 186 | test_dirs = filter((lambda x: path.isdir(path.join(test_base_dir, x))), |
72 | 174 | os.listdir(test_base_dir)) | 187 | os.listdir(test_base_dir)) |
73 | 188 | if sub_dir: | ||
74 | 189 | target_dir = os.path.join(target_dir, sub_dir) | ||
75 | 175 | for test_dir in test_dirs: | 190 | for test_dir in test_dirs: |
76 | 176 | test_dir = path.join(test_base_dir, test_dir) | 191 | test_dir = path.join(test_base_dir, test_dir) |
77 | 177 | print('Moving %s to %s' % (test_dir, target_dir)) | 192 | print('Moving %s to %s' % (test_dir, target_dir)) |
78 | @@ -194,9 +209,14 @@ | |||
79 | 194 | print('Only keeping entries with x-source') | 209 | print('Only keeping entries with x-source') |
80 | 195 | manifest = [entry for entry in manifest if 'x-source' in entry] | 210 | manifest = [entry for entry in manifest if 'x-source' in entry] |
81 | 196 | for entry in manifest: | 211 | for entry in manifest: |
82 | 212 | subdir = py2_subdir | ||
83 | 213 | if 'x-test' in entry and 'autopilot' in entry['x-test']: | ||
84 | 214 | subdir = None | ||
85 | 197 | get_bzr_tests(entry['x-source']['vcs-bzr'], | 215 | get_bzr_tests(entry['x-source']['vcs-bzr'], |
86 | 198 | entry['x-source']['vcs-bzr-revno'], | 216 | entry['x-source']['vcs-bzr-revno'], |
88 | 199 | test_dir) | 217 | 'tests/autopilot', |
89 | 218 | test_dir, | ||
90 | 219 | subdir) | ||
91 | 200 | 220 | ||
92 | 201 | 221 | ||
93 | 202 | def main(): | 222 | def main(): |
94 | @@ -208,6 +228,7 @@ | |||
95 | 208 | series = adb.shell('lsb_release -c -s').strip() | 228 | series = adb.shell('lsb_release -c -s').strip() |
96 | 209 | lp = LP(series, arch) | 229 | lp = LP(series, arch) |
97 | 210 | test_dir = tempfile.mkdtemp() | 230 | test_dir = tempfile.mkdtemp() |
98 | 231 | os.mkdir(os.path.join(test_dir, py2_subdir)) | ||
99 | 211 | atexit.register(cleanup, test_dir) | 232 | atexit.register(cleanup, test_dir) |
100 | 212 | fetch_test_base(adb, test_dir) | 233 | fetch_test_base(adb, test_dir) |
101 | 213 | fetch_click_tests(adb, test_dir, args.user, args.click) | 234 | fetch_click_tests(adb, test_dir, args.user, args.click) |
102 | 214 | 235 | ||
103 | === modified file 'phablet-test-run' | |||
104 | --- phablet-test-run 2014-01-20 19:59:53 +0000 | |||
105 | +++ phablet-test-run 2014-03-06 13:08:36 +0000 | |||
106 | @@ -100,10 +100,18 @@ | |||
107 | 100 | 100 | ||
108 | 101 | run_autopilot_test() { | 101 | run_autopilot_test() { |
109 | 102 | exec_with_adb chmod 666 /dev/uinput | 102 | exec_with_adb chmod 666 /dev/uinput |
110 | 103 | |||
111 | 104 | # First let's try python3, without python2 path | ||
112 | 105 | defaultpy="/usr/bin/python3" | ||
113 | 106 | import_error=$(adb shell "cd /home/phablet/autopilot; $defaultpy -c 'import $TESTSUITE'") | ||
114 | 107 | if [ -n "$import_error" ]; then | ||
115 | 108 | defaultpy="PYTHONPATH=/home/phablet/autopilot/legacy-py2:/home/phablet/autopilot /usr/bin/python" | ||
116 | 109 | fi | ||
117 | 110 | |||
118 | 103 | # adb shell always returns 0, so we have to do some hackery to get the | 111 | # adb shell always returns 0, so we have to do some hackery to get the |
119 | 104 | # actual RC from the test | 112 | # actual RC from the test |
120 | 105 | { | 113 | { |
122 | 106 | apbase="sudo -iu $USER bash -ic \"cd /home/phablet/autopilot; autopilot run $VERBOSE" | 114 | apbase="sudo -iu $USER bash -ic \"cd /home/phablet/autopilot; $defaultpy -m autopilot.run run $VERBOSE" |
123 | 107 | if [ "$RESULTDIR" ]; then | 115 | if [ "$RESULTDIR" ]; then |
124 | 108 | adb shell "$apbase -o /tmp/test_results.xml -f xml $TESTSUITE\"; echo ADB_RC=\$?" | 116 | adb shell "$apbase -o /tmp/test_results.xml -f xml $TESTSUITE\"; echo ADB_RC=\$?" |
125 | 109 | echo "***** Test summary *****" | 117 | echo "***** Test summary *****" |
/home/autopilot/ can have python3 modules, or python3 & 2 bilingual modules (e.g. uitoolkit, unity8 emulators etc. are expected to be bilingual)
/home/autopilot /legacy- py2 can only have python2 modules.
I've modified get_bzr_tests to fetch into python2 location is autopilot_dir is not defined.
If autopilot_dir is defined, it's fetched into python3 location.
Ditto grabbing tests from binary packages, although that doesn't grab python3 tests yet.
The phablet-test-run portion, modifies how autopilot is launched to carefully set PYTHONPATH for python2-only (this is achieved by passing -E (ignore pythonpath) to python3, which is dropped if autopilot re-execs under python2)