Merge lp:~le-chi-thu/linaro-ci/add-testplan-support into lp:linaro-ci
- add-testplan-support
- Merge into lci-build-tools
Status: | Merged |
---|---|
Merged at revision: | 62 |
Proposed branch: | lp:~le-chi-thu/linaro-ci/add-testplan-support |
Merge into: | lp:linaro-ci |
Diff against target: |
129 lines (+43/-34) 1 file modified
get_latest_ci_hwpack (+43/-34) |
To merge this branch: | bzr merge lp:~le-chi-thu/linaro-ci/add-testplan-support |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Deepti B. Kalakeri (community) | Approve | ||
Le Chi Thu (community) | Needs Resubmitting | ||
Review via email: mp+104743@code.launchpad.net |
Commit message
Description of the change
Add support to define test plan for lava.
the format is : LAVA_TEST_
options and timeout is optional.
Examples:
LAVA_TEST_
LAVA_TEST_
if the LAVA_TEST_PLAN is not define, the default will be "ltp,pwrmgmt". It mean the current jobs do not need to update.
Deepti B. Kalakeri (deeptik) wrote : | # |
Deepti B. Kalakeri (deeptik) : | # |
Le Chi Thu (le-chi-thu) wrote : | # |
ltp take options - see the $OPTIONS macro in the test definition file.
RUNSTEPS = ['cd build && sudo ./runltp $(OPTIONS)']
http://
Now the json file is correct generate.
- 62. By Le Chi Thu <email address hidden> <email address hidden>
-
Update after review 1
Deepti B. Kalakeri (deeptik) wrote : | # |
Thanks for addressing the review, I have few more.
1) Please retain the boot_linaro_image stuff as it was before you made the
changes, since this block does not change, I would like to have it in the
dict itself.
2) I did not see the 80 columns rule being followed, for ex line no: 99
spans to 100 columns.
3) I guess you did not see the following comments in the previous review
(i)
"
>
re.compile(
>
The regex above should not match first part(:100) of the following
LAVA_TEST_
The above regex matches :100 as testcase which is invalid.
"
(ii)
> +
> + if r.group("options"):
> + lava_test_
> r.group("options")
> +
> + template[
> +
>
It would be good if combined both the for loops into one for loop.
could you please address all of them and resubmit.
On Tue, May 8, 2012 at 5:53 PM, Le Chi Thu <email address hidden> wrote:
> Review: Resubmit
>
> ltp take options - see the $OPTIONS macro in the test definition file.
> RUNSTEPS = ['cd build && sudo ./runltp $(OPTIONS)']
>
> http://
>
> Now the json file is correct generate.
> --
>
> https:/
> You are reviewing the proposed merge of
> lp:~le-chi-thu/linaro-ci/add-testplan-support into lp:linaro-ci.
>
--
Thanks and Regards,
Deepti
Infrastructure Team Member, Linaro Platform Teams
Linaro.org | Open source software for ARM SoCs
Follow Linaro: http://
http://
Le Chi Thu (le-chi-thu) wrote : | # |
Fixed according to comments.
- 63. By Le Chi Thu <email address hidden> <email address hidden>
-
Updated after review 2
Deepti B. Kalakeri (deeptik) wrote : | # |
Your changes fail when we pass LAVA_TEST_
python ./get_latest_
Traceback (most recent call last):
File "./get_
lava_
AttributeError: 'NoneType' object has no attribute 'group'
The following changes fix the above problem and includes other changes like aligning to 80 columns rule.
=== modified file 'get_latest_
--- get_latest_
+++ get_latest_
@@ -16,7 +16,7 @@
job_name = os.getenv(
-testplan = os.getenv(
+testplan = os.getenv(
template = {
"timeout": 20000,
@@ -66,50 +66,48 @@
metadata[
metadata[
metadata[
-
-
rootfsdate, rootfsbuild = rootfs_
metadata[
metadata[
metadata[
-
template[
testcases = testplan.split(',')
-
-regex = re.compile(
-
-# lava_test_install command
-
+testcases = [tc.strip() for tc in testplan.
+regex = re.compile(
+ "(:(?P<
+
+# Dict for lava_test_install
lava_test_
lava_test_
lava_test_
-# where in the template dict we shall insert test commands
+# The positions in the template dict to insert test install/run commands
install_cmd_pos = 1
run_cmd_pos = 2
for testcase in testcases:
r = regex.search(
- lava_test_
-
- # append lava_test_run commands
- timeout = 4800
- if r.group('timeout'):
- timeout = r.group('timeout')
-
- lava_test_
- lava_test_
- lava_test_
- "timeout" : timeout }
-
- if r.group("options"):
- lava_test_
-
- template[
- run_cmd_pos = run_cmd_pos + 1
+ if r.group("testcase") != None:
+ lava_test_
+ append(
+
+ # append lava_test_run commands
+ timeout = 4800
+ if r.group('timeout'):
+ timeout = r.group('timeout')
+
+ lava_test_
+ lava_test_
+ lava_test_
+ ...
- 64. By Le Chi Thu <email address hidden> <email address hidden>
-
Updated after review 3
Le Chi Thu (le-chi-thu) : | # |
Deepti B. Kalakeri (deeptik) wrote : | # |
The changes looks ok. Thanks a lot.
Deepti B. Kalakeri (deeptik) : | # |
Preview Diff
1 | === modified file 'get_latest_ci_hwpack' | |||
2 | --- get_latest_ci_hwpack 2012-04-04 12:14:43 +0000 | |||
3 | +++ get_latest_ci_hwpack 2012-05-14 10:48:21 +0000 | |||
4 | @@ -5,6 +5,7 @@ | |||
5 | 5 | 5 | ||
6 | 6 | Usage: ./gentemplate <target> <hwpack_type> <image_type> | 6 | Usage: ./gentemplate <target> <hwpack_type> <image_type> |
7 | 7 | """ | 7 | """ |
8 | 8 | import re | ||
9 | 8 | 9 | ||
10 | 9 | from find_latest import find_latest_hwpack, find_latest_rootfs, find_ci_latest | 10 | from find_latest import find_latest_hwpack, find_latest_rootfs, find_ci_latest |
11 | 10 | import json | 11 | import json |
12 | @@ -15,8 +16,7 @@ | |||
13 | 15 | job_name = os.getenv("KERNEL_JOB_NAME", "unknown") + '_' + \ | 16 | job_name = os.getenv("KERNEL_JOB_NAME", "unknown") + '_' + \ |
14 | 16 | os.getenv("HWPACK_BUILD_DATE", "unknown") + '_daily test' | 17 | os.getenv("HWPACK_BUILD_DATE", "unknown") + '_daily test' |
15 | 17 | 18 | ||
18 | 18 | 19 | testplan = os.getenv("LAVA_TEST_PLAN", "ltp, pwrmgmt") | |
17 | 19 | |||
19 | 20 | 20 | ||
20 | 21 | template = { | 21 | template = { |
21 | 22 | "timeout": 20000, | 22 | "timeout": 20000, |
22 | @@ -25,36 +25,9 @@ | |||
23 | 25 | "command": "deploy_linaro_image" | 25 | "command": "deploy_linaro_image" |
24 | 26 | }, | 26 | }, |
25 | 27 | { | 27 | { |
26 | 28 | "command": "lava_test_install", | ||
27 | 29 | "parameters": | ||
28 | 30 | { | ||
29 | 31 | "tests": [ | ||
30 | 32 | "ltp", | ||
31 | 33 | "pwrmgmt" | ||
32 | 34 | ], | ||
33 | 35 | "timeout": 4000 | ||
34 | 36 | } | ||
35 | 37 | }, | ||
36 | 38 | { | ||
37 | 39 | "command": "boot_linaro_image" | 28 | "command": "boot_linaro_image" |
38 | 40 | }, | 29 | }, |
39 | 41 | { | 30 | { |
40 | 42 | "command": "lava_test_run", | ||
41 | 43 | "parameters": | ||
42 | 44 | { | ||
43 | 45 | "test_name": "ltp", | ||
44 | 46 | "timeout": 4800 | ||
45 | 47 | } | ||
46 | 48 | }, | ||
47 | 49 | { | ||
48 | 50 | "command": "lava_test_run", | ||
49 | 51 | "parameters": | ||
50 | 52 | { | ||
51 | 53 | "test_name": "pwrmgmt", | ||
52 | 54 | "timeout": 4800 | ||
53 | 55 | } | ||
54 | 56 | }, | ||
55 | 57 | { | ||
56 | 58 | "command": "submit_results", | 31 | "command": "submit_results", |
57 | 59 | "parameters": { | 32 | "parameters": { |
58 | 60 | "stream": os.getenv("BUNDLE_STREAM_NAME", "/anonymous/plars/"), | 33 | "stream": os.getenv("BUNDLE_STREAM_NAME", "/anonymous/plars/"), |
59 | @@ -63,7 +36,7 @@ | |||
60 | 63 | } | 36 | } |
61 | 64 | ], | 37 | ], |
62 | 65 | "job_name": job_name, | 38 | "job_name": job_name, |
64 | 66 | "device_type": os.getenv("BOARD_TYPE", "unknown") | 39 | "device_type": os.getenv("BOARD_TYPE", "unknown") |
65 | 67 | } | 40 | } |
66 | 68 | 41 | ||
67 | 69 | url=os.getenv("JOB_URL", "http://snapshots.linaro.org/kernel-hwpack/") | 42 | url=os.getenv("JOB_URL", "http://snapshots.linaro.org/kernel-hwpack/") |
68 | @@ -71,7 +44,8 @@ | |||
69 | 71 | hwpack_dir_url=os.getenv("KERNEL_JOB_NAME", "unknown") | 44 | hwpack_dir_url=os.getenv("KERNEL_JOB_NAME", "unknown") |
70 | 72 | hwpack_url_path = os.path.join(url, hwpack_parent_url) | 45 | hwpack_url_path = os.path.join(url, hwpack_parent_url) |
71 | 73 | hwpack_url_path = os.path.join(hwpack_url_path, hwpack_dir_url) | 46 | hwpack_url_path = os.path.join(hwpack_url_path, hwpack_dir_url) |
73 | 74 | hwpack_url = find_ci_latest(hwpack_url_path) | 47 | |
74 | 48 | hwpack_url = find_ci_latest(hwpack_url_path) | ||
75 | 75 | rootfs_id, rootfs_url = find_latest_rootfs(imagetype) | 49 | rootfs_id, rootfs_url = find_latest_rootfs(imagetype) |
76 | 76 | 50 | ||
77 | 77 | deploy_params = { 'hwpack':hwpack_url, 'rootfs':rootfs_url } | 51 | deploy_params = { 'hwpack':hwpack_url, 'rootfs':rootfs_url } |
78 | @@ -92,13 +66,48 @@ | |||
79 | 92 | metadata['kernel.git_log_info'] = os.getenv("GIT_LOG", "unknown") | 66 | metadata['kernel.git_log_info'] = os.getenv("GIT_LOG", "unknown") |
80 | 93 | metadata['hwpack.type'] = hwpacktype | 67 | metadata['hwpack.type'] = hwpacktype |
81 | 94 | metadata['hwpack.date'] = hwpackdate | 68 | metadata['hwpack.date'] = hwpackdate |
82 | 95 | |||
83 | 96 | |||
84 | 97 | rootfsdate, rootfsbuild = rootfs_id.split('.') | 69 | rootfsdate, rootfsbuild = rootfs_id.split('.') |
85 | 98 | metadata['rootfs.type'] = imagetype | 70 | metadata['rootfs.type'] = imagetype |
86 | 99 | metadata['rootfs.date'] = rootfsdate | 71 | metadata['rootfs.date'] = rootfsdate |
87 | 100 | metadata['rootfs.build'] = rootfsbuild | 72 | metadata['rootfs.build'] = rootfsbuild |
88 | 101 | |||
89 | 102 | template['actions'][0]['metadata'] = metadata | 73 | template['actions'][0]['metadata'] = metadata |
90 | 103 | 74 | ||
91 | 75 | testcases = testplan.split(',') | ||
92 | 76 | testcases = [tc.strip() for tc in testplan.split(',')] | ||
93 | 77 | regex = re.compile("(?P<testcase>^\w+)(\((?P<options>[^\)]+)\))?"\ | ||
94 | 78 | "(:(?P<timeout>\d+))?") | ||
95 | 79 | |||
96 | 80 | # Dict for lava_test_install | ||
97 | 81 | lava_test_install_command = {} | ||
98 | 82 | lava_test_install_command["command"] = "lava_test_install" | ||
99 | 83 | lava_test_install_command["parameters"] = { "tests" : [], "timeout" : 4000 } | ||
100 | 84 | |||
101 | 85 | # The positions in the template dict to insert test install/run commands | ||
102 | 86 | install_cmd_pos = 1 | ||
103 | 87 | run_cmd_pos = 2 | ||
104 | 88 | |||
105 | 89 | for testcase in testcases: | ||
106 | 90 | r = regex.search(testcase) | ||
107 | 91 | if r.group("testcase") != None: | ||
108 | 92 | lava_test_install_command["parameters"]["tests"].\ | ||
109 | 93 | append(r.group("testcase")) | ||
110 | 94 | |||
111 | 95 | # append lava_test_run commands | ||
112 | 96 | timeout = 4800 | ||
113 | 97 | if r.group('timeout'): | ||
114 | 98 | timeout = r.group('timeout') | ||
115 | 99 | |||
116 | 100 | lava_test_run_command = {} | ||
117 | 101 | lava_test_run_command["command"] = "lava_test_run" | ||
118 | 102 | lava_test_run_command["parameters"] = {"test_name" : r.group('testcase'), | ||
119 | 103 | "timeout" : timeout } | ||
120 | 104 | |||
121 | 105 | if r.group("options"): | ||
122 | 106 | lava_test_run_command["parameters"]["test_options"] = r.group("options") | ||
123 | 107 | |||
124 | 108 | template['actions'].insert(run_cmd_pos, lava_test_run_command) | ||
125 | 109 | run_cmd_pos = run_cmd_pos + 1 | ||
126 | 110 | |||
127 | 111 | # append the lava test install command | ||
128 | 112 | template['actions'].insert(install_cmd_pos, lava_test_install_command) | ||
129 | 104 | print json.dumps(template, indent=2) | 113 | print json.dumps(template, indent=2) |
Thanks for the changes. Please see my comments inline.
On Fri, May 4, 2012 at 7:38 PM, Le Chi Thu <email address hidden> wrote:
> Le Chi Thu has proposed merging /code.launchpad .net/~le- chi-thu/ linaro- ci/add- testplan- support/ +merge/ 104743<https:/ /code.launchpad .net/%7Ele- chi-thu/ linaro- ci/add- testplan- support/ +merge/ 104743> PLAN=<testcase> (<options> ):<timeout> ,... PLAN=ltp, pwrmgmt PLAN=test_ abc(option_ abc):100
> lp:~le-chi-thu/linaro-ci/add-testplan-support into lp:linaro-ci.
>
> Requested reviews:
> Deepti B. Kalakeri (deeptik)
>
> For more details, see:
>
> https:/
>
> Add support to define test plan for lava.
>
> the format is : LAVA_TEST_
> options and timeout is optional.
>
> Examples:
> LAVA_TEST_
> LAVA_TEST_
>
Could you give me an example of the options_abc ?
>
> if the LAVA_TEST_PLAN is not define, the default will be "ltp,pwrmgmt". It
> mean the current jobs do not need to update.
>
The default test that needs to be run should as well include
boot_linaro_image. Without booting the image how will you be able to run
the tests ?
> -- /code.launchpad .net/~le- chi-thu/ linaro- ci/add- testplan- support/ +merge/ 104743<https:/ /code.launchpad .net/%7Ele- chi-thu/ linaro- ci/add- testplan- support/ +merge/ 104743> ci_hwpack' ci_hwpack 2012-04-04 12:14:43 +0000 ci_hwpack 2012-05-04 14:07:20 +0000 "KERNEL_ JOB_NAME" , "unknown") + '_' + \ "HWPACK_ BUILD_DATE" , "unknown") + '_daily test' "LAVA_TEST_ PLAN", "ltp,pwrmgmt") linaro_ image" install" ,
>
> https:/
> You are requested to review the proposed merge of
> lp:~le-chi-thu/linaro-ci/add-testplan-support into lp:linaro-ci.
>
> === modified file 'get_latest_
> --- get_latest_
> +++ get_latest_
> @@ -5,6 +5,7 @@
>
> Usage: ./gentemplate <target> <hwpack_type> <image_type>
> """
> +import re
>
> from find_latest import find_latest_hwpack, find_latest_rootfs,
> find_ci_latest
> import json
> @@ -15,8 +16,7 @@
> job_name = os.getenv(
> os.getenv(
>
> -
> -
> +testplan = os.getenv(
>
> template = {
> "timeout": 20000,
> @@ -25,36 +25,6 @@
> "command": "deploy_
> },
> {
> - "command": "lava_test_
> - "parameters":
> - {
> - "tests": [
> - "ltp",
> - "pwrmgmt"
> - ],
> - "timeout": 4000
> - }
> - },
> - {
> - "command": "boot_linaro_image"
> - },
>
This should not be deleted, as we need to boot the image and then can we
run the tests.
> - { "BUNDLE_ STREAM_ NAME", "/anonymous/ plars/" ), dir_url= os.getenv( "KERNEL_ JOB_NAME" , "unknown") join(hwpack_ url_path, hwpack_dir_url)
> - "command": "lava_test_run",
> - "parameters":
> - {
> - "test_name": "ltp",
> - "timeout": 4800
> - }
> - },
> - {
> - "command": "lava_test_run",
> - "parameters":
> - {
> - "test_name": "pwrmgmt",
> - "timeout": 4800
> - }
> - },
> - {
> "command": "submit_results",
> "parameters": {
> "stream": os.getenv(
> @@ -71,6 +41,9 @@
> hwpack_
> hwpack_url_path = os.path.join(url, hwpack_parent_url)
> hwpack_url_path = os.path.
> +
> +
> +
>
Please delete the blank lines, you can leave a single bla...