Merge lp:~le-chi-thu/lava-dispatcher/improve-logging into lp:lava-dispatcher

Proposed by Le Chi Thu
Status: Merged
Merged at revision: 119
Proposed branch: lp:~le-chi-thu/lava-dispatcher/improve-logging
Merge into: lp:lava-dispatcher
Diff against target: 531 lines (+113/-25) (has conflicts)
14 files modified
.bzrignore (+1/-0)
lava-dispatch (+18/-1)
lava_dispatcher/actions/android_0xbench.py (+3/-2)
lava_dispatcher/actions/android_basic.py (+2/-1)
lava_dispatcher/actions/android_deploy.py (+8/-6)
lava_dispatcher/actions/boot_control.py (+4/-0)
lava_dispatcher/actions/deploy.py (+24/-2)
lava_dispatcher/actions/launch_control.py (+31/-5)
lava_dispatcher/actions/lava-test.py (+4/-0)
lava_dispatcher/android_client.py (+6/-4)
lava_dispatcher/client.py (+4/-3)
lava_dispatcher/config.py (+2/-1)
lava_dispatcher/default-config/lava-dispatcher/README (+4/-0)
lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf (+2/-0)
Text conflict in lava_dispatcher/actions/deploy.py
Text conflict in lava_dispatcher/actions/launch_control.py
To merge this branch: bzr merge lp:~le-chi-thu/lava-dispatcher/improve-logging
Reviewer Review Type Date Requested Status
Paul Larson (community) Approve
Le Chi Thu Pending
Review via email: mp+76548@code.launchpad.net

This proposal supersedes a proposal from 2011-08-10.

Description of the change

All print outs from lava-dispatcher are prefix with <<LAVA_DISPATCHER>>. More informations are printed out to easier follow the flow of dispatcher.

Replaced the python configuration file with one option in lava-dispatcher.conf file.

# Enable python logging level to debug. 0 - logging level is INFO, 1- logging level is DEBUG
LOGGING_LEVEL_DEBUG = 0

To post a comment you must log in.
Revision history for this message
Spring Zhang (qzhang) wrote : Posted in a previous version of this proposal

9 +def dispatcher_print(msg):
10 + print "\n<<LAVA_DISPATCHER>> %s"%msg
do you consider if some message will redirect to sys.stderr? Currently there is no such statement.

There are also android_0xbench.py and android_basic.py existing print statements if you want to make all actions use the function.

Revision history for this message
Paul Larson (pwlars) wrote : Posted in a previous version of this proposal

Good idea to improve the logging in the dispatcher, but if we are going to go to the trouble of doing, let's really improve it and use python.logging.

review: Needs Fixing
Revision history for this message
Spring Zhang (qzhang) wrote : Posted in a previous version of this proposal

Actually, I have a bp on: https://blueprints.launchpad.net/lava-dispatcher/+spec/linaro-platforms-o-dispatcher-logging, I'll start after milestone 11.08 soon. I will get your prefix in it too.

Revision history for this message
Le Chi Thu (le-chi-thu) wrote : Posted in a previous version of this proposal

Spring: Let abandon this MP and go ahead improve the logging with python
logging lib.

On 12 August 2011 04:28, Spring Zhang <email address hidden> wrote:

> Actually, I have a bp on:
> https://blueprints.launchpad.net/lava-dispatcher/+spec/linaro-platforms-o-dispatcher-logging,
> I'll start after milestone 11.08 soon. I will get your prefix in it too.
> --
>
> https://code.launchpad.net/~le-chi-thu/lava-dispatcher/improve-logging/+merge/71120
> You are the owner of lp:~le-chi-thu/lava-dispatcher/improve-logging.
>

Revision history for this message
Le Chi Thu (le-chi-thu) wrote : Posted in a previous version of this proposal

Hi

I replaced the print statements with python standard logging library. Please review.

BR

/Chi Thu

review: Needs Resubmitting
Revision history for this message
Paul Larson (pwlars) wrote : Posted in a previous version of this proposal

8 +./.idea
What's this for?

31 +# load logging configuration file
32 +for directory in load_config_paths('lava-dispatcher'):
33 + path = os.path.join(directory, 'logging.conf')
34 + if os.path.exists(path):
35 + logging.config.fileConfig(path)
36 + break
You should be able to just use get_config() for this (see lava_dispatcher/config.py)

477 +[formatters]
478 +keys: detailed,simple
479 +
480 +[handlers]
481 +keys: console
482 +
483 +[loggers]
484 +keys: root
485 +
486 +[formatter_simple]
487 +format:<LAVA_DISPATCHER>%(asctime)s %(levelname)s: %(message)s
488 +datefmt:%Y-%m-%d %I:%M:%S %p
489 +
490 +[formatter_detailed]
491 +format:<LAVA_DISPATCHER>%(asctime)s %(levelname)s %(module)s:%(lineno)d: %(message)s
492 +datefmt:%Y-%m-%d %I:%M:%S %p
493 +
494 +[handler_console]
495 +class: StreamHandler
496 +args: []
497 +formatter: simple
498 +
499 +[logger_root]
500 +level: DEBUG
501 +name: LAVA_DISPATCHER
502 +handlers: console
Perhaps I'm just tired and missing it, but I don't see where you are actually using all these options. Also, I think it's overkill. I don't think we should really need to specify anything other than the log level, and I'm thinking perhaps it should be set to some default value in the main dispatcher config file, but be something we can override with an option passed to the job.

Otherwise, looking much better!

review: Needs Fixing
Revision history for this message
Le Chi Thu (le-chi-thu) wrote : Posted in a previous version of this proposal

>8 +./.idea
>What's this for?

.idea is created by the IDE (PyCharm) I use containing PyCharm project
files..

> You should be able to just use get_config() for this (see
> lava_dispatcher/config.py)
>
> I saw and understand the get_config method in config.py but It does not fit
the need of logging.

I need to pass the url of the configuration file to the python logging
library. So I use the load_config_paths to search for that file.

get_config get keys and values inside the configuration file.

477 +[formatters]
> 478 +keys: detailed,simple
>
> 502 +handlers: console
> Perhaps I'm just tired and missing it, but I don't see where you are
> actually using all these options. Also, I think it's overkill. I don't
> think we should really need to specify anything other than the log level,
> and I'm thinking perhaps it should be set to some default value in the main
> dispatcher config file, but be something we can override with an option
> passed to the job.
>

This is how python logging configuration commonl done. This file is a simple
template and can be override by copy the file to users lava-dispatcher
configuration directories and modify the content.

See example in http://antonym.org/2005/03/a-real-python-logging-example.html

> Otherwise, looking much better!
> --
>
> https://code.launchpad.net/~le-chi-thu/lava-dispatcher/improve-logging/+merge/71120
> You are the owner of lp:~le-chi-thu/lava-dispatcher/improve-logging.
>

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote : Posted in a previous version of this proposal

Hi, overall the change looks good. I share Paul's concerns about the configuration though -- in particular, it's not possible to 'stack' configurations as it is with the other configuration files we now have. This is in large part because logging.config basically sucks (if you could pass a ConfigParser instance to a function to configure the logging, there wouldn't be this problem).

Also, in my experience, people don't actually use fileConfig (I don't think I've worked on a single project that uses it in fact). I agree with Paul that we should start simple, and just allow a key 'log_level' in the general lava-dispatcher.conf file to set a log level, and when we want to customize other things, start from this base line.

Revision history for this message
Le Chi Thu (le-chi-thu) wrote : Posted in a previous version of this proposal

Just updated to simplify the configuration of logging level.

118. By Paul Larson

Support lava-android-test in lava-dispatcher for android testing

Revision history for this message
Paul Larson (pwlars) wrote :

As discussed earlier today, I changed this slightly to make the option use the python logging module levels, and just called the option LOGGING_LEVEL.

Also, please make sure to select linaro-validation when requesting merge proposals so that everyone see it :)

review: Approve
119. By Paul Larson

Merge python logging branch from ChiThu

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file '.bzrignore'
--- .bzrignore 2011-09-09 00:57:07 +0000
+++ .bzrignore 2011-09-22 10:41:46 +0000
@@ -6,3 +6,4 @@
6./dist6./dist
7./lava_dispatcher.egg-info7./lava_dispatcher.egg-info
8./build8./build
9./.idea
9\ No newline at end of file10\ No newline at end of file
1011
=== modified file 'lava-dispatch'
--- lava-dispatch 2011-08-17 00:52:46 +0000
+++ lava-dispatch 2011-09-22 10:41:46 +0000
@@ -22,9 +22,11 @@
22import optparse22import optparse
23import os23import os
24import sys24import sys
25import logging.config
2526
26from lava_dispatcher import LavaTestJob27from lava_dispatcher import LavaTestJob
2728from lava_dispatcher.config import load_config_paths
29from lava_dispatcher.config import get_config
2830
29parser = optparse.OptionParser('%prog: lava-dispatch <json job file>')31parser = optparse.OptionParser('%prog: lava-dispatch <json job file>')
30parser.add_option(32parser.add_option(
@@ -32,6 +34,7 @@
3234
33(options, args) = parser.parse_args()35(options, args) = parser.parse_args()
3436
37
35if len(args) != 1:38if len(args) != 1:
36 parser.print_help()39 parser.print_help()
37 sys.exit(1)40 sys.exit(1)
@@ -44,6 +47,20 @@
44with open(args[0]) as fd:47with open(args[0]) as fd:
45 jobdata = fd.read()48 jobdata = fd.read()
4649
50# config the python logging
51
52FORMAT = '<LAVA_DISPATCHER>%(asctime)s %(levelname)s: %(message)s'
53DATEFMT= '%Y-%m-%d %I:%M:%S %p'
54logging.basicConfig(format=FORMAT,datefmt=DATEFMT)
55
56config = get_config("lava-dispatcher")
57logging_debug = config.get("LOGGING_LEVEL_DEBUG")
58if (logging_debug == "1"):
59 logging.root.setLevel(logging.DEBUG)
60else:
61 logging.root.setLevel(logging.INFO)
62
47job = LavaTestJob(jobdata, oob_file)63job = LavaTestJob(jobdata, oob_file)
64
48#FIXME Return status65#FIXME Return status
49job.run()66job.run()
5067
=== modified file 'lava_dispatcher/actions/android_0xbench.py'
--- lava_dispatcher/actions/android_0xbench.py 2011-07-20 06:53:10 +0000
+++ lava_dispatcher/actions/android_0xbench.py 2011-09-22 10:41:46 +0000
@@ -22,6 +22,7 @@
22from lava_dispatcher.actions import BaseAndroidAction22from lava_dispatcher.actions import BaseAndroidAction
23import time23import time
24import pexpect24import pexpect
25import logging
2526
26class cmd_test_android_0xbench(BaseAndroidAction):27class cmd_test_android_0xbench(BaseAndroidAction):
27 def run(self):28 def run(self):
@@ -30,7 +31,7 @@
30 time.sleep(30)31 time.sleep(30)
31 if not self.check_sys_bootup():32 if not self.check_sys_bootup():
32 # TODO: Fetch the logcat message as attached33 # TODO: Fetch the logcat message as attached
33 print "0xbench Test: sys bootup fail, aborted"34 logging.warning("0xbench Test: sys bootup fail, aborted")
34 return35 return
3536
36 self.client.android_logcat_clear()37 self.client.android_logcat_clear()
@@ -46,7 +47,7 @@
46 try:47 try:
47 self.client.android_logcat_monitor(pattern, timeout = 1200)48 self.client.android_logcat_monitor(pattern, timeout = 1200)
48 except pexpect.TIMEOUT:49 except pexpect.TIMEOUT:
49 print "0xbench Test: TIMEOUT Fail"50 logging.warning("0xbench Test: TIMEOUT Fail")
50 raise51 raise
51 finally:52 finally:
52 self.client.android_logcat_stop()53 self.client.android_logcat_stop()
5354
=== modified file 'lava_dispatcher/actions/android_basic.py'
--- lava_dispatcher/actions/android_basic.py 2011-09-05 22:39:47 +0000
+++ lava_dispatcher/actions/android_basic.py 2011-09-22 10:41:46 +0000
@@ -24,6 +24,7 @@
24import time24import time
25import pexpect25import pexpect
26import sys26import sys
27import logging
27from datetime import datetime28from datetime import datetime
28from lava_dispatcher.android_util import savebundlefile29from lava_dispatcher.android_util import savebundlefile
2930
@@ -34,7 +35,7 @@
34 time.sleep(30)35 time.sleep(30)
35 if not self.check_sys_bootup():36 if not self.check_sys_bootup():
36 # TODO: Fetch the logcat message as attachment37 # TODO: Fetch the logcat message as attachment
37 print "monkey run test skipped: sys bootup fail"38 logging.warning("monkey run test skipped: sys bootup fail")
38 return39 return
3940
40 TIMEFORMAT = '%Y-%m-%dT%H:%M:%SZ'41 TIMEFORMAT = '%Y-%m-%dT%H:%M:%SZ'
4142
=== modified file 'lava_dispatcher/actions/android_deploy.py'
--- lava_dispatcher/actions/android_deploy.py 2011-09-14 04:18:40 +0000
+++ lava_dispatcher/actions/android_deploy.py 2011-09-22 10:41:46 +0000
@@ -24,6 +24,8 @@
24import sys24import sys
25import shutil25import shutil
26import traceback26import traceback
27import logging
28
27from tempfile import mkdtemp29from tempfile import mkdtemp
28from lava_dispatcher.utils import download, download_with_cache30from lava_dispatcher.utils import download, download_with_cache
29from lava_dispatcher.client import CriticalError31from lava_dispatcher.client import CriticalError
@@ -33,14 +35,14 @@
33 LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir35 LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir
34 LAVA_IMAGE_URL = self.context.lava_image_url36 LAVA_IMAGE_URL = self.context.lava_image_url
35 client = self.client37 client = self.client
36 print "deploying Android on %s" % client.hostname38 logging.info("deploying Android on %s" % client.hostname)
37 print " boot: %s" % boot39 logging.info(" boot: %s" % boot)
38 print " system: %s" % system40 logging.info(" system: %s" % system)
39 print " data: %s" % data41 logging.info(" data: %s" % data)
40 print "Booting master image"42 logging.info("Booting master image")
41 client.boot_master_image()43 client.boot_master_image()
4244
43 print "Waiting for network to come up"45 logging.info("Waiting for network to come up")
44 try:46 try:
45 client.wait_network_up()47 client.wait_network_up()
46 except:48 except:
4749
=== modified file 'lava_dispatcher/actions/boot_control.py'
--- lava_dispatcher/actions/boot_control.py 2011-09-16 02:37:57 +0000
+++ lava_dispatcher/actions/boot_control.py 2011-09-22 10:41:46 +0000
@@ -20,6 +20,8 @@
20# along20# along
21# with this program; if not, see <http://www.gnu.org/licenses>.21# with this program; if not, see <http://www.gnu.org/licenses>.
2222
23import logging
24
23from lava_dispatcher.actions import BaseAction, BaseAndroidAction25from lava_dispatcher.actions import BaseAction, BaseAndroidAction
24from lava_dispatcher.client import CriticalError26from lava_dispatcher.client import CriticalError
2527
@@ -44,6 +46,7 @@
44 client.proc.sendline("")46 client.proc.sendline("")
45 status = 'pass'47 status = 'pass'
46 try:48 try:
49 logging.info("Boot Linaro image")
47 client.boot_linaro_image()50 client.boot_linaro_image()
48 except:51 except:
49 status = 'fail'52 status = 'fail'
@@ -56,4 +59,5 @@
56 """59 """
57 def run(self):60 def run(self):
58 client = self.client61 client = self.client
62 logging.info("Boot Master image")
59 client.boot_master_image()63 client.boot_master_image()
6064
=== modified file 'lava_dispatcher/actions/deploy.py'
--- lava_dispatcher/actions/deploy.py 2011-09-16 02:37:57 +0000
+++ lava_dispatcher/actions/deploy.py 2011-09-22 10:41:46 +0000
@@ -24,6 +24,7 @@
24import shutil24import shutil
25import traceback25import traceback
26from tempfile import mkdtemp26from tempfile import mkdtemp
27import logging
2728
28from lava_dispatcher.actions import BaseAction29from lava_dispatcher.actions import BaseAction
29from lava_dispatcher.utils import download, download_with_cache30from lava_dispatcher.utils import download, download_with_cache
@@ -35,15 +36,22 @@
35 LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir36 LAVA_IMAGE_TMPDIR = self.context.lava_image_tmpdir
36 LAVA_IMAGE_URL = self.context.lava_image_url37 LAVA_IMAGE_URL = self.context.lava_image_url
37 client = self.client38 client = self.client
39<<<<<<< TREE
38 print "deploying on %s" % client.hostname40 print "deploying on %s" % client.hostname
39 print " hwpack: %s" % hwpack41 print " hwpack: %s" % hwpack
40 print " rootfs: %s" % rootfs42 print " rootfs: %s" % rootfs
41 if kernel_matrix:43 if kernel_matrix:
42 print " package: %s" % kernel_matrix[0]44 print " package: %s" % kernel_matrix[0]
43 print "Booting master image"45 print "Booting master image"
46=======
47 logging.info("deploying on %s" % client.hostname)
48 logging.info(" hwpack: %s" % hwpack)
49 logging.info(" rootfs: %s" % rootfs)
50 logging.info("Booting master image")
51>>>>>>> MERGE-SOURCE
44 client.boot_master_image()52 client.boot_master_image()
4553
46 print "Waiting for network to come up"54 logging.info("Waiting for network to come up")
47 try:55 try:
48 client.wait_network_up()56 client.wait_network_up()
49 except:57 except:
@@ -51,6 +59,7 @@
51 client.sio.write(tb)59 client.sio.write(tb)
52 raise CriticalError("Unable to reach LAVA server, check network")60 raise CriticalError("Unable to reach LAVA server, check network")
5361
62<<<<<<< TREE
54 if kernel_matrix:63 if kernel_matrix:
55 hwpack = self.refresh_hwpack(kernel_matrix, hwpack, use_cache)64 hwpack = self.refresh_hwpack(kernel_matrix, hwpack, use_cache)
56 #make new hwpack downloadable65 #make new hwpack downloadable
@@ -59,6 +68,10 @@
59 LAVA_IMAGE_URL, hwpack])68 LAVA_IMAGE_URL, hwpack])
60 print " hwpack with new kernel: %s" % hwpack69 print " hwpack with new kernel: %s" % hwpack
6170
71=======
72 logging.info("About to handle with the build")
73
74>>>>>>> MERGE-SOURCE
62 try:75 try:
63 boot_tgz, root_tgz = self.generate_tarballs(hwpack, rootfs, 76 boot_tgz, root_tgz = self.generate_tarballs(hwpack, rootfs,
64 use_cache)77 use_cache)
@@ -131,16 +144,24 @@
131 os.chmod(tarball_dir, 0755)144 os.chmod(tarball_dir, 0755)
132 #fix me: if url is not http-prefix, copy it to tarball_dir145 #fix me: if url is not http-prefix, copy it to tarball_dir
133 if use_cache:146 if use_cache:
147 logging.info("Downloading the %s file using cache" % hwpack_url)
134 hwpack_path = download_with_cache(hwpack_url, tarball_dir, lava_cachedir)148 hwpack_path = download_with_cache(hwpack_url, tarball_dir, lava_cachedir)
149
150 logging.info("Downloading the %s file using cache" % rootfs_url)
135 rootfs_path = download_with_cache(rootfs_url, tarball_dir, lava_cachedir)151 rootfs_path = download_with_cache(rootfs_url, tarball_dir, lava_cachedir)
136 else:152 else:
153 logging.info("Downloading the %s file" % hwpack_url)
137 hwpack_path = download(hwpack_url, tarball_dir)154 hwpack_path = download(hwpack_url, tarball_dir)
155
156 logging.info("Downloading the %s file" % rootfs_url)
138 rootfs_path = download(rootfs_url, tarball_dir)157 rootfs_path = download(rootfs_url, tarball_dir)
139158
140 image_file = os.path.join(tarball_dir, "lava.img")159 image_file = os.path.join(tarball_dir, "lava.img")
141 cmd = ("sudo linaro-media-create --hwpack-force-yes --dev %s "160 cmd = ("sudo linaro-media-create --hwpack-force-yes --dev %s "
142 "--image_file %s --binary %s --hwpack %s --image_size 3G" %161 "--image_file %s --binary %s --hwpack %s --image_size 3G" %
143 (client.device_type, image_file, rootfs_path, hwpack_path))162 (client.device_type, image_file, rootfs_path, hwpack_path))
163 logging.info("Executing the linaro-media-create command")
164 logging.info(cmd)
144 rc, output = getstatusoutput(cmd)165 rc, output = getstatusoutput(cmd)
145 if rc:166 if rc:
146 shutil.rmtree(tarball_dir)167 shutil.rmtree(tarball_dir)
@@ -163,7 +184,7 @@
163184
164 def deploy_linaro_rootfs(self, rootfs):185 def deploy_linaro_rootfs(self, rootfs):
165 client = self.client186 client = self.client
166 print "Deploying linaro image"187 logging.info("Deploying linaro image")
167 client.run_cmd_master('umount /dev/disk/by-label/testrootfs')188 client.run_cmd_master('umount /dev/disk/by-label/testrootfs')
168 client.run_cmd_master(189 client.run_cmd_master(
169 'mkfs.ext3 -q /dev/disk/by-label/testrootfs -L testrootfs')190 'mkfs.ext3 -q /dev/disk/by-label/testrootfs -L testrootfs')
@@ -189,6 +210,7 @@
189210
190 def deploy_linaro_bootfs(self, bootfs):211 def deploy_linaro_bootfs(self, bootfs):
191 client = self.client212 client = self.client
213 logging.info("Deploying linaro bootfs")
192 client.run_cmd_master('umount /dev/disk/by-label/testboot')214 client.run_cmd_master('umount /dev/disk/by-label/testboot')
193 client.run_cmd_master(215 client.run_cmd_master(
194 'mkfs.vfat /dev/disk/by-label/testboot -n testboot')216 'mkfs.vfat /dev/disk/by-label/testboot -n testboot')
195217
=== modified file 'lava_dispatcher/actions/launch_control.py'
--- lava_dispatcher/actions/launch_control.py 2011-09-20 02:35:31 +0000
+++ lava_dispatcher/actions/launch_control.py 2011-09-22 10:41:46 +0000
@@ -23,6 +23,8 @@
23import os23import os
24import shutil24import shutil
25import tarfile25import tarfile
26import logging
27
26from lava_dispatcher.actions import BaseAction28from lava_dispatcher.actions import BaseAction
27from lava_dispatcher.client import OperationFailed29from lava_dispatcher.client import OperationFailed
28from lava_dispatcher.utils import download30from lava_dispatcher.utils import download
@@ -33,8 +35,16 @@
3335
34class cmd_submit_results_on_host(BaseAction):36class cmd_submit_results_on_host(BaseAction):
35 def run(self, server, stream):37 def run(self, server, stream):
38<<<<<<< TREE
36 dashboard = _get_dashboard(server)39 dashboard = _get_dashboard(server)
37 40
41=======
42 logging.info("Executing submit_results_on_host command")
43 xmlrpc_url = "%s/xml-rpc/" % server
44 srv = xmlrpclib.ServerProxy(xmlrpc_url,
45 allow_none=True, use_datetime=True)
46
47>>>>>>> MERGE-SOURCE
38 #Upload bundle files to dashboard48 #Upload bundle files to dashboard
39 bundle_list = os.listdir("/tmp/%s" % self.context.lava_result_dir)49 bundle_list = os.listdir("/tmp/%s" % self.context.lava_result_dir)
40 for bundle_name in bundle_list:50 for bundle_name in bundle_list:
@@ -47,9 +57,9 @@
47 print >> self.context.oob_file, 'dashboard-put-result:', \57 print >> self.context.oob_file, 'dashboard-put-result:', \
48 dashboard.put_ex(content, job_name, stream)58 dashboard.put_ex(content, job_name, stream)
49 except xmlrpclib.Fault, err:59 except xmlrpclib.Fault, err:
50 print "xmlrpclib.Fault occurred"60 logging.warning("xmlrpclib.Fault occurred")
51 print "Fault code: %d" % err.faultCode61 logging.warning("Fault code: %d" % err.faultCode)
52 print "Fault string: %s" % err.faultString62 logging.warning("Fault string: %s" % err.faultString)
5363
54 # After uploading, remove the bundle file at the host side64 # After uploading, remove the bundle file at the host side
55 os.remove(bundle)65 os.remove(bundle)
@@ -63,8 +73,18 @@
63 :param server: URL of the lava-dashboard server RPC endpoint73 :param server: URL of the lava-dashboard server RPC endpoint
64 :param stream: Stream on the lava-dashboard server to save the result to74 :param stream: Stream on the lava-dashboard server to save the result to
65 """75 """
76<<<<<<< TREE
66 #Create l-d server connection77 #Create l-d server connection
67 dashboard = _get_dashboard(server)78 dashboard = _get_dashboard(server)
79=======
80 #Create l-c server connection
81 xmlrpc_url = "%s/xml-rpc/" % server
82
83 logging.info("Executing submit_results command to server "+xmlrpc_url)
84
85 srv = xmlrpclib.ServerProxy(xmlrpc_url,
86 allow_none=True, use_datetime=True)
87>>>>>>> MERGE-SOURCE
6888
69 client = self.client89 client = self.client
70 try:90 try:
@@ -82,6 +102,7 @@
82 client.run_cmd_master('umount /mnt/root')102 client.run_cmd_master('umount /mnt/root')
83103
84 #Create tarball of all results104 #Create tarball of all results
105 logging.info("Creating lava results tarball")
85 client.run_cmd_master('cd /tmp')106 client.run_cmd_master('cd /tmp')
86 client.run_cmd_master(107 client.run_cmd_master(
87 'tar czf /tmp/lava_results.tgz -C /tmp/%s .' % self.context.lava_result_dir)108 'tar czf /tmp/lava_results.tgz -C /tmp/%s .' % self.context.lava_result_dir)
@@ -101,6 +122,8 @@
101122
102 # download test result with a retry mechanism123 # download test result with a retry mechanism
103 # set retry timeout to 2mins124 # set retry timeout to 2mins
125
126 logging.info("About to download the result tarball to host")
104 now = time.time()127 now = time.time()
105 timeout = 120128 timeout = 120
106 try:129 try:
@@ -111,9 +134,10 @@
111 if time.time() >= now+timeout:134 if time.time() >= now+timeout:
112 raise135 raise
113 except:136 except:
114 print traceback.format_exc()137 logging.warning(traceback.format_exc())
115 status = 'fail'138 status = 'fail'
116 err_msg = err_msg + " Can't retrieve test case results."139 err_msg = err_msg + " Can't retrieve test case results."
140 logging.warning(err_msg)
117141
118 client.run_cmd_master('kill %1')142 client.run_cmd_master('kill %1')
119143
@@ -127,15 +151,17 @@
127 self.all_bundles.append(json.loads(content))151 self.all_bundles.append(json.loads(content))
128 tar.close()152 tar.close()
129 except:153 except:
130 print traceback.format_exc()154 logging.warning(traceback.format_exc())
131 status = 'fail'155 status = 'fail'
132 err_msg = err_msg + " Some test case result appending failed."156 err_msg = err_msg + " Some test case result appending failed."
157 logging.warning(err_msg)
133 finally:158 finally:
134 shutil.rmtree(tarball_dir)159 shutil.rmtree(tarball_dir)
135 else:160 else:
136 status = 'fail'161 status = 'fail'
137 err_msg = err_msg + "Getting master image IP address failed, \162 err_msg = err_msg + "Getting master image IP address failed, \
138no test case result retrived."163no test case result retrived."
164 logging.warning(err_msg)
139165
140 #flush the serial log166 #flush the serial log
141 client.run_shell_command("")167 client.run_shell_command("")
142168
=== modified file 'lava_dispatcher/actions/lava-test.py'
--- lava_dispatcher/actions/lava-test.py 2011-09-16 02:37:57 +0000
+++ lava_dispatcher/actions/lava-test.py 2011-09-22 10:41:46 +0000
@@ -22,6 +22,8 @@
2222
23from datetime import datetime23from datetime import datetime
24import traceback24import traceback
25import logging
26
25from lava_dispatcher.actions import BaseAction27from lava_dispatcher.actions import BaseAction
26from lava_dispatcher.client import OperationFailed, CriticalError28from lava_dispatcher.client import OperationFailed, CriticalError
2729
@@ -77,6 +79,7 @@
7779
78class cmd_lava_test_run(BaseAction):80class cmd_lava_test_run(BaseAction):
79 def run(self, test_name, timeout=-1):81 def run(self, test_name, timeout=-1):
82 logging.info("Executing lava_test_run %s command" % test_name)
80 #Make sure in test image now83 #Make sure in test image now
81 client = self.client84 client = self.client
82 client.in_test_shell()85 client.in_test_shell()
@@ -96,6 +99,7 @@
96 lava-test deployment to test image rootfs by chroot99 lava-test deployment to test image rootfs by chroot
97 """100 """
98 def run(self, tests, install_python = None, register = None, timeout=2400):101 def run(self, tests, install_python = None, register = None, timeout=2400):
102 logging.info("Executing lava_test_install (%s) command" % ",".join(tests))
99 client = self.client103 client = self.client
100104
101 _setup_testrootfs(client)105 _setup_testrootfs(client)
102106
=== modified file 'lava_dispatcher/android_client.py'
--- lava_dispatcher/android_client.py 2011-09-08 04:28:39 +0000
+++ lava_dispatcher/android_client.py 2011-09-22 10:41:46 +0000
@@ -19,6 +19,8 @@
1919
20import pexpect20import pexpect
21import sys21import sys
22import logging
23
22from lava_dispatcher.client import LavaClient, OperationFailed24from lava_dispatcher.client import LavaClient, OperationFailed
23from utils import string_to_list25from utils import string_to_list
2426
@@ -80,7 +82,7 @@
8082
81 def android_logcat_stop(self):83 def android_logcat_stop(self):
82 self.proc.sendcontrol('C')84 self.proc.sendcontrol('C')
83 print "logcat cancelled"85 logging.info("logcat cancelled")
8486
85 # adb cound be connected through network87 # adb cound be connected through network
86 def android_adb_connect(self, dev_ip):88 def android_adb_connect(self, dev_ip):
@@ -108,7 +110,7 @@
108 self.run_cmd_tester(110 self.run_cmd_tester(
109 'netcfg %s dhcp' % network_interface, timeout=60)111 'netcfg %s dhcp' % network_interface, timeout=60)
110 except:112 except:
111 print "netcfg %s dhcp exception" % network_interface113 logging.warning("netcfg %s dhcp exception" % network_interface)
112 return False114 return False
113115
114 # Check network ip and setup adb connection116 # Check network ip and setup adb connection
@@ -119,7 +121,7 @@
119 try:121 try:
120 id = self.proc.expect([ip_pattern, pexpect.EOF], timeout=60)122 id = self.proc.expect([ip_pattern, pexpect.EOF], timeout=60)
121 except:123 except:
122 print "ifconfig can not match ip pattern"124 logging.warning("ifconfig can not match ip pattern")
123 return False125 return False
124 if id == 0:126 if id == 0:
125 match_group = self.proc.match.groups()127 match_group = self.proc.match.groups()
@@ -127,7 +129,7 @@
127 device_ip = match_group[0]129 device_ip = match_group[0]
128 adb_status, dev_name = self.android_adb_connect(device_ip)130 adb_status, dev_name = self.android_adb_connect(device_ip)
129 if adb_status == True:131 if adb_status == True:
130 print "dev_name = " + dev_name132 logging.info("dev_name = " + dev_name)
131 result = self.run_adb_shell_command(dev_name, "echo 1", "1")133 result = self.run_adb_shell_command(dev_name, "echo 1", "1")
132 self.android_adb_disconnect(device_ip)134 self.android_adb_disconnect(device_ip)
133 return result135 return result
134136
=== modified file 'lava_dispatcher/client.py'
--- lava_dispatcher/client.py 2011-09-15 07:02:08 +0000
+++ lava_dispatcher/client.py 2011-09-22 10:41:46 +0000
@@ -24,6 +24,7 @@
24from cStringIO import StringIO24from cStringIO import StringIO
25import traceback25import traceback
26from utils import string_to_list26from utils import string_to_list
27import logging
2728
28class LavaClient(object):29class LavaClient(object):
29 def __init__(self, context, config):30 def __init__(self, context, config):
@@ -203,7 +204,7 @@
203 try:204 try:
204 self.wait_network_up()205 self.wait_network_up()
205 except:206 except:
206 print traceback.format_exc()207 logging.warning(traceback.format_exc())
207 return None208 return None
208 #tty device uses minimal match, see pexpect wiki209 #tty device uses minimal match, see pexpect wiki
209 #pattern1 = ".*\n(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"210 #pattern1 = ".*\n(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"
@@ -215,10 +216,10 @@
215 #self.proc.sendline("")216 #self.proc.sendline("")
216 id = self.proc.expect([pattern1, pexpect.EOF,217 id = self.proc.expect([pattern1, pexpect.EOF,
217 pexpect.TIMEOUT], timeout=5)218 pexpect.TIMEOUT], timeout=5)
218 print "\nmatching pattern is %s" % id219 logging.info("\nmatching pattern is %s" % id)
219 if id == 0:220 if id == 0:
220 ip = self.proc.match.groups()[0]221 ip = self.proc.match.groups()[0]
221 print "Master IP is %s" % ip222 logging.info("Master IP is %s" % ip)
222 return ip223 return ip
223 else:224 else:
224 return None225 return None
225226
=== modified file 'lava_dispatcher/config.py'
--- lava_dispatcher/config.py 2011-09-09 00:52:17 +0000
+++ lava_dispatcher/config.py 2011-09-22 10:41:46 +0000
@@ -21,6 +21,7 @@
21from ConfigParser import ConfigParser21from ConfigParser import ConfigParser
22import os22import os
23import StringIO23import StringIO
24import logging
2425
2526
26default_config_path = os.path.join(27default_config_path = os.path.join(
@@ -60,7 +61,7 @@
60 config_files.reverse()61 config_files.reverse()
61 if cp is None:62 if cp is None:
62 cp = ConfigParser()63 cp = ConfigParser()
63 print "About to read %s" % str(config_files)64 logging.debug("About to read %s" % str(config_files))
64 for path in config_files:65 for path in config_files:
65 _read_into(path, cp)66 _read_into(path, cp)
66 return cp67 return cp
6768
=== modified file 'lava_dispatcher/default-config/lava-dispatcher/README'
--- lava_dispatcher/default-config/lava-dispatcher/README 2011-09-08 23:35:05 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/README 2011-09-22 10:41:46 +0000
@@ -39,3 +39,7 @@
39 a line "device_type = <device type>", although other settings can39 a line "device_type = <device type>", although other settings can
40 be included here. You will definitely need to tell lava-dispatcher40 be included here. You will definitely need to tell lava-dispatcher
41 about the devices you have!41 about the devices you have!
42
43* logging.conf
44
45 This file defines settings for Python logging
42\ No newline at end of file46\ No newline at end of file
4347
=== modified file 'lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf'
--- lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf 2011-09-08 23:35:05 +0000
+++ lava_dispatcher/default-config/lava-dispatcher/lava-dispatcher.conf 2011-09-22 10:41:46 +0000
@@ -17,3 +17,5 @@
17# Location for caching downloaded artifacts such as hwpacks and images17# Location for caching downloaded artifacts such as hwpacks and images
18LAVA_CACHEDIR = /linaro/images/cache18LAVA_CACHEDIR = /linaro/images/cache
1919
20# Enable python logging level to debug. 0 - logging level is INFO, 1- logging level is DEBUG
21LOGGING_LEVEL_DEBUG = 0
20\ No newline at end of file22\ No newline at end of file

Subscribers

People subscribed via source and target branches