Merge lp:~pwlars/lava-test/json-compat-fix into lp:lava-test/0.0

[Paul Larson]

These are needed in preparation for being able to submit results to launch-control. Launch-control currently fails silently and accepts invalid results, but if you look into it further you can see that it doesn't properly handle them unless these fixes are in place.

Thanks,
Paul Larson

[James Westby (james-w)]

Hi,

I find these sorts of things a little confusing:

28 - device['desc'] = desc
29 + device['attributes'] = desc

Can we have the local variables not share the name of a key they are
not for?

There are also more code changes than test changes, is the test
coverage good enough in this area?

Thanks,

James

review: Needs Information

[Zygmunt Krynicki (zyga)]

These are needed in preparation for being able to submit results to launch-control. Launch-control currently fails silently and accepts invalid results,

Yes, you can upload anything as validation and processing is going to be asynchronous anyway. Note that you _can_ check wat went wrong by looking at BundleDeserializationError model right in the web UI.

 but if you look into it further you can see that it doesn't properly handle them unless these fixes are in place.

It can handle correct documents, they just need to adhere to schema.

Best regards.
Zygmunt Krynicki

[lp:~pwlars/lava-test/json-compat-fix]

36. By Paul Larson on 2010-09-27

match variable names to field names better in hwprofile

37. By Paul Larson on 2010-09-27

Make hw_profile tests more complete

[Paul Larson (pwlars)]

> 28 - device['desc'] = desc
> 29 + device['attributes'] = desc
> Can we have the local variables not share the name of a key they are
> not for?
Done

> There are also more code changes than test changes, is the test
> coverage good enough in this area?
Changed this a bit. The tests look at the whole dict that should result from converting the cpuinfo, meminfo, etc. This doesn't add more tests, but makes the existing tests more complete.

[James Westby (james-w)]

Hi,

Much better, thanks.

34 - device['desc'] = desc
35 + device['attributes'] = desc

I don't know how tricky it would be to clean that one up as well, but
if you can that would be great.

Thanks,

James

review: Approve

[Paul Larson (pwlars)]

On Mon, 2010-09-27 at 15:22 +0000, James Westby wrote:
> 34 - device['desc'] = desc
> 35 + device['attributes'] = desc
Bah, missed that one. Thanks!

=== modified file 'abrek/hwprofile.py'
--- abrek/hwprofile.py	2010-09-18 04:25:23 +0000
+++ abrek/hwprofile.py	2010-09-27 14:22:38 +0000
@@ -95,8 +95,8 @@
         for c in range(len(cpudevs)):
             device = {}
             device['device_type'] = 'device.cpu'
-            device['attributes'] = 'Processor #{0}'.format(c)
-            device['desc'] = cpudevs[c]
+            device['description'] = 'Processor #{0}'.format(c)
+            device['attributes'] = cpudevs[c]
             devices.append(device)
     except IOError:
         print >> sys.stderr, "WARNING: Could not read cpu information"
@@ -113,11 +113,11 @@
     machine = os.uname()[-1]
     if machine in ('i686', 'x86_64'):
         try:
-            name = read_file('/sys/class/dmi/id/board_name') or None
+            description = read_file('/sys/class/dmi/id/board_name') or None
             vendor = read_file('/sys/class/dmi/id/board_vendor') or None
             version = read_file('/sys/class/dmi/id/board_version') or None
-            if name:
-                device['attributes'] = name.strip()
+            if description:
+                device['description'] = description.strip()
             if vendor:
                 desc['vendor'] = vendor.strip()
             if version:
@@ -140,7 +140,7 @@
             return devices
     else:
         return devices
-    device['desc'] = desc
+    device['attributes'] = desc
     device['device_type'] = 'device.board'
     devices.append(device)
     return devices
@@ -166,11 +166,11 @@
                 kind = 'RAM'
             else:
                 kind = 'swap'
-            name = "{capacity}MiB of {kind}".format(
+            description = "{capacity}MiB of {kind}".format(
                 capacity=capacity >> 20, kind=kind)
             device = {}
-            device['attributes'] = name
-            device['desc'] = {'capacity': capacity, 'kind': kind}
+            device['description'] = description
+            device['attributes'] = {'capacity': capacity, 'kind': kind}
             device['device_type'] = "device.mem"
             devices.append(device)
     except IOError:
@@ -183,18 +183,18 @@
     """
     pattern = re.compile(
               "^Bus \d{3} Device \d{3}: ID (?P<vendor_id>[0-9a-f]{4}):"
-              "(?P<product_id>[0-9a-f]{4}) (?P<name>.*)$")
+              "(?P<product_id>[0-9a-f]{4}) (?P<description>.*)$")
     devices = []
     for line in commands.getoutput('lsusb').splitlines():
         match = pattern.match(line)
         if match:
-            vendor_id, product_id, name = match.groups()
-            desc = {}
+            vendor_id, product_id, description = match.groups()
+            attributes = {}
             device = {}
-            desc['vendor_id'] = int(vendor_id, 16)
-            desc['product_id'] = int(product_id, 16)
-            device['desc'] = desc
-            device['attributes'] = name
+            attributes['vendor_id'] = int(vendor_id, 16)
+            attributes['product_id'] = int(product_id, 16)
+            device['attributes'] = attributes
+            device['description'] = description
             device['device_type'] = 'device.usb'
             devices.append(device)
     return devices

=== modified file 'abrek/testdef.py'
--- abrek/testdef.py	2010-09-21 22:35:55 +0000
+++ abrek/testdef.py	2010-09-27 14:22:38 +0000
@@ -92,17 +92,18 @@
 
     def _savetestdata(self):
         TIMEFORMAT = '%Y-%m-%dT%H:%M:%SZ'
-        testdata = {}
+        testdata = [{}]
+        testdata['format'] = "Dashboard Bundle Format 1.0"
         filename = os.path.join(self.resultsdir, 'testdata.json')
-        testdata['test_id'] = self.testname
-        testdata['analyzer_assigned_uuid'] = str(uuid1())
-        testdata['time_check_performed'] = False
-        testdata['analyzer_assigned_date'] = datetime.strftime(
+        testdata[0]['test_id'] = self.testname
+        testdata[0]['analyzer_assigned_uuid'] = str(uuid1())
+        testdata[0]['time_check_performed'] = False
+        testdata[0]['analyzer_assigned_date'] = datetime.strftime(
                                              self.runner.starttime,TIMEFORMAT)
         hw = hwprofile.get_hw_context()
-        testdata['hw_context'] = hw
+        testdata[0]['hw_context'] = hw
         sw = swprofile.get_sw_context()
-        testdata['sw_context'] = sw
+        testdata[0]['sw_context'] = sw
         write_file(json.dumps(testdata), filename)
 
     def run(self, quiet=False):

=== modified file 'tests/test_hwprofile.py'
--- tests/test_hwprofile.py	2010-09-18 04:26:46 +0000
+++ tests/test_hwprofile.py	2010-09-27 14:22:38 +0000
@@ -79,22 +79,54 @@
     def test_get_cpu_devs(self):
         fake_file('/proc/cpuinfo', ARM_CPUINFO_FILE)
         devs = abrek.hwprofile.get_cpu_devs()
-        processor = "ARMv7 Processor rev 3 (v7l)"
-        self.assertEqual(processor, devs[0]['desc']['Processor'])
+        cpuinfo = {
+            'attributes': {
+                'CPU implementer': '0x41',
+                'Features': 'swp half thumb fastmult vfp edsp neon vfpv3*',
+                'CPU architecture': '7',
+                'BogoMIPS': '483.16',
+                'Hardware': 'OMAP3 Beagle Board',
+                'CPU revision': '3',
+                'CPU part': '0xc08',
+                'Serial': '0000000000000000',
+                'Processor': 'ARMv7 Processor rev 3 (v7l)',
+                'CPU variant': '0x1',
+                'Revision': '0020'},
+            'description': 'Processor #0',
+            'device_type': 'device.cpu'}
+        self.assertEqual(cpuinfo, devs[0])
 
     def test_get_board_devs(self):
         fake_file('/sys/class/dmi/id/board_name', FAKE_BOARDNAME_FILE)
         devs = abrek.hwprofile.get_board_devs()
-        self.assertEqual(FAKE_BOARDNAME_FILE, devs[0]['attributes'])
+        boardinfo = {
+            'attributes': {
+                'version': 'Not Available',
+                'vendor': 'LENOVO'},
+            'description': 'XXXXXXX',
+            'device_type': 'device.board'}
+        self.assertEqual(boardinfo, devs[0])
 
     def test_get_mem_devs(self):
         fake_file('/proc/meminfo', FAKE_MEMINFO_FILE)
         devs = abrek.hwprofile.get_mem_devs()
-        self.assertEqual(243937280, devs[0]['desc']['capacity'])
+        meminfo = {
+            'attributes': {
+                'kind': 'RAM',
+                'capacity': 243937280},
+            'description': '232MiB of RAM',
+            'device_type': 'device.mem'}
+        self.assertEqual(meminfo, devs[0])
 
     def test_get_usb_devs(self):
         devs = abrek.hwprofile.get_usb_devs()
-        self.assertEqual('device.usb', devs[0]['device_type'])
+        usbinfo = {
+            'attributes': {
+                'vendor_id': 7531,
+                'product_id': 1},
+            'description': 'Linux Foundation 1.1 root hub',
+            'device_type': 'device.usb'}
+        self.assertEqual(usbinfo, devs[0])
 
 
 class MissingFiles(TestCaseWithFixtures):