submission parser breaks with ppc64el <processor> information

Bug #1306255 reported by Daniel Manrique
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Checkbox Provider - Resource
Fix Released
High
Unassigned

Bug Description

This was reported as inability to parse a submission with this error:

Traceback (most recent call last):
  File "checkbox_parser.py", line 182, in <module>
    sys.exit(main())
  File "checkbox_parser.py", line 171, in main
    messages = parse_stream(stream)
  File "checkbox_parser.py", line 24, in parse_stream
    parser.run(TestRun, messages=messages)
  File "/src/checkbox/bzr/legacy/checkbox-old/checkbox/parsers/submission.py", line 570, in run
    self.parseRoot(result, root)
  File "/src/checkbox/bzr/legacy/checkbox-old/checkbox/parsers/submission.py", line 555, in parseRoot
    parser(result, child)
  File "/src/checkbox/bzr/legacy/checkbox-old/checkbox/parsers/submission.py", line 383, in parseHardware
    parser(result, child)
  File "/src/checkbox/bzr/legacy/checkbox-old/checkbox/parsers/submission.py", line 420, in parseProcessors
    properties = self._getProperties(child)
  File "/src/checkbox/bzr/legacy/checkbox-old/checkbox/parsers/submission.py", line 309, in _getProperties
    name, value = self._getProperty(child)
  File "/src/checkbox/bzr/legacy/checkbox-old/checkbox/parsers/submission.py", line 301, in _getProperty
    return (node.get("name"), self._getValueAsType(node))
  File "/src/checkbox/bzr/legacy/checkbox-old/checkbox/parsers/submission.py", line 323, in _getValueAsType
    return str(node.text.strip())
AttributeError: 'NoneType' object has no attribute 'strip'

Looking at the submission parser more closely, this happens when trying to parse the <processors> section of the submission. On an x86 system it looks like this (showing only one processor out of 4 the system has):
    <processors>
      <processor id="0" name="0">
        <property name="bogomips" type="str">5183</property>
        <property name="cache" type="str">3145728</property>
        <property name="count" type="str">4</property>
        <property name="model" type="str">Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz</property>
        <property name="model_number" type="str">6</property>
        <property name="model_revision" type="str">7</property>
        <property name="model_version" type="str">42</property>
        <property name="other" type="str">fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid</property>
        <property name="platform" type="str">x86_64</property>
        <property name="speed" type="str">2601</property>
        <property name="type" type="str">GenuineIntel</property>
      </processor>
  </processors>

Whereas ppc64el has this:
    <processors>
      <processor id="0" name="0">
        <property name="bogomips" type="str">0</property>
        <property name="cache" type="str">0</property>
        <property name="count" type="str">2</property>
        <property name="model" type="str">POWER7 (architected), altive
        <property name="model_number" type="str"></property>
        <property name="model_revision" type="str"></property>
        <property name="model_version" type="str">2.1 (pvr 004a 0201)<
        <property name="other" type="str"></property>
        <property name="platform" type="str">pSeries</property>
        <property name="speed" type="str">4283</property>
        <property name="type" type="str">pSeries</property>
      </processor>
    </processors>

model_number, model_revision and "other" are empty in ppc64el, and this is specifically what's causing the crash. If I fill in bogus values there, things work as expected.

This presumably comes from the cpuinfo_resource script, which produces this for x86:

bogomips: 4789
cache: 4194304
count: 4
model: Intel(R) Core(TM) i7-4500U CPU @ 1.80GHz
model_number: 6
model_revision: 1
model_version: 69
other: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid
platform: x86_64
speed: 2401
type: GenuineIntel

and this for ppc64el:

bogomips: 0
cache: 0
count: 2
model: POWER7 (architected), altivec supported
model_number:
model_revision:
model_version: 2.1 (pvr 004a 0201)
other:
platform: pSeries
speed: 4283
type: pSeries

The quick way to solve it is to change cpuinfo_resource script so that it doesn't output keys with empty values (I'm hoping they won't appear in the xml then; I tested and if I just remove those empty keys in the XML, the submission parses fine), and/or it outputs reasonable model_number, _revision and other for ppc64el (essentially: if we can find them some way, add them; but ALSO add a check before printing out key/value pairs, and if the value is empty, skip printing the key).

At some point we may also want to make the parser more robust against this kind of "malformed" data.

As a workaround to enable sending of these submissions:

- Open the XML file with a text editor
- Look for the <processors> section
- For each processor, if the model_number, model_revision and other elements are empty, just delete them (delete the whole line).
- Save submission and upload normally.

Related branches

Daniel Manrique (roadmr)
affects: checkbox → plainbox-provider-resource
Revision history for this message
Daniel Manrique (roadmr) wrote :

I'll send a quick fix to avoid printing empty keys (rather, keys with empty values). We can later modify the CpuinfoParser in checkbox_support to get more data items in ppc64el systems if desired.

Zygmunt Krynicki (zyga)
Changed in plainbox-provider-resource:
milestone: none → 0.4
Revision history for this message
Ashley Lai (alai) wrote :
Changed in plainbox-provider-resource:
milestone: 0.4 → 0.5
Changed in plainbox-provider-resource:
status: Triaged → Fix Committed
Changed in plainbox-provider-resource:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.