Merge lp:~xnox/landscape-client/s390x into lp:~landscape/landscape-client/trunk

[Dimitri John Ledkov]

Add support for s390x CPU architecture

[Alberto Donato (ack)]

Looks good, +1

A few nits inline.

Also, make lint reports a few errors:

landscape/monitor/tests/test_processorinfo.py:370:1: E302 expected 2 blank lines, found 1
landscape/monitor/tests/test_processorinfo.py:379:64: W291 trailing whitespace
landscape/monitor/tests/test_processorinfo.py:380:80: E501 line too long (89 characters)
landscape/monitor/tests/test_processorinfo.py:381:80: E501 line too long (95 characters)
landscape/monitor/tests/test_processorinfo.py:382:80: E501 line too long (90 characters)
landscape/monitor/tests/test_processorinfo.py:383:80: E501 line too long (97 characters)
landscape/monitor/tests/test_processorinfo.py:384:80: E501 line too long (94 characters)
landscape/monitor/tests/test_processorinfo.py:385:80: E501 line too long (95 characters)
landscape/monitor/tests/test_processorinfo.py:393:80: E501 line too long (84 characters)
landscape/monitor/tests/test_processorinfo.py:427:80: E501 line too long (164 characters)
landscape/monitor/tests/test_processorinfo.py:447:80: E501 line too long (164 characters)
landscape/monitor/tests/test_processorinfo.py:474:80: E501 line too long (139 characters)
landscape/monitor/processorinfo.py:278:15: E225 missing whitespace around operator
landscape/monitor/processorinfo.py:291:1: W293 blank line contains whitespace

Some are related to the sample content in the test case. I think you can drop those lines (or just cut out some of the content, since they're not actually considered by the parser.

review: Approve

[Adam Collard (adam-collard)]

Can we extract an (X86 equivalent) value for cache-size?

review: Needs Information

[Dimitri John Ledkov (xnox)]

We are targeting these two microprocessors:
https://en.wikipedia.org/wiki/IBM_z13_(microprocessor)
https://en.wikipedia.org/wiki/IBM_zEC12_(microprocessor)

L1,L2 caches are per core, L3 is shared. The L3 shared cache is not reported, but for the below z chip it would have been 64MB.

There is also storage controller per compute drawer with like extra 960MiB of L4 cache, per two chips, thus 480 MiB per core.

The cache-size reported up to landscape for x86_64 for my cpu is the shared L3 cache, for z that kind of matches the shared L3 cache as reported:
cache4 : level=3 type=Unified scope=Shared size=65536K line_size=256 associativity=16

Or do you want the largest one? Cause then it would be L4 cache (non-existant on x86_64) from the co-processing unit:
cache5 : level=4 type=Unified scope=Shared size=491520K line_size=256 associativity=30

I guess i should go for the largest one? Cause i guess ultimately cpuinfo simply reports the largest cache. Then for z13 that would be 480 MiB. I'll add that in, just to show off really how big these things are. =)

[lp:~xnox/landscape-client/s390x]

829. By Dimitri John Ledkov on 2016-02-05

pep8, review comments, and add largest cache-size.

[Dimitri John Ledkov (xnox)]

Please re-review and merge. All comments addressed.

[Adam Collard (adam-collard)]

review: Approve

[lp:~xnox/landscape-client/s390x]

830. By Dimitri John Ledkov on 2016-02-05

fix copy&paste, and try to hint string with #noqa

831. By Dimitri John Ledkov on 2016-02-05

use the right noqa

=== modified file 'landscape/monitor/processorinfo.py'
--- landscape/monitor/processorinfo.py	2014-04-30 15:58:40 +0000
+++ landscape/monitor/processorinfo.py	2016-02-05 10:14:41 +0000
@@ -263,7 +263,58 @@
         return processors
 
 
+class S390XMessageFactory:
+    """Factory for s390x-based processors provides processor information.
+
+    @param source_filename: The file name of the data source.
+    """
+
+    def __init__(self, source_filename):
+        self._source_filename = source_filename
+
+    def create_message(self):
+        """Returns a list containing information about each processor."""
+        processors = []
+        vendor = None
+        cache_size = 0
+        file = open(self._source_filename)
+
+        try:
+            current = None
+
+            for line in file:
+                parts = line.split(":", 1)
+                key = parts[0].strip()
+
+                if key == "vendor_id":
+                    vendor = parts[1].strip()
+                    continue
+
+                if key.startswith("cache"):
+                    for word in parts[1].split():
+                        if word.startswith("size="):
+                            cache_size = int(word[5:-1])
+                            continue
+
+                if key.startswith("processor "):
+                    id = int(key.split()[1])
+                    model = parts[1].split()[-1]
+                    current = {
+                        "processor-id": id,
+                        "model": model,
+                        "vendor": vendor,
+                        "cache-size": cache_size,
+                    }
+                    processors.append(current)
+                    continue
+        finally:
+            file.close()
+
+        return processors
+
+
 message_factories = [("(arm*|aarch64)", ARMMessageFactory),
                      ("ppc(64)?", PowerPCMessageFactory),
                      ("sparc[64]", SparcMessageFactory),
-                     ("i[3-7]86|x86_64", X86MessageFactory)]
+                     ("i[3-7]86|x86_64", X86MessageFactory),
+                     ("s390x", S390XMessageFactory)]

=== modified file 'landscape/monitor/tests/test_processorinfo.py'
--- landscape/monitor/tests/test_processorinfo.py	2014-05-01 14:10:46 +0000
+++ landscape/monitor/tests/test_processorinfo.py	2016-02-05 10:14:41 +0000
@@ -368,6 +368,46 @@
         self.assertEqual(processor_1["processor-id"], 1)
 
 
+class S390XMessageTest(LandscapeTest):
+    """Tests for s390x message builder."""
+
+    helpers = [MonitorHelper]
+
+    S390X = """
+vendor_id       : IBM/S390
+# processors    : 4
+bogomips per cpu: 3033.00
+features	: esan3 zarch stfle msa ldisp eimm dfp etf3eh highgprs 
+cache0          : level=1 type=Data scope=Private size=128K line_size=256 associativity=8
+cache1          : level=1 type=Instruction scope=Private size=96K line_size=256 associativity=6
+cache2          : level=2 type=Data scope=Private size=2048K line_size=256 associativity=8
+cache3          : level=2 type=Instruction scope=Private size=2048K line_size=256 associativity=8
+cache4          : level=3 type=Unified scope=Shared size=65536K line_size=256 associativity=16
+cache5          : level=4 type=Unified scope=Shared size=491520K line_size=256 associativity=30
+processor 0: version = FF,  identification = 018F67,  machine = 2964
+processor 1: version = FF,  identification = 018F67,  machine = 2964
+processor 2: version = FF,  identification = 018F67,  machine = 2964
+processor 3: version = FF,  identification = 018F67,  machine = 2964
+""" # noqa
+
+    def test_read_sample_s390x_data(self):
+        """Ensure the plugin can parse /proc/cpuinfo from an IBM zSeries machine."""
+        filename = self.makeFile(self.S390X)
+        plugin = ProcessorInfo(machine_name="s390x",
+                               source_filename=filename)
+        message = plugin.create_message()
+        self.assertEqual("processor-info", message["type"])
+        self.assertEqual(4, len(message["processors"]))
+
+        for id, processor in enumerate(message["processors"]):
+            self.assertEqual(
+                {"vendor": "IBM/S390",
+                 "model": "2964",
+                 "processor-id": id,
+                 "cache-size": 491520,
+                 }, processor)
+
+
 class X86MessageTest(LandscapeTest):
     """Test for x86-specific message handling."""