Merge lp:~sophia-wu/opencompute/add-ocp-system-event-log-job into lp:opencompute/checkbox

[Sophia Wu]

Verify the system event log must be capable of saving at least 256 entries

[Jeff Lane  (bladernr)]

Hi Sophia.... thanks for submitting this. In general it looks good but there are a couple things:

1: Why is there a separate whitelist for this? Does this need it's own whitelist, or would this be in a general certification remote whitelist? If that's the case, the whitelist should simply be called opencompute-certify-remote.whitelist.

2: if you want the config file me.cfg to be installed, you need to add it to the approrpiate files in the debian directory. here are teh places where the examples directory is referenced:

debian/checkbox-hw-collection.install:usr/share/checkbox/examples/checkbox-hw-collection.ini
debian/checkbox.install:usr/share/checkbox/examples/checkbox.ini
debian/checkbox.install:usr/share/checkbox/examples/network.cfg
debian/checkbox.install:usr/share/checkbox/examples/virtualization.cfg
debian/checkbox.install:usr/share/checkbox/examples/org.freedesktop.policykit.checkbox.policy usr/share/polkit-1/actions/
debian/checkbox-ocp-cli.install:usr/share/checkbox/examples/checkbox-ocp-cli.ini
debian/checkbox-ocp-gtk.install:usr/share/checkbox/examples/checkbox-ocp-gtk.ini
debian/checkbox-ocp-qt.install:usr/share/checkbox/examples/checkbox-ocp-qt.ini
debian/checkbox-ocp-urwid.install:usr/share/checkbox/examples/checkbox-ocp-urwid.ini
debian/checkbox.postinst:cp /usr/share/checkbox/examples/network.cfg /etc/checkbox.d/
debian/checkbox.postinst:cp /usr/share/checkbox/examples/virtualization.cfg /etc/checkbox.d/

You're probably interested in checkbox.install and checkbox.postinst

Without adding it to those files, the config file will not be installed with the package.

3: in the ipmi_sel_entries script, you reference ../examples/me.cfg twice. The proper installed location is /etc/checkbox.d/ (See 2 above).

4: Finally, you also need to update debian/changelog and add an entry under your name that briefly describes the changes you've made. See other entries in the changelog for examples.

Fix those few items (and just let me know more about item 1) and I'll push this through.

Thanks!

review: Needs Fixing

[Sophia Wu (sophia-wu)]

Hi Jeff,

Thanks for your feedback. It's very helpful for me.

And about item 1,
Due to some other test cases that we haven't commit now, they will run ME stress test.
I think it's better to separate the whitelists, one is for retrieving ME data and another is for stressing ME.

Thanks,
Sophia

[lp:~sophia-wu/opencompute/add-ocp-system-event-log-job]

2170. By Nelson Chu on 2014-02-25

Adds OCP CPU and Memory tests and jobs

2171. By Jeff Lane  on 2014-03-25

Adds new job files to the setup.cfg file so they will be properly installed. This is just a cleanup of something I missed on initial review.

2172. By Nelson Chu on 2014-03-25

Add OCP hardware information job

2173. By Sophia Wu on 2014-03-31

resolve conflicts

2174. By Sophia Wu on 2014-03-31

Add OCP system event log job

Unmerged revisions

=== added file 'data/whitelists/opencompute-certify-remoteME.whitelist'
--- data/whitelists/opencompute-certify-remoteME.whitelist	1970-01-01 00:00:00 +0000
+++ data/whitelists/opencompute-certify-remoteME.whitelist	2014-02-14 09:48:31 +0000
@@ -0,0 +1,46 @@
+## This whitelist is intended for use inside Canonical's test labs.  The tests
+## conained in this list are the same as those contained in the
+## server-selftest.whitelist file that OEMs may use. The difference here is that
+## some of these tests depend on a specific network environment and may not run
+## properly in your test environment.  To avoid false failures, please use the 
+## server-selftest.whitelist instead.
+# Resource Jobs
+block_device
+cdimage
+cpuinfo
+device
+dmi
+dpkg
+efi
+environment
+gconf
+lsb
+meminfo
+module
+optical_drive
+package
+sleep
+uname
+#Info attachment jobs
+__info__
+cpuinfo_attachment
+dmesg_attachment
+dmi_attachment
+dmidecode_attachment
+efi_attachment
+lspci_attachment
+meminfo_attachment
+modprobe_attachment
+modules_attachment
+sysctl_attachment
+sysfs_attachment
+udev_attachment
+lsmod_attachment
+acpi_sleep_attachment
+info/hdparm
+info/hdparm_.*.txt
+installer_debug.gz
+info/disk_partitions
+# Actual test cases
+__TC-002-0011-System_Log__
+TC-002-0011-001-System_Log_Entries

=== modified file 'debian/changelog'
--- debian/changelog	2013-11-05 20:04:15 +0000
+++ debian/changelog	2014-02-14 09:48:31 +0000
@@ -1,5 +1,10 @@
 checkbox (1.16.13~OCP) UNRELEASED; urgency=low
 
+  [ Sophia Wu ]
+  * jobs/TC-002-0011-System_Log.txt - added new IPMI out-of-band test for Intel ME
+  * examples/me.cfg - added new config file for DCMI/IPMI test
+   
+
   [ Jeff Marcom ]
   * Updated scripts/network script from lp:checkbox
 

=== modified file 'debian/checkbox.install'
--- debian/checkbox.install	2013-07-05 20:01:00 +0000
+++ debian/checkbox.install	2014-02-14 09:48:31 +0000
@@ -8,6 +8,7 @@
 usr/share/checkbox/examples/checkbox.ini
 usr/share/checkbox/examples/network.cfg
 usr/share/checkbox/examples/virtualization.cfg
+usr/share/checkbox/examples/me.cfg
 usr/share/checkbox/examples/org.freedesktop.policykit.checkbox.policy usr/share/polkit-1/actions/
 usr/share/checkbox/install/*
 usr/share/checkbox/jobs/*

=== modified file 'debian/checkbox.postinst'
--- debian/checkbox.postinst	2013-07-05 16:43:48 +0000
+++ debian/checkbox.postinst	2014-02-14 09:48:31 +0000
@@ -5,5 +5,6 @@
 
 cp /usr/share/checkbox/examples/network.cfg /etc/checkbox.d/
 cp /usr/share/checkbox/examples/virtualization.cfg /etc/checkbox.d/
+cp /usr/share/checkbox/examples/me.cfg /etc/checkbox.d/
 
 #DEBHELPER#

=== added file 'examples/me.cfg'
--- examples/me.cfg	1970-01-01 00:00:00 +0000
+++ examples/me.cfg	2014-02-14 09:48:31 +0000
@@ -0,0 +1,15 @@
+#Please config ME IP of server that you want to test
+#for example, if ME ip of your server is 10.0.0.1, 
+#then you must replace x.x.x.x to 10.0.0.1 in [Targets]
+
+[Targets]
+Target1: x.x.x.x
+
+#please config ME account(USER/PASSWORD) of server that you want to test
+#for example, if ME account of your server is ADMIN/ADMIN, 
+#then you must replace OCP/OCP to ADMIN/ADMIN in [Account]
+
+[Account]
+USER: OCP
+PASSWORD: OCP
+

=== added file 'jobs/TC-002-0011-System_Log.txt'
--- jobs/TC-002-0011-System_Log.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-002-0011-System_Log.txt	2014-02-14 09:48:31 +0000
@@ -0,0 +1,11 @@
+plugin: shell
+name: TC-002-0011-001-System_Log_Entries
+requires: package.name == 'ipmitool'
+user: root
+command: ipmi_sel_entries
+description: 
+  1. Use ipmitool to collect event log information
+  2. Calculate entries number of system event log
+  3. Criteria: the A log must be capable of saving at least 256 entries
+
+ 

=== added file 'scripts/ipmi_sel_entries'
--- scripts/ipmi_sel_entries	1970-01-01 00:00:00 +0000
+++ scripts/ipmi_sel_entries	2014-02-14 09:48:31 +0000
@@ -0,0 +1,147 @@
+#!/usr/bin/env python3
+"""
+Copyright (C) 2010-2013 by Cloud Computing Center for Mobile Applications 
+Industrial Technology Research Institute
+
+File Name
+  ipmi_sel_entries
+  1. Use ipmitool to collect event log information
+  2. Calculate entries number of system event log
+  3. Criteria: the A log must be capable of saving at least 256 entries
+
+Description
+  Use ipmitool to get system event log info and calculate whether 
+  total number of entries is more than 256.
+
+Authors
+  Sophia Wu <[e-mail]>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 3,
+as published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+import sys
+import time
+import shlex
+import re
+import subprocess
+from subprocess import (
+    CalledProcessError,
+    check_call,
+    check_output
+)
+from argparse import (
+    ArgumentParser,
+    RawTextHelpFormatter
+)
+import configparser
+
+#Retrieve entries number that already used
+def get_entry_used(output):
+    entry_used = re.search(r'Entries\D*([0-9 ]+)', output)
+    return entry_used.group(1).strip()
+
+#Retrieve free entries that available
+def get_entry_free(output):
+    entry_free = re.search(r'# Free Units\D*([0-9 ]+)', output)
+    return entry_free.group(1).strip()
+
+#Use ipmitool to retrieve sel information
+def get_sel_info(host_ip, user, password):
+    cmd = 'ipmitool -H {} -U {} -P {} sel'.format(host_ip, user, password)
+    ipmi_sel_return = check_output(shlex.split(cmd), universal_newlines=True)
+    return ipmi_sel_return
+
+def sel_entry_test(args):
+
+    #Default config file to config requirement info for DCMI in-band/out-of-band access 
+    DEFAULT_CFG = "/etc/checkbox.d/me.cfg"
+    if not "config" in vars(args):
+        config_file = DEFAULT_CFG
+    else:
+        config_file = args.config
+
+    config = configparser.RawConfigParser()
+
+    try:
+        config.readfp(open(config_file))
+    except IOError:
+        print("No config file found")
+        return 10
+    
+    # Acquire ME IP/Credential parameters from config file
+    try:
+        targets_options = config.options('Targets')
+        targets_list = []
+        for target_key in targets_options:
+            targets_list.append(config.get('Targets', target_key))
+        if not targets_list:
+            print("Invalid or Empty targets")
+            return 20
+    except configparser.Error:
+        print("Invalid or Empty targets")
+        return 30
+
+    try:
+        user_value = config.get('Account', 'USER')
+        passwd_value = config.get('Account', 'PASSWORD')
+    except configparser.Error:
+        print("Invalid or Empty credential info")
+        return 40
+
+    for target in targets_list:
+
+        if not target or not user_value or not passwd_value:
+            print("Require Taget IP, Account(USER/PASSWORD) for DCMI out-of-band access")
+            return 50
+        else:
+            print("SUT =", target)
+
+            try:
+                sel_return = get_sel_info(target, user_value, passwd_value)
+                time.sleep(5)
+            except CalledProcessError as command_exception:
+                print("Failed executing ipmi command, Reason: %s." %(command_exception))
+                return 60
+
+            try:
+                print("Used Entry:", int(get_entry_used(sel_return)))
+                print("Free Entry:", int(get_entry_free(sel_return)))
+            except:
+                print("Error to parse SEL info")
+                return 70
+
+            try:
+                entry_space = int(get_entry_used(sel_return))+int(get_entry_free(sel_return))
+            except ArithmeticError:
+                print("Arithmetic Error")
+                return 80
+
+            if entry_space < 256:
+                print("Entries Number:", int(entry_space), "Entries Number is less than 256")
+                return 90
+            else:
+                print("Entries Number of SUT is more than 256.")
+    return 0
+
+def main():
+
+    intro_message = "Default config location is /etc/checkbox.d/me.cfg"
+    parser = ArgumentParser(description=intro_message,
+                            formatter_class=RawTextHelpFormatter)
+    parser.add_argument('--config', default="/etc/checkbox.d/me.cfg",
+                        help="Supply config file for getting default credential")
+    args = parser.parse_args()
+    return sel_entry_test(args)
+
+if __name__ == "__main__":
+    sys.exit(main())