Merge lp:~nelson-chu/opencompute/add-ocp-certified-jobs into lp:opencompute/checkbox

[Nelson Chu]

[Jeff Lane  (bladernr)]

Hi Nelson...

I really hate to do this, but I'm going to reject this Merge Proposal. Here's why:

Please, PLEASE make several, smaller, targeted merge requests rather than one large 1500 line merge. The only time a merge should have that many lines of changed code is if it adds or changes a very large chunk of code, not adding so many small, individual files.

Smaller merge requests make it a lot easier to review, make suggestions and get things merged in a more timely fashion.

I would suggest breaking this merge request up into smaller chunks divided based on testing topic. So, for example submit one merge request with your proposed changes for:
TC-001-0001-CPU_Memory
whcih should include the whitelist changes, the added scripts, added/modified job files, etc.

Then do the same in different merge requests for:
TC-001-0002-Platform_Controller_Hub
TC-001-0003-BIOS_setup_menu
TC-001-0004-BIOS_remote_update
TC-001-0005-BIOS_event_log
and so on.

Now, a word about job naming and whitelists that apply across all these changes:

1: Dont put whitespace (space, tab) in the job name.
    Rather than TC-001-0001-001 CPU information, you should use something like these:
        TC-001-0001-001-CPU_information
        TC-001-0001/001_CPU_information
2: I don't know that it will hurt, but try to remove any blank lines in the whitelist file, the sections are identifiable by the __SUITENAME__ items on the list, so blank lines are not necessary.
3: You CAN, however, add comments preceding them with the # character
4: remember that every job called __SUITENAME__ must exist in the local.txt.in file (see that file for examples)
    That job is a special job that causes the suitename.txt.in file to be read in when checkbox launches.

Beyond those suggestions, We'll deal with individual problems in the smaller merge requests I've asked for.

Also, given that you are new to this process, using smaller merge requests will make it a LOT easier for me to help show you through the process as well as acquaint you with the way checkbox works and how the code needs to look.

review: Disapprove

Unmerged revisions

2169. By Nelson Chu on 2013-11-25

Add OCP certified jobs

=== added file 'data/whitelists/opencompute-certify-local.whitelist'
--- data/whitelists/opencompute-certify-local.whitelist	1970-01-01 00:00:00 +0000
+++ data/whitelists/opencompute-certify-local.whitelist	2013-11-25 07:01:57 +0000
@@ -0,0 +1,100 @@
+# Resource Jobs
+block_device
+cdimage
+cpuinfo
+device
+dmi
+dpkg
+efi
+environment
+gconf
+lsb
+meminfo
+module
+optical_drive
+package
+sleep
+uname
+#Info attachment jobs
+__info__
+cpuinfo_attachment
+dmesg_attachment
+dmi_attachment
+dmidecode_attachment
+efi_attachment
+lspci_attachment
+lshw_attachment
+mcelog_attachment
+meminfo_attachment
+modprobe_attachment
+modules_attachment
+sysctl_attachment
+sysfs_attachment
+udev_attachment
+lsmod_attachment
+acpi_sleep_attachment
+info/hdparm
+info/hdparm_.*.txt
+installer_debug.gz
+info/disk_partitions
+# Actual test cases
+__TC-001-0001-CPU_Memory__
+TC-001-0001-001 CPU information
+TC-001-0001-002 CPU topology
+TC-001-0001-003 Single Processor Mode
+TC-001-0001-004 Intel QPI
+TC-001-0001-005 Memory Information
+
+__TC-001-0002-Platform_Controller_Hub__
+TC-001-0002-001 SATA port
+TC-001-0002-002 USB 2.0
+TC-001-0002-003 Mini-SAS port
+
+__TC-001-0003-BIOS_setup_menu__
+TC-001-0003-001 BIOS setup menu
+
+__TC-001-0004-BIOS_remote_update__
+TC-001-0004-001 Scenario 1
+TC-001-0004-002 Scenario 2
+TC-001-0004-003 Scenario 3
+
+__TC-001-0005-BIOS_event_log__
+TC-001-0005-001 BIOS event log
+
+__TC-002-0001-Initial_machine_provisioning__
+TC-002-0001-001 Inventory information
+TC-002-0001-002 IP lease information
+TC-002-0001-003 Monitor attachment
+
+__TC-002-0004-Boot_order_PXE__
+TC-002-0004-001 Boot Order / PXE
+
+__TC-002-0005-Remote_serial_console__
+TC-002-0005-001 Enable SOL
+TC-002-0005-002 SOL operation
+
+__TC-003-0001-Hardware_information__
+TC-003-0001-001 CPU information
+TC-003-0001-002 Memory information
+TC-003-0001-003 Disk information
+TC-003-0001-004 BIOS information
+TC-003-0001-005 ME information
+TC-003-0001-006 NIC information
+TC-003-0001-007 RAID information
+TC-003-0001-008 HBA information
+
+___TC-003-0002-Idle_test__
+TC-003-0002-001 12 hours idle
+
+__TC-003-0003-Network_performance__
+TC-003-0003-001 Network performance
+
+__TC-003-0004-Disk_performance__
+TC-003-0004-001 Disk performance
+
+__TC-004-0001-Knox_testing__
+TC-004-0001-001-RAID_card_information
+TC-004-0002-001-Get_all_HDD
+TC-004-0003-001-Build_raid_6
+TC-004-0004-001-Read/Write_file
+TC-004-0005-001-Rebuild_Raid

=== added file 'jobs/TC-001-0001-CPU_Memory.txt'
--- jobs/TC-001-0001-CPU_Memory.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-001-0001-CPU_Memory.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,25 @@
+plugin: shell
+name: TC-001-0001-001 CPU information
+requires: package.name == 'lshw'
+command: cpu_info
+description:
+ 1. Gathering CPU information by using lshw command.
+ 2. Output CPU model and L1, L2, L3 cache size.
+ 3. Pass criteria: CPU model should belong to Intel Xeon processor E5-2600 product family and Last Level Cache (LLC) should be larger than 20MB
+
+plugin: shell
+name: TC-001-0001-002 CPU topology
+command: processor_topology
+description:
+ 1. Gathering CPU information by using lscpu command.
+ 2. Output the total number of CPU, the number of Thread per core, the number of Core per Socket, and the number of Socket.
+ 3. Pass criteria: It should be 8 cores per CPU and 2 Threads per core (that is, up to 16 threads with Hyper-Threading Technology per CPU).
+
+plugin: shell
+name: TC-001-0001-003 Memory Information
+requires: package.name == 'lshw'
+command: memory_info
+description:
+ 1. Gathering memory information by using lshw command.
+ 2. Output memory module, vendor, size and slot.
+ 3. Pass criteria: 2 DDR3 slots per channel per processor (total of 16 DIMMs on the motherboard).

=== added file 'jobs/TC-001-0002-Platform_Controller_Hub.txt'
--- jobs/TC-001-0002-Platform_Controller_Hub.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-001-0002-Platform_Controller_Hub.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,11 @@
+plugin: shell
+name: TC-001-0002-001 SATA port
+command: echo "manual"; exit 1
+description:
+ individual SATA 6Gps ports from SATA port 0/1.
+
+plugin: shell
+name: TC-001-0002-002 USB 2.0
+command: echo "manual"; exit 1
+description:
+ USB 2.0 ports (2 designed in this board: one front connector, one vertical onboard)

=== added file 'jobs/TC-001-0003-BIOS_setup_menu.txt'
--- jobs/TC-001-0003-BIOS_setup_menu.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-001-0003-BIOS_setup_menu.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,5 @@
+plugin: shell
+name: TC-001-0003-001 BIOS setup menu
+command: echo "manual"; exit 1
+description:
+ BIOS setup menu.

=== added file 'jobs/TC-001-0004-BIOS_remote_update.txt'
--- jobs/TC-001-0004-BIOS_remote_update.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-001-0004-BIOS_remote_update.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,17 @@
+plugin: shell
+name: TC-001-0004-001 Scenario 1
+command: echo "Scenario 1: Sample/Audit BIOS settings."; echo "Not implemented yet"; exit 1
+description:
+ Scenario 1: Sample/Audit BIOS settings.
+
+plugin: shell
+name: TC-001-0004-002 Scenario 2
+command: echo "Scenario 2: Update BIOS with pre-configured set of BIOS settings."; echo "Not implemented yet";exit 1
+description:
+ Scenario 2: Update BIOS with pre-configured set of BIOS settings.
+
+plugin: shell
+name: TC-001-0004-003 Scenario 3
+command: echo "Scenario 3: BIOS/firmware update with a new revision."; echo "Not implemented yet"; exit 1
+description:
+ Scenario 1: BIOS/firmware update with a new revision.

=== added file 'jobs/TC-001-0005-BIOS_event_log.txt'
--- jobs/TC-001-0005-BIOS_event_log.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-001-0005-BIOS_event_log.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,5 @@
+plugin: shell
+name: TC-001-0005-001 BIOS event log
+command: echo "manual"; exit 1
+description:
+ BIOS event log. Logged errors and error threshold setting.

=== added file 'jobs/TC-002-0001-Initial_machine_provisioning.txt'
--- jobs/TC-002-0001-Initial_machine_provisioning.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-002-0001-Initial_machine_provisioning.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,17 @@
+plugin: shell
+name: TC-002-0001-001 Inventory information
+command: echo "MAC address and/or other inventory information feeds/bar codes from the machine's manufacturer."; echo "Not implemented yet"; exit 1
+description:
+ MAC address and/or other inventory information feeds/bar codes from the machine's manufacturer.
+
+plugin: shell
+name: TC-002-0001-002 IP lease information
+command: echo "Searching through IP lease information to find machines that have gotten a DHCP leases for their management interfaces."; echo "Not implemented yet"; exit 1
+description:
+ Searching through IP lease information to find machines that have gotten a DHCP leases for their management interfaces.
+
+plugin: shell
+name: TC-002-0001-003 Monitor attachment
+command: echo "manual"; exit 1
+description:
+ Attach KVM to setup machine.

=== added file 'jobs/TC-002-0004-Boot_order_PXE.txt'
--- jobs/TC-002-0004-Boot_order_PXE.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-002-0004-Boot_order_PXE.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,5 @@
+plugin: shell
+name: TC-002-0004-001 Boot Order / PXE
+command: echo "Not implemented yet"; exit 1
+description:
+ Boot form PXE.

=== added file 'jobs/TC-002-0005-Remote_serial_console.txt'
--- jobs/TC-002-0005-Remote_serial_console.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-002-0005-Remote_serial_console.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,14 @@
+plugin: shell
+name: TC-002-0005-001 Enable SOL
+command: echo "manual"; exit 1
+description:
+ SOL BIOS setup.
+
+plugin: shell
+name: TC-002-0005-002 SOL operation
+command: echo "manual"; exit 1
+description:
+ 1. Remote control the SUT by using IPMI SOL.
+ 2. SOL that will allow to operate the SUT on: pre-boot process, BIOS setup, boot process, access to the installed OS
+ 3. Pass criteria: Successfully operate the SUT in the scenarios listed above.
+

=== added file 'jobs/TC-003-0001-Hardware_information.txt'
--- jobs/TC-003-0001-Hardware_information.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-003-0001-Hardware_information.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,54 @@
+plugin: shell
+name: TC-003-0001-001 CPU information
+command: echo "refer to TC-001-0001-001"
+description:
+ To verify CPU information with specification. Show CPU type and speed.
+
+plugin: shell
+name: TC-003-0001-002 Memory information
+command: echo "refer to TC-001-0001-003"
+description:
+ To verify memory information with specification. Show memory vendor, location and size.
+
+plugin: shell
+name: TC-003-0001-003 Disk information
+requires: package.name == 'lshw'
+command: disk_info
+description:
+ 1. Gathering disk information by using lshw command.
+ 2. Output Disk type, vendor, product, capacity.
+ 3. Pass criteria: If catch the information then pass.
+
+plugin: shell
+name: TC-003-0001-004 BIOS information
+requires: package.name == 'lshw'
+command: bios_info
+description:
+ 1. Gathering BIOS information by using lshw command.
+ 2. Output BIOS vendor, version and release date.
+ 3. Pass criteria: If catch the information then pass.
+
+plugin: shell
+name: TC-003-0001-005 ME information
+command: echo "Show Intel ME information."; echo "Refer to DCMI job."
+description:
+ Show Intel ME information.
+
+plugin: shell
+name: TC-003-0001-006 NIC information
+command: network_device_info
+description:
+ 1. Gathering NIC information by using udevadm command.
+ 2. Output Interface, product, vendor, driver, device path.
+ 3. Pass criteria: If catch the information then pass.
+
+plugin: shell
+name: TC-003-0001-007 RAID card information
+requires:
+ package.name == 'megacli'
+ package.name == 'megactl'
+command: raid_info
+description:
+ 1. Gathering LSI RAID card information by using megasasctl command.
+ 2. Show adapter, product name, memory, BBU, serial no.
+ 3. Pass criteria: If catch the information then pass.

=== added file 'jobs/TC-003-0002-Idle_test.txt'
--- jobs/TC-003-0002-Idle_test.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-003-0002-Idle_test.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,5 @@
+plugin: shell
+name: TC-003-0002-001 12 hours idle
+command: echo "Leave server sitting idle for 12 hours. Verify after 12 hours that the system is still stable."; echo "Not implemented yet"; exit 1
+description:
+ Leave server sitting idle for 12 hours. Verify after 12 hours that the system is still stable.

=== added file 'jobs/TC-003-0003-Network_performance.txt'
--- jobs/TC-003-0003-Network_performance.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-003-0003-Network_performance.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,5 @@
+plugin: shell
+name: TC-003-0003-001 Network performance
+command: echo "Tx/Rx Gbps for 10G NIC spec. >9.5 Gbps"; echo "Not implemented yet"; exit 1
+description:
+ Tx/Rx Gbps for 10G NIC spec. > 9.5 Gbps

=== added file 'jobs/TC-003-0004-Disk_performance.txt'
--- jobs/TC-003-0004-Disk_performance.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-003-0004-Disk_performance.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,6 @@
+plugin: shell
+name: TC-003-0004-001 Disk performance
+command: echo "SMART threshold, grown defect list glist = reallocated sectors = 200 for 4TB, glist = 100 for 2TB for all vendors.
+SMART HDD temp = 60C."; echo "Not implemented yet"; exit 1
+description:
+ Test disk performance.

=== added file 'jobs/TC-004-0001-Knox_testing.txt'
--- jobs/TC-004-0001-Knox_testing.txt	1970-01-01 00:00:00 +0000
+++ jobs/TC-004-0001-Knox_testing.txt	2013-11-25 07:01:57 +0000
@@ -0,0 +1,45 @@
+plugin: shell
+name: TC-004-0001-001-RAID_card_information
+command: echo "refer to TC-003-0001-007"
+description:
+ RAID card information
+
+plugin: shell
+name: TC-004-0002-001-Get_all_HDD
+requires:
+ package.name == 'megacli'
+ package.name == 'megactl'
+depends: TC-004-0001-001 RAID card information
+command: raid_hdd_info
+description:
+ Get knox all HDD.
+
+plugin: shell
+name: TC-004-0003-001-Build_raid_6
+requires:
+ package.name == 'megacli'
+ package.name == 'megactl'
+depends: TC-004-0002-001-Get_all_HDD
+command: create_raid6
+description:
+ Build Raid 6
+
+plugin: shell
+name: TC-004-0004-001-Read/Write_file
+depends: TC-004-0003-001-Build_raid_6
+description: Test RAID Read/Write file
+user: root
+command:
+ exit 1
+
+plugin: shell
+name: TC-004-0005-001-Rebuild_Raid
+requires:
+ package.name == 'megacli'
+ package.name == 'megactl'
+depends: TC-004-0004-001-Read/Write_file
+environ: CHECKBOX_DATA
+user: root
+command: rebulid_raid -l $CHECKBOX_DATA/rebulid_raid.log
+description:
+ Re-build Raid

=== added file 'scripts/bios_info'
--- scripts/bios_info	1970-01-01 00:00:00 +0000
+++ scripts/bios_info	2013-11-25 07:01:57 +0000
@@ -0,0 +1,63 @@
+#!/usr/bin/env python3
+"""
+Copyright (C) 2010-2013 by Cloud Computing Center for Mobile Applications
+Industrial Technology Research Institute
+
+bios_info
+  Gathering BIOS information by using lshw command.
+  And output BIOS vendor, version and release date.
+
+Authors
+  Nelson Chu <[e-mail]>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 3,
+as published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+
+import os
+import sys
+import xml.etree.ElementTree as ET
+from subprocess import check_output
+
+
+def main():
+    attribute = ['vendor', 'version', 'date']
+    command = 'lshw -xml'
+    with open(os.devnull, "w") as NULL:
+       hwinfo_xml = check_output(command, stderr=NULL, shell=True)
+    root = ET.fromstring(hwinfo_xml)
+
+    # Parser lshw XML for gather BIOS information.
+    bios_info = root.findall(".//node[@id='firmware']")
+
+    if not bios_info:
+        print("Can not parser any BIOS information.")
+        return 10
+
+    for bios in bios_info:
+        for attr in attribute:
+            if bios.find(attr) is None:
+                print(("Can not found BIOS %s") %attr)
+                return 20
+
+        for attr in attribute:
+            if attr == 'date':
+                print(('release date=%s.') %(bios.find(attr).text)),
+                continue
+            print(('%s=%s,') %(attr, bios.find(attr).text)),
+        print()
+
+    return 0
+
+if __name__ == '__main__':
+    sys.exit(main())

=== added file 'scripts/cpu_info'
--- scripts/cpu_info	1970-01-01 00:00:00 +0000
+++ scripts/cpu_info	2013-11-25 07:01:57 +0000
@@ -0,0 +1,77 @@
+#!/usr/bin/env python3
+"""
+Copyright (C) 2010-2013 by Cloud Computing Center for Mobile Applications 
+Industrial Technology Research Institute
+
+cpu_info
+  Gathering CPU information by using lshw command.
+  And output CPU model and L1, L2, L3 cache size.
+  L3 Cache should be larger than 20MB.
+
+Authors
+  Nelson Chu <[e-mail]>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 3,
+as published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+
+import os
+import re
+import sys
+import xml.etree.ElementTree as ET
+from subprocess import check_output
+
+def main():
+    command = 'lshw -xml'
+    with open(os.devnull, "w") as NULL:
+       hwinfo_xml = check_output(command, stderr=NULL, shell=True)
+    root = ET.fromstring(hwinfo_xml)
+
+    # Parser lshw XML for gather processor information.
+    processor = root.findall(".//product/..[@class='processor']")
+
+    if not processor:
+        print("Can not parser any processor information.")
+        return 10
+
+    for cpu in processor:
+        if cpu.find('product') is None:
+            print("Can not found processor product.")
+            return 20
+        print(cpu.find('product').text)
+
+        cache_list = cpu.findall(".//size/..[@class='memory']")
+        if not cache_list:
+            print("Can not found any CPU cache.")
+            return 30
+
+        for cache in cache_list:
+            if cache.find('size') is None or cache.find('slot') is None:
+                print("Can not access Last Level Cache (LLC).")
+                return 40
+
+            cache_size = int(cache.find('size').text) / 1024
+            if cache_size > 1024:
+                cache_size = cache_size / 1024
+                print(('%s %d MB') %(cache.find('slot').text, cache_size))
+                if re.search('L3', cache.find('slot').text):
+                    if cache_size < 20:
+                        print('L3 cache size less than 20MB.')
+                        return 50
+            else:
+                print(('%s %d KB') %(cache.find('slot').text, cache_size))
+
+    return 0
+
+if __name__ == '__main__':
+    sys.exit(main())

=== added file 'scripts/create_raid6'
--- scripts/create_raid6	1970-01-01 00:00:00 +0000
+++ scripts/create_raid6	2013-11-25 07:01:57 +0000
@@ -0,0 +1,121 @@
+#!/usr/bin/env python3
+"""
+Copyright (C) 2010-2013 by Cloud Computing Center for Mobile Applications
+Industrial Technology Research Institute
+
+create_raid6
+  Using megacli command build RAID 6 on knox.
+
+  Warning: Make sure there is no RAID configuration in LSI RAID card 
+           before this program is executed.
+
+Authors
+  Nelson Chu <[e-mail]>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 3,
+as published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+
+import os
+import re
+import sys
+from subprocess import check_output, check_call
+
+
+def get_adapter():
+    """
+    Gather all adapter number if there are multiple RAID card.
+    """
+
+    command = 'megacli -CfgDsply -Aall|grep Adapter'
+    adapter_list = []
+
+    adapter_info = check_output(command, shell=True)
+    adapter_info = adapter_info.decode('utf-8')
+
+    for adapter in adapter_info.strip().split('\n'):
+        adapter_list.append(adapter.strip().split(' ')[-1])
+    return adapter_list
+
+
+def get_all_disk(adapter):
+    """
+    Gather all disks Enclosure and Slot number, and make a 
+    Enclosure:Slot pair list.
+    """
+
+    command = 'megacli -PDList -A{0}'.format(adapter) \
+        + '|grep -E "Enclosure Device ID|Slot Number"'
+
+    disk_list = []
+    disk_info = check_output(command, shell=True)
+    disk_info = disk_info.decode('utf-8')
+
+    for line in disk_info.strip().split('\n'):
+        if line.startswith('Enclosure'):
+            match = re.search(r'\d+', line)
+            disk_list.append(match.group(0))
+        if line.startswith('Slot'):
+            match = re.search(r'\d+', line)
+            enclosure = disk_list.pop()
+            E_S = '%s:%s' %(enclosure, match.group(0))
+            disk_list.append(E_S)
+    return disk_list
+
+
+def build_raid(adapter, disk_list):
+    """
+    Use all disk creatd RAID 6 and set last disk as hot spare.
+    """
+
+    # use the last disk as hot spare
+    spare = disk_list.pop()
+
+    disk = ','.join(disk_list)
+    command = 'megacli -CfgLDadd -r6 [{0}] WB Direct -Hsp[{1}] -a{2}'.format(
+        disk, spare, adapter)
+
+    check_call(command, shell=True)
+
+
+def main():
+    knox_dict = {}
+
+    # get RAID card adapter number
+    try:
+        adapter_list = get_adapter()
+    except Exception as e:
+        print('Error: %s' %e)
+        return 10
+
+    # get all disk in RAID card
+    try:
+        for adapter in adapter_list:
+            knox_dict[adapter] = get_all_disk(adapter)
+    except Exception as e:
+        print('Error: %s' %e)
+        return 20
+
+    # use all HDD build RAID
+    try:
+        for adapter, disk_list in knox_dict.items():
+            build_raid(adapter, disk_list)
+    except Exception as e:
+        print('Error: Knox build raid 6 failed.')
+        return 30
+
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main())

=== added file 'scripts/disk_info'
--- scripts/disk_info	1970-01-01 00:00:00 +0000
+++ scripts/disk_info	2013-11-25 07:01:57 +0000
@@ -0,0 +1,66 @@
+#!/usr/bin/env python3
+"""
+Copyright (C) 2010-2013 by Cloud Computing Center for Mobile Applications
+Industrial Technology Research Institute
+
+disk_info
+  Gathering disk information by using lshw command.
+  And output disk type, vendor, product, capacity.
+
+Authors
+  Nelson Chu <[e-mail]>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 3,
+as published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+
+import sys
+import xml.etree.ElementTree as ET
+from subprocess import Popen, PIPE
+
+def main():
+    attribute = ['description', 'vendor', 'product', 'size']
+    command = 'lshw -xml'
+    hwinfo_xml = Popen(command, stdout=PIPE, stderr=PIPE,
+                       shell=True).communicate()[0]
+    root = ET.fromstring(hwinfo_xml)
+
+    # Parser lshw XML for gather disk information.
+    disk_list = root.findall(".//node[@class='disk']")
+
+    if not disk_list:
+        print("Can not parser any disk information.")
+        return 10
+
+    for disk in disk_list:
+        for attr in attribute:
+            if disk.find(attr) is None:
+                print(("Can not found disk %s") %attr)
+                return 20
+
+        disk_size = int(disk.find('size').text) / (1000**3)
+        for attr in attribute:
+            if attr == 'description':
+                print(('Type=%s,') %disk.find(attr).text),
+                continue
+            elif attr == 'size':
+                print(('%s=%dGB.') %(attr, disk_size)),
+                continue
+            else:
+                print(('%s=%s,') %(attr, disk.find(attr).text)),
+        print()
+
+    return 0
+
+if __name__ == '__main__':
+    sys.exit(main())

=== added file 'scripts/memory_info'
--- scripts/memory_info	1970-01-01 00:00:00 +0000
+++ scripts/memory_info	2013-11-25 07:01:57 +0000
@@ -0,0 +1,65 @@
+#!/usr/bin/env python3
+"""
+Copyright (C) 2010-2013 by Cloud Computing Center for Mobile Applications
+Industrial Technology Research Institute
+
+memory_info
+  Gathering memory information by using lshw command.
+  And output memory module, vendor, size and slot.
+
+Authors
+  Nelson Chu <[e-mail]>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 3,
+as published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+
+import sys
+import xml.etree.ElementTree as ET
+from subprocess import Popen, PIPE
+
+def main():
+    attribute = ['description', 'vendor', 'slot', 'size']
+    command = 'lshw -xml'
+    hwinfo_xml = Popen(command, stdout=PIPE, stderr=PIPE,
+                       shell=True).communicate()[0]
+    root = ET.fromstring(hwinfo_xml)
+
+    # Parser lshw XML for gather memory information.
+    memory_list = root.findall(".//clock/..[@class='memory']")
+
+    if not memory_list:
+        print("Can not parser any memory information.")
+        return 10
+
+    for dimm in memory_list:
+        for attr in attribute:
+            if dimm.find(attr) is None:
+                print(("Can not found memory %s") %attr)
+                return 20
+
+        memory_size = int(dimm.find('size').text) / (1024**3)
+        for attr in attribute:
+            if attr == 'description':
+                print(('%s,') %(dimm.find(attr).text)),
+                continue
+            elif attr == 'size':
+                print(('%s=%dGB.') %(attr, memory_size)),
+                continue
+            print(('%s=%s,') %(attr, dimm.find(attr).text)),
+        print()
+
+    return 0
+
+if __name__ == '__main__':
+    sys.exit(main())

=== added file 'scripts/processor_topology'
--- scripts/processor_topology	1970-01-01 00:00:00 +0000
+++ scripts/processor_topology	2013-11-25 07:01:57 +0000
@@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+'''
+cpu_topology
+Written by Jeffrey Lane <[e-mail]>
+'''
+import sys
+import os
+from subprocess import check_output
+
+class proc_cpuinfo():
+    '''
+    Class to get and handle information from /proc/cpuinfo
+    Creates a dictionary of data gleaned from that file.
+    '''
+    def __init__(self):
+        self.cpuinfo = {}
+        cpu_fh = open('/proc/cpuinfo', 'r')
+        try:
+            temp = cpu_fh.readlines()
+        finally:
+            cpu_fh.close()
+
+        for i in temp:
+            if i.startswith('processor'):
+                key = 'cpu' + (i.split(':')[1].strip())
+                self.cpuinfo[key] = {'core_id':'', 'physical_package_id':''}
+            elif i.startswith('core id'):
+                self.cpuinfo[key].update({'core_id': i.split(':')[1].strip()})
+            elif i.startswith('physical id'):
+                self.cpuinfo[key].update({'physical_package_id':
+                                          i.split(':')[1].strip()})
+            else:
+                continue
+
+
+class sysfs_cpu():
+    '''
+    Class to get and handle information from sysfs as relates to CPU topology
+    Creates an informational class to present information on various CPUs
+    '''
+
+    def __init__(self, proc):
+        self.syscpu = {}
+        self.path = '/sys/devices/system/cpu/' + proc + '/topology'
+        items = ['core_id', 'physical_package_id']
+        for i in items:
+            syscpu_fh = open(os.path.join(self.path, i), 'r')
+            try:
+                self.syscpu[i] = syscpu_fh.readline().strip()
+            finally:
+                syscpu_fh.close()
+
+
+def compare(proc_cpu, sys_cpu):
+    cpu_map = {}
+    '''
+    If there is only 1 CPU the test don't look for core_id
+    and physical_package_id because those information are absent in
+    /proc/cpuinfo on singlecore system
+    '''
+    for key in proc_cpu.keys():
+        if 'cpu1' not in proc_cpu:
+            cpu_map[key] = True
+        else:
+            for subkey in proc_cpu[key].keys():
+                if proc_cpu[key][subkey] == sys_cpu[key][subkey]:
+                    cpu_map[key] = True
+                else:
+                    cpu_map[key] = False
+    return cpu_map
+
+
+def main():
+    cpuinfo = proc_cpuinfo()
+    sys_cpu = {}
+    keys = cpuinfo.cpuinfo.keys()
+    for k in keys:
+        sys_cpu[k] = sysfs_cpu(k).syscpu
+    cpu_map = compare(cpuinfo.cpuinfo, sys_cpu)
+    if False in cpu_map.values() or len(cpu_map) < 1:
+        print("FAIL: CPU Topology is incorrect", file=sys.stderr)
+        print("-" * 52, file=sys.stderr)
+        print("{0}{1}".format("/proc/cpuinfo".center(30), "sysfs".center(25)),
+                file=sys.stderr)
+        print("{0}{1}{2}{3}{1}{2}".format(
+                "CPU".center(6),
+                "Physical ID".center(13),
+                "Core ID".center(9),
+                "|".center(3)), file=sys.stderr)
+        for key in sorted(sys_cpu.keys()):
+            print("{0}{1}{2}{3}{4}{5}".format(
+                  key.center(6),
+                  cpuinfo.cpuinfo[key]['physical_package_id'].center(13),
+                  cpuinfo.cpuinfo[key]['core_id'].center(9),
+                  "|".center(3),
+                  sys_cpu[key]['physical_package_id'].center(13),
+                  sys_cpu[key]['core_id'].center(9)), file=sys.stderr)
+        return 1
+    else:
+        # Output the total number of CPU, the number of Thread per core, 
+        # the number of Core per Socket, and the number of Socket.
+        # Revised by Nelson Chu <[e-mail]>
+        command = 'lscpu'
+
+        with open(os.devnull, "w") as NULL:
+            cpu_info = check_output(command, stderr=NULL, shell=True)
+
+        cpu_info = cpu_info.decode()
+
+        for cpu in cpu_info.split('\n'):
+            if cpu.startswith("CPU(s)"):
+                print(cpu + ',')
+            if cpu.startswith("Thread(s) per core"):
+                print(cpu + ',')
+            if cpu.startswith("Core(s) per socket"):
+                print(cpu + ',')
+            if cpu.startswith("Socket(s)"):
+                print(cpu + '.')
+
+        return 0
+
+if __name__ == '__main__':
+    sys.exit(main())

=== added file 'scripts/raid_hdd_info'
--- scripts/raid_hdd_info	1970-01-01 00:00:00 +0000
+++ scripts/raid_hdd_info	2013-11-25 07:01:57 +0000
@@ -0,0 +1,59 @@
+#!/usr/bin/env python3
+"""
+Copyright (C) 2010-2013 by Cloud Computing Center for Mobile Applications
+Industrial Technology Research Institute
+
+raid_hdd_info
+  Get all knox HDD information by using megasasctl command.
+  Show total HDD number in knox and check these disks status is ready or not.
+  Pass criteria: All knox disks information and status are correct.
+
+Authors
+  Nelson Chu <[e-mail]>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 3,
+as published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+
+import sys
+from subprocess import Popen, PIPE
+
+def main():
+    command = 'megasasctl |grep -E "a[0-9]+e[0-9]+s[0-9]+ +[0-9]+GiB"'
+    hdd_info = Popen(command, stdout=PIPE, stderr=PIPE,
+                     shell=True).communicate()[0]
+    hdd_info = hdd_info.decode('utf-8')
+    disk_number = 0
+    fail = 0
+
+    # Check is there has any disk.
+    if not hdd_info:
+        print('There is no disk in knox.')
+        return 10
+
+    for disk in hdd_info.strip().split('\n'):
+        disk_number = disk_number+1
+        # Check disk status is ready or not
+        if disk.strip().split(' ')[-1] != 'ready':
+            fail = fail+1
+
+    print(('Total %d HDDs in knox.') %disk_number)
+
+    if fail:
+        print(('There are %d disk status not in ready!') %fail)
+        return 20
+
+    return 0
+
+if __name__ == '__main__':
+    sys.exit(main())

=== added file 'scripts/raid_info'
--- scripts/raid_info	1970-01-01 00:00:00 +0000
+++ scripts/raid_info	2013-11-25 07:01:57 +0000
@@ -0,0 +1,49 @@
+#!/bin/bash
+#
+# Copyright (C) 2010-2013 by Cloud Computing Center for Mobile Applications
+# Industrial Technology Research Institute
+#
+# raid_info
+#   Gathering LSI RAID card information by using megasasctl command.
+#   Show adapter, product name, memory, BBU, serial no.
+#
+# Authors
+#   Nelson Chu <[e-mail]>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3,
+# as published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+
+check_return_code() {
+    if [ "${1}" -ne "0" ]; then
+        echo "ERROR: ${2}" >&2
+        exit ${1}
+    fi
+}
+
+show=`megacli -CfgDsply -Aall|grep -E \
+    'Adapter:|Product Name:|Memory:|BBU:|Serial No:'`
+check_return_code $? "There is no LSI MegaRAID SAS cards found."
+
+n=0
+echo "$show" | {
+    while IFS= read -r line; do 
+	echo $line"."
+	n=$(($n + 1))
+    done
+
+    if [ $n -ne "5" ]; then
+	check_return_code 10 "LSI RAID attributes are not correct."
+    fi
+
+    exit 0
+}

=== added file 'scripts/read_write_file'
--- scripts/read_write_file	1970-01-01 00:00:00 +0000
+++ scripts/read_write_file	2013-11-25 07:01:57 +0000
@@ -0,0 +1,177 @@
+#!/usr/bin/env python3
+"""
+Copyright (C) 2010-2013 by Cloud Computing Center for Mobile Applications
+Industrial Technology Research Institute
+
+read_write_file
+ Create one partition and format on device then test disk I/O.
+
+Authors
+  Nelson Chu <[e-mail]>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 3,
+as published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import os
+import re
+import sys
+import logging
+from subprocess import check_output, check_call, STDOUT
+from argparse import ArgumentParser, RawTextHelpFormatter
+
+log_formatter = '%(asctime)s [%(levelname)-4s] %(funcName)s: %(message)s'
+logging.basicConfig(level=logging.DEBUG,
+                    format=log_formatter,
+                    datefmt='%Y-%m-%d %H:%M:%S')
+
+
+def read_file_test(mount_point, file_size):
+    command = "dd if={0}/test_file of=/dev/null ".format(mount_point) \
+        + "bs=1M count={0} iflag=direct".format(file_size)
+
+    output = check_output(command, stderr=STDOUT, shell=True)
+    output = output.decode("utf-8")
+
+    match = re.search(r'\((.*)\)\D*(\d+\.\d+)\D*(\d+.*)', output)
+    if not match:
+        logging.error('Does not match patterns of read file test.')
+        return False
+
+    logging.info('Read file successful: %s in %s seconds, speed= %s.'
+                %match.group(1,2,3))
+    return True
+
+
+def write_file_test(mount_point, file_size):
+    command = "dd if=/dev/zero of={0}/test_file ".format(mount_point) \
+        + "bs=1M count={0} oflag=direct".format(file_size)
+
+    output = check_output(command, stderr=STDOUT, shell=True)
+    output = output.decode("utf-8")
+
+    match = re.search(r'\((.*)\)\D*(\d+\.\d+)\D*(\d+.*)', output)
+    if not match:
+        logging.error('Does not match patterns of write file test.')
+        return False
+
+    logging.info('Write file successful: %s in %s seconds, speed= %s.'
+                %match.group(1,2,3))
+    return True
+
+
+def umount_filesystem(mount_point):
+    umount_cmd = "umount {0}".format(mount_point)
+    with open(os.devnull, "w") as NULL:
+        check_call(umount_cmd, stdout=NULL, stderr=NULL, shell=True)
+    logging.debug('Umount %s directory successful.' %mount_point)
+    
+
+def mount_filesystem(device):
+    mount_point = '/mnt/' + os.path.basename(device)
+    if not os.path.exists(mount_point):
+        os.mkdir(mount_point)
+        logging.debug('Create %s directory successful!' %mount_point)
+    else:
+        if os.path.ismount(mount_point):
+            logging.debug('The %s directory is already mounted.' %mount_point)
+            umount_filesystem(mount_point)
+    
+    mount_cmd = "mount {0} {1}".format(device, mount_point)
+    with open(os.devnull, "w") as NULL:
+        check_call(mount_cmd, stdout=NULL, stderr=NULL, shell=True)
+    logging.debug('Mount %s on %s directory successful.' 
+                %(device, mount_point))
+    return mount_point
+
+
+def format_device(device):
+    command = "mkfs.ext4 -F {0}".format(device)
+    logging.debug('Format %s device beginning.' %device)
+    with open(os.devnull, "w") as NULL:
+        check_call(command, stdout=NULL, stderr=NULL, shell=True)
+    logging.debug('Format %s completed successfully.' %device)
+
+
+def run(device, file_size):
+
+    # format device
+    try:
+        format_device(device)
+    except Exception as e:
+        logging.error('%s' %e)
+        return 20
+
+    # mount device on system
+    try:
+        mount_point = mount_filesystem(device)
+    except Exception as e:
+        logging.error('%s' %e)
+        return 30
+
+    # write file test
+    try:
+        if not write_file_test(mount_point, file_size):
+            return 35
+    except Exception as e:
+        logging.error('%s' %e)
+        return 40
+
+    # read file test
+    try:
+        if not read_file_test(mount_point, file_size):
+            return 45
+    except Exception as e:
+        logging.error('%s' %e)
+        return 50
+
+    # umount device
+    try:
+        umount_filesystem(mount_point)
+    except Exception as e:
+        logging.error('%s' %e)
+        return 60
+
+    return 0
+
+
+def main():
+    description_text = 'read_write_file\n\tTest the device read/write file ' \
+        + 'correctly.\n\n\n\tWarning: This program will create one partition' \
+        + ' on device and format it.\n\t\t  Please make sure what you are ' \
+        + 'doing!\n\n\n\tRequirement: Give a device name on the system.'
+
+    parser = ArgumentParser(description=description_text,
+                            formatter_class=RawTextHelpFormatter)
+
+    parser.add_argument('-d', '--device', type=str, required=True,
+                      help=('The device name which used to read write test.\n'
+                            '[Example: sda]'))
+    parser.add_argument('-m', '--megabyte', type=int,
+                        default=5000,
+                        help=('The test file size. [Default: 5GB]'))
+
+    args = parser.parse_args()
+
+    device = args.device
+    if not device.startswith('/dev/'):
+        device = '/dev/' + device
+
+    if not os.path.exists(device):
+        parser.print_help()
+        return 10
+
+    return run(device, args.megabyte)
+
+
+if __name__ == '__main__':
+    sys.exit(main())

=== added file 'scripts/rebulid_raid'
--- scripts/rebulid_raid	1970-01-01 00:00:00 +0000
+++ scripts/rebulid_raid	2013-11-25 07:01:57 +0000
@@ -0,0 +1,277 @@
+#!/usr/bin/env python3
+"""
+Copyright (C) 2010-2013 by Cloud Computing Center for Mobile Applications
+Industrial Technology Research Institute
+
+rebulid_raid
+  Test RAID 6 auto rebuild feature in hot spare mode.
+
+Authors
+  Nelson Chu <[e-mail]>
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License version 3,
+as published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+
+import os
+import re
+import sys
+import time
+import random
+import logging
+from subprocess import check_output, check_call
+from argparse import ArgumentParser, RawTextHelpFormatter
+
+logFormatter = logging.Formatter(
+    "%(asctime)s [%(levelname)-5.5s] %(funcName)s: %(message)s", 
+    datefmt='%Y-%m-%d %H:%M:%S')
+logger = logging.getLogger()
+logger.setLevel(logging.DEBUG)
+
+
+def get_adapter():
+    """
+    Gather all adapter number if there are multiple RAID card.
+    """
+
+    command = 'megacli -CfgDsply -Aall|grep Adapter'
+    adapter_list = []
+
+    adapter_info = check_output(command, shell=True)
+    adapter_info = adapter_info.decode('utf-8')
+
+    for adapter in adapter_info.strip().split('\n'):
+        adapter_list.append(adapter.strip().split(' ')[-1])
+    logger.debug('adapter_list: %s' %adapter_list)
+    return adapter_list
+
+
+def get_all_disk(adapter):
+    """
+    Gather all disks Enclosure and Slot number, and make a 
+    Enclosure:Slot pair list.
+    """
+
+    command = ('megacli -PDList -A%s|grep -E "Enclosure Device ID|Slot Numbe"' 
+               %adapter)
+    disk_list = []
+    disk_info = check_output(command, shell=True)
+    disk_info = disk_info.decode('utf-8')
+
+    for line in disk_info.strip().split('\n'):
+        if line.startswith('Enclosure'):
+            match = re.search(r'\d+', line)
+            disk_list.append(match.group(0))
+        if line.startswith('Slot'):
+            match = re.search(r'\d+', line)
+            enclosure = disk_list.pop()
+            E_S = '%s:%s' %(enclosure, match.group(0))
+            disk_list.append(E_S)
+
+    logger.debug('adapter: %s, disk_list: %s' %(adapter, disk_list))
+    return disk_list
+
+
+def set_disk_offline(adapter, disk_list):
+    """
+    Use all disk creatd RAID 6 and set last disk as hot spare.
+    """
+
+    # Random offline one disk in disk list.
+    disk = disk_list[random.randint(0, len(disk_list)-1)]
+    command = ('megacli -PDOffline -PhysDrv [%s] -a%s' %(disk, adapter))
+    with open(os.devnull, "w") as NULL:
+        check_call(command, stdout=NULL, stderr=NULL, shell=True)
+    logger.info('Random offline one disk in disk list, Enclosure: %s, Slot: %s' 
+                %(disk.split(':')[0], disk.split(':')[1]))
+
+
+def find_hotspare_disk(adapter):
+    """
+    Find hotspare disk in adapter
+    """
+    command = "megasasctl |grep -E '^a%s.*hotspare'" %adapter
+    output = check_output(command, shell=True)
+    output = output.decode('utf-8')
+    hotspare = output.strip()
+
+    if len(hotspare.split('\n')) != 1:
+        logger.error('Hotspare disk number is not correct.' \
+                         + 'It can be only one hotspare in a ' \
+                         + 'adapter.')
+        return False
+
+    match = re.match(r'a(\d+)e(\d+)s(\d+)',hotspare)
+    if not match:
+        logger.error('Can not found enclosure and slot number of hotspare.')
+        return False
+
+    hotspare_E_S = "%s:%s" %(match.group(2), match.group(3))
+    logger.info('Found hotspare, Enclosure: %s, Slot: %s' 
+                %(match.group(2), match.group(3)))
+    return hotspare_E_S
+
+
+def check_rebuild_status(adapter, spare):
+    command = 'megacli -PDRbld -ShowProg -PhysDrv [%s] -A%s|grep -i rebuild' \
+        %(spare, adapter)
+    output = check_output(command, shell=True)
+    output = output.decode('utf-8')
+    logger.debug('%s' %output.strip())
+
+    match = re.search(r'Completed (\d+)% in (\d+) Minutes', output)
+    if not match:
+        return True
+    return False
+
+
+def all_rebuild_status(adapter_list, spare_dict):
+    for adapter in adapter_list:
+        if not check_rebuild_status(adapter, spare_dict[adapter]):
+            return False
+    return True
+
+
+def confirm_rebuild_status(adapter, spare):
+    enclosure, slot = spare.split(':')
+    command = ("megasasctl|grep -E '^a%se%ss%s'" %(adapter, enclosure, slot))
+    output = check_output(command, shell=True)
+    output = output.decode('utf-8')
+
+    match = re.search(r'online', output)
+    if not match:
+        logger.error('Hotspare status did not change to online. Adapter: %s, ' \
+                         +'Hotspare: [$s:$s]' %(adapter, enclosure, slot))
+        return False
+    return True
+
+
+def run(Adapter= None, Hsp=None, disk_list=None):
+    disk_dict = {}
+    spare_dict = {}
+    adapter_list = []
+    rebuild_status = {}
+
+    try:
+        if Adapter:
+            adapter_list.append(Adapter)
+        else:
+            adapter_list = get_adapter()
+    except Exception as e:
+        logger.error('%s' %e)
+        return 10
+
+    try:
+        if Adapter and disk_list:
+            disk_dict[Adapter] = disk_list
+        else:
+            for adapter in adapter_list:
+                disk_dict[adapter] = get_all_disk(adapter)
+    except Exception as e:
+        logger.error('%s' %e)
+        return 20
+
+    try:
+        if Adapter and Hsp:
+            spare_dict[Adapter] = Hsp
+        else:
+            for adapter in adapter_list:
+                spare_dict[adapter] = find_hotspare_disk(adapter)
+                if not spare_dict[adapter]:
+                    return 30
+    except Exception as e:
+        logger.error('%s' %e)
+        return 40
+
+    try:
+        for adapter in adapter_list:
+            # Remove hotspare disk from disk dict.
+            if spare_dict[adapter] in disk_dict[adapter]:
+                disk_dict[adapter].remove(spare_dict[adapter])
+            set_disk_offline(adapter, disk_dict[adapter])
+    except Exception as e:
+        logger.error('%s' %e)
+        return 50
+
+    # Wait for rebuild process
+    time.sleep(5)
+
+    try:
+        while(True):
+            if not all_rebuild_status(adapter_list, spare_dict):
+                # Check rebuild RAID status every 15 min.
+                time.sleep(900)
+                continue
+            break
+    except Exception as e:
+        logger.error('%s' %e)
+        return 60
+
+    check = 0
+    try:
+        for adapter in adapter_list:
+            if not confirm_rebuild_status(adapter, spare_dict[adapter]):
+                logger.error('Rebuild RAID failed, Adapter: %s, Hotspare: %s' 
+                             %(adapter, spare_dict[adapter]))
+                check = 1
+        if check != 0:
+            return 70
+    except Exception as e:
+        logger.error('%s' %e)
+        return 80
+
+    logger.info('All adapter rebuild RAID successful!')
+    return 0
+
+
+def main():
+    description_text = 'rebulit_raid\n\tTests LSI RAID card auto rebuild ' \
+        + 'function.\n\n\n\tWarning: This program will rebuild RAID ' \
+        + 'automaticity.\n\t\t Please make sure what you are doing!\n\n\n' \
+        + '\tRequirement: It can be only one hotspare in a adapter.'
+
+    parser = ArgumentParser(description=description_text,
+                            formatter_class=RawTextHelpFormatter)
+
+    parser.add_argument('-l', '--log', type=str,
+                      default='/tmp/rebulid_raid.log',
+                      help=('Specify the location and name of the log file.\n'
+                            '[Default: %(default)s]'))
+    parser.add_argument('-P', '--PhysDrv', type=str,
+                        help=('The physical drive enclosure and slot of RAID.'
+                              '\n[Example: --PhysDrv E0:S0,E1:S1,...] '))
+    parser.add_argument('-H', '--Hsp', type=str,
+                        help=('The hotspare\'s enclosure and slot of RAID.'
+                              '\n[Example: --Hsp E0:S0] '))
+    parser.add_argument('-A', '--Adapter', type=str,
+                        help=('The adapter number of RAID card.'
+                              '\n[Example: 0'))
+    args = parser.parse_args()
+
+    file_handler = logging.FileHandler(args.log)
+    file_handler.setFormatter(logFormatter)
+    console_handler = logging.StreamHandler()
+    console_handler.setFormatter(logFormatter)
+    console_handler.setLevel(logging.INFO)
+    logger.addHandler(file_handler)
+    logger.addHandler(console_handler)
+
+    disk_list = None
+    if args.PhysDrv:
+       disk_list = args.PhysDrv.split(',')
+
+    logger.info('Rebuild process beginning.')
+    return run(args.Adapter, args.Hsp, disk_list)
+
+if __name__ == '__main__':
+    sys.exit(main())