Merge lp:~cr3/checkbox-certification/attachments_move into lp:checkbox-certification

[Marc Tardif]

Merged moving of attachment jobs in client certification whitelists.

This moves the attachment jobs from immediately after the resource jobs to the very end of the client certification whitelists. This makes testing slightly more efficient.

Note that the change was not done for the server certification whitelists because they are already automated.

[Jeff Lane  (bladernr)]

All the newer jobs appear at the end of the list, where the original attachment jobs were all alphabetized to make reading easier.

Since they don't depend on running in a particular order, do you think it would be worthwhile to take this opportunity to clean them up a bit to make reading them easier down the road?

Also, another thought, would it be worthwhile to move the other attachment jobs into that section, or do you think they'd be better off where they are right now (thinking of things like the suspend_single_log attach job, suspend_30_cycles_log attach job, and other similar ones that run after actual tests).

review: Needs Information

[Jeff Lane  (bladernr)]

I meant to add that I'm fine with this as is, the comment above is curiosity rather than critique.

[Marc Tardif (cr3)]

I don't understand what you mean by "the original attachment jobs were all alphabetized to make reading easier." I thought I kept the original order, or did I miss something?

As for adding some of the other attachments, I think it's easier for sections to map to the corresponding file as closely as possible. For example, the attachment jobs under the __info__ section in the whitelists all map to the jobs in the info.txt.in file. I realize this has been affected quite significantly by your changes to move automated tests at the bottom of the whitelist. However, that was such a significant improvement that the benefits outweighed the readability. In this case, I think that the few other attachments scattered throughout the other parts of the whitelist file don't impact running checkbox as much so I'd keep things simple.

[Jeff Lane  (bladernr)]

Nevermind... they haven't changed as much as I thought they had. For example, an older version of the "certification" whitelist shows this:

codecs_attach
cpuinfo_attach
dmesg_attach
dmi_attach
dmidecode_attach
lspci_attach
modprobe_attach
modules_attach
sysctl_attach
udev_attach

Where the current one shows this:
codecs_attachment
cpuinfo_attachment
dmesg_attachment
dmi_attachment
dmidecode_attachment
efi_attachment
lspci_attachment
meminfo_attachment
modprobe_attachment
modules_attachment
sysctl_attachment
sysfs_attachment
udev_attachment
lsmod_attachment
acpi_sleep_attachment
info/hdparm
info/hdparm_.*.txt
installer_debug.gz
info/touchpad_driver

I was just wondering if alphabetizing them would make them easier to read in a side by side comparison when we're making sure new jobs have been properly applied between lists.

For example:

list1 list2 list3
A A B
B C G
C D F
D E A
E G C
F B D
G F E

Just an exaggerated example to show what I mean. In any case, this is no reason to hold up your merge request, so I'll take it offline instead.

[Jeff Lane  (bladernr)]

So as I said, no reason to hold this up on a tangential discussion that doesn't really affect this merge. Looks good, approve.

review: Approve

=== modified file 'data/client-cert.whitelist'
--- data/client-cert.whitelist	2012-10-17 19:25:07 +0000
+++ data/client-cert.whitelist	2012-10-18 19:45:24 +0000
@@ -24,27 +24,6 @@
 touchpad_scroll
 uname
 usb
-#Info attachment jobs
-__info__
-codecs_attachment
-cpuinfo_attachment
-dmesg_attachment
-dmi_attachment
-dmidecode_attachment
-efi_attachment
-lspci_attachment
-meminfo_attachment
-modprobe_attachment
-modules_attachment
-sysctl_attachment
-sysfs_attachment
-udev_attachment
-lsmod_attachment
-acpi_sleep_attachment
-info/hdparm
-info/hdparm_.*.txt
-installer_debug.gz
-info/touchpad_driver
 # Tests grouped by categories. Categories start with '__'
 __audio__
 audio/alsa_info_collect
@@ -270,3 +249,24 @@
 power-management/hibernate-30-cycle-log-attach
 __stress__
 stress/cpu_stress_test
+#Info attachment jobs
+__info__
+codecs_attachment
+cpuinfo_attachment
+dmesg_attachment
+dmi_attachment
+dmidecode_attachment
+efi_attachment
+lspci_attachment
+meminfo_attachment
+modprobe_attachment
+modules_attachment
+sysctl_attachment
+sysfs_attachment
+udev_attachment
+lsmod_attachment
+acpi_sleep_attachment
+info/hdparm
+info/hdparm_.*.txt
+installer_debug.gz
+info/touchpad_driver
 
=== modified file 'data/client-selftest.whitelist'
--- data/client-selftest.whitelist	2012-10-17 19:25:07 +0000
+++ data/client-selftest.whitelist	2012-10-18 19:45:24 +0000
@@ -22,27 +22,6 @@
 touchpad_scroll
 uname
 usb
-#Info attachment jobs
-__info__
-codecs_attachment
-cpuinfo_attachment
-dmesg_attachment
-dmi_attachment
-dmidecode_attachment
-efi_attachment
-lspci_attachment
-meminfo_attachment
-modprobe_attachment
-modules_attachment
-sysctl_attachment
-sysfs_attachment
-udev_attachment
-lsmod_attachment
-acpi_sleep_attachment
-info/hdparm
-info/hdparm_.*.txt
-installer_debug.gz
-info/touchpad_driver
 # Tests grouped by categories. Categories start with '__'
 __audio__
 audio/alsa_info_collect
@@ -274,3 +253,24 @@
 power-management/hibernate-30-cycle-log-attach
 __stress__
 stress/cpu_stress_test
+#Info attachment jobs
+__info__
+codecs_attachment
+cpuinfo_attachment
+dmesg_attachment
+dmi_attachment
+dmidecode_attachment
+efi_attachment
+lspci_attachment
+meminfo_attachment
+modprobe_attachment
+modules_attachment
+sysctl_attachment
+sysfs_attachment
+udev_attachment
+lsmod_attachment
+acpi_sleep_attachment
+info/hdparm
+info/hdparm_.*.txt
+installer_debug.gz
+info/touchpad_driver
 
=== modified file 'debian/changelog'
--- debian/changelog	2012-10-17 12:32:22 +0000
+++ debian/changelog	2012-10-18 19:45:24 +0000
@@ -57,6 +57,7 @@
   * Migrated project to Python 3.
   * Munged submission.xml to be consistent with direct submissions.
   * Fixed whitelists to reflect changes in optical tests (LP: #990560)
+  * Moved attachment jobs at the end of client whitelists.
 
   [ Daniel Manrique ]
   * Added alsa_info_collect and attachment jobs to certification whitelist,