Merge ~ltrager/maas:fix_tests_hardware_type_tags into maas:master

[Lee Trager]

Fix tests broken by hardware type tags.

With the addition of hardware type to scripts scripts are now auto tagged
with their hardware type. Some tests did not expect that. Also fixed naming in some script metadata and add apt install section to memtester.sh.

[Andres Rodriguez (andreserl)]

quick question inline

review: Needs Information

[Newell Jensen (newell-jensen)]

LGTM

review: Approve

[Newell Jensen (newell-jensen)]

Not going to stop you from landing but splitting up the drive by fix for s/disk/storage might be better in another branch.

[Andres Rodriguez (andreserl)]

review: Approve

[Andres Rodriguez (andreserl)]

I'm approving on the condition we review the naming convention because, it doesn't really seem user readable.

Also, I noticed that hardware_type: storage to define the type of a *Test* is not the right parameter.

[MAAS Lander (maas-lander)]

LANDING
-b fix_tests_hardware_type_tags lp:~ltrager/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/230/consoleText

5d4efe9... by Lee Trager on 2017-08-20

Fix intermittent script failures.

3a96221... by Lee Trager on 2017-08-20

Merge branch 'fix_tests_hardware_type_tags_2' into fix_tests_hardware_type_tags

diff --git a/src/maasserver/api/tests/test_machine.py b/src/maasserver/api/tests/test_machine.py
index 63e0aa6..46edcc4 100644
--- a/src/maasserver/api/tests/test_machine.py
+++ b/src/maasserver/api/tests/test_machine.py
@@ -1019,10 +1019,14 @@ class TestMachineAPI(APITestCase.ForUser):
             'enable_ssh': "true",
             'skip_networking': 1,
             'commissioning_scripts': ','.join([
-                choice(commissioning_script_selected_by_tag.tags),
+                choice([
+                    tag for tag in commissioning_script_selected_by_tag.tags
+                    if 'tag' in tag]),
                 commissioning_script_selected_by_name.name]),
             'testing_scripts': ','.join([
-                choice(testing_script_selected_by_tag.tags),
+                choice([
+                    tag for tag in testing_script_selected_by_tag.tags
+                    if 'tag' in tag]),
                 testing_script_selected_by_name.name]),
             })
         self.assertEqual(http.client.OK, response.status_code)
diff --git a/src/maasserver/api/tests/test_node.py b/src/maasserver/api/tests/test_node.py
index 07fa7e7..706a3a0 100644
--- a/src/maasserver/api/tests/test_node.py
+++ b/src/maasserver/api/tests/test_node.py
@@ -591,8 +591,10 @@ class TestPowerMixin(APITestCase.ForUser):
             'op': 'test',
             'enable_ssh': "true",
             'testing_scripts': ','.join([
-                random.choice(testing_script_selected_by_tag.tags),
-                testing_script_selected_by_name.name]),
+                random.choice([
+                    tag for tag in testing_script_selected_by_tag.tags
+                    if 'tag' in tag
+                ]), testing_script_selected_by_name.name]),
             })
         self.assertEqual(http.client.OK, response.status_code)
         node = reload_object(node)
diff --git a/src/metadataserver/builtin_scripts/badblocks.py b/src/metadataserver/builtin_scripts/badblocks.py
index 4402a4e..d686a51 100644
--- a/src/metadataserver/builtin_scripts/badblocks.py
+++ b/src/metadataserver/builtin_scripts/badblocks.py
@@ -31,7 +31,7 @@
 #     title: Bad blocks
 #     description: The number of bad blocks found on the storage device.
 # parameters:
-#   disk: {type: storage}
+#   storage: {type: storage}
 # destructive: {{if 'destructive' in name}}True{{else}}False{{endif}}
 # --- End MAAS 1.0 script metadata ---
 
diff --git a/src/metadataserver/builtin_scripts/memtester.sh b/src/metadataserver/builtin_scripts/memtester.sh
index c217919..994d15e 100644
--- a/src/metadataserver/builtin_scripts/memtester.sh
+++ b/src/metadataserver/builtin_scripts/memtester.sh
@@ -24,6 +24,8 @@
 # title: Memory integrity
 # description: Run memtester against all available userspace memory.
 # script_type: test
+# packages:
+#   apt: memtester
 # --- End MAAS 1.0 script metadata ---
 
 sudo -n apt-get install -q -y memtester
diff --git a/src/metadataserver/builtin_scripts/smartctl.py b/src/metadataserver/builtin_scripts/smartctl.py
index 9591e0b..03c9fc3 100644
--- a/src/metadataserver/builtin_scripts/smartctl.py
+++ b/src/metadataserver/builtin_scripts/smartctl.py
@@ -28,7 +28,7 @@
 # hardware_type: storage
 # parallel: instance
 # parameters:
-#   disk: {type: storage}
+#   storage: {type: storage}
 # packages:
 #   apt: smartctl
 # timeout: {{timeout}}
diff --git a/src/metadataserver/models/tests/test_scriptset.py b/src/metadataserver/models/tests/test_scriptset.py
index b18b363..443ea58 100644
--- a/src/metadataserver/models/tests/test_scriptset.py
+++ b/src/metadataserver/models/tests/test_scriptset.py
@@ -197,7 +197,9 @@ class TestScriptSetManager(MAASServerTestCase):
 
         script_set = ScriptSet.objects.create_commissioning_script_set(
             node, scripts=[
-                random.choice(script_selected_by_tag.tags),
+                random.choice([
+                    tag for tag in script_selected_by_tag.tags
+                    if 'tag' in tag]),
                 script_selected_by_name.name,
                 script_selected_by_id.id,
             ])
@@ -274,7 +276,9 @@ class TestScriptSetManager(MAASServerTestCase):
 
         script_set = ScriptSet.objects.create_testing_script_set(
             node, scripts=[
-                random.choice(script_selected_by_tag.tags),
+                random.choice([
+                    tag for tag in script_selected_by_tag.tags
+                    if 'tag' in tag]),
                 script_selected_by_name.name,
                 script_selected_by_id.id,
             ])
diff --git a/utilities/find-intermittent-errors b/utilities/find-intermittent-errors
index 7d4841c..105ad9c 100755
--- a/utilities/find-intermittent-errors
+++ b/utilities/find-intermittent-errors
@@ -47,16 +47,13 @@ function stats()
 
     echo "Tests run: $(ls $OUTPUT_DIR | wc -l)"
     for i in $OUTPUT_DIR/*; do
-	 cat $i | awk '/FAIL$/ {
-            for(i=1;i<=NF-2;i++) {
-                printf("%s ", $i);
-            }
-            printf("\n");
+	cat $i | awk '/^(FAIL): .*$/ {
+            print($2);
             count++;
         }
         END {
             if(count == 0) print("No failures");
-        }'
+        }' | sort -u
     done | sort | uniq -c | sort -r -n -k 1
 }