Merge lp:~matsubara/ubuntu-test-cases/lamp-bug-1443987 into lp:ubuntu-test-cases/server

[Diogo Matsubara]

Update lamp test case that checks the number of default mysql tables installed

Hi, this branch fixes the failing vivid smoke test for LAMP[1] by updating the number of default MySQL tables the test looks for. I have a question though, if I update this branch, will older releases such as trusty rely on this new test case? If yes, those are going to fail as well since on trusty the default mysql version installs only 24 tables, IIRC.

Should I special case the test case or does trusty use another branch for its test runs?

[1] http://d-jenkins.ubuntu-ci:8080/job/vivid-server-i386-smoke-lamp/

[Paul Larson (pwlars)]

Would this break things in the trusty jobs though? It looks like there have been no daily images for trusty in a while, but I'm guessing there will be at some point right?

review: Needs Information

[Diogo Matsubara (matsubara)]

I believe it might break the test case for Trusty. Looking at the console output of the last Trusty run[1] it branches the test suite from lp:ubuntu-test-cases/server

So, what's usually the way to handle cases like this? Have a specific branch for trusty and update the trusty jobs to use that branch or special case in the test suite itself?

Thanks for the review!

[1] http://d-jenkins.ubuntu-ci:8080/job/trusty-server-i386-smoke-lamp/174/consoleFull of the last run

[Paul Larson (pwlars)]

I don't think this is something we've had to deal with before, so there's not a "usual" solution, but I'm sure we can come up with something that doesn't require forking for each release. How about either:
1. detecting whether we are on a release before or after vivid and adjusting the expectation based on that

-or-

2. Improving the test to look for at least a few known table names that exist in both releases rather than just look at the number of list items after calling split().

I think I like #2 better personally, but I don't think that's the only solution.

[lp:~matsubara/ubuntu-test-cases/lamp-bug-1443987]

101. By Diogo Matsubara on 2015-04-16

update LAMP tests to check for some expected tables in the MySQL database as well as checking for a minimum of 24 tables in it.

[Diogo Matsubara (matsubara)]

Hi Paul,

I updated the tests to take into account the fact that there are differences between Trusty and Vivid default MySQL install. It now checks for existing tables on both installs as well as checking for a minimum of 24 tables.

I used this data to come up with the expected table names:

trusty-mysql-tables.out -> http://paste.ubuntu.com/10833234/
vivid-mysql-tables.out -> http://paste.ubuntu.com/10833235/
diff -u trusty-mysql-tables.out vivid-mysql-tables.out -> http://paste.ubuntu.com/10833233/

[Paul Larson (pwlars)]

This seems better to me. +1

review: Approve

[Robie Basak (racb)]

I'm happy that this modification of the test case will continue to effectively test the MySQL install in both Trusty and Vivid. +1.

review: Approve

[Diogo Matsubara (matsubara)]

Thanks for the review Robie and Paul. I tried to push the changes to lp:ubuntu-test-cases/server but I don't have permission (One needs to be in ~ubuntu-server-dev). Could you push it for me please?

[lp:~matsubara/ubuntu-test-cases/lamp-bug-1443987]

102. By Diogo Matsubara on 2015-04-30

fix typo in set method.

=== modified file 'testsuites/lamp/test_mysql_show_tables/tc_control'
--- testsuites/lamp/test_mysql_show_tables/tc_control	2012-11-14 10:56:53 +0000
+++ testsuites/lamp/test_mysql_show_tables/tc_control	2015-04-30 08:39:51 +0000
@@ -3,10 +3,12 @@
 pre_conditions: The machine has been provisioned with a LAMP server
 action: |
   1. List the tables in the database mysql
-  2. Check if there are 24 tables in the database
+  2. Check a list of default tables exist in the database
+  3. Check if there are 24 or more tables in the database
 expected_results: |
   1. The query lists the tables in mysql database
-  2. There are 24 tables in it
+  2. There are some default tables in it
+  3. There are 24 or more tables in it
 type: userland
 run_as: root
 timeout: 200

=== modified file 'testsuites/lamp/test_mysql_show_tables/test.py'
--- testsuites/lamp/test_mysql_show_tables/test.py	2013-03-01 17:39:50 +0000
+++ testsuites/lamp/test_mysql_show_tables/test.py	2015-04-30 08:39:51 +0000
@@ -27,6 +27,33 @@
 logging.basicConfig(level=logging.INFO)
 
 
+# Tables known to exist on a default install for Trusty and Vivid.
+EXPECTED_TABLES = set([
+    'columns_priv',
+    'db',
+    'event',
+    'func',
+    'general_log',
+    'help_category',
+    'help_keyword',
+    'help_relation',
+    'help_topic',
+    'ndb_binlog_index',
+    'plugin',
+    'proc',
+    'procs_priv',
+    'proxies_priv',
+    'servers',
+    'slow_log',
+    'tables_priv',
+    'time_zone',
+    'time_zone_leap_second',
+    'time_zone_name',
+    'time_zone_transition',
+    'time_zone_transition_type',
+    'user'])
+
+
 class LampTest(unittest.TestCase):
     def testMysqlShowTables(self):
         cmd = ["mysql", "-e", "show tables",
@@ -37,7 +64,13 @@
 
         self.assertEqual(stderr, None)
         logging.debug("Cmd output: %s" % (output))
-        self.assertEqual(len(output.strip().split("\n")), 24)
+        result = output.strip().split("\n")
+        # Check that the output contains some default tables.
+        self.assertTrue(EXPECTED_TABLES.issubset(set(result)))
+        # Check for a minimum of 24 tables as the number might vary between
+        # releases. See bug 1443987.
+        self.assertGreaterEqual(len(result), 24)
+
 
 if __name__ == '__main__':
     unittest.main()