Merge lp:~psivaa/qa-iso-static-validation-test/qa-iso-static-validation-test into lp:qa-iso-static-validation-test

[Para Siva]

Fixing the report.html file failures on server tests

[lp:~psivaa/qa-iso-static-validation-test/qa-iso-static-validation-test]

15. By Paul Larson on 2012-09-18

pep8 cleanup

[Paul Larson (pwlars)]

Interesting, do we know how the problems page gets generated? Are there ever situations where it shows up in reports.html but not in the problem page? I assumed that one pulled from the other, or at least used the same process to find these uninstallable packages.

+ fh_probs_page = urllib2.urlopen(self.probs_url).read()
What happens if this fails?

[Para Siva (psivaa)]

I was under the impression that both of them are generated independently. Please see below a reply from cjwatson about one of my queries about some entries in report.html.

"psivaa: yep. they're all listed on http://people.canonical.com/~ubuntu-archive/testing/quantal_probs.html, so it isn't an image build problem."

So this means there are occasions where the entries in report.html are not present in _probs.html and in that case it shows an image build issue. Hence needs reporting. But if the package is present on both reports we do not have to worry about them because the dev teams are aware of them

Re: not handling the exception for urlopen, I'll add it and resubmit.

Thanks

[lp:~psivaa/qa-iso-static-validation-test/qa-iso-static-validation-test]

16. By Para Siva on 2012-09-18

report.html fix number 2 handling url open excetion

17. By Para Siva on 2012-09-18

Fixing known report.html file failures

[Paul Larson (pwlars)]

Good!

19 +
20 + self.probs_url = os.path.join(ARCHIVE_TEST_URL, self.st_release + '_probs.html')
21
22 + self.probs_url = os.path.join(ARCHIVE_TEST_URL, self.st_release + '_probs.html')
Duplicate line?

Some minor nitpicks:
41 + self.fail("Failed to fetch problems.html from the server")
You have an extra space after "from"

pep8 issues (easy cleanups, but since I just pushed a branch cleaning things like this, seems appropriate not to introduce more)
iso_static_validation.py:107:1: W293 blank line contains whitespace
iso_static_validation.py:108:80: E501 line too long (88 characters)
iso_static_validation.py:112:5: E301 expected 1 blank line, found 0
iso_static_validation.py:399:46: E231 missing whitespace after ','

For bonus points, it's a good habit to start coding things in a way that are python3 compliant. So things like exception handling should be, for example:
except urllib2.HTTPError as e:

If you'd like though, just fix the pep8 stuff and we can do the python3 compliance as another fix later on. It's not really a requirement right now, but would be a good idea considering the future direction of python.

review: Needs Fixing

[lp:~psivaa/qa-iso-static-validation-test/qa-iso-static-validation-test]

18. By Para Siva on 2012-09-18

pep8 fixes for the report.html fix

[Para Siva (psivaa)]

review: Needs Resubmitting

[Paul Larson (pwlars)]

review: Approve

=== modified file 'iso_static_validation.py'
--- iso_static_validation.py	2012-09-18 13:30:01 +0000
+++ iso_static_validation.py	2012-09-18 16:18:34 +0000
@@ -33,9 +33,8 @@
 import shutil
 
 # Defaults
-#DEFAULT_ISOROOT = os.path.expanduser(
-#    '/var/lib/ubuntu-server-iso-testing/isos/ubuntu/')
 DEFAULT_URL = 'http://cdimage.ubuntu.com/'
+ARCHIVE_TEST_URL = 'http://people.canonical.com/~ubuntu-archive/testing/'
 
 #constants
 ONE_MB_BLOCK = 2 ** 20
@@ -106,6 +105,9 @@
         else:
             self.fail("Image name be in the form of release-variant-arch.iso")
 
+        self.probs_url = os.path.join(ARCHIVE_TEST_URL, self.st_release +
+            '_probs.html')
+
     # Test if the sha check sum of the iso matches that is given in the server
     def test_sha256_checksum(self):
         current_iso_digest = {}
@@ -382,11 +384,19 @@
             logging.error("Failed to fetch URL '%s': %s .", report_url, e)
             self.fail("Failed to fetch report.html from  the server")
 
-        # check the number of failed packages
-        report_page_lines = re.findall('there were ["]?([^" >]+)',
+        uninst_pkgs = re.findall('"?([^" >]+ )produces uninstallable binaries',
                                        fh_report_page)
-        logging.debug('Check for conflicting packages in the report file')
-        self.assertEqual(['0'], report_page_lines)
+
+        try:
+            fh_probs_page = urllib2.urlopen(self.probs_url).read()
+        except urllib2.HTTPError, e:
+            logging.error("Failed to fetch URL '%s': %s .", self.probs_url, e)
+            self.fail("Failed to fetch problems.html from the server")
+
+        # If a package is listed in report.html and not in problems page
+        # then it's a concern and needs attention
+        for uninst_pkg in uninst_pkgs:
+            self.assertIn(uninst_pkg.rstrip(), fh_probs_page)
 
     # Test if the size of the image fits a 700MB disc
     def test_size(self):