Merge ~ian-may/+git/autotest-client-tests:jammy-focal-pts-compat-mp-v2 into ~canonical-kernel-team/+git/autotest-client-tests:master

[Ian May]

v2 Changes

- Add wrapper function for getting platform object via platform or distro module.

- Further investigation into openssl-3.0.1 test failure with squid.internal proxy. This can be replicated outside of autotest and PTS using

Works
wget https://github.com/openssl/openssl/archive/refs/tags/openssl-3.0.0.tar.gz

Fails
export https_proxy="http://squid.internal:3128"
wget https://github.com/openssl/openssl/archive/refs/tags/openssl-3.0.0.tar.gz

DGX Performance testing is moving from Bionic/Focal to Focal/Jammy. The ubuntu_performance_pts test needs some fixups for Jammy. These patches provide updates for Jammy compatibility with pts/john-the-ripper and pts/openssl

[Andrei Gherzan (agherzan)]

I'm unsure if I understand the "UBUNTU: SAUCE: ubuntu_performance_pts: update user-config.xml" change. I would recommend expanding git log a bit.

[Po-Hsu Lin (cypressyew)]

Apart from the comment from Andrei, I am curious why we don't need to use squid.internal proxy anymore.

Also, with older releases the platform.dist() will error out with:
  AttributeError: module 'platform' has no attribute 'dist'

Do we need to run this test with Focal or even older releases? If the answer is yes then we might need to modify this part.

Other than this the code is looking good.

review: Needs Information

[Ian May (ian-may)]

> Apart from the comment from Andrei, I am curious why we don't need to use
> squid.internal proxy anymore.
>

I apologize for not providing better context. When using 'squid.internal' and running the openssl test "phoronix-test-suite batch-benchmark openssl-3.0.1". The test fails to download. I'll update the commit message with a summary.

$ sudo phoronix-test-suite batch-benchmark openssl-3.0.1

    Evaluating External Test Dependencies ........................................................................................................................................................................................................................

Phoronix Test Suite v10.8.4

    To Install: pts/openssl-3.0.1

    Determining File Requirements ................................................................................................................................................................................................................................
    Searching Download Caches ....................................................................................................................................................................................................................................

    1 Test To Install
        1 File To Download [14.41MB]
        430MB Of Disk Space Is Needed
        2 Minutes, 32 Seconds Estimated Install Time

    pts/openssl-3.0.1:
        Test Installation 1 of 1
        1 File Needed [14.41 MB / 1 Minute]
        Downloading: openssl-openssl-3.0.0.tar.gz [14.41MB]
            Download Failed: https://github.com/openssl/openssl/archive/refs/tags/openssl-3.0.0.tar.gz
            If able to locate the file elsewhere, place it in the download cache and re-run the command.

            Download Cache: /var/lib/phoronix-test-suite/download-cache/
            File Name: openssl-openssl-3.0.0.tar.gz
            SHA256: b39653402e398a2b89c3f1779f09fe273fda6b6d7873d2816b954b949a185b83
            Downloading of needed test files failed.

    [PROBLEM] pts/openssl-3.0.1 is not installed.

Removing the proxy the test downloads fine. If you know of a way to make this work via squid.internal I can pursue that method for as a resolution.

> Also, with older releases the platform.dist() will error out with:
> AttributeError: module 'platform' has no attribute 'dist'
>
> Do we need to run this test with Focal or even older releases? If the answer
> is yes then we might need to modify this part.

Thanks, there is a wrapper script for running the DGX Perf tests, it installs python2-minimal on Focal and Jammy which is why I didn't see any issues with platform.dist() in my testing. I'm adding a patch that will wrap that call to protect against errors on Python 3.8 and up.

>
> Other than this the code is looking good.

[Po-Hsu Lin (cypressyew)]

LGTM. Thanks!

review: Approve

[Po-Hsu Lin (cypressyew)]

Applied and pushed, thanks!

diff --git a/ubuntu_performance_pts/control b/ubuntu_performance_pts/control
index 9ccb7d7..80d6767 100644
--- a/ubuntu_performance_pts/control
+++ b/ubuntu_performance_pts/control
@@ -17,7 +17,6 @@ result = job.run_test_detail(NAME, test_name='setup', tag='setup', subtest='setu
 if result == 'GOOD':
     jobs = [
         [ 'john-the-ripper', 'john-the-ripper-blowfish', 30 ],
-        [ 'john-the-ripper', 'john-the-ripper-des', 30 ],
         [ 'openssl', 'openssl', 30 ],
         [ 'povray', 'povray', 80 ],
         [ 'ttsiod-renderer', 'ttsiod-renderer', 30 ],
diff --git a/ubuntu_performance_pts/phoronix-test-suite_10.8.4_all.deb b/ubuntu_performance_pts/phoronix-test-suite_10.8.4_all.deb
new file mode 100644
index 0000000..d3bcd8b
Binary files /dev/null and b/ubuntu_performance_pts/phoronix-test-suite_10.8.4_all.deb differ
diff --git a/ubuntu_performance_pts/phoronix-test-suite_7.8.0_all.deb b/ubuntu_performance_pts/phoronix-test-suite_7.8.0_all.deb
deleted file mode 100644
index 797f6bb..0000000
Binary files a/ubuntu_performance_pts/phoronix-test-suite_7.8.0_all.deb and /dev/null differ
diff --git a/ubuntu_performance_pts/ubuntu_performance_pts.py b/ubuntu_performance_pts/ubuntu_performance_pts.py
index 60228c8..5130438 100644
--- a/ubuntu_performance_pts/ubuntu_performance_pts.py
+++ b/ubuntu_performance_pts/ubuntu_performance_pts.py
@@ -86,13 +86,16 @@ class ubuntu_performance_pts(test.test):
         if result.returncode != 0:
             print("WARNING: could not set swap %s" % ("on" if swap_on else "off"))
 
-    def install_required_pkgs(self):
-        arch    = platform.processor()
+    def get_platform_distro(self):
         try:
-            series = platform.dist()[2]
+            return platform.dist()
         except AttributeError:
             import distro
-            series = distro.codename()
+            return distro.linux_distribution(full_distribution_name=False)
+
+    def install_required_pkgs(self):
+        arch    = platform.processor()
+        series = self.get_platform_distro()[2]
         release = platform.release()
 
         pkgs = [
@@ -136,7 +139,7 @@ class ubuntu_performance_pts(test.test):
         self.results += self.setup_config(pwd.getpwnam(username)[5])
         self.results += self.setup_config(os.path.expanduser("~"))
 
-        self.results += utils.system_output('dpkg -i %s' % os.path.join(self.bindir, 'phoronix-test-suite_7.8.0_all.deb'), retain_output=True)
+        self.results += utils.system_output('dpkg -i %s' % os.path.join(self.bindir, 'phoronix-test-suite_10.8.4_all.deb'), retain_output=True)
         self.results += utils.system_output('phoronix-test-suite enterprise-setup', retain_output=True)
         self.results += utils.system_output('yes n | phoronix-test-suite batch-setup', retain_output=True)
 
@@ -228,15 +231,18 @@ class ubuntu_performance_pts(test.test):
             print("NOTRUN: test not run, no data")
 
     def run_john_the_ripper_blowfish(self, test_name, tag):
-        cmd = 'export PRESET_OPTIONS="john-the-ripper.run-test=Blowfish"; %s phoronix-test-suite batch-benchmark john-the-ripper-1.7.2' % force_times_to_run
+        if self.get_platform_distro()[1].split('.')[0] > "20":
+            cmd = 'export PRESET_OPTIONS="john-the-ripper.run-test=Blowfish"; %s phoronix-test-suite batch-benchmark john-the-ripper-1.8.0' % force_times_to_run
+        else:
+            cmd = 'export PRESET_OPTIONS="john-the-ripper.run-test=Blowfish"; %s phoronix-test-suite batch-benchmark john-the-ripper-1.7.2' % force_times_to_run
         self.print_stats('john_the_ripper_blowfish', cmd)
 
-    def run_john_the_ripper_des(self, test_name, tag):
-        cmd = 'export PRESET_OPTIONS="john-the-ripper.run-test=Traditional DES"; %s phoronix-test-suite batch-benchmark john-the-ripper-1.6.2' % force_times_to_run
-        self.print_stats('john_the_ripper_des', cmd)
-
     def run_openssl(self, test_name, tag):
-        cmd = '%s phoronix-test-suite batch-benchmark openssl-1.11.0' % force_times_to_run
+        #If series is greater than focal(20) we need to use Openssl-3
+        if self.get_platform_distro()[1].split('.')[0] > "20":
+            cmd = 'export PRESET_OPTIONS="openssl.algo=RSA4096"; %s phoronix-test-suite batch-benchmark openssl-3.0.1' % force_times_to_run
+        else:
+            cmd = 'export PRESET_OPTIONS="openssl.algo=RSA4096"; %s phoronix-test-suite batch-benchmark openssl-1.11.0' % force_times_to_run
         self.print_stats(test_name, cmd)
 
     def run_povray(self, test_name, tag):
@@ -277,7 +283,6 @@ class ubuntu_performance_pts(test.test):
             'git': self.run_generic,
             'build-llvm': self.run_generic,
             'john-the-ripper-blowfish' : self.run_john_the_ripper_blowfish,
-            'john-the-ripper-des': self.run_john_the_ripper_des,
             'openssl': self.run_openssl,
             'osbench-files': self.run_osbench_files,
             'osbench-processes': self.run_osbench_files,
diff --git a/ubuntu_performance_pts/user-config.xml b/ubuntu_performance_pts/user-config.xml
index de80ed8..19ba6cb 100644
--- a/ubuntu_performance_pts/user-config.xml
+++ b/ubuntu_performance_pts/user-config.xml
@@ -56,8 +56,8 @@
       <NoInternetCommunication>FALSE</NoInternetCommunication>
       <NoNetworkCommunication>FALSE</NoNetworkCommunication>
       <Timeout>20</Timeout>
-      <ProxyAddress>squid.internal</ProxyAddress>
-      <ProxyPort>3128</ProxyPort>
+      <ProxyAddress></ProxyAddress>
+      <ProxyPort></ProxyPort>
       <ProxyUser></ProxyUser>
       <ProxyPassword></ProxyPassword>
     </Networking>