Merge ~ian-may/+git/autotest-client-tests:jammy-focal-pts-compat-mp into ~canonical-kernel-team/+git/autotest-client-tests:master

[Ian May]

DGX Performance testing is moving from Bionic/Focal to Focal/Jammy. The ubuntu_performance_pts test needs some fixups for Jammy. These patches provide updates for Jammy compatibility with pts/john-the-ripper and pts/openssl

[Andrei Gherzan (agherzan)]

I'm unsure if I understand the "UBUNTU: SAUCE: ubuntu_performance_pts: update user-config.xml" change. I would recommend expanding git log a bit.

[Po-Hsu Lin (cypressyew)]

Apart from the comment from Andrei, I am curious why we don't need to use squid.internal proxy anymore.

Also, with older releases the platform.dist() will error out with:
  AttributeError: module 'platform' has no attribute 'dist'

Do we need to run this test with Focal or even older releases? If the answer is yes then we might need to modify this part.

Other than this the code is looking good.

review: Needs Information

[Ian May (ian-may)]

> Apart from the comment from Andrei, I am curious why we don't need to use
> squid.internal proxy anymore.
>

I apologize for not providing better context. When using 'squid.internal' and running the openssl test "phoronix-test-suite batch-benchmark openssl-3.0.1". The test fails to download. I'll update the commit message with a summary.

$ sudo phoronix-test-suite batch-benchmark openssl-3.0.1

    Evaluating External Test Dependencies ........................................................................................................................................................................................................................

Phoronix Test Suite v10.8.4

    To Install: pts/openssl-3.0.1

    Determining File Requirements ................................................................................................................................................................................................................................
    Searching Download Caches ....................................................................................................................................................................................................................................

    1 Test To Install
        1 File To Download [14.41MB]
        430MB Of Disk Space Is Needed
        2 Minutes, 32 Seconds Estimated Install Time

    pts/openssl-3.0.1:
        Test Installation 1 of 1
        1 File Needed [14.41 MB / 1 Minute]
        Downloading: openssl-openssl-3.0.0.tar.gz [14.41MB]
            Download Failed: https://github.com/openssl/openssl/archive/refs/tags/openssl-3.0.0.tar.gz
            If able to locate the file elsewhere, place it in the download cache and re-run the command.

            Download Cache: /var/lib/phoronix-test-suite/download-cache/
            File Name: openssl-openssl-3.0.0.tar.gz
            SHA256: b39653402e398a2b89c3f1779f09fe273fda6b6d7873d2816b954b949a185b83
            Downloading of needed test files failed.

    [PROBLEM] pts/openssl-3.0.1 is not installed.

Removing the proxy the test downloads fine. If you know of a way to make this work via squid.internal I can pursue that method for as a resolution.

> Also, with older releases the platform.dist() will error out with:
> AttributeError: module 'platform' has no attribute 'dist'
>
> Do we need to run this test with Focal or even older releases? If the answer
> is yes then we might need to modify this part.

Thanks, there is a wrapper script for running the DGX Perf tests, it installs python2-minimal on Focal and Jammy which is why I didn't see any issues with platform.dist() in my testing. I'm adding a patch that will wrap that call to protect against errors on Python 3.8 and up.

>
> Other than this the code is looking good.

Unmerged commits

f950d5c... by Ian May on 2023-06-02

UBUNTU: SAUCE: ubuntu_performance_pts: update user-config.xml

Signed-off-by: Ian May <email address hidden>

ecf7d5d... by Ian May on 2023-05-26

UBUNTU: SAUCE: ubuntu_performance_pts: update openssl test

For Jammy compat we need to test against Openssl-3. If
running on Jammy use the pts openssl-3.0.1 test.

Add a PRESET_OPTION so we are only running the test we want
to measure.

Signed-off-by: Ian May <email address hidden>

e52690f... by Ian May on 2023-06-02

UBUNTU: SAUCE: ubuntu_performance_pts: update john-the-ripper version

For Jammy compat, update to john-the-ripper-1.8.0 to fix a compile error that was
introduced using gcc-11. Continue running john-the-ripper-1.7.2 for Focal.

Signed-off-by: Ian May <email address hidden>

fa0a4a1... by Ian May on 2023-05-31

UBUNTU: SAUCE: ubuntu_performance_pts: remove john-the-ripper DES test

This test is not valid on Focal and later. We no longer
run Perf tests on Bionic so we can remove the test.

Signed-off-by: Ian May <email address hidden>

ba505b4... by Ian May on 2023-01-24

UBUNTU: SAUCE: ubuntu_performance_pts update version of PTS

This version provides proper functionality on Jammy, such as
better batch interactive support and updated JTR and openssl
tests.

Signed-off-by: Ian May <email address hidden>

diff --git a/ubuntu_performance_pts/control b/ubuntu_performance_pts/control
index 9ccb7d7..80d6767 100644
--- a/ubuntu_performance_pts/control
+++ b/ubuntu_performance_pts/control
@@ -17,7 +17,6 @@ result = job.run_test_detail(NAME, test_name='setup', tag='setup', subtest='setu
 if result == 'GOOD':
     jobs = [
         [ 'john-the-ripper', 'john-the-ripper-blowfish', 30 ],
-        [ 'john-the-ripper', 'john-the-ripper-des', 30 ],
         [ 'openssl', 'openssl', 30 ],
         [ 'povray', 'povray', 80 ],
         [ 'ttsiod-renderer', 'ttsiod-renderer', 30 ],
diff --git a/ubuntu_performance_pts/phoronix-test-suite_10.8.4_all.deb b/ubuntu_performance_pts/phoronix-test-suite_10.8.4_all.deb
 new file mode 100644
index 0000000..d3bcd8b
 Binary files /dev/null and b/ubuntu_performance_pts/phoronix-test-suite_10.8.4_all.deb differ
diff --git a/ubuntu_performance_pts/phoronix-test-suite_7.8.0_all.deb b/ubuntu_performance_pts/phoronix-test-suite_7.8.0_all.deb
 deleted file mode 100644
index 797f6bb..0000000
 Binary files a/ubuntu_performance_pts/phoronix-test-suite_7.8.0_all.deb and /dev/null differ
diff --git a/ubuntu_performance_pts/ubuntu_performance_pts.py b/ubuntu_performance_pts/ubuntu_performance_pts.py
index 60228c8..b9d91c7 100644
--- a/ubuntu_performance_pts/ubuntu_performance_pts.py
+++ b/ubuntu_performance_pts/ubuntu_performance_pts.py
@@ -136,7 +136,7 @@ class ubuntu_performance_pts(test.test):
         self.results += self.setup_config(pwd.getpwnam(username)[5])
         self.results += self.setup_config(os.path.expanduser("~"))
 
-        self.results += utils.system_output('dpkg -i %s' % os.path.join(self.bindir, 'phoronix-test-suite_7.8.0_all.deb'), retain_output=True)
+        self.results += utils.system_output('dpkg -i %s' % os.path.join(self.bindir, 'phoronix-test-suite_10.8.4_all.deb'), retain_output=True)
         self.results += utils.system_output('phoronix-test-suite enterprise-setup', retain_output=True)
         self.results += utils.system_output('yes n | phoronix-test-suite batch-setup', retain_output=True)
 
@@ -228,15 +228,18 @@ class ubuntu_performance_pts(test.test):
             print("NOTRUN: test not run, no data")
 
     def run_john_the_ripper_blowfish(self, test_name, tag):
-        cmd = 'export PRESET_OPTIONS="john-the-ripper.run-test=Blowfish"; %s phoronix-test-suite batch-benchmark john-the-ripper-1.7.2' % force_times_to_run
+        if platform.dist()[1].split('.')[0] > "20":
+            cmd = 'export PRESET_OPTIONS="john-the-ripper.run-test=Blowfish"; %s phoronix-test-suite batch-benchmark john-the-ripper-1.8.0' % force_times_to_run
+        else:
+            cmd = 'export PRESET_OPTIONS="john-the-ripper.run-test=Blowfish"; %s phoronix-test-suite batch-benchmark john-the-ripper-1.7.2' % force_times_to_run
         self.print_stats('john_the_ripper_blowfish', cmd)
 
-    def run_john_the_ripper_des(self, test_name, tag):
-        cmd = 'export PRESET_OPTIONS="john-the-ripper.run-test=Traditional DES"; %s phoronix-test-suite batch-benchmark john-the-ripper-1.6.2' % force_times_to_run
-        self.print_stats('john_the_ripper_des', cmd)
-
     def run_openssl(self, test_name, tag):
-        cmd = '%s phoronix-test-suite batch-benchmark openssl-1.11.0' % force_times_to_run
+        #If series is greater than focal(20) we need to use Openssl-3
+        if platform.dist()[1].split('.')[0] > "20":
+            cmd = 'export PRESET_OPTIONS="openssl.algo=RSA4096"; %s phoronix-test-suite batch-benchmark openssl-3.0.1' % force_times_to_run
+        else:
+            cmd = 'export PRESET_OPTIONS="openssl.algo=RSA4096"; %s phoronix-test-suite batch-benchmark openssl-1.11.0' % force_times_to_run
         self.print_stats(test_name, cmd)
 
     def run_povray(self, test_name, tag):
@@ -277,7 +280,6 @@ class ubuntu_performance_pts(test.test):
             'git': self.run_generic,
             'build-llvm': self.run_generic,
             'john-the-ripper-blowfish' : self.run_john_the_ripper_blowfish,
-            'john-the-ripper-des': self.run_john_the_ripper_des,
             'openssl': self.run_openssl,
             'osbench-files': self.run_osbench_files,
             'osbench-processes': self.run_osbench_files,
diff --git a/ubuntu_performance_pts/user-config.xml b/ubuntu_performance_pts/user-config.xml
index de80ed8..19ba6cb 100644
--- a/ubuntu_performance_pts/user-config.xml
+++ b/ubuntu_performance_pts/user-config.xml
@@ -56,8 +56,8 @@
       <NoInternetCommunication>FALSE</NoInternetCommunication>
       <NoNetworkCommunication>FALSE</NoNetworkCommunication>
       <Timeout>20</Timeout>
-      <ProxyAddress>squid.internal</ProxyAddress>
-      <ProxyPort>3128</ProxyPort>
+      <ProxyAddress></ProxyAddress>
+      <ProxyPort></ProxyPort>
       <ProxyUser></ProxyUser>
       <ProxyPassword></ProxyPassword>
     </Networking>