Unit test failure with OpenSSL 1.1.1

seems some key deprecated ? can we check [0] above to know which of follow param lead to error?
best way would be within 1.1.1 env to consturct a command string and try it ..

|RuntimeError: OpenSSL error: *** WARNING : deprecated key derivation used.
|Using -iter or -pbkdf2 would be better.

    def _run_ssl(self, text, decrypt=False):
        cmd = ['openssl', 'aes-128-cbc', '-A', '-a', '-pass',
               'pass:%s' % self._shared, '-nosalt']
        if decrypt:
            cmd.append('-d')
        out, err = utils.execute(*cmd,
                                 process_input=encodeutils.safe_encode(text))
        if err:
            raise RuntimeError(_('OpenSSL error: %s') % err)
        return out

I'm hitting this as well now that we have openssl 1.1.1 in cosmic-proposed. This is affecting rocky and above for ubuntu. Unfortunately this is preventing our unit tests from running successfully for an 18.0.1 release. To recreate:

lxc launch ubuntu-daily:cosmic c1
lxc exec c1 /bin/bash
root@c1:~# cat >> /etc/apt/sources.list << EOF
deb http://archive.ubuntu.com/ubuntu cosmic-proposed main restricted
deb http://archive.ubuntu.com/ubuntu cosmic-proposed universe
EOF
root@c1:~# sudo apt update
root@c1:~# sudo apt dist-upgrade --yes
root@c1:~# apt policy openssl # should be at openssl 1.1.1-1ubuntu2
root@c1:~# sudo apt install python-dev git gcc tox --yes
root@c1:~# git clone https://github.com/openstack/nova
root@c1:~# cd nova
root@c1:~/nova# tox -e py27 # results in failures: https://paste.ubuntu.com/p/3W39Vy87Sy/

By any chance can the importance of this bug be increased?

I also hit this bug! Any news since then?

This is not just a testing issue, it means that xenapi will not able to talk to xen agent at runtime, with openssl 1.1.1 binary.

Since openssl binary is executed, it's a bit hard to determine if it failed or not. As it generates genuine errors and warning in stderr.

In this case the password derivation function has been deprecated in OpenSSL but it still works. I don't know what xen api agent can or cannot accept, thus I don't think it is safe to upgrade the openssl command to use stronger key derivation. Instead, we should whitelist the harmless warning and not treat it as an error.

I do not believe the string is translated in OpenSSL upstream.

Please see the attached path.

It would be copyright canonical, with OpenStack CLA signed. But i'm not sure when I will have time to submit this patch upstream properly.

Fix proposed to branch: master
Review: https://review.openstack.org/635533

@xnox, thanks for the patch. I've submitted it to the upstream master branch. Once that lands I'll start backporting to stable branches and Ubuntu.

Reviewed: https://review.opendev.org/635533
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=1da71fa4ab1d7d0f580cd5cbc97f2dfd2e1c378a
Submitter: Zuul
Branch: master

commit 1da71fa4ab1d7d0f580cd5cbc97f2dfd2e1c378a
Author: Corey Bryant <email address hidden>
Date: Thu Feb 7 10:12:54 2019 -0500

    xenapi/agent: Change openssl error handling

    Prior to this patch, if the openssl command returned a zero exit code
    and wrote details to stderr, nova would raise a RuntimeError exception.
    This patch changes the behavior to only raise a RuntimeError exception
    when openssl returns a non-zero exit code. Regardless of the exit code
    a warning will always be logged with stderr details if stderr is not
    None. Note that processutils.execute will now raise a
    processutils.ProcessExecutionError exception for any non-zero exit code
    since we are passing check_exit_code=True, which we convert to a
    Runtime error.

    Thanks to Dimitri John Ledkov <email address hidden> and Eric Fried
    <email address hidden> for helping with this patch.

    Change-Id: I212ac2b5ccd93e00adb7b9fe102fcb70857c6073
    Partial-Bug: #1771506

This bug was fixed in the package nova - 2:19.0.0-0ubuntu4

---------------
nova (2:19.0.0-0ubuntu4) eoan; urgency=medium

  * d/p/xenapi-agent-change-openssl-error-handling.patch: Cherry-picked from
    upstream to ensure xenapi agent only raises a RuntimeError exception
    when openssl returns a non-zero exit code (LP: #1771506).

 -- Corey Bryant <email address hidden> Wed, 01 May 2019 17:10:47 -0400

New versions of nova with this fix have been uploaded to eoan, disco, cosmic, and bionic. Stable release uploads are awaiting review from the SRU team [1].

[1]
https://launchpad.net/ubuntu/disco/+queue?queue_state=1&queue_text=nova
https://launchpad.net/ubuntu/cosmic/+queue?queue_state=1&queue_text=nova
https://launchpad.net/ubuntu/bionic/+queue?queue_state=1&queue_text=nova

An upload of nova to disco-proposed has been rejected from the upload queue for the following reason: "The .changes file doesn't incorporate changes in 2:19.0.0-0ubuntu2.1 please reupload.".

Hello Thomas, or anyone else affected,

Accepted nova into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:19.0.0-0ubuntu2.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Thomas, or anyone else affected,

Accepted nova into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:18.1.0-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Thomas, or anyone else affected,

Accepted nova into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:17.0.9-0ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Thomas, or anyone else affected,

Accepted nova into stein-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:stein-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-stein-needed to verification-stein-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-stein-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Thomas, or anyone else affected,

Accepted nova into rocky-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:rocky-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-rocky-needed to verification-rocky-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-rocky-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Thomas, or anyone else affected,

Accepted nova into queens-proposed. The package will build now and be available in the Ubuntu Cloud Archive in a few hours, and then in the -proposed repository.

Please help us by testing this new package. To enable the -proposed repository:

  sudo add-apt-repository cloud-archive:queens-proposed
  sudo apt-get update

Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-queens-needed to verification-queens-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-queens-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Thomas, or anyone else affected,

Accepted nova into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:19.0.0-0ubuntu2.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Thomas, or anyone else affected,

Accepted nova into cosmic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:18.1.0-0ubuntu3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-cosmic to verification-done-cosmic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-cosmic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello Thomas, or anyone else affected,

Accepted nova into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/nova/2:17.0.9-0ubuntu3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Bionic is great, previously it would fail nova.tests.unit.virt.xenapi.test_xenapi.XenAPIDiffieHellmanTestCase tests but now they pass.

Cosmic/Disco/Eoan are also correctly fixed at runtime, however the unittests that exercise this runtime issue are force skipped since the skip-openssl-1.1.1-tests.patch is still applied. We should drop skip-openssl-1.1.1-tests.patch from Cosmic/Disco/Eoan in the subsequent uploads. I've now uploaded dropping skip-openssl-1.1.1-tests.patch into Eoan.

Pass (with nitpicks on cosmic/disco).

@Dimitri, thanks very much. I've pushed changes to cosmic and disco branches to drop the skip-openssl-1.1.1-tests.patch patch and have built them successfully (locally) for disco, cosmic, bionic-stein, and bionic-rocky. I'm going to hold off on uploads just for that change as nova has a lot of churn and they'll get picked up on the next SRU.

This has also built successfully in the queens cloud archive.

Reviewed: https://review.opendev.org/656307
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=8241b47967adb792a4254eeb58fda1fc55edf314
Submitter: Zuul
Branch: stable/rocky

commit 8241b47967adb792a4254eeb58fda1fc55edf314
Author: Corey Bryant <email address hidden>
Date: Thu Feb 7 10:12:54 2019 -0500

    xenapi/agent: Change openssl error handling

    Prior to this patch, if the openssl command returned a zero exit code
    and wrote details to stderr, nova would raise a RuntimeError exception.
    This patch changes the behavior to only raise a RuntimeError exception
    when openssl returns a non-zero exit code. Regardless of the exit code
    a warning will always be logged with stderr details if stderr is not
    None. Note that processutils.execute will now raise a
    processutils.ProcessExecutionError exception for any non-zero exit code
    since we are passing check_exit_code=True, which we convert to a
    Runtime error.

    Thanks to Dimitri John Ledkov <email address hidden> and Eric Fried
    <email address hidden> for helping with this patch.

    Conflicts:
        nova/virt/xenapi/agent.py

    NOTE(coreycb): The conflict is due to
    Ibe2f478288db42f8168b52dfc14d85ab92ace74b not being in stable/rocky.

    Change-Id: I212ac2b5ccd93e00adb7b9fe102fcb70857c6073
    Partial-Bug: #1771506
    (cherry picked from commit 1da71fa4ab1d7d0f580cd5cbc97f2dfd2e1c378a)
    (cherry picked from commit 64793cf6f77c5ba7c9ea51662d936c7545ffce8c)

Reviewed: https://review.opendev.org/656304
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=64793cf6f77c5ba7c9ea51662d936c7545ffce8c
Submitter: Zuul
Branch: stable/stein

commit 64793cf6f77c5ba7c9ea51662d936c7545ffce8c
Author: Corey Bryant <email address hidden>
Date: Thu Feb 7 10:12:54 2019 -0500

    xenapi/agent: Change openssl error handling

    Prior to this patch, if the openssl command returned a zero exit code
    and wrote details to stderr, nova would raise a RuntimeError exception.
    This patch changes the behavior to only raise a RuntimeError exception
    when openssl returns a non-zero exit code. Regardless of the exit code
    a warning will always be logged with stderr details if stderr is not
    None. Note that processutils.execute will now raise a
    processutils.ProcessExecutionError exception for any non-zero exit code
    since we are passing check_exit_code=True, which we convert to a
    Runtime error.

    Thanks to Dimitri John Ledkov <email address hidden> and Eric Fried
    <email address hidden> for helping with this patch.

    Change-Id: I212ac2b5ccd93e00adb7b9fe102fcb70857c6073
    Partial-Bug: #1771506
    (cherry picked from commit 1da71fa4ab1d7d0f580cd5cbc97f2dfd2e1c378a)

This bug was fixed in the package nova - 2:17.0.9-0ubuntu3

---------------
nova (2:17.0.9-0ubuntu3) bionic; urgency=medium

  * d/p/bug_1825882.patch: Cherry-picked from upstream to ensure
    virsh disk attach does not fail silently (LP: #1825882).
  * d/p/bug_1826523.patch: Cherry-picked from upstream to ensure
    always disconnect volumes after libvirt exceptions (LP: #1826523).

nova (2:17.0.9-0ubuntu2) bionic; urgency=medium

  * d/p/xenapi-agent-change-openssl-error-handling.patch: Cherry-picked from
    upstream to ensure xenapi agent only raises a RuntimeError exception
    when openssl returns a non-zero exit code (LP: #1771506).
  * d/p/skip-double-word-hacking-test.patch: Cherry-picked from upstream
    to work-around test_hacking failures with new versions of python
    (LP: #1782786).

 -- Sahid Orentino Ferdjaoui <email address hidden> Thu, 16 May 2019 11:06:11 +0200

This bug was fixed in the package nova - 2:18.1.0-0ubuntu3

---------------
nova (2:18.1.0-0ubuntu3) cosmic; urgency=medium

  * d/p/bug_1825882.patch: Cherry-picked from upstream to ensure
    virsh disk attach does not fail silently (LP: #1825882).
  * d/p/bug_1826523.patch: Cherry-picked from upstream to ensure
    always disconnect volumes after libvirt exceptions (LP: #1826523).

nova (2:18.1.0-0ubuntu2) cosmic; urgency=medium

  * d/p/xenapi-agent-change-openssl-error-handling.patch: Cherry-picked from
    upstream to ensure xenapi agent only raises a RuntimeError exception
    when openssl returns a non-zero exit code (LP: #1771506).

 -- Sahid Orentino Ferdjaoui <email address hidden> Thu, 16 May 2019 10:58:45 +0200

Reviewed: https://review.opendev.org/656308
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=5b0adaa0ca5f757bb224d1ffac0c6705b03ee2ed
Submitter: Zuul
Branch: stable/queens

commit 5b0adaa0ca5f757bb224d1ffac0c6705b03ee2ed
Author: Corey Bryant <email address hidden>
Date: Thu Feb 7 10:12:54 2019 -0500

    xenapi/agent: Change openssl error handling

    Prior to this patch, if the openssl command returned a zero exit code
    and wrote details to stderr, nova would raise a RuntimeError exception.
    This patch changes the behavior to only raise a RuntimeError exception
    when openssl returns a non-zero exit code. Regardless of the exit code
    a warning will always be logged with stderr details if stderr is not
    None. Note that processutils.execute will now raise a
    processutils.ProcessExecutionError exception for any non-zero exit code
    since we are passing check_exit_code=True, which we convert to a
    Runtime error.

    Thanks to Dimitri John Ledkov <email address hidden> and Eric Fried
    <email address hidden> for helping with this patch.

    Conflicts:
        nova/virt/xenapi/agent.py

    NOTE(coreycb): The conflict is due to
    Ibe2f478288db42f8168b52dfc14d85ab92ace74b not being in stable/queens.

    Change-Id: I212ac2b5ccd93e00adb7b9fe102fcb70857c6073
    Partial-Bug: #1771506
    (cherry picked from commit 1da71fa4ab1d7d0f580cd5cbc97f2dfd2e1c378a)
    (cherry picked from commit 64793cf6f77c5ba7c9ea51662d936c7545ffce8c)
    (cherry picked from commit 82de38ad4ce86c5398538a8635713a86407216d0)

The verification of the Stable Release Update for nova has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package nova - 2:19.0.0-0ubuntu2.3

---------------
nova (2:19.0.0-0ubuntu2.3) disco; urgency=medium

  * d/p/bug_1825882.patch: Cherry-picked from upstream to ensure
    virsh disk attach does not fail silently (LP: #1825882).
  * d/p/bug_1826523.patch: Cherry-picked from upstream to ensure
    always disconnect volumes after libvirt exceptions (LP: #1826523).

nova (2:19.0.0-0ubuntu2.2) disco; urgency=medium

  * d/p/xenapi-agent-change-openssl-error-handling.patch: Cherry-picked from
    upstream to ensure xenapi agent only raises a RuntimeError exception
    when openssl returns a non-zero exit code (LP: #1771506).

nova (2:19.0.0-0ubuntu2.1) disco; urgency=medium

  * d/gbp.conf: Create stable/stein branch.
  * d/p/eventlet-monkey-patching-should-be-as-early-as-possible.patch:
    Cherry-picked from upstream stable/stein review to fix py3+wsgi+ssl crash
    (LP: #1808951).

 -- Sahid Orentino Ferdjaoui <email address hidden> Thu, 16 May 2019 10:54:46 +0200