MRE updates 19.11.14 / 21.11.3 / 22.11.1

Bug #2002404 reported by Christian Ehrhardt 
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
dpdk (Ubuntu)
Fix Released
Undecided
Unassigned
Focal
Fix Released
Undecided
Christian Ehrhardt 
Jammy
Fix Released
Undecided
Christian Ehrhardt 
Kinetic
Fix Released
Undecided
Christian Ehrhardt 
Lunar
Fix Released
Undecided
Unassigned

Bug Description

Now that 22.11.1 is complete in lunar we can look at that.

Hi,
This bug tracks an update for the DPDK packages in:
- Focal 19.11.13 -> 19.11.14
- Jammy & Kinetic 21.11.2 -> 21.11.3

This update includes bugfixes only following the SRU policy exception defined at https://wiki.ubuntu.com/StableReleaseUpdates/DPDK.

No new features added, a classic stable release with a bunch of fixes aggregated and enhanced testing by the companies being part of the DPDK community.

[Impact]

Stable release update so not directly applicable; see the exception policy document linked above.
For Reference - former cases are here:
- https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1784816
- https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1817675
- https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1836365
- https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1912464
- https://bugs.launchpad.net/ubuntu/+source/dpdk/+bug/1940913

[Major Changes]

Listed by upstream in detail

Focal - 19.11.14:
https://doc.dpdk.org/guides-19.11/rel_notes/release_19_11.html#id45

Jammy & Kinetic - 21.11.3:
https://doc.dpdk.org/guides-21.11/rel_notes/release_21_11.html#id5

[Test Plan]

See https://wiki.ubuntu.com/StableReleaseUpdates/DPDK#SRU_TestVerify

[Regression Potential]

Upstream performs extensive testing before release, giving us a high degree of confidence in the general case. There problems are most likely to manifest in Ubuntu-specific integrations, such as in relation to the versions of dependencies available and other packaging-specific matters.
Therefore that is what our verification focuses on.

Related branches

tags: added: server-todo
Changed in dpdk (Ubuntu Lunar):
status: New → Fix Released
Changed in dpdk (Ubuntu Kinetic):
status: New → Confirmed
Changed in dpdk (Ubuntu Jammy):
status: New → Confirmed
Changed in dpdk (Ubuntu Focal):
status: New → Confirmed
description: updated
summary: - MRE updates 19.11.14 and 20.11.7
+ MRE updates 19.11.14 / 21.11.3 / 22.11.1
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :
Changed in dpdk (Ubuntu Focal):
assignee: nobody → Christian Ehrhardt  (paelzer)
Changed in dpdk (Ubuntu Jammy):
assignee: nobody → Christian Ehrhardt  (paelzer)
Changed in dpdk (Ubuntu Kinetic):
assignee: nobody → Christian Ehrhardt  (paelzer)
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

All built fine, waiting for dep8 tests and MR review.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

MRs approved, tests ok => uploaded to -unapproved

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

FYI over the weekend 2.11.1 also migrated to lunar-release together with python.
With that there should be nothing left stalling this except finding time to SRU review, accept and test it, but since it isn't super-urgent I can wait until I see the update here on the bug that it is accepted to -unapproved.

Changed in dpdk (Ubuntu Focal):
status: Confirmed → In Progress
Changed in dpdk (Ubuntu Jammy):
status: Confirmed → In Progress
Changed in dpdk (Ubuntu Kinetic):
status: Confirmed → In Progress
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Christian, or anyone else affected,

Accepted dpdk into kinetic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dpdk/21.11.3-0ubuntu0.22.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-kinetic to verification-done-kinetic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-kinetic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in dpdk (Ubuntu Kinetic):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-kinetic
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Christian, or anyone else affected,

Accepted dpdk into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dpdk/21.11.3-0ubuntu0.22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in dpdk (Ubuntu Jammy):
status: In Progress → Fix Committed
tags: added: verification-needed-jammy
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Christian, or anyone else affected,

Accepted dpdk into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dpdk/19.11.14-0ubuntu0.20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in dpdk (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed-focal
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (dpdk/21.11.3-0ubuntu0.22.04.1)

All autopkgtests for the newly accepted dpdk (21.11.3-0ubuntu0.22.04.1) for jammy have finished running.
The following regressions have been reported in tests triggered by the package:

dpdk/21.11.3-0ubuntu0.22.04.1 (arm64, amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/jammy/update_excuses.html#dpdk

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Test issues:
- arm64 was resolved by a retry, that is due to some chip features in our test environemnt which the next release will make non-flaky
- amd64 was a timeout on something that worked fine before and had no changes
  - local retry good
  - retry in infrastructure good as well now

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Detailed regression tests started:
- Focal with 19.11.14-0ubuntu0.20.04.1 fully passed

PHYS_TESTPMD
VIRT_SPAWNGUESTS
VIRT_TESTPMD
VIRT_L2FWD
BENCH_OVS
VUC_BENCH_OVSDPDK
VUC_ENDURANCE_STARTSTOP
VUC_ENDURANCE_ADDREMOVEPORT

1.0.0 (09:51:59): phys (BM) tests
  1.1.0 (09:51:59): initialize environment
    1.1.1 (09:56:05): testpmd => Pass
    1.1.2 (09:57:06): check testpmd output => Pass
2.0.0 (09:57:06): prep virtual test environment

1.0.0 (09:59:44): virt tests
  1.1.0 (09:59:44): initialize environment
3.0.0 (10:00:16): performance tests
  3.1.0 (10:00:16): prep benchmarks
  3.2.0 (10:00:41): performance tests
    3.2.1 (10:00:50): test guest-openvswitch for OVS-5CPU => Pass
    3.2.2 (10:20:38): test guest-dpdk-vhost-user-client-multiq for OVSDPDK-VUC => Pass
4.0.0 (10:41:55): VUC endurance checks
  4.1.0 (10:41:55): prep VUC endurance tests
    4.1.1 (10:58:26): start stop guests (client) => Pass
    4.1.2 (12:20:12): add/remove ports (client) => Pass
  4.2.0 (12:29:45): Final cleanup+ echo ''

Now upgrading to Jammy

tags: added: verification-done-focal
removed: verification-needed-focal
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Jammy: 21.11.3-0ubuntu0.22.04.1

VIRT_SPAWNGUESTS
VIRT_TESTPMD
VIRT_L2FWD
BENCH_OVS
VUC_BENCH_OVSDPDK
VUC_ENDURANCE_STARTSTOP
VUC_ENDURANCE_ADDREMOVEPORT

1.0.0 (07:46:45): phys (BM) tests
  1.1.0 (07:46:45): initialize environment
    1.1.1 (07:48:06): testpmd => Pass
    1.1.2 (07:48:56): check testpmd output => Pass
2.0.0 (07:48:56): prep virtual test environment

1.0.0 (07:51:13): virt tests
  1.1.0 (07:51:13): initialize environment
3.0.0 (07:52:08): performance tests
  3.1.0 (07:52:08): prep benchmarks
  3.2.0 (07:52:29): performance tests
    3.2.1 (07:52:37): test guest-openvswitch for OVS-5CPU => Pass
    3.2.2 (08:11:41): test guest-dpdk-vhost-user-client-multiq for OVSDPDK-VUC => Pass
4.0.0 (08:30:54): VUC endurance checks
  4.1.0 (08:30:54): prep VUC endurance tests
    4.1.1 (08:42:23): start stop guests (client) => Pass
    4.1.2 (09:54:44): add/remove ports (client) => Pass
  4.2.0 (10:04:12): Final cleanup

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Sory, but Kinetic 21.11.3-0ubuntu0.22.10.1 takes a while.

It seems there is an unrelated change that breaks ipv4/dhcp in some of the testbeds.
I'm tracking this down if it is due to the new DPDK or anything else, but that will take some extra time.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

The openvswitch run in between is broken even without running against proposed.
Actually it is the external network configuration which hands me no IP via dhcp anymore.
That leads to issues doing apt-updates from the guests once connected directly to the external network via OVS or OVS-DPDK.

I've now run all other tests, they are good:

1.0.0 (07:00:18): phys (BM) tests
  1.1.0 (07:00:18): initialize environment
    1.1.1 (07:01:35): testpmd => Pass
    1.1.2 (07:02:24): check testpmd output => Pass
2.0.0 (07:02:24): prep virtual test environment

With some manual pondering I got the OVS-DPDK based run with vhostuserclient connections (the one we most care for) done:
    3.2.1 (07:52:15): test guest-dpdk-vhost-user-client-multiq for OVSDPDK-VUC => Pass

^^ sounds like 50 minutes later, but that took 3 days to get working :-/

1. Just as we do with migration-reference/0 in autopkgtest I have verified that those sub-cases fail with proposed just as much.
2. We know from the many runs so far that it also affects not only non-proposed but also pure OVS runs (without DPDK in the mix).
3. Furthermore it seems mostly flaky, failing on different stages of the test each time.

Gladly Jammy and Focal (more important) were fine and fully complete before this happened.

I'd consider kinetic verified to the extend it can be verified at the moment.
If you object, then just wait until kinetic is EOL and update just all the others (I think that would be silly, but fulfill the requirements without spending too much time while being blocked on this).

tags: added: verification-done verification-done-jammy verification-done-kinetic
removed: verification-needed verification-needed-jammy verification-needed-kinetic
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

I verified the test results and am satisfied that they show the executed planned test case, and that the results are correct.

I understand that kinetic needed some hand holding, and that the current dpdk in kinetic has the same issues, and pure OVS tests runs (without DPDK) are also affected.

The package built correctly in all architectures and Ubuntu releases it was meant for.

There are no DEP8 regressions.

There is a focal SRU freeze at the moment, so the focal packages cannot be accepted at the moment.

There is no halted phasing on the previous update.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dpdk - 21.11.3-0ubuntu0.22.10.1

---------------
dpdk (21.11.3-0ubuntu0.22.10.1) kinetic; urgency=medium

  * Merge LTS stable release 21.11.3 (LP: #2002404)
    fixing various issues from descriptions to hard crashes
    - Full release notes are available at:
      https://doc.dpdk.org/guides-21.11/rel_notes/release_21_11.html#id5

 -- Christian Ehrhardt <email address hidden> Tue, 17 Jan 2023 11:33:59 +0100

Changed in dpdk (Ubuntu Kinetic):
status: Fix Committed → Fix Released
Revision history for this message
Andreas Hasenack (ahasenack) wrote : Update Released

The verification of the Stable Release Update for dpdk has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dpdk - 21.11.3-0ubuntu0.22.04.1

---------------
dpdk (21.11.3-0ubuntu0.22.04.1) jammy; urgency=medium

  * Merge LTS stable release 21.11.3 (LP: #2002404)
    fixing various issues from descriptions to hard crashes
    - Full release notes are available at:
      https://doc.dpdk.org/guides-21.11/rel_notes/release_21_11.html#id5

 -- Christian Ehrhardt <email address hidden> Tue, 17 Jan 2023 11:31:40 +0100

Changed in dpdk (Ubuntu Jammy):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package dpdk - 19.11.14-0ubuntu0.20.04.1

---------------
dpdk (19.11.14-0ubuntu0.20.04.1) focal; urgency=medium

  * Merge LTS stable release 19.11.13 (LP: #2002404)
    fixing various issues from descriptions to hard crashes
    - Full release notes are available at:
      https://doc.dpdk.org/guides-19.11/rel_notes/release_19_11.html#id45

 -- Christian Ehrhardt <email address hidden> Tue, 17 Jan 2023 11:48:20 +0100

Changed in dpdk (Ubuntu Focal):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.