mask current unreliable tests in bionic and cosmic

Bug #1806347 reported by Christian Ehrhardt 
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
systemd (Ubuntu)
Fix Released
Undecided
Unassigned
Bionic
New
Undecided
Unassigned
Cosmic
New
Undecided
Unassigned

Bug Description

Hi,
a recent SRU made me check the success rate of systemd autopkgtests.
It looks grim atm:

None of the fails look related to the actual uploads BTW

Bionic:
last 7 ppc64 - http://autopkgtest.ubuntu.com/packages/s/systemd/bionic/ppc64el
last 7 s390x - http://autopkgtest.ubuntu.com/packages/s/systemd/bionic/s390x
At least those with "last 7" of them is exactly since systemd/237-3ubuntu10.10 and only the "upstream" test is failing.

last 26 amd64 - http://autopkgtest.ubuntu.com/packages/s/systemd/bionic/amd64
last 12 i386 - http://autopkgtest.ubuntu.com/packages/s/systemd/bionic/i386
If anything those fail since the last kernel update, but that isn't sure.

last 80 armhf - http://autopkgtest.ubuntu.com/packages/s/systemd/bionic/armhf
Here always "root-unittests" fails, maybe not container compatible?

Cosmic:
last ~40 ppc64 - http://autopkgtest.ubuntu.com/packages/s/systemd/cosmic/ppc64el
In addition to the "upstream" fails seen in Bionic there are some gdm induced errors which we just fixed in https://git.launchpad.net/~ubuntu-core-dev/ubuntu/+source/systemd/commit/?id=3006fedda1d1ca3f04c5f593e8018bb6d1196025

last 4 s390x - http://autopkgtest.ubuntu.com/packages/s/systemd/cosmic/s390x
last 16 on amd64 - http://autopkgtest.ubuntu.com/packages/s/systemd/cosmic/amd64
last 8 arm64 - http://autopkgtest.ubuntu.com/packages/s/systemd/cosmic/arm64
last 4 i386 - http://autopkgtest.ubuntu.com/packages/s/systemd/cosmic/i386
The tests that fail most of the time are "boot-smoke" and upstream"

In general the test "upstream" fails most (amd64/s390x/ppc64).
Maybe that one should be (partially) disabled on the next upload as the rest is at least semi reliable (doable with retries).

The same applies for test "root-unittests" on armhf, please just skip that one and retain the rest.

So maybe on the next systemd update those could be addressed or at least improved, but for now this is fake-blocking way too many SRUs - to lets add an all arch badtest for now.

TL;DR on changes that I'd request for next systemd upload:
- Bionic skip "upstream" test (unless you have the magic dust to make it good again)
- Bionic skip root-unittest on armhf
- Cosmic skip "upstream" test
- Cosmic also implement gdm3 fix we just had in Disco
- Cosmic skip boot-smoke test on arm64 & amd64

I'm leaving this bug open to consider it on the next systemd upload as suggested.
For now I see no other way than to propose an MP to force-badtest.

Changed in systemd (Ubuntu):
status: New → Fix Released
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

(for now) Britney hints are supplied to unblock several SRUs that block on these tests while not causing it.
This bug will stay open to (hopefully) track and address it for real on the next systemd upload.

=> https://code.launchpad.net/~paelzer/britney/hints-ubuntu-bionic-sru-systemd/+merge/359987
=> https://code.launchpad.net/~paelzer/britney/hints-ubuntu-cosmic-sru-systemd/+merge/359986

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Issues were fixed by the last upload implementing most of the suggested fixes.
At least the more recent tests all worked for me.
Cancelling the MPs and set to fix released.

Changed in systemd (Ubuntu Bionic):
status: New → Fix Released
Changed in systemd (Ubuntu Cosmic):
status: New → Fix Released
Changed in systemd (Ubuntu Bionic):
status: Fix Released → New
Changed in systemd (Ubuntu Cosmic):
status: Fix Released → New
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

After actually waking up I realized that they still are broken, retriggering the tests to make sure they really work.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

No, in fact it is still as bad :-/
@xnox since the MPs marking them as badtest for now were ignored, should I open them up again or how do we plan to resolve that short term?

Can/Should I ask the SRU team to ignore them (instead of the MP to make Britney do the same)?

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.