curtin

skip_by_date shows MTU tests as passing when we believe they should be failing

Bug #1855148 reported by Dan Watkins
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
curtin
Fix Committed
Medium
Unassigned

Bug Description

e.g. in https://jenkins.ubuntu.com/server/job/curtin-vmtest-devel-amd64-proposed/84/

09:16:36 ======================================================================
09:16:36 ERROR: test suite for <class 'vmtests.test_network_mtu.EoanTestNetworkMtu'>
09:16:36 ----------------------------------------------------------------------
09:16:36 Traceback (most recent call last):
09:16:36 File "/usr/lib/python3/dist-packages/nose/plugins/multiprocess.py", line 788, in run
09:16:36 self.setUp()
09:16:36 File "/usr/lib/python3/dist-packages/nose/suite.py", line 292, in setUp
09:16:36 self.setupContext(ancestor)
09:16:36 File "/usr/lib/python3/dist-packages/nose/plugins/multiprocess.py", line 770, in setupContext
09:16:36 super(NoSharedFixtureContextSuite, self).setupContext(context)
09:16:36 File "/usr/lib/python3/dist-packages/nose/suite.py", line 315, in setupContext
09:16:36 try_run(context, names)
09:16:36 File "/usr/lib/python3/dist-packages/nose/util.py", line 471, in try_run
09:16:36 return func()
09:16:36 File "/var/lib/jenkins/servers/server/workspace/curtin-vmtest-devel-amd64-proposed/curtin-84/tests/vmtests/__init__.py", line 495, in wrapper
09:16:36 raise RuntimeError(msg)
09:16:36 RuntimeError: skip_by_date(EoanTestNetworkMtu.setUpClass) LP: #1671951 fixby=2019-10-02 removeby=2019-10-23: (REMOVE_WORKAROUND) Passed.

Related branches

~raharper/curtin:fix/vmtest-network-vlan-skip
Dan Watkins (community): Approve
Server Team CI bot: Approve (continuous-integration)
Diff: 42 lines (+6/-6)
2 files modified
tests/vmtests/test_network_ipv6_vlan.py (+2/-2)
tests/vmtests/test_network_vlan.py (+4/-4)
Ryan Harper (raharper)
Changed in curtin:
importance: Undecided → Medium
status: New → Triaged
Revision history for this message
Ryan Harper (raharper) wrote :

I've sorted this out. Once the skip_by_date fixby date has passed, we invoke the test; in this case the test looked like this:

def test_ip_output(self):
   pass

Which passes; but without actually running the test.

What we need to do instead is:

def test_ip_output(self):
    super().test_ip_output()

Revision history for this message
Dan Watkins (oddbloke) wrote : Re: [Bug 1855148] Re: skip_by_date shows MTU tests as passing when we believe they should be failing

On Tue, Feb 04, 2020 at 09:04:05PM -0000, Ryan Harper wrote:
> I've sorted this out. Once the skip_by_date fixby date has passed, we
> invoke the test; in this case the test looked like this:
>
> def test_ip_output(self):
> pass
>
> Which passes; but without actually running the test.
>
> What we need to do instead is:
>
> def test_ip_output(self):
> super().test_ip_output()

Could we achieve the same by entirely omitting the definition of
test_ip_output in the subclass?

Revision history for this message
Ryan Harper (raharper) wrote :

On Tue, Feb 4, 2020 at 3:20 PM Dan Watkins <email address hidden>
wrote:

> On Tue, Feb 04, 2020 at 09:04:05PM -0000, Ryan Harper wrote:
> > I've sorted this out. Once the skip_by_date fixby date has passed, we
> > invoke the test; in this case the test looked like this:
> >
> > def test_ip_output(self):
> > pass
> >
> > Which passes; but without actually running the test.
> >
> > What we need to do instead is:
> >
> > def test_ip_output(self):
> > super().test_ip_output()
>
> Could we achieve the same by entirely omitting the definition of
> test_ip_output in the subclass?
>

I only wanted to skip this one test, so I need to override it to decorate
it.

>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1855148
>
> Title:
> skip_by_date shows MTU tests as passing when we believe they should be
> failing
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/curtin/+bug/1855148/+subscriptions
>

Revision history for this message
Dan Watkins (oddbloke) wrote :

On Tue, Feb 04, 2020 at 09:26:20PM -0000, Ryan Harper wrote:
> I only wanted to skip this one test, so I need to override it to decorate
> it.

Aha, makes perfect sense. Thanks!

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

This bug is fixed with commit 2a2bcf03 to curtin on branch master.
To view that commit see the following URL:
https://git.launchpad.net/curtin/commit/?id=2a2bcf03

Changed in curtin:
status: Triaged → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.