Merge lp:~raharper/curtin/trunk.vmtest-remove-bug-timers into lp:~curtin-dev/curtin/trunk

[Ryan Harper]

vmtest: fix artful networking

- Remove skip_by_date calls in vmtests that are no longer valid
- ifenslave brings in ifupdown, so filter out ifenslave package if target
  release is artful
- Adjust artful bonding test to skip checking for ifslave and instead check
  that ifenslave is *not* installed
- Drop mtu settings in network-static-routes configuration, (not needed)
- Move static routes under an interface subnet to be compatible with
  netplan format which requires routes under an interface
- Refactor ip_route_show parsing, ignoring default route and fetching
  variable settings like 'proto'.
- Skiptest bridging in artful, need a cloud-init fix for stp in netplan
- Skiptest for mtu in artful, need networkd to support mtu6 support

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:532
https://jenkins.ubuntu.com/server/job/curtin-ci/644/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/644
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/644
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/644
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/644

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/644/rebuild

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

For testing, generally this affects the following path:

./tools/jenkins-runner -vv --nologcapture tests/vmtests/test_network*

Note, that while we're specifically looking for all of the ArtfulTest* tests to now pass (or skip)
we need to ensure that we don't regress non-artful releases.

[Chad Smith (chad.smith)]

Looks pretty good, couple of nits and questions.

[Ryan Harper (raharper)]

Thanks for the review, I'll fix up several things you've mentioned.

[lp:~raharper/curtin/trunk.vmtest-remove-bug-timers]

533. By Ryan Harper on 2017-10-04

Fix spelling of 'additional' in curthooks log message

534. By Ryan Harper on 2017-10-04

Remove unneeded .lower(), switch to an equality check

535. By Ryan Harper on 2017-10-04

Raise a ValueError in ip_route_show regex failure to match

536. By Ryan Harper on 2017-10-04

Just check for empty dpkg-query status in Artful Bonding tests

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:536
https://jenkins.ubuntu.com/server/job/curtin-ci/645/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/645
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/645
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/645
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/645

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/645/rebuild

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

On Thu, Oct 5, 2017 at 11:21 AM, Scott Moser <email address hidden> wrote:

>
>
> Diff comments:
>
> > === modified file 'curtin/commands/curthooks.py'
> > --- curtin/commands/curthooks.py 2017-08-04 19:53:29 +0000
> > +++ curtin/commands/curthooks.py 2017-10-04 20:53:13 +0000
> > @@ -687,6 +689,19 @@
> > if pkg not in needed_packages:
> > needed_packages.append(pkg)
> >
> > + # do not install ifenslave if target release is artful as it
> > + # triggers an install of ifupdown which will break network rendering
> > + if 'ifenslave' in needed_packages:
> > + codename, _ = util.subp(['lsb_release', '--codename',
> '--short'],
>
> util.lsb_release(target=target)
>

I didn't want the dict =); will fix.

>
> > + capture=True, target=target)
> > + # drop the newline
> > + codename = codename.strip()
> > + LOG.debug('Target release codename: %s', codename)
> > + if codename == 'artful':
> > + LOG.debug("Skipping install of package 'ifenslave' to
> prevent"
> > + " network configutation errors on release %s",
> codename)
> > + needed_packages.remove('ifenslave')
>
> what about bridge-utils and vlan ?
>

bridge-utils recommends ifupdown but does not install
vlan does not bring in ifupdown either.

>
> it looks like we will install those in some scenarios too.
> maybe they dont break stuff, but ... fix ?
>
> > +
> > if needed_packages:
> > state = util.load_command_environment()
> > with events.ReportEventStack(
>
>
> --
> https://code.launchpad.net/~raharper/curtin/trunk.vmtest-
> remove-bug-timers/+merge/331757
> You are the owner of lp:~raharper/curtin/trunk.vmtest-remove-bug-timers.
>

[lp:~raharper/curtin/trunk.vmtest-remove-bug-timers]

537. By Ryan Harper on 2017-10-05

Use util.lsb_release()

[Scott Moser (smoser)]

but vlan and bridge-utils are not needed or desired in artful, right?

[Scott Moser (smoser)]

that means best case we install some stuff that odesnt get used and slow down the install.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:537
https://jenkins.ubuntu.com/server/job/curtin-ci/646/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/646
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/646
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/646
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/646

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/646/rebuild

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

vlan appears to be installed already,
bridge-utils while not needed directly to create bridges, is highly useful
for modifying/displaying bridge config. I don't think it's bad to install
it.

On Thu, Oct 5, 2017 at 11:44 AM, Scott Moser <email address hidden> wrote:

> but vlan and bridge-utils are not needed or desired in artful, right?
>
> --
> https://code.launchpad.net/~raharper/curtin/trunk.vmtest-
> remove-bug-timers/+merge/331757
> You are the owner of lp:~raharper/curtin/trunk.vmtest-remove-bug-timers.
>

[Ryan Harper (raharper)]

On Thu, Oct 5, 2017 at 1:38 PM, Scott Moser <email address hidden> wrote:

>
>
> Diff comments:
>
> >
> > === modified file 'examples/tests/network_alias.yaml'
> > --- examples/tests/network_alias.yaml 2017-06-13 22:10:10 +0000
> > +++ examples/tests/network_alias.yaml 2017-10-05 16:35:19 +0000
> > @@ -9,11 +9,9 @@
> > subnets:
> > - type: static
> > address: 10.47.98.1/24
> > - mtu: 1501
> > - type: static
> > address: 2001:4800:78ff:1b:be76:4eff:fe06:ffac
> > netmask: 'ffff:ffff:ffff:ffff::'
> > - mtu: 1480
>
> you say "not needed" in your commit message, but why?
> and isnt it concerning that this diddnt have fallout in tests ?
>

It does have fallout for netplan which doesn;'t support ipv6 mtu;
however, the *scope* of this test is multiple ip address per interface
The mtu itself has no affect on assigning multiple addresses to a single
interface.

The failure to apply an ipv6 MTU in this test is not germane; we're
only testing if we can set multiple ips.

>
> > # multi_v4_alias: multiple v4 addrs on same interface
> > - type: physical
> > name: interface1
> >
> > === modified file 'tests/vmtests/test_network.py'
> > --- tests/vmtests/test_network.py 2017-08-02 15:46:35 +0000
> > +++ tests/vmtests/test_network.py 2017-10-05 16:35:19 +0000
> > @@ -285,14 +285,19 @@
> > ip_route_show = self.load_collect_file("ip_route_show")
> > logger.debug("ip route show:\n{}".format(ip_route_show))
> > for line in [line for line in ip_route_show.split('\n')
> > - if 'src' in line]:
> > + if 'src' in line and not
> line.startswith('default')]:
> > + print('ip_route_show: line: %s' % line)
> > m = re.search(r'^(?P<network>\S+)\sdev\s' +
> > r'(?P<devname>\S+)\s+' +
> > - r'proto kernel\s+scope link' +
> > - r'\s+src\s(?P<src_ip>\S+)',
> > + r'proto\s(?P<proto>\S+)\s+' +
> > + r'scope\s(?P<scope>\S+)\s+' +
> > + r'src\s(?P<src_ip>\S+)',
> > line)
> > - route_info = m.groupdict('')
> > - logger.debug(route_info)
> > + if m:
> > + route_info = m.groupdict('')
> > + logger.debug(route_info)
> > + else:
> > + raise ValueError('Failed match ip_route_show line: %s'
> % line)
>
> bikeshed: this is probably more of a RuntimeException
> kind of "got stuff i didnt expect".
> doesnt matter. take input or not.
>

Sure

>
> >
> > routes = {
> > '4': route_n,
> > @@ -390,7 +395,8 @@
> > gateways.append(subnet.get('gateway'))
> > for route in subnet.get('routes', []):
> > gateways += __find_gw_config(route)
> > - return gateways
> > + # drop duplicate gateways (static routes)
>
> this does have the side affect of sorting your list.
>

That's ok, we iterate ...

[lp:~raharper/curtin/trunk.vmtest-remove-bug-timers]

538. By Ryan Harper on 2017-10-05

Merge from smoser

539. By Ryan Harper on 2017-10-05

Fix up merge issues

[Scott Moser (smoser)]

started vmtest run at https://jenkins.ubuntu.com/server/view/curtin/job/curtin-vmtest-devel-debug/5/

[Scott Moser (smoser)]

i'm good with this. lets see it pass the test run above and then pull.

review: Approve

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:539
https://jenkins.ubuntu.com/server/job/curtin-ci/647/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/647
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/647
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/647
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/647

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/647/rebuild

review: Approve (continuous-integration)

[Ryan Harper (raharper)]

Computer says yes!

Ran 2167 tests in 9300.984s

OK (SKIP=161)
Thu, 05 Oct 2017 23:51:46 +0000: vmtest end [0] in 9302s
+ RET=0

=== modified file 'curtin/commands/curthooks.py'
--- curtin/commands/curthooks.py	2017-08-04 19:53:29 +0000
+++ curtin/commands/curthooks.py	2017-10-05 21:13:20 +0000
@@ -663,6 +663,8 @@
         found_reqs = mapped_config['handler'](cfg, mapped_config['mapping'])
         needed_packages.extend(found_reqs)
 
+    LOG.debug('Curtin config dependencies requires additional packages: %s',
+              needed_packages)
     return needed_packages
 
 
@@ -675,8 +677,8 @@
     '''
 
     installed_packages = util.get_installed_packages(target)
-    needed_packages = [pkg for pkg in detect_required_packages(cfg)
-                       if pkg not in installed_packages]
+    needed_packages = set([pkg for pkg in detect_required_packages(cfg)
+                           if pkg not in installed_packages])
 
     arch_packages = {
         's390x': [('s390-tools', 'zipl')],
@@ -685,16 +687,28 @@
     for pkg, cmd in arch_packages.get(platform.machine(), []):
         if not util.which(cmd, target=target):
             if pkg not in needed_packages:
-                needed_packages.append(pkg)
+                needed_packages.add(pkg)
+
+    # FIXME: This needs cleaning up.
+    # do not install certain packages on artful as they are no longer needed.
+    # ifenslave specifically causes issuse due to dependency on ifupdown.
+    codename = util.lsb_release(target=target).get('codename')
+    if codename == 'artful':
+        drops = set(['bridge-utils', 'ifenslave', 'vlan'])
+        if needed_packages.union(drops):
+            LOG.debug("Skipping install of %s.  Not needed on artful.",
+                      needed_packages.union(drops))
+            needed_packages = needed_packages.difference(drops)
 
     if needed_packages:
+        to_add = list(sorted(needed_packages))
         state = util.load_command_environment()
         with events.ReportEventStack(
                 name=state.get('report_stack_prefix'),
                 reporting_enabled=True, level="INFO",
                 description="Installing packages on target system: " +
-                str(needed_packages)):
-            util.install_packages(needed_packages, target=target)
+                str(to_add)):
+            util.install_packages(to_add, target=target)
 
 
 def system_upgrade(cfg, target):

=== modified file 'examples/tests/network_alias.yaml'
--- examples/tests/network_alias.yaml	2017-06-13 22:10:10 +0000
+++ examples/tests/network_alias.yaml	2017-10-05 21:13:20 +0000
@@ -9,11 +9,9 @@
           subnets:
               - type: static
                 address: 10.47.98.1/24
-                mtu: 1501
               - type: static
                 address: 2001:4800:78ff:1b:be76:4eff:fe06:ffac
                 netmask: 'ffff:ffff:ffff:ffff::'
-                mtu: 1480
         # multi_v4_alias: multiple v4 addrs on same interface
         - type: physical
           name: interface1

=== modified file 'examples/tests/network_static_routes.yaml'
--- examples/tests/network_static_routes.yaml	2017-01-25 00:16:35 +0000
+++ examples/tests/network_static_routes.yaml	2017-10-05 21:13:20 +0000
@@ -10,18 +10,13 @@
               - address: 172.23.31.42/26
                 gateway: 172.23.31.2
                 type: static
-      - type: route
-        id: 4
-        metric: 0
-        destination: 10.0.0.0/12
-        gateway: 172.23.31.1
-      - type: route
-        id: 5
-        metric: 0
-        destination: 192.168.0.0/16
-        gateway: 172.23.31.1
-      - type: route
-        id: 6
-        metric: 1
-        destination: 10.200.0.0/16
-        gateway: 172.23.31.1
+                routes:
+                  - gateway: 172.23.31.1
+                    network: 10.0.0.0/12
+                    metric: 0
+                  - gateway: 172.23.31.1
+                    network: 192.168.0.0/16
+                    metric: 0
+                  - gateway: 172.23.31.1
+                    network: 10.200.0.0/16
+                    metric: 1

=== modified file 'tests/vmtests/__init__.py'
--- tests/vmtests/__init__.py	2017-10-02 15:32:45 +0000
+++ tests/vmtests/__init__.py	2017-10-05 21:13:20 +0000
@@ -336,7 +336,12 @@
     __test__ = False
     arch_skip = []
     boot_timeout = BOOT_TIMEOUT
-    collect_scripts = []
+    collect_scripts = [textwrap.dedent("""
+        cd OUTPUT_COLLECT_D
+        dpkg-query --show \
+            --showformat='${db:Status-Abbrev}\t${Package}\t${Version}\n' \
+            > debian-packages.txt 2> debian-packages.txt.err
+    """)]
     conf_file = "examples/tests/basic.yaml"
     nr_cpus = None
     dirty_disks = False
@@ -369,6 +374,8 @@
     target_krel = None
     target_ftype = "vmtest.root-tgz"
 
+    _debian_packages = None
+
     def shortDescription(self):
         return None
 
@@ -1154,6 +1161,18 @@
                 fp.write(json.dumps(data, indent=2, sort_keys=True,
                                     separators=(',', ': ')) + "\n")
 
+    @property
+    def debian_packages(self):
+        if self._debian_packages is None:
+            data = self.load_collect_file("debian-packages.txt")
+            pkgs = {}
+            for line in data.splitlines():
+                # lines are <status>\t<
+                status, pkg, ver = line.split('\t')
+                pkgs[pkg] = {'status': status, 'version': ver}
+            self._debian_packages = pkgs
+        return self._debian_packages
+
 
 class PsuedoVMBaseClass(VMBaseClass):
     # This mimics much of the VMBaseClass just with faster setUpClass

=== modified file 'tests/vmtests/test_network.py'
--- tests/vmtests/test_network.py	2017-08-02 15:46:35 +0000
+++ tests/vmtests/test_network.py	2017-10-05 21:13:20 +0000
@@ -19,7 +19,7 @@
     extra_nics = []
     # XXX: command | tee output is required for Centos under SELinux
     # http://danwalsh.livejournal.com/22860.html
-    collect_scripts = [textwrap.dedent("""
+    collect_scripts = VMBaseClass.collect_scripts + [textwrap.dedent("""
         cd OUTPUT_COLLECT_D
         echo "waiting for ipv6 to settle" && sleep 5
         route -n | tee first_route_n
@@ -39,9 +39,6 @@
         cp -av /var/log/upstart ./upstart ||:
         cp -av /etc/cloud ./etc_cloud
         cp -av /var/log/cloud*.log ./
-        dpkg-query -W -f '${Version}' cloud-init |tee dpkg_cloud-init_version
-        dpkg-query -W -f '${Version}' nplan |tee dpkg_nplan_version
-        dpkg-query -W -f '${Version}' systemd |tee dpkg_systemd_version
         rpm -q --queryformat '%{VERSION}\n' cloud-init |tee rpm_ci_version
         V=/usr/lib/python*/*-packages/cloudinit/version.py;
         grep -c NETWORK_CONFIG_V2 $V |tee cloudinit_passthrough_available
@@ -285,14 +282,19 @@
         ip_route_show = self.load_collect_file("ip_route_show")
         logger.debug("ip route show:\n{}".format(ip_route_show))
         for line in [line for line in ip_route_show.split('\n')
-                     if 'src' in line]:
+                     if 'src' in line and not line.startswith('default')]:
+            print('ip_route_show: line: %s' % line)
             m = re.search(r'^(?P<network>\S+)\sdev\s' +
                           r'(?P<devname>\S+)\s+' +
-                          r'proto kernel\s+scope link' +
-                          r'\s+src\s(?P<src_ip>\S+)',
+                          r'proto\s(?P<proto>\S+)\s+' +
+                          r'scope\s(?P<scope>\S+)\s+' +
+                          r'src\s(?P<src_ip>\S+)',
                           line)
-            route_info = m.groupdict('')
-            logger.debug(route_info)
+            if m:
+                route_info = m.groupdict('')
+                logger.debug(route_info)
+            else:
+                raise ValueError('Failed match ip_route_show line: %s' % line)
 
         routes = {
             '4': route_n,
@@ -390,7 +392,8 @@
                     gateways.append(subnet.get('gateway'))
                 for route in subnet.get('routes', []):
                     gateways += __find_gw_config(route)
-                return gateways
+                # drop duplicate gateways (static routes)
+                return list(set(gateways))
 
             # handle gateways by looking at routing table
             configured_gws = __find_gw_config(subnet)
@@ -408,7 +411,8 @@
 
                 print('found_gws: %s\nexpected: %s' % (found_gws,
                                                        configured_gws))
-                self.assertEqual(len(found_gws), len(configured_gws))
+                # we only need to check that we found at least one as we walk
+                self.assertGreater(len(found_gws), 0)
                 for fgw in found_gws:
                     if ":" in gw_ip:
                         (dest, gw, flags, metric, ref, use, iface) = \

=== modified file 'tests/vmtests/test_network_bonding.py'
--- tests/vmtests/test_network_bonding.py	2017-08-02 15:46:35 +0000
+++ tests/vmtests/test_network_bonding.py	2017-10-05 21:13:20 +0000
@@ -8,19 +8,10 @@
 
 class TestNetworkBondingAbs(TestNetworkBaseTestsAbs):
     conf_file = "examples/tests/bonding_network.yaml"
-    collect_scripts = TestNetworkBaseTestsAbs.collect_scripts + [
-        textwrap.dedent("""
-        cd OUTPUT_COLLECT_D
-        dpkg-query -W -f '${Status}' ifenslave > ifenslave_installed
-        """)]
-
-    def test_output_files_exist_ifenslave(self):
-        self.output_files_exist(["ifenslave_installed"])
 
     def test_ifenslave_installed(self):
-        status = self.load_collect_file("ifenslave_installed")
-        logger.debug('ifenslave installed: {}'.format(status))
-        self.assertEqual('install ok installed', status)
+        self.assertIn("ifenslave", self.debian_packages,
+                      "ifenslave deb not installed")
 
 
 class CentosTestNetworkBondingAbs(TestNetworkBondingAbs):
@@ -49,12 +40,10 @@
 
 class PreciseHWETTestBonding(relbase.precise_hwe_t, TestNetworkBondingAbs):
     __test__ = True
-    # package names on precise are different, need to check on ifenslave-2.6
-    collect_scripts = TestNetworkBondingAbs.collect_scripts + [
-        textwrap.dedent("""
-        cd OUTPUT_COLLECT_D
-        dpkg-query -W -f '${Status}' ifenslave-2.6 > ifenslave_installed
-        """)]
+
+    def test_ifenslave_installed(self):
+        self.assertIn("ifenslave-2.6", self.debian_packages,
+                      "ifenslave deb not installed")
 
 
 class TrustyTestBonding(relbase.trusty, TestNetworkBondingAbs):
@@ -88,6 +77,16 @@
 class ArtfulTestBonding(relbase.artful, TestNetworkBondingAbs):
     __test__ = True
 
+    def test_ifenslave_installed(self):
+        """Artful should not have ifenslave installed."""
+        pass
+
+    def test_ifenslave_not_installed(self):
+        """Confirm that ifenslave is not installed on artful"""
+        self.assertNotIn('ifenslave', self.debian_packages,
+                         "ifenslave is not expected in artful: %s" %
+                         self.debian_packages.get('ifenslave'))
+
 
 class Centos66TestNetworkBonding(centos_relbase.centos66fromxenial,
                                  CentosTestNetworkBondingAbs):

=== modified file 'tests/vmtests/test_network_bridging.py'
--- tests/vmtests/test_network_bridging.py	2017-08-22 14:50:10 +0000
+++ tests/vmtests/test_network_bridging.py	2017-10-05 21:13:20 +0000
@@ -101,7 +101,6 @@
         grep -r . /sys/class/net/br0 > sysfs_br0
         grep -r . /sys/class/net/br0/brif/eth1 > sysfs_br0_eth1
         grep -r . /sys/class/net/br0/brif/eth2 > sysfs_br0_eth2
-        dpkg-query -W -f '${Status}' bridge-utils 2>&1 > bridge-utils_installed
         """)]
 
     def test_output_files_exist_bridge(self):
@@ -110,10 +109,8 @@
                                  "sysfs_br0_eth2"])
 
     def test_bridge_utils_installed(self):
-        self.output_files_exist(["bridge-utils_installed"])
-        status = self.load_collect_file("bridge-utils_installed").strip()
-        logger.debug('bridge-utils installed: {}'.format(status))
-        self.assertEqual('install ok installed', status)
+        self.assertIn("bridge-utils", self.debian_packages,
+                      "bridge-utilsi deb not installed")
 
     def test_bridge_params(self):
 
@@ -225,12 +222,21 @@
 class ZestyTestBridging(relbase.zesty, TestBridgeNetworkAbs):
     __test__ = True
 
+
+class ArtfulTestBridging(relbase.artful, TestBridgeNetworkAbs):
+    __test__ = True
+
     @classmethod
     def setUpClass(cls):
-        cls.skip_by_date(cls.__name__, cls.release, "1706752",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
+        cls.skip_by_date(cls.__name__, cls.release, "1721157",
+                         fixby=(2017, 10, 16), removeby=(2017, 11, 16))
         super().setUpClass()
 
+    def test_bridge_utils_installed(self):
+        """bridge-utils not needed in artful."""
+        pass
 
-class ArtfulTestBridging(relbase.artful, TestBridgeNetworkAbs):
-    __test__ = True
+    def test_bridge_utils_not_installed(self):
+        self.assertNotIn("bridge-utils", self.debian_packages,
+                         "bridge-utils is not expected in artful: %s" %
+                         self.debian_packages.get('bridge-utils'))

=== modified file 'tests/vmtests/test_network_enisource.py'
--- tests/vmtests/test_network_enisource.py	2017-08-02 15:46:35 +0000
+++ tests/vmtests/test_network_enisource.py	2017-10-05 21:13:20 +0000
@@ -26,6 +26,8 @@
         This testcase then uses curtin.net.deb_parse_config method to
         extract information about what curtin wrote and compare that
         with what was actually configured (which we capture via ifconfig)
+
+        Note: This test is *not* valid for Artful as it has no ENI.
     """
 
     conf_file = "examples/tests/network_source.yaml"
@@ -98,7 +100,3 @@
 
 class ZestyTestNetworkENISource(relbase.zesty, TestNetworkENISource):
     __test__ = True
-
-
-class ArtfulTestNetworkENISource(relbase.artful, TestNetworkENISource):
-    __test__ = True

=== modified file 'tests/vmtests/test_network_ipv6.py'
--- tests/vmtests/test_network_ipv6.py	2017-08-22 14:50:10 +0000
+++ tests/vmtests/test_network_ipv6.py	2017-10-05 21:13:20 +0000
@@ -67,21 +67,10 @@
 class XenialTestNetworkIPV6(relbase.xenial, TestNetworkIPV6Abs):
     __test__ = True
 
-    @classmethod
-    def test_ip_output(cls):
-        cls.skip_by_date(cls.__name__, cls.release, bugnum="1701097",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
-
 
 class ZestyTestNetworkIPV6(relbase.zesty, TestNetworkIPV6Abs):
     __test__ = True
 
-    @classmethod
-    def setUpClass(cls):
-        cls.skip_by_date(cls.__name__, cls.release, "ci-003c6678e",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
-        super().setUpClass()
-
 
 class ArtfulTestNetworkIPV6(relbase.artful, TestNetworkIPV6Abs):
     __test__ = True

=== modified file 'tests/vmtests/test_network_ipv6_enisource.py'
--- tests/vmtests/test_network_ipv6_enisource.py	2017-08-22 14:50:10 +0000
+++ tests/vmtests/test_network_ipv6_enisource.py	2017-10-05 21:13:20 +0000
@@ -31,25 +31,11 @@
 class XenialTestNetworkIPV6ENISource(relbase.xenial, TestNetworkIPV6ENISource):
     __test__ = True
 
-    @classmethod
-    def test_ip_output(cls):
-        cls.skip_by_date(cls.__name__, cls.release, bugnum="1701097",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
-
 
 class ZestyTestNetworkIPV6ENISource(relbase.zesty, TestNetworkIPV6ENISource):
     __test__ = True
 
-    @classmethod
-    def test_ip_output(cls):
-        cls.skip_by_date(cls.__name__, cls.release, bugnum="1701097",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
-
-
+
+# Artful no longer has eni/ifupdown
 class ArtfulTestNetworkIPV6ENISource(relbase.artful, TestNetworkIPV6ENISource):
-    __test__ = True
-
-    @classmethod
-    def test_ip_output(cls):
-        cls.skip_by_date(cls.__name__, cls.release, bugnum="1701097",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
+    __test__ = False

=== modified file 'tests/vmtests/test_network_ipv6_vlan.py'
--- tests/vmtests/test_network_ipv6_vlan.py	2017-08-22 14:50:10 +0000
+++ tests/vmtests/test_network_ipv6_vlan.py	2017-10-05 21:13:20 +0000
@@ -44,12 +44,6 @@
 class ZestyTestNetworkIPV6Vlan(relbase.zesty, TestNetworkIPV6VlanAbs):
     __test__ = True
 
-    @classmethod
-    def setUpClass(cls):
-        cls.skip_by_date(cls.__name__, cls.release, bugnum="ci-003c6678e",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
-        super().setUpClass()
-
 
 class ArtfulTestNetworkIPV6Vlan(relbase.artful, TestNetworkIPV6VlanAbs):
     __test__ = True

=== modified file 'tests/vmtests/test_network_mtu.py'
--- tests/vmtests/test_network_mtu.py	2017-08-22 14:50:10 +0000
+++ tests/vmtests/test_network_mtu.py	2017-10-05 21:13:20 +0000
@@ -3,6 +3,7 @@
 from .test_network_ipv6 import TestNetworkIPV6Abs
 
 import textwrap
+import unittest
 
 
 class TestNetworkMtuAbs(TestNetworkIPV6Abs):
@@ -133,30 +134,25 @@
     def test_etc_resolvconf(self):
         pass
 
-    @classmethod
-    def test_ip_output(cls):
-        cls.skip_by_date(cls.__name__, cls.release, bugnum="1706973",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
-
-    @classmethod
-    def test_ipv6_mtu_smaller_than_ipv4_v6_iface_first(cls):
-        cls.skip_by_date(cls.__name__, cls.release, bugnum="1706973",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
-
-    @classmethod
-    def test_ipv6_mtu_smaller_than_ipv4_non_default(cls):
-        cls.skip_by_date(cls.__name__, cls.release, bugnum="1706973",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
-
-    @classmethod
-    def test_ipv6_mtu_higher_than_default_no_ipv4_iface_up(cls):
-        cls.skip_by_date(cls.__name__, cls.release, bugnum="1706973",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
-
-    @classmethod
-    def test_ipv6_mtu_higher_than_default_no_ipv4_iface_v6_iface_first(cls):
-        cls.skip_by_date(cls.__name__, cls.release, bugnum="1706973",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
+    @unittest.skip("Sysconfig does not support mixed v4/v6 MTU: LP:#1706973")
+    def test_ip_output(self):
+        pass
+
+    @unittest.skip("Sysconfig does not support mixed v4/v6 MTU: LP:#1706973")
+    def test_ipv6_mtu_smaller_than_ipv4_v6_iface_first(self):
+        pass
+
+    @unittest.skip("Sysconfig does not support mixed v4/v6 MTU: LP:#1706973")
+    def test_ipv6_mtu_smaller_than_ipv4_non_default(self):
+        pass
+
+    @unittest.skip("Sysconfig does not support mixed v4/v6 MTU: LP:#1706973")
+    def test_ipv6_mtu_higher_than_default_no_ipv4_iface_up(self):
+        pass
+
+    @unittest.skip("Sysconfig does not support mixed v4/v6 MTU: LP:#1706973")
+    def test_ipv6_mtu_higher_than_default_no_ipv4_iface_v6_iface_first(self):
+        pass
 
 
 class PreciseHWETTestNetworkMtu(relbase.precise_hwe_t, TestNetworkMtuAbs):
@@ -204,6 +200,12 @@
 class ArtfulTestNetworkMtu(relbase.artful, TestNetworkMtuAbs):
     __test__ = True
 
+    @classmethod
+    def setUpClass(cls):
+        cls.skip_by_date(cls.__name__, cls.release, "1671951",
+                         fixby=(2017, 10, 20), removeby=(2018, 1, 23))
+        super().setUpClass()
+
 
 class Centos66TestNetworkMtu(centos_relbase.centos66fromxenial,
                              CentosTestNetworkMtuAbs):

=== modified file 'tests/vmtests/test_network_vlan.py'
--- tests/vmtests/test_network_vlan.py	2017-08-22 14:50:10 +0000
+++ tests/vmtests/test_network_vlan.py	2017-10-05 21:13:20 +0000
@@ -12,7 +12,6 @@
     collect_scripts = TestNetworkBaseTestsAbs.collect_scripts + [
         textwrap.dedent("""
              cd OUTPUT_COLLECT_D
-             dpkg-query -W -f '${Status}' vlan > vlan_installed
              ip -d link show interface1.2667 |tee ip_link_show_interface1.2667
              ip -d link show interface1.2668 |tee ip_link_show_interface1.2668
              ip -d link show interface1.2669 |tee ip_link_show_interface1.2669
@@ -31,15 +30,12 @@
     def test_output_files_exist_vlan(self):
         link_files = ["ip_link_show_%s" % vlan['name']
                       for vlan in self.get_vlans()]
-        self.output_files_exist(["vlan_installed"] + link_files)
+        self.output_files_exist(link_files)
 
     def test_vlan_installed(self):
-        status = self.load_collect_file("vlan_installed").strip()
-        logger.debug('vlan installed?: %s', status)
-        self.assertEqual('install ok installed', status)
+        self.assertIn("vlan", self.debian_packages, "vlan deb not installed")
 
     def test_vlan_enabled(self):
-
         # we must have at least one
         self.assertGreaterEqual(len(self.get_vlans()), 1)
 
@@ -103,12 +99,6 @@
 class ZestyTestNetworkVlan(relbase.zesty, TestNetworkVlanAbs):
     __test__ = True
 
-    @classmethod
-    def setUpClass(cls):
-        cls.skip_by_date(cls.__name__, cls.release, "ci-003c6678e",
-                         fixby=(2017, 9, 18), removeby=(2017, 9, 27))
-        super().setUpClass()
-
 
 class ArtfulTestNetworkVlan(relbase.artful, TestNetworkVlanAbs):
     __test__ = True