curtin

Merge ~smoser/curtin:cleanup/always-use-bootif into curtin:master

  1. Git
  2. lp:~smoser/curtin
  3. cleanup/always-use-bootif
  4. Merge into master
Proposed by Scott Moser
Status: Merged
Approved by: Scott Moser
Approved revision: 5e65be2056fc88db15d684b1833c86d4e299107c
Merge reported by: Scott Moser
Merged at revision: df7f58977c3d4e4630be894e295eae44be68af88
Proposed branch: ~smoser/curtin:cleanup/always-use-bootif
Merge into: curtin:master
Diff against target: 192 lines (+31/-22)
10 files modified
tests/vmtests/__init__.py (+31/-12)
tests/vmtests/test_centos_basic.py (+0/-2)
tests/vmtests/test_network.py (+0/-1)
tests/vmtests/test_network_alias.py (+0/-1)
tests/vmtests/test_network_bonding.py (+0/-1)
tests/vmtests/test_network_bridging.py (+0/-1)
tests/vmtests/test_network_ipv6.py (+0/-1)
tests/vmtests/test_network_mtu.py (+0/-1)
tests/vmtests/test_network_static.py (+0/-1)
tests/vmtests/test_network_vlan.py (+0/-1)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Approve
Ryan Harper (community) Approve
Review via email: mp+337670@code.launchpad.net

Commit message

vmtest: always boot with BOOTIF and ip=:::::BOOTIF:dhcp

This excercises part of boot that was excercised in MAAS but not here.
Cloud-initramfs-dyn-netconf translates BOOTIF to the device name.

See bug #1749019 for some more context.

Description of the change

see commit message

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:22ccd46450cc183154d2c445ea7d545db5c740eb
https://jenkins.ubuntu.com/server/job/curtin-ci/843/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/843/console
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/843/console
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/843/console
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/843/console

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/843/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

https://jenkins.ubuntu.com/server/view/curtin/job/curtin-vmtest-devel-amd64/305/console
has a run of 56e5408653d9f9b87baf84d3aacc80dce4d3d675

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:56e5408653d9f9b87baf84d3aacc80dce4d3d675
https://jenkins.ubuntu.com/server/job/curtin-ci/844/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/844/console
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/844/console
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/844/console
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/844/console

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/844/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:d1e2eb8cae9fba2774782fd0038e65a898972a21
https://jenkins.ubuntu.com/server/job/curtin-ci/845/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/845
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/845
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/845
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/845

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/845/rebuild

review: Approve (continuous-integration)
Revision history for this message
Ryan Harper (raharper) :
Revision history for this message
Scott Moser (smoser) :
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:4978209a9ef4e790cebb04caed84f3ade97f86cc
https://jenkins.ubuntu.com/server/job/curtin-ci/854/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/854
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/854
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/854
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/854

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/854/rebuild

review: Approve (continuous-integration)
Revision history for this message
Ryan Harper (raharper) wrote :

Nice, thanks for the refactor and comments. Approved with a vmtest run passing.

review: Approve
Revision history for this message
Scott Moser (smoser) wrote :

Ryan,
good catch on hostname.
you were right, I documented the *wrong* format.
Apparently googles' answer to 'klibc ipconfig' was wrong:
  https://kernel.googlesource.com/pub/scm/libs/klibc/klibc/+/klibc-0.130/ipconfig/README
There it is documented:
  addr:server:gateway:netmask:interface:proto

But Ubuntu's version:
  https://git.launchpad.net/ubuntu/+source/klibc/tree/usr/kinit/ipconfig/README.ipconfig
documents (and uses):
  <client-ip>:<server-ip>:<gw-ip>:<netmask>:<hostname>:<device>:<autoconf>

So my code was right, but doc was wrong. They're fixed now.

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

PASSED: Continuous integration, rev:5e65be2056fc88db15d684b1833c86d4e299107c
https://jenkins.ubuntu.com/server/job/curtin-ci/906/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/906
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/906
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/906
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/906

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/906/rebuild

review: Approve (continuous-integration)
Revision history for this message
Scott Moser (smoser) wrote :

Jenkins run started at
 https://jenkins.ubuntu.com/server/view/curtin/job/curtin-vmtest-devel-debug/54/console

Revision history for this message
Scott Moser (smoser) wrote :

The above run failed on these two tests:
  tests/vmtests/test_network_bridging.py:Centos66TestBridgeNetwork
  tests/vmtests/test_network_alias.py:Centos66TestNetworkAlias

I did not reproduce those failures on diglett, and re-ran again
in jenkins just those tests at:

  https://jenkins.ubuntu.com/server/view/curtin/job/curtin-vmtest-devel-debug/55/console

That was green.

Ryan mentioned in IRC: "those are load related failures".

so... I think this is ready to go.

Revision history for this message
Scott Moser (smoser) wrote :

An upstream commit landed for this bug.

To view that commit see the following URL:
https://git.launchpad.net/curtin/commit/?id=df7f5897

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/tests/vmtests/__init__.py b/tests/vmtests/__init__.py
2index b5aa538..4f32c45 100644
3--- a/tests/vmtests/__init__.py
4+++ b/tests/vmtests/__init__.py
5@@ -708,8 +708,8 @@ class VMBaseClass(TestCase):
6 cmd.extend([
7 "--root-arg=root=%s" % root_url,
8 "--append=overlayroot=tmpfs",
9- "--append=ip=dhcp", # enable networking
10 ])
11+
12 # getting resolvconf configured is only fixed in bionic
13 # the iscsi_auto handles resolvconf setup via call to
14 # configure_networking in initramfs
15@@ -733,7 +733,7 @@ class VMBaseClass(TestCase):
16 cls.network_state = curtin_net.parse_net_config(cls.conf_file)
17 logger.debug("Network state: {}".format(cls.network_state))
18
19- # build -n arg list with macaddrs from net_config physical config
20+ # build --netdev=arg list with 'physical' nics from net_config
21 macs = []
22 interfaces = {}
23 if cls.network_state:
24@@ -744,16 +744,14 @@ class VMBaseClass(TestCase):
25 hwaddr = iface.get('mac_address')
26 if iface['type'] == 'physical' and hwaddr:
27 macs.append(hwaddr)
28- netdevs = []
29- if len(macs) > 0:
30- # take first mac and mark it as the boot interface to prevent DHCP
31- # on multiple interfaces which can hang the install.
32- cmd.extend(["--append=BOOTIF=01-%s" % macs[0].replace(":", "-")])
33- for mac in macs:
34- netdevs.extend(["--netdev=" + DEFAULT_BRIDGE +
35- ",mac={}".format(mac)])
36- else:
37- netdevs.extend(["--netdev=" + DEFAULT_BRIDGE])
38+
39+ if len(macs) == 0:
40+ macs = ["52:54:00:12:34:01"]
41+
42+ netdevs = ["--netdev=%s,mac=%s" % (DEFAULT_BRIDGE, m) for m in macs]
43+
44+ # Add kernel parameters to simulate network boot from first nic.
45+ cmd.extend(kernel_boot_cmdline_for_mac(macs[0]))
46
47 # build disk arguments
48 disks = []
49@@ -1737,6 +1735,27 @@ def get_lan_ip():
50 return addr
51
52
53+def kernel_boot_cmdline_for_mac(mac):
54+ """Return kernel command line arguments for initramfs dhcp on mac.
55+
56+ Ubuntu initramfs respect klibc's ip= format for network config in
57+ initramfs. That format is:
58+ ip=addr:server:gateway:netmask:interface:proto
59+ see /usr/share/doc/libklibc/README.ipconfig.gz for more info.
60+
61+ If no 'interface' field is provided, dhcp will be tried on all. To allow
62+ specifying the interface in ip= parameter without knowing the name of the
63+ device that the kernel will choose, cloud-initramfs-dyn-netconf replaces
64+ 'BOOTIF' in the ip= parameter with the name found in BOOTIF.
65+
66+ Network bootloaders append to kernel command line
67+ BOOTIF=01-<mac-address> to indicate which mac they booted from.
68+
69+ Paired with BOOTIF replacement this ends up being: ip=::::eth0:dhcp."""
70+ return ["--append=ip=:::::BOOTIF:dhcp",
71+ "--append=BOOTIF=01-%s" % mac.replace(":", "-")]
72+
73+
74 def is_unsupported_ubuntu(release):
75 global _UNSUPPORTED_UBUNTU
76 udi = 'ubuntu-distro-info'
77diff --git a/tests/vmtests/test_centos_basic.py b/tests/vmtests/test_centos_basic.py
78index b576279..7857e74 100644
79--- a/tests/vmtests/test_centos_basic.py
80+++ b/tests/vmtests/test_centos_basic.py
81@@ -11,7 +11,6 @@ import textwrap
82 class CentosTestBasicAbs(VMBaseClass):
83 __test__ = False
84 conf_file = "examples/tests/centos_basic.yaml"
85- extra_kern_args = "BOOTIF=eth0-52:54:00:12:34:00"
86 # XXX: command | tee output is required for Centos under SELinux
87 # http://danwalsh.livejournal.com/22860.html
88 collect_scripts = VMBaseClass.collect_scripts + [textwrap.dedent(
89@@ -74,7 +73,6 @@ class Centos66FromXenialTestBasic(relbase.centos66fromxenial,
90
91 class CentosTestBasicNetworkAbs(TestNetworkBaseTestsAbs):
92 conf_file = "examples/tests/centos_basic.yaml"
93- extra_kern_args = "BOOTIF=eth0-52:54:00:12:34:00"
94 collect_scripts = TestNetworkBaseTestsAbs.collect_scripts + [
95 textwrap.dedent("""
96 cd OUTPUT_COLLECT_D
97diff --git a/tests/vmtests/test_network.py b/tests/vmtests/test_network.py
98index 6ce4262..59a25fe 100644
99--- a/tests/vmtests/test_network.py
100+++ b/tests/vmtests/test_network.py
101@@ -437,7 +437,6 @@ class TestNetworkBasicAbs(TestNetworkBaseTestsAbs):
102
103 class CentosTestNetworkBasicAbs(TestNetworkBaseTestsAbs):
104 conf_file = "examples/tests/centos_basic.yaml"
105- extra_kern_args = "BOOTIF=eth0-52:54:00:12:34:00"
106 collect_scripts = TestNetworkBaseTestsAbs.collect_scripts + [
107 textwrap.dedent("""
108 cd OUTPUT_COLLECT_D
109diff --git a/tests/vmtests/test_network_alias.py b/tests/vmtests/test_network_alias.py
110index 258554f..903b395 100644
111--- a/tests/vmtests/test_network_alias.py
112+++ b/tests/vmtests/test_network_alias.py
113@@ -19,7 +19,6 @@ class TestNetworkAliasAbs(TestNetworkBaseTestsAbs):
114
115
116 class CentosTestNetworkAliasAbs(TestNetworkAliasAbs):
117- extra_kern_args = "BOOTIF=eth0-52:54:00:12:34:00"
118 collect_scripts = TestNetworkAliasAbs.collect_scripts + [
119 textwrap.dedent("""
120 cd OUTPUT_COLLECT_D
121diff --git a/tests/vmtests/test_network_bonding.py b/tests/vmtests/test_network_bonding.py
122index 24cf60f..7d07413 100644
123--- a/tests/vmtests/test_network_bonding.py
124+++ b/tests/vmtests/test_network_bonding.py
125@@ -16,7 +16,6 @@ class TestNetworkBondingAbs(TestNetworkBaseTestsAbs):
126
127
128 class CentosTestNetworkBondingAbs(TestNetworkBondingAbs):
129- extra_kern_args = "BOOTIF=eth0-52:54:00:12:34:00"
130 collect_scripts = TestNetworkBondingAbs.collect_scripts + [
131 textwrap.dedent("""
132 cd OUTPUT_COLLECT_D
133diff --git a/tests/vmtests/test_network_bridging.py b/tests/vmtests/test_network_bridging.py
134index 5691b00..ca8964e 100644
135--- a/tests/vmtests/test_network_bridging.py
136+++ b/tests/vmtests/test_network_bridging.py
137@@ -184,7 +184,6 @@ class TestBridgeNetworkAbs(TestNetworkBaseTestsAbs):
138
139
140 class CentosTestBridgeNetworkAbs(TestBridgeNetworkAbs):
141- extra_kern_args = "BOOTIF=eth0-52:54:00:12:34:00"
142 collect_scripts = TestBridgeNetworkAbs.collect_scripts + [
143 textwrap.dedent("""
144 cd OUTPUT_COLLECT_D
145diff --git a/tests/vmtests/test_network_ipv6.py b/tests/vmtests/test_network_ipv6.py
146index 9bbfc1e..6d87dcf 100644
147--- a/tests/vmtests/test_network_ipv6.py
148+++ b/tests/vmtests/test_network_ipv6.py
149@@ -25,7 +25,6 @@ class TestNetworkIPV6Abs(TestNetworkBaseTestsAbs):
150
151
152 class CentosTestNetworkIPV6Abs(TestNetworkIPV6Abs):
153- extra_kern_args = "BOOTIF=eth0-bc:76:4e:06:96:b3"
154 collect_scripts = TestNetworkIPV6Abs.collect_scripts + [
155 textwrap.dedent("""
156 cd OUTPUT_COLLECT_D
157diff --git a/tests/vmtests/test_network_mtu.py b/tests/vmtests/test_network_mtu.py
158index 86f4e48..41b1383 100644
159--- a/tests/vmtests/test_network_mtu.py
160+++ b/tests/vmtests/test_network_mtu.py
161@@ -120,7 +120,6 @@ class TestNetworkMtuAbs(TestNetworkIPV6Abs):
162
163 class CentosTestNetworkMtuAbs(TestNetworkMtuAbs):
164 conf_file = "examples/tests/network_mtu.yaml"
165- extra_kern_args = "BOOTIF=eth0-52:54:00:12:34:00"
166 collect_scripts = TestNetworkMtuAbs.collect_scripts + [
167 textwrap.dedent("""
168 cd OUTPUT_COLLECT_D
169diff --git a/tests/vmtests/test_network_static.py b/tests/vmtests/test_network_static.py
170index 2d226c0..d96d3eb 100644
171--- a/tests/vmtests/test_network_static.py
172+++ b/tests/vmtests/test_network_static.py
173@@ -13,7 +13,6 @@ class TestNetworkStaticAbs(TestNetworkBaseTestsAbs):
174
175
176 class CentosTestNetworkStaticAbs(TestNetworkStaticAbs):
177- extra_kern_args = "BOOTIF=eth0-52:54:00:12:34:00"
178 collect_scripts = TestNetworkBaseTestsAbs.collect_scripts + [
179 textwrap.dedent("""
180 cd OUTPUT_COLLECT_D
181diff --git a/tests/vmtests/test_network_vlan.py b/tests/vmtests/test_network_vlan.py
182index 00eb445..3cb6eae 100644
183--- a/tests/vmtests/test_network_vlan.py
184+++ b/tests/vmtests/test_network_vlan.py
185@@ -54,7 +54,6 @@ class TestNetworkVlanAbs(TestNetworkBaseTestsAbs):
186
187
188 class CentosTestNetworkVlanAbs(TestNetworkVlanAbs):
189- extra_kern_args = "BOOTIF=eth0-d4:be:d9:a8:49:13"
190 collect_scripts = TestNetworkVlanAbs.collect_scripts + [
191 textwrap.dedent("""
192 cd OUTPUT_COLLECT_D

Subscribers

People subscribed via source and target branches