Merge ~goneri/cloud-init:freebsd_net_renderer into cloud-init:master

Proposed by Gonéri Le Bouder on 2019-04-07
Status: Needs review
Proposed branch: ~goneri/cloud-init:freebsd_net_renderer
Merge into: cloud-init:master
Diff against target: 998 lines (+292/-537)
9 files modified
cloudinit/distros/__init__.py (+1/-1)
cloudinit/distros/freebsd.py (+23/-419)
cloudinit/net/__init__.py (+30/-0)
cloudinit/net/freebsd.py (+123/-0)
cloudinit/net/renderers.py (+3/-1)
doc/rtd/topics/network-config.rst (+1/-1)
tests/unittests/test_distros/test_netconfig.py (+82/-114)
tests/unittests/test_net.py (+0/-1)
tests/unittests/test_net_freebsd.py (+29/-0)
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Needs Fixing on 2019-11-23
Igor Galić (community) Needs Fixing on 2019-10-31
Ryan Harper 2019-04-07 Needs Information on 2019-07-31
Review via email: mp+365641@code.launchpad.net

Commit message

freebsd: introduce the freebsd renderer

Refactoring of the FreeBSD code base to provide a real network renderer
for FreeBSD.
Use the generic update_sysconfig_file() from rhel_util to handle the
access to /etc/rc.conf.
Interfaces are not automatically renamed by FreeBSD using
the following configuration in /etc/rc.conf:

```
ifconfig_fxp0_name="eth0"
```

To post a comment you must log in.
Ryan Harper (raharper) wrote :

Thanks for refactoring the BSD support. I've a few comments inline.

review: Needs Fixing
Gonéri Le Bouder (goneri) wrote :

I pushed an update to address your comments.

Ryan Harper (raharper) wrote :

Thanks. I've added a few more questions/comments inline.

review: Needs Fixing
Gonéri Le Bouder (goneri) wrote :

Thanks Ryan for the review.

Ryan Harper (raharper) :
Gonéri Le Bouder (goneri) wrote :

Hi Ryan,

I think I've address all your comments. I prefer to keep the interface renaming unchanged for now because:

- the current code base is complex and Linux specific, it means the logic has to be full reimplemented
- the current implementation is already broken, and nobody complains so I'm not afraid my patch will bring any regression
- the new implementation will cover most of the use-cases and follow FreeBSD philosophy.

So, I think the current patch already bring some good by reducing the complexity of the code base, and worth to be merge, even if this feature is not fully functional.

It may be interesting to revisit the renaming issue the day https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=118111 is merged.

Ryan Harper (raharper) :
review: Needs Fixing
Gonéri Le Bouder (goneri) wrote :

> Ryan, could you elaborate on what needs to be fixed?

Oh, I just realized you answered with inline comments. sorry.

Gonéri Le Bouder (goneri) wrote :

Ryan, I think I've addressed all the comments.

Ryan Harper (raharper) wrote :

Thanks for the update. I've added some additional comments inline. Thanks for continue to work on this.

review: Needs Information
Gonéri Le Bouder (goneri) wrote :

Done

Andrey Fesenko (f0andrey) wrote :

Thanks it's fix first start network.

I'm submit update in port FreeBSD
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240372

No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want jenkins to rebuild you need to trigger it yourself):
https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/365641/+edit-commit-message
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1123/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1123//rebuild

review: Needs Fixing (continuous-integration)

No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want jenkins to rebuild you need to trigger it yourself):
https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/365641/+edit-commit-message
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1132/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1132//rebuild

review: Needs Fixing (continuous-integration)

No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want jenkins to rebuild you need to trigger it yourself):
https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/365641/+edit-commit-message
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1133/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1133//rebuild

review: Needs Fixing (continuous-integration)

No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want jenkins to rebuild you need to trigger it yourself):
https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/365641/+edit-commit-message
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1134/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1134//rebuild

review: Needs Fixing (continuous-integration)

No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want jenkins to rebuild you need to trigger it yourself):
https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/365641/+edit-commit-message
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1135/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1135//rebuild

review: Needs Fixing (continuous-integration)

PASSED: Continuous integration, rev:fb42c3a3a00d9700957cbf4269d35ad0b7d161d8
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1137/
Executed test runs:
    SUCCESS: Checkout
    SUCCESS: Unit & Style Tests
    SUCCESS: Ubuntu LTS: Build
    SUCCESS: Ubuntu LTS: Integration
    IN_PROGRESS: Declarative: Post Actions

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1137//rebuild

review: Approve (continuous-integration)
Igor Galić (i.galic) wrote :

i've had a look and tried the commands locally, and it appears this won't reliably work on any installation.

review: Needs Fixing
Gonéri Le Bouder (goneri) wrote :

You are a bit harsh because, you don't point any major problem in the patch. I've updated it to take care of the colon in the interface name, but I don't think this is something really common. Do you actually have an ifconfig output example?

I will continue the review here:
https://github.com/canonical/cloud-init/pull/61

FAILED: Continuous integration, rev:0f6af36387dc15ba9ddd72b56fdb81b5208f9568
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1287/
Executed test runs:
    SUCCESS: Checkout
    FAILED: Unit & Style Tests

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1287//rebuild

review: Needs Fixing (continuous-integration)
Igor Galić (i.galic) wrote :

please forgive my harsh tone, i only now realised my comment is saying "on any installation" instead of "on every" 🙇‍♀️

Unmerged commits

0f6af36... by Gonéri Le Bouder on 2019-03-24

freebsd: introduce the freebsd renderer

Refactoring of the FreeBSD code base to provide a real network renderer
for FreeBSD.
Use the generic update_sysconfig_file() from rhel_util to handle the
access to /etc/rc.conf.
Interfaces are not automatically renamed by FreeBSD using
the following configuration in /etc/rc.conf:

```
ifconfig_fxp0_name="eth0"
```

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/cloudinit/distros/__init__.py b/cloudinit/distros/__init__.py
2index 00bdee3..75bb441 100644
3--- a/cloudinit/distros/__init__.py
4+++ b/cloudinit/distros/__init__.py
5@@ -145,7 +145,7 @@ class Distro(object):
6 # Write it out
7
8 # pylint: disable=assignment-from-no-return
9- # We have implementations in arch, freebsd and gentoo still
10+ # We have implementations in arch and gentoo still
11 dev_names = self._write_network(settings)
12 # pylint: enable=assignment-from-no-return
13 # Now try to bring them up
14diff --git a/cloudinit/distros/freebsd.py b/cloudinit/distros/freebsd.py
15index c55f899..3224887 100644
16--- a/cloudinit/distros/freebsd.py
17+++ b/cloudinit/distros/freebsd.py
18@@ -13,12 +13,10 @@ import re
19 from cloudinit import distros
20 from cloudinit import helpers
21 from cloudinit import log as logging
22+from cloudinit import net
23 from cloudinit import ssh_util
24 from cloudinit import util
25-
26-from cloudinit.distros import net_util
27-from cloudinit.distros.parsers.resolv_conf import ResolvConf
28-
29+from cloudinit.distros import rhel_util
30 from cloudinit.settings import PER_INSTANCE
31
32 LOG = logging.getLogger(__name__)
33@@ -29,9 +27,8 @@ class Distro(distros.Distro):
34 rc_conf_fn = "/etc/rc.conf"
35 login_conf_fn = '/etc/login.conf'
36 login_conf_fn_bak = '/etc/login.conf.orig'
37- resolv_conf_fn = '/etc/resolv.conf'
38 ci_sudoers_fn = '/usr/local/etc/sudoers.d/90-cloud-init-users'
39- default_primary_nic = 'hn0'
40+ hostname_conf_fn = '/etc/rc.conf'
41
42 def __init__(self, name, cfg, paths):
43 distros.Distro.__init__(self, name, cfg, paths)
44@@ -40,99 +37,8 @@ class Distro(distros.Distro):
45 # should only happen say once per instance...)
46 self._runner = helpers.Runners(paths)
47 self.osfamily = 'freebsd'
48- self.ipv4_pat = re.compile(r"\s+inet\s+\d+[.]\d+[.]\d+[.]\d+")
49 cfg['ssh_svcname'] = 'sshd'
50
51- # Updates a key in /etc/rc.conf.
52- def updatercconf(self, key, value):
53- LOG.debug("Checking %s for: %s = %s", self.rc_conf_fn, key, value)
54- conf = self.loadrcconf()
55- config_changed = False
56- if key not in conf:
57- LOG.debug("Adding key in %s: %s = %s", self.rc_conf_fn, key,
58- value)
59- conf[key] = value
60- config_changed = True
61- else:
62- for item in conf.keys():
63- if item == key and conf[item] != value:
64- conf[item] = value
65- LOG.debug("Changing key in %s: %s = %s", self.rc_conf_fn,
66- key, value)
67- config_changed = True
68-
69- if config_changed:
70- LOG.info("Writing %s", self.rc_conf_fn)
71- buf = StringIO()
72- for keyval in conf.items():
73- buf.write('%s="%s"\n' % keyval)
74- util.write_file(self.rc_conf_fn, buf.getvalue())
75-
76- # Load the contents of /etc/rc.conf and store all keys in a dict. Make sure
77- # quotes are ignored:
78- # hostname="bla"
79- def loadrcconf(self):
80- RE_MATCH = re.compile(r'^(\w+)\s*=\s*(.*)\s*')
81- conf = {}
82- lines = util.load_file(self.rc_conf_fn).splitlines()
83- for line in lines:
84- m = RE_MATCH.match(line)
85- if not m:
86- LOG.debug("Skipping line from /etc/rc.conf: %s", line)
87- continue
88- key = m.group(1).rstrip()
89- val = m.group(2).rstrip()
90- # Kill them quotes (not completely correct, aka won't handle
91- # quoted values, but should be ok ...)
92- if val[0] in ('"', "'"):
93- val = val[1:]
94- if val[-1] in ('"', "'"):
95- val = val[0:-1]
96- if len(val) == 0:
97- LOG.debug("Skipping empty value from /etc/rc.conf: %s", line)
98- continue
99- conf[key] = val
100- return conf
101-
102- def readrcconf(self, key):
103- conf = self.loadrcconf()
104- try:
105- val = conf[key]
106- except KeyError:
107- val = None
108- return val
109-
110- # NOVA will inject something like eth0, rewrite that to use the FreeBSD
111- # adapter. Since this adapter is based on the used driver, we need to
112- # figure out which interfaces are available. On KVM platforms this is
113- # vtnet0, where Xen would use xn0.
114- def getnetifname(self, dev):
115- LOG.debug("Translating network interface %s", dev)
116- if dev.startswith('lo'):
117- return dev
118-
119- n = re.search(r'\d+$', dev)
120- index = n.group(0)
121-
122- (out, _err) = util.subp(['ifconfig', '-a'])
123- ifconfigoutput = [x for x in (out.strip()).splitlines()
124- if len(x.split()) > 0]
125- bsddev = 'NOT_FOUND'
126- for line in ifconfigoutput:
127- m = re.match(r'^\w+', line)
128- if m:
129- if m.group(0).startswith('lo'):
130- continue
131- # Just settle with the first non-lo adapter we find, since it's
132- # rather unlikely there will be multiple nicdrivers involved.
133- bsddev = m.group(0)
134- break
135-
136- # Replace the index with the one we're after.
137- bsddev = re.sub(r'\d+$', index, bsddev)
138- LOG.debug("Using network interface %s", bsddev)
139- return bsddev
140-
141 def _select_hostname(self, hostname, fqdn):
142 # Should be FQDN if available. See rc.conf(5) in FreeBSD
143 if fqdn:
144@@ -140,21 +46,18 @@ class Distro(distros.Distro):
145 return hostname
146
147 def _read_system_hostname(self):
148- sys_hostname = self._read_hostname(filename=None)
149- return ('rc.conf', sys_hostname)
150+ sys_hostname = self._read_hostname(self.hostname_conf_fn)
151+ return (self.hostname_conf_fn, sys_hostname)
152
153 def _read_hostname(self, filename, default=None):
154- hostname = None
155- try:
156- hostname = self.readrcconf('hostname')
157- except IOError:
158- pass
159- if not hostname:
160+ (_exists, contents) = rhel_util.read_sysconfig_file(filename)
161+ if contents.get('hostname'):
162+ return contents['hostname']
163+ else:
164 return default
165- return hostname
166
167 def _write_hostname(self, hostname, filename):
168- self.updatercconf('hostname', hostname)
169+ rhel_util.update_sysconfig_file(filename, {'hostname': hostname})
170
171 def create_group(self, name, members):
172 group_add_cmd = ['pw', '-n', name]
173@@ -275,309 +178,16 @@ class Distro(distros.Distro):
174 keys = set(kwargs['ssh_authorized_keys']) or []
175 ssh_util.setup_user_keys(keys, name, options=None)
176
177- @staticmethod
178- def get_ifconfig_list():
179- cmd = ['ifconfig', '-l']
180- (nics, err) = util.subp(cmd, rcs=[0, 1])
181- if len(err):
182- LOG.warning("Error running %s: %s", cmd, err)
183- return None
184- return nics
185-
186- @staticmethod
187- def get_ifconfig_ifname_out(ifname):
188- cmd = ['ifconfig', ifname]
189- (if_result, err) = util.subp(cmd, rcs=[0, 1])
190- if len(err):
191- LOG.warning("Error running %s: %s", cmd, err)
192- return None
193- return if_result
194-
195- @staticmethod
196- def get_ifconfig_ether():
197- cmd = ['ifconfig', '-l', 'ether']
198- (nics, err) = util.subp(cmd, rcs=[0, 1])
199- if len(err):
200- LOG.warning("Error running %s: %s", cmd, err)
201- return None
202- return nics
203-
204- @staticmethod
205- def get_interface_mac(ifname):
206- if_result = Distro.get_ifconfig_ifname_out(ifname)
207- for item in if_result.splitlines():
208- if item.find('ether ') != -1:
209- mac = str(item.split()[1])
210- if mac:
211- return mac
212-
213- @staticmethod
214- def get_devicelist():
215- nics = Distro.get_ifconfig_list()
216- return nics.split()
217-
218- @staticmethod
219- def get_ipv6():
220- ipv6 = []
221- nics = Distro.get_devicelist()
222- for nic in nics:
223- if_result = Distro.get_ifconfig_ifname_out(nic)
224- for item in if_result.splitlines():
225- if item.find("inet6 ") != -1 and item.find("scopeid") == -1:
226- ipv6.append(nic)
227- return ipv6
228-
229- def get_ipv4(self):
230- ipv4 = []
231- nics = Distro.get_devicelist()
232- for nic in nics:
233- if_result = Distro.get_ifconfig_ifname_out(nic)
234- for item in if_result.splitlines():
235- print(item)
236- if self.ipv4_pat.match(item):
237- ipv4.append(nic)
238- return ipv4
239-
240- def is_up(self, ifname):
241- if_result = Distro.get_ifconfig_ifname_out(ifname)
242- pat = "^" + ifname
243- for item in if_result.splitlines():
244- if re.match(pat, item):
245- flags = item.split('<')[1].split('>')[0]
246- if flags.find("UP") != -1:
247- return True
248-
249- def _get_current_rename_info(self, check_downable=True):
250- """Collect information necessary for rename_interfaces."""
251- names = Distro.get_devicelist()
252- bymac = {}
253- for n in names:
254- bymac[Distro.get_interface_mac(n)] = {
255- 'name': n, 'up': self.is_up(n), 'downable': None}
256-
257- nics_with_addresses = set()
258- if check_downable:
259- nics_with_addresses = set(self.get_ipv4() + self.get_ipv6())
260-
261- for d in bymac.values():
262- d['downable'] = (d['up'] is False or
263- d['name'] not in nics_with_addresses)
264-
265- return bymac
266-
267- def _rename_interfaces(self, renames):
268- if not len(renames):
269- LOG.debug("no interfaces to rename")
270- return
271-
272- current_info = self._get_current_rename_info()
273-
274- cur_bymac = {}
275- for mac, data in current_info.items():
276- cur = data.copy()
277- cur['mac'] = mac
278- cur_bymac[mac] = cur
279-
280- def update_byname(bymac):
281- return dict((data['name'], data)
282- for data in bymac.values())
283-
284- def rename(cur, new):
285- util.subp(["ifconfig", cur, "name", new], capture=True)
286-
287- def down(name):
288- util.subp(["ifconfig", name, "down"], capture=True)
289-
290- def up(name):
291- util.subp(["ifconfig", name, "up"], capture=True)
292-
293- ops = []
294- errors = []
295- ups = []
296- cur_byname = update_byname(cur_bymac)
297- tmpname_fmt = "cirename%d"
298- tmpi = -1
299-
300- for mac, new_name in renames:
301- cur = cur_bymac.get(mac, {})
302- cur_name = cur.get('name')
303- cur_ops = []
304- if cur_name == new_name:
305- # nothing to do
306- continue
307-
308- if not cur_name:
309- errors.append("[nic not present] Cannot rename mac=%s to %s"
310- ", not available." % (mac, new_name))
311- continue
312-
313- if cur['up']:
314- msg = "[busy] Error renaming mac=%s from %s to %s"
315- if not cur['downable']:
316- errors.append(msg % (mac, cur_name, new_name))
317- continue
318- cur['up'] = False
319- cur_ops.append(("down", mac, new_name, (cur_name,)))
320- ups.append(("up", mac, new_name, (new_name,)))
321-
322- if new_name in cur_byname:
323- target = cur_byname[new_name]
324- if target['up']:
325- msg = "[busy-target] Error renaming mac=%s from %s to %s."
326- if not target['downable']:
327- errors.append(msg % (mac, cur_name, new_name))
328- continue
329- else:
330- cur_ops.append(("down", mac, new_name, (new_name,)))
331-
332- tmp_name = None
333- while tmp_name is None or tmp_name in cur_byname:
334- tmpi += 1
335- tmp_name = tmpname_fmt % tmpi
336-
337- cur_ops.append(("rename", mac, new_name, (new_name, tmp_name)))
338- target['name'] = tmp_name
339- cur_byname = update_byname(cur_bymac)
340- if target['up']:
341- ups.append(("up", mac, new_name, (tmp_name,)))
342-
343- cur_ops.append(("rename", mac, new_name, (cur['name'], new_name)))
344- cur['name'] = new_name
345- cur_byname = update_byname(cur_bymac)
346- ops += cur_ops
347-
348- opmap = {'rename': rename, 'down': down, 'up': up}
349- if len(ops) + len(ups) == 0:
350- if len(errors):
351- LOG.debug("unable to do any work for renaming of %s", renames)
352- else:
353- LOG.debug("no work necessary for renaming of %s", renames)
354- else:
355- LOG.debug("achieving renaming of %s with ops %s",
356- renames, ops + ups)
357-
358- for op, mac, new_name, params in ops + ups:
359- try:
360- opmap.get(op)(*params)
361- except Exception as e:
362- errors.append(
363- "[unknown] Error performing %s%s for %s, %s: %s" %
364- (op, params, mac, new_name, e))
365- if len(errors):
366- raise Exception('\n'.join(errors))
367-
368- def apply_network_config_names(self, netcfg):
369- renames = []
370- for ent in netcfg.get('config', {}):
371- if ent.get('type') != 'physical':
372- continue
373- mac = ent.get('mac_address')
374- name = ent.get('name')
375- if not mac:
376- continue
377- renames.append([mac, name])
378- return self._rename_interfaces(renames)
379-
380- @classmethod
381 def generate_fallback_config(self):
382- nics = Distro.get_ifconfig_ether()
383- if nics is None:
384- LOG.debug("Fail to get network interfaces")
385- return None
386- potential_interfaces = nics.split()
387- connected = []
388- for nic in potential_interfaces:
389- pat = "^" + nic
390- if_result = Distro.get_ifconfig_ifname_out(nic)
391- for item in if_result.split("\n"):
392- if re.match(pat, item):
393- flags = item.split('<')[1].split('>')[0]
394- if flags.find("RUNNING") != -1:
395- connected.append(nic)
396- if connected:
397- potential_interfaces = connected
398- names = list(sorted(potential_interfaces))
399- default_pri_nic = Distro.default_primary_nic
400- if default_pri_nic in names:
401- names.remove(default_pri_nic)
402- names.insert(0, default_pri_nic)
403- target_name = None
404- target_mac = None
405- for name in names:
406- mac = Distro.get_interface_mac(name)
407- if mac:
408- target_name = name
409- target_mac = mac
410- break
411- if target_mac and target_name:
412- nconf = {'config': [], 'version': 1}
413+ nconf = {'config': [], 'version': 1}
414+ for mac, name in net.get_interfaces_by_mac().items():
415 nconf['config'].append(
416- {'type': 'physical', 'name': target_name,
417- 'mac_address': target_mac, 'subnets': [{'type': 'dhcp'}]})
418- return nconf
419- else:
420- return None
421-
422- def _write_network(self, settings):
423- entries = net_util.translate_network(settings)
424- nameservers = []
425- searchdomains = []
426- dev_names = entries.keys()
427- for (device, info) in entries.items():
428- # Skip the loopback interface.
429- if device.startswith('lo'):
430- continue
431-
432- dev = self.getnetifname(device)
433-
434- LOG.info('Configuring interface %s', dev)
435-
436- if info.get('bootproto') == 'static':
437- LOG.debug('Configuring dev %s with %s / %s', dev,
438- info.get('address'), info.get('netmask'))
439- # Configure an ipv4 address.
440- ifconfig = (info.get('address') + ' netmask ' +
441- info.get('netmask'))
442-
443- # Configure the gateway.
444- self.updatercconf('defaultrouter', info.get('gateway'))
445-
446- if 'dns-nameservers' in info:
447- nameservers.extend(info['dns-nameservers'])
448- if 'dns-search' in info:
449- searchdomains.extend(info['dns-search'])
450- else:
451- ifconfig = 'DHCP'
452-
453- self.updatercconf('ifconfig_' + dev, ifconfig)
454-
455- # Try to read the /etc/resolv.conf or just start from scratch if that
456- # fails.
457- try:
458- resolvconf = ResolvConf(util.load_file(self.resolv_conf_fn))
459- resolvconf.parse()
460- except IOError:
461- util.logexc(LOG, "Failed to parse %s, use new empty file",
462- self.resolv_conf_fn)
463- resolvconf = ResolvConf('')
464- resolvconf.parse()
465-
466- # Add some nameservers
467- for server in nameservers:
468- try:
469- resolvconf.add_nameserver(server)
470- except ValueError:
471- util.logexc(LOG, "Failed to add nameserver %s", server)
472-
473- # And add any searchdomains.
474- for domain in searchdomains:
475- try:
476- resolvconf.add_search_domain(domain)
477- except ValueError:
478- util.logexc(LOG, "Failed to add search domain %s", domain)
479- util.write_file(self.resolv_conf_fn, str(resolvconf), 0o644)
480+ {'type': 'physical', 'name': name,
481+ 'mac_address': mac, 'subnets': [{'type': 'dhcp'}]})
482+ return nconf
483
484- return dev_names
485+ def _write_network_config(self, netconfig):
486+ return self._supported_write_network_config(netconfig)
487
488 def apply_locale(self, locale, out_fn=None):
489 # Adjust the locals value to the new value
490@@ -605,18 +215,12 @@ class Distro(distros.Distro):
491 util.logexc(LOG, "Failed to restore %s backup",
492 self.login_conf_fn)
493
494- def _bring_up_interface(self, device_name):
495- if device_name.startswith('lo'):
496- return
497- dev = self.getnetifname(device_name)
498- cmd = ['/etc/rc.d/netif', 'start', dev]
499- LOG.debug("Attempting to bring up interface %s using command %s",
500- dev, cmd)
501- # This could return 1 when the interface has already been put UP by the
502- # OS. This is just fine.
503- (_out, err) = util.subp(cmd, rcs=[0, 1])
504- if len(err):
505- LOG.warning("Error running %s: %s", cmd, err)
506+ def apply_network_config_names(self, netconfig):
507+ # This is handled by the freebsd network renderer. It writes in
508+ # /etc/rc.conf a line with the following format:
509+ # ifconfig_OLDNAME_name=NEWNAME
510+ # FreeBSD network script will rename the interface automatically.
511+ return
512
513 def install_packages(self, pkglist):
514 self.update_package_sources()
515diff --git a/cloudinit/net/__init__.py b/cloudinit/net/__init__.py
516index bd80637..7048f14 100644
517--- a/cloudinit/net/__init__.py
518+++ b/cloudinit/net/__init__.py
519@@ -765,6 +765,36 @@ def get_ib_interface_hwaddr(ifname, ethernet_format):
520
521
522 def get_interfaces_by_mac():
523+ if util.is_FreeBSD():
524+ return get_interfaces_by_mac_on_freebsd()
525+ else:
526+ return get_interfaces_by_mac_on_linux()
527+
528+
529+def get_interfaces_by_mac_on_freebsd():
530+ (out, _) = util.subp(['ifconfig', '-a', 'ether'])
531+ # flatten each interface block in a single line
532+ def flatten(out):
533+ curr_block = ''
534+ for l in out.split('\n'):
535+ if l.startswith(' '):
536+ curr_block += l
537+ else:
538+ if curr_block:
539+ yield curr_block
540+ curr_block = l
541+ yield curr_block
542+ # looks for interface and mac in a list of flatten block
543+ def find_mac(flat_list):
544+ for block in flat_list:
545+ m = re.search(r"^(\S*): .*ether\s([\da-f:]{17}).*", block)
546+ if m:
547+ yield (m.group(1), m.group(2))
548+ results = {i[0]: i[1] for i in find_mac(flatten(out))}
549+ return results
550+
551+
552+def get_interfaces_by_mac_on_linux():
553 """Build a dictionary of tuples {mac: name}.
554
555 Bridges and any devices that have a 'stolen' mac are excluded."""
556diff --git a/cloudinit/net/freebsd.py b/cloudinit/net/freebsd.py
557new file mode 100644
558index 0000000..066d5eb
559--- /dev/null
560+++ b/cloudinit/net/freebsd.py
561@@ -0,0 +1,123 @@
562+# This file is part of cloud-init. See LICENSE file for license information.
563+
564+import re
565+
566+from cloudinit import log as logging
567+from cloudinit import net
568+from cloudinit import util
569+from cloudinit.distros import rhel_util
570+from cloudinit.distros.parsers.resolv_conf import ResolvConf
571+
572+from . import renderer
573+
574+LOG = logging.getLogger(__name__)
575+
576+
577+class Renderer(renderer.Renderer):
578+ resolv_conf_fn = 'etc/resolv.conf'
579+ rc_conf_fn = 'etc/rc.conf'
580+
581+ def __init__(self, config=None):
582+ if not config:
583+ config = {}
584+ self.dhcp_interfaces = []
585+ self._postcmds = config.get('postcmds', True)
586+
587+ def _render_route(self, route, indent=""):
588+ pass
589+
590+ def _render_iface(self, iface, render_hwaddress=False):
591+ pass
592+
593+ def _write_network(self, settings, target=None):
594+ nameservers = []
595+ searchdomains = []
596+ ifname_by_mac = net.get_interfaces_by_mac()
597+ for interface in settings.iter_interfaces():
598+ device_name = interface.get("name")
599+ device_mac = interface.get("mac_address")
600+ if device_name and re.match(r'^lo\d+$', device_name):
601+ continue
602+ if device_mac and device_name:
603+ cur_name = ifname_by_mac[device_mac]
604+ if not cur_name:
605+ LOG.info('Cannot find any device with MAC %s', device_mac)
606+ continue
607+ if cur_name != device_name:
608+ rhel_util.update_sysconfig_file(
609+ util.target_path(target, self.rc_conf_fn), {
610+ 'ifconfig_%s_name' % cur_name: device_name})
611+ elif device_mac:
612+ device_name = ifname_by_mac[device_mac]
613+
614+ subnet = interface.get("subnets", [])[0]
615+ LOG.info('Configuring interface %s', device_name)
616+
617+ if subnet.get('type') == 'static':
618+ LOG.debug('Configuring dev %s with %s / %s', device_name,
619+ subnet.get('address'), subnet.get('netmask'))
620+ # Configure an ipv4 address.
621+ ifconfig = (subnet.get('address') + ' netmask ' +
622+ subnet.get('netmask'))
623+
624+ # Configure the gateway.
625+ rhel_util.update_sysconfig_file(
626+ util.target_path(target, self.rc_conf_fn), {
627+ 'defaultrouter': subnet.get('gateway')})
628+
629+ if 'dns_nameservers' in subnet:
630+ nameservers.extend(subnet['dns_nameservers'])
631+ if 'dns_search' in subnet:
632+ searchdomains.extend(subnet['dns_search'])
633+ else:
634+ self.dhcp_interfaces.append(device_name)
635+ ifconfig = 'DHCP'
636+
637+ rhel_util.update_sysconfig_file(
638+ util.target_path(target, self.rc_conf_fn), {
639+ 'ifconfig_' + device_name: ifconfig})
640+
641+ # Try to read the /etc/resolv.conf or just start from scratch if that
642+ # fails.
643+ try:
644+ resolvconf = ResolvConf(util.load_file(self.resolv_conf_fn))
645+ resolvconf.parse()
646+ except IOError:
647+ util.logexc(LOG, "Failed to parse %s, use new empty file",
648+ self.resolv_conf_fn)
649+ resolvconf = ResolvConf('')
650+ resolvconf.parse()
651+
652+ # Add some nameservers
653+ for server in nameservers:
654+ try:
655+ resolvconf.add_nameserver(server)
656+ except ValueError:
657+ util.logexc(LOG, "Failed to add nameserver %s", server)
658+
659+ # And add any searchdomains.
660+ for domain in searchdomains:
661+ try:
662+ resolvconf.add_search_domain(domain)
663+ except ValueError:
664+ util.logexc(LOG, "Failed to add search domain %s", domain)
665+ util.write_file(self.resolv_conf_fn, str(resolvconf), 0o644)
666+ self.start_services()
667+
668+ def render_network_state(self, network_state, templates=None, target=None):
669+ self._write_network(network_state, target=target)
670+
671+ def start_services(self):
672+ if not self._postcmds:
673+ LOG.debug("freebsd generate postcmd disabled")
674+ return
675+
676+ util.subp(['service', 'netif', 'restart'], capture=True)
677+ util.subp(['service', 'routing', 'restart'], capture=True)
678+ for dhcp_interface in self.dhcp_interfaces:
679+ util.subp(['service', 'dhclient', 'restart', dhcp_interface],
680+ capture=True)
681+
682+
683+def available(target=None):
684+ return util.is_FreeBSD()
685diff --git a/cloudinit/net/renderers.py b/cloudinit/net/renderers.py
686index 5117b4a..b98dbbe 100644
687--- a/cloudinit/net/renderers.py
688+++ b/cloudinit/net/renderers.py
689@@ -1,17 +1,19 @@
690 # This file is part of cloud-init. See LICENSE file for license information.
691
692 from . import eni
693+from . import freebsd
694 from . import netplan
695 from . import RendererNotFoundError
696 from . import sysconfig
697
698 NAME_TO_RENDERER = {
699 "eni": eni,
700+ "freebsd": freebsd,
701 "netplan": netplan,
702 "sysconfig": sysconfig,
703 }
704
705-DEFAULT_PRIORITY = ["eni", "sysconfig", "netplan"]
706+DEFAULT_PRIORITY = ["eni", "sysconfig", "netplan", "freebsd"]
707
708
709 def search(priority=None, target=None, first=False):
710diff --git a/doc/rtd/topics/network-config.rst b/doc/rtd/topics/network-config.rst
711index 51ced4d..1520ba9 100644
712--- a/doc/rtd/topics/network-config.rst
713+++ b/doc/rtd/topics/network-config.rst
714@@ -191,7 +191,7 @@ supplying an updated configuration in cloud-config. ::
715
716 system_info:
717 network:
718- renderers: ['netplan', 'eni', 'sysconfig']
719+ renderers: ['netplan', 'eni', 'sysconfig', 'freebsd']
720
721
722 Network Configuration Tools
723diff --git a/tests/unittests/test_distros/test_netconfig.py b/tests/unittests/test_distros/test_netconfig.py
724index 6720995..4c14d8c 100644
725--- a/tests/unittests/test_distros/test_netconfig.py
726+++ b/tests/unittests/test_distros/test_netconfig.py
727@@ -1,5 +1,6 @@
728 # This file is part of cloud-init. See LICENSE file for license information.
729
730+import copy
731 import os
732 from six import StringIO
733 from textwrap import dedent
734@@ -14,7 +15,7 @@ from cloudinit.distros.parsers.sys_conf import SysConf
735 from cloudinit import helpers
736 from cloudinit import settings
737 from cloudinit.tests.helpers import (
738- FilesystemMockingTestCase, dir2dict, populate_dir)
739+ FilesystemMockingTestCase, dir2dict)
740 from cloudinit import util
741
742
743@@ -213,128 +214,95 @@ class TestNetCfgDistroBase(FilesystemMockingTestCase):
744 self.assertEqual(v, b2[k])
745
746
747-class TestNetCfgDistroFreebsd(TestNetCfgDistroBase):
748+class TestNetCfgDistroFreeBSD(TestNetCfgDistroBase):
749
750- frbsd_ifout = """\
751-hn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
752- options=51b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,LRO>
753- ether 00:15:5d:4c:73:00
754- inet6 fe80::215:5dff:fe4c:7300%hn0 prefixlen 64 scopeid 0x2
755- inet 10.156.76.127 netmask 0xfffffc00 broadcast 10.156.79.255
756- nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
757- media: Ethernet autoselect (10Gbase-T <full-duplex>)
758- status: active
759+ def setUp(self):
760+ super(TestNetCfgDistroFreeBSD, self).setUp()
761+ self.distro = self._get_distro('freebsd', renderers=['freebsd'])
762+
763+ def _apply_and_verify_freebsd(self, apply_fn, config, expected_cfgs=None,
764+ bringup=False):
765+ if not expected_cfgs:
766+ raise ValueError('expected_cfg must not be None')
767+
768+ tmpd = None
769+ with mock.patch('cloudinit.net.freebsd.available') as m_avail:
770+ m_avail.return_value = True
771+ with self.reRooted(tmpd) as tmpd:
772+ util.ensure_dir('/etc')
773+ util.ensure_file('/etc/rc.conf')
774+ util.ensure_file('/etc/resolv.conf')
775+ apply_fn(config, bringup)
776+
777+ results = dir2dict(tmpd)
778+ for cfgpath, expected in expected_cfgs.items():
779+ print("----------")
780+ print(expected)
781+ print("^^^^ expected | rendered VVVVVVV")
782+ print(results[cfgpath])
783+ print("----------")
784+ self.assertEqual(
785+ set(expected.split('\n')),
786+ set(results[cfgpath].split('\n')))
787+ self.assertEqual(0o644, get_mode(cfgpath, tmpd))
788+
789+ @mock.patch('cloudinit.net.get_interfaces_by_mac')
790+ def test_apply_network_config_freebsd_standard(self, ifaces_mac):
791+ ifaces_mac.return_value = {
792+ '00:15:5d:4c:73:00': 'eth0',
793+ }
794+ rc_conf_expected = """\
795+defaultrouter=192.168.1.254
796+ifconfig_eth0='192.168.1.5 netmask 255.255.255.0'
797+ifconfig_eth1=DHCP
798 """
799
800- @mock.patch('cloudinit.distros.freebsd.Distro.get_ifconfig_list')
801- @mock.patch('cloudinit.distros.freebsd.Distro.get_ifconfig_ifname_out')
802- def test_get_ip_nic_freebsd(self, ifname_out, iflist):
803- frbsd_distro = self._get_distro('freebsd')
804- iflist.return_value = "lo0 hn0"
805- ifname_out.return_value = self.frbsd_ifout
806- res = frbsd_distro.get_ipv4()
807- self.assertEqual(res, ['lo0', 'hn0'])
808- res = frbsd_distro.get_ipv6()
809- self.assertEqual(res, [])
810-
811- @mock.patch('cloudinit.distros.freebsd.Distro.get_ifconfig_ether')
812- @mock.patch('cloudinit.distros.freebsd.Distro.get_ifconfig_ifname_out')
813- @mock.patch('cloudinit.distros.freebsd.Distro.get_interface_mac')
814- def test_generate_fallback_config_freebsd(self, mac, ifname_out, if_ether):
815- frbsd_distro = self._get_distro('freebsd')
816-
817- if_ether.return_value = 'hn0'
818- ifname_out.return_value = self.frbsd_ifout
819- mac.return_value = '00:15:5d:4c:73:00'
820- res = frbsd_distro.generate_fallback_config()
821- self.assertIsNotNone(res)
822-
823- def test_simple_write_freebsd(self):
824- fbsd_distro = self._get_distro('freebsd')
825-
826- rc_conf = '/etc/rc.conf'
827- read_bufs = {
828- rc_conf: 'initial-rc-conf-not-validated',
829- '/etc/resolv.conf': 'initial-resolv-conf-not-validated',
830+ expected_cfgs = {
831+ '/etc/rc.conf': rc_conf_expected,
832+ '/etc/resolv.conf': ''
833+ }
834+ self._apply_and_verify_freebsd(self.distro.apply_network_config,
835+ V1_NET_CFG,
836+ expected_cfgs=expected_cfgs.copy())
837+
838+ @mock.patch('cloudinit.net.get_interfaces_by_mac')
839+ def test_apply_network_config_freebsd_ifrename(self, ifaces_mac):
840+ ifaces_mac.return_value = {
841+ '00:15:5d:4c:73:00': 'vtnet0',
842 }
843+ rc_conf_expected = """\
844+ifconfig_vtnet0_name=eth0
845+defaultrouter=192.168.1.254
846+ifconfig_eth0='192.168.1.5 netmask 255.255.255.0'
847+ifconfig_eth1=DHCP
848+"""
849
850- tmpd = self.tmp_dir()
851- populate_dir(tmpd, read_bufs)
852- with self.reRooted(tmpd):
853- with mock.patch("cloudinit.distros.freebsd.util.subp",
854- return_value=('vtnet0', '')):
855- fbsd_distro.apply_network(BASE_NET_CFG, False)
856- results = dir2dict(tmpd)
857-
858- self.assertIn(rc_conf, results)
859- self.assertCfgEquals(
860- dedent('''\
861- ifconfig_vtnet0="192.168.1.5 netmask 255.255.255.0"
862- ifconfig_vtnet1="DHCP"
863- defaultrouter="192.168.1.254"
864- '''), results[rc_conf])
865- self.assertEqual(0o644, get_mode(rc_conf, tmpd))
866-
867- def test_simple_write_freebsd_from_v2eni(self):
868- fbsd_distro = self._get_distro('freebsd')
869-
870- rc_conf = '/etc/rc.conf'
871- read_bufs = {
872- rc_conf: 'initial-rc-conf-not-validated',
873- '/etc/resolv.conf': 'initial-resolv-conf-not-validated',
874+ V1_NET_CFG_RENAME = copy.deepcopy(V1_NET_CFG)
875+ V1_NET_CFG_RENAME['config'][0]['mac_address'] = '00:15:5d:4c:73:00'
876+
877+ expected_cfgs = {
878+ '/etc/rc.conf': rc_conf_expected,
879+ '/etc/resolv.conf': ''
880 }
881+ self._apply_and_verify_freebsd(self.distro.apply_network_config,
882+ V1_NET_CFG_RENAME,
883+ expected_cfgs=expected_cfgs.copy())
884
885- tmpd = self.tmp_dir()
886- populate_dir(tmpd, read_bufs)
887- with self.reRooted(tmpd):
888- with mock.patch("cloudinit.distros.freebsd.util.subp",
889- return_value=('vtnet0', '')):
890- fbsd_distro.apply_network(BASE_NET_CFG_FROM_V2, False)
891- results = dir2dict(tmpd)
892-
893- self.assertIn(rc_conf, results)
894- self.assertCfgEquals(
895- dedent('''\
896- ifconfig_vtnet0="192.168.1.5 netmask 255.255.255.0"
897- ifconfig_vtnet1="DHCP"
898- defaultrouter="192.168.1.254"
899- '''), results[rc_conf])
900- self.assertEqual(0o644, get_mode(rc_conf, tmpd))
901-
902- def test_apply_network_config_fallback_freebsd(self):
903- fbsd_distro = self._get_distro('freebsd')
904-
905- # a weak attempt to verify that we don't have an implementation
906- # of _write_network_config or apply_network_config in fbsd now,
907- # which would make this test not actually test the fallback.
908- self.assertRaises(
909- NotImplementedError, fbsd_distro._write_network_config,
910- BASE_NET_CFG)
911-
912- # now run
913- mynetcfg = {
914- 'config': [{"type": "physical", "name": "eth0",
915- "mac_address": "c0:d6:9f:2c:e8:80",
916- "subnets": [{"type": "dhcp"}]}],
917- 'version': 1}
918-
919- rc_conf = '/etc/rc.conf'
920- read_bufs = {
921- rc_conf: 'initial-rc-conf-not-validated',
922- '/etc/resolv.conf': 'initial-resolv-conf-not-validated',
923+ @mock.patch('cloudinit.net.get_interfaces_by_mac')
924+ def test_apply_network_config_freebsd_nameserver(self, ifaces_mac):
925+ ifaces_mac.return_value = {
926+ '00:15:5d:4c:73:00': 'eth0',
927 }
928
929- tmpd = self.tmp_dir()
930- populate_dir(tmpd, read_bufs)
931- with self.reRooted(tmpd):
932- with mock.patch("cloudinit.distros.freebsd.util.subp",
933- return_value=('vtnet0', '')):
934- fbsd_distro.apply_network_config(mynetcfg, bring_up=False)
935- results = dir2dict(tmpd)
936-
937- self.assertIn(rc_conf, results)
938- self.assertCfgEquals('ifconfig_vtnet0="DHCP"', results[rc_conf])
939- self.assertEqual(0o644, get_mode(rc_conf, tmpd))
940+ V1_NET_CFG_DNS = copy.deepcopy(V1_NET_CFG)
941+ ns = ['1.2.3.4']
942+ V1_NET_CFG_DNS['config'][0]['subnets'][0]['dns_nameservers'] = ns
943+ expected_cfgs = {
944+ '/etc/resolv.conf': 'nameserver 1.2.3.4\n'
945+ }
946+ self._apply_and_verify_freebsd(self.distro.apply_network_config,
947+ V1_NET_CFG_DNS,
948+ expected_cfgs=expected_cfgs.copy())
949
950
951 class TestNetCfgDistroUbuntuEni(TestNetCfgDistroBase):
952diff --git a/tests/unittests/test_net.py b/tests/unittests/test_net.py
953index 0f45dc3..47a50e7 100644
954--- a/tests/unittests/test_net.py
955+++ b/tests/unittests/test_net.py
956@@ -2409,7 +2409,6 @@ DEFAULT_DEV_ATTRS = {
957 }
958 }
959
960-
961 def _setup_test(tmp_dir, mock_get_devicelist, mock_read_sys_net,
962 mock_sys_dev_path, dev_attrs=None):
963 if not dev_attrs:
964diff --git a/tests/unittests/test_net_freebsd.py b/tests/unittests/test_net_freebsd.py
965new file mode 100644
966index 0000000..8adb14e
967--- /dev/null
968+++ b/tests/unittests/test_net_freebsd.py
969@@ -0,0 +1,29 @@
970+from cloudinit import net
971+
972+from cloudinit.tests.helpers import (CiTestCase, mock)
973+
974+
975+IFCONFIG_FREEBSD = """vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
976+ options=6c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
977+ ether 52:54:00:50:b7:0d
978+re0.33: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
979+ options=80003<RXCSUM,TXCSUM,LINKSTATE>
980+ ether 80:00:73:63:5c:48
981+ groups: vlan
982+ vlan: 33 vlanpcp: 0 parent interface: re0
983+ media: Ethernet autoselect (1000baseT <full-duplex,master>)
984+ status: active
985+ nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
986+lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
987+ options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
988+"""
989+
990+class TestInterfacesByMac(CiTestCase):
991+
992+ @mock.patch('cloudinit.util.subp')
993+ @mock.patch('cloudinit.util.is_FreeBSD')
994+ def test_get_interfaces_by_mac(self, mock_is_FreeBSD, mock_subp):
995+ mock_is_FreeBSD.return_value = True
996+ mock_subp.return_value = (IFCONFIG_FREEBSD, 0)
997+ a = net.get_interfaces_by_mac()
998+ assert a == {"vtnet0": "52:54:00:50:b7:0d", "re0.33": "80:00:73:63:5c:48"}

Subscribers

People subscribed via source and target branches