Bionic netplan render invalid yaml duplicate anchor declaration for nameserver entries

Can you please run 'sudo cloud-init collect-logs' and attach the log file and the contents of /etc/netplan/

cloud init logs.

netplan yaml file.

From what I see, aliases come from the base library (pyyaml) which uses aliases if object references are used:

https://git.launchpad.net/~usd-import-team/ubuntu/+source/pyyaml/tree/lib3/yaml/serializer.py?h=ubuntu/bionic#n11
    ANCHOR_TEMPLATE = 'id%03d'
# ...
    def generate_anchor(self, node):
        self.last_anchor_id += 1
        return self.ANCHOR_TEMPLATE % self.last_anchor_id

In [2]: import yaml

In [6]: a = {'name': 'foo', 'mtu': '1500'}
   ...: b = {'name': 'bar', 'mtu': '9000'}
   ...: c = {'ref1': a, 'ref2': a, 'ref3': b, 'ref4': b}
   ...: print(yaml.dump(c))
   ...:
   ...:
ref1: &id001 {mtu: '1500', name: foo}
ref2: *id001
ref3: &id002 {mtu: '9000', name: bar}
ref4: *id002

So there must be something wrong with how cloud-init produces the aggregated yaml file.

I think this happens because it renders yaml for each section and bond interfaces and VLAN interfaces belong to different sections while nameservers object is identical and is reused in both sections (e.g. 2 bonds, 2 VLAN interfaces both use the same nameservers section but give you an anchor *per section* as the second entry in each section uses the same nameservers key which results in id001 being present).

https://github.com/cloud-init/cloud-init/blob/ubuntu/18.4-0ubuntu1_18.04.1/cloudinit/net/netplan.py#L359-L367

        def _render_section(name, section):
            if section:
                dump = util.yaml_dumps({name: section},
                                       explicit_start=False,
                                       explicit_end=False)
                txt = util.indent(dump, ' ' * 4)
                return [txt]
            return []

        content.append("network:\n version: 2\n")
        content += _render_section('ethernets', ethernets)
        content += _render_section('wifis', wifis)
        content += _render_section('bonds', bonds) # <--- first id001 gets generated
        content += _render_section('bridges', bridges)
        content += _render_section('vlans', vlans) # <--- second id001 gets generated

Yes, the per-section rendering means there are duplicate anchors.

I'll have a branch up that will remove the use of aliases.

This bug is fixed with commit cf308366 to cloud-init on branch master.
To view that commit see the following URL:
https://git.launchpad.net/cloud-init/commit/?id=cf308366

Status changed to 'Confirmed' because the bug affects multiple users.

Subscribing field-critical as it blocks a deployment

This bug is fixed in master and as such is available for testing or usage in the cloud-init daily PPA. This is available at the URL below:

https://launchpad.net/~cloud-init-dev/+archive/ubuntu/daily

We have an SRU of cloud-init that is in progress and wrapping up in the next couple of days. The plan is to start a new SRU containing this fix as soon as the current one is done. We will do whatever we can to help expedite that SRU to release this fix.

@Joshua Thanks a lot for your help!

Our first SRU is done, the next SRU is started and it will contain the fix. This can be tracked in the following bug report:

https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1819067

This bug was fixed in the package cloud-init - 18.5-45-g3554ffe8-0ubuntu1

---------------
cloud-init (18.5-45-g3554ffe8-0ubuntu1) disco; urgency=medium

  * New upstream snapshot.
    - cloud-init-per: POSIX sh does not support string subst, use sed
      (LP: #1819222)

 -- Daniel Watkins <email address hidden> Fri, 08 Mar 2019 17:42:34 -0500

SRU is complete, fix is released. Daily images should pick this up on Mar 20 or Mar 21

Marking all fixed-released and unsubscribing field-critical.

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3331