Duplicate values in ARPIPTargets prevents systemd-networkd to process the netdev file due to a validation error in Ubuntu 18.04LTS.
Commits:
* Prevent to have duplicate values set to ARPIPTargets in NetDev files (LP: #1915837)
* Added a test case to check ARPIPTargets value in NetDev files (LP: #1915837) (#199)
* Improved the test case to trigger multiple pass on the YAML config (LP: #1915837) (#199)
* Reduced the test case content to the required base minimum configuration items (LP: #1915837) (#199)
Co-authored-by: Kivanc Sunkar <email address hidden>
NM Integration: API update & Parser/Generator split (#191)
This is 7th in a series of pull requests to provide the API needed for implementation of a NetworkManager YAML backend. It builds upon #189
Changes contained in this PR:
* Split the code/modules into a YAML/netplan generator (netplan.o in addition to nm.o, networkd.o, openvswitch.o, ...)
* Split the code/modules into a keyfile/NM parser (parse-nm.o in addition to our YAML/netplan parser in parse.o)
* Rename others top-level device type/stanza to nm-devices to make it more explicit and print a warning when using it
* Move netplan_get_id_from_nm_filename back to utils.o, as it does not fit into the parse-nm.o module anymore (it is no parser specific functionality). Apply a small improvement to detect netplan_ids in testing scenario, where rootdir is provided.
* Unify the new netplan generator module API with the other generators: void write_netplan_conf(const NetplanNetDefinition* def, const char* rootdir)
* Unify the new keyfile parser module API with the other parser: gboolean netplan_parse_keyfile(const char* filename, GError** error)
Improvements for the NetworkManager YAML backend integration (#189)
This PR is 6th in a series of pull requests to prepare for the NetworkManager YAML backend. It builds upon #183 and contains a few improvements and fixes, enabling the NetworkManager test-suite (i.e. make check) to fully pass, especially the tests in the patched keyfile plugin (src/settings/plugins/keyfile/tests/test-keyfile-settings).
It implements the following functionality:
Adding some basic support for the modems YAML schema, to allow matching of (physical) modem interfaces and enable the detection of GSM vs CDMA connections, which are distinct in NM while using the same definition in Netplan.
Switching the passthrough key-value pairs to a DataList (instead of HashTable), to keep order of the keyfile elements, which is relevant in some cases (e.g. for tc.qdiscs)
Avoid parsing and serialization of type other connections, as they might not contain the relevant handlers. Use full passthrough mode for those connections, to avoid parsing failures.
Allow independent modification of Netdef ID & match.name, to enable changing the connection.interface-name from nmcli, while keeping the netdef ID equal to a previously existing ID. This enables overriding of existing netdef ID by 90-NM-... YAML file, keeping the same Netdef ID.
Implement handling of keyfile UUIDs for each NM connection profile
Implement handling of empty keyfile groups (i.e. relevant for [bridge])
Commits:
* nm:nm-keyfile: allow to define empty Keyfile groups, like [bridge] or [proxy]
* passthrough: switch from GHashTable to GDataList, as some of the keyfile values need to be ordered, like tc.qdisks or tc.filters
* nm: special handling for [tc] group, where keys can contain dots
* serialize: write match stanza only for physical devices
* parse: modems are physical links and can have a 'match' stanza
* parse: improve clearing of netdefs
* nm: improve annotation of passthrouh settings
* nm-keyfile: improve removal of supported keys
Also, removes the group, if all of its keys were removed
* nm: improve detection of GSM connection
GSM/CDMA can also be defined on a type=bluetooth connection – not just
on MODEMS connections
* nm:nm-keyfile:serialize: set wakeonlan only for ethernet devices
* nm-keyfile:serialize: some modem stanzas
* nm-keyfile:serialize: handle OTHER type via full passthrough mode
* nm:nm-keyfile:serialize: improve interface-name/match handling
* nm: improve handling of NM uuid
* nm: improve passthrough.connection.type handling
* nm: improve handling of wifi.mode
* tests: add and adopt test cases
* nm-keyfile: allow to pass a previously known netdef ID, to enable overrides
* parse: allow changing match.name independenly of netdef ID
* nm-keyfile: update GSM vs CDMA comment
* parse: avoid creation of 'netdefs' hashmap in netplan_parse_yaml
Implement NM keyfile passthrough mode as a fallback mechanism (#183)
This is 5th in a series of pull requests to prepare for the NetworkManager YAML backend. It builds upon #187
It introduces a new (undocumented) top level type others, which needs to be used when a NM connection of unsupported type needs to be written. If used the connection.type needs to be specified by a passthrough setting, e.g.:
network:
version: 2
others:
renderer: NetworkManager
networkmanager:
uuid: ...
passthrough: connection.type: vxlan ipv4.method: auto
...
others is undocumented on purpose, as this is not supposed to be used in regular netplan configs, but can be used as a fallback mechanism, if a given network connection type is not yet implemented in netplan. It can then still be used via passthrough mode.
Furthermore, this PR introduced a new module nm-keyfile to libnetplan, which contains some logic to help with integration of the NetworkManager YAML settings backend. Especially the netplan_render_yaml_from_nm_keyfile function, which transforms a given GKeyFile* structure into a valid NetplanNetDefinition* structure and uses the serializer to transform this into a valid netplan YAML and save it to disk. For now this makes heavy use the the passthrough fallback mechanism, but it will be extended to place keyfile settings into the correct netplan schema in the future, step by step.
The nm.c module was extended to make use of the passthrough fallback mechanism, to render a valid NetworkManager keyfile out of this data and write it to disk. It uses the internal netplan data structure to generate the keyfile as usual, but allows to extend (or override) specified settings via passthrough if they are not (yet/fully) supported.
As discussed in #181, the netplan_get_id_from_nm_filename function is moved into this new nm-keyfile module from utils as this might be a better place for keyfile specific functionality.
Currently this PR is based upon the slyon/nm-4 branch. It will be rebased once PR #187 is merged, but the underlying base shouldn't change too much, so it should be fine to review this already.
Commits:
* nm-keyfile: Add netplan_render_yaml_from_nm_keyfile API
* configmanager: handle 'others' key
* nm: Implement fallback/passthrough mode
* nm-keyfile: support wifis, modems, bridges in addition to ethernets (and others)
* nm-keyfile: cleanup
* nm-keyfile: support bonds, vlans, tunnels
* validation: check others vs passthrough
* nm: fallback generator: allow dotted groups (i.e. wireguard-peer)
* nm: Update copyright
* nm-keyfile: handle connection-type alias
* WIP: nm: fallback integration
* nm-keyfile: modularize and clear handled fallback keys
* WIP: nm: fallback wifi_mode & interface_name
* WIP: netplan-yaml: draft YAML export
* NM: split into keyfile parser and YAML serializer
* nm-keyfile: cleanup
* nm: cleanup
* parse: cleanup
* nm: mention overriden settings in keyfile comment
* nm: allow to specify name (NM id) from backend_settings
* nm: simplify connection.type error handling
* test_nm_backend: adopt tests
* nm-keyfile: move netplan_get_id_from_nm_filename from util
* tests: move generator tests into generator/test_passthrough.py
* nm-keyfile: clear netdefs and improve docs
* nm: improve comments/docs
* nm-keyfile: support canonical names in addition to alias
* nm-keyfile: improve formatting
* doc: mention the device type
Prepare passthrough mode for NM backend and YAML serializer (#187)
This is 4th in a series of pull requests. It enables parsing of the passthrough keyword inside the networkmanager backend settings, inside any netdef or wifi AP. Furthermore, it add a new serialize.c/h module, which can take a single netdef and render it into a YAML file. Testing is implemented by rendering a given YAML file into a new YAML file, and comparing the 1:1 match.
The YAML serializer does (by far) not yet support all supported netplan settings, but only the minimal set to enable the NetworkManager backend via passthrough mode.
The NM backend settings need to be available in every wifi AP definition (in addition to the netdef itself), as in the NetworkManager world, each wifi connection is a separate connection profile, while in the netplan world the wifi networks/AP are combined under a single device/netdef. But the settings passthrough (i.e. fallback mode) needs to be available for each individual NM connection profile, therefore we need them in the AP struct.
The passthrough setting is a mapping inside backend specific settings of key-value pairs, where the keys are separated by a dot in the format KEYFILE_GROUP.KEYFILE_KEY. Those values are passed through to the keyfile as is, and can be used as a fallback mechanism, where a specific feature is not yet implemented.
NM: refactor keyfile generator, by using GLib's keyfile writer instead of custom writer (#184)
This is the 3rd in a series of pull requests to enable the implementation of a YAML/netplan backend for NetworkManager.
In this PR netplan's current keyfile generation code (in nm.c) is refactored to make use of GLib's keyfile writer, instead of using a custom, home grown approach. This way the keyfile settings can easily be read, written and overwritten (e.g. by the keyfile passthrough/fallback settings in #183) and it should be more robust overall.
While on it: Clean up the usage of keyfile alias for ethernet, wifi and wifi-security, which NetworkManager writes by default since a long time.
Implement netplan_generate and netplan_delete_connection APIs (#182)
This is 2nd in a series of pull request implementing the functionality in libnetplan to provide a NetworkManager YAML backend. It builds upon #181
* It improves the netplan set CLI a bit, to delete a file, if only network: {version: 2} is left.
* It implements a netplan_generate function, which will call netplan generate in the background by spawning another process.
* At some point the generate binary should be refactored, to call functions of libnetplan instead, so we could use this same (refactored) functionality inside the library directly
* But for now this is as good as it gets...
* It implements a netplan_delete_connection function, which will delete a connection from the YAML structure of a file (or potentially the whole file, if it is empty afterwards), by utilizing the netplan set network.TYPE.IFNAME=NULL functionality by spawning another process.
* At some point we should get rid of this Python/C split of the YAML parsers, to have all the functionality available inside the C library.
* But for now this is as good as it gets, without duplicating any logic...
