Merge ~slyon/network-manager:netplan-integration into network-manager:ubuntu/master

Proposed by Lukas Märdian
Status: Merged
Merged at revision: bcd43413ecea3493bfbcf091b95508a4120ba7d3
Proposed branch: ~slyon/network-manager:netplan-integration
Merge into: network-manager:ubuntu/master
Diff against target: 1558 lines (+1454/-1)
11 files modified
debian/changelog (+14/-0)
debian/control (+4/-0)
debian/network-manager.postinst (+33/-0)
debian/network-manager.preinst (+18/-0)
debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch (+134/-0)
debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch (+525/-0)
debian/patches/series (+2/-0)
debian/rules (+2/-1)
debian/tests/control (+4/-0)
debian/tests/nm.py (+1/-0)
debian/tests/nm_netplan.py (+717/-0)
Reviewer Review Type Date Requested Status
Sebastien Bacher Approve
Danilo Egea Gondolfo (community) Approve
Lukas Märdian Abstain
Review via email: mp+423735@code.launchpad.net

Description of the change

Deploying the NetworkManager-Netplan integration (a.k.a "Netplan everywhere") on Ubuntu Desktop by default.

This depends on libnetplan >= v0.106 and we cannot depend on libnetplan on i386, so we're dropping the patch to the NetworkManager daemon binary on i386, using quilt (that binary isn't built on i386 anyway).

On installation of the package any existing configuration from /etc/NetworkManager/system-connections will be backed up in /root/NetworkManaker.bak

On updade of the package, any existing configuration from /etc/NetworkManager/system-connections will be migrated to /etc/netplan and translated into corresponding YAML files. Missing settings will be handled usinng the "passthrough" and "nm-devices" fallback mechanism.

Netplan patches are managed via git-buildpackage patch-queue (Using "Gbp-Pq: Topic netplan")

To post a comment you must log in.
Revision history for this message
Olivier Gayot (ogayot) wrote :

Thanks! A few comments inline.

Revision history for this message
Danilo Egea Gondolfo (danilogondolfo) wrote :

Left one comment inline about the migration script.

review: Needs Fixing
Revision history for this message
Sebastien Bacher (seb128) wrote :

Confirming the issue raised by Danilo, the script failed to migrate several connections using a ca-cert and pointing to a filename stored in directory with a 'é' in its name

it also displayed some warnings (translated from french so the wording might not be the correct english one)

> Warning: there is another connection with the name '...'. Refer to the connection using its uuid '...'

review: Needs Fixing
Revision history for this message
Sebastien Bacher (seb128) wrote :

Unsure if that's specific to the change there or a new mantic issue but the autopkgtest against a ppa testbuild failed

https://autopkgtest.ubuntu.com/results/autopkgtest-mantic-ubuntu-desktop-transitions/mantic/amd64/n/network-manager/20230508_144934_e64d7@/log.gz

'Error: failed to reload connections: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: Object does not exist at path “/org/freedesktop/NetworkManager/Settings”.
dpkg: error processing package network-manager (--configure):
 installed network-manager package post-installation script subprocess returned error exit status 1'

Revision history for this message
Lukas Märdian (slyon) wrote :

Thanks!

I've fixed the utf-8 (non-ASCII keyfiles) issue reported by @seb128 and @danilogondolfo as suggested by Danilo (i.e. checking for the "text/plain" mime type instead).

Thanks Sebastien for the "Error: failed to reload connections: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod" error, that's a good catch! It didn't happen on my Lunar autopkgtests, but I could reproduce it on Mantic, too. Seems to be some kind of race condition when debhelper tries to re-start NetworkManager.service after upgrade and then the postinst script is calling into "nmcli", while NM is not yet fully up. I've added a small "nm-online -s" line in there to make it wait for NM, which now passes the autopkgtests on Mantic again:
```
autopkgtest [15:07:36]: @@@@@@@@@@@@@@@@@@@@ summary
wpa-dhclient PASS
nm.py PASS
killswitches-no-urfkill PASS
urfkill-integration PASS
nm_netplan.py PASS
qemu-system-x86_64: terminating on signal 15 from pid 1944756 (/usr/bin/python3)
```

For the "Warning: there is another connection with the name '...'. Refer to the connection using its uuid '...'" message, I don't really know where it comes from, as we're only modifying connections through nmcli, using their UUID identifiers... Could you try to provide a reproducer for that?

PTAL.

Revision history for this message
Sebastien Bacher (seb128) wrote :

Thanks for the tweaks, I restored the connections from the backup dir, tried to the new revision and now hit

+ echo Migrating Ziggo (03c8f2a7-268d-4765-b626-efcc02dd686c) to /etc/netplan
Migrating Ziggo (03c8f2a7-268d-4765-b626-efcc02dd686c) to /etc/netplan
+ nmcli con mod 03c8f2a7-268d-4765-b626-efcc02dd686c con-name Ziggo.NETPLAN_MIGRATE
Erreur : la modification de la connexion « Ziggo.NETPLAN_MIGRATE » a échoué : Remote peer disconnected
dpkg: erreur de traitement du paquet network-manager (--install) :
 le sous-processus paquet network-manager script post-installation installé a renvoyé un état de sortie d'erreur 1

there is a NetworkManager crash report from that time in /var/crash, it hit the
nms-keyfile-writer.c assert on l538 which is part of this changeset, the journal includes

> <error> [1683703791.7570] BUG: the profile cannot be stored in keyfile format without becoming unusable: invalid connection: 802-1x.identity: la propriété est manquante
and
> <warn> [1683703795.8560] keyfile: load: "/run/NetworkManager/system-connections/netplan-NM-03c8f2a7-268d-4765-b626-efcc02dd686c-Ziggo.nmconnection": failed to load connection: invalid connection: 802-1x.identity: la propriété est manquante

I'm sending the corresponding file via email

So it seems we have a bug there that n-m is hitting an assertion, also it feels wrong than failing to migrate one connection is failing the package installation and letting the packaging system in a state where most users will not be able to recover from easily

review: Needs Fixing
Revision history for this message
Lukas Märdian (slyon) wrote :

Thank you very much for the valuable feedback! I've rebased the branch to properly include the current fixes and moved forward to a v4 Netplan-integration patch, which makes use of "#if WITH_NETPLAN" inside the NM codebase, to allow "--enable-netplan"/"--disable-netplan" buildflags (enabled by default), so we can easily disable it on i386 and don't need that quilt hack anymore.

Also, thanks for providing the test files by email. I agree we shouldn't fail the installation of the package when the migration of a single connection profile fails, it should just skip that connection and keep it as a keyfile. I'll be working on that!

I'll also investigate that crash, but I think we should not block on that one (once we are able to skip non-migrated connections). I'll try to collect all the information and create a corresponding bug report.

Revision history for this message
Lukas Märdian (slyon) wrote :

So Danilo found the root cause of the "802-1x.identity" issue (crash), which is a bug in libnetplan (Netplan's keyfile parser). This will be handled via in bug #2016625 (details to follow there)

Revision history for this message
Lukas Märdian (slyon) wrote :

I've just pushed another update to the branch, which handles migration failures gracefully:
Before migration it will create a temporary backup of the keyfile in and restore that keyfile backup, should the migration fail.

We're working on the crash (libnetplan bug) separately, as mentioned above. I've pushed the current state of this branch into the "Netplan Everywhere" PPA via https://code.launchpad.net/~canonical-foundations/+recipe/network-manager-netplan-lunar-gu

Let me know what you think!

Revision history for this message
Sebastien Bacher (seb128) wrote :

Thanks, I can confirm that the new changes make the package install without error and that it rollbacks the problematic connection. The autopkgtests issue is also resolve

The warning I mentioned earlier about the exiting name/refer by uid is printing by

> nmcli con mod daffb72f-c980-4ede-95cf-31fb7d24484d con-name Hotspot

There is another connection with the same name. The command is correctly issued with the uid so I think the warning is safe to ignore

I will upload the changes to mantic today unless you would prefer to see the netplan fix landing first?

review: Approve
Revision history for this message
Danilo Egea Gondolfo (danilogondolfo) :
Revision history for this message
Danilo Egea Gondolfo (danilogondolfo) wrote :

I left an inline comment about some code we probably want to remove from the #if #endif block.

Revision history for this message
Lukas Märdian (slyon) wrote :

Thank you for the additional reviews!

@seb128, please go ahead and upload the changes into Mantic, we'll be fixing (lib-)netplan in parallel.

@danilogondolfo, you're right it is NetworkManager code. But in fact we're not patching it out, but rather creating a copy of that block for the Netplan codepath. So we should be safe ignoring that whole block inside #if WITH_NETPLAN ... #endif if the Netplan buildflag is disabled. It is visible in the GitHub diff, see: https://github.com/slyon/NetworkManager/commit/aaa791cb61b24a9416db18f2290f711bc9a8f26e#diff-e46473e1ac3ce2315efbd5fbad59951998e274ca21244f6b9e807f20e67db58aL344

review: Abstain
Revision history for this message
Danilo Egea Gondolfo (danilogondolfo) :
review: Approve
Revision history for this message
Sebastien Bacher (seb128) wrote :

Thanks, merged and uploaded to Ubuntu now!

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/debian/changelog b/debian/changelog
2index f362bb1..e2e79df 100644
3--- a/debian/changelog
4+++ b/debian/changelog
5@@ -1,3 +1,17 @@
6+network-manager (1.42.4-1ubuntu3) mantic; urgency=medium
7+
8+ [ Lukas Märdian ]
9+ * d/{control,rules,patches}: Prepare libnetplan build (non i386)
10+ * debian/patches/netplan: Add libnetplan backend integration patch
11+ * d/network-manager.preinst: backup previous configuration automatically
12+ * d/network-manager.postinst: Trigger Netplan migration on install/upgrade
13+
14+ [ Danilo Egea Gondolfo ]
15+ * d/t/nm.py: Fix autopkgtests when Netplan is in use
16+ * d/t/nm_netplan.py: Add autopkgtests for the netplan integration
17+
18+ -- Lukas Märdian <slyon@ubuntu.com> Wed, 10 May 2023 11:56:58 +0200
19+
20 network-manager (1.42.4-1ubuntu2) lunar; urgency=medium
21
22 * d/t/nm.py: Fix autopkgtests with NM-1.42's 'lo' connection (LP: #2009543)
23diff --git a/debian/control b/debian/control
24index 603370d..dd8bf9d 100644
25--- a/debian/control
26+++ b/debian/control
27@@ -38,6 +38,9 @@ Build-Depends: debhelper-compat (= 13),
28 python3-dbus <!nocheck>,
29 python3-pexpect <!nocheck>,
30 iproute2 <!nocheck>,
31+ libyaml-dev [!i386],
32+ libnetplan-dev (>= 0.106~) [!i386],
33+ netplan.io (>= 0.106~) [!i386] <!nocheck>,
34 Standards-Version: 4.6.2
35 Rules-Requires-Root: no
36 XS-Debian-Vcs-Git: https://salsa.debian.org/utopia-team/network-manager.git
37@@ -51,6 +54,7 @@ Architecture: linux-any
38 Pre-Depends: ${misc:Pre-Depends}
39 Depends: ${shlibs:Depends},
40 ${misc:Depends},
41+ netplan.io (>= 0.106~) [!i386],
42 libnm0 (= ${binary:Version}),
43 isc-dhcp-client,
44 default-dbus-system-bus | dbus-system-bus,
45diff --git a/debian/network-manager.postinst b/debian/network-manager.postinst
46index e2cb049..0db377b 100644
47--- a/debian/network-manager.postinst
48+++ b/debian/network-manager.postinst
49@@ -81,3 +81,36 @@ esac
50
51 #DEBHELPER#
52
53+# Run "Netplan Everywhere" migration after debhelper (re-)started
54+# NetworkManager.service for us. On every package upgrade.
55+DIR="/etc/NetworkManager/system-connections"
56+if [ "$1" = "configure" ] && [ -d "$DIR" ]; then
57+ mkdir -p /run/netplan/nm-migrate
58+ for CON in /etc/NetworkManager/system-connections/*; do
59+ TYPE=$(file -bi "$CON" | cut -s -d ";" -f 1)
60+ [ "$TYPE" = "text/plain" ] || continue # skip non-keyfiles
61+ UUID=$(grep "^uuid=" "$CON" | cut -c 6-)
62+ if [ -n "$UUID" ]
63+ then
64+ # Wait for NetworkManager startup to complete,
65+ # so we can safely use nmcli. Wait in every interation to handle
66+ # a crashed NetworkManager in the previous migraiton step.
67+ nm-online -qs || (echo "SKIP: NetworkManager is not ready ..." 1>&2 && continue)
68+ BACKUP="/run/netplan/nm-migrate/"$(basename "$CON")
69+ cp "$CON" "$BACKUP"
70+ ORIG_NAME=$(nmcli --get-values connection.id con show "$UUID")
71+ echo "Migrating $ORIG_NAME ($UUID) to /etc/netplan" 1>&2
72+ # Touch the connection's ID (con-name) to trigger its migration.
73+ # The Netplan integration will translate the original NM keyfile from
74+ # /etc/NetworkManager/system-connections/* to a YAML file located in
75+ # /etc/netplan/90-NM-*.yaml and re-generate a corresponding keyfile in
76+ # /run/NetworkManager/system-connections/netplan-NM-*.nmconnection
77+ nmcli con mod "$UUID" con-name "$ORIG_NAME" || \
78+ (echo "FAILED. Restoring backup ..." 1>&2 && mv "$BACKUP" "$CON" && \
79+ rm -f "/etc/netplan/90-NM-$UUID"*.yaml)
80+ rm -f "$BACKUP" # clear backup (if it still exists)
81+ fi
82+ done
83+ rm -rf /run/netplan/nm-migrate # cleanup after ourselves
84+ (nm-online -qs && nmcli con reload) || echo "WARNING: NetworkManager could not reload connections ..." 1>&2
85+fi
86diff --git a/debian/network-manager.preinst b/debian/network-manager.preinst
87new file mode 100644
88index 0000000..886ab77
89--- /dev/null
90+++ b/debian/network-manager.preinst
91@@ -0,0 +1,18 @@
92+#!/bin/sh
93+
94+set -e
95+
96+#DEBHELPER#
97+
98+DIR="/etc/NetworkManager/system-connections"
99+CNT=$(ls -1 "$DIR" | wc -l)
100+if ([ "$1" = "upgrade" ] || [ "$1" = "install" ]) && [ -d "$DIR" ] && [ "$CNT" -ge 1 ]; then
101+ # create backup directory if it does not yet exist
102+ mkdir -p /root/NetworkManager.bak || true
103+ BAK="/root/NetworkManager.bak/system-connections_$2"
104+ if [ -d "$BAK" ]; then
105+ rm -r "$BAK"
106+ fi
107+ # copy current system-connections to the backup directory
108+ cp -r "$DIR" "$BAK"
109+fi
110diff --git a/debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch b/debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch
111new file mode 100644
112index 0000000..f3ae212
113--- /dev/null
114+++ b/debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch
115@@ -0,0 +1,134 @@
116+From 9dd31963bd89ccba5e96f26506996e792177bdab Mon Sep 17 00:00:00 2001
117+From: Lukas Märdian <slyon@ubuntu.com>
118+Date: Tue, 9 May 2023 17:09:48 +0200
119+Subject: [PATCH 1/2] netplan: Adopt buildsystems for Netplan integration
120+
121+Autotools and Meson will define a "WITH_NETPLAN" variable with the
122+values 1 or 0 accordingly, using the config.h header.
123+---
124+ Makefile.am | 4 +++-
125+ configure.ac | 17 +++++++++++++++++
126+ meson.build | 10 ++++++++++
127+ meson_options.txt | 1 +
128+ src/core/meson.build | 1 +
129+ 5 files changed, 32 insertions(+), 1 deletion(-)
130+
131+diff --git a/Makefile.am b/Makefile.am
132+index a452cacac4..677c078655 100644
133+--- a/Makefile.am
134++++ b/Makefile.am
135+@@ -2588,7 +2588,8 @@ $(src_core_libNetworkManagerBase_la_OBJECTS): $(src_libnm_core_public_mkenums_h)
136+
137+ ###############################################################################
138+
139+-src_core_libNetworkManager_la_CPPFLAGS = $(src_core_cppflags)
140++src_core_libNetworkManager_la_CPPFLAGS = $(src_core_cppflags) \
141++ $(NETPLAN_CFLAGS)
142+
143+ src_core_libNetworkManager_la_SOURCES = \
144+ \
145+@@ -2803,6 +2804,7 @@ src_core_libNetworkManager_la_LIBADD = \
146+ $(LIBAUDIT_LIBS) \
147+ $(LIBPSL_LIBS) \
148+ $(LIBCURL_LIBS) \
149++ $(NETPLAN_LIBS) \
150+ $(NULL)
151+
152+ $(src_core_libNetworkManager_la_OBJECTS): $(src_libnm_core_public_mkenums_h)
153+diff --git a/configure.ac b/configure.ac
154+index b0cf78dc03..6b4c1467b6 100644
155+--- a/configure.ac
156++++ b/configure.ac
157+@@ -880,6 +880,22 @@ else
158+ AC_DEFINE(WITH_OPENVSWITCH, 0, [Whether we build with OVS plugin])
159+ fi
160+
161++# Netplan integration
162++AC_ARG_ENABLE(netplan, AS_HELP_STRING([--enable-netplan], [Enable Netplan integration]))
163++if test "${enable_netplan}" != "no"; then
164++ PKG_CHECK_MODULES(NETPLAN, [netplan >= 0.106], [enable_netplan=yes], [enable_netplan=no])
165++ if test "$enable_netplan" != "yes"; then
166++ AC_MSG_ERROR(Netplan is required)
167++ fi
168++ enable_netplan='yes'
169++fi
170++AM_CONDITIONAL(WITH_NETPLAN, test "${enable_netplan}" = "yes")
171++if test "${enable_netplan}" = "yes" ; then
172++ AC_DEFINE(WITH_NETPLAN, 1, [Whether we build with Netplan integration])
173++else
174++ AC_DEFINE(WITH_NETPLAN, 0, [Whether we build with Netplan integration])
175++fi
176++
177+ # DHCP client support
178+ AC_ARG_WITH([dhclient],
179+ AS_HELP_STRING([--with-dhclient=yes|no|path], [Enable dhclient support]))
180+@@ -1400,6 +1416,7 @@ echo " ofono: $with_ofono"
181+ echo " concheck: $enable_concheck"
182+ echo " libteamdctl: $enable_teamdctl"
183+ echo " ovs: $enable_ovs"
184++echo " netplan: $enable_netplan"
185+ echo " nmcli: $build_nmcli"
186+ echo " nmtui: $build_nmtui"
187+ echo " nm-cloud-setup: $with_nm_cloud_setup"
188+diff --git a/meson.build b/meson.build
189+index 6813e52ac1..e808729c22 100644
190+--- a/meson.build
191++++ b/meson.build
192+@@ -274,6 +274,8 @@ config_h.set10('HAVE_LIBSYSTEMD', libsystemd_dep.found())
193+ systemd_dep = dependency('systemd', required: false)
194+ have_systemd_200 = systemd_dep.found() and systemd_dep.version().version_compare('>= 200')
195+
196++libnetplan_dep = dependency('netplan', version: '>= 0.106', required: false)
197++
198+ gio_unix_dep = dependency('gio-unix-2.0', version: '>= 2.40')
199+
200+ glib_dep = declare_dependency(
201+@@ -646,6 +648,13 @@ if enable_ovs
202+ endif
203+ config_h.set10('WITH_OPENVSWITCH', enable_ovs)
204+
205++# Netplan integration
206++enable_netplan = get_option('netplan')
207++if enable_netplan
208++ assert(libnetplan_dep.found(), 'libnetplan is needed for Netplan integration.')
209++endif
210++config_h.set10('WITH_NETPLAN', enable_netplan)
211++
212+ # DNS resolv.conf managers
213+ config_dns_rc_manager_default = get_option('config_dns_rc_manager_default')
214+ config_h.set_quoted('NM_CONFIG_DEFAULT_MAIN_RC_MANAGER', config_dns_rc_manager_default)
215+@@ -1064,6 +1073,7 @@ output += ' ofono: ' + enable_ofono.to_string() + '\n'
216+ output += ' concheck: ' + enable_concheck.to_string() + '\n'
217+ output += ' libteamdctl: ' + enable_teamdctl.to_string() + '\n'
218+ output += ' ovs: ' + enable_ovs.to_string() + '\n'
219++output += ' netplan: ' + enable_netplan.to_string() + '\n'
220+ output += ' nmcli: ' + enable_nmcli.to_string() + '\n'
221+ output += ' nmtui: ' + enable_nmtui.to_string() + '\n'
222+ output += ' nm-cloud-setup: ' + enable_nm_cloud_setup.to_string() + '\n'
223+diff --git a/meson_options.txt b/meson_options.txt
224+index 4e359f9e92..9f52b4f6e4 100644
225+--- a/meson_options.txt
226++++ b/meson_options.txt
227+@@ -37,6 +37,7 @@ option('ofono', type: 'boolean', value: false, description: 'Enable oFono suppor
228+ option('concheck', type: 'boolean', value: true, description: 'enable connectivity checking support')
229+ option('teamdctl', type: 'boolean', value: false, description: 'enable Teamd control support')
230+ option('ovs', type: 'boolean', value: true, description: 'enable Open vSwitch support')
231++option('netplan', type: 'boolean', value: true, description: 'Enable Netplan integration')
232+ option('nmcli', type: 'boolean', value: true, description: 'Build nmcli')
233+ option('nmtui', type: 'boolean', value: true, description: 'Build nmtui')
234+ option('nm_cloud_setup', type: 'boolean', value: false, description: 'Build nm-cloud-setup, a tool for automatically configure networking in cloud (EXPERIMENTAL!)')
235+diff --git a/src/core/meson.build b/src/core/meson.build
236+index 6c1d463ed1..35ece13f06 100644
237+--- a/src/core/meson.build
238++++ b/src/core/meson.build
239+@@ -71,6 +71,7 @@ nm_deps = [
240+ libndp_dep,
241+ libudev_dep,
242+ logind_dep,
243++ libnetplan_dep,
244+ ]
245+
246+ if enable_concheck
247+--
248+2.39.2
249+
250diff --git a/debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch b/debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch
251new file mode 100644
252index 0000000..1d959f2
253--- /dev/null
254+++ b/debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch
255@@ -0,0 +1,525 @@
256+From aaa791cb61b24a9416db18f2290f711bc9a8f26e Mon Sep 17 00:00:00 2001
257+From: Lukas Märdian <lukas.maerdian@canonical.com>
258+Date: Tue, 2 Feb 2021 15:52:05 +0100
259+Subject: [PATCH 2/2] netplan: make use of libnetplan for YAML backend
260+
261+Origin: https://github.com/slyon/NetworkManager/tree/netplan-nm-1.42
262+Forwarded: no, https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/556
263+Last-Update: 2023-05-11
264+
265+This patch modifies NetworkManager's nms-keyfile-plugin in a way to
266+write YAML connections (according to the netplan spec) to
267+/etc/netplan/*.yaml instead of NM's native keyfile connection profiles
268+in /etc/NetworkManager/system-connections/*.nmconnection.
269+
270+Whenever a connection profile is to be written (add/modify) the keyfile,
271+generated internally by NM, is passed into libnetplan's
272+"netplan_parser_load_keyfile()" API, validated via
273+"netplan_state_import_parser_results()" and converted to a netplan YAML
274+config by calling libnetplan's "netplan_netdef_write_yaml()" API. The
275+internal keyfile is thrown away afterwards.
276+
277+Whenever a connection profile is to be deleted the netplan-/netdef-id is
278+extracted from the ephemeral keyfile in /run/NetworkManager/system-connections
279+via "netplan_get_id_from_nm_filepath()" and the corresponding YAML is
280+updated/deleted by calling "netplan_delete_connection()".
281+
282+Each time the YAML data was modified, NetworkManager calls
283+"netplan generate" to produce new ephemeral keyfile connections in
284+/run/NetworkManager/system-connections for NM to read-back. This way the
285+netplan generator can be used as intended (no need for duplicated
286+keyfile export functionality) and the nms-keyfile-writer can be re-used
287+without any patching needed.
288+
289+V2:
290++ ported to NetworkManager 1.36.6 (Ubuntu Jammy LTS/Core 22.04)
291++ test-keyfile-settings.c: clear netplan YAML config from previous runs
292++ nms-keyfile-writer.c: avoid double-free of `path` on exit, caused by gs_free
293+
294+V3:
295++ ported to NM 1.40.6 (Ubuntu Lunar), using new libnetplan API (v0.106)
296++ ignore .nm-generated connections (LP: #1998207)
297+
298+V4:
299++ encapsulated all Netplan code in "#if WITH_NETPLAN" as provided by the
300+ buildsystems via config.h (see previous commit)
301++ nms-keyfile-writer.c: increase buffer size to account for ".nmconnection"
302+ suffix in netplan_netdef_get_output_filename()
303+
304+Co-authored-by: Alfonso Sanchez-Beato <alfonso.sanchez-beato@canonical.com>
305+Co-authored-by: Danilo Egea Gondolfo <danilo.egea.gondolfo@canonical.com>
306+---
307+ .../plugins/keyfile/nms-keyfile-plugin.c | 34 ++++
308+ .../plugins/keyfile/nms-keyfile-utils.c | 18 ++
309+ .../plugins/keyfile/nms-keyfile-utils.h | 4 +
310+ .../plugins/keyfile/nms-keyfile-writer.c | 173 ++++++++++++++++++
311+ .../keyfile/tests/test-keyfile-settings.c | 61 +++++-
312+ 5 files changed, 286 insertions(+), 4 deletions(-)
313+
314+diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c b/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c
315+index 1d7de8d24b..4b986dc0c8 100644
316+--- a/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c
317++++ b/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c
318+@@ -12,6 +12,9 @@
319+ #include <unistd.h>
320+ #include <sys/types.h>
321+ #include <sys/time.h>
322++#if WITH_NETPLAN
323++#include <netplan/util.h>
324++#endif
325+
326+ #include "libnm-std-aux/c-list-util.h"
327+ #include "libnm-glib-aux/nm-c-list.h"
328+@@ -309,6 +312,12 @@ _load_file(NMSKeyfilePlugin *self,
329+ gs_free char *full_filename = NULL;
330+ struct stat st;
331+
332++ #if WITH_NETPLAN
333++ // Handle all netplan generated connections via STORAGE_TYPE_ETC, as they live in /etc/netplan
334++ if (g_str_has_prefix(filename, "netplan-"))
335++ storage_type = NMS_KEYFILE_STORAGE_TYPE_ETC;
336++ #endif
337++
338+ if (_ignore_filename(storage_type, filename)) {
339+ gs_free char *nmmeta = NULL;
340+ gs_free char *loaded_path = NULL;
341+@@ -584,6 +593,9 @@ reload_connections(NMSettingsPlugin *plugin,
342+ NM_SETT_UTIL_STORAGES_INIT(storages_new, nms_keyfile_storage_destroy);
343+ int i;
344+
345++ #if WITH_NETPLAN
346++ generate_netplan(NULL);
347++ #endif
348+ _load_dir(self, NMS_KEYFILE_STORAGE_TYPE_RUN, priv->dirname_run, &storages_new);
349+ if (priv->dirname_etc)
350+ _load_dir(self, NMS_KEYFILE_STORAGE_TYPE_ETC, priv->dirname_etc, &storages_new);
351+@@ -1008,6 +1020,15 @@ delete_connection(NMSettingsPlugin *plugin, NMSettingsStorage *storage_x, GError
352+ previous_filename = nms_keyfile_storage_get_filename(storage);
353+ uuid = nms_keyfile_storage_get_uuid(storage);
354+
355++ #if WITH_NETPLAN
356++ nm_auto_unref_keyfile GKeyFile *key_file = NULL;
357++ key_file = g_key_file_new ();
358++ if (!g_key_file_load_from_file (key_file, previous_filename, G_KEY_FILE_NONE, error))
359++ return FALSE;
360++ g_autofree gchar* ssid = NULL;
361++ ssid = g_key_file_get_string(key_file, "wifi", "ssid", NULL);
362++ #endif
363++
364+ if (!NM_IN_SET(storage->storage_type,
365+ NMS_KEYFILE_STORAGE_TYPE_ETC,
366+ NMS_KEYFILE_STORAGE_TYPE_RUN)) {
367+@@ -1033,6 +1054,19 @@ delete_connection(NMSettingsPlugin *plugin, NMSettingsStorage *storage_x, GError
368+ } else
369+ operation_message = "deleted from disk";
370+
371++ #if WITH_NETPLAN
372++ g_autofree gchar *netplan_id = NULL;
373++ ssize_t netplan_id_size = 0;
374++
375++ netplan_id = g_malloc0(strlen(previous_filename));
376++ netplan_id_size = netplan_get_id_from_nm_filepath(previous_filename, ssid, netplan_id, strlen(previous_filename) - 1);
377++ if (netplan_id_size > 0) {
378++ _LOGI ("deleting netplan connection: %s", netplan_id);
379++ netplan_delete_connection(netplan_id, NULL);
380++ generate_netplan(NULL);
381++ }
382++ #endif
383++
384+ _LOGT("commit: deleted \"%s\", %s %s (%s%s%s%s)",
385+ previous_filename,
386+ storage->is_meta_data ? "meta-data" : "profile",
387+diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-utils.c b/src/core/settings/plugins/keyfile/nms-keyfile-utils.c
388+index 7c0e329e2d..a91ee6997a 100644
389+--- a/src/core/settings/plugins/keyfile/nms-keyfile-utils.c
390++++ b/src/core/settings/plugins/keyfile/nms-keyfile-utils.c
391+@@ -399,3 +399,21 @@ nms_keyfile_utils_check_file_permissions(NMSKeyfileFiletype filetype,
392+ NM_SET_OUT(out_st, st);
393+ return TRUE;
394+ }
395++
396++#if WITH_NETPLAN
397++gboolean
398++generate_netplan(const char* rootdir)
399++{
400++ /* TODO: call the io.netplan.Netplan.Generate() DBus method directly, after
401++ * finding a way to pass the --root-dir parameter via DBus, to make it work
402++ * inside NM's unit-tests where netplan needs to read & generate outside of
403++ * /etc/netplan and /run/{systemd,NetworkManager} */
404++ const gchar *argv[] = { "netplan", "generate", NULL , NULL, NULL };
405++ if (rootdir) {
406++ argv[2] = "--root-dir";
407++ argv[3] = rootdir;
408++ }
409++ return g_spawn_sync(NULL, (gchar**)argv, NULL, G_SPAWN_SEARCH_PATH,
410++ NULL, NULL, NULL, NULL, NULL, NULL);
411++}
412++#endif
413+diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-utils.h b/src/core/settings/plugins/keyfile/nms-keyfile-utils.h
414+index 0fd83bdf35..9ed25d9cfd 100644
415+--- a/src/core/settings/plugins/keyfile/nms-keyfile-utils.h
416++++ b/src/core/settings/plugins/keyfile/nms-keyfile-utils.h
417+@@ -68,4 +68,8 @@ gboolean nms_keyfile_utils_check_file_permissions(NMSKeyfileFiletype filetype,
418+ struct stat *out_st,
419+ GError **error);
420+
421++#if WITH_NETPLAN
422++gboolean generate_netplan(const char* rootdir);
423++#endif
424++
425+ #endif /* __NMS_KEYFILE_UTILS_H__ */
426+diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-writer.c b/src/core/settings/plugins/keyfile/nms-keyfile-writer.c
427+index ad6f277ce3..e5015ab74b 100644
428+--- a/src/core/settings/plugins/keyfile/nms-keyfile-writer.c
429++++ b/src/core/settings/plugins/keyfile/nms-keyfile-writer.c
430+@@ -12,6 +12,14 @@
431+ #include <sys/stat.h>
432+ #include <unistd.h>
433+
434++#if WITH_NETPLAN
435++#include <net/if.h>
436++#include <netplan/parse.h>
437++#include <netplan/parse-nm.h>
438++#include <netplan/util.h>
439++#include <netplan/netplan.h>
440++#endif
441++
442+ #include "libnm-core-intern/nm-keyfile-internal.h"
443+
444+ #include "nms-keyfile-utils.h"
445+@@ -201,6 +209,9 @@ _internal_write_connection(NMConnection *connection,
446+ char **out_path,
447+ NMConnection **out_reread,
448+ gboolean *out_reread_same,
449++ #if WITH_NETPLAN
450++ const char *rootdir,
451++ #endif
452+ GError **error)
453+ {
454+ nm_auto_unref_keyfile GKeyFile *kf_file = NULL;
455+@@ -409,11 +420,161 @@ _internal_write_connection(NMConnection *connection,
456+ if (existing_path && !existing_path_read_only && !nm_streq(path, existing_path))
457+ unlink(existing_path);
458+
459++ #if WITH_NETPLAN
460++ NetplanParser *npp = NULL;
461++ NetplanState *np_state = NULL;
462++
463++ /* NETPLAN: write only non-temporary files to /etc/netplan/... */
464++ if (!is_volatile && !is_nm_generated && !is_external &&
465++ strstr(keyfile_dir, "/etc/NetworkManager/system-connections")) {
466++ g_autofree gchar *ssid = g_key_file_get_string(kf_file, "wifi", "ssid", NULL);
467++ g_autofree gchar *escaped_ssid = ssid ?
468++ g_uri_escape_string(ssid, NULL, TRUE) : NULL;
469++ g_autofree gchar *netplan_id = NULL;
470++ ssize_t netplan_id_size = 0;
471++ NetplanNetDefinition *netdef = NULL;
472++ NetplanStateIterator state_iter;
473++ const gchar* kf_path = path;
474++
475++ if (existing_path && strstr(existing_path, "system-connections/netplan-")) {
476++ netplan_id = g_malloc0(strlen(existing_path));
477++ netplan_id_size = netplan_get_id_from_nm_filepath(existing_path, ssid, netplan_id, strlen(existing_path) - 1);
478++ if (netplan_id_size <= 0) {
479++ g_free(netplan_id);
480++ netplan_id = NULL;
481++ }
482++ }
483++
484++ if (netplan_id && existing_path) {
485++ GFile* from = g_file_new_for_path(path);
486++ GFile* to = g_file_new_for_path(existing_path);
487++ g_file_copy(from, to, G_FILE_COPY_OVERWRITE, NULL, NULL, NULL, NULL);
488++ kf_path = existing_path;
489++ }
490++
491++ // push keyfile into libnetplan for parsing (using existing_path, if available,
492++ // to be able to extract the original netdef_id and override existing settings)
493++ npp = netplan_parser_new();
494++
495++ if (!netplan_parser_load_keyfile(npp, kf_path, &local_err)) {
496++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
497++ "netplan: YAML translation failed: %s", local_err->message);
498++ goto netplan_parser_error;
499++ }
500++
501++ np_state = netplan_state_new();
502++ if (!netplan_state_import_parser_results(np_state, npp, &local_err)) {
503++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
504++ "netplan: YAML validation failed: %s", local_err->message);
505++ goto netplan_error;
506++ }
507++
508++ netplan_state_iterator_init(np_state, &state_iter);
509++ /* At this point we have a single netdef in the netplan state */
510++ netdef = netplan_state_iterator_next(&state_iter);
511++
512++ if (!netdef) {
513++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
514++ "netplan: Netplan state has no network definitions");
515++ goto netplan_error;
516++ }
517++
518++ if (!netplan_netdef_write_yaml(np_state, netdef, rootdir, &local_err)) {
519++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
520++ "netplan: Failed to generate YAML: %s", local_err->message);
521++ goto netplan_error;
522++ }
523++
524++ /* Delete same connection-profile provided by legacy netplan plugin.
525++ * TODO: drop legacy connection handling after 24.04 LTS */
526++ g_autofree gchar* legacy_path = NULL;
527++ legacy_path = g_strdup_printf("/etc/netplan/NM-%s.yaml", nm_connection_get_uuid (connection));
528++ if (g_file_test(legacy_path, G_FILE_TEST_EXISTS | G_FILE_TEST_IS_REGULAR)) {
529++ g_debug("Deleting legacy netplan connection: %s", legacy_path);
530++ unlink(legacy_path);
531++ }
532++
533++ /* Clear original keyfile in /etc/NetworkManager/system-connections/,
534++ * we've written the /etc/netplan/*.yaml file instead. */
535++ unlink(path);
536++ if (!generate_netplan(rootdir)) {
537++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
538++ "netplan generate failed");
539++ goto netplan_error;
540++ }
541++
542++ // Calculating the maximum space needed to store the new keyfile path
543++ ssize_t path_size = strlen(path) + strlen(nm_connection_get_uuid(connection)) + IF_NAMESIZE + 1;
544++ if (escaped_ssid)
545++ path_size += strlen(escaped_ssid);
546++ path_size += 50; // give some extra buffer, e.g. when going from ConName to ConName.nmconnection
547++
548++ g_free(path);
549++ path = g_malloc0(path_size);
550++ path_size = netplan_netdef_get_output_filename(netdef, ssid, path, path_size);
551++
552++ if (path_size <= 0) {
553++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
554++ "netplan: couldn't determine the keyfile path");
555++ goto netplan_error;
556++ }
557++
558++ if (rootdir) {
559++ char* final_path = g_build_path(G_DIR_SEPARATOR_S, rootdir, path, NULL);
560++ g_free(path);
561++ path = final_path;
562++ }
563++
564++ netplan_state_clear(&np_state);
565++ netplan_parser_clear(&npp);
566++
567++ /* re-read again: this time the connection profile newly generated by netplan in /run/... */
568++ if ( out_reread
569++ || out_reread_same) {
570++ gs_free_error GError *reread_error = NULL;
571++
572++ //XXX: why does the _from_keyfile function behave differently?
573++ //reread = nms_keyfile_reader_from_keyfile (kf_file, path, NULL, profile_dir, FALSE, &reread_error);
574++ reread = nms_keyfile_reader_from_file (path, profile_dir, NULL, NULL, NULL, NULL, NULL, NULL, &reread_error);
575++
576++ if ( !reread
577++ || !nm_connection_normalize (reread, NULL, NULL, &reread_error)) {
578++ nm_log_err (LOGD_SETTINGS, "BUG: the profile cannot be stored in keyfile format without becoming unusable: %s", reread_error->message);
579++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
580++ "keyfile writer produces an invalid connection: %s",
581++ reread_error->message);
582++ nm_assert_not_reached ();
583++ return FALSE;
584++ }
585++
586++ if (out_reread_same) {
587++ reread_same = !!nm_connection_compare (reread, connection, NM_SETTING_COMPARE_FLAG_EXACT);
588++
589++ nm_assert (reread_same == nm_connection_compare (connection, reread, NM_SETTING_COMPARE_FLAG_EXACT));
590++ nm_assert (reread_same == ({
591++ gs_unref_hashtable GHashTable *_settings = NULL;
592++
593++ ( nm_connection_diff (reread, connection, NM_SETTING_COMPARE_FLAG_EXACT, &_settings)
594++ && !_settings);
595++ }));
596++ }
597++ }
598++ }
599++ #endif
600++
601+ NM_SET_OUT(out_reread, g_steal_pointer(&reread));
602+ NM_SET_OUT(out_reread_same, reread_same);
603+ NM_SET_OUT(out_path, g_steal_pointer(&path));
604+
605+ return TRUE;
606++
607++#if WITH_NETPLAN
608++netplan_error:
609++ netplan_state_clear(&np_state);
610++netplan_parser_error:
611++ netplan_parser_clear(&npp);
612++ return FALSE;
613++#endif
614+ }
615+
616+ gboolean
617+@@ -454,6 +615,9 @@ nms_keyfile_writer_connection(NMConnection *connection,
618+ out_path,
619+ out_reread,
620+ out_reread_same,
621++ #if WITH_NETPLAN
622++ NULL,
623++ #endif
624+ error);
625+ }
626+
627+@@ -467,6 +631,12 @@ nms_keyfile_writer_test_connection(NMConnection *connection,
628+ gboolean *out_reread_same,
629+ GError **error)
630+ {
631++ #if WITH_NETPLAN
632++ gchar *rootdir = g_strdup(keyfile_dir);
633++ if (g_str_has_suffix (keyfile_dir, "/run/NetworkManager/system-connections")) {
634++ rootdir[strlen(rootdir)-38] = '\0'; /* 38 = strlen("/run/NetworkManager/...") */
635++ }
636++ #endif
637+ return _internal_write_connection(connection,
638+ FALSE,
639+ FALSE,
640+@@ -486,5 +656,8 @@ nms_keyfile_writer_test_connection(NMConnection *connection,
641+ out_path,
642+ out_reread,
643+ out_reread_same,
644++ #if WITH_NETPLAN
645++ rootdir,
646++ #endif
647+ error);
648+ }
649+diff --git a/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c b/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c
650+index 83019babb1..eb59a91eab 100644
651+--- a/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c
652++++ b/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c
653+@@ -24,8 +24,15 @@
654+
655+ #include "nm-test-utils-core.h"
656+
657++#if WITH_NETPLAN
658++#define TEST_KEYFILES_DIR_OLD NM_BUILD_SRCDIR"/src/core/settings/plugins/keyfile/tests/keyfiles"
659++#define TEST_SCRATCH_DIR_OLD NM_BUILD_BUILDDIR"/src/core/settings/plugins/keyfile/tests/keyfiles"
660++#define TEST_KEYFILES_DIR TEST_KEYFILES_DIR_OLD"/run/NetworkManager/system-connections"
661++#define TEST_SCRATCH_DIR TEST_SCRATCH_DIR_OLD"/run/NetworkManager/system-connections"
662++#else
663+ #define TEST_KEYFILES_DIR NM_BUILD_SRCDIR "/src/core/settings/plugins/keyfile/tests/keyfiles"
664+ #define TEST_SCRATCH_DIR NM_BUILD_BUILDDIR "/src/core/settings/plugins/keyfile/tests/keyfiles"
665++#endif
666+
667+ /*****************************************************************************/
668+
669+@@ -113,6 +120,11 @@ assert_reread_and_unlink(NMConnection *connection,
670+ static void
671+ assert_reread_same(NMConnection *connection, NMConnection *reread)
672+ {
673++ #if WITH_NETPLAN
674++ // Netplan does some normalization already, so compare normalized connections
675++ nm_connection_normalize (connection, NULL, NULL, NULL);
676++ nm_connection_normalize (reread, NULL, NULL, NULL);
677++ #endif
678+ nmtst_assert_connection_verifies_without_normalization(reread);
679+ nmtst_assert_connection_equals(connection, TRUE, reread, FALSE);
680+ }
681+@@ -789,6 +801,10 @@ test_write_wireless_connection(void)
682+ bssid,
683+ NM_SETTING_WIRELESS_SSID,
684+ ssid,
685++ #if WITH_NETPLAN
686++ //XXX: netplan uses explicit "infrastructure" mode
687++ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA,
688++ #endif
689+ NM_SETTING_WIRED_MTU,
690+ 1000,
691+ NULL);
692+@@ -870,7 +886,12 @@ test_write_string_ssid(void)
693+ nm_connection_add_setting(connection, NM_SETTING(s_wireless));
694+
695+ ssid = g_bytes_new(tmpssid, sizeof(tmpssid));
696+- g_object_set(s_wireless, NM_SETTING_WIRELESS_SSID, ssid, NULL);
697++ g_object_set(s_wireless, NM_SETTING_WIRELESS_SSID, ssid,
698++ #if WITH_NETPLAN
699++ //XXX: netplan uses explicit "infrastructure" mode
700++ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA,
701++ #endif
702++ NULL);
703+ g_bytes_unref(ssid);
704+
705+ /* IP4 setting */
706+@@ -953,7 +974,12 @@ test_write_intlist_ssid(void)
707+ nm_connection_add_setting(connection, NM_SETTING(s_wifi));
708+
709+ ssid = g_bytes_new(tmpssid, sizeof(tmpssid));
710+- g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, NULL);
711++ g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid,
712++ #if WITH_NETPLAN
713++ //XXX: netplan uses explicit "infrastructure" mode
714++ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA,
715++ #endif
716++ NULL);
717+ g_bytes_unref(ssid);
718+
719+ /* IP4 setting */
720+@@ -1053,7 +1079,12 @@ test_write_intlike_ssid(void)
721+ nm_connection_add_setting(connection, NM_SETTING(s_wifi));
722+
723+ ssid = g_bytes_new(tmpssid, sizeof(tmpssid));
724+- g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, NULL);
725++ g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid,
726++ #if WITH_NETPLAN
727++ //XXX: netplan uses explicit "infrastructure" mode
728++ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA,
729++ #endif
730++ NULL);
731+ g_bytes_unref(ssid);
732+
733+ /* IP4 setting */
734+@@ -1115,7 +1146,12 @@ test_write_intlike_ssid_2(void)
735+ nm_connection_add_setting(connection, NM_SETTING(s_wifi));
736+
737+ ssid = g_bytes_new(tmpssid, sizeof(tmpssid));
738+- g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, NULL);
739++ g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid,
740++ #if WITH_NETPLAN
741++ //XXX: netplan uses explicit "infrastructure" mode
742++ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA,
743++ #endif
744++ NULL);
745+ g_bytes_unref(ssid);
746+
747+ /* IP4 setting */
748+@@ -2845,12 +2881,29 @@ main(int argc, char **argv)
749+
750+ nmtst_init_assert_logging(&argc, &argv, "INFO", "DEFAULT");
751+
752++ #if WITH_NETPLAN
753++ if (g_mkdir_with_parents(TEST_SCRATCH_DIR_OLD, 0755) != 0) {
754++ errsv = errno;
755++ g_error("failure to create test directory \"%s\": %s",
756++ TEST_SCRATCH_DIR_OLD,
757++ nm_strerror_native(errsv));
758++ }
759++ // Prepare netplan test directories
760++ g_mkdir_with_parents (TEST_SCRATCH_DIR_OLD"/etc/netplan", 0755);
761++ g_mkdir_with_parents (TEST_SCRATCH_DIR_OLD"/run/NetworkManager", 0755);
762++ // link "keyfiles/" to "run/NetworkManager/system-connections"
763++ const gchar *args[] = { "/bin/ln", "-s", TEST_KEYFILES_DIR_OLD, TEST_KEYFILES_DIR, NULL };
764++ g_spawn_sync(NULL, (gchar**)args, NULL, G_SPAWN_DEFAULT, NULL, NULL, NULL, NULL, NULL, NULL);
765++ // clear netplan YAML config from previous runs
766++ g_spawn_command_line_sync("/bin/sh -c 'rm -f " TEST_KEYFILES_DIR_OLD "/etc/netplan/*.yaml'", NULL, NULL, NULL, NULL);
767++ #else
768+ if (g_mkdir_with_parents(TEST_SCRATCH_DIR, 0755) != 0) {
769+ errsv = errno;
770+ g_error("failure to create test directory \"%s\": %s",
771+ TEST_SCRATCH_DIR,
772+ nm_strerror_native(errsv));
773+ }
774++ #endif
775+
776+ /* The tests */
777+ g_test_add_func("/keyfile/test_read_valid_wired_connection", test_read_valid_wired_connection);
778+--
779+2.39.2
780+
781diff --git a/debian/patches/series b/debian/patches/series
782index c2344eb..2e570aa 100644
783--- a/debian/patches/series
784+++ b/debian/patches/series
785@@ -3,3 +3,5 @@ Force-online-state-with-unmanaged-devices.patch
786 # Ubuntu patches
787 Provide-access-to-some-of-NM-s-interfaces-to-whoopsie.patch
788 Update-dnsmasq-parameters.patch
789+netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch
790+netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch
791diff --git a/debian/rules b/debian/rules
792index edcb573..7d336f1 100755
793--- a/debian/rules
794+++ b/debian/rules
795@@ -4,6 +4,7 @@ include /usr/share/dpkg/architecture.mk
796
797 ifeq ($(shell dpkg-vendor --is Ubuntu && echo yes) $(DEB_HOST_ARCH), yes i386)
798 BUILD_PACKAGES += -Nnetwork-manager
799+ NETPLAN += --disable-netplan
800 endif
801
802 # Disable lto here regardless of whether we enable the configure flag
803@@ -61,7 +62,7 @@ override_dh_auto_configure:
804 --enable-lto \
805 --disable-more-warnings \
806 --disable-modify-system \
807- --disable-ovs
808+ --disable-ovs $(NETPLAN)
809
810 override_dh_install:
811 find debian/tmp -name '*.la' -print -delete
812diff --git a/debian/tests/control b/debian/tests/control
813index 795b9ba..1985292 100644
814--- a/debian/tests/control
815+++ b/debian/tests/control
816@@ -13,3 +13,7 @@ Restrictions: needs-root allow-stderr isolation-machine skippable
817 Tests: urfkill-integration
818 Depends: network-manager, build-essential, linux-headers-generic [!i386], rfkill, urfkill
819 Restrictions: needs-root allow-stderr isolation-machine skippable
820+
821+Tests: nm_netplan.py
822+Depends: python3, gir1.2-nm-1.0, network-manager, netplan.io, openvpn, easy-rsa, network-manager-openvpn
823+Restrictions: needs-root allow-stderr isolation-container
824diff --git a/debian/tests/nm.py b/debian/tests/nm.py
825index 47345db..f7a03b7 100755
826--- a/debian/tests/nm.py
827+++ b/debian/tests/nm.py
828@@ -70,6 +70,7 @@ class NetworkManagerTest(network_test_base.NetworkTestBase):
829 "/etc/NetworkManager",
830 "/var/lib/NetworkManager",
831 "/run/NetworkManager",
832+ "/etc/netplan",
833 ]:
834 subprocess.check_call(["mount", "-n", "-t", "tmpfs", "none", d])
835 self.addCleanup(subprocess.call, ["umount", d])
836diff --git a/debian/tests/nm_netplan.py b/debian/tests/nm_netplan.py
837new file mode 100644
838index 0000000..cc86bcf
839--- /dev/null
840+++ b/debian/tests/nm_netplan.py
841@@ -0,0 +1,717 @@
842+#!/usr/bin/python3
843+
844+__author__ = "Danilo Egea Gondolfo <danilo.egea.gondolfo@canonical.com>"
845+__copyright__ = "(C) 2023 Canonical Ltd."
846+__license__ = "GPL v2 or later"
847+
848+from glob import glob
849+import json
850+import shutil
851+import socket
852+import subprocess
853+import sys
854+import os
855+from time import sleep
856+import unittest
857+import yaml
858+
859+import gi
860+gi.require_version("NM", "1.0")
861+from gi.repository import NM, GLib, Gio
862+
863+nmclient = NM.Client.new()
864+
865+class TestNetplan(unittest.TestCase):
866+
867+ def setUp(self):
868+ self._stop_network_manager()
869+
870+ self._start_nm()
871+ sleep(1)
872+ self.nmclient = NM.Client.new()
873+
874+ def tearDown(self):
875+ pass
876+
877+ def _start_nm(self, auto_connect=True):
878+ """This method is basically a copy of start_nm() from nm.py
879+ without the parts we don't need
880+ """
881+
882+ if not os.path.exists("/run/NetworkManager"):
883+ os.mkdir("/run/NetworkManager")
884+ for d in [
885+ "/etc/NetworkManager",
886+ "/var/lib/NetworkManager",
887+ "/run/NetworkManager",
888+ "/etc/netplan",
889+ ]:
890+ subprocess.check_call(["mount", "-n", "-t", "tmpfs", "none", d])
891+ self.addCleanup(subprocess.call, ["umount", d])
892+ os.mkdir("/etc/NetworkManager/system-connections")
893+
894+ denylist = ""
895+ for iface in os.listdir("/sys/class/net"):
896+ if iface in ['bonding_masters']:
897+ continue
898+ with open("/sys/class/net/%s/address" % iface) as f:
899+ if denylist:
900+ denylist += ";"
901+ denylist += "mac:%s" % f.read().strip()
902+
903+ conf = "/etc/NetworkManager/NetworkManager.conf"
904+ extra_main = ""
905+ if not auto_connect:
906+ extra_main += "no-auto-default=*\n"
907+
908+ with open(conf, "w") as f:
909+ f.write(
910+ "[main]\nplugins=keyfile\n%s\n[keyfile]\nunmanaged-devices=%s\n"
911+ % (extra_main, denylist)
912+ )
913+
914+ log = "/tmp/NetworkManager.log"
915+ f_log = os.open(log, os.O_CREAT | os.O_WRONLY | os.O_SYNC)
916+
917+ # build NM command line
918+ argv = ["NetworkManager", "--log-level=debug", "--debug", "--config=" + conf]
919+ # allow specifying extra arguments
920+ argv += os.environ.get("NM_TEST_DAEMON_ARGS", "").strip().split()
921+
922+ p = subprocess.Popen(argv, stdout=f_log, stderr=subprocess.STDOUT)
923+ # automatically terminate process at end of test case
924+ self.addCleanup(p.wait)
925+ self.addCleanup(p.terminate)
926+ self.addCleanup(os.close, f_log)
927+ self.addCleanup(self._clear_connections)
928+
929+ self._process_glib_events()
930+
931+ def _process_glib_events(self):
932+ """Process pending GLib main loop events"""
933+
934+ context = GLib.MainContext.default()
935+ while context.iteration(False):
936+ pass
937+
938+ def _restart_network_manager(self):
939+ cmd = ['systemctl', 'restart', 'NetworkManager']
940+ subprocess.call(cmd, stdout=subprocess.DEVNULL)
941+
942+ def _stop_network_manager(self):
943+ cmd = ['systemctl', 'stop', 'NetworkManager']
944+ subprocess.call(cmd, stdout=subprocess.DEVNULL)
945+
946+ def _add_connection(self, connection):
947+ main_loop = GLib.MainLoop()
948+ def add_cb(client, result, data):
949+ self.nmclient.add_connection_finish(result)
950+ main_loop.quit()
951+
952+ self.nmclient.add_connection_async(connection, True, None, add_cb, None)
953+ main_loop.run()
954+
955+ def _delete_connection(self, connection):
956+ uuid = connection.get_uuid()
957+ cmd = ['nmcli', 'con', 'del', uuid]
958+ subprocess.call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
959+
960+ def _delete_interface(self, connection):
961+ iface = connection.get_interface_name()
962+ if iface:
963+ cmd = ['ip', 'link', 'del', iface]
964+ subprocess.call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
965+
966+ def _nmcli(self, parameters):
967+ cmd = ['nmcli'] + parameters
968+ subprocess.call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
969+
970+ def _bridge_show(self, bridge):
971+ cmd = ['bridge', '-j', 'link', 'show', bridge]
972+ ret = subprocess.run(cmd, capture_output=True)
973+ return json.loads(ret.stdout)
974+
975+ def _load_netplan_yaml_for_connection(self, connection):
976+ filename = '/etc/netplan/90-NM-' + connection.get_uuid() + '.yaml'
977+
978+ file = open(filename)
979+ data = yaml.safe_load(file)
980+ file.close()
981+ return data
982+
983+ def _get_number_of_yaml_files(self):
984+ return len(self._get_list_of_yaml_files())
985+
986+ def _get_list_of_yaml_files(self):
987+ return glob("/etc/netplan/90-NM-*")
988+
989+ def _commit_and_save_connection(self, connection):
990+ main_loop = GLib.MainLoop()
991+
992+ def commit_cb(client, result, data):
993+ connection.commit_changes_finish(result)
994+ main_loop.quit()
995+
996+ connection.commit_changes_async(True, None, commit_cb, None)
997+ main_loop.run()
998+
999+ def _clear_connections(self):
1000+ for conn in self.nmclient.get_connections():
1001+ self._delete_connection(conn)
1002+ self._delete_interface(conn)
1003+
1004+ def _netplan_generate(self):
1005+ cmd = ['netplan', 'generate']
1006+ ret = subprocess.run(cmd, capture_output=True)
1007+
1008+ def _nmcli_con_reload(self):
1009+ self._nmcli(['con', 'reload'])
1010+
1011+ # Tests
1012+
1013+ def test_create_a_simple_bridge_with_dhcp(self):
1014+
1015+ conn = NM.SimpleConnection.new()
1016+ settings = NM.SettingConnection.new()
1017+ settings.set_property(NM.SETTING_CONNECTION_ID, "bridge0")
1018+ settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "bridge0")
1019+ settings.set_property(NM.SETTING_CONNECTION_TYPE, "bridge")
1020+
1021+ bridge = NM.SettingBridge.new()
1022+ ipv4 = NM.SettingIP4Config.new()
1023+ ipv4.set_property(NM.SETTING_IP_CONFIG_METHOD, "auto")
1024+ ipv6 = NM.SettingIP6Config.new()
1025+ ipv6.set_property(NM.SETTING_IP_CONFIG_METHOD, "auto")
1026+
1027+ conn.add_setting(settings)
1028+ conn.add_setting(ipv4)
1029+ conn.add_setting(ipv6)
1030+ conn.add_setting(bridge)
1031+
1032+ # There should be zero netplan NM yaml before adding a connection
1033+ self.assertEqual(self._get_number_of_yaml_files(), 0)
1034+
1035+ self._add_connection(conn)
1036+
1037+ # There should be one netplan NM yaml after adding a connection
1038+ self.assertEqual(self._get_number_of_yaml_files(), 1)
1039+
1040+ connection = self.nmclient.get_connection_by_id("bridge0")
1041+ yaml_data = self._load_netplan_yaml_for_connection(connection)
1042+
1043+ # Validating some of the expected flags
1044+ self.assertTrue(yaml_data['network']['bridges']['bridge0']['dhcp4'])
1045+ self.assertTrue(yaml_data['network']['bridges']['bridge0']['dhcp6'])
1046+
1047+ self._delete_connection(connection)
1048+
1049+ # There should be zero netplan NM yaml after deleting a connection
1050+ self.assertEqual(self._get_number_of_yaml_files(), 0)
1051+
1052+ def test_create_a_simple_bridge_with_ip_addresses(self):
1053+ conn = NM.SimpleConnection.new()
1054+ settings = NM.SettingConnection.new()
1055+ settings.set_property(NM.SETTING_CONNECTION_ID, "bridge0")
1056+ settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "bridge0")
1057+ settings.set_property(NM.SETTING_CONNECTION_TYPE, "bridge")
1058+
1059+ bridge = NM.SettingBridge.new()
1060+
1061+ ipv4 = NM.SettingIP4Config.new()
1062+ ipv4.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
1063+ ip4_addr1 = NM.IPAddress.new(socket.AF_INET, "10.20.30.40", 24)
1064+ ip4_addr2 = NM.IPAddress.new(socket.AF_INET, "10.20.30.41", 24)
1065+ ipv4.add_address(ip4_addr1)
1066+ ipv4.add_address(ip4_addr2)
1067+
1068+ ipv6 = NM.SettingIP6Config.new()
1069+ ipv6.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
1070+ ip6_addr1 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::1", 64)
1071+ ip6_addr2 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::2", 64)
1072+ ipv6.add_address(ip6_addr1)
1073+ ipv6.add_address(ip6_addr2)
1074+
1075+ conn.add_setting(settings)
1076+ conn.add_setting(ipv4)
1077+ conn.add_setting(ipv6)
1078+ conn.add_setting(bridge)
1079+
1080+ # There should be zero netplan NM yaml before adding a connection
1081+ self.assertEqual(self._get_number_of_yaml_files(), 0)
1082+
1083+ self._add_connection(conn)
1084+
1085+ # There should be one netplan NM yaml after adding a connection
1086+ self.assertEqual(self._get_number_of_yaml_files(), 1)
1087+
1088+ connection = self.nmclient.get_connection_by_id("bridge0")
1089+ yaml_data = self._load_netplan_yaml_for_connection(connection)
1090+
1091+ # Validating some of the expected flags
1092+ self.assertNotIn('dhcp4', yaml_data['network']['bridges']['bridge0'])
1093+ self.assertNotIn('dhcp6', yaml_data['network']['bridges']['bridge0'])
1094+
1095+ addresses = yaml_data['network']['bridges']['bridge0']['addresses']
1096+ expected_addresses = ['10.20.30.40/24', '10.20.30.41/24', 'dead:beef::1/64', 'dead:beef::2/64']
1097+
1098+ self.assertListEqual(addresses, expected_addresses)
1099+
1100+ self._delete_connection(connection)
1101+
1102+ # There should be zero netplan NM yaml after deleting a connection
1103+ self.assertEqual(self._get_number_of_yaml_files(), 0)
1104+
1105+ def test_create_a_simple_bridge_with_ip_and_member(self):
1106+
1107+ conn = NM.SimpleConnection.new()
1108+ settings = NM.SettingConnection.new()
1109+ settings.set_property(NM.SETTING_CONNECTION_ID, "bridge0")
1110+ settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "bridge0")
1111+ settings.set_property(NM.SETTING_CONNECTION_TYPE, "bridge")
1112+
1113+ bridge = NM.SettingBridge.new()
1114+
1115+ ipv4 = NM.SettingIP4Config.new()
1116+ ipv4.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
1117+ ip4_addr1 = NM.IPAddress.new(socket.AF_INET, "10.20.30.40", 24)
1118+ ip4_addr2 = NM.IPAddress.new(socket.AF_INET, "10.20.30.41", 24)
1119+ ipv4.add_address(ip4_addr1)
1120+ ipv4.add_address(ip4_addr2)
1121+
1122+ ipv6 = NM.SettingIP6Config.new()
1123+ ipv6.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
1124+ ip6_addr1 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::1", 64)
1125+ ip6_addr2 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::2", 64)
1126+ ipv6.add_address(ip6_addr1)
1127+ ipv6.add_address(ip6_addr2)
1128+
1129+ conn.add_setting(settings)
1130+ conn.add_setting(ipv4)
1131+ conn.add_setting(ipv6)
1132+ conn.add_setting(bridge)
1133+
1134+ # There should be zero netplan NM yaml before adding a connection
1135+ self.assertEqual(self._get_number_of_yaml_files(), 0)
1136+
1137+ # Adding the bridge
1138+ self._add_connection(conn)
1139+
1140+ # There should be one netplan NM yaml after adding a connection
1141+ self.assertEqual(self._get_number_of_yaml_files(), 1)
1142+
1143+ # Creating a tap0 device to be a bridge member
1144+ tap0 = NM.SimpleConnection.new()
1145+ tap0_conn_settings = NM.SettingConnection.new()
1146+ tap0_conn_settings.set_property(NM.SETTING_CONNECTION_ID, "tap0")
1147+ tap0_conn_settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "tap0")
1148+ tap0_conn_settings.set_property(NM.SETTING_CONNECTION_TYPE, "tun")
1149+ tap0_conn_settings.set_property(NM.SETTING_CONNECTION_SLAVE_TYPE, "bridge")
1150+ tap0_conn_settings.set_property(NM.SETTING_CONNECTION_MASTER, "bridge0")
1151+
1152+ tap0_settings = NM.SettingTun.new()
1153+ tap0_settings.set_property(NM.SETTING_TUN_MODE, NM.SettingTunMode.TAP)
1154+
1155+ tap0.add_setting(tap0_conn_settings)
1156+ tap0.add_setting(tap0_settings)
1157+ self._add_connection(tap0)
1158+
1159+ # There should be two netplan NM yaml after adding the tap
1160+ self.assertEqual(self._get_number_of_yaml_files(), 2)
1161+
1162+ bridge0_connection = self.nmclient.get_connection_by_id("bridge0")
1163+ yaml_data = self._load_netplan_yaml_for_connection(bridge0_connection)
1164+
1165+ # Validating some of the bridge expected flags
1166+ addresses = yaml_data['network']['bridges']['bridge0']['addresses']
1167+ expected_addresses = ['10.20.30.40/24', '10.20.30.41/24', 'dead:beef::1/64', 'dead:beef::2/64']
1168+
1169+ self.assertListEqual(addresses, expected_addresses)
1170+
1171+ # Validating if tap0 is attached to the bridge
1172+ # It might take a while for the new interface be created...
1173+ limit = 10
1174+ while (show_bridge := self._bridge_show('bridge0')) == [] and limit > 0:
1175+ sleep(1)
1176+ limit = limit - 1
1177+ self.assertEqual(show_bridge[0]['master'], 'bridge0')
1178+ self.assertEqual(show_bridge[0]['ifname'], 'tap0')
1179+
1180+ tap0_connection = self.nmclient.get_connection_by_id("tap0")
1181+ tap_yaml = self._load_netplan_yaml_for_connection(tap0_connection)
1182+
1183+ tap0_uuid = tap0_connection.get_uuid()
1184+ # Validating that the bridge information is in the tap yaml
1185+ self.assertEqual(tap_yaml['network']['nm-devices']['NM-' + tap0_uuid]['networkmanager']['passthrough']['connection.master'], 'bridge0')
1186+
1187+ self._delete_connection(tap0_connection)
1188+ # There should be one netplan NM yaml after deleting the tap
1189+ self.assertEqual(self._get_number_of_yaml_files(), 1)
1190+
1191+ self._delete_connection(bridge0_connection)
1192+ # There should be zero netplan NM yaml after deleting the bridge
1193+ self.assertEqual(self._get_number_of_yaml_files(), 0)
1194+
1195+ @unittest.skip('XXX: the netplan yaml file will be deleted when we try to change the connetion')
1196+ def test_create_an_interface_and_change_it(self):
1197+ """Add a tap interface and change it after create adding IP addresses to it."""
1198+
1199+ # Creating a tap0 device to be a bridge member
1200+ tap0 = NM.SimpleConnection.new()
1201+ tap0_conn_settings = NM.SettingConnection.new()
1202+ tap0_conn_settings.set_property(NM.SETTING_CONNECTION_ID, "tap0")
1203+ tap0_conn_settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "tap0")
1204+ tap0_conn_settings.set_property(NM.SETTING_CONNECTION_TYPE, "tun")
1205+
1206+ tap0_settings = NM.SettingTun.new()
1207+ tap0_settings.set_property(NM.SETTING_TUN_MODE, NM.SettingTunMode.TAP)
1208+
1209+ tap0.add_setting(tap0_conn_settings)
1210+ tap0.add_setting(tap0_settings)
1211+ self._add_connection(tap0)
1212+
1213+ # There should be one netplan NM yaml after adding a connection
1214+ self.assertEqual(self._get_number_of_yaml_files(), 1)
1215+
1216+ tap0_connection = self.nmclient.get_connection_by_id("tap0")
1217+
1218+ ipv4_settings = tap0_connection.get_setting_ip4_config()
1219+ ipv4_settings.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
1220+ ip4_addr1 = NM.IPAddress.new(socket.AF_INET, "10.20.30.40", 24)
1221+ ip4_addr2 = NM.IPAddress.new(socket.AF_INET, "10.20.30.41", 24)
1222+ ipv4_settings.add_address(ip4_addr1)
1223+ ipv4_settings.add_address(ip4_addr2)
1224+
1225+ ipv6_settings = tap0_connection.get_setting_ip6_config()
1226+ ipv6_settings.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
1227+ ip6_addr1 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::1", 64)
1228+ ip6_addr2 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::2", 64)
1229+ ipv6_settings.add_address(ip6_addr1)
1230+ ipv6_settings.add_address(ip6_addr2)
1231+
1232+ self._commit_and_save_connection(tap0_connection)
1233+
1234+ # There should be one netplan NM yaml files after chaging the only existing connection
1235+ self.assertEqual(self._get_number_of_yaml_files(), 1)
1236+
1237+ self._delete_connection(tap0_connection)
1238+
1239+ # There should be zero netplan NM yaml files after removing the only existing connection
1240+ self.assertEqual(self._get_number_of_yaml_files(), 0)
1241+
1242+ def test_nmcli_add_device_and_change_it(self):
1243+ """Uses the nmcli to add a connection and validates if the
1244+ Netplan YAML file has the expected configuration.
1245+ It also changes the configuration changing the ipv4 method from auto
1246+ to manual and adding an IP address. After the change the Netplan YAML
1247+ file should have the same name.
1248+ """
1249+
1250+ self.assertEqual(self._get_number_of_yaml_files(), 0)
1251+
1252+ nmcli_add = ['con', 'add', 'type', 'tun', 'mode', 'tap', 'ifname', 'tap0', 'ipv4.method', 'auto']
1253+ self._nmcli(nmcli_add)
1254+
1255+ self.assertEqual(self._get_number_of_yaml_files(), 1)
1256+
1257+ files_before = self._get_list_of_yaml_files()
1258+
1259+ # After OOB changes, the client must be refreshed apparently
1260+ self.nmclient = NM.Client.new()
1261+
1262+ conn = self.nmclient.get_connection_by_id("tun-tap0")
1263+ uuid = conn.get_uuid()
1264+ data = self._load_netplan_yaml_for_connection(conn)
1265+
1266+ ip4_method = data.get('network') \
1267+ .get('nm-devices') \
1268+ .get('NM-' + uuid) \
1269+ .get('networkmanager') \
1270+ .get('passthrough') \
1271+ .get('ipv4.method')
1272+
1273+ self.assertEqual(ip4_method, 'auto')
1274+
1275+ nmcli_mod = ['con', 'mod', 'tun-tap0', 'ipv4.method', 'manual', 'ipv4.addresses', '10.20.30.40/24']
1276+ self._nmcli(nmcli_mod)
1277+
1278+ self.assertEqual(self._get_number_of_yaml_files(), 1)
1279+
1280+ files_after = self._get_list_of_yaml_files()
1281+
1282+ self.assertListEqual(files_before, files_after)
1283+
1284+ data = self._load_netplan_yaml_for_connection(conn)
1285+
1286+ ip4_method = data.get('network') \
1287+ .get('nm-devices') \
1288+ .get('NM-' + uuid) \
1289+ .get('networkmanager') \
1290+ .get('passthrough') \
1291+ .get('ipv4.method')
1292+
1293+ ip4_addr = data.get('network') \
1294+ .get('nm-devices') \
1295+ .get('NM-' + uuid) \
1296+ .get('networkmanager') \
1297+ .get('passthrough') \
1298+ .get('ipv4.address1')
1299+
1300+ self.assertEqual(ip4_method, 'manual')
1301+ self.assertEqual(ip4_addr, '10.20.30.40/24')
1302+
1303+ self._delete_connection(conn)
1304+ self.assertEqual(self._get_number_of_yaml_files(), 0)
1305+
1306+ def test_nmcli_add_wifi_connection(self):
1307+ """Create a wifi connection via nmcli and check if the expected
1308+ fields were added to the netplan yaml file."""
1309+
1310+ ssid = 'My network SSID'
1311+ passwd = 'secretpasswd'
1312+ method = 'wpa-psk'
1313+ ip = '10.20.30.40/24'
1314+ nmcli_add = ['con', 'add', 'type', 'wifi', 'ssid', ssid,
1315+ 'wifi-sec.key-mgmt', method, 'wifi-sec.psk', passwd,
1316+ 'ipv4.method', 'manual', 'ipv4.addresses', ip]
1317+
1318+ self.assertEqual(self._get_number_of_yaml_files(), 0)
1319+
1320+ self._nmcli(nmcli_add)
1321+
1322+ self.assertEqual(self._get_number_of_yaml_files(), 1)
1323+
1324+ # After OOB changes, the client must be refreshed apparently
1325+ self.nmclient = NM.Client.new()
1326+
1327+ conn = self.nmclient.get_connection_by_id("wifi")
1328+ uuid = conn.get_uuid()
1329+ data = self._load_netplan_yaml_for_connection(conn)
1330+
1331+ ap_name = list(data.get('network') \
1332+ .get('wifis') \
1333+ .get('NM-' + uuid) \
1334+ .get('access-points') \
1335+ .keys())[0]
1336+
1337+ ip_addr = data.get('network') \
1338+ .get('wifis') \
1339+ .get('NM-' + uuid) \
1340+ .get('addresses')
1341+
1342+ auth_method = data.get('network') \
1343+ .get('wifis') \
1344+ .get('NM-' + uuid) \
1345+ .get('access-points') \
1346+ .get(ap_name) \
1347+ .get('auth') \
1348+ .get('key-management')
1349+
1350+ auth_passwd = data.get('network') \
1351+ .get('wifis') \
1352+ .get('NM-' + uuid) \
1353+ .get('access-points') \
1354+ .get(ap_name) \
1355+ .get('auth') \
1356+ .get('password')
1357+
1358+ self.assertEqual(ap_name, ssid)
1359+ self.assertListEqual(ip_addr, [ip])
1360+ self.assertEqual(auth_method, 'psk')
1361+ self.assertEqual(auth_passwd, passwd)
1362+
1363+ self._delete_connection(conn)
1364+ self.assertEqual(self._get_number_of_yaml_files(), 0)
1365+
1366+ def test_create_connection_via_netplan(self):
1367+ """
1368+ Create a connection via netplan generate and check if NM will pick it up
1369+ """
1370+
1371+ netplan_yaml = '''network:
1372+ renderer: NetworkManager
1373+ ethernets:
1374+ eth123:
1375+ dhcp4: true'''
1376+
1377+ with open('/etc/netplan/10-test.yaml', 'w') as f:
1378+ f.write(netplan_yaml)
1379+
1380+ self._netplan_generate();
1381+ self._nmcli_con_reload()
1382+ self.nmclient = NM.Client.new()
1383+
1384+ expected = None
1385+ for conn in self.nmclient.get_connections():
1386+ if conn.get_id() == 'netplan-eth123':
1387+ expected = conn
1388+
1389+ self.assertIsNotNone(expected)
1390+
1391+ def test_create_connection_via_netplan_and_remove_via_nmcli(self):
1392+ """
1393+ Create a connection via netplan generate and remove it with nmcli.
1394+
1395+ The interface should be removed from the yaml file.
1396+ """
1397+
1398+ netplan_yaml = '''network:
1399+ renderer: NetworkManager
1400+ ethernets:
1401+ eth123:
1402+ dhcp4: true
1403+ eth456:
1404+ dhcp4: true'''
1405+
1406+ with open('/etc/netplan/10-test.yaml', 'w') as f:
1407+ f.write(netplan_yaml)
1408+
1409+ self._netplan_generate();
1410+ self._nmcli_con_reload()
1411+ self.nmclient = NM.Client.new()
1412+
1413+ expected1 = None
1414+ expected2 = None
1415+ for conn in self.nmclient.get_connections():
1416+ if conn.get_id() == 'netplan-eth123':
1417+ expected1 = conn
1418+ if conn.get_id() == 'netplan-eth456':
1419+ expected2 = conn
1420+
1421+ self.assertIsNotNone(expected1)
1422+ self.assertIsNotNone(expected1)
1423+
1424+ self._delete_connection(expected1)
1425+
1426+ with open('/etc/netplan/10-test.yaml', 'r') as f:
1427+ yaml_data = yaml.safe_load(f)
1428+
1429+ # eth123 shouldn't exist anymore
1430+ self.assertIsNone(yaml_data.get('network').get('ethernets').get('eth123'))
1431+
1432+ self.assertIsNotNone(yaml_data.get('network').get('ethernets').get('eth456'))
1433+
1434+ def test_create_connection_via_netplan_and_change_it_via_nmcli(self):
1435+ """
1436+ Create a connection via netplan generate and change it via nmcli.
1437+ """
1438+
1439+ netplan_yaml = '''network:
1440+ renderer: NetworkManager
1441+ ethernets:
1442+ eth123:
1443+ dhcp4: false
1444+ dhcp6: false
1445+ eth456:
1446+ dhcp4: true'''
1447+
1448+ with open('/etc/netplan/10-test.yaml', 'w') as f:
1449+ f.write(netplan_yaml)
1450+
1451+ self._netplan_generate();
1452+ self._nmcli_con_reload()
1453+ self._nmcli(['con', 'mod', 'netplan-eth123', 'ipv4.method', 'auto'])
1454+
1455+ # eth123.dhcp4 should be overriden by 90-NM-<UUID>.yaml (from 10-test.yaml)
1456+ # The output of 'netplan get' should account for that.
1457+ out = subprocess.check_output(['netplan', 'get'], universal_newlines=True)
1458+ yaml_data = yaml.safe_load(out)
1459+ dhcp = yaml_data.get('network').get('ethernets').get('eth123').get('dhcp4')
1460+ self.assertTrue(dhcp)
1461+
1462+ def test_openvpn_connection(self):
1463+ """ Test case for LP#1998207"""
1464+
1465+ server_config = """dev tun
1466+ca /tmp/openvpn/pki/ca.crt
1467+cert /tmp/openvpn/pki/issued/server.crt
1468+key /tmp/openvpn/pki/private/server.key
1469+dh /tmp/openvpn/pki/dh.pem
1470+server 10.8.0.0 255.255.255.0
1471+keepalive 10 120
1472+cipher AES-256-GCM
1473+compress lz4-v2
1474+push "compress lz4-v2"
1475+user root
1476+log /tmp/openvpn.log
1477+group root
1478+"""
1479+
1480+ client_config = """client
1481+dev tun
1482+remote 127.0.0.1 1194
1483+nobind
1484+ca /tmp/openvpn/pki/ca.crt
1485+cert /tmp/openvpn/pki/issued/client.crt
1486+key /tmp/openvpn/pki/private/client.key
1487+cipher AES-256-GCM
1488+"""
1489+
1490+ # The minimum DH size accepted by OpenVPN these days is 2048.
1491+ # It might take a while to be generated (like almost a minute)
1492+ # It would be faster to use shared keys instead of TLS but it
1493+ # seems it's not an option anymore in OpenVPN
1494+ openvpn_spinup_script = """/usr/share/easy-rsa/easyrsa init-pki
1495+EASYRSA_BATCH=1 /usr/share/easy-rsa/easyrsa build-ca nopass
1496+EASYRSA_BATCH=1 /usr/share/easy-rsa/easyrsa build-server-full server nopass
1497+EASYRSA_BATCH=1 /usr/share/easy-rsa/easyrsa build-client-full client nopass
1498+/usr/share/easy-rsa/easyrsa gen-dh
1499+"""
1500+
1501+ tmpdir = '/tmp/openvpn'
1502+ self.addCleanup(shutil.rmtree, tmpdir)
1503+ os.mkdir(tmpdir)
1504+ os.chdir(tmpdir)
1505+
1506+ with open('openvpn_spinup.sh', 'w') as f:
1507+ f.write(openvpn_spinup_script)
1508+
1509+ with open('server.conf', 'w') as f:
1510+ f.write(server_config)
1511+
1512+ with open('client.conf', 'w') as f:
1513+ f.write(client_config)
1514+
1515+ cmd = ['bash', 'openvpn_spinup.sh']
1516+ subprocess.call(cmd, stdout=subprocess.DEVNULL)
1517+
1518+ openvpn_server_cmd = ['openvpn', '--config', 'server.conf']
1519+ p_server = subprocess.Popen(openvpn_server_cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
1520+ sleep(1) # Let's give OpenVPN a second to start
1521+
1522+ # Add a useless default route to the loopback interface so NM will allow starting the VPN connection.
1523+ # Apparently, if it doesn't own an *active connection* with a default route, it will not allow
1524+ # us to start the VPN client.
1525+ # As we set the main ethernet device as unmanaged, NM will not 'own' a default route when it starts.
1526+ # Unfortunately, by doing this, NM will take over the interface 'lo' and add a new yaml file to /etc/netplan
1527+ self._nmcli(['con', 'mod', 'lo', 'ipv4.gateway', '10.8.0.254'])
1528+
1529+ # Create an OpenVPN connection based on the configuration found in client.conf
1530+ self._nmcli(['con', 'import', 'type', 'openvpn', 'file', 'client.conf'])
1531+
1532+ # At this point we should have 2 yaml files
1533+ # One for the tun0 NM took over and one for the VPN connection
1534+ self.assertEqual(self._get_number_of_yaml_files(), 2,
1535+ msg='More than expected YAML files were found after creating the connection')
1536+
1537+ self._nmcli(['con', 'up', 'client'])
1538+ sleep(2) # Let's give NM a couple of seconds to settle down
1539+ # We still should have 2 files after starting the client
1540+ self.assertEqual(self._get_number_of_yaml_files(), 2,
1541+ msg='More than expected YAML files were found after starting the connection')
1542+
1543+ self._nmcli(['con', 'down', 'client'])
1544+ sleep(2) # Let's give NM a couple of seconds to settle down
1545+ # We still should have 2 files after stopping the client
1546+ self.assertEqual(self._get_number_of_yaml_files(), 2,
1547+ msg='More than expected YAML files were found after stopping the connection')
1548+
1549+ p_server.terminate()
1550+ p_server.wait()
1551+ # We still should have 2 files after stopping the server
1552+ self.assertEqual(self._get_number_of_yaml_files(), 2,
1553+ msg='More than expected YAML files were found after stopping the OpenVPN server')
1554+
1555+
1556+if __name__ == '__main__':
1557+ runner = unittest.TextTestRunner(stream=sys.stdout, verbosity=2)
1558+ unittest.main(testRunner=runner)

Subscribers

People subscribed via source and target branches

to all changes: