NetworkManager

Merge ~slyon/network-manager:netplan-integration into network-manager:ubuntu/master

  1. Git
  2. lp:~slyon/network-manager
  3. netplan-integration
  4. Merge into ubuntu/master
Proposed by Lukas Märdian
Status: Merged
Merged at revision: bcd43413ecea3493bfbcf091b95508a4120ba7d3
Proposed branch: ~slyon/network-manager:netplan-integration
Merge into: network-manager:ubuntu/master
Diff against target: 1558 lines (+1454/-1)
11 files modified
debian/changelog (+14/-0)
debian/control (+4/-0)
debian/network-manager.postinst (+33/-0)
debian/network-manager.preinst (+18/-0)
debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch (+134/-0)
debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch (+525/-0)
debian/patches/series (+2/-0)
debian/rules (+2/-1)
debian/tests/control (+4/-0)
debian/tests/nm.py (+1/-0)
debian/tests/nm_netplan.py (+717/-0)
Reviewer Review Type Date Requested Status
Sebastien Bacher Approve
Danilo Egea Gondolfo (community) Approve
Lukas Märdian Abstain
Review via email: mp+423735@code.launchpad.net

Description of the change

Deploying the NetworkManager-Netplan integration (a.k.a "Netplan everywhere") on Ubuntu Desktop by default.

This depends on libnetplan >= v0.106 and we cannot depend on libnetplan on i386, so we're dropping the patch to the NetworkManager daemon binary on i386, using quilt (that binary isn't built on i386 anyway).

On installation of the package any existing configuration from /etc/NetworkManager/system-connections will be backed up in /root/NetworkManaker.bak

On updade of the package, any existing configuration from /etc/NetworkManager/system-connections will be migrated to /etc/netplan and translated into corresponding YAML files. Missing settings will be handled usinng the "passthrough" and "nm-devices" fallback mechanism.

Netplan patches are managed via git-buildpackage patch-queue (Using "Gbp-Pq: Topic netplan")

To post a comment you must log in.
Revision history for this message
Olivier Gayot (ogayot) wrote :

Thanks! A few comments inline.

Revision history for this message
Danilo Egea Gondolfo (danilogondolfo) wrote :

Left one comment inline about the migration script.

review: Needs Fixing
Revision history for this message
Sebastien Bacher (seb128) wrote :

Confirming the issue raised by Danilo, the script failed to migrate several connections using a ca-cert and pointing to a filename stored in directory with a 'é' in its name

it also displayed some warnings (translated from french so the wording might not be the correct english one)

> Warning: there is another connection with the name '...'. Refer to the connection using its uuid '...'

review: Needs Fixing
Revision history for this message
Sebastien Bacher (seb128) wrote :

Unsure if that's specific to the change there or a new mantic issue but the autopkgtest against a ppa testbuild failed

https://autopkgtest.ubuntu.com/results/autopkgtest-mantic-ubuntu-desktop-transitions/mantic/amd64/n/network-manager/20230508_144934_e64d7@/log.gz

'Error: failed to reload connections: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: Object does not exist at path “/org/freedesktop/NetworkManager/Settings”.
dpkg: error processing package network-manager (--configure):
 installed network-manager package post-installation script subprocess returned error exit status 1'

Revision history for this message
Lukas Märdian (slyon) wrote :

Thanks!

I've fixed the utf-8 (non-ASCII keyfiles) issue reported by @seb128 and @danilogondolfo as suggested by Danilo (i.e. checking for the "text/plain" mime type instead).

Thanks Sebastien for the "Error: failed to reload connections: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod" error, that's a good catch! It didn't happen on my Lunar autopkgtests, but I could reproduce it on Mantic, too. Seems to be some kind of race condition when debhelper tries to re-start NetworkManager.service after upgrade and then the postinst script is calling into "nmcli", while NM is not yet fully up. I've added a small "nm-online -s" line in there to make it wait for NM, which now passes the autopkgtests on Mantic again:
```
autopkgtest [15:07:36]: @@@@@@@@@@@@@@@@@@@@ summary
wpa-dhclient PASS
nm.py PASS
killswitches-no-urfkill PASS
urfkill-integration PASS
nm_netplan.py PASS
qemu-system-x86_64: terminating on signal 15 from pid 1944756 (/usr/bin/python3)
```

For the "Warning: there is another connection with the name '...'. Refer to the connection using its uuid '...'" message, I don't really know where it comes from, as we're only modifying connections through nmcli, using their UUID identifiers... Could you try to provide a reproducer for that?

PTAL.

Revision history for this message
Sebastien Bacher (seb128) wrote :

Thanks for the tweaks, I restored the connections from the backup dir, tried to the new revision and now hit

+ echo Migrating Ziggo (03c8f2a7-268d-4765-b626-efcc02dd686c) to /etc/netplan
Migrating Ziggo (03c8f2a7-268d-4765-b626-efcc02dd686c) to /etc/netplan
+ nmcli con mod 03c8f2a7-268d-4765-b626-efcc02dd686c con-name Ziggo.NETPLAN_MIGRATE
Erreur : la modification de la connexion « Ziggo.NETPLAN_MIGRATE » a échoué : Remote peer disconnected
dpkg: erreur de traitement du paquet network-manager (--install) :
 le sous-processus paquet network-manager script post-installation installé a renvoyé un état de sortie d'erreur 1

there is a NetworkManager crash report from that time in /var/crash, it hit the
nms-keyfile-writer.c assert on l538 which is part of this changeset, the journal includes

> <error> [1683703791.7570] BUG: the profile cannot be stored in keyfile format without becoming unusable: invalid connection: 802-1x.identity: la propriété est manquante
and
> <warn> [1683703795.8560] keyfile: load: "/run/NetworkManager/system-connections/netplan-NM-03c8f2a7-268d-4765-b626-efcc02dd686c-Ziggo.nmconnection": failed to load connection: invalid connection: 802-1x.identity: la propriété est manquante

I'm sending the corresponding file via email

So it seems we have a bug there that n-m is hitting an assertion, also it feels wrong than failing to migrate one connection is failing the package installation and letting the packaging system in a state where most users will not be able to recover from easily

review: Needs Fixing
Revision history for this message
Lukas Märdian (slyon) wrote :

Thank you very much for the valuable feedback! I've rebased the branch to properly include the current fixes and moved forward to a v4 Netplan-integration patch, which makes use of "#if WITH_NETPLAN" inside the NM codebase, to allow "--enable-netplan"/"--disable-netplan" buildflags (enabled by default), so we can easily disable it on i386 and don't need that quilt hack anymore.

Also, thanks for providing the test files by email. I agree we shouldn't fail the installation of the package when the migration of a single connection profile fails, it should just skip that connection and keep it as a keyfile. I'll be working on that!

I'll also investigate that crash, but I think we should not block on that one (once we are able to skip non-migrated connections). I'll try to collect all the information and create a corresponding bug report.

Revision history for this message
Lukas Märdian (slyon) wrote :

So Danilo found the root cause of the "802-1x.identity" issue (crash), which is a bug in libnetplan (Netplan's keyfile parser). This will be handled via in bug #2016625 (details to follow there)

Revision history for this message
Lukas Märdian (slyon) wrote :

I've just pushed another update to the branch, which handles migration failures gracefully:
Before migration it will create a temporary backup of the keyfile in and restore that keyfile backup, should the migration fail.

We're working on the crash (libnetplan bug) separately, as mentioned above. I've pushed the current state of this branch into the "Netplan Everywhere" PPA via https://code.launchpad.net/~canonical-foundations/+recipe/network-manager-netplan-lunar-gu

Let me know what you think!

Revision history for this message
Sebastien Bacher (seb128) wrote :

Thanks, I can confirm that the new changes make the package install without error and that it rollbacks the problematic connection. The autopkgtests issue is also resolve

The warning I mentioned earlier about the exiting name/refer by uid is printing by

> nmcli con mod daffb72f-c980-4ede-95cf-31fb7d24484d con-name Hotspot

There is another connection with the same name. The command is correctly issued with the uid so I think the warning is safe to ignore

I will upload the changes to mantic today unless you would prefer to see the netplan fix landing first?

review: Approve
Revision history for this message
Danilo Egea Gondolfo (danilogondolfo) :
Revision history for this message
Danilo Egea Gondolfo (danilogondolfo) wrote :

I left an inline comment about some code we probably want to remove from the #if #endif block.

Revision history for this message
Lukas Märdian (slyon) wrote :

Thank you for the additional reviews!

@seb128, please go ahead and upload the changes into Mantic, we'll be fixing (lib-)netplan in parallel.

@danilogondolfo, you're right it is NetworkManager code. But in fact we're not patching it out, but rather creating a copy of that block for the Netplan codepath. So we should be safe ignoring that whole block inside #if WITH_NETPLAN ... #endif if the Netplan buildflag is disabled. It is visible in the GitHub diff, see: https://github.com/slyon/NetworkManager/commit/aaa791cb61b24a9416db18f2290f711bc9a8f26e#diff-e46473e1ac3ce2315efbd5fbad59951998e274ca21244f6b9e807f20e67db58aL344

review: Abstain
Revision history for this message
Danilo Egea Gondolfo (danilogondolfo) :
review: Approve
Revision history for this message
Sebastien Bacher (seb128) wrote :

Thanks, merged and uploaded to Ubuntu now!

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/debian/changelog b/debian/changelog
index f362bb1..e2e79df 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,17 @@
1network-manager (1.42.4-1ubuntu3) mantic; urgency=medium
2
3 [ Lukas Märdian ]
4 * d/{control,rules,patches}: Prepare libnetplan build (non i386)
5 * debian/patches/netplan: Add libnetplan backend integration patch
6 * d/network-manager.preinst: backup previous configuration automatically
7 * d/network-manager.postinst: Trigger Netplan migration on install/upgrade
8
9 [ Danilo Egea Gondolfo ]
10 * d/t/nm.py: Fix autopkgtests when Netplan is in use
11 * d/t/nm_netplan.py: Add autopkgtests for the netplan integration
12
13 -- Lukas Märdian <slyon@ubuntu.com> Wed, 10 May 2023 11:56:58 +0200
14
1network-manager (1.42.4-1ubuntu2) lunar; urgency=medium15network-manager (1.42.4-1ubuntu2) lunar; urgency=medium
216
3 * d/t/nm.py: Fix autopkgtests with NM-1.42's 'lo' connection (LP: #2009543)17 * d/t/nm.py: Fix autopkgtests with NM-1.42's 'lo' connection (LP: #2009543)
diff --git a/debian/control b/debian/control
index 603370d..dd8bf9d 100644
--- a/debian/control
+++ b/debian/control
@@ -38,6 +38,9 @@ Build-Depends: debhelper-compat (= 13),
38 python3-dbus <!nocheck>,38 python3-dbus <!nocheck>,
39 python3-pexpect <!nocheck>,39 python3-pexpect <!nocheck>,
40 iproute2 <!nocheck>,40 iproute2 <!nocheck>,
41 libyaml-dev [!i386],
42 libnetplan-dev (>= 0.106~) [!i386],
43 netplan.io (>= 0.106~) [!i386] <!nocheck>,
41Standards-Version: 4.6.244Standards-Version: 4.6.2
42Rules-Requires-Root: no45Rules-Requires-Root: no
43XS-Debian-Vcs-Git: https://salsa.debian.org/utopia-team/network-manager.git46XS-Debian-Vcs-Git: https://salsa.debian.org/utopia-team/network-manager.git
@@ -51,6 +54,7 @@ Architecture: linux-any
51Pre-Depends: ${misc:Pre-Depends}54Pre-Depends: ${misc:Pre-Depends}
52Depends: ${shlibs:Depends},55Depends: ${shlibs:Depends},
53 ${misc:Depends},56 ${misc:Depends},
57 netplan.io (>= 0.106~) [!i386],
54 libnm0 (= ${binary:Version}),58 libnm0 (= ${binary:Version}),
55 isc-dhcp-client,59 isc-dhcp-client,
56 default-dbus-system-bus | dbus-system-bus,60 default-dbus-system-bus | dbus-system-bus,
diff --git a/debian/network-manager.postinst b/debian/network-manager.postinst
index e2cb049..0db377b 100644
--- a/debian/network-manager.postinst
+++ b/debian/network-manager.postinst
@@ -81,3 +81,36 @@ esac
8181
82#DEBHELPER#82#DEBHELPER#
8383
84# Run "Netplan Everywhere" migration after debhelper (re-)started
85# NetworkManager.service for us. On every package upgrade.
86DIR="/etc/NetworkManager/system-connections"
87if [ "$1" = "configure" ] && [ -d "$DIR" ]; then
88 mkdir -p /run/netplan/nm-migrate
89 for CON in /etc/NetworkManager/system-connections/*; do
90 TYPE=$(file -bi "$CON" | cut -s -d ";" -f 1)
91 [ "$TYPE" = "text/plain" ] || continue # skip non-keyfiles
92 UUID=$(grep "^uuid=" "$CON" | cut -c 6-)
93 if [ -n "$UUID" ]
94 then
95 # Wait for NetworkManager startup to complete,
96 # so we can safely use nmcli. Wait in every interation to handle
97 # a crashed NetworkManager in the previous migraiton step.
98 nm-online -qs || (echo "SKIP: NetworkManager is not ready ..." 1>&2 && continue)
99 BACKUP="/run/netplan/nm-migrate/"$(basename "$CON")
100 cp "$CON" "$BACKUP"
101 ORIG_NAME=$(nmcli --get-values connection.id con show "$UUID")
102 echo "Migrating $ORIG_NAME ($UUID) to /etc/netplan" 1>&2
103 # Touch the connection's ID (con-name) to trigger its migration.
104 # The Netplan integration will translate the original NM keyfile from
105 # /etc/NetworkManager/system-connections/* to a YAML file located in
106 # /etc/netplan/90-NM-*.yaml and re-generate a corresponding keyfile in
107 # /run/NetworkManager/system-connections/netplan-NM-*.nmconnection
108 nmcli con mod "$UUID" con-name "$ORIG_NAME" || \
109 (echo "FAILED. Restoring backup ..." 1>&2 && mv "$BACKUP" "$CON" && \
110 rm -f "/etc/netplan/90-NM-$UUID"*.yaml)
111 rm -f "$BACKUP" # clear backup (if it still exists)
112 fi
113 done
114 rm -rf /run/netplan/nm-migrate # cleanup after ourselves
115 (nm-online -qs && nmcli con reload) || echo "WARNING: NetworkManager could not reload connections ..." 1>&2
116fi
diff --git a/debian/network-manager.preinst b/debian/network-manager.preinst
84new file mode 100644117new file mode 100644
index 0000000..886ab77
--- /dev/null
+++ b/debian/network-manager.preinst
@@ -0,0 +1,18 @@
1#!/bin/sh
2
3set -e
4
5#DEBHELPER#
6
7DIR="/etc/NetworkManager/system-connections"
8CNT=$(ls -1 "$DIR" | wc -l)
9if ([ "$1" = "upgrade" ] || [ "$1" = "install" ]) && [ -d "$DIR" ] && [ "$CNT" -ge 1 ]; then
10 # create backup directory if it does not yet exist
11 mkdir -p /root/NetworkManager.bak || true
12 BAK="/root/NetworkManager.bak/system-connections_$2"
13 if [ -d "$BAK" ]; then
14 rm -r "$BAK"
15 fi
16 # copy current system-connections to the backup directory
17 cp -r "$DIR" "$BAK"
18fi
diff --git a/debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch b/debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch
0new file mode 10064419new file mode 100644
index 0000000..f3ae212
--- /dev/null
+++ b/debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch
@@ -0,0 +1,134 @@
1From 9dd31963bd89ccba5e96f26506996e792177bdab Mon Sep 17 00:00:00 2001
2From: Lukas Märdian <slyon@ubuntu.com>
3Date: Tue, 9 May 2023 17:09:48 +0200
4Subject: [PATCH 1/2] netplan: Adopt buildsystems for Netplan integration
5
6Autotools and Meson will define a "WITH_NETPLAN" variable with the
7values 1 or 0 accordingly, using the config.h header.
8---
9 Makefile.am | 4 +++-
10 configure.ac | 17 +++++++++++++++++
11 meson.build | 10 ++++++++++
12 meson_options.txt | 1 +
13 src/core/meson.build | 1 +
14 5 files changed, 32 insertions(+), 1 deletion(-)
15
16diff --git a/Makefile.am b/Makefile.am
17index a452cacac4..677c078655 100644
18--- a/Makefile.am
19+++ b/Makefile.am
20@@ -2588,7 +2588,8 @@ $(src_core_libNetworkManagerBase_la_OBJECTS): $(src_libnm_core_public_mkenums_h)
21
22 ###############################################################################
23
24-src_core_libNetworkManager_la_CPPFLAGS = $(src_core_cppflags)
25+src_core_libNetworkManager_la_CPPFLAGS = $(src_core_cppflags) \
26+ $(NETPLAN_CFLAGS)
27
28 src_core_libNetworkManager_la_SOURCES = \
29 \
30@@ -2803,6 +2804,7 @@ src_core_libNetworkManager_la_LIBADD = \
31 $(LIBAUDIT_LIBS) \
32 $(LIBPSL_LIBS) \
33 $(LIBCURL_LIBS) \
34+ $(NETPLAN_LIBS) \
35 $(NULL)
36
37 $(src_core_libNetworkManager_la_OBJECTS): $(src_libnm_core_public_mkenums_h)
38diff --git a/configure.ac b/configure.ac
39index b0cf78dc03..6b4c1467b6 100644
40--- a/configure.ac
41+++ b/configure.ac
42@@ -880,6 +880,22 @@ else
43 AC_DEFINE(WITH_OPENVSWITCH, 0, [Whether we build with OVS plugin])
44 fi
45
46+# Netplan integration
47+AC_ARG_ENABLE(netplan, AS_HELP_STRING([--enable-netplan], [Enable Netplan integration]))
48+if test "${enable_netplan}" != "no"; then
49+ PKG_CHECK_MODULES(NETPLAN, [netplan >= 0.106], [enable_netplan=yes], [enable_netplan=no])
50+ if test "$enable_netplan" != "yes"; then
51+ AC_MSG_ERROR(Netplan is required)
52+ fi
53+ enable_netplan='yes'
54+fi
55+AM_CONDITIONAL(WITH_NETPLAN, test "${enable_netplan}" = "yes")
56+if test "${enable_netplan}" = "yes" ; then
57+ AC_DEFINE(WITH_NETPLAN, 1, [Whether we build with Netplan integration])
58+else
59+ AC_DEFINE(WITH_NETPLAN, 0, [Whether we build with Netplan integration])
60+fi
61+
62 # DHCP client support
63 AC_ARG_WITH([dhclient],
64 AS_HELP_STRING([--with-dhclient=yes|no|path], [Enable dhclient support]))
65@@ -1400,6 +1416,7 @@ echo " ofono: $with_ofono"
66 echo " concheck: $enable_concheck"
67 echo " libteamdctl: $enable_teamdctl"
68 echo " ovs: $enable_ovs"
69+echo " netplan: $enable_netplan"
70 echo " nmcli: $build_nmcli"
71 echo " nmtui: $build_nmtui"
72 echo " nm-cloud-setup: $with_nm_cloud_setup"
73diff --git a/meson.build b/meson.build
74index 6813e52ac1..e808729c22 100644
75--- a/meson.build
76+++ b/meson.build
77@@ -274,6 +274,8 @@ config_h.set10('HAVE_LIBSYSTEMD', libsystemd_dep.found())
78 systemd_dep = dependency('systemd', required: false)
79 have_systemd_200 = systemd_dep.found() and systemd_dep.version().version_compare('>= 200')
80
81+libnetplan_dep = dependency('netplan', version: '>= 0.106', required: false)
82+
83 gio_unix_dep = dependency('gio-unix-2.0', version: '>= 2.40')
84
85 glib_dep = declare_dependency(
86@@ -646,6 +648,13 @@ if enable_ovs
87 endif
88 config_h.set10('WITH_OPENVSWITCH', enable_ovs)
89
90+# Netplan integration
91+enable_netplan = get_option('netplan')
92+if enable_netplan
93+ assert(libnetplan_dep.found(), 'libnetplan is needed for Netplan integration.')
94+endif
95+config_h.set10('WITH_NETPLAN', enable_netplan)
96+
97 # DNS resolv.conf managers
98 config_dns_rc_manager_default = get_option('config_dns_rc_manager_default')
99 config_h.set_quoted('NM_CONFIG_DEFAULT_MAIN_RC_MANAGER', config_dns_rc_manager_default)
100@@ -1064,6 +1073,7 @@ output += ' ofono: ' + enable_ofono.to_string() + '\n'
101 output += ' concheck: ' + enable_concheck.to_string() + '\n'
102 output += ' libteamdctl: ' + enable_teamdctl.to_string() + '\n'
103 output += ' ovs: ' + enable_ovs.to_string() + '\n'
104+output += ' netplan: ' + enable_netplan.to_string() + '\n'
105 output += ' nmcli: ' + enable_nmcli.to_string() + '\n'
106 output += ' nmtui: ' + enable_nmtui.to_string() + '\n'
107 output += ' nm-cloud-setup: ' + enable_nm_cloud_setup.to_string() + '\n'
108diff --git a/meson_options.txt b/meson_options.txt
109index 4e359f9e92..9f52b4f6e4 100644
110--- a/meson_options.txt
111+++ b/meson_options.txt
112@@ -37,6 +37,7 @@ option('ofono', type: 'boolean', value: false, description: 'Enable oFono suppor
113 option('concheck', type: 'boolean', value: true, description: 'enable connectivity checking support')
114 option('teamdctl', type: 'boolean', value: false, description: 'enable Teamd control support')
115 option('ovs', type: 'boolean', value: true, description: 'enable Open vSwitch support')
116+option('netplan', type: 'boolean', value: true, description: 'Enable Netplan integration')
117 option('nmcli', type: 'boolean', value: true, description: 'Build nmcli')
118 option('nmtui', type: 'boolean', value: true, description: 'Build nmtui')
119 option('nm_cloud_setup', type: 'boolean', value: false, description: 'Build nm-cloud-setup, a tool for automatically configure networking in cloud (EXPERIMENTAL!)')
120diff --git a/src/core/meson.build b/src/core/meson.build
121index 6c1d463ed1..35ece13f06 100644
122--- a/src/core/meson.build
123+++ b/src/core/meson.build
124@@ -71,6 +71,7 @@ nm_deps = [
125 libndp_dep,
126 libudev_dep,
127 logind_dep,
128+ libnetplan_dep,
129 ]
130
131 if enable_concheck
132--
1332.39.2
134
diff --git a/debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch b/debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch
0new file mode 100644135new file mode 100644
index 0000000..1d959f2
--- /dev/null
+++ b/debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch
@@ -0,0 +1,525 @@
1From aaa791cb61b24a9416db18f2290f711bc9a8f26e Mon Sep 17 00:00:00 2001
2From: Lukas Märdian <lukas.maerdian@canonical.com>
3Date: Tue, 2 Feb 2021 15:52:05 +0100
4Subject: [PATCH 2/2] netplan: make use of libnetplan for YAML backend
5
6Origin: https://github.com/slyon/NetworkManager/tree/netplan-nm-1.42
7Forwarded: no, https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/556
8Last-Update: 2023-05-11
9
10This patch modifies NetworkManager's nms-keyfile-plugin in a way to
11write YAML connections (according to the netplan spec) to
12/etc/netplan/*.yaml instead of NM's native keyfile connection profiles
13in /etc/NetworkManager/system-connections/*.nmconnection.
14
15Whenever a connection profile is to be written (add/modify) the keyfile,
16generated internally by NM, is passed into libnetplan's
17"netplan_parser_load_keyfile()" API, validated via
18"netplan_state_import_parser_results()" and converted to a netplan YAML
19config by calling libnetplan's "netplan_netdef_write_yaml()" API. The
20internal keyfile is thrown away afterwards.
21
22Whenever a connection profile is to be deleted the netplan-/netdef-id is
23extracted from the ephemeral keyfile in /run/NetworkManager/system-connections
24via "netplan_get_id_from_nm_filepath()" and the corresponding YAML is
25updated/deleted by calling "netplan_delete_connection()".
26
27Each time the YAML data was modified, NetworkManager calls
28"netplan generate" to produce new ephemeral keyfile connections in
29/run/NetworkManager/system-connections for NM to read-back. This way the
30netplan generator can be used as intended (no need for duplicated
31keyfile export functionality) and the nms-keyfile-writer can be re-used
32without any patching needed.
33
34V2:
35+ ported to NetworkManager 1.36.6 (Ubuntu Jammy LTS/Core 22.04)
36+ test-keyfile-settings.c: clear netplan YAML config from previous runs
37+ nms-keyfile-writer.c: avoid double-free of `path` on exit, caused by gs_free
38
39V3:
40+ ported to NM 1.40.6 (Ubuntu Lunar), using new libnetplan API (v0.106)
41+ ignore .nm-generated connections (LP: #1998207)
42
43V4:
44+ encapsulated all Netplan code in "#if WITH_NETPLAN" as provided by the
45 buildsystems via config.h (see previous commit)
46+ nms-keyfile-writer.c: increase buffer size to account for ".nmconnection"
47 suffix in netplan_netdef_get_output_filename()
48
49Co-authored-by: Alfonso Sanchez-Beato <alfonso.sanchez-beato@canonical.com>
50Co-authored-by: Danilo Egea Gondolfo <danilo.egea.gondolfo@canonical.com>
51---
52 .../plugins/keyfile/nms-keyfile-plugin.c | 34 ++++
53 .../plugins/keyfile/nms-keyfile-utils.c | 18 ++
54 .../plugins/keyfile/nms-keyfile-utils.h | 4 +
55 .../plugins/keyfile/nms-keyfile-writer.c | 173 ++++++++++++++++++
56 .../keyfile/tests/test-keyfile-settings.c | 61 +++++-
57 5 files changed, 286 insertions(+), 4 deletions(-)
58
59diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c b/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c
60index 1d7de8d24b..4b986dc0c8 100644
61--- a/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c
62+++ b/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c
63@@ -12,6 +12,9 @@
64 #include <unistd.h>
65 #include <sys/types.h>
66 #include <sys/time.h>
67+#if WITH_NETPLAN
68+#include <netplan/util.h>
69+#endif
70
71 #include "libnm-std-aux/c-list-util.h"
72 #include "libnm-glib-aux/nm-c-list.h"
73@@ -309,6 +312,12 @@ _load_file(NMSKeyfilePlugin *self,
74 gs_free char *full_filename = NULL;
75 struct stat st;
76
77+ #if WITH_NETPLAN
78+ // Handle all netplan generated connections via STORAGE_TYPE_ETC, as they live in /etc/netplan
79+ if (g_str_has_prefix(filename, "netplan-"))
80+ storage_type = NMS_KEYFILE_STORAGE_TYPE_ETC;
81+ #endif
82+
83 if (_ignore_filename(storage_type, filename)) {
84 gs_free char *nmmeta = NULL;
85 gs_free char *loaded_path = NULL;
86@@ -584,6 +593,9 @@ reload_connections(NMSettingsPlugin *plugin,
87 NM_SETT_UTIL_STORAGES_INIT(storages_new, nms_keyfile_storage_destroy);
88 int i;
89
90+ #if WITH_NETPLAN
91+ generate_netplan(NULL);
92+ #endif
93 _load_dir(self, NMS_KEYFILE_STORAGE_TYPE_RUN, priv->dirname_run, &storages_new);
94 if (priv->dirname_etc)
95 _load_dir(self, NMS_KEYFILE_STORAGE_TYPE_ETC, priv->dirname_etc, &storages_new);
96@@ -1008,6 +1020,15 @@ delete_connection(NMSettingsPlugin *plugin, NMSettingsStorage *storage_x, GError
97 previous_filename = nms_keyfile_storage_get_filename(storage);
98 uuid = nms_keyfile_storage_get_uuid(storage);
99
100+ #if WITH_NETPLAN
101+ nm_auto_unref_keyfile GKeyFile *key_file = NULL;
102+ key_file = g_key_file_new ();
103+ if (!g_key_file_load_from_file (key_file, previous_filename, G_KEY_FILE_NONE, error))
104+ return FALSE;
105+ g_autofree gchar* ssid = NULL;
106+ ssid = g_key_file_get_string(key_file, "wifi", "ssid", NULL);
107+ #endif
108+
109 if (!NM_IN_SET(storage->storage_type,
110 NMS_KEYFILE_STORAGE_TYPE_ETC,
111 NMS_KEYFILE_STORAGE_TYPE_RUN)) {
112@@ -1033,6 +1054,19 @@ delete_connection(NMSettingsPlugin *plugin, NMSettingsStorage *storage_x, GError
113 } else
114 operation_message = "deleted from disk";
115
116+ #if WITH_NETPLAN
117+ g_autofree gchar *netplan_id = NULL;
118+ ssize_t netplan_id_size = 0;
119+
120+ netplan_id = g_malloc0(strlen(previous_filename));
121+ netplan_id_size = netplan_get_id_from_nm_filepath(previous_filename, ssid, netplan_id, strlen(previous_filename) - 1);
122+ if (netplan_id_size > 0) {
123+ _LOGI ("deleting netplan connection: %s", netplan_id);
124+ netplan_delete_connection(netplan_id, NULL);
125+ generate_netplan(NULL);
126+ }
127+ #endif
128+
129 _LOGT("commit: deleted \"%s\", %s %s (%s%s%s%s)",
130 previous_filename,
131 storage->is_meta_data ? "meta-data" : "profile",
132diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-utils.c b/src/core/settings/plugins/keyfile/nms-keyfile-utils.c
133index 7c0e329e2d..a91ee6997a 100644
134--- a/src/core/settings/plugins/keyfile/nms-keyfile-utils.c
135+++ b/src/core/settings/plugins/keyfile/nms-keyfile-utils.c
136@@ -399,3 +399,21 @@ nms_keyfile_utils_check_file_permissions(NMSKeyfileFiletype filetype,
137 NM_SET_OUT(out_st, st);
138 return TRUE;
139 }
140+
141+#if WITH_NETPLAN
142+gboolean
143+generate_netplan(const char* rootdir)
144+{
145+ /* TODO: call the io.netplan.Netplan.Generate() DBus method directly, after
146+ * finding a way to pass the --root-dir parameter via DBus, to make it work
147+ * inside NM's unit-tests where netplan needs to read & generate outside of
148+ * /etc/netplan and /run/{systemd,NetworkManager} */
149+ const gchar *argv[] = { "netplan", "generate", NULL , NULL, NULL };
150+ if (rootdir) {
151+ argv[2] = "--root-dir";
152+ argv[3] = rootdir;
153+ }
154+ return g_spawn_sync(NULL, (gchar**)argv, NULL, G_SPAWN_SEARCH_PATH,
155+ NULL, NULL, NULL, NULL, NULL, NULL);
156+}
157+#endif
158diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-utils.h b/src/core/settings/plugins/keyfile/nms-keyfile-utils.h
159index 0fd83bdf35..9ed25d9cfd 100644
160--- a/src/core/settings/plugins/keyfile/nms-keyfile-utils.h
161+++ b/src/core/settings/plugins/keyfile/nms-keyfile-utils.h
162@@ -68,4 +68,8 @@ gboolean nms_keyfile_utils_check_file_permissions(NMSKeyfileFiletype filetype,
163 struct stat *out_st,
164 GError **error);
165
166+#if WITH_NETPLAN
167+gboolean generate_netplan(const char* rootdir);
168+#endif
169+
170 #endif /* __NMS_KEYFILE_UTILS_H__ */
171diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-writer.c b/src/core/settings/plugins/keyfile/nms-keyfile-writer.c
172index ad6f277ce3..e5015ab74b 100644
173--- a/src/core/settings/plugins/keyfile/nms-keyfile-writer.c
174+++ b/src/core/settings/plugins/keyfile/nms-keyfile-writer.c
175@@ -12,6 +12,14 @@
176 #include <sys/stat.h>
177 #include <unistd.h>
178
179+#if WITH_NETPLAN
180+#include <net/if.h>
181+#include <netplan/parse.h>
182+#include <netplan/parse-nm.h>
183+#include <netplan/util.h>
184+#include <netplan/netplan.h>
185+#endif
186+
187 #include "libnm-core-intern/nm-keyfile-internal.h"
188
189 #include "nms-keyfile-utils.h"
190@@ -201,6 +209,9 @@ _internal_write_connection(NMConnection *connection,
191 char **out_path,
192 NMConnection **out_reread,
193 gboolean *out_reread_same,
194+ #if WITH_NETPLAN
195+ const char *rootdir,
196+ #endif
197 GError **error)
198 {
199 nm_auto_unref_keyfile GKeyFile *kf_file = NULL;
200@@ -409,11 +420,161 @@ _internal_write_connection(NMConnection *connection,
201 if (existing_path && !existing_path_read_only && !nm_streq(path, existing_path))
202 unlink(existing_path);
203
204+ #if WITH_NETPLAN
205+ NetplanParser *npp = NULL;
206+ NetplanState *np_state = NULL;
207+
208+ /* NETPLAN: write only non-temporary files to /etc/netplan/... */
209+ if (!is_volatile && !is_nm_generated && !is_external &&
210+ strstr(keyfile_dir, "/etc/NetworkManager/system-connections")) {
211+ g_autofree gchar *ssid = g_key_file_get_string(kf_file, "wifi", "ssid", NULL);
212+ g_autofree gchar *escaped_ssid = ssid ?
213+ g_uri_escape_string(ssid, NULL, TRUE) : NULL;
214+ g_autofree gchar *netplan_id = NULL;
215+ ssize_t netplan_id_size = 0;
216+ NetplanNetDefinition *netdef = NULL;
217+ NetplanStateIterator state_iter;
218+ const gchar* kf_path = path;
219+
220+ if (existing_path && strstr(existing_path, "system-connections/netplan-")) {
221+ netplan_id = g_malloc0(strlen(existing_path));
222+ netplan_id_size = netplan_get_id_from_nm_filepath(existing_path, ssid, netplan_id, strlen(existing_path) - 1);
223+ if (netplan_id_size <= 0) {
224+ g_free(netplan_id);
225+ netplan_id = NULL;
226+ }
227+ }
228+
229+ if (netplan_id && existing_path) {
230+ GFile* from = g_file_new_for_path(path);
231+ GFile* to = g_file_new_for_path(existing_path);
232+ g_file_copy(from, to, G_FILE_COPY_OVERWRITE, NULL, NULL, NULL, NULL);
233+ kf_path = existing_path;
234+ }
235+
236+ // push keyfile into libnetplan for parsing (using existing_path, if available,
237+ // to be able to extract the original netdef_id and override existing settings)
238+ npp = netplan_parser_new();
239+
240+ if (!netplan_parser_load_keyfile(npp, kf_path, &local_err)) {
241+ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
242+ "netplan: YAML translation failed: %s", local_err->message);
243+ goto netplan_parser_error;
244+ }
245+
246+ np_state = netplan_state_new();
247+ if (!netplan_state_import_parser_results(np_state, npp, &local_err)) {
248+ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
249+ "netplan: YAML validation failed: %s", local_err->message);
250+ goto netplan_error;
251+ }
252+
253+ netplan_state_iterator_init(np_state, &state_iter);
254+ /* At this point we have a single netdef in the netplan state */
255+ netdef = netplan_state_iterator_next(&state_iter);
256+
257+ if (!netdef) {
258+ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
259+ "netplan: Netplan state has no network definitions");
260+ goto netplan_error;
261+ }
262+
263+ if (!netplan_netdef_write_yaml(np_state, netdef, rootdir, &local_err)) {
264+ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
265+ "netplan: Failed to generate YAML: %s", local_err->message);
266+ goto netplan_error;
267+ }
268+
269+ /* Delete same connection-profile provided by legacy netplan plugin.
270+ * TODO: drop legacy connection handling after 24.04 LTS */
271+ g_autofree gchar* legacy_path = NULL;
272+ legacy_path = g_strdup_printf("/etc/netplan/NM-%s.yaml", nm_connection_get_uuid (connection));
273+ if (g_file_test(legacy_path, G_FILE_TEST_EXISTS | G_FILE_TEST_IS_REGULAR)) {
274+ g_debug("Deleting legacy netplan connection: %s", legacy_path);
275+ unlink(legacy_path);
276+ }
277+
278+ /* Clear original keyfile in /etc/NetworkManager/system-connections/,
279+ * we've written the /etc/netplan/*.yaml file instead. */
280+ unlink(path);
281+ if (!generate_netplan(rootdir)) {
282+ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
283+ "netplan generate failed");
284+ goto netplan_error;
285+ }
286+
287+ // Calculating the maximum space needed to store the new keyfile path
288+ ssize_t path_size = strlen(path) + strlen(nm_connection_get_uuid(connection)) + IF_NAMESIZE + 1;
289+ if (escaped_ssid)
290+ path_size += strlen(escaped_ssid);
291+ path_size += 50; // give some extra buffer, e.g. when going from ConName to ConName.nmconnection
292+
293+ g_free(path);
294+ path = g_malloc0(path_size);
295+ path_size = netplan_netdef_get_output_filename(netdef, ssid, path, path_size);
296+
297+ if (path_size <= 0) {
298+ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
299+ "netplan: couldn't determine the keyfile path");
300+ goto netplan_error;
301+ }
302+
303+ if (rootdir) {
304+ char* final_path = g_build_path(G_DIR_SEPARATOR_S, rootdir, path, NULL);
305+ g_free(path);
306+ path = final_path;
307+ }
308+
309+ netplan_state_clear(&np_state);
310+ netplan_parser_clear(&npp);
311+
312+ /* re-read again: this time the connection profile newly generated by netplan in /run/... */
313+ if ( out_reread
314+ || out_reread_same) {
315+ gs_free_error GError *reread_error = NULL;
316+
317+ //XXX: why does the _from_keyfile function behave differently?
318+ //reread = nms_keyfile_reader_from_keyfile (kf_file, path, NULL, profile_dir, FALSE, &reread_error);
319+ reread = nms_keyfile_reader_from_file (path, profile_dir, NULL, NULL, NULL, NULL, NULL, NULL, &reread_error);
320+
321+ if ( !reread
322+ || !nm_connection_normalize (reread, NULL, NULL, &reread_error)) {
323+ nm_log_err (LOGD_SETTINGS, "BUG: the profile cannot be stored in keyfile format without becoming unusable: %s", reread_error->message);
324+ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED,
325+ "keyfile writer produces an invalid connection: %s",
326+ reread_error->message);
327+ nm_assert_not_reached ();
328+ return FALSE;
329+ }
330+
331+ if (out_reread_same) {
332+ reread_same = !!nm_connection_compare (reread, connection, NM_SETTING_COMPARE_FLAG_EXACT);
333+
334+ nm_assert (reread_same == nm_connection_compare (connection, reread, NM_SETTING_COMPARE_FLAG_EXACT));
335+ nm_assert (reread_same == ({
336+ gs_unref_hashtable GHashTable *_settings = NULL;
337+
338+ ( nm_connection_diff (reread, connection, NM_SETTING_COMPARE_FLAG_EXACT, &_settings)
339+ && !_settings);
340+ }));
341+ }
342+ }
343+ }
344+ #endif
345+
346 NM_SET_OUT(out_reread, g_steal_pointer(&reread));
347 NM_SET_OUT(out_reread_same, reread_same);
348 NM_SET_OUT(out_path, g_steal_pointer(&path));
349
350 return TRUE;
351+
352+#if WITH_NETPLAN
353+netplan_error:
354+ netplan_state_clear(&np_state);
355+netplan_parser_error:
356+ netplan_parser_clear(&npp);
357+ return FALSE;
358+#endif
359 }
360
361 gboolean
362@@ -454,6 +615,9 @@ nms_keyfile_writer_connection(NMConnection *connection,
363 out_path,
364 out_reread,
365 out_reread_same,
366+ #if WITH_NETPLAN
367+ NULL,
368+ #endif
369 error);
370 }
371
372@@ -467,6 +631,12 @@ nms_keyfile_writer_test_connection(NMConnection *connection,
373 gboolean *out_reread_same,
374 GError **error)
375 {
376+ #if WITH_NETPLAN
377+ gchar *rootdir = g_strdup(keyfile_dir);
378+ if (g_str_has_suffix (keyfile_dir, "/run/NetworkManager/system-connections")) {
379+ rootdir[strlen(rootdir)-38] = '\0'; /* 38 = strlen("/run/NetworkManager/...") */
380+ }
381+ #endif
382 return _internal_write_connection(connection,
383 FALSE,
384 FALSE,
385@@ -486,5 +656,8 @@ nms_keyfile_writer_test_connection(NMConnection *connection,
386 out_path,
387 out_reread,
388 out_reread_same,
389+ #if WITH_NETPLAN
390+ rootdir,
391+ #endif
392 error);
393 }
394diff --git a/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c b/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c
395index 83019babb1..eb59a91eab 100644
396--- a/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c
397+++ b/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c
398@@ -24,8 +24,15 @@
399
400 #include "nm-test-utils-core.h"
401
402+#if WITH_NETPLAN
403+#define TEST_KEYFILES_DIR_OLD NM_BUILD_SRCDIR"/src/core/settings/plugins/keyfile/tests/keyfiles"
404+#define TEST_SCRATCH_DIR_OLD NM_BUILD_BUILDDIR"/src/core/settings/plugins/keyfile/tests/keyfiles"
405+#define TEST_KEYFILES_DIR TEST_KEYFILES_DIR_OLD"/run/NetworkManager/system-connections"
406+#define TEST_SCRATCH_DIR TEST_SCRATCH_DIR_OLD"/run/NetworkManager/system-connections"
407+#else
408 #define TEST_KEYFILES_DIR NM_BUILD_SRCDIR "/src/core/settings/plugins/keyfile/tests/keyfiles"
409 #define TEST_SCRATCH_DIR NM_BUILD_BUILDDIR "/src/core/settings/plugins/keyfile/tests/keyfiles"
410+#endif
411
412 /*****************************************************************************/
413
414@@ -113,6 +120,11 @@ assert_reread_and_unlink(NMConnection *connection,
415 static void
416 assert_reread_same(NMConnection *connection, NMConnection *reread)
417 {
418+ #if WITH_NETPLAN
419+ // Netplan does some normalization already, so compare normalized connections
420+ nm_connection_normalize (connection, NULL, NULL, NULL);
421+ nm_connection_normalize (reread, NULL, NULL, NULL);
422+ #endif
423 nmtst_assert_connection_verifies_without_normalization(reread);
424 nmtst_assert_connection_equals(connection, TRUE, reread, FALSE);
425 }
426@@ -789,6 +801,10 @@ test_write_wireless_connection(void)
427 bssid,
428 NM_SETTING_WIRELESS_SSID,
429 ssid,
430+ #if WITH_NETPLAN
431+ //XXX: netplan uses explicit "infrastructure" mode
432+ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA,
433+ #endif
434 NM_SETTING_WIRED_MTU,
435 1000,
436 NULL);
437@@ -870,7 +886,12 @@ test_write_string_ssid(void)
438 nm_connection_add_setting(connection, NM_SETTING(s_wireless));
439
440 ssid = g_bytes_new(tmpssid, sizeof(tmpssid));
441- g_object_set(s_wireless, NM_SETTING_WIRELESS_SSID, ssid, NULL);
442+ g_object_set(s_wireless, NM_SETTING_WIRELESS_SSID, ssid,
443+ #if WITH_NETPLAN
444+ //XXX: netplan uses explicit "infrastructure" mode
445+ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA,
446+ #endif
447+ NULL);
448 g_bytes_unref(ssid);
449
450 /* IP4 setting */
451@@ -953,7 +974,12 @@ test_write_intlist_ssid(void)
452 nm_connection_add_setting(connection, NM_SETTING(s_wifi));
453
454 ssid = g_bytes_new(tmpssid, sizeof(tmpssid));
455- g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, NULL);
456+ g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid,
457+ #if WITH_NETPLAN
458+ //XXX: netplan uses explicit "infrastructure" mode
459+ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA,
460+ #endif
461+ NULL);
462 g_bytes_unref(ssid);
463
464 /* IP4 setting */
465@@ -1053,7 +1079,12 @@ test_write_intlike_ssid(void)
466 nm_connection_add_setting(connection, NM_SETTING(s_wifi));
467
468 ssid = g_bytes_new(tmpssid, sizeof(tmpssid));
469- g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, NULL);
470+ g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid,
471+ #if WITH_NETPLAN
472+ //XXX: netplan uses explicit "infrastructure" mode
473+ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA,
474+ #endif
475+ NULL);
476 g_bytes_unref(ssid);
477
478 /* IP4 setting */
479@@ -1115,7 +1146,12 @@ test_write_intlike_ssid_2(void)
480 nm_connection_add_setting(connection, NM_SETTING(s_wifi));
481
482 ssid = g_bytes_new(tmpssid, sizeof(tmpssid));
483- g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, NULL);
484+ g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid,
485+ #if WITH_NETPLAN
486+ //XXX: netplan uses explicit "infrastructure" mode
487+ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA,
488+ #endif
489+ NULL);
490 g_bytes_unref(ssid);
491
492 /* IP4 setting */
493@@ -2845,12 +2881,29 @@ main(int argc, char **argv)
494
495 nmtst_init_assert_logging(&argc, &argv, "INFO", "DEFAULT");
496
497+ #if WITH_NETPLAN
498+ if (g_mkdir_with_parents(TEST_SCRATCH_DIR_OLD, 0755) != 0) {
499+ errsv = errno;
500+ g_error("failure to create test directory \"%s\": %s",
501+ TEST_SCRATCH_DIR_OLD,
502+ nm_strerror_native(errsv));
503+ }
504+ // Prepare netplan test directories
505+ g_mkdir_with_parents (TEST_SCRATCH_DIR_OLD"/etc/netplan", 0755);
506+ g_mkdir_with_parents (TEST_SCRATCH_DIR_OLD"/run/NetworkManager", 0755);
507+ // link "keyfiles/" to "run/NetworkManager/system-connections"
508+ const gchar *args[] = { "/bin/ln", "-s", TEST_KEYFILES_DIR_OLD, TEST_KEYFILES_DIR, NULL };
509+ g_spawn_sync(NULL, (gchar**)args, NULL, G_SPAWN_DEFAULT, NULL, NULL, NULL, NULL, NULL, NULL);
510+ // clear netplan YAML config from previous runs
511+ g_spawn_command_line_sync("/bin/sh -c 'rm -f " TEST_KEYFILES_DIR_OLD "/etc/netplan/*.yaml'", NULL, NULL, NULL, NULL);
512+ #else
513 if (g_mkdir_with_parents(TEST_SCRATCH_DIR, 0755) != 0) {
514 errsv = errno;
515 g_error("failure to create test directory \"%s\": %s",
516 TEST_SCRATCH_DIR,
517 nm_strerror_native(errsv));
518 }
519+ #endif
520
521 /* The tests */
522 g_test_add_func("/keyfile/test_read_valid_wired_connection", test_read_valid_wired_connection);
523--
5242.39.2
525
diff --git a/debian/patches/series b/debian/patches/series
index c2344eb..2e570aa 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -3,3 +3,5 @@ Force-online-state-with-unmanaged-devices.patch
3# Ubuntu patches3# Ubuntu patches
4Provide-access-to-some-of-NM-s-interfaces-to-whoopsie.patch4Provide-access-to-some-of-NM-s-interfaces-to-whoopsie.patch
5Update-dnsmasq-parameters.patch5Update-dnsmasq-parameters.patch
6netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch
7netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch
diff --git a/debian/rules b/debian/rules
index edcb573..7d336f1 100755
--- a/debian/rules
+++ b/debian/rules
@@ -4,6 +4,7 @@ include /usr/share/dpkg/architecture.mk
44
5ifeq ($(shell dpkg-vendor --is Ubuntu && echo yes) $(DEB_HOST_ARCH), yes i386)5ifeq ($(shell dpkg-vendor --is Ubuntu && echo yes) $(DEB_HOST_ARCH), yes i386)
6 BUILD_PACKAGES += -Nnetwork-manager6 BUILD_PACKAGES += -Nnetwork-manager
7 NETPLAN += --disable-netplan
7endif8endif
89
9# Disable lto here regardless of whether we enable the configure flag10# Disable lto here regardless of whether we enable the configure flag
@@ -61,7 +62,7 @@ override_dh_auto_configure:
61 --enable-lto \62 --enable-lto \
62 --disable-more-warnings \63 --disable-more-warnings \
63 --disable-modify-system \64 --disable-modify-system \
64 --disable-ovs65 --disable-ovs $(NETPLAN)
6566
66override_dh_install:67override_dh_install:
67 find debian/tmp -name '*.la' -print -delete68 find debian/tmp -name '*.la' -print -delete
diff --git a/debian/tests/control b/debian/tests/control
index 795b9ba..1985292 100644
--- a/debian/tests/control
+++ b/debian/tests/control
@@ -13,3 +13,7 @@ Restrictions: needs-root allow-stderr isolation-machine skippable
13Tests: urfkill-integration13Tests: urfkill-integration
14Depends: network-manager, build-essential, linux-headers-generic [!i386], rfkill, urfkill14Depends: network-manager, build-essential, linux-headers-generic [!i386], rfkill, urfkill
15Restrictions: needs-root allow-stderr isolation-machine skippable15Restrictions: needs-root allow-stderr isolation-machine skippable
16
17Tests: nm_netplan.py
18Depends: python3, gir1.2-nm-1.0, network-manager, netplan.io, openvpn, easy-rsa, network-manager-openvpn
19Restrictions: needs-root allow-stderr isolation-container
diff --git a/debian/tests/nm.py b/debian/tests/nm.py
index 47345db..f7a03b7 100755
--- a/debian/tests/nm.py
+++ b/debian/tests/nm.py
@@ -70,6 +70,7 @@ class NetworkManagerTest(network_test_base.NetworkTestBase):
70 "/etc/NetworkManager",70 "/etc/NetworkManager",
71 "/var/lib/NetworkManager",71 "/var/lib/NetworkManager",
72 "/run/NetworkManager",72 "/run/NetworkManager",
73 "/etc/netplan",
73 ]:74 ]:
74 subprocess.check_call(["mount", "-n", "-t", "tmpfs", "none", d])75 subprocess.check_call(["mount", "-n", "-t", "tmpfs", "none", d])
75 self.addCleanup(subprocess.call, ["umount", d])76 self.addCleanup(subprocess.call, ["umount", d])
diff --git a/debian/tests/nm_netplan.py b/debian/tests/nm_netplan.py
76new file mode 10064477new file mode 100644
index 0000000..cc86bcf
--- /dev/null
+++ b/debian/tests/nm_netplan.py
@@ -0,0 +1,717 @@
1#!/usr/bin/python3
2
3__author__ = "Danilo Egea Gondolfo <danilo.egea.gondolfo@canonical.com>"
4__copyright__ = "(C) 2023 Canonical Ltd."
5__license__ = "GPL v2 or later"
6
7from glob import glob
8import json
9import shutil
10import socket
11import subprocess
12import sys
13import os
14from time import sleep
15import unittest
16import yaml
17
18import gi
19gi.require_version("NM", "1.0")
20from gi.repository import NM, GLib, Gio
21
22nmclient = NM.Client.new()
23
24class TestNetplan(unittest.TestCase):
25
26 def setUp(self):
27 self._stop_network_manager()
28
29 self._start_nm()
30 sleep(1)
31 self.nmclient = NM.Client.new()
32
33 def tearDown(self):
34 pass
35
36 def _start_nm(self, auto_connect=True):
37 """This method is basically a copy of start_nm() from nm.py
38 without the parts we don't need
39 """
40
41 if not os.path.exists("/run/NetworkManager"):
42 os.mkdir("/run/NetworkManager")
43 for d in [
44 "/etc/NetworkManager",
45 "/var/lib/NetworkManager",
46 "/run/NetworkManager",
47 "/etc/netplan",
48 ]:
49 subprocess.check_call(["mount", "-n", "-t", "tmpfs", "none", d])
50 self.addCleanup(subprocess.call, ["umount", d])
51 os.mkdir("/etc/NetworkManager/system-connections")
52
53 denylist = ""
54 for iface in os.listdir("/sys/class/net"):
55 if iface in ['bonding_masters']:
56 continue
57 with open("/sys/class/net/%s/address" % iface) as f:
58 if denylist:
59 denylist += ";"
60 denylist += "mac:%s" % f.read().strip()
61
62 conf = "/etc/NetworkManager/NetworkManager.conf"
63 extra_main = ""
64 if not auto_connect:
65 extra_main += "no-auto-default=*\n"
66
67 with open(conf, "w") as f:
68 f.write(
69 "[main]\nplugins=keyfile\n%s\n[keyfile]\nunmanaged-devices=%s\n"
70 % (extra_main, denylist)
71 )
72
73 log = "/tmp/NetworkManager.log"
74 f_log = os.open(log, os.O_CREAT | os.O_WRONLY | os.O_SYNC)
75
76 # build NM command line
77 argv = ["NetworkManager", "--log-level=debug", "--debug", "--config=" + conf]
78 # allow specifying extra arguments
79 argv += os.environ.get("NM_TEST_DAEMON_ARGS", "").strip().split()
80
81 p = subprocess.Popen(argv, stdout=f_log, stderr=subprocess.STDOUT)
82 # automatically terminate process at end of test case
83 self.addCleanup(p.wait)
84 self.addCleanup(p.terminate)
85 self.addCleanup(os.close, f_log)
86 self.addCleanup(self._clear_connections)
87
88 self._process_glib_events()
89
90 def _process_glib_events(self):
91 """Process pending GLib main loop events"""
92
93 context = GLib.MainContext.default()
94 while context.iteration(False):
95 pass
96
97 def _restart_network_manager(self):
98 cmd = ['systemctl', 'restart', 'NetworkManager']
99 subprocess.call(cmd, stdout=subprocess.DEVNULL)
100
101 def _stop_network_manager(self):
102 cmd = ['systemctl', 'stop', 'NetworkManager']
103 subprocess.call(cmd, stdout=subprocess.DEVNULL)
104
105 def _add_connection(self, connection):
106 main_loop = GLib.MainLoop()
107 def add_cb(client, result, data):
108 self.nmclient.add_connection_finish(result)
109 main_loop.quit()
110
111 self.nmclient.add_connection_async(connection, True, None, add_cb, None)
112 main_loop.run()
113
114 def _delete_connection(self, connection):
115 uuid = connection.get_uuid()
116 cmd = ['nmcli', 'con', 'del', uuid]
117 subprocess.call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
118
119 def _delete_interface(self, connection):
120 iface = connection.get_interface_name()
121 if iface:
122 cmd = ['ip', 'link', 'del', iface]
123 subprocess.call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
124
125 def _nmcli(self, parameters):
126 cmd = ['nmcli'] + parameters
127 subprocess.call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
128
129 def _bridge_show(self, bridge):
130 cmd = ['bridge', '-j', 'link', 'show', bridge]
131 ret = subprocess.run(cmd, capture_output=True)
132 return json.loads(ret.stdout)
133
134 def _load_netplan_yaml_for_connection(self, connection):
135 filename = '/etc/netplan/90-NM-' + connection.get_uuid() + '.yaml'
136
137 file = open(filename)
138 data = yaml.safe_load(file)
139 file.close()
140 return data
141
142 def _get_number_of_yaml_files(self):
143 return len(self._get_list_of_yaml_files())
144
145 def _get_list_of_yaml_files(self):
146 return glob("/etc/netplan/90-NM-*")
147
148 def _commit_and_save_connection(self, connection):
149 main_loop = GLib.MainLoop()
150
151 def commit_cb(client, result, data):
152 connection.commit_changes_finish(result)
153 main_loop.quit()
154
155 connection.commit_changes_async(True, None, commit_cb, None)
156 main_loop.run()
157
158 def _clear_connections(self):
159 for conn in self.nmclient.get_connections():
160 self._delete_connection(conn)
161 self._delete_interface(conn)
162
163 def _netplan_generate(self):
164 cmd = ['netplan', 'generate']
165 ret = subprocess.run(cmd, capture_output=True)
166
167 def _nmcli_con_reload(self):
168 self._nmcli(['con', 'reload'])
169
170 # Tests
171
172 def test_create_a_simple_bridge_with_dhcp(self):
173
174 conn = NM.SimpleConnection.new()
175 settings = NM.SettingConnection.new()
176 settings.set_property(NM.SETTING_CONNECTION_ID, "bridge0")
177 settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "bridge0")
178 settings.set_property(NM.SETTING_CONNECTION_TYPE, "bridge")
179
180 bridge = NM.SettingBridge.new()
181 ipv4 = NM.SettingIP4Config.new()
182 ipv4.set_property(NM.SETTING_IP_CONFIG_METHOD, "auto")
183 ipv6 = NM.SettingIP6Config.new()
184 ipv6.set_property(NM.SETTING_IP_CONFIG_METHOD, "auto")
185
186 conn.add_setting(settings)
187 conn.add_setting(ipv4)
188 conn.add_setting(ipv6)
189 conn.add_setting(bridge)
190
191 # There should be zero netplan NM yaml before adding a connection
192 self.assertEqual(self._get_number_of_yaml_files(), 0)
193
194 self._add_connection(conn)
195
196 # There should be one netplan NM yaml after adding a connection
197 self.assertEqual(self._get_number_of_yaml_files(), 1)
198
199 connection = self.nmclient.get_connection_by_id("bridge0")
200 yaml_data = self._load_netplan_yaml_for_connection(connection)
201
202 # Validating some of the expected flags
203 self.assertTrue(yaml_data['network']['bridges']['bridge0']['dhcp4'])
204 self.assertTrue(yaml_data['network']['bridges']['bridge0']['dhcp6'])
205
206 self._delete_connection(connection)
207
208 # There should be zero netplan NM yaml after deleting a connection
209 self.assertEqual(self._get_number_of_yaml_files(), 0)
210
211 def test_create_a_simple_bridge_with_ip_addresses(self):
212 conn = NM.SimpleConnection.new()
213 settings = NM.SettingConnection.new()
214 settings.set_property(NM.SETTING_CONNECTION_ID, "bridge0")
215 settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "bridge0")
216 settings.set_property(NM.SETTING_CONNECTION_TYPE, "bridge")
217
218 bridge = NM.SettingBridge.new()
219
220 ipv4 = NM.SettingIP4Config.new()
221 ipv4.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
222 ip4_addr1 = NM.IPAddress.new(socket.AF_INET, "10.20.30.40", 24)
223 ip4_addr2 = NM.IPAddress.new(socket.AF_INET, "10.20.30.41", 24)
224 ipv4.add_address(ip4_addr1)
225 ipv4.add_address(ip4_addr2)
226
227 ipv6 = NM.SettingIP6Config.new()
228 ipv6.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
229 ip6_addr1 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::1", 64)
230 ip6_addr2 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::2", 64)
231 ipv6.add_address(ip6_addr1)
232 ipv6.add_address(ip6_addr2)
233
234 conn.add_setting(settings)
235 conn.add_setting(ipv4)
236 conn.add_setting(ipv6)
237 conn.add_setting(bridge)
238
239 # There should be zero netplan NM yaml before adding a connection
240 self.assertEqual(self._get_number_of_yaml_files(), 0)
241
242 self._add_connection(conn)
243
244 # There should be one netplan NM yaml after adding a connection
245 self.assertEqual(self._get_number_of_yaml_files(), 1)
246
247 connection = self.nmclient.get_connection_by_id("bridge0")
248 yaml_data = self._load_netplan_yaml_for_connection(connection)
249
250 # Validating some of the expected flags
251 self.assertNotIn('dhcp4', yaml_data['network']['bridges']['bridge0'])
252 self.assertNotIn('dhcp6', yaml_data['network']['bridges']['bridge0'])
253
254 addresses = yaml_data['network']['bridges']['bridge0']['addresses']
255 expected_addresses = ['10.20.30.40/24', '10.20.30.41/24', 'dead:beef::1/64', 'dead:beef::2/64']
256
257 self.assertListEqual(addresses, expected_addresses)
258
259 self._delete_connection(connection)
260
261 # There should be zero netplan NM yaml after deleting a connection
262 self.assertEqual(self._get_number_of_yaml_files(), 0)
263
264 def test_create_a_simple_bridge_with_ip_and_member(self):
265
266 conn = NM.SimpleConnection.new()
267 settings = NM.SettingConnection.new()
268 settings.set_property(NM.SETTING_CONNECTION_ID, "bridge0")
269 settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "bridge0")
270 settings.set_property(NM.SETTING_CONNECTION_TYPE, "bridge")
271
272 bridge = NM.SettingBridge.new()
273
274 ipv4 = NM.SettingIP4Config.new()
275 ipv4.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
276 ip4_addr1 = NM.IPAddress.new(socket.AF_INET, "10.20.30.40", 24)
277 ip4_addr2 = NM.IPAddress.new(socket.AF_INET, "10.20.30.41", 24)
278 ipv4.add_address(ip4_addr1)
279 ipv4.add_address(ip4_addr2)
280
281 ipv6 = NM.SettingIP6Config.new()
282 ipv6.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
283 ip6_addr1 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::1", 64)
284 ip6_addr2 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::2", 64)
285 ipv6.add_address(ip6_addr1)
286 ipv6.add_address(ip6_addr2)
287
288 conn.add_setting(settings)
289 conn.add_setting(ipv4)
290 conn.add_setting(ipv6)
291 conn.add_setting(bridge)
292
293 # There should be zero netplan NM yaml before adding a connection
294 self.assertEqual(self._get_number_of_yaml_files(), 0)
295
296 # Adding the bridge
297 self._add_connection(conn)
298
299 # There should be one netplan NM yaml after adding a connection
300 self.assertEqual(self._get_number_of_yaml_files(), 1)
301
302 # Creating a tap0 device to be a bridge member
303 tap0 = NM.SimpleConnection.new()
304 tap0_conn_settings = NM.SettingConnection.new()
305 tap0_conn_settings.set_property(NM.SETTING_CONNECTION_ID, "tap0")
306 tap0_conn_settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "tap0")
307 tap0_conn_settings.set_property(NM.SETTING_CONNECTION_TYPE, "tun")
308 tap0_conn_settings.set_property(NM.SETTING_CONNECTION_SLAVE_TYPE, "bridge")
309 tap0_conn_settings.set_property(NM.SETTING_CONNECTION_MASTER, "bridge0")
310
311 tap0_settings = NM.SettingTun.new()
312 tap0_settings.set_property(NM.SETTING_TUN_MODE, NM.SettingTunMode.TAP)
313
314 tap0.add_setting(tap0_conn_settings)
315 tap0.add_setting(tap0_settings)
316 self._add_connection(tap0)
317
318 # There should be two netplan NM yaml after adding the tap
319 self.assertEqual(self._get_number_of_yaml_files(), 2)
320
321 bridge0_connection = self.nmclient.get_connection_by_id("bridge0")
322 yaml_data = self._load_netplan_yaml_for_connection(bridge0_connection)
323
324 # Validating some of the bridge expected flags
325 addresses = yaml_data['network']['bridges']['bridge0']['addresses']
326 expected_addresses = ['10.20.30.40/24', '10.20.30.41/24', 'dead:beef::1/64', 'dead:beef::2/64']
327
328 self.assertListEqual(addresses, expected_addresses)
329
330 # Validating if tap0 is attached to the bridge
331 # It might take a while for the new interface be created...
332 limit = 10
333 while (show_bridge := self._bridge_show('bridge0')) == [] and limit > 0:
334 sleep(1)
335 limit = limit - 1
336 self.assertEqual(show_bridge[0]['master'], 'bridge0')
337 self.assertEqual(show_bridge[0]['ifname'], 'tap0')
338
339 tap0_connection = self.nmclient.get_connection_by_id("tap0")
340 tap_yaml = self._load_netplan_yaml_for_connection(tap0_connection)
341
342 tap0_uuid = tap0_connection.get_uuid()
343 # Validating that the bridge information is in the tap yaml
344 self.assertEqual(tap_yaml['network']['nm-devices']['NM-' + tap0_uuid]['networkmanager']['passthrough']['connection.master'], 'bridge0')
345
346 self._delete_connection(tap0_connection)
347 # There should be one netplan NM yaml after deleting the tap
348 self.assertEqual(self._get_number_of_yaml_files(), 1)
349
350 self._delete_connection(bridge0_connection)
351 # There should be zero netplan NM yaml after deleting the bridge
352 self.assertEqual(self._get_number_of_yaml_files(), 0)
353
354 @unittest.skip('XXX: the netplan yaml file will be deleted when we try to change the connetion')
355 def test_create_an_interface_and_change_it(self):
356 """Add a tap interface and change it after create adding IP addresses to it."""
357
358 # Creating a tap0 device to be a bridge member
359 tap0 = NM.SimpleConnection.new()
360 tap0_conn_settings = NM.SettingConnection.new()
361 tap0_conn_settings.set_property(NM.SETTING_CONNECTION_ID, "tap0")
362 tap0_conn_settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "tap0")
363 tap0_conn_settings.set_property(NM.SETTING_CONNECTION_TYPE, "tun")
364
365 tap0_settings = NM.SettingTun.new()
366 tap0_settings.set_property(NM.SETTING_TUN_MODE, NM.SettingTunMode.TAP)
367
368 tap0.add_setting(tap0_conn_settings)
369 tap0.add_setting(tap0_settings)
370 self._add_connection(tap0)
371
372 # There should be one netplan NM yaml after adding a connection
373 self.assertEqual(self._get_number_of_yaml_files(), 1)
374
375 tap0_connection = self.nmclient.get_connection_by_id("tap0")
376
377 ipv4_settings = tap0_connection.get_setting_ip4_config()
378 ipv4_settings.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
379 ip4_addr1 = NM.IPAddress.new(socket.AF_INET, "10.20.30.40", 24)
380 ip4_addr2 = NM.IPAddress.new(socket.AF_INET, "10.20.30.41", 24)
381 ipv4_settings.add_address(ip4_addr1)
382 ipv4_settings.add_address(ip4_addr2)
383
384 ipv6_settings = tap0_connection.get_setting_ip6_config()
385 ipv6_settings.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual")
386 ip6_addr1 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::1", 64)
387 ip6_addr2 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::2", 64)
388 ipv6_settings.add_address(ip6_addr1)
389 ipv6_settings.add_address(ip6_addr2)
390
391 self._commit_and_save_connection(tap0_connection)
392
393 # There should be one netplan NM yaml files after chaging the only existing connection
394 self.assertEqual(self._get_number_of_yaml_files(), 1)
395
396 self._delete_connection(tap0_connection)
397
398 # There should be zero netplan NM yaml files after removing the only existing connection
399 self.assertEqual(self._get_number_of_yaml_files(), 0)
400
401 def test_nmcli_add_device_and_change_it(self):
402 """Uses the nmcli to add a connection and validates if the
403 Netplan YAML file has the expected configuration.
404 It also changes the configuration changing the ipv4 method from auto
405 to manual and adding an IP address. After the change the Netplan YAML
406 file should have the same name.
407 """
408
409 self.assertEqual(self._get_number_of_yaml_files(), 0)
410
411 nmcli_add = ['con', 'add', 'type', 'tun', 'mode', 'tap', 'ifname', 'tap0', 'ipv4.method', 'auto']
412 self._nmcli(nmcli_add)
413
414 self.assertEqual(self._get_number_of_yaml_files(), 1)
415
416 files_before = self._get_list_of_yaml_files()
417
418 # After OOB changes, the client must be refreshed apparently
419 self.nmclient = NM.Client.new()
420
421 conn = self.nmclient.get_connection_by_id("tun-tap0")
422 uuid = conn.get_uuid()
423 data = self._load_netplan_yaml_for_connection(conn)
424
425 ip4_method = data.get('network') \
426 .get('nm-devices') \
427 .get('NM-' + uuid) \
428 .get('networkmanager') \
429 .get('passthrough') \
430 .get('ipv4.method')
431
432 self.assertEqual(ip4_method, 'auto')
433
434 nmcli_mod = ['con', 'mod', 'tun-tap0', 'ipv4.method', 'manual', 'ipv4.addresses', '10.20.30.40/24']
435 self._nmcli(nmcli_mod)
436
437 self.assertEqual(self._get_number_of_yaml_files(), 1)
438
439 files_after = self._get_list_of_yaml_files()
440
441 self.assertListEqual(files_before, files_after)
442
443 data = self._load_netplan_yaml_for_connection(conn)
444
445 ip4_method = data.get('network') \
446 .get('nm-devices') \
447 .get('NM-' + uuid) \
448 .get('networkmanager') \
449 .get('passthrough') \
450 .get('ipv4.method')
451
452 ip4_addr = data.get('network') \
453 .get('nm-devices') \
454 .get('NM-' + uuid) \
455 .get('networkmanager') \
456 .get('passthrough') \
457 .get('ipv4.address1')
458
459 self.assertEqual(ip4_method, 'manual')
460 self.assertEqual(ip4_addr, '10.20.30.40/24')
461
462 self._delete_connection(conn)
463 self.assertEqual(self._get_number_of_yaml_files(), 0)
464
465 def test_nmcli_add_wifi_connection(self):
466 """Create a wifi connection via nmcli and check if the expected
467 fields were added to the netplan yaml file."""
468
469 ssid = 'My network SSID'
470 passwd = 'secretpasswd'
471 method = 'wpa-psk'
472 ip = '10.20.30.40/24'
473 nmcli_add = ['con', 'add', 'type', 'wifi', 'ssid', ssid,
474 'wifi-sec.key-mgmt', method, 'wifi-sec.psk', passwd,
475 'ipv4.method', 'manual', 'ipv4.addresses', ip]
476
477 self.assertEqual(self._get_number_of_yaml_files(), 0)
478
479 self._nmcli(nmcli_add)
480
481 self.assertEqual(self._get_number_of_yaml_files(), 1)
482
483 # After OOB changes, the client must be refreshed apparently
484 self.nmclient = NM.Client.new()
485
486 conn = self.nmclient.get_connection_by_id("wifi")
487 uuid = conn.get_uuid()
488 data = self._load_netplan_yaml_for_connection(conn)
489
490 ap_name = list(data.get('network') \
491 .get('wifis') \
492 .get('NM-' + uuid) \
493 .get('access-points') \
494 .keys())[0]
495
496 ip_addr = data.get('network') \
497 .get('wifis') \
498 .get('NM-' + uuid) \
499 .get('addresses')
500
501 auth_method = data.get('network') \
502 .get('wifis') \
503 .get('NM-' + uuid) \
504 .get('access-points') \
505 .get(ap_name) \
506 .get('auth') \
507 .get('key-management')
508
509 auth_passwd = data.get('network') \
510 .get('wifis') \
511 .get('NM-' + uuid) \
512 .get('access-points') \
513 .get(ap_name) \
514 .get('auth') \
515 .get('password')
516
517 self.assertEqual(ap_name, ssid)
518 self.assertListEqual(ip_addr, [ip])
519 self.assertEqual(auth_method, 'psk')
520 self.assertEqual(auth_passwd, passwd)
521
522 self._delete_connection(conn)
523 self.assertEqual(self._get_number_of_yaml_files(), 0)
524
525 def test_create_connection_via_netplan(self):
526 """
527 Create a connection via netplan generate and check if NM will pick it up
528 """
529
530 netplan_yaml = '''network:
531 renderer: NetworkManager
532 ethernets:
533 eth123:
534 dhcp4: true'''
535
536 with open('/etc/netplan/10-test.yaml', 'w') as f:
537 f.write(netplan_yaml)
538
539 self._netplan_generate();
540 self._nmcli_con_reload()
541 self.nmclient = NM.Client.new()
542
543 expected = None
544 for conn in self.nmclient.get_connections():
545 if conn.get_id() == 'netplan-eth123':
546 expected = conn
547
548 self.assertIsNotNone(expected)
549
550 def test_create_connection_via_netplan_and_remove_via_nmcli(self):
551 """
552 Create a connection via netplan generate and remove it with nmcli.
553
554 The interface should be removed from the yaml file.
555 """
556
557 netplan_yaml = '''network:
558 renderer: NetworkManager
559 ethernets:
560 eth123:
561 dhcp4: true
562 eth456:
563 dhcp4: true'''
564
565 with open('/etc/netplan/10-test.yaml', 'w') as f:
566 f.write(netplan_yaml)
567
568 self._netplan_generate();
569 self._nmcli_con_reload()
570 self.nmclient = NM.Client.new()
571
572 expected1 = None
573 expected2 = None
574 for conn in self.nmclient.get_connections():
575 if conn.get_id() == 'netplan-eth123':
576 expected1 = conn
577 if conn.get_id() == 'netplan-eth456':
578 expected2 = conn
579
580 self.assertIsNotNone(expected1)
581 self.assertIsNotNone(expected1)
582
583 self._delete_connection(expected1)
584
585 with open('/etc/netplan/10-test.yaml', 'r') as f:
586 yaml_data = yaml.safe_load(f)
587
588 # eth123 shouldn't exist anymore
589 self.assertIsNone(yaml_data.get('network').get('ethernets').get('eth123'))
590
591 self.assertIsNotNone(yaml_data.get('network').get('ethernets').get('eth456'))
592
593 def test_create_connection_via_netplan_and_change_it_via_nmcli(self):
594 """
595 Create a connection via netplan generate and change it via nmcli.
596 """
597
598 netplan_yaml = '''network:
599 renderer: NetworkManager
600 ethernets:
601 eth123:
602 dhcp4: false
603 dhcp6: false
604 eth456:
605 dhcp4: true'''
606
607 with open('/etc/netplan/10-test.yaml', 'w') as f:
608 f.write(netplan_yaml)
609
610 self._netplan_generate();
611 self._nmcli_con_reload()
612 self._nmcli(['con', 'mod', 'netplan-eth123', 'ipv4.method', 'auto'])
613
614 # eth123.dhcp4 should be overriden by 90-NM-<UUID>.yaml (from 10-test.yaml)
615 # The output of 'netplan get' should account for that.
616 out = subprocess.check_output(['netplan', 'get'], universal_newlines=True)
617 yaml_data = yaml.safe_load(out)
618 dhcp = yaml_data.get('network').get('ethernets').get('eth123').get('dhcp4')
619 self.assertTrue(dhcp)
620
621 def test_openvpn_connection(self):
622 """ Test case for LP#1998207"""
623
624 server_config = """dev tun
625ca /tmp/openvpn/pki/ca.crt
626cert /tmp/openvpn/pki/issued/server.crt
627key /tmp/openvpn/pki/private/server.key
628dh /tmp/openvpn/pki/dh.pem
629server 10.8.0.0 255.255.255.0
630keepalive 10 120
631cipher AES-256-GCM
632compress lz4-v2
633push "compress lz4-v2"
634user root
635log /tmp/openvpn.log
636group root
637"""
638
639 client_config = """client
640dev tun
641remote 127.0.0.1 1194
642nobind
643ca /tmp/openvpn/pki/ca.crt
644cert /tmp/openvpn/pki/issued/client.crt
645key /tmp/openvpn/pki/private/client.key
646cipher AES-256-GCM
647"""
648
649 # The minimum DH size accepted by OpenVPN these days is 2048.
650 # It might take a while to be generated (like almost a minute)
651 # It would be faster to use shared keys instead of TLS but it
652 # seems it's not an option anymore in OpenVPN
653 openvpn_spinup_script = """/usr/share/easy-rsa/easyrsa init-pki
654EASYRSA_BATCH=1 /usr/share/easy-rsa/easyrsa build-ca nopass
655EASYRSA_BATCH=1 /usr/share/easy-rsa/easyrsa build-server-full server nopass
656EASYRSA_BATCH=1 /usr/share/easy-rsa/easyrsa build-client-full client nopass
657/usr/share/easy-rsa/easyrsa gen-dh
658"""
659
660 tmpdir = '/tmp/openvpn'
661 self.addCleanup(shutil.rmtree, tmpdir)
662 os.mkdir(tmpdir)
663 os.chdir(tmpdir)
664
665 with open('openvpn_spinup.sh', 'w') as f:
666 f.write(openvpn_spinup_script)
667
668 with open('server.conf', 'w') as f:
669 f.write(server_config)
670
671 with open('client.conf', 'w') as f:
672 f.write(client_config)
673
674 cmd = ['bash', 'openvpn_spinup.sh']
675 subprocess.call(cmd, stdout=subprocess.DEVNULL)
676
677 openvpn_server_cmd = ['openvpn', '--config', 'server.conf']
678 p_server = subprocess.Popen(openvpn_server_cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
679 sleep(1) # Let's give OpenVPN a second to start
680
681 # Add a useless default route to the loopback interface so NM will allow starting the VPN connection.
682 # Apparently, if it doesn't own an *active connection* with a default route, it will not allow
683 # us to start the VPN client.
684 # As we set the main ethernet device as unmanaged, NM will not 'own' a default route when it starts.
685 # Unfortunately, by doing this, NM will take over the interface 'lo' and add a new yaml file to /etc/netplan
686 self._nmcli(['con', 'mod', 'lo', 'ipv4.gateway', '10.8.0.254'])
687
688 # Create an OpenVPN connection based on the configuration found in client.conf
689 self._nmcli(['con', 'import', 'type', 'openvpn', 'file', 'client.conf'])
690
691 # At this point we should have 2 yaml files
692 # One for the tun0 NM took over and one for the VPN connection
693 self.assertEqual(self._get_number_of_yaml_files(), 2,
694 msg='More than expected YAML files were found after creating the connection')
695
696 self._nmcli(['con', 'up', 'client'])
697 sleep(2) # Let's give NM a couple of seconds to settle down
698 # We still should have 2 files after starting the client
699 self.assertEqual(self._get_number_of_yaml_files(), 2,
700 msg='More than expected YAML files were found after starting the connection')
701
702 self._nmcli(['con', 'down', 'client'])
703 sleep(2) # Let's give NM a couple of seconds to settle down
704 # We still should have 2 files after stopping the client
705 self.assertEqual(self._get_number_of_yaml_files(), 2,
706 msg='More than expected YAML files were found after stopping the connection')
707
708 p_server.terminate()
709 p_server.wait()
710 # We still should have 2 files after stopping the server
711 self.assertEqual(self._get_number_of_yaml_files(), 2,
712 msg='More than expected YAML files were found after stopping the OpenVPN server')
713
714
715if __name__ == '__main__':
716 runner = unittest.TextTestRunner(stream=sys.stdout, verbosity=2)
717 unittest.main(testRunner=runner)

Subscribers

People subscribed via source and target branches

to all changes: