Merge ~slyon/network-manager:netplan-integration into network-manager:ubuntu/master
- Git
- lp:~slyon/network-manager
- netplan-integration
- Merge into ubuntu/master
Status: | Merged |
---|---|
Merged at revision: | bcd43413ecea3493bfbcf091b95508a4120ba7d3 |
Proposed branch: | ~slyon/network-manager:netplan-integration |
Merge into: | network-manager:ubuntu/master |
Diff against target: |
1558 lines (+1454/-1) 11 files modified
debian/changelog (+14/-0) debian/control (+4/-0) debian/network-manager.postinst (+33/-0) debian/network-manager.preinst (+18/-0) debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch (+134/-0) debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch (+525/-0) debian/patches/series (+2/-0) debian/rules (+2/-1) debian/tests/control (+4/-0) debian/tests/nm.py (+1/-0) debian/tests/nm_netplan.py (+717/-0) |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Sebastien Bacher | Approve | ||
Danilo Egea Gondolfo (community) | Approve | ||
Lukas Märdian | Abstain | ||
Review via email: mp+423735@code.launchpad.net |
Commit message
Description of the change
Deploying the NetworkManager-
This depends on libnetplan >= v0.106 and we cannot depend on libnetplan on i386, so we're dropping the patch to the NetworkManager daemon binary on i386, using quilt (that binary isn't built on i386 anyway).
On installation of the package any existing configuration from /etc/NetworkMan
On updade of the package, any existing configuration from /etc/NetworkMan
Netplan patches are managed via git-buildpackage patch-queue (Using "Gbp-Pq: Topic netplan")
Olivier Gayot (ogayot) wrote : | # |
Danilo Egea Gondolfo (danilogondolfo) wrote : | # |
Left one comment inline about the migration script.
Sebastien Bacher (seb128) wrote : | # |
Confirming the issue raised by Danilo, the script failed to migrate several connections using a ca-cert and pointing to a filename stored in directory with a 'é' in its name
it also displayed some warnings (translated from french so the wording might not be the correct english one)
> Warning: there is another connection with the name '...'. Refer to the connection using its uuid '...'
Sebastien Bacher (seb128) wrote : | # |
Unsure if that's specific to the change there or a new mantic issue but the autopkgtest against a ppa testbuild failed
'Error: failed to reload connections: GDBus.Error:
dpkg: error processing package network-manager (--configure):
installed network-manager package post-installation script subprocess returned error exit status 1'
Lukas Märdian (slyon) wrote : | # |
Thanks!
I've fixed the utf-8 (non-ASCII keyfiles) issue reported by @seb128 and @danilogondolfo as suggested by Danilo (i.e. checking for the "text/plain" mime type instead).
Thanks Sebastien for the "Error: failed to reload connections: GDBus.Error:
```
autopkgtest [15:07:36]: @@@@@@@
wpa-dhclient PASS
nm.py PASS
killswitches-
urfkill-integration PASS
nm_netplan.py PASS
qemu-system-x86_64: terminating on signal 15 from pid 1944756 (/usr/bin/python3)
```
For the "Warning: there is another connection with the name '...'. Refer to the connection using its uuid '...'" message, I don't really know where it comes from, as we're only modifying connections through nmcli, using their UUID identifiers... Could you try to provide a reproducer for that?
PTAL.
Sebastien Bacher (seb128) wrote : | # |
Thanks for the tweaks, I restored the connections from the backup dir, tried to the new revision and now hit
+ echo Migrating Ziggo (03c8f2a7-
Migrating Ziggo (03c8f2a7-
+ nmcli con mod 03c8f2a7-
Erreur : la modification de la connexion « Ziggo.NETPLAN_
dpkg: erreur de traitement du paquet network-manager (--install) :
le sous-processus paquet network-manager script post-installation installé a renvoyé un état de sortie d'erreur 1
there is a NetworkManager crash report from that time in /var/crash, it hit the
nms-keyfile-
> <error> [1683703791.7570] BUG: the profile cannot be stored in keyfile format without becoming unusable: invalid connection: 802-1x.identity: la propriété est manquante
and
> <warn> [1683703795.8560] keyfile: load: "/run/NetworkMa
I'm sending the corresponding file via email
So it seems we have a bug there that n-m is hitting an assertion, also it feels wrong than failing to migrate one connection is failing the package installation and letting the packaging system in a state where most users will not be able to recover from easily
Lukas Märdian (slyon) wrote : | # |
Thank you very much for the valuable feedback! I've rebased the branch to properly include the current fixes and moved forward to a v4 Netplan-integration patch, which makes use of "#if WITH_NETPLAN" inside the NM codebase, to allow "--enable-
Also, thanks for providing the test files by email. I agree we shouldn't fail the installation of the package when the migration of a single connection profile fails, it should just skip that connection and keep it as a keyfile. I'll be working on that!
I'll also investigate that crash, but I think we should not block on that one (once we are able to skip non-migrated connections). I'll try to collect all the information and create a corresponding bug report.
Lukas Märdian (slyon) wrote : | # |
So Danilo found the root cause of the "802-1x.identity" issue (crash), which is a bug in libnetplan (Netplan's keyfile parser). This will be handled via in bug #2016625 (details to follow there)
Lukas Märdian (slyon) wrote : | # |
I've just pushed another update to the branch, which handles migration failures gracefully:
Before migration it will create a temporary backup of the keyfile in and restore that keyfile backup, should the migration fail.
We're working on the crash (libnetplan bug) separately, as mentioned above. I've pushed the current state of this branch into the "Netplan Everywhere" PPA via https:/
Let me know what you think!
Sebastien Bacher (seb128) wrote : | # |
Thanks, I can confirm that the new changes make the package install without error and that it rollbacks the problematic connection. The autopkgtests issue is also resolve
The warning I mentioned earlier about the exiting name/refer by uid is printing by
> nmcli con mod daffb72f-
There is another connection with the same name. The command is correctly issued with the uid so I think the warning is safe to ignore
I will upload the changes to mantic today unless you would prefer to see the netplan fix landing first?
Danilo Egea Gondolfo (danilogondolfo) : | # |
Danilo Egea Gondolfo (danilogondolfo) wrote : | # |
I left an inline comment about some code we probably want to remove from the #if #endif block.
Lukas Märdian (slyon) wrote : | # |
Thank you for the additional reviews!
@seb128, please go ahead and upload the changes into Mantic, we'll be fixing (lib-)netplan in parallel.
@danilogondolfo, you're right it is NetworkManager code. But in fact we're not patching it out, but rather creating a copy of that block for the Netplan codepath. So we should be safe ignoring that whole block inside #if WITH_NETPLAN ... #endif if the Netplan buildflag is disabled. It is visible in the GitHub diff, see: https:/
Danilo Egea Gondolfo (danilogondolfo) : | # |
Sebastien Bacher (seb128) wrote : | # |
Thanks, merged and uploaded to Ubuntu now!
Preview Diff
1 | diff --git a/debian/changelog b/debian/changelog |
2 | index f362bb1..e2e79df 100644 |
3 | --- a/debian/changelog |
4 | +++ b/debian/changelog |
5 | @@ -1,3 +1,17 @@ |
6 | +network-manager (1.42.4-1ubuntu3) mantic; urgency=medium |
7 | + |
8 | + [ Lukas Märdian ] |
9 | + * d/{control,rules,patches}: Prepare libnetplan build (non i386) |
10 | + * debian/patches/netplan: Add libnetplan backend integration patch |
11 | + * d/network-manager.preinst: backup previous configuration automatically |
12 | + * d/network-manager.postinst: Trigger Netplan migration on install/upgrade |
13 | + |
14 | + [ Danilo Egea Gondolfo ] |
15 | + * d/t/nm.py: Fix autopkgtests when Netplan is in use |
16 | + * d/t/nm_netplan.py: Add autopkgtests for the netplan integration |
17 | + |
18 | + -- Lukas Märdian <slyon@ubuntu.com> Wed, 10 May 2023 11:56:58 +0200 |
19 | + |
20 | network-manager (1.42.4-1ubuntu2) lunar; urgency=medium |
21 | |
22 | * d/t/nm.py: Fix autopkgtests with NM-1.42's 'lo' connection (LP: #2009543) |
23 | diff --git a/debian/control b/debian/control |
24 | index 603370d..dd8bf9d 100644 |
25 | --- a/debian/control |
26 | +++ b/debian/control |
27 | @@ -38,6 +38,9 @@ Build-Depends: debhelper-compat (= 13), |
28 | python3-dbus <!nocheck>, |
29 | python3-pexpect <!nocheck>, |
30 | iproute2 <!nocheck>, |
31 | + libyaml-dev [!i386], |
32 | + libnetplan-dev (>= 0.106~) [!i386], |
33 | + netplan.io (>= 0.106~) [!i386] <!nocheck>, |
34 | Standards-Version: 4.6.2 |
35 | Rules-Requires-Root: no |
36 | XS-Debian-Vcs-Git: https://salsa.debian.org/utopia-team/network-manager.git |
37 | @@ -51,6 +54,7 @@ Architecture: linux-any |
38 | Pre-Depends: ${misc:Pre-Depends} |
39 | Depends: ${shlibs:Depends}, |
40 | ${misc:Depends}, |
41 | + netplan.io (>= 0.106~) [!i386], |
42 | libnm0 (= ${binary:Version}), |
43 | isc-dhcp-client, |
44 | default-dbus-system-bus | dbus-system-bus, |
45 | diff --git a/debian/network-manager.postinst b/debian/network-manager.postinst |
46 | index e2cb049..0db377b 100644 |
47 | --- a/debian/network-manager.postinst |
48 | +++ b/debian/network-manager.postinst |
49 | @@ -81,3 +81,36 @@ esac |
50 | |
51 | #DEBHELPER# |
52 | |
53 | +# Run "Netplan Everywhere" migration after debhelper (re-)started |
54 | +# NetworkManager.service for us. On every package upgrade. |
55 | +DIR="/etc/NetworkManager/system-connections" |
56 | +if [ "$1" = "configure" ] && [ -d "$DIR" ]; then |
57 | + mkdir -p /run/netplan/nm-migrate |
58 | + for CON in /etc/NetworkManager/system-connections/*; do |
59 | + TYPE=$(file -bi "$CON" | cut -s -d ";" -f 1) |
60 | + [ "$TYPE" = "text/plain" ] || continue # skip non-keyfiles |
61 | + UUID=$(grep "^uuid=" "$CON" | cut -c 6-) |
62 | + if [ -n "$UUID" ] |
63 | + then |
64 | + # Wait for NetworkManager startup to complete, |
65 | + # so we can safely use nmcli. Wait in every interation to handle |
66 | + # a crashed NetworkManager in the previous migraiton step. |
67 | + nm-online -qs || (echo "SKIP: NetworkManager is not ready ..." 1>&2 && continue) |
68 | + BACKUP="/run/netplan/nm-migrate/"$(basename "$CON") |
69 | + cp "$CON" "$BACKUP" |
70 | + ORIG_NAME=$(nmcli --get-values connection.id con show "$UUID") |
71 | + echo "Migrating $ORIG_NAME ($UUID) to /etc/netplan" 1>&2 |
72 | + # Touch the connection's ID (con-name) to trigger its migration. |
73 | + # The Netplan integration will translate the original NM keyfile from |
74 | + # /etc/NetworkManager/system-connections/* to a YAML file located in |
75 | + # /etc/netplan/90-NM-*.yaml and re-generate a corresponding keyfile in |
76 | + # /run/NetworkManager/system-connections/netplan-NM-*.nmconnection |
77 | + nmcli con mod "$UUID" con-name "$ORIG_NAME" || \ |
78 | + (echo "FAILED. Restoring backup ..." 1>&2 && mv "$BACKUP" "$CON" && \ |
79 | + rm -f "/etc/netplan/90-NM-$UUID"*.yaml) |
80 | + rm -f "$BACKUP" # clear backup (if it still exists) |
81 | + fi |
82 | + done |
83 | + rm -rf /run/netplan/nm-migrate # cleanup after ourselves |
84 | + (nm-online -qs && nmcli con reload) || echo "WARNING: NetworkManager could not reload connections ..." 1>&2 |
85 | +fi |
86 | diff --git a/debian/network-manager.preinst b/debian/network-manager.preinst |
87 | new file mode 100644 |
88 | index 0000000..886ab77 |
89 | --- /dev/null |
90 | +++ b/debian/network-manager.preinst |
91 | @@ -0,0 +1,18 @@ |
92 | +#!/bin/sh |
93 | + |
94 | +set -e |
95 | + |
96 | +#DEBHELPER# |
97 | + |
98 | +DIR="/etc/NetworkManager/system-connections" |
99 | +CNT=$(ls -1 "$DIR" | wc -l) |
100 | +if ([ "$1" = "upgrade" ] || [ "$1" = "install" ]) && [ -d "$DIR" ] && [ "$CNT" -ge 1 ]; then |
101 | + # create backup directory if it does not yet exist |
102 | + mkdir -p /root/NetworkManager.bak || true |
103 | + BAK="/root/NetworkManager.bak/system-connections_$2" |
104 | + if [ -d "$BAK" ]; then |
105 | + rm -r "$BAK" |
106 | + fi |
107 | + # copy current system-connections to the backup directory |
108 | + cp -r "$DIR" "$BAK" |
109 | +fi |
110 | diff --git a/debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch b/debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch |
111 | new file mode 100644 |
112 | index 0000000..f3ae212 |
113 | --- /dev/null |
114 | +++ b/debian/patches/netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch |
115 | @@ -0,0 +1,134 @@ |
116 | +From 9dd31963bd89ccba5e96f26506996e792177bdab Mon Sep 17 00:00:00 2001 |
117 | +From: Lukas Märdian <slyon@ubuntu.com> |
118 | +Date: Tue, 9 May 2023 17:09:48 +0200 |
119 | +Subject: [PATCH 1/2] netplan: Adopt buildsystems for Netplan integration |
120 | + |
121 | +Autotools and Meson will define a "WITH_NETPLAN" variable with the |
122 | +values 1 or 0 accordingly, using the config.h header. |
123 | +--- |
124 | + Makefile.am | 4 +++- |
125 | + configure.ac | 17 +++++++++++++++++ |
126 | + meson.build | 10 ++++++++++ |
127 | + meson_options.txt | 1 + |
128 | + src/core/meson.build | 1 + |
129 | + 5 files changed, 32 insertions(+), 1 deletion(-) |
130 | + |
131 | +diff --git a/Makefile.am b/Makefile.am |
132 | +index a452cacac4..677c078655 100644 |
133 | +--- a/Makefile.am |
134 | ++++ b/Makefile.am |
135 | +@@ -2588,7 +2588,8 @@ $(src_core_libNetworkManagerBase_la_OBJECTS): $(src_libnm_core_public_mkenums_h) |
136 | + |
137 | + ############################################################################### |
138 | + |
139 | +-src_core_libNetworkManager_la_CPPFLAGS = $(src_core_cppflags) |
140 | ++src_core_libNetworkManager_la_CPPFLAGS = $(src_core_cppflags) \ |
141 | ++ $(NETPLAN_CFLAGS) |
142 | + |
143 | + src_core_libNetworkManager_la_SOURCES = \ |
144 | + \ |
145 | +@@ -2803,6 +2804,7 @@ src_core_libNetworkManager_la_LIBADD = \ |
146 | + $(LIBAUDIT_LIBS) \ |
147 | + $(LIBPSL_LIBS) \ |
148 | + $(LIBCURL_LIBS) \ |
149 | ++ $(NETPLAN_LIBS) \ |
150 | + $(NULL) |
151 | + |
152 | + $(src_core_libNetworkManager_la_OBJECTS): $(src_libnm_core_public_mkenums_h) |
153 | +diff --git a/configure.ac b/configure.ac |
154 | +index b0cf78dc03..6b4c1467b6 100644 |
155 | +--- a/configure.ac |
156 | ++++ b/configure.ac |
157 | +@@ -880,6 +880,22 @@ else |
158 | + AC_DEFINE(WITH_OPENVSWITCH, 0, [Whether we build with OVS plugin]) |
159 | + fi |
160 | + |
161 | ++# Netplan integration |
162 | ++AC_ARG_ENABLE(netplan, AS_HELP_STRING([--enable-netplan], [Enable Netplan integration])) |
163 | ++if test "${enable_netplan}" != "no"; then |
164 | ++ PKG_CHECK_MODULES(NETPLAN, [netplan >= 0.106], [enable_netplan=yes], [enable_netplan=no]) |
165 | ++ if test "$enable_netplan" != "yes"; then |
166 | ++ AC_MSG_ERROR(Netplan is required) |
167 | ++ fi |
168 | ++ enable_netplan='yes' |
169 | ++fi |
170 | ++AM_CONDITIONAL(WITH_NETPLAN, test "${enable_netplan}" = "yes") |
171 | ++if test "${enable_netplan}" = "yes" ; then |
172 | ++ AC_DEFINE(WITH_NETPLAN, 1, [Whether we build with Netplan integration]) |
173 | ++else |
174 | ++ AC_DEFINE(WITH_NETPLAN, 0, [Whether we build with Netplan integration]) |
175 | ++fi |
176 | ++ |
177 | + # DHCP client support |
178 | + AC_ARG_WITH([dhclient], |
179 | + AS_HELP_STRING([--with-dhclient=yes|no|path], [Enable dhclient support])) |
180 | +@@ -1400,6 +1416,7 @@ echo " ofono: $with_ofono" |
181 | + echo " concheck: $enable_concheck" |
182 | + echo " libteamdctl: $enable_teamdctl" |
183 | + echo " ovs: $enable_ovs" |
184 | ++echo " netplan: $enable_netplan" |
185 | + echo " nmcli: $build_nmcli" |
186 | + echo " nmtui: $build_nmtui" |
187 | + echo " nm-cloud-setup: $with_nm_cloud_setup" |
188 | +diff --git a/meson.build b/meson.build |
189 | +index 6813e52ac1..e808729c22 100644 |
190 | +--- a/meson.build |
191 | ++++ b/meson.build |
192 | +@@ -274,6 +274,8 @@ config_h.set10('HAVE_LIBSYSTEMD', libsystemd_dep.found()) |
193 | + systemd_dep = dependency('systemd', required: false) |
194 | + have_systemd_200 = systemd_dep.found() and systemd_dep.version().version_compare('>= 200') |
195 | + |
196 | ++libnetplan_dep = dependency('netplan', version: '>= 0.106', required: false) |
197 | ++ |
198 | + gio_unix_dep = dependency('gio-unix-2.0', version: '>= 2.40') |
199 | + |
200 | + glib_dep = declare_dependency( |
201 | +@@ -646,6 +648,13 @@ if enable_ovs |
202 | + endif |
203 | + config_h.set10('WITH_OPENVSWITCH', enable_ovs) |
204 | + |
205 | ++# Netplan integration |
206 | ++enable_netplan = get_option('netplan') |
207 | ++if enable_netplan |
208 | ++ assert(libnetplan_dep.found(), 'libnetplan is needed for Netplan integration.') |
209 | ++endif |
210 | ++config_h.set10('WITH_NETPLAN', enable_netplan) |
211 | ++ |
212 | + # DNS resolv.conf managers |
213 | + config_dns_rc_manager_default = get_option('config_dns_rc_manager_default') |
214 | + config_h.set_quoted('NM_CONFIG_DEFAULT_MAIN_RC_MANAGER', config_dns_rc_manager_default) |
215 | +@@ -1064,6 +1073,7 @@ output += ' ofono: ' + enable_ofono.to_string() + '\n' |
216 | + output += ' concheck: ' + enable_concheck.to_string() + '\n' |
217 | + output += ' libteamdctl: ' + enable_teamdctl.to_string() + '\n' |
218 | + output += ' ovs: ' + enable_ovs.to_string() + '\n' |
219 | ++output += ' netplan: ' + enable_netplan.to_string() + '\n' |
220 | + output += ' nmcli: ' + enable_nmcli.to_string() + '\n' |
221 | + output += ' nmtui: ' + enable_nmtui.to_string() + '\n' |
222 | + output += ' nm-cloud-setup: ' + enable_nm_cloud_setup.to_string() + '\n' |
223 | +diff --git a/meson_options.txt b/meson_options.txt |
224 | +index 4e359f9e92..9f52b4f6e4 100644 |
225 | +--- a/meson_options.txt |
226 | ++++ b/meson_options.txt |
227 | +@@ -37,6 +37,7 @@ option('ofono', type: 'boolean', value: false, description: 'Enable oFono suppor |
228 | + option('concheck', type: 'boolean', value: true, description: 'enable connectivity checking support') |
229 | + option('teamdctl', type: 'boolean', value: false, description: 'enable Teamd control support') |
230 | + option('ovs', type: 'boolean', value: true, description: 'enable Open vSwitch support') |
231 | ++option('netplan', type: 'boolean', value: true, description: 'Enable Netplan integration') |
232 | + option('nmcli', type: 'boolean', value: true, description: 'Build nmcli') |
233 | + option('nmtui', type: 'boolean', value: true, description: 'Build nmtui') |
234 | + option('nm_cloud_setup', type: 'boolean', value: false, description: 'Build nm-cloud-setup, a tool for automatically configure networking in cloud (EXPERIMENTAL!)') |
235 | +diff --git a/src/core/meson.build b/src/core/meson.build |
236 | +index 6c1d463ed1..35ece13f06 100644 |
237 | +--- a/src/core/meson.build |
238 | ++++ b/src/core/meson.build |
239 | +@@ -71,6 +71,7 @@ nm_deps = [ |
240 | + libndp_dep, |
241 | + libudev_dep, |
242 | + logind_dep, |
243 | ++ libnetplan_dep, |
244 | + ] |
245 | + |
246 | + if enable_concheck |
247 | +-- |
248 | +2.39.2 |
249 | + |
250 | diff --git a/debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch b/debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch |
251 | new file mode 100644 |
252 | index 0000000..1d959f2 |
253 | --- /dev/null |
254 | +++ b/debian/patches/netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch |
255 | @@ -0,0 +1,525 @@ |
256 | +From aaa791cb61b24a9416db18f2290f711bc9a8f26e Mon Sep 17 00:00:00 2001 |
257 | +From: Lukas Märdian <lukas.maerdian@canonical.com> |
258 | +Date: Tue, 2 Feb 2021 15:52:05 +0100 |
259 | +Subject: [PATCH 2/2] netplan: make use of libnetplan for YAML backend |
260 | + |
261 | +Origin: https://github.com/slyon/NetworkManager/tree/netplan-nm-1.42 |
262 | +Forwarded: no, https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/556 |
263 | +Last-Update: 2023-05-11 |
264 | + |
265 | +This patch modifies NetworkManager's nms-keyfile-plugin in a way to |
266 | +write YAML connections (according to the netplan spec) to |
267 | +/etc/netplan/*.yaml instead of NM's native keyfile connection profiles |
268 | +in /etc/NetworkManager/system-connections/*.nmconnection. |
269 | + |
270 | +Whenever a connection profile is to be written (add/modify) the keyfile, |
271 | +generated internally by NM, is passed into libnetplan's |
272 | +"netplan_parser_load_keyfile()" API, validated via |
273 | +"netplan_state_import_parser_results()" and converted to a netplan YAML |
274 | +config by calling libnetplan's "netplan_netdef_write_yaml()" API. The |
275 | +internal keyfile is thrown away afterwards. |
276 | + |
277 | +Whenever a connection profile is to be deleted the netplan-/netdef-id is |
278 | +extracted from the ephemeral keyfile in /run/NetworkManager/system-connections |
279 | +via "netplan_get_id_from_nm_filepath()" and the corresponding YAML is |
280 | +updated/deleted by calling "netplan_delete_connection()". |
281 | + |
282 | +Each time the YAML data was modified, NetworkManager calls |
283 | +"netplan generate" to produce new ephemeral keyfile connections in |
284 | +/run/NetworkManager/system-connections for NM to read-back. This way the |
285 | +netplan generator can be used as intended (no need for duplicated |
286 | +keyfile export functionality) and the nms-keyfile-writer can be re-used |
287 | +without any patching needed. |
288 | + |
289 | +V2: |
290 | ++ ported to NetworkManager 1.36.6 (Ubuntu Jammy LTS/Core 22.04) |
291 | ++ test-keyfile-settings.c: clear netplan YAML config from previous runs |
292 | ++ nms-keyfile-writer.c: avoid double-free of `path` on exit, caused by gs_free |
293 | + |
294 | +V3: |
295 | ++ ported to NM 1.40.6 (Ubuntu Lunar), using new libnetplan API (v0.106) |
296 | ++ ignore .nm-generated connections (LP: #1998207) |
297 | + |
298 | +V4: |
299 | ++ encapsulated all Netplan code in "#if WITH_NETPLAN" as provided by the |
300 | + buildsystems via config.h (see previous commit) |
301 | ++ nms-keyfile-writer.c: increase buffer size to account for ".nmconnection" |
302 | + suffix in netplan_netdef_get_output_filename() |
303 | + |
304 | +Co-authored-by: Alfonso Sanchez-Beato <alfonso.sanchez-beato@canonical.com> |
305 | +Co-authored-by: Danilo Egea Gondolfo <danilo.egea.gondolfo@canonical.com> |
306 | +--- |
307 | + .../plugins/keyfile/nms-keyfile-plugin.c | 34 ++++ |
308 | + .../plugins/keyfile/nms-keyfile-utils.c | 18 ++ |
309 | + .../plugins/keyfile/nms-keyfile-utils.h | 4 + |
310 | + .../plugins/keyfile/nms-keyfile-writer.c | 173 ++++++++++++++++++ |
311 | + .../keyfile/tests/test-keyfile-settings.c | 61 +++++- |
312 | + 5 files changed, 286 insertions(+), 4 deletions(-) |
313 | + |
314 | +diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c b/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c |
315 | +index 1d7de8d24b..4b986dc0c8 100644 |
316 | +--- a/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c |
317 | ++++ b/src/core/settings/plugins/keyfile/nms-keyfile-plugin.c |
318 | +@@ -12,6 +12,9 @@ |
319 | + #include <unistd.h> |
320 | + #include <sys/types.h> |
321 | + #include <sys/time.h> |
322 | ++#if WITH_NETPLAN |
323 | ++#include <netplan/util.h> |
324 | ++#endif |
325 | + |
326 | + #include "libnm-std-aux/c-list-util.h" |
327 | + #include "libnm-glib-aux/nm-c-list.h" |
328 | +@@ -309,6 +312,12 @@ _load_file(NMSKeyfilePlugin *self, |
329 | + gs_free char *full_filename = NULL; |
330 | + struct stat st; |
331 | + |
332 | ++ #if WITH_NETPLAN |
333 | ++ // Handle all netplan generated connections via STORAGE_TYPE_ETC, as they live in /etc/netplan |
334 | ++ if (g_str_has_prefix(filename, "netplan-")) |
335 | ++ storage_type = NMS_KEYFILE_STORAGE_TYPE_ETC; |
336 | ++ #endif |
337 | ++ |
338 | + if (_ignore_filename(storage_type, filename)) { |
339 | + gs_free char *nmmeta = NULL; |
340 | + gs_free char *loaded_path = NULL; |
341 | +@@ -584,6 +593,9 @@ reload_connections(NMSettingsPlugin *plugin, |
342 | + NM_SETT_UTIL_STORAGES_INIT(storages_new, nms_keyfile_storage_destroy); |
343 | + int i; |
344 | + |
345 | ++ #if WITH_NETPLAN |
346 | ++ generate_netplan(NULL); |
347 | ++ #endif |
348 | + _load_dir(self, NMS_KEYFILE_STORAGE_TYPE_RUN, priv->dirname_run, &storages_new); |
349 | + if (priv->dirname_etc) |
350 | + _load_dir(self, NMS_KEYFILE_STORAGE_TYPE_ETC, priv->dirname_etc, &storages_new); |
351 | +@@ -1008,6 +1020,15 @@ delete_connection(NMSettingsPlugin *plugin, NMSettingsStorage *storage_x, GError |
352 | + previous_filename = nms_keyfile_storage_get_filename(storage); |
353 | + uuid = nms_keyfile_storage_get_uuid(storage); |
354 | + |
355 | ++ #if WITH_NETPLAN |
356 | ++ nm_auto_unref_keyfile GKeyFile *key_file = NULL; |
357 | ++ key_file = g_key_file_new (); |
358 | ++ if (!g_key_file_load_from_file (key_file, previous_filename, G_KEY_FILE_NONE, error)) |
359 | ++ return FALSE; |
360 | ++ g_autofree gchar* ssid = NULL; |
361 | ++ ssid = g_key_file_get_string(key_file, "wifi", "ssid", NULL); |
362 | ++ #endif |
363 | ++ |
364 | + if (!NM_IN_SET(storage->storage_type, |
365 | + NMS_KEYFILE_STORAGE_TYPE_ETC, |
366 | + NMS_KEYFILE_STORAGE_TYPE_RUN)) { |
367 | +@@ -1033,6 +1054,19 @@ delete_connection(NMSettingsPlugin *plugin, NMSettingsStorage *storage_x, GError |
368 | + } else |
369 | + operation_message = "deleted from disk"; |
370 | + |
371 | ++ #if WITH_NETPLAN |
372 | ++ g_autofree gchar *netplan_id = NULL; |
373 | ++ ssize_t netplan_id_size = 0; |
374 | ++ |
375 | ++ netplan_id = g_malloc0(strlen(previous_filename)); |
376 | ++ netplan_id_size = netplan_get_id_from_nm_filepath(previous_filename, ssid, netplan_id, strlen(previous_filename) - 1); |
377 | ++ if (netplan_id_size > 0) { |
378 | ++ _LOGI ("deleting netplan connection: %s", netplan_id); |
379 | ++ netplan_delete_connection(netplan_id, NULL); |
380 | ++ generate_netplan(NULL); |
381 | ++ } |
382 | ++ #endif |
383 | ++ |
384 | + _LOGT("commit: deleted \"%s\", %s %s (%s%s%s%s)", |
385 | + previous_filename, |
386 | + storage->is_meta_data ? "meta-data" : "profile", |
387 | +diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-utils.c b/src/core/settings/plugins/keyfile/nms-keyfile-utils.c |
388 | +index 7c0e329e2d..a91ee6997a 100644 |
389 | +--- a/src/core/settings/plugins/keyfile/nms-keyfile-utils.c |
390 | ++++ b/src/core/settings/plugins/keyfile/nms-keyfile-utils.c |
391 | +@@ -399,3 +399,21 @@ nms_keyfile_utils_check_file_permissions(NMSKeyfileFiletype filetype, |
392 | + NM_SET_OUT(out_st, st); |
393 | + return TRUE; |
394 | + } |
395 | ++ |
396 | ++#if WITH_NETPLAN |
397 | ++gboolean |
398 | ++generate_netplan(const char* rootdir) |
399 | ++{ |
400 | ++ /* TODO: call the io.netplan.Netplan.Generate() DBus method directly, after |
401 | ++ * finding a way to pass the --root-dir parameter via DBus, to make it work |
402 | ++ * inside NM's unit-tests where netplan needs to read & generate outside of |
403 | ++ * /etc/netplan and /run/{systemd,NetworkManager} */ |
404 | ++ const gchar *argv[] = { "netplan", "generate", NULL , NULL, NULL }; |
405 | ++ if (rootdir) { |
406 | ++ argv[2] = "--root-dir"; |
407 | ++ argv[3] = rootdir; |
408 | ++ } |
409 | ++ return g_spawn_sync(NULL, (gchar**)argv, NULL, G_SPAWN_SEARCH_PATH, |
410 | ++ NULL, NULL, NULL, NULL, NULL, NULL); |
411 | ++} |
412 | ++#endif |
413 | +diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-utils.h b/src/core/settings/plugins/keyfile/nms-keyfile-utils.h |
414 | +index 0fd83bdf35..9ed25d9cfd 100644 |
415 | +--- a/src/core/settings/plugins/keyfile/nms-keyfile-utils.h |
416 | ++++ b/src/core/settings/plugins/keyfile/nms-keyfile-utils.h |
417 | +@@ -68,4 +68,8 @@ gboolean nms_keyfile_utils_check_file_permissions(NMSKeyfileFiletype filetype, |
418 | + struct stat *out_st, |
419 | + GError **error); |
420 | + |
421 | ++#if WITH_NETPLAN |
422 | ++gboolean generate_netplan(const char* rootdir); |
423 | ++#endif |
424 | ++ |
425 | + #endif /* __NMS_KEYFILE_UTILS_H__ */ |
426 | +diff --git a/src/core/settings/plugins/keyfile/nms-keyfile-writer.c b/src/core/settings/plugins/keyfile/nms-keyfile-writer.c |
427 | +index ad6f277ce3..e5015ab74b 100644 |
428 | +--- a/src/core/settings/plugins/keyfile/nms-keyfile-writer.c |
429 | ++++ b/src/core/settings/plugins/keyfile/nms-keyfile-writer.c |
430 | +@@ -12,6 +12,14 @@ |
431 | + #include <sys/stat.h> |
432 | + #include <unistd.h> |
433 | + |
434 | ++#if WITH_NETPLAN |
435 | ++#include <net/if.h> |
436 | ++#include <netplan/parse.h> |
437 | ++#include <netplan/parse-nm.h> |
438 | ++#include <netplan/util.h> |
439 | ++#include <netplan/netplan.h> |
440 | ++#endif |
441 | ++ |
442 | + #include "libnm-core-intern/nm-keyfile-internal.h" |
443 | + |
444 | + #include "nms-keyfile-utils.h" |
445 | +@@ -201,6 +209,9 @@ _internal_write_connection(NMConnection *connection, |
446 | + char **out_path, |
447 | + NMConnection **out_reread, |
448 | + gboolean *out_reread_same, |
449 | ++ #if WITH_NETPLAN |
450 | ++ const char *rootdir, |
451 | ++ #endif |
452 | + GError **error) |
453 | + { |
454 | + nm_auto_unref_keyfile GKeyFile *kf_file = NULL; |
455 | +@@ -409,11 +420,161 @@ _internal_write_connection(NMConnection *connection, |
456 | + if (existing_path && !existing_path_read_only && !nm_streq(path, existing_path)) |
457 | + unlink(existing_path); |
458 | + |
459 | ++ #if WITH_NETPLAN |
460 | ++ NetplanParser *npp = NULL; |
461 | ++ NetplanState *np_state = NULL; |
462 | ++ |
463 | ++ /* NETPLAN: write only non-temporary files to /etc/netplan/... */ |
464 | ++ if (!is_volatile && !is_nm_generated && !is_external && |
465 | ++ strstr(keyfile_dir, "/etc/NetworkManager/system-connections")) { |
466 | ++ g_autofree gchar *ssid = g_key_file_get_string(kf_file, "wifi", "ssid", NULL); |
467 | ++ g_autofree gchar *escaped_ssid = ssid ? |
468 | ++ g_uri_escape_string(ssid, NULL, TRUE) : NULL; |
469 | ++ g_autofree gchar *netplan_id = NULL; |
470 | ++ ssize_t netplan_id_size = 0; |
471 | ++ NetplanNetDefinition *netdef = NULL; |
472 | ++ NetplanStateIterator state_iter; |
473 | ++ const gchar* kf_path = path; |
474 | ++ |
475 | ++ if (existing_path && strstr(existing_path, "system-connections/netplan-")) { |
476 | ++ netplan_id = g_malloc0(strlen(existing_path)); |
477 | ++ netplan_id_size = netplan_get_id_from_nm_filepath(existing_path, ssid, netplan_id, strlen(existing_path) - 1); |
478 | ++ if (netplan_id_size <= 0) { |
479 | ++ g_free(netplan_id); |
480 | ++ netplan_id = NULL; |
481 | ++ } |
482 | ++ } |
483 | ++ |
484 | ++ if (netplan_id && existing_path) { |
485 | ++ GFile* from = g_file_new_for_path(path); |
486 | ++ GFile* to = g_file_new_for_path(existing_path); |
487 | ++ g_file_copy(from, to, G_FILE_COPY_OVERWRITE, NULL, NULL, NULL, NULL); |
488 | ++ kf_path = existing_path; |
489 | ++ } |
490 | ++ |
491 | ++ // push keyfile into libnetplan for parsing (using existing_path, if available, |
492 | ++ // to be able to extract the original netdef_id and override existing settings) |
493 | ++ npp = netplan_parser_new(); |
494 | ++ |
495 | ++ if (!netplan_parser_load_keyfile(npp, kf_path, &local_err)) { |
496 | ++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED, |
497 | ++ "netplan: YAML translation failed: %s", local_err->message); |
498 | ++ goto netplan_parser_error; |
499 | ++ } |
500 | ++ |
501 | ++ np_state = netplan_state_new(); |
502 | ++ if (!netplan_state_import_parser_results(np_state, npp, &local_err)) { |
503 | ++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED, |
504 | ++ "netplan: YAML validation failed: %s", local_err->message); |
505 | ++ goto netplan_error; |
506 | ++ } |
507 | ++ |
508 | ++ netplan_state_iterator_init(np_state, &state_iter); |
509 | ++ /* At this point we have a single netdef in the netplan state */ |
510 | ++ netdef = netplan_state_iterator_next(&state_iter); |
511 | ++ |
512 | ++ if (!netdef) { |
513 | ++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED, |
514 | ++ "netplan: Netplan state has no network definitions"); |
515 | ++ goto netplan_error; |
516 | ++ } |
517 | ++ |
518 | ++ if (!netplan_netdef_write_yaml(np_state, netdef, rootdir, &local_err)) { |
519 | ++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED, |
520 | ++ "netplan: Failed to generate YAML: %s", local_err->message); |
521 | ++ goto netplan_error; |
522 | ++ } |
523 | ++ |
524 | ++ /* Delete same connection-profile provided by legacy netplan plugin. |
525 | ++ * TODO: drop legacy connection handling after 24.04 LTS */ |
526 | ++ g_autofree gchar* legacy_path = NULL; |
527 | ++ legacy_path = g_strdup_printf("/etc/netplan/NM-%s.yaml", nm_connection_get_uuid (connection)); |
528 | ++ if (g_file_test(legacy_path, G_FILE_TEST_EXISTS | G_FILE_TEST_IS_REGULAR)) { |
529 | ++ g_debug("Deleting legacy netplan connection: %s", legacy_path); |
530 | ++ unlink(legacy_path); |
531 | ++ } |
532 | ++ |
533 | ++ /* Clear original keyfile in /etc/NetworkManager/system-connections/, |
534 | ++ * we've written the /etc/netplan/*.yaml file instead. */ |
535 | ++ unlink(path); |
536 | ++ if (!generate_netplan(rootdir)) { |
537 | ++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED, |
538 | ++ "netplan generate failed"); |
539 | ++ goto netplan_error; |
540 | ++ } |
541 | ++ |
542 | ++ // Calculating the maximum space needed to store the new keyfile path |
543 | ++ ssize_t path_size = strlen(path) + strlen(nm_connection_get_uuid(connection)) + IF_NAMESIZE + 1; |
544 | ++ if (escaped_ssid) |
545 | ++ path_size += strlen(escaped_ssid); |
546 | ++ path_size += 50; // give some extra buffer, e.g. when going from ConName to ConName.nmconnection |
547 | ++ |
548 | ++ g_free(path); |
549 | ++ path = g_malloc0(path_size); |
550 | ++ path_size = netplan_netdef_get_output_filename(netdef, ssid, path, path_size); |
551 | ++ |
552 | ++ if (path_size <= 0) { |
553 | ++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED, |
554 | ++ "netplan: couldn't determine the keyfile path"); |
555 | ++ goto netplan_error; |
556 | ++ } |
557 | ++ |
558 | ++ if (rootdir) { |
559 | ++ char* final_path = g_build_path(G_DIR_SEPARATOR_S, rootdir, path, NULL); |
560 | ++ g_free(path); |
561 | ++ path = final_path; |
562 | ++ } |
563 | ++ |
564 | ++ netplan_state_clear(&np_state); |
565 | ++ netplan_parser_clear(&npp); |
566 | ++ |
567 | ++ /* re-read again: this time the connection profile newly generated by netplan in /run/... */ |
568 | ++ if ( out_reread |
569 | ++ || out_reread_same) { |
570 | ++ gs_free_error GError *reread_error = NULL; |
571 | ++ |
572 | ++ //XXX: why does the _from_keyfile function behave differently? |
573 | ++ //reread = nms_keyfile_reader_from_keyfile (kf_file, path, NULL, profile_dir, FALSE, &reread_error); |
574 | ++ reread = nms_keyfile_reader_from_file (path, profile_dir, NULL, NULL, NULL, NULL, NULL, NULL, &reread_error); |
575 | ++ |
576 | ++ if ( !reread |
577 | ++ || !nm_connection_normalize (reread, NULL, NULL, &reread_error)) { |
578 | ++ nm_log_err (LOGD_SETTINGS, "BUG: the profile cannot be stored in keyfile format without becoming unusable: %s", reread_error->message); |
579 | ++ g_set_error (error, NM_SETTINGS_ERROR, NM_SETTINGS_ERROR_FAILED, |
580 | ++ "keyfile writer produces an invalid connection: %s", |
581 | ++ reread_error->message); |
582 | ++ nm_assert_not_reached (); |
583 | ++ return FALSE; |
584 | ++ } |
585 | ++ |
586 | ++ if (out_reread_same) { |
587 | ++ reread_same = !!nm_connection_compare (reread, connection, NM_SETTING_COMPARE_FLAG_EXACT); |
588 | ++ |
589 | ++ nm_assert (reread_same == nm_connection_compare (connection, reread, NM_SETTING_COMPARE_FLAG_EXACT)); |
590 | ++ nm_assert (reread_same == ({ |
591 | ++ gs_unref_hashtable GHashTable *_settings = NULL; |
592 | ++ |
593 | ++ ( nm_connection_diff (reread, connection, NM_SETTING_COMPARE_FLAG_EXACT, &_settings) |
594 | ++ && !_settings); |
595 | ++ })); |
596 | ++ } |
597 | ++ } |
598 | ++ } |
599 | ++ #endif |
600 | ++ |
601 | + NM_SET_OUT(out_reread, g_steal_pointer(&reread)); |
602 | + NM_SET_OUT(out_reread_same, reread_same); |
603 | + NM_SET_OUT(out_path, g_steal_pointer(&path)); |
604 | + |
605 | + return TRUE; |
606 | ++ |
607 | ++#if WITH_NETPLAN |
608 | ++netplan_error: |
609 | ++ netplan_state_clear(&np_state); |
610 | ++netplan_parser_error: |
611 | ++ netplan_parser_clear(&npp); |
612 | ++ return FALSE; |
613 | ++#endif |
614 | + } |
615 | + |
616 | + gboolean |
617 | +@@ -454,6 +615,9 @@ nms_keyfile_writer_connection(NMConnection *connection, |
618 | + out_path, |
619 | + out_reread, |
620 | + out_reread_same, |
621 | ++ #if WITH_NETPLAN |
622 | ++ NULL, |
623 | ++ #endif |
624 | + error); |
625 | + } |
626 | + |
627 | +@@ -467,6 +631,12 @@ nms_keyfile_writer_test_connection(NMConnection *connection, |
628 | + gboolean *out_reread_same, |
629 | + GError **error) |
630 | + { |
631 | ++ #if WITH_NETPLAN |
632 | ++ gchar *rootdir = g_strdup(keyfile_dir); |
633 | ++ if (g_str_has_suffix (keyfile_dir, "/run/NetworkManager/system-connections")) { |
634 | ++ rootdir[strlen(rootdir)-38] = '\0'; /* 38 = strlen("/run/NetworkManager/...") */ |
635 | ++ } |
636 | ++ #endif |
637 | + return _internal_write_connection(connection, |
638 | + FALSE, |
639 | + FALSE, |
640 | +@@ -486,5 +656,8 @@ nms_keyfile_writer_test_connection(NMConnection *connection, |
641 | + out_path, |
642 | + out_reread, |
643 | + out_reread_same, |
644 | ++ #if WITH_NETPLAN |
645 | ++ rootdir, |
646 | ++ #endif |
647 | + error); |
648 | + } |
649 | +diff --git a/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c b/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c |
650 | +index 83019babb1..eb59a91eab 100644 |
651 | +--- a/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c |
652 | ++++ b/src/core/settings/plugins/keyfile/tests/test-keyfile-settings.c |
653 | +@@ -24,8 +24,15 @@ |
654 | + |
655 | + #include "nm-test-utils-core.h" |
656 | + |
657 | ++#if WITH_NETPLAN |
658 | ++#define TEST_KEYFILES_DIR_OLD NM_BUILD_SRCDIR"/src/core/settings/plugins/keyfile/tests/keyfiles" |
659 | ++#define TEST_SCRATCH_DIR_OLD NM_BUILD_BUILDDIR"/src/core/settings/plugins/keyfile/tests/keyfiles" |
660 | ++#define TEST_KEYFILES_DIR TEST_KEYFILES_DIR_OLD"/run/NetworkManager/system-connections" |
661 | ++#define TEST_SCRATCH_DIR TEST_SCRATCH_DIR_OLD"/run/NetworkManager/system-connections" |
662 | ++#else |
663 | + #define TEST_KEYFILES_DIR NM_BUILD_SRCDIR "/src/core/settings/plugins/keyfile/tests/keyfiles" |
664 | + #define TEST_SCRATCH_DIR NM_BUILD_BUILDDIR "/src/core/settings/plugins/keyfile/tests/keyfiles" |
665 | ++#endif |
666 | + |
667 | + /*****************************************************************************/ |
668 | + |
669 | +@@ -113,6 +120,11 @@ assert_reread_and_unlink(NMConnection *connection, |
670 | + static void |
671 | + assert_reread_same(NMConnection *connection, NMConnection *reread) |
672 | + { |
673 | ++ #if WITH_NETPLAN |
674 | ++ // Netplan does some normalization already, so compare normalized connections |
675 | ++ nm_connection_normalize (connection, NULL, NULL, NULL); |
676 | ++ nm_connection_normalize (reread, NULL, NULL, NULL); |
677 | ++ #endif |
678 | + nmtst_assert_connection_verifies_without_normalization(reread); |
679 | + nmtst_assert_connection_equals(connection, TRUE, reread, FALSE); |
680 | + } |
681 | +@@ -789,6 +801,10 @@ test_write_wireless_connection(void) |
682 | + bssid, |
683 | + NM_SETTING_WIRELESS_SSID, |
684 | + ssid, |
685 | ++ #if WITH_NETPLAN |
686 | ++ //XXX: netplan uses explicit "infrastructure" mode |
687 | ++ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA, |
688 | ++ #endif |
689 | + NM_SETTING_WIRED_MTU, |
690 | + 1000, |
691 | + NULL); |
692 | +@@ -870,7 +886,12 @@ test_write_string_ssid(void) |
693 | + nm_connection_add_setting(connection, NM_SETTING(s_wireless)); |
694 | + |
695 | + ssid = g_bytes_new(tmpssid, sizeof(tmpssid)); |
696 | +- g_object_set(s_wireless, NM_SETTING_WIRELESS_SSID, ssid, NULL); |
697 | ++ g_object_set(s_wireless, NM_SETTING_WIRELESS_SSID, ssid, |
698 | ++ #if WITH_NETPLAN |
699 | ++ //XXX: netplan uses explicit "infrastructure" mode |
700 | ++ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA, |
701 | ++ #endif |
702 | ++ NULL); |
703 | + g_bytes_unref(ssid); |
704 | + |
705 | + /* IP4 setting */ |
706 | +@@ -953,7 +974,12 @@ test_write_intlist_ssid(void) |
707 | + nm_connection_add_setting(connection, NM_SETTING(s_wifi)); |
708 | + |
709 | + ssid = g_bytes_new(tmpssid, sizeof(tmpssid)); |
710 | +- g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, NULL); |
711 | ++ g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, |
712 | ++ #if WITH_NETPLAN |
713 | ++ //XXX: netplan uses explicit "infrastructure" mode |
714 | ++ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA, |
715 | ++ #endif |
716 | ++ NULL); |
717 | + g_bytes_unref(ssid); |
718 | + |
719 | + /* IP4 setting */ |
720 | +@@ -1053,7 +1079,12 @@ test_write_intlike_ssid(void) |
721 | + nm_connection_add_setting(connection, NM_SETTING(s_wifi)); |
722 | + |
723 | + ssid = g_bytes_new(tmpssid, sizeof(tmpssid)); |
724 | +- g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, NULL); |
725 | ++ g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, |
726 | ++ #if WITH_NETPLAN |
727 | ++ //XXX: netplan uses explicit "infrastructure" mode |
728 | ++ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA, |
729 | ++ #endif |
730 | ++ NULL); |
731 | + g_bytes_unref(ssid); |
732 | + |
733 | + /* IP4 setting */ |
734 | +@@ -1115,7 +1146,12 @@ test_write_intlike_ssid_2(void) |
735 | + nm_connection_add_setting(connection, NM_SETTING(s_wifi)); |
736 | + |
737 | + ssid = g_bytes_new(tmpssid, sizeof(tmpssid)); |
738 | +- g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, NULL); |
739 | ++ g_object_set(s_wifi, NM_SETTING_WIRELESS_SSID, ssid, |
740 | ++ #if WITH_NETPLAN |
741 | ++ //XXX: netplan uses explicit "infrastructure" mode |
742 | ++ NM_SETTING_WIRELESS_MODE, NM_SETTING_WIRELESS_MODE_INFRA, |
743 | ++ #endif |
744 | ++ NULL); |
745 | + g_bytes_unref(ssid); |
746 | + |
747 | + /* IP4 setting */ |
748 | +@@ -2845,12 +2881,29 @@ main(int argc, char **argv) |
749 | + |
750 | + nmtst_init_assert_logging(&argc, &argv, "INFO", "DEFAULT"); |
751 | + |
752 | ++ #if WITH_NETPLAN |
753 | ++ if (g_mkdir_with_parents(TEST_SCRATCH_DIR_OLD, 0755) != 0) { |
754 | ++ errsv = errno; |
755 | ++ g_error("failure to create test directory \"%s\": %s", |
756 | ++ TEST_SCRATCH_DIR_OLD, |
757 | ++ nm_strerror_native(errsv)); |
758 | ++ } |
759 | ++ // Prepare netplan test directories |
760 | ++ g_mkdir_with_parents (TEST_SCRATCH_DIR_OLD"/etc/netplan", 0755); |
761 | ++ g_mkdir_with_parents (TEST_SCRATCH_DIR_OLD"/run/NetworkManager", 0755); |
762 | ++ // link "keyfiles/" to "run/NetworkManager/system-connections" |
763 | ++ const gchar *args[] = { "/bin/ln", "-s", TEST_KEYFILES_DIR_OLD, TEST_KEYFILES_DIR, NULL }; |
764 | ++ g_spawn_sync(NULL, (gchar**)args, NULL, G_SPAWN_DEFAULT, NULL, NULL, NULL, NULL, NULL, NULL); |
765 | ++ // clear netplan YAML config from previous runs |
766 | ++ g_spawn_command_line_sync("/bin/sh -c 'rm -f " TEST_KEYFILES_DIR_OLD "/etc/netplan/*.yaml'", NULL, NULL, NULL, NULL); |
767 | ++ #else |
768 | + if (g_mkdir_with_parents(TEST_SCRATCH_DIR, 0755) != 0) { |
769 | + errsv = errno; |
770 | + g_error("failure to create test directory \"%s\": %s", |
771 | + TEST_SCRATCH_DIR, |
772 | + nm_strerror_native(errsv)); |
773 | + } |
774 | ++ #endif |
775 | + |
776 | + /* The tests */ |
777 | + g_test_add_func("/keyfile/test_read_valid_wired_connection", test_read_valid_wired_connection); |
778 | +-- |
779 | +2.39.2 |
780 | + |
781 | diff --git a/debian/patches/series b/debian/patches/series |
782 | index c2344eb..2e570aa 100644 |
783 | --- a/debian/patches/series |
784 | +++ b/debian/patches/series |
785 | @@ -3,3 +3,5 @@ Force-online-state-with-unmanaged-devices.patch |
786 | # Ubuntu patches |
787 | Provide-access-to-some-of-NM-s-interfaces-to-whoopsie.patch |
788 | Update-dnsmasq-parameters.patch |
789 | +netplan/0001-netplan-Adopt-buildsystems-for-Netplan-integration.patch |
790 | +netplan/0002-netplan-make-use-of-libnetplan-for-YAML-backend.patch |
791 | diff --git a/debian/rules b/debian/rules |
792 | index edcb573..7d336f1 100755 |
793 | --- a/debian/rules |
794 | +++ b/debian/rules |
795 | @@ -4,6 +4,7 @@ include /usr/share/dpkg/architecture.mk |
796 | |
797 | ifeq ($(shell dpkg-vendor --is Ubuntu && echo yes) $(DEB_HOST_ARCH), yes i386) |
798 | BUILD_PACKAGES += -Nnetwork-manager |
799 | + NETPLAN += --disable-netplan |
800 | endif |
801 | |
802 | # Disable lto here regardless of whether we enable the configure flag |
803 | @@ -61,7 +62,7 @@ override_dh_auto_configure: |
804 | --enable-lto \ |
805 | --disable-more-warnings \ |
806 | --disable-modify-system \ |
807 | - --disable-ovs |
808 | + --disable-ovs $(NETPLAN) |
809 | |
810 | override_dh_install: |
811 | find debian/tmp -name '*.la' -print -delete |
812 | diff --git a/debian/tests/control b/debian/tests/control |
813 | index 795b9ba..1985292 100644 |
814 | --- a/debian/tests/control |
815 | +++ b/debian/tests/control |
816 | @@ -13,3 +13,7 @@ Restrictions: needs-root allow-stderr isolation-machine skippable |
817 | Tests: urfkill-integration |
818 | Depends: network-manager, build-essential, linux-headers-generic [!i386], rfkill, urfkill |
819 | Restrictions: needs-root allow-stderr isolation-machine skippable |
820 | + |
821 | +Tests: nm_netplan.py |
822 | +Depends: python3, gir1.2-nm-1.0, network-manager, netplan.io, openvpn, easy-rsa, network-manager-openvpn |
823 | +Restrictions: needs-root allow-stderr isolation-container |
824 | diff --git a/debian/tests/nm.py b/debian/tests/nm.py |
825 | index 47345db..f7a03b7 100755 |
826 | --- a/debian/tests/nm.py |
827 | +++ b/debian/tests/nm.py |
828 | @@ -70,6 +70,7 @@ class NetworkManagerTest(network_test_base.NetworkTestBase): |
829 | "/etc/NetworkManager", |
830 | "/var/lib/NetworkManager", |
831 | "/run/NetworkManager", |
832 | + "/etc/netplan", |
833 | ]: |
834 | subprocess.check_call(["mount", "-n", "-t", "tmpfs", "none", d]) |
835 | self.addCleanup(subprocess.call, ["umount", d]) |
836 | diff --git a/debian/tests/nm_netplan.py b/debian/tests/nm_netplan.py |
837 | new file mode 100644 |
838 | index 0000000..cc86bcf |
839 | --- /dev/null |
840 | +++ b/debian/tests/nm_netplan.py |
841 | @@ -0,0 +1,717 @@ |
842 | +#!/usr/bin/python3 |
843 | + |
844 | +__author__ = "Danilo Egea Gondolfo <danilo.egea.gondolfo@canonical.com>" |
845 | +__copyright__ = "(C) 2023 Canonical Ltd." |
846 | +__license__ = "GPL v2 or later" |
847 | + |
848 | +from glob import glob |
849 | +import json |
850 | +import shutil |
851 | +import socket |
852 | +import subprocess |
853 | +import sys |
854 | +import os |
855 | +from time import sleep |
856 | +import unittest |
857 | +import yaml |
858 | + |
859 | +import gi |
860 | +gi.require_version("NM", "1.0") |
861 | +from gi.repository import NM, GLib, Gio |
862 | + |
863 | +nmclient = NM.Client.new() |
864 | + |
865 | +class TestNetplan(unittest.TestCase): |
866 | + |
867 | + def setUp(self): |
868 | + self._stop_network_manager() |
869 | + |
870 | + self._start_nm() |
871 | + sleep(1) |
872 | + self.nmclient = NM.Client.new() |
873 | + |
874 | + def tearDown(self): |
875 | + pass |
876 | + |
877 | + def _start_nm(self, auto_connect=True): |
878 | + """This method is basically a copy of start_nm() from nm.py |
879 | + without the parts we don't need |
880 | + """ |
881 | + |
882 | + if not os.path.exists("/run/NetworkManager"): |
883 | + os.mkdir("/run/NetworkManager") |
884 | + for d in [ |
885 | + "/etc/NetworkManager", |
886 | + "/var/lib/NetworkManager", |
887 | + "/run/NetworkManager", |
888 | + "/etc/netplan", |
889 | + ]: |
890 | + subprocess.check_call(["mount", "-n", "-t", "tmpfs", "none", d]) |
891 | + self.addCleanup(subprocess.call, ["umount", d]) |
892 | + os.mkdir("/etc/NetworkManager/system-connections") |
893 | + |
894 | + denylist = "" |
895 | + for iface in os.listdir("/sys/class/net"): |
896 | + if iface in ['bonding_masters']: |
897 | + continue |
898 | + with open("/sys/class/net/%s/address" % iface) as f: |
899 | + if denylist: |
900 | + denylist += ";" |
901 | + denylist += "mac:%s" % f.read().strip() |
902 | + |
903 | + conf = "/etc/NetworkManager/NetworkManager.conf" |
904 | + extra_main = "" |
905 | + if not auto_connect: |
906 | + extra_main += "no-auto-default=*\n" |
907 | + |
908 | + with open(conf, "w") as f: |
909 | + f.write( |
910 | + "[main]\nplugins=keyfile\n%s\n[keyfile]\nunmanaged-devices=%s\n" |
911 | + % (extra_main, denylist) |
912 | + ) |
913 | + |
914 | + log = "/tmp/NetworkManager.log" |
915 | + f_log = os.open(log, os.O_CREAT | os.O_WRONLY | os.O_SYNC) |
916 | + |
917 | + # build NM command line |
918 | + argv = ["NetworkManager", "--log-level=debug", "--debug", "--config=" + conf] |
919 | + # allow specifying extra arguments |
920 | + argv += os.environ.get("NM_TEST_DAEMON_ARGS", "").strip().split() |
921 | + |
922 | + p = subprocess.Popen(argv, stdout=f_log, stderr=subprocess.STDOUT) |
923 | + # automatically terminate process at end of test case |
924 | + self.addCleanup(p.wait) |
925 | + self.addCleanup(p.terminate) |
926 | + self.addCleanup(os.close, f_log) |
927 | + self.addCleanup(self._clear_connections) |
928 | + |
929 | + self._process_glib_events() |
930 | + |
931 | + def _process_glib_events(self): |
932 | + """Process pending GLib main loop events""" |
933 | + |
934 | + context = GLib.MainContext.default() |
935 | + while context.iteration(False): |
936 | + pass |
937 | + |
938 | + def _restart_network_manager(self): |
939 | + cmd = ['systemctl', 'restart', 'NetworkManager'] |
940 | + subprocess.call(cmd, stdout=subprocess.DEVNULL) |
941 | + |
942 | + def _stop_network_manager(self): |
943 | + cmd = ['systemctl', 'stop', 'NetworkManager'] |
944 | + subprocess.call(cmd, stdout=subprocess.DEVNULL) |
945 | + |
946 | + def _add_connection(self, connection): |
947 | + main_loop = GLib.MainLoop() |
948 | + def add_cb(client, result, data): |
949 | + self.nmclient.add_connection_finish(result) |
950 | + main_loop.quit() |
951 | + |
952 | + self.nmclient.add_connection_async(connection, True, None, add_cb, None) |
953 | + main_loop.run() |
954 | + |
955 | + def _delete_connection(self, connection): |
956 | + uuid = connection.get_uuid() |
957 | + cmd = ['nmcli', 'con', 'del', uuid] |
958 | + subprocess.call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) |
959 | + |
960 | + def _delete_interface(self, connection): |
961 | + iface = connection.get_interface_name() |
962 | + if iface: |
963 | + cmd = ['ip', 'link', 'del', iface] |
964 | + subprocess.call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) |
965 | + |
966 | + def _nmcli(self, parameters): |
967 | + cmd = ['nmcli'] + parameters |
968 | + subprocess.call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) |
969 | + |
970 | + def _bridge_show(self, bridge): |
971 | + cmd = ['bridge', '-j', 'link', 'show', bridge] |
972 | + ret = subprocess.run(cmd, capture_output=True) |
973 | + return json.loads(ret.stdout) |
974 | + |
975 | + def _load_netplan_yaml_for_connection(self, connection): |
976 | + filename = '/etc/netplan/90-NM-' + connection.get_uuid() + '.yaml' |
977 | + |
978 | + file = open(filename) |
979 | + data = yaml.safe_load(file) |
980 | + file.close() |
981 | + return data |
982 | + |
983 | + def _get_number_of_yaml_files(self): |
984 | + return len(self._get_list_of_yaml_files()) |
985 | + |
986 | + def _get_list_of_yaml_files(self): |
987 | + return glob("/etc/netplan/90-NM-*") |
988 | + |
989 | + def _commit_and_save_connection(self, connection): |
990 | + main_loop = GLib.MainLoop() |
991 | + |
992 | + def commit_cb(client, result, data): |
993 | + connection.commit_changes_finish(result) |
994 | + main_loop.quit() |
995 | + |
996 | + connection.commit_changes_async(True, None, commit_cb, None) |
997 | + main_loop.run() |
998 | + |
999 | + def _clear_connections(self): |
1000 | + for conn in self.nmclient.get_connections(): |
1001 | + self._delete_connection(conn) |
1002 | + self._delete_interface(conn) |
1003 | + |
1004 | + def _netplan_generate(self): |
1005 | + cmd = ['netplan', 'generate'] |
1006 | + ret = subprocess.run(cmd, capture_output=True) |
1007 | + |
1008 | + def _nmcli_con_reload(self): |
1009 | + self._nmcli(['con', 'reload']) |
1010 | + |
1011 | + # Tests |
1012 | + |
1013 | + def test_create_a_simple_bridge_with_dhcp(self): |
1014 | + |
1015 | + conn = NM.SimpleConnection.new() |
1016 | + settings = NM.SettingConnection.new() |
1017 | + settings.set_property(NM.SETTING_CONNECTION_ID, "bridge0") |
1018 | + settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "bridge0") |
1019 | + settings.set_property(NM.SETTING_CONNECTION_TYPE, "bridge") |
1020 | + |
1021 | + bridge = NM.SettingBridge.new() |
1022 | + ipv4 = NM.SettingIP4Config.new() |
1023 | + ipv4.set_property(NM.SETTING_IP_CONFIG_METHOD, "auto") |
1024 | + ipv6 = NM.SettingIP6Config.new() |
1025 | + ipv6.set_property(NM.SETTING_IP_CONFIG_METHOD, "auto") |
1026 | + |
1027 | + conn.add_setting(settings) |
1028 | + conn.add_setting(ipv4) |
1029 | + conn.add_setting(ipv6) |
1030 | + conn.add_setting(bridge) |
1031 | + |
1032 | + # There should be zero netplan NM yaml before adding a connection |
1033 | + self.assertEqual(self._get_number_of_yaml_files(), 0) |
1034 | + |
1035 | + self._add_connection(conn) |
1036 | + |
1037 | + # There should be one netplan NM yaml after adding a connection |
1038 | + self.assertEqual(self._get_number_of_yaml_files(), 1) |
1039 | + |
1040 | + connection = self.nmclient.get_connection_by_id("bridge0") |
1041 | + yaml_data = self._load_netplan_yaml_for_connection(connection) |
1042 | + |
1043 | + # Validating some of the expected flags |
1044 | + self.assertTrue(yaml_data['network']['bridges']['bridge0']['dhcp4']) |
1045 | + self.assertTrue(yaml_data['network']['bridges']['bridge0']['dhcp6']) |
1046 | + |
1047 | + self._delete_connection(connection) |
1048 | + |
1049 | + # There should be zero netplan NM yaml after deleting a connection |
1050 | + self.assertEqual(self._get_number_of_yaml_files(), 0) |
1051 | + |
1052 | + def test_create_a_simple_bridge_with_ip_addresses(self): |
1053 | + conn = NM.SimpleConnection.new() |
1054 | + settings = NM.SettingConnection.new() |
1055 | + settings.set_property(NM.SETTING_CONNECTION_ID, "bridge0") |
1056 | + settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "bridge0") |
1057 | + settings.set_property(NM.SETTING_CONNECTION_TYPE, "bridge") |
1058 | + |
1059 | + bridge = NM.SettingBridge.new() |
1060 | + |
1061 | + ipv4 = NM.SettingIP4Config.new() |
1062 | + ipv4.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual") |
1063 | + ip4_addr1 = NM.IPAddress.new(socket.AF_INET, "10.20.30.40", 24) |
1064 | + ip4_addr2 = NM.IPAddress.new(socket.AF_INET, "10.20.30.41", 24) |
1065 | + ipv4.add_address(ip4_addr1) |
1066 | + ipv4.add_address(ip4_addr2) |
1067 | + |
1068 | + ipv6 = NM.SettingIP6Config.new() |
1069 | + ipv6.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual") |
1070 | + ip6_addr1 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::1", 64) |
1071 | + ip6_addr2 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::2", 64) |
1072 | + ipv6.add_address(ip6_addr1) |
1073 | + ipv6.add_address(ip6_addr2) |
1074 | + |
1075 | + conn.add_setting(settings) |
1076 | + conn.add_setting(ipv4) |
1077 | + conn.add_setting(ipv6) |
1078 | + conn.add_setting(bridge) |
1079 | + |
1080 | + # There should be zero netplan NM yaml before adding a connection |
1081 | + self.assertEqual(self._get_number_of_yaml_files(), 0) |
1082 | + |
1083 | + self._add_connection(conn) |
1084 | + |
1085 | + # There should be one netplan NM yaml after adding a connection |
1086 | + self.assertEqual(self._get_number_of_yaml_files(), 1) |
1087 | + |
1088 | + connection = self.nmclient.get_connection_by_id("bridge0") |
1089 | + yaml_data = self._load_netplan_yaml_for_connection(connection) |
1090 | + |
1091 | + # Validating some of the expected flags |
1092 | + self.assertNotIn('dhcp4', yaml_data['network']['bridges']['bridge0']) |
1093 | + self.assertNotIn('dhcp6', yaml_data['network']['bridges']['bridge0']) |
1094 | + |
1095 | + addresses = yaml_data['network']['bridges']['bridge0']['addresses'] |
1096 | + expected_addresses = ['10.20.30.40/24', '10.20.30.41/24', 'dead:beef::1/64', 'dead:beef::2/64'] |
1097 | + |
1098 | + self.assertListEqual(addresses, expected_addresses) |
1099 | + |
1100 | + self._delete_connection(connection) |
1101 | + |
1102 | + # There should be zero netplan NM yaml after deleting a connection |
1103 | + self.assertEqual(self._get_number_of_yaml_files(), 0) |
1104 | + |
1105 | + def test_create_a_simple_bridge_with_ip_and_member(self): |
1106 | + |
1107 | + conn = NM.SimpleConnection.new() |
1108 | + settings = NM.SettingConnection.new() |
1109 | + settings.set_property(NM.SETTING_CONNECTION_ID, "bridge0") |
1110 | + settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "bridge0") |
1111 | + settings.set_property(NM.SETTING_CONNECTION_TYPE, "bridge") |
1112 | + |
1113 | + bridge = NM.SettingBridge.new() |
1114 | + |
1115 | + ipv4 = NM.SettingIP4Config.new() |
1116 | + ipv4.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual") |
1117 | + ip4_addr1 = NM.IPAddress.new(socket.AF_INET, "10.20.30.40", 24) |
1118 | + ip4_addr2 = NM.IPAddress.new(socket.AF_INET, "10.20.30.41", 24) |
1119 | + ipv4.add_address(ip4_addr1) |
1120 | + ipv4.add_address(ip4_addr2) |
1121 | + |
1122 | + ipv6 = NM.SettingIP6Config.new() |
1123 | + ipv6.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual") |
1124 | + ip6_addr1 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::1", 64) |
1125 | + ip6_addr2 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::2", 64) |
1126 | + ipv6.add_address(ip6_addr1) |
1127 | + ipv6.add_address(ip6_addr2) |
1128 | + |
1129 | + conn.add_setting(settings) |
1130 | + conn.add_setting(ipv4) |
1131 | + conn.add_setting(ipv6) |
1132 | + conn.add_setting(bridge) |
1133 | + |
1134 | + # There should be zero netplan NM yaml before adding a connection |
1135 | + self.assertEqual(self._get_number_of_yaml_files(), 0) |
1136 | + |
1137 | + # Adding the bridge |
1138 | + self._add_connection(conn) |
1139 | + |
1140 | + # There should be one netplan NM yaml after adding a connection |
1141 | + self.assertEqual(self._get_number_of_yaml_files(), 1) |
1142 | + |
1143 | + # Creating a tap0 device to be a bridge member |
1144 | + tap0 = NM.SimpleConnection.new() |
1145 | + tap0_conn_settings = NM.SettingConnection.new() |
1146 | + tap0_conn_settings.set_property(NM.SETTING_CONNECTION_ID, "tap0") |
1147 | + tap0_conn_settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "tap0") |
1148 | + tap0_conn_settings.set_property(NM.SETTING_CONNECTION_TYPE, "tun") |
1149 | + tap0_conn_settings.set_property(NM.SETTING_CONNECTION_SLAVE_TYPE, "bridge") |
1150 | + tap0_conn_settings.set_property(NM.SETTING_CONNECTION_MASTER, "bridge0") |
1151 | + |
1152 | + tap0_settings = NM.SettingTun.new() |
1153 | + tap0_settings.set_property(NM.SETTING_TUN_MODE, NM.SettingTunMode.TAP) |
1154 | + |
1155 | + tap0.add_setting(tap0_conn_settings) |
1156 | + tap0.add_setting(tap0_settings) |
1157 | + self._add_connection(tap0) |
1158 | + |
1159 | + # There should be two netplan NM yaml after adding the tap |
1160 | + self.assertEqual(self._get_number_of_yaml_files(), 2) |
1161 | + |
1162 | + bridge0_connection = self.nmclient.get_connection_by_id("bridge0") |
1163 | + yaml_data = self._load_netplan_yaml_for_connection(bridge0_connection) |
1164 | + |
1165 | + # Validating some of the bridge expected flags |
1166 | + addresses = yaml_data['network']['bridges']['bridge0']['addresses'] |
1167 | + expected_addresses = ['10.20.30.40/24', '10.20.30.41/24', 'dead:beef::1/64', 'dead:beef::2/64'] |
1168 | + |
1169 | + self.assertListEqual(addresses, expected_addresses) |
1170 | + |
1171 | + # Validating if tap0 is attached to the bridge |
1172 | + # It might take a while for the new interface be created... |
1173 | + limit = 10 |
1174 | + while (show_bridge := self._bridge_show('bridge0')) == [] and limit > 0: |
1175 | + sleep(1) |
1176 | + limit = limit - 1 |
1177 | + self.assertEqual(show_bridge[0]['master'], 'bridge0') |
1178 | + self.assertEqual(show_bridge[0]['ifname'], 'tap0') |
1179 | + |
1180 | + tap0_connection = self.nmclient.get_connection_by_id("tap0") |
1181 | + tap_yaml = self._load_netplan_yaml_for_connection(tap0_connection) |
1182 | + |
1183 | + tap0_uuid = tap0_connection.get_uuid() |
1184 | + # Validating that the bridge information is in the tap yaml |
1185 | + self.assertEqual(tap_yaml['network']['nm-devices']['NM-' + tap0_uuid]['networkmanager']['passthrough']['connection.master'], 'bridge0') |
1186 | + |
1187 | + self._delete_connection(tap0_connection) |
1188 | + # There should be one netplan NM yaml after deleting the tap |
1189 | + self.assertEqual(self._get_number_of_yaml_files(), 1) |
1190 | + |
1191 | + self._delete_connection(bridge0_connection) |
1192 | + # There should be zero netplan NM yaml after deleting the bridge |
1193 | + self.assertEqual(self._get_number_of_yaml_files(), 0) |
1194 | + |
1195 | + @unittest.skip('XXX: the netplan yaml file will be deleted when we try to change the connetion') |
1196 | + def test_create_an_interface_and_change_it(self): |
1197 | + """Add a tap interface and change it after create adding IP addresses to it.""" |
1198 | + |
1199 | + # Creating a tap0 device to be a bridge member |
1200 | + tap0 = NM.SimpleConnection.new() |
1201 | + tap0_conn_settings = NM.SettingConnection.new() |
1202 | + tap0_conn_settings.set_property(NM.SETTING_CONNECTION_ID, "tap0") |
1203 | + tap0_conn_settings.set_property(NM.SETTING_CONNECTION_INTERFACE_NAME, "tap0") |
1204 | + tap0_conn_settings.set_property(NM.SETTING_CONNECTION_TYPE, "tun") |
1205 | + |
1206 | + tap0_settings = NM.SettingTun.new() |
1207 | + tap0_settings.set_property(NM.SETTING_TUN_MODE, NM.SettingTunMode.TAP) |
1208 | + |
1209 | + tap0.add_setting(tap0_conn_settings) |
1210 | + tap0.add_setting(tap0_settings) |
1211 | + self._add_connection(tap0) |
1212 | + |
1213 | + # There should be one netplan NM yaml after adding a connection |
1214 | + self.assertEqual(self._get_number_of_yaml_files(), 1) |
1215 | + |
1216 | + tap0_connection = self.nmclient.get_connection_by_id("tap0") |
1217 | + |
1218 | + ipv4_settings = tap0_connection.get_setting_ip4_config() |
1219 | + ipv4_settings.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual") |
1220 | + ip4_addr1 = NM.IPAddress.new(socket.AF_INET, "10.20.30.40", 24) |
1221 | + ip4_addr2 = NM.IPAddress.new(socket.AF_INET, "10.20.30.41", 24) |
1222 | + ipv4_settings.add_address(ip4_addr1) |
1223 | + ipv4_settings.add_address(ip4_addr2) |
1224 | + |
1225 | + ipv6_settings = tap0_connection.get_setting_ip6_config() |
1226 | + ipv6_settings.set_property(NM.SETTING_IP_CONFIG_METHOD, "manual") |
1227 | + ip6_addr1 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::1", 64) |
1228 | + ip6_addr2 = NM.IPAddress.new(socket.AF_INET6, "dead:beef::2", 64) |
1229 | + ipv6_settings.add_address(ip6_addr1) |
1230 | + ipv6_settings.add_address(ip6_addr2) |
1231 | + |
1232 | + self._commit_and_save_connection(tap0_connection) |
1233 | + |
1234 | + # There should be one netplan NM yaml files after chaging the only existing connection |
1235 | + self.assertEqual(self._get_number_of_yaml_files(), 1) |
1236 | + |
1237 | + self._delete_connection(tap0_connection) |
1238 | + |
1239 | + # There should be zero netplan NM yaml files after removing the only existing connection |
1240 | + self.assertEqual(self._get_number_of_yaml_files(), 0) |
1241 | + |
1242 | + def test_nmcli_add_device_and_change_it(self): |
1243 | + """Uses the nmcli to add a connection and validates if the |
1244 | + Netplan YAML file has the expected configuration. |
1245 | + It also changes the configuration changing the ipv4 method from auto |
1246 | + to manual and adding an IP address. After the change the Netplan YAML |
1247 | + file should have the same name. |
1248 | + """ |
1249 | + |
1250 | + self.assertEqual(self._get_number_of_yaml_files(), 0) |
1251 | + |
1252 | + nmcli_add = ['con', 'add', 'type', 'tun', 'mode', 'tap', 'ifname', 'tap0', 'ipv4.method', 'auto'] |
1253 | + self._nmcli(nmcli_add) |
1254 | + |
1255 | + self.assertEqual(self._get_number_of_yaml_files(), 1) |
1256 | + |
1257 | + files_before = self._get_list_of_yaml_files() |
1258 | + |
1259 | + # After OOB changes, the client must be refreshed apparently |
1260 | + self.nmclient = NM.Client.new() |
1261 | + |
1262 | + conn = self.nmclient.get_connection_by_id("tun-tap0") |
1263 | + uuid = conn.get_uuid() |
1264 | + data = self._load_netplan_yaml_for_connection(conn) |
1265 | + |
1266 | + ip4_method = data.get('network') \ |
1267 | + .get('nm-devices') \ |
1268 | + .get('NM-' + uuid) \ |
1269 | + .get('networkmanager') \ |
1270 | + .get('passthrough') \ |
1271 | + .get('ipv4.method') |
1272 | + |
1273 | + self.assertEqual(ip4_method, 'auto') |
1274 | + |
1275 | + nmcli_mod = ['con', 'mod', 'tun-tap0', 'ipv4.method', 'manual', 'ipv4.addresses', '10.20.30.40/24'] |
1276 | + self._nmcli(nmcli_mod) |
1277 | + |
1278 | + self.assertEqual(self._get_number_of_yaml_files(), 1) |
1279 | + |
1280 | + files_after = self._get_list_of_yaml_files() |
1281 | + |
1282 | + self.assertListEqual(files_before, files_after) |
1283 | + |
1284 | + data = self._load_netplan_yaml_for_connection(conn) |
1285 | + |
1286 | + ip4_method = data.get('network') \ |
1287 | + .get('nm-devices') \ |
1288 | + .get('NM-' + uuid) \ |
1289 | + .get('networkmanager') \ |
1290 | + .get('passthrough') \ |
1291 | + .get('ipv4.method') |
1292 | + |
1293 | + ip4_addr = data.get('network') \ |
1294 | + .get('nm-devices') \ |
1295 | + .get('NM-' + uuid) \ |
1296 | + .get('networkmanager') \ |
1297 | + .get('passthrough') \ |
1298 | + .get('ipv4.address1') |
1299 | + |
1300 | + self.assertEqual(ip4_method, 'manual') |
1301 | + self.assertEqual(ip4_addr, '10.20.30.40/24') |
1302 | + |
1303 | + self._delete_connection(conn) |
1304 | + self.assertEqual(self._get_number_of_yaml_files(), 0) |
1305 | + |
1306 | + def test_nmcli_add_wifi_connection(self): |
1307 | + """Create a wifi connection via nmcli and check if the expected |
1308 | + fields were added to the netplan yaml file.""" |
1309 | + |
1310 | + ssid = 'My network SSID' |
1311 | + passwd = 'secretpasswd' |
1312 | + method = 'wpa-psk' |
1313 | + ip = '10.20.30.40/24' |
1314 | + nmcli_add = ['con', 'add', 'type', 'wifi', 'ssid', ssid, |
1315 | + 'wifi-sec.key-mgmt', method, 'wifi-sec.psk', passwd, |
1316 | + 'ipv4.method', 'manual', 'ipv4.addresses', ip] |
1317 | + |
1318 | + self.assertEqual(self._get_number_of_yaml_files(), 0) |
1319 | + |
1320 | + self._nmcli(nmcli_add) |
1321 | + |
1322 | + self.assertEqual(self._get_number_of_yaml_files(), 1) |
1323 | + |
1324 | + # After OOB changes, the client must be refreshed apparently |
1325 | + self.nmclient = NM.Client.new() |
1326 | + |
1327 | + conn = self.nmclient.get_connection_by_id("wifi") |
1328 | + uuid = conn.get_uuid() |
1329 | + data = self._load_netplan_yaml_for_connection(conn) |
1330 | + |
1331 | + ap_name = list(data.get('network') \ |
1332 | + .get('wifis') \ |
1333 | + .get('NM-' + uuid) \ |
1334 | + .get('access-points') \ |
1335 | + .keys())[0] |
1336 | + |
1337 | + ip_addr = data.get('network') \ |
1338 | + .get('wifis') \ |
1339 | + .get('NM-' + uuid) \ |
1340 | + .get('addresses') |
1341 | + |
1342 | + auth_method = data.get('network') \ |
1343 | + .get('wifis') \ |
1344 | + .get('NM-' + uuid) \ |
1345 | + .get('access-points') \ |
1346 | + .get(ap_name) \ |
1347 | + .get('auth') \ |
1348 | + .get('key-management') |
1349 | + |
1350 | + auth_passwd = data.get('network') \ |
1351 | + .get('wifis') \ |
1352 | + .get('NM-' + uuid) \ |
1353 | + .get('access-points') \ |
1354 | + .get(ap_name) \ |
1355 | + .get('auth') \ |
1356 | + .get('password') |
1357 | + |
1358 | + self.assertEqual(ap_name, ssid) |
1359 | + self.assertListEqual(ip_addr, [ip]) |
1360 | + self.assertEqual(auth_method, 'psk') |
1361 | + self.assertEqual(auth_passwd, passwd) |
1362 | + |
1363 | + self._delete_connection(conn) |
1364 | + self.assertEqual(self._get_number_of_yaml_files(), 0) |
1365 | + |
1366 | + def test_create_connection_via_netplan(self): |
1367 | + """ |
1368 | + Create a connection via netplan generate and check if NM will pick it up |
1369 | + """ |
1370 | + |
1371 | + netplan_yaml = '''network: |
1372 | + renderer: NetworkManager |
1373 | + ethernets: |
1374 | + eth123: |
1375 | + dhcp4: true''' |
1376 | + |
1377 | + with open('/etc/netplan/10-test.yaml', 'w') as f: |
1378 | + f.write(netplan_yaml) |
1379 | + |
1380 | + self._netplan_generate(); |
1381 | + self._nmcli_con_reload() |
1382 | + self.nmclient = NM.Client.new() |
1383 | + |
1384 | + expected = None |
1385 | + for conn in self.nmclient.get_connections(): |
1386 | + if conn.get_id() == 'netplan-eth123': |
1387 | + expected = conn |
1388 | + |
1389 | + self.assertIsNotNone(expected) |
1390 | + |
1391 | + def test_create_connection_via_netplan_and_remove_via_nmcli(self): |
1392 | + """ |
1393 | + Create a connection via netplan generate and remove it with nmcli. |
1394 | + |
1395 | + The interface should be removed from the yaml file. |
1396 | + """ |
1397 | + |
1398 | + netplan_yaml = '''network: |
1399 | + renderer: NetworkManager |
1400 | + ethernets: |
1401 | + eth123: |
1402 | + dhcp4: true |
1403 | + eth456: |
1404 | + dhcp4: true''' |
1405 | + |
1406 | + with open('/etc/netplan/10-test.yaml', 'w') as f: |
1407 | + f.write(netplan_yaml) |
1408 | + |
1409 | + self._netplan_generate(); |
1410 | + self._nmcli_con_reload() |
1411 | + self.nmclient = NM.Client.new() |
1412 | + |
1413 | + expected1 = None |
1414 | + expected2 = None |
1415 | + for conn in self.nmclient.get_connections(): |
1416 | + if conn.get_id() == 'netplan-eth123': |
1417 | + expected1 = conn |
1418 | + if conn.get_id() == 'netplan-eth456': |
1419 | + expected2 = conn |
1420 | + |
1421 | + self.assertIsNotNone(expected1) |
1422 | + self.assertIsNotNone(expected1) |
1423 | + |
1424 | + self._delete_connection(expected1) |
1425 | + |
1426 | + with open('/etc/netplan/10-test.yaml', 'r') as f: |
1427 | + yaml_data = yaml.safe_load(f) |
1428 | + |
1429 | + # eth123 shouldn't exist anymore |
1430 | + self.assertIsNone(yaml_data.get('network').get('ethernets').get('eth123')) |
1431 | + |
1432 | + self.assertIsNotNone(yaml_data.get('network').get('ethernets').get('eth456')) |
1433 | + |
1434 | + def test_create_connection_via_netplan_and_change_it_via_nmcli(self): |
1435 | + """ |
1436 | + Create a connection via netplan generate and change it via nmcli. |
1437 | + """ |
1438 | + |
1439 | + netplan_yaml = '''network: |
1440 | + renderer: NetworkManager |
1441 | + ethernets: |
1442 | + eth123: |
1443 | + dhcp4: false |
1444 | + dhcp6: false |
1445 | + eth456: |
1446 | + dhcp4: true''' |
1447 | + |
1448 | + with open('/etc/netplan/10-test.yaml', 'w') as f: |
1449 | + f.write(netplan_yaml) |
1450 | + |
1451 | + self._netplan_generate(); |
1452 | + self._nmcli_con_reload() |
1453 | + self._nmcli(['con', 'mod', 'netplan-eth123', 'ipv4.method', 'auto']) |
1454 | + |
1455 | + # eth123.dhcp4 should be overriden by 90-NM-<UUID>.yaml (from 10-test.yaml) |
1456 | + # The output of 'netplan get' should account for that. |
1457 | + out = subprocess.check_output(['netplan', 'get'], universal_newlines=True) |
1458 | + yaml_data = yaml.safe_load(out) |
1459 | + dhcp = yaml_data.get('network').get('ethernets').get('eth123').get('dhcp4') |
1460 | + self.assertTrue(dhcp) |
1461 | + |
1462 | + def test_openvpn_connection(self): |
1463 | + """ Test case for LP#1998207""" |
1464 | + |
1465 | + server_config = """dev tun |
1466 | +ca /tmp/openvpn/pki/ca.crt |
1467 | +cert /tmp/openvpn/pki/issued/server.crt |
1468 | +key /tmp/openvpn/pki/private/server.key |
1469 | +dh /tmp/openvpn/pki/dh.pem |
1470 | +server 10.8.0.0 255.255.255.0 |
1471 | +keepalive 10 120 |
1472 | +cipher AES-256-GCM |
1473 | +compress lz4-v2 |
1474 | +push "compress lz4-v2" |
1475 | +user root |
1476 | +log /tmp/openvpn.log |
1477 | +group root |
1478 | +""" |
1479 | + |
1480 | + client_config = """client |
1481 | +dev tun |
1482 | +remote 127.0.0.1 1194 |
1483 | +nobind |
1484 | +ca /tmp/openvpn/pki/ca.crt |
1485 | +cert /tmp/openvpn/pki/issued/client.crt |
1486 | +key /tmp/openvpn/pki/private/client.key |
1487 | +cipher AES-256-GCM |
1488 | +""" |
1489 | + |
1490 | + # The minimum DH size accepted by OpenVPN these days is 2048. |
1491 | + # It might take a while to be generated (like almost a minute) |
1492 | + # It would be faster to use shared keys instead of TLS but it |
1493 | + # seems it's not an option anymore in OpenVPN |
1494 | + openvpn_spinup_script = """/usr/share/easy-rsa/easyrsa init-pki |
1495 | +EASYRSA_BATCH=1 /usr/share/easy-rsa/easyrsa build-ca nopass |
1496 | +EASYRSA_BATCH=1 /usr/share/easy-rsa/easyrsa build-server-full server nopass |
1497 | +EASYRSA_BATCH=1 /usr/share/easy-rsa/easyrsa build-client-full client nopass |
1498 | +/usr/share/easy-rsa/easyrsa gen-dh |
1499 | +""" |
1500 | + |
1501 | + tmpdir = '/tmp/openvpn' |
1502 | + self.addCleanup(shutil.rmtree, tmpdir) |
1503 | + os.mkdir(tmpdir) |
1504 | + os.chdir(tmpdir) |
1505 | + |
1506 | + with open('openvpn_spinup.sh', 'w') as f: |
1507 | + f.write(openvpn_spinup_script) |
1508 | + |
1509 | + with open('server.conf', 'w') as f: |
1510 | + f.write(server_config) |
1511 | + |
1512 | + with open('client.conf', 'w') as f: |
1513 | + f.write(client_config) |
1514 | + |
1515 | + cmd = ['bash', 'openvpn_spinup.sh'] |
1516 | + subprocess.call(cmd, stdout=subprocess.DEVNULL) |
1517 | + |
1518 | + openvpn_server_cmd = ['openvpn', '--config', 'server.conf'] |
1519 | + p_server = subprocess.Popen(openvpn_server_cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) |
1520 | + sleep(1) # Let's give OpenVPN a second to start |
1521 | + |
1522 | + # Add a useless default route to the loopback interface so NM will allow starting the VPN connection. |
1523 | + # Apparently, if it doesn't own an *active connection* with a default route, it will not allow |
1524 | + # us to start the VPN client. |
1525 | + # As we set the main ethernet device as unmanaged, NM will not 'own' a default route when it starts. |
1526 | + # Unfortunately, by doing this, NM will take over the interface 'lo' and add a new yaml file to /etc/netplan |
1527 | + self._nmcli(['con', 'mod', 'lo', 'ipv4.gateway', '10.8.0.254']) |
1528 | + |
1529 | + # Create an OpenVPN connection based on the configuration found in client.conf |
1530 | + self._nmcli(['con', 'import', 'type', 'openvpn', 'file', 'client.conf']) |
1531 | + |
1532 | + # At this point we should have 2 yaml files |
1533 | + # One for the tun0 NM took over and one for the VPN connection |
1534 | + self.assertEqual(self._get_number_of_yaml_files(), 2, |
1535 | + msg='More than expected YAML files were found after creating the connection') |
1536 | + |
1537 | + self._nmcli(['con', 'up', 'client']) |
1538 | + sleep(2) # Let's give NM a couple of seconds to settle down |
1539 | + # We still should have 2 files after starting the client |
1540 | + self.assertEqual(self._get_number_of_yaml_files(), 2, |
1541 | + msg='More than expected YAML files were found after starting the connection') |
1542 | + |
1543 | + self._nmcli(['con', 'down', 'client']) |
1544 | + sleep(2) # Let's give NM a couple of seconds to settle down |
1545 | + # We still should have 2 files after stopping the client |
1546 | + self.assertEqual(self._get_number_of_yaml_files(), 2, |
1547 | + msg='More than expected YAML files were found after stopping the connection') |
1548 | + |
1549 | + p_server.terminate() |
1550 | + p_server.wait() |
1551 | + # We still should have 2 files after stopping the server |
1552 | + self.assertEqual(self._get_number_of_yaml_files(), 2, |
1553 | + msg='More than expected YAML files were found after stopping the OpenVPN server') |
1554 | + |
1555 | + |
1556 | +if __name__ == '__main__': |
1557 | + runner = unittest.TextTestRunner(stream=sys.stdout, verbosity=2) |
1558 | + unittest.main(testRunner=runner) |
Thanks! A few comments inline.