lp:~ogayot/curtin

Author: Olivier Gayot
Author Date: 2024-04-15 10:04:22 UTC

util: fix lack of string interpolation when raising TiemoutError

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2024-03-22 16:02:24 UTC

apt-config: avoid APT complaints about sources backup files

Before apt-config makes a change to ubuntu.sources or another source
file, it backs up the original file by appending the .curtin.old
extension.

However, by default, APT will emit a warning when a file with an unknown
extension is present in the etc/apt/sources.list.d directory. Currently,
known extensions are .sources and .list so when APT sees our .curtin.old
files, it does emit a warning.

 > N: Ignoring file 'ubuntu.sources.curtin.old' in directory
 '/etc/apt/sources.list.d/' as it has an invalid filename extension

Through Dir::Ignore-Files-Silently directives, APT can be configured to
ignore (i.e., not warn) some of the files that have unknown extensions.
By default, the list includes:

 Dir::Ignore-Files-Silently:: "\.disabled$";
 Dir::Ignore-Files-Silently:: "\.bak$";
 Dir::Ignore-Files-Silently:: "\.dpkg-[a-z]+$";
 Dir::Ignore-Files-Silently:: "\.ucf-[a-z]+$";
 Dir::Ignore-Files-Silently:: "\.save$";
 Dir::Ignore-Files-Silently:: "\.orig$";
 Dir::Ignore-Files-Silently:: "\.distUpgrade$";

We now back up the files using the .curtin.orig extension instead of
.curtin.old so that APT does not complain.

LP: #2058741

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2024-03-11 11:18:08 UTC

apt: ensure systemd knows it runs in a chroot, when executing postinst

Since we added the --mount-proc option to unshare, the postinst script
for openssh-server (and most likely other packages) started failing with
the following error when `systemctl daemon-reload` was invoked:

  > Failed to connect to bus: No data available

Before the option was added, it would simply do nothing because systemd
rightly understood it was running in a chroot.

To determine if we are running in a chroot, systemd checks if
/proc/1/root (corresponding to the init process) and / are the same
inode. If they are different, systemd assumes we are in a chroot.

However, we are running apt-get in a new PID namespace which means that
in the new namespace, apt-get gets assigned PID 1 and is therefore the
"init" process.

Now that /proc is properly mounted in the chroot, when systemd compares
/proc/1/root and /, it sees they are identical because the init process
(which is apt-get) is actually running inside the chroot.

Without the --mount-proc option, /proc/1 in the chroot would still refer
to the systemd init process (running outside the chroot), so it would
work properly.

With the SYSTEMD_OFFLINE variable, one can "force" systemd to assume
it is running in a chroot. Let's use it when running commands in a
chroot, and the variable is not already defined.

LP: #2056570

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2024-03-11 11:18:08 UTC

apt: ensure systemd knows it runs in a chroot, when executing postinst

Since we added the --mount-proc option to unshare, the postinst script
for openssh-server (and most likely other packages) started failing with
the following error when `systemctl daemon-reload` was invoked:

  > Failed to connect to bus: No data available

Before the option was added, it would simply do nothing because systemd
rightly understood it was running in a chroot.

To determine if we are running in a chroot, systemd checks if
/proc/1/root (corresponding to the init process) and / are the same
inode. If they are different, systemd assumes we are in a chroot.

However, we are running apt-get in a new PID namespace which means that
in the new namespace, apt-get gets assigned PID 1 and is therefore the
"init" process.

Now that /proc is properly mounted in the chroot, when systemd compares
/proc/1/root and /, it sees they are identical because the init process
(which is apt-get) is actually running inside the chroot.

Without the --mount-proc option, /proc/1 in the chroot would still refer
to the systemd init process (running outside the chroot), so it would
work properly.

With the SYSTEMD_OFFLINE variable, one can "force" systemd to assume
it is running in a chroot. Let's use it.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2024-03-08 13:30:39 UTC

system-install: log output of commands if they fail

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2024-03-08 10:34:01 UTC

apt-config: enable deb822 sources from 24.04 onwards

This reverts commit a7640fdcac396f9f09044dc7ca7553043ce4231c and applies
further modifications to enable deb822 sources starting from 24.04 and
not 23.10.

LP: #2056308

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-10-26 09:44:54 UTC

curthooks/install-grub: apply efivarfs workaround if system is affected

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-09-20 09:28:05 UTC

block_meta: also add recovery key if zkey is used

When zkey is properly setup, we do not invoke cryptsetup luksFormat
ourselves. Instead we lean on zkey to invoke cryptsetup luksFormat for
us.

zkey seems to have no native support for invoking cryptsetup luksAddKey,
so we need to manually call it if we want to add a recovery key in a
second slot.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-09-07 10:55:58 UTC

apt_source: do not produce a deb822 source stanza with zero suite

When no suite is set for a given repository, we end up with a
deb822 styled source such as:

 Type: deb
 Suites:
 URIs: http://archive.ubuntu.com/ubuntu

This is invalid and apt/apt-get will raise the following error:

E: Malformed entry 2 in sources file /etc/apt/sources.list.d/ubuntu.sources (Suite)

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-08-10 14:09:56 UTC

block_meta: add luks recovery key if requested

If the storage configuration contains the key "recovery_keyfile", call
cryptsetup luksAddKey after cryptsetup luksFormat so that the key
specified is added as a recovery key (i.e., a normal key in the second
key slot).

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-07-06 14:35:05 UTC

curthooks: fix exception when passing 'kernel: null'

When we don't want curthooks to install a kernel, we can pass:

  kernel: null

However, currently, it makes the code raise an exception because we run
.get() on the kernel object even if it is None.

     config.merge_config(mapping, kernel_cfg.get('mapping', {}))
 AttributeError: 'NoneType' object has no attribute 'get'
 'NoneType' object has no attribute 'get'

Fixed by returning gracefully without installing any kernel upon
encountering this configuration.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-05-02 14:14:19 UTC

Add file a ( do not merge )

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-04-25 14:23:17 UTC

block: sleep and retry wiping part. if open(O_EXCL) returns EBUSY

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-04-24 10:28:43 UTC

block_meta_v2: don't solely expect flag='logical' for logical partitions

Most of the time, a logical partition has its flag set to 'local'.
However, it some cases, it can also have flag set to 'swap', 'boot' and
possibly other values.

To determine if a partition is actually logical on a DOS partition
table, we need to check the partition number.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2023-04-03 16:49:44 UTC

block-meta: fix failed disk lookup when WWN includes extension

curtin's storage config extracts the WWN of a disk from one of the
following udev variables:

 * ID_WWN_WITH_EXTENSION (preferred)
 * ID_WWN

However, when trying to look the disk up later in the block-meta code,
curtin only tries to match the WWN value against the ID_WWN variable
(and another variable related to DM multipath).

This means that the disk can not be be found just based on the wwn if
the ID_WWN and ID_WWN_WITH_EXTENSION variables don't hold the same
value.

In the past, that was often okay because other fields (such as disk path
or serial) would still make the disk lookup succeed.

However, the following patch introduced a restriction. In v2, all fields
specified must now match for the disk lookup to be successful:

8c5f87ed block_meta: all fields on a disk action must match with v2 config

Fixed by matching against the ID_WWN_WITH_EXTENSION (preferred) and then
ID_WWN.

(cherry picked from commit 73da3c4f47ea22d58f57371594a396c5434ab260)

Author: Olivier Gayot
Author Date: 2023-04-03 09:40:41 UTC

block-meta: fix failed disk lookup when WWN includes extension

curtin's storage config extracts the WWN of a disk from one of the
following udev variables:

 * ID_WWN_WITH_EXTENSION (preferred)
 * ID_WWN

However, when trying to look the disk up later in the block-meta code,
curtin only tries to match the WWN value against the ID_WWN variable
(and another variable related to DM multipath).

This means that the disk can not be be found just based on the wwn if
the ID_WWN and ID_WWN_WITH_EXTENSION variables don't hold the same
value.

In the past, that was often okay because other fields (such as disk path
or serial) would still make the disk lookup succeed.

However, the following patch introduced a restriction. In v2, all fields
specified must now match for the disk lookup to be successful:

8c5f87ed block_meta: all fields on a disk action must match with v2 config

Fixed by matching against the ID_WWN_WITH_EXTENSION (preferred) and then
ID_WWN.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Michael Hudson-Doyle
Author Date: 2022-03-10 22:14:00 UTC

have v2 get_path_to_storage_volume promote multipath members to multipath device more generally

Author: Olivier Gayot
Author Date: 2022-11-03 14:51:17 UTC

distro: fix typo s/packge/package/

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-08-08 07:47:15 UTC

install: add option to append to log file

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-07-26 09:36:10 UTC

curtin: do not reject rhel9 and derivatives

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-07-08 12:17:12 UTC

doc: remove duplicate section about extract subcommand

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-01-20 14:16:58 UTC

Make sure curthooks do not discard APT preferences

Although APT preferences applied properly when running curtin
apt-config, we observed during a full installation using Subiquity that
the preferences would not be honored when installing packages from the
"packages" autoinstall section.

This happened because the installation of packages occur after running
curthooks. Curtooks call handle_apt with an almost empty configuration.
Therefore, we would overwrite the preferences by removing the
etc/apt/preferences.d/90curtin.pref file.

It is to be noted that the same behavior occurs for the APT proxy.

Fixed by not removing etc/apt/preferences.d/90curtin.pref when the
configuration does not contain APT preferences.

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-01-12 15:34:13 UTC

Remove leftover debug print statement

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-01-07 08:29:01 UTC

Fix format of examples/apt-source.yaml so it can be passed to apt-config

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Olivier Gayot
Author Date: 2022-01-07 08:28:27 UTC

Add support for APT preferences in apt-config

apt-config now supports a list of APT preferences (i.e. pinning rules)
as in the following example:

apt:
  preferences:
    - {package: "python3-*", pin: "origin *ubuntu.com*", pin-priority: 200}
    - {package: "python-*", pin: "origin *ubuntu.com*", pin-priority: -1}

These preferences are deployed under
<target>/etc/apt/preferences.d/90-curtin.pref using the format specified
in apt_preferences(5).

  Package: python3-*
  Pin: origin *ubuntu.com*
  Pin-Priority: 200

  Package: python-*
  Pin: origin *ubuntu.com*
  Pin-Priority: -1

Signed-off-by: Olivier Gayot <olivier.gayot@canonical.com>

Author: Dan Bungert
Author Date: 2021-12-09 03:08:18 UTC

block: output partition device path

reconstructing this later is non-trivial for some devices

Author: Dan Bungert
Author Date: 2021-12-07 03:23:51 UTC

21.3 merge for Jammy

Author: Dan Bungert
Author Date: 2021-09-20 20:54:57 UTC

commands/apt: use python-apt for sources.list

Start using python{,3}-apt for sources.list handling.
generate_sources_list now operates in a pipeline-like model, where each
stage performs one specific transformation on the list of entries.

Author: Paride Legovini
Author Date: 2021-01-27 20:47:05 UTC

releasing curtin version 21.2-0ubuntu1~16.04.1

Author: Paride Legovini
Author Date: 2021-01-27 20:39:57 UTC

releasing curtin version 21.2-0ubuntu1~18.04.1

Author: Paride Legovini
Author Date: 2021-01-27 20:26:44 UTC

releasing curtin version 21.2-0ubuntu1~20.04.1

Author: Paride Legovini
Author Date: 2020-09-25 20:46:55 UTC

releasing curtin version 20.2-0ubuntu1

Author: Ryan Harper
Author Date: 2020-02-14 15:44:17 UTC

releasing curtin version 19.3-26-g82f23e3d-0ubuntu1~19.10.1

Author: Dan Watkins
Author Date: 2019-09-10 21:57:21 UTC

Revert "d/control: update Depends for new probert package names"

This reverts commit 43c0a1474c4d32cebf0e940e555a7ac6851dbe9c.

(I incorrectly didn't fix the Depends during the preceding merge, but
its existence is still sufficient for this revert to not cause merge
conflicts into master.)

Author: Ryan Harper
Author Date: 2019-06-12 20:55:08 UTC

update changelog

Author: Dan Watkins
Author Date: 2019-05-21 21:03:54 UTC

release 19.1

Author: Ryan Harper
Author Date: 2018-05-18 19:02:29 UTC

releasing curtin version 18.1-17-gae48e86f-0ubuntu1~17.10.1

Author: Scott Moser
Author Date: 2017-12-19 18:34:42 UTC

Import version 0.1.0~bzr532-0ubuntu1~17.04.1