lp:~bjornt/maas

Author: Björn Tillenius
Author Date: 2024-03-13 14:48:45 UTC

Bug #2056740: Can't commission/deploy AMT machines

The maas-power script was passed "--boot-mode pxe" for AMT machines
and failed, since such a parameter wasn't defined.

All other power drivers, including the newer AMT one, sets PXE boot every
time the machine is powered on. I changed things, so that "boot-mode" is
no longer passed, and the older AMT driver also sets the machine to PXE
boot whenever it's turned on.

(cherry picked from commit 116ccedb102a78136d9e1d62d3144850e861947e)

Author: Björn Tillenius
Author Date: 2024-03-13 09:07:33 UTC

Bug #2057750: Can't pxe boot legacy non-uefi machines

The rewrite rules in maas agent had 'bootloader' instead of
'bootloaders' for non-UEFI bootloaders.

(cherry picked from commit c9c16ac166c1eab8a25ea95a43b9f6138d58ac26)

Author: Björn Tillenius
Author Date: 2024-03-13 10:32:46 UTC

Bug #2056740: Can't commission/deploy AMT machines

The maas-power script was passed "--boot-mode pxe" for AMT machines
and failed, since such a parameter wasn't defined.

All other power drivers, including the newer AMT one, sets PXE boot every
time the machine is powered on. I changed things, so that "boot-mode" is
no longer passed, and the older AMT driver also sets the machine to PXE
boot whenever it's turned on.

Author: Björn Tillenius
Author Date: 2024-03-13 08:08:10 UTC

Bug #2057750: Can't pxe boot legacy non-uefi machines

The rewrite rules in maas agent had 'bootloader' instead of
'bootloaders' for non-UEFI bootloaders.

Author: Adam Collard
Author Date: 2024-02-13 10:16:34 UTC

fix: LP:2052958 cope with disks with missing serial metadata

(cherry picked from commit f57ed04744e7110659d7b6833b2ad2b1571a020e)

Author: Björn Tillenius
Author Date: 2024-02-20 08:30:13 UTC

failed to detect a valid IP address from 'None'

In the configure workflow, the query for getting the region IPs
included None values, which broke the workflow.

Changed the query not to include any None values.

Author: Björn Tillenius
Author Date: 2024-02-14 13:54:38 UTC

Bug #2053033: Creating MAAS Virsh VM does not work (libvirt: error)

For some reason virsh prints out an error about executing
pkttyagent. This happens only on some machines, and we don't know why.

But the error seems harmless. This patch ignores the error and
lets the connection succeed.

(cherry picked from commit f0005e39bcadfc366c76e713c444739179743c84)
(cherry picked from commit 9eb3ca242e3c57f1b7ab1a1b6537ee4dfffb2aa6)
(cherry picked from commit 052a895bfac55e414bf5b02e5e28122bff12a532)

Author: Björn Tillenius
Author Date: 2024-02-14 11:44:33 UTC

Bug #2053033: Creating MAAS Virsh VM does not work (libvirt: error)

For some reason virsh prints out an error about executing
pkttyagent. This happens only on some machines, and we don't know why.

But the error seems harmless. This patch ignores the error and
lets the connection succeed.

(cherry picked from commit f0005e39bcadfc366c76e713c444739179743c84)
(cherry picked from commit 9eb3ca242e3c57f1b7ab1a1b6537ee4dfffb2aa6)

Author: Björn Tillenius
Author Date: 2024-02-14 11:44:33 UTC

Bug #2053033: Creating MAAS Virsh VM does not work (libvirt: error)

For some reason virsh prints out an error about executing
pkttyagent. This happens only on some machines, and we don't know why.

But the error seems harmless. This patch ignores the error and
lets the connection succeed.

(cherry picked from commit f0005e39bcadfc366c76e713c444739179743c84)
(cherry picked from commit 9eb3ca242e3c57f1b7ab1a1b6537ee4dfffb2aa6)

Author: Björn Tillenius
Author Date: 2024-02-14 10:29:40 UTC

Bug #2053033: Creating MAAS Virsh VM does not work (libvirt: error)

For some reason virsh prints out an error about executing
pkttyagent. This happens only on some machines, and we don't know why.

But the error seems harmless. This patch ignores the error and
lets the connection succeed.

(cherry picked from commit f0005e39bcadfc366c76e713c444739179743c84)

Author: Björn Tillenius
Author Date: 2024-02-14 09:31:00 UTC

Bug #2053033: Creating MAAS Virsh VM does not work (libvirt: error)

For some reason virsh prints out an error about executing
pkttyagent. This happens only on some machines, and we don't know why.

But the error seems harmless. This patch ignores the error and
lets the connection succeed.

Author: Björn Tillenius
Author Date: 2024-02-01 15:50:01 UTC

Lint.

Author: Björn Tillenius
Author Date: 2023-11-14 16:02:38 UTC

Add a new asyncio worker that can run workflows.

Author: Björn Tillenius
Author Date: 2023-09-01 07:04:15 UTC

Initial version inter-tech power driver.

Author: Björn Tillenius
Author Date: 2023-11-21 09:52:35 UTC

Add more logging for when power action fails.

We had the issue that a machine wouldn't turn on when told
to do so, but the logs didn't give any clues that this was
the case.

Author: Björn Tillenius
Author Date: 2023-11-20 16:30:40 UTC

Don't reference self inside of a @transaction block.

Inside a @transaction block, a new transaction is being created
and 'self' is a stale reference from an old transaction.

Author: Björn Tillenius
Author Date: 2023-11-02 10:16:01 UTC

Bug #204254: Machine sometimes revert to old status after power control Edit

The async methods related to power control uses multiple threads
and transactions, but they all used the same, stale, Node object.

When using deferToDatabase and transational(), you must always
get fresh values from the database.

Author: Björn Tillenius
Author Date: 2023-11-01 10:08:21 UTC

Allow maas-power to work with multiple-choice fields.

The IPMI driver allows multiple workaround flags to passed, but maas-power
wouldn't allow that.

Author: Björn Tillenius
Author Date: 2023-10-31 08:19:21 UTC

Bug #2041854: Bump timeout for Vault secret migration

MAAS can be busy and take more than 30 seconds to restart.

Bump timeout to be twice as long.

Author: Björn Tillenius
Author Date: 2023-09-22 09:47:31 UTC

Flag images that can be deployed to memory in the websocket handler.

bootresources.poll() now returns the 'canBeDeployToMemory' flag
for all resources.

Author: Björn Tillenius
Author Date: 2023-09-22 08:48:13 UTC

Allow non-Ubuntu OSes to be deployed to memory.

For uploaded files, deploying to memory is supported
if it's tgz or txz image. That's what the initramfs
supports.

Author: Björn Tillenius
Author Date: 2023-09-27 09:00:32 UTC

Use f-strings.

Author: Björn Tillenius
Author Date: 2023-09-25 08:49:50 UTC

Add missing type annotation.

Author: Björn Tillenius
Author Date: 2023-09-22 06:49:27 UTC

Fix intermittent test_POST_creates_boot_resource failure.

Author: Björn Tillenius
Author Date: 2023-09-14 07:10:25 UTC

Typo.

Author: Björn Tillenius
Author Date: 2023-09-12 07:05:08 UTC

WIP make tarball ephemeral deploys work.

Author: Björn Tillenius
Author Date: 2023-09-04 13:47:17 UTC

WIP

Author: Björn Tillenius
Author Date: 2023-07-20 10:13:40 UTC

WIP - zone handler.

Author: Björn Tillenius
Author Date: 2023-07-10 09:45:06 UTC

Add tests for the db_conn FastAPI dependency.

Author: Björn Tillenius
Author Date: 2023-07-11 12:58:05 UTC

Close db connections for each test.

Author: Björn Tillenius
Author Date: 2023-07-05 15:17:53 UTC

Rework maasapiserver tests not to use pytest-postgresql.

Author: Björn Tillenius
Author Date: 2023-07-05 10:41:05 UTC

Extract the non-Django database setup into its own pytest plugin.

The new database plugin still uses Django to setup the database,
but it doesn't import Django directly.

This allows the same fixtures to be used for sqlalchemy-based tests.

Author: Björn Tillenius
Author Date: 2023-07-05 15:17:53 UTC

Rework maasapiserver tests not to use pytest-postgresql.

Author: Björn Tillenius
Author Date: 2023-07-05 09:25:04 UTC

Fix URL.

Author: Björn Tillenius
Author Date: 2023-04-27 15:28:47 UTC

Add another whitespace.

Author: Björn Tillenius
Author Date: 2022-11-21 14:37:14 UTC

Add foreign key constraints for current_foo_script_set attributes.

Author: Björn Tillenius
Author Date: 2022-11-18 13:15:39 UTC

Move Script metadataserver models to maasserver.

Author: Björn Tillenius
Author Date: 2022-11-17 16:12:14 UTC

Move Node metadataserver models to maasserver.

Author: Björn Tillenius
Author Date: 2022-11-21 14:37:14 UTC

Add foreign key constraints for current_foo_script_set attributes.

Author: Alberto Donato
Author Date: 2023-02-03 09:39:12 UTC

Bug #2009805: machine deploy install_kvm=True fails

exclude discovered IPs when selecting which to use for VM hosts deploys

(cherry picked from commit 91442ef7aecbcd2caf345b8abad579876f5d9448)

Author: Björn Tillenius
Author Date: 2023-03-13 12:24:06 UTC

LP #2009805: machine deploy install_kvm=True fails

Only consider IPs that have been configured by MAAS when choosing the right IP
for the KVM endpoint.

Previously it considered any IP associated with the interface, including
DISCOVERED ones that where used while commissioning the machine.

I couldn't find a reliable way of testing this. There is already a test
for this, and most likely if failed intermittently.

Author: Björn Tillenius
Author Date: 2023-03-13 11:11:32 UTC

Bug #2003310: Refresh scripts are not re-run if they pass, but fail to report the results to the region

While refreshing the hardware information for controllers,
the scripts might all pass, but when reporting the results
back to the metadata server, something went wrong.

The exception was swallowed and ignored, though, so the
networking service didn't noticed it and assumed that
all scripts had passed.

This changes things so that the signal exception is caught
in the networking service itself, so it's aware that something
went wrong.

(cherry picked from commit 42cdcb554f65574d216aaff06ad592675920922a)

Author: Björn Tillenius
Author Date: 2023-03-10 14:25:33 UTC

Improve log message.

Author: Björn Tillenius
Author Date: 2023-03-09 13:55:17 UTC

Add some debug logging to debug bug #2003310.

Author: Björn Tillenius
Author Date: 2023-01-27 09:04:45 UTC

Make sqlalchemy handlers use new ip model.

Author: Björn Tillenius
Author Date: 2023-01-25 10:06:43 UTC

Allow boot_interface_id to be None in the sqlalchemy handlers.

Author: Björn Tillenius
Author Date: 2023-01-24 14:59:00 UTC

Fix sample data generation to generate valid vids.

MAAS ensures that the vid for VLANs are less than 4095.

Author: Björn Tillenius
Author Date: 2023-01-20 17:18:56 UTC

LP #1990172: "20-maas-03-machine-resources" commissioning script improperly reports a Pass when the test fails

The results was piped to 'tee', but pipefail wasn't set.

(cherry picked from commit 15f64a96a59ef39cde0bbb06b79948e05e03c147)

Author: Björn Tillenius
Author Date: 2023-01-20 16:37:13 UTC

LP #1990172: "20-maas-03-machine-resources" commissioning script improperly reports a Pass when the test fails

The results was piped to 'tee', but pipefail wasn't set.

Author: Björn Tillenius
Author Date: 2023-01-20 11:36:03 UTC

Run the performance tests together with the normal pytest tests.

They are quick to run with an empty database, and by running them
CI will ensure that they don't break.

Also change run-perf-tests-ci to not hide the output, to make
it easier to debug failures.

Author: Björn Tillenius
Author Date: 2023-01-17 14:32:40 UTC

Add an sqlalchemy handler that performs multiple queries.

The purpose of that is to see if we can optimize each query
individually to make it perform better.

Also change the way status_message is calculated, from a CTE
to a scalar subquery. This makes the query significantly faster.

Author: Björn Tillenius
Author Date: 2022-12-07 09:23:39 UTC

Add sqlalchemy core handler that gets all data in one query.

Currently it's not optimized at all. It returns the right data, but probably
needs to be rewritten in order to be faster.

There's also a handler using sqlalchemy ORM, but it's incomplete and disabled
for now.

Author: Björn Tillenius
Author Date: 2023-01-03 15:18:55 UTC

Make seeds reporting work when no setup has been done.

Author: Björn Tillenius
Author Date: 2023-01-05 10:12:45 UTC

Control the garbage collection for the performance tests.

Before, one test could create a lot of garbage that then got
collected while running another test. That made the second test
look slower than it actually was.

Now we collect the garbage before a test being run, and also
collect it after the test is being run, including the last collection
in the timing of the test.

Author: Björn Tillenius
Author Date: 2022-12-02 08:17:19 UTC

Compare ids instead of the length.

Author: Björn Tillenius
Author Date: 2022-11-28 16:50:41 UTC

Add debug logging to test_handles_missing_system_handler_on_notification.

It sometimes fails intermittently. I have a theory on why, but I want to see
some debug logs from a failure to confirm my finding, since I can't reproduce
it locally.

Author: Björn Tillenius
Author Date: 2022-11-24 14:48:55 UTC

Add a pytest DB fixture to set up the database for the tests.

You now no longer have to run the pytest tests that require a DB using
bin/database. Instead you run bin/pytest and it will set up the DB
automatically. It still uses the bin/database infrastructure under the hood,
and it will re-use the DB by default. So the time it takes to run the tests is
the same as before.

For the performance tests, it's possible to pass in a DB dump to use as well.

I removed pytest-django, since we want to control how the DB is
created and cleaned up.

Author: Björn Tillenius
Author Date: 2022-11-28 14:26:15 UTC

Print out the seeds for pytest test only if they fail.

Author: Björn Tillenius
Author Date: 2022-11-25 10:07:58 UTC

Fix tests.

Author: Björn Tillenius
Author Date: 2022-11-24 15:19:22 UTC

Drop default scope

Author: Björn Tillenius
Author Date: 2022-11-23 10:52:32 UTC

Remove the @perf_test decorator.

It's not needed, and it was somewhat broken. It reverted back to
a savepoint, unless commit_transaction was true. In that case
it simply removed the savepoint. I'm not quite sure what the intent
was there. If it was to simulate a commit, it was in the wrong
place, since it was outside the block that measures the duration.

The transaction is rolled back elsewhere for each tests, so no
need for a custom decorator. The decorator also had logic for
skipping the tests if django wasn't loaded, but I don't see
a use case for that.

Author: Björn Tillenius
Author Date: 2022-11-23 08:17:27 UTC

Make it move explicit what the measure in the performance tests

Before, we measured how long the whole test took to run. That includes
any local setup you do in the test, but also the setup of any fixtures
the test uses.

Author: Björn Tillenius
Author Date: 2022-11-22 14:41:50 UTC

Add a performance tests for listing all machines with the websocket.

With this it's easier to track what the default page size should be.

Author: Björn Tillenius
Author Date: 2022-11-22 14:28:31 UTC

No implicit, hardcoded, path for bin/test.perf

bin/test.perf always passed src/maasperf to pyttest. This made it
impossible to run only a single test.

CI uses utilities/run-perf-tests-ci, which already explicitly passes
src/maasperf as the path.

Author: Björn Tillenius
Author Date: 2022-11-21 10:24:11 UTC

Lint.

Author: Björn Tillenius
Author Date: 2022-10-11 09:26:00 UTC

Fix schemaspy after switch to jammy.

Author: Björn Tillenius
Author Date: 2022-10-28 13:05:36 UTC

Don't return ipv6 boot addresses for ipv4 subnets.

An ipv4 subnet won't be able to use ipv6 addresses, and it would break the
DHCP config.

(cherry picked from commit 425a78cfd666e05757bf6f72e63c48bdf0f807df)

Author: Björn Tillenius
Author Date: 2022-11-15 09:13:12 UTC

LP #1990383: Link subnet on new machine

Fixed the API so that it allows editing interfaces the same way the UI allows.

Author: Björn Tillenius
Author Date: 2022-11-15 08:26:31 UTC

Rename parameter.

Author: Björn Tillenius
Author Date: 2022-11-11 13:50:50 UTC

LP #1996074: Machine details stuck at "Loading" for machines with no disks

The issue was that if you delete all disks from a machine, the testing
node's current scriptset will be empty. Then when the machine was deploy, the
creation of the installation scriptset would clean up the empty testing
scripset, leaving a reference to a deleted row.

I've changed things so that when you create a new scriptset, we only clean up
scriptsets of the same type. I also fixed the tests, since they didn't actually
create any empty sets. The reason the tests passed before was that they set the
max limit to 1.

Now the real issue is that Node.current_testing_script_set doesn't have a
foreign key in the database, and that the 'on_delete=SET_NULL' doesn't work.
But considering that might be hard to fix, I think it's worth landing this
workaround for now.

Author: Björn Tillenius
Author Date: 2022-11-10 13:42:14 UTC

Fix test_PUT_admin_can_change_power_type intermittent failure.

If the power type is 'lxd', it fails, since it can't generate the certificate.

Author: Björn Tillenius
Author Date: 2022-11-09 17:20:16 UTC

Fix lint.

Author: Björn Tillenius
Author Date: 2022-11-09 16:28:30 UTC

LP #1835271: Ephemeral deployment keeps cloud-inits autogenerated netplan config

Author: Björn Tillenius
Author Date: 2022-11-09 10:03:24 UTC

Make snap-bind-mounts work with jammy.

Author: Björn Tillenius
Author Date: 2022-10-28 12:40:01 UTC

Don't return ipv6 boot addresses for ipv4 subnets.

An ipv4 subnet won't be able to use ipv6 addresses, and it would break the
DHCP config.

Author: Björn Tillenius
Author Date: 2022-10-24 09:11:38 UTC

LP #1937138: Calling mark_intro_complete doesn't respond correctly

Change UserHandler not to operate on self.user directly. self.user
is the autenticated user and should only be used to check the authentication
and authorizations.

Author: Björn Tillenius
Author Date: 2022-10-14 15:26:52 UTC

Add a few different test cases.

Author: Björn Tillenius
Author Date: 2022-10-12 12:42:51 UTC

Use hostname as label and value when grouping by parent in the websocket.

Author: Björn Tillenius
Author Date: 2022-09-30 10:15:12 UTC

Review fixes:
  * Exit after help
  * Don't switch the LXD project
  * Better help message at the end

Author: Björn Tillenius
Author Date: 2022-09-23 09:18:06 UTC

Explicity delete device children only when release a machine.

Previously, only devices could be a child of a machine. Now that
machines (like VMs and DPUs) can be children, we need to explicitly
filter out the non-device nodes.

DPUs shouldn't be deleted, and VMs will be deleted when the VM
host is deleted. There are checks that prevents a VM host from
being released.

Author: Björn Tillenius
Author Date: 2022-09-21 16:16:14 UTC

Bug #1989970: Can't enlist machines on subnets with DNS set

Fix the logic in build_metadata_url() to get the right rack
controller IP to use in case an external DNS is used.

Also fix get_apt_url() to use the same logic.

Additional logic was added to deal with DHCP relays.

(cherry picked from commit 2517f3acf519fc6d7f16e9e6b51b42180af1dc73)

Author: Björn Tillenius
Author Date: 2022-09-21 14:58:00 UTC

Review fixes.

Author: Björn Tillenius
Author Date: 2022-09-09 07:48:49 UTC

Prepare for 3.2.5 release

Author: Björn Tillenius
Author Date: 2022-07-05 10:38:55 UTC

Fix test_perf_list_machines_MachinesHander_direct_call.

When creating the emitter, it didn't pass in anonymous=False, which meant that
the default anonymous=True was being used.

When anonymous=True, a lot less data is returned.

Author: Gavin Lin
Author Date: 2022-06-03 15:16:33 UTC

Add Raritan PDU power driver.

(cherry picked from commit 2aa1d2b147aa8d544c1bff1742cce9047d3f1ee6)

Author: Björn Tillenius
Author Date: 2022-06-03 14:17:02 UTC

Add stats for power drivers.

Also added an attribute to BMC, so that we can track whether the BMC was
auto-detected in commissioning, or whether the user created it manually.

(cherry picked from commit ab360b09b24e750d106e07778574e72a8dfc99fc)

Author: Björn Tillenius
Author Date: 2022-06-03 14:00:25 UTC

Fix intermittent test failure.

Author: Björn Tillenius
Author Date: 2022-06-01 15:34:13 UTC

Remove unused code from Node.set_power_config().

There was code that checked for self.bmc, even though it couldn't really
be None, except for in unit tests.

Author: Björn Tillenius
Author Date: 2022-05-24 13:29:31 UTC

Move store_node_power_parameters to metadataserver.api.

That's the only place where it's used.

Author: Björn Tillenius
Author Date: 2022-05-20 14:49:00 UTC

Fix make lint-shell

Author: Björn Tillenius
Author Date: 2022-04-13 09:25:24 UTC

Serialize Subnets in batch.

Author: Björn Tillenius
Author Date: 2022-04-14 11:39:43 UTC

Fix test failures.

Author: Björn Tillenius
Author Date: 2022-03-30 14:22:36 UTC

Add discovered

Author: Björn Tillenius
Author Date: 2022-03-07 13:41:44 UTC

Don't use TYPE_CHECKING.

It was used incorrect, breaking real execution for the code. Given that
importing Machine is cheap, since the sampledata code already makes use
of it, it doesn't make much sense to optimize things using TYPE_CHECKING
checks here.

Author: Björn Tillenius
Author Date: 2022-03-04 12:44:30 UTC

Factor out common logic.

Author: Björn Tillenius
Author Date: 2022-02-28 11:40:39 UTC

Don't specify PYTHONPATH when building the snap.

Having a PYTHONPATH there breaks the build if a systemwide pip is
installed.

I tried building it without having build-environment defined. The snap
built ok, and the installed snap started up without any problems.

Author: Björn Tillenius
Author Date: 2022-02-24 08:14:36 UTC

Move FakeCommissioningData to maasserver.testing.

This makes it easier to reuse it for other tests and for sampledata.

Author: Björn Tillenius
Author Date: 2022-02-16 10:11:06 UTC

Make sure the subnets are big enough. Don't create new users when creating events.

Author: Björn Tillenius
Author Date: 2022-02-11 10:34:13 UTC

Review comments.

Author: Björn Tillenius
Author Date: 2022-02-09 14:12:06 UTC

Allow VLAN names to be cleared in websocket and API.

Also ensure that you need to be an admin to edit the VLAN.