Merge ~r00ta/maas:lp-2058273-sync-bootresources-deterministic into maas:3.5

Proposed by Jacopo Rota
Status: Superseded
Proposed branch: ~r00ta/maas:lp-2058273-sync-bootresources-deterministic
Merge into: maas:3.5
Diff against target: 34 lines (+5/-3)
1 file modified
src/maasserver/workflow/bootresource.py (+5/-3)
Reviewer Review Type Date Requested Status
MAAS Lander Approve
MAAS Maintainers Pending
Review via email: mp+462615@code.launchpad.net

This proposal supersedes a proposal from 2024-03-18.

This proposal has been superseded by a proposal from 2024-03-18.

Commit message

fix: lp-2058273. Make the sync-bootresources workflow deterministic

Description of the change

The temporal workflows must be deterministic. This MP aims to use the random generator provided by the temporal sdk to randomize the source list for the images in order to balance the load on the available regions.

To post a comment you must log in.
4010fa6... by Jacopo Rota

fix: lp-2058273. Make the sync-bootresources workflow deterministic

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b lp-2058273-sync-bootresources-deterministic lp:~r00ta/maas/+git/maas into -b 3.5 lp:~maas-committers/maas

STATUS: SUCCESS
COMMIT: a0a79dbf6c9b9a5184bd61c68c80c3eb086f9c8a

review: Approve

Unmerged commits

4010fa6... by Jacopo Rota

fix: lp-2058273. Make the sync-bootresources workflow deterministic

81a9976... by Jacopo Rota

fix: lp-2058037. Ensure get-bootresourcefile-endpoints activity fails if the region is not exposing its ip yet.

3b58876... by Anton Troyanov

fix: /run/maas should be owned by maas

`/run` is usually a tmpfs mount, and it is not persistent across reboots, so
`/run/maas` should not be created using .dirs, .install or .postinst scripts.

MAAS has various services that store their directories and files under
`/run/maas`, but because some services are run as root (e.g. dhcp), this leads
to ownership issues when `mkdir -p` is used (e.g. DHCP will create
`/run/maas/dhcp`)

In order to solve ownership issue, this commit changes DHCP service directory
to `/run/maas-dhcp` while keeping `/run/maas` only for files related to MAAS
itself (not external 3rd party services)

Resolves LP:2056222
Resolves LP:2056225

b7f2542... by Jones Ogolo <email address hidden>

Update initial SQL

6466b0c... by Jones Ogolo <email address hidden>

Update maas-ui to 84ff1641a
feat: update image sync (tooltip) information (#5352)

07a613f... by Peter Makowski

Update maas-ui to 2384eb408
fix: cannot stop importing images MAASENG-2864 (#5351)

d312185... by Peter Makowski

Update maas-ui to 064adb558
fix: cannot delete image while downloading MAASENG-2863 (#5350)

Fixes: lp#232057935

5d5a0aa... by Jones Ogolo <email address hidden>

Update maas-ui to 7d88fb2f4
feat: move non-partition collapsible forms to side panel (#5349)

7896226... by Anton Troyanov

chore: reflect workflow type in wfid

069e184... by Alexsander de Souza

fix(bootresource): lp-2057979. Avoid workflow id clash.

ID must be unique within a namespace

Update scan failed

At least one of the branches involved have failed to scan. You can manually schedule a rescan if required.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/src/maasserver/workflow/bootresource.py b/src/maasserver/workflow/bootresource.py
index 4097f7e..559d03b 100644
--- a/src/maasserver/workflow/bootresource.py
+++ b/src/maasserver/workflow/bootresource.py
@@ -2,14 +2,13 @@ import asyncio
2from asyncio import gather2from asyncio import gather
3from dataclasses import dataclass, field, replace3from dataclasses import dataclass, field, replace
4from datetime import datetime, timedelta4from datetime import datetime, timedelta
5import random
6from typing import Coroutine, Sequence5from typing import Coroutine, Sequence
76
8from aiohttp.client_exceptions import ClientError7from aiohttp.client_exceptions import ClientError
9from temporalio import activity, workflow8from temporalio import activity, workflow
10from temporalio.common import RetryPolicy, WorkflowIDReusePolicy9from temporalio.common import RetryPolicy, WorkflowIDReusePolicy
11from temporalio.exceptions import ApplicationError10from temporalio.exceptions import ApplicationError
12from temporalio.workflow import ActivityCancellationType11from temporalio.workflow import ActivityCancellationType, random
1312
14from maasserver.utils.bootresource import (13from maasserver.utils.bootresource import (
15 get_bootresource_store_path,14 get_bootresource_store_path,
@@ -298,9 +297,12 @@ class SyncBootResourcesWorkflow:
298 for reg in sources297 for reg in sources
299 for ep in endpoints[reg]298 for ep in endpoints[reg]
300 ]299 ]
300 # In order to balance the workload on the regions we randomize the order of the source_list. We use the random
301 # generator provided by Temporal to keep the workflow execution deterministic.
302 generator = random()
301 new_res = replace(303 new_res = replace(
302 res,304 res,
303 source_list=random.sample(eps, min(len(eps), MAX_SOURCES)),305 source_list=generator.sample(eps, min(len(eps), MAX_SOURCES)),
304 )306 )
305 for region in missing:307 for region in missing:
306 sync_jobs.append(_schedule(new_res, region))308 sync_jobs.append(_schedule(new_res, region))

Subscribers

People subscribed via source and target branches