Merge ~r00ta/maas:MAASENG-2165-support-filter-by-ephemeral-deployment into maas:master

[Jacopo Rota]

support filter by ephemeral_deploy property in machine list websocket handler

This MP aims to implement the task https://warthogs.atlassian.net/browse/MAASENG-2165 .

In particular, the user can now filter by the `ephemeral_deploy` property.
If `ephemeral_deploy=True`, then all the machines that are ephemerally deployed will be included in the result.
If `ephemeral_deploy=False`, then all the machines that are ephemerally deployed are excluded from the result.

If nothing is specified, all the machines are included in the result.

[MAAS Lander (maas-lander)]

UNIT TESTS
-b MAASENG-2165-support-filter-by-ephemeral-deployment lp:~r00ta/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/3551/console
COMMIT: d1d5232ab24fbc174ec1089a7ddb85fec1ab9927

review: Needs Fixing

[Björn Tillenius (bjornt)]

+1

review: Approve

996f30e... by Jacopo Rota on 2023-09-15

use deployment_target instead of ephemeral_deploy in filters

0b8d227... by Jacopo Rota on 2023-09-15

refactoring

5a73f84... by Jacopo Rota on 2023-09-15

refactoring

[MAAS Lander (maas-lander)]

UNIT TESTS
-b MAASENG-2165-support-filter-by-ephemeral-deployment lp:~r00ta/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/3553/console
COMMIT: 0b8d2277ff4e43dd6426f9ff475d1c0d4366e8ba

review: Needs Fixing

[MAAS Lander (maas-lander)]

UNIT TESTS
-b MAASENG-2165-support-filter-by-ephemeral-deployment lp:~r00ta/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/3554/console
COMMIT: 5a73f84cdf3dfc61c28f55f341e5b02656a38f6d

review: Needs Fixing

[Björn Tillenius (bjornt)]

+1

review: Approve

d6bf4ca... by Jacopo Rota on 2023-09-15

add deployment_target in filter_groups response

[MAAS Lander (maas-lander)]

UNIT TESTS
-b MAASENG-2165-support-filter-by-ephemeral-deployment lp:~r00ta/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/3557/console
COMMIT: d6bf4ca7567c36f8d5ab9ce68c43dd8ef1593095

review: Needs Fixing

[Jacopo Rota (r00ta)]

jenkins: !test

[MAAS Lander (maas-lander)]

UNIT TESTS
-b MAASENG-2165-support-filter-by-ephemeral-deployment lp:~r00ta/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/3559/console
COMMIT: d6bf4ca7567c36f8d5ab9ce68c43dd8ef1593095

review: Needs Fixing

ff755a7... by Jacopo Rota on 2023-09-15

fix dynamic test

[MAAS Lander (maas-lander)]

UNIT TESTS
-b MAASENG-2165-support-filter-by-ephemeral-deployment lp:~r00ta/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/3560/console
COMMIT: ff755a7195c105b87d124a5c36aa74dc85dd08b8

review: Needs Fixing

be2448c... by Jacopo Rota on 2023-09-15

fix dynamic test

[MAAS Lander (maas-lander)]

UNIT TESTS
-b MAASENG-2165-support-filter-by-ephemeral-deployment lp:~r00ta/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: SUCCESS
COMMIT: be2448cae6c5f6a3878dab5eeb018b87a31bc099

review: Approve

diff --git a/src/maasserver/enum.py b/src/maasserver/enum.py
index e12bd46..6b98fb2 100644
--- a/src/maasserver/enum.py
+++ b/src/maasserver/enum.py
@@ -411,6 +411,16 @@ class POWER_STATE:
 POWER_STATE_CHOICES = enum_choices(POWER_STATE, transform=str.capitalize)
 
 
+class DEPLOYMENT_TARGET:
+    # A node has been deployed ephemerally
+    MEMORY = "memory"
+
+    DISK = "disk"
+
+
+DEPLOYMENT_TARGET_CHOICES = enum_choices(DEPLOYMENT_TARGET)
+
+
 class BOOT_RESOURCE_TYPE:
     """Possible types for `BootResource`."""
 
diff --git a/src/maasserver/node_constraint_filter_forms.py b/src/maasserver/node_constraint_filter_forms.py
index 41614e5..c76a668 100644
--- a/src/maasserver/node_constraint_filter_forms.py
+++ b/src/maasserver/node_constraint_filter_forms.py
@@ -17,6 +17,8 @@ from django.forms.fields import Field
 from netaddr import IPAddress
 
 from maasserver.enum import (
+    DEPLOYMENT_TARGET,
+    DEPLOYMENT_TARGET_CHOICES,
     NODE_STATUS,
     NODE_STATUS_SHORT_LABEL_CHOICES,
     POWER_STATE,
@@ -633,6 +635,7 @@ STATIC_FILTER_FIELDS = (
     "power_state",
     "simple_status",
     "status",
+    "deployment_target",
 )
 
 
@@ -934,9 +937,24 @@ class FilterNodeForm(forms.Form):
         clean_prefix="=",
     )
 
+    deployment_target = ConstrainedMultipleChoiceField(
+        label="Deployment target",
+        choices=DEPLOYMENT_TARGET_CHOICES,
+        required=False,
+        clean_prefix="=",
+    )
+
+    not_deployment_target = ConstrainedMultipleChoiceField(
+        label="Deployment target",
+        choices=DEPLOYMENT_TARGET_CHOICES,
+        required=False,
+        clean_prefix="=",
+    )
+
     ignore_unknown_constraints = False
 
     NODE_FILTERS = {
+        "deployment_target": ("ephemeral_deploy", _match_any),
         "fabric_classes": (
             "current_config__interface__vlan__fabric__class_type",
             _match_any,
@@ -965,6 +983,7 @@ class FilterNodeForm(forms.Form):
     }
 
     NODE_EXCLUDES = {
+        "not_deployment_target": ("ephemeral_deploy", _match_any),
         "not_fabric_classes": (
             "current_config__interface__vlan__fabric__class_type",
             _match_any,
@@ -1147,6 +1166,27 @@ class FilterNodeForm(forms.Form):
         values = self.cleaned_data["not_power_state"]
         return [getattr(POWER_STATE, a.upper()) for a in values]
 
+    def _convert_deployment_target_to_ephemeral_deploy(self, target):
+        if target is None:
+            return None
+        if target == DEPLOYMENT_TARGET.MEMORY:
+            return True
+        return False
+
+    def clean_deployment_target(self):
+        values = self.cleaned_data["deployment_target"]
+        return [
+            self._convert_deployment_target_to_ephemeral_deploy(a)
+            for a in values
+        ]
+
+    def clean_not_deployment_target(self):
+        values = self.cleaned_data["not_deployment_target"]
+        return [
+            self._convert_deployment_target_to_ephemeral_deploy(a)
+            for a in values
+        ]
+
     def clean(self):
         if not self.ignore_unknown_constraints:
             unknown_constraints = set(self.data).difference(set(self.fields))
diff --git a/src/maasserver/tests/test_node_constraint_filter_forms.py b/src/maasserver/tests/test_node_constraint_filter_forms.py
index 2414112..6a4c0b7 100644
--- a/src/maasserver/tests/test_node_constraint_filter_forms.py
+++ b/src/maasserver/tests/test_node_constraint_filter_forms.py
@@ -14,6 +14,7 @@ from testtools.matchers import (
 )
 
 from maasserver.enum import (
+    DEPLOYMENT_TARGET,
     FILESYSTEM_GROUP_TYPE,
     FILESYSTEM_TYPE,
     INTERFACE_TYPE,
@@ -1555,6 +1556,8 @@ class TestFilterNodeForm(MAASServerTestCase, FilterConstraintsMixin):
             "not_owner": factory.make_User().username,
             "power_state": POWER_STATE.ON,
             "not_power_state": POWER_STATE.OFF,
+            "deployment_target": DEPLOYMENT_TARGET.MEMORY,
+            "not_deployment_target": DEPLOYMENT_TARGET.DISK,
         }
         form = FilterNodeForm(data=constraints)
         self.assertTrue(form.is_valid(), form.errors)
@@ -2123,6 +2126,8 @@ class TestAcquireNodeForm(MAASServerTestCase, FilterConstraintsMixin):
             "not_owner": factory.make_User().username,
             "power_state": POWER_STATE.ON,
             "not_power_state": POWER_STATE.OFF,
+            "deployment_target": DEPLOYMENT_TARGET.MEMORY,
+            "not_deployment_target": DEPLOYMENT_TARGET.DISK,
         }
         form = AcquireNodeForm(data=constraints)
         self.assertTrue(form.is_valid(), form.errors)
diff --git a/src/maasserver/websockets/handlers/node.py b/src/maasserver/websockets/handlers/node.py
index 03baa03..f60d52c 100644
--- a/src/maasserver/websockets/handlers/node.py
+++ b/src/maasserver/websockets/handlers/node.py
@@ -15,6 +15,7 @@ from django.db.models import Prefetch, Subquery
 from lxml import etree
 
 from maasserver.enum import (
+    DEPLOYMENT_TARGET_CHOICES,
     FILESYSTEM_FORMAT_TYPE_CHOICES,
     FILESYSTEM_FORMAT_TYPE_CHOICES_DICT,
     INTERFACE_TYPE,
@@ -1601,6 +1602,11 @@ class NodeHandler(TimestampedModelHandler):
                         {"key": choice, "label": str(choice)}
                         for choice in (True, False)
                     ]
+                elif key == "deployment_target":
+                    return [
+                        {"key": choice, "label": str(choice)}
+                        for choice in DEPLOYMENT_TARGET_CHOICES
+                    ]
             else:
                 return self._get_dynamic_filter_options(key)
 
diff --git a/src/maasserver/websockets/handlers/tests/test_machine.py b/src/maasserver/websockets/handlers/tests/test_machine.py
index 5cc29bd..d8655bd 100644
--- a/src/maasserver/websockets/handlers/tests/test_machine.py
+++ b/src/maasserver/websockets/handlers/tests/test_machine.py
@@ -5908,6 +5908,20 @@ class TestMachineHandlerNewSchema(MAASServerTestCase):
                     "for_grouping": False,
                 },
                 {
+                    "key": "deployment_target",
+                    "label": "Deployment target",
+                    "dynamic": False,
+                    "type": "list[str]",
+                    "for_grouping": False,
+                },
+                {
+                    "key": "not_deployment_target",
+                    "label": "Deployment target",
+                    "dynamic": True,
+                    "type": "list[str]",
+                    "for_grouping": False,
+                },
+                {
                     "key": "storage",
                     "label": "Storage",
                     "dynamic": True,
diff --git a/src/tests/maasserver/websockets/handlers/test_machine.py b/src/tests/maasserver/websockets/handlers/test_machine.py
index f073699..7f98d26 100644
--- a/src/tests/maasserver/websockets/handlers/test_machine.py
+++ b/src/tests/maasserver/websockets/handlers/test_machine.py
@@ -339,6 +339,47 @@ class TestMachineHandlerNewSchema:
         )
         assert result["groups"][0]["count"] == 1
 
+    def test_filter_deployment_target(self):
+        user, session = factory.make_User_with_session()
+        node_with_ephemeral_deployment = factory.make_Node(
+            owner=user, status=NODE_STATUS.NEW, ephemeral_deploy=True
+        )
+        node_with_standard_deployment = factory.make_Node(
+            owner=user, status=NODE_STATUS.NEW, ephemeral_deploy=False
+        )
+        handler = MachineHandler(
+            user, {}, None, session_id=session.session_key
+        )
+
+        result = handler.list_ids({})
+        assert result["groups"][0]["count"] == 2
+
+        result = handler.list_ids(
+            {
+                "filter": {
+                    "deployment_target": ["=memory"],
+                }
+            }
+        )
+        assert result["groups"][0]["count"] == 1
+        assert (
+            result["groups"][0]["items"][0]["id"]
+            == node_with_ephemeral_deployment.id
+        )
+
+        result = handler.list_ids(
+            {
+                "filter": {
+                    "deployment_target": ["=disk"],
+                }
+            }
+        )
+        assert result["groups"][0]["count"] == 1
+        assert (
+            result["groups"][0]["items"][0]["id"]
+            == node_with_standard_deployment.id
+        )
+
     def test_filter_counters(self):
         user, session = factory.make_User_with_session()
         nodes = [factory.make_Machine(owner=user) for _ in range(2)]