Merge lp:~ubuntudotcom1/maas/bug-1391228 into lp:~maas-committers/maas/trunk

Proposed by ubuntudotcom1
Status: Merged
Approved by: ubuntudotcom1
Approved revision: no longer in the source branch.
Merged at revision: 3776
Proposed branch: lp:~ubuntudotcom1/maas/bug-1391228
Merge into: lp:~maas-committers/maas/trunk
Diff against target: 2146 lines (+1693/-82)
5 files modified
src/maasserver/api/nodes.py (+211/-51)
src/maasserver/api/tests/test_nodes.py (+842/-28)
src/maasserver/models/eventtype.py (+10/-2)
src/maasserver/urls_api.py (+4/-1)
src/maasserver/views/nodes.py (+626/-0)
To merge this branch: bzr merge lp:~ubuntudotcom1/maas/bug-1391228
Reviewer Review Type Date Requested Status
Blake Rouse (community) Approve
Review via email: mp+254576@code.launchpad.net

Commit message

Addresses bug 1391228 with a new api operation for retrieving event logs (per the MAAS team Event API spec).

Description of the change

Addresses bug 1391228 with a new api operation for retrieving event logs (per the MAAS team Event API spec).

To post a comment you must log in.
Revision history for this message
Blake Rouse (blake-rouse) wrote :

Overall the implementation looks good. Just a few things that need to be fixed. The biggest one is that 'id' should be renamed to 'system_id'.

review: Needs Fixing
Revision history for this message
ubuntudotcom1 (ubuntudotcom1) wrote :

Address comments. Moved log levels to inverse dict in models/eventtype.py as LOGGING_LEVELS_BY_NAME.

Revision history for this message
Blake Rouse (blake-rouse) wrote :

Thanks for all the fixes. Looks good.

review: Approve
Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (2.3 MiB)

The attempt to merge lp:~ubuntudotcom1/maas/bug-1391228 into lp:maas failed. Below is the output from the failed tests.

Ign http://security.ubuntu.com trusty-security InRelease
Get:1 http://security.ubuntu.com trusty-security Release.gpg [933 B]
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Get:2 http://security.ubuntu.com trusty-security Release [63.5 kB]
Ign http://nova.clouds.archive.ubuntu.com trusty-updates InRelease
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Get:3 http://nova.clouds.archive.ubuntu.com trusty-updates Release.gpg [933 B]
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Get:4 http://nova.clouds.archive.ubuntu.com trusty-updates Release [63.5 kB]
Get:5 http://security.ubuntu.com trusty-security/main Sources [76.1 kB]
Get:6 http://security.ubuntu.com trusty-security/universe Sources [19.1 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Get:7 http://security.ubuntu.com trusty-security/main amd64 Packages [251 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Get:8 http://security.ubuntu.com trusty-security/universe amd64 Packages [91.6 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Get:9 http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources [190 kB]
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Get:10 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources [108 kB]
Get:11 http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages [490 kB]
Get:12 http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages [263 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Fetched 1,618 kB in 3s (488 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
     --no-install-recommends install apache2 authbind bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm pep8 postgresql pyflakes python-apt python-bson python-bzrlib python-convoy python-coverage python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-extras python-fixtures python-flake8 python-formencode python-hivex python-httplib2 python-iscpy python-jinja2 python-jsonschema python-lockfile python-lxml python-mock python-netaddr python-netifaces python-nose python-...

Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (85.0 KiB)

The attempt to merge lp:~ubuntudotcom1/maas/bug-1391228 into lp:maas failed. Below is the output from the failed tests.

Ign http://security.ubuntu.com trusty-security InRelease
Hit http://security.ubuntu.com trusty-security Release.gpg
Hit http://security.ubuntu.com trusty-security Release
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Ign http://nova.clouds.archive.ubuntu.com trusty-updates InRelease
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Hit http://nova.clouds.archive.ubuntu.com trusty-updates Release.gpg
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Hit http://security.ubuntu.com trusty-security/main Sources
Hit http://nova.clouds.archive.ubuntu.com trusty-updates Release
Hit http://security.ubuntu.com trusty-security/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Hit http://security.ubuntu.com trusty-security/main amd64 Packages
Hit http://security.ubuntu.com trusty-security/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
     --no-install-recommends install apache2 authbind bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm pep8 postgresql pyflakes python-apt python-bson python-bzrlib python-convoy python-coverage python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-extras python-fixtures python-flake8 python-formencode python-hivex python-httplib2 python-iscpy python-jinja2 python-jsonschema python-lockfile python-lxml python-mock python-netaddr python-netifaces python-nose python-oauth python-openssl python-paramiko python-pexpect python-pip python-pocket-lint python-psycopg2 python-pyinotify python-pyparsing python-seamicroclient python-simplejson ...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'src/maasserver/api/nodes.py'
2--- src/maasserver/api/nodes.py 2015-03-30 18:17:56 +0000
3+++ src/maasserver/api/nodes.py 2015-04-03 19:28:00 +0000
4@@ -5,7 +5,7 @@
5 absolute_import,
6 print_function,
7 unicode_literals,
8- )
9+)
10
11 str = None
12
13@@ -14,17 +14,22 @@
14 "AnonNodesHandler",
15 "NodeHandler",
16 "NodesHandler",
17+ "EventsHandler",
18 "store_node_power_parameters",
19- ]
20+]
21
22 from base64 import b64decode
23+import logging
24+import urllib
25
26 import bson
27 import crochet
28 from django.conf import settings
29 from django.core.exceptions import PermissionDenied
30+from django.core.urlresolvers import reverse
31 from django.http import HttpResponse
32 from django.shortcuts import get_object_or_404
33+from formencode.validators import Int
34 from maasserver import locks
35 from maasserver.api.logger import maaslog
36 from maasserver.api.support import (
37@@ -38,6 +43,7 @@
38 get_oauth_token,
39 get_optional_list,
40 get_optional_param,
41+ get_overridden_query_dict,
42 )
43 from maasserver.clusterrpc.power_parameters import get_power_types
44 from maasserver.dns.config import dns_update_zones
45@@ -64,9 +70,11 @@
46 NodeActionForm,
47 )
48 from maasserver.models import (
49+ Event,
50 MACAddress,
51 Node,
52 )
53+from maasserver.models.eventtype import LOGGING_LEVELS_BY_NAME
54 from maasserver.models.node import RELEASABLE_STATUSES
55 from maasserver.models.nodeprobeddetails import get_single_probed_details
56 from maasserver.node_action import Commission
57@@ -74,6 +82,7 @@
58 from maasserver.rpc import getClientFor
59 from maasserver.utils import find_nodegroup
60 from maasserver.utils.orm import get_first
61+from maasserver.views.nodes import event_to_dict
62 from piston.utils import rc
63 from provisioningserver.power.poweraction import (
64 PowerActionFail,
65@@ -118,8 +127,11 @@
66 'model',
67 'serial',
68 'tags',
69- )),
70- )
71+ )),
72+)
73+
74+MAX_EVENT_LOG_COUNT = 1000
75+DEFAULT_EVENT_LOG_LIMIT = 100
76
77
78 def store_node_power_parameters(node, request):
79@@ -148,6 +160,195 @@
80 node.save()
81
82
83+def filtered_nodes_list_from_request(request):
84+ """List Nodes visible to the user, optionally filtered by criteria.
85+
86+ :param hostname: An optional hostname. Only events relating to the node
87+ with the matching hostname will be returned. This can be specified
88+ multiple times to get events relating to more than one node.
89+ :param mac_address: An optional MAC address. Only events relating to the
90+ node owning the specified MAC address will be returned. This can be
91+ specified multiple times to get events relating to more than one node.
92+ :param id: An optional list of system ids. Only events relating to the
93+ nodes with matching system ids will be returned.
94+ :param zone: An optional name for a physical zone. Only events relating to
95+ the nodes in the zone will be returned.
96+ :param agent_name: An optional agent name. Only events relating to the
97+ nodes with matching agent names will be returned.
98+ """
99+ # Get filters from request.
100+ match_ids = get_optional_list(request.GET, 'id')
101+
102+ match_macs = get_optional_list(request.GET, 'mac_address')
103+ if match_macs is not None:
104+ invalid_macs = [
105+ mac for mac in match_macs if MAC_RE.match(mac) is None]
106+ if len(invalid_macs) != 0:
107+ raise MAASAPIValidationError(
108+ "Invalid MAC address(es): %s" % ", ".join(invalid_macs))
109+
110+ # Fetch nodes and apply filters.
111+ nodes = Node.nodes.get_nodes(
112+ request.user, NODE_PERMISSION.VIEW, ids=match_ids)
113+ if match_macs is not None:
114+ nodes = nodes.filter(macaddress__mac_address__in=match_macs)
115+ match_hostnames = get_optional_list(request.GET, 'hostname')
116+ if match_hostnames is not None:
117+ nodes = nodes.filter(hostname__in=match_hostnames)
118+ match_zone_name = request.GET.get('zone', None)
119+ if match_zone_name is not None:
120+ nodes = nodes.filter(zone__name=match_zone_name)
121+ match_agent_name = request.GET.get('agent_name', None)
122+ if match_agent_name is not None:
123+ nodes = nodes.filter(agent_name=match_agent_name)
124+
125+ return nodes
126+
127+
128+class EventsHandler(OperationsHandler):
129+ """Retrieve filtered node events.
130+
131+ A specific Node's events is identified by specifying one or more
132+ ids, hostnames, or mac addresses as a list.
133+ """
134+ api_doc_section_name = "Events"
135+
136+ create = read = update = delete = None
137+
138+ model = Event
139+
140+ all_params = (
141+ 'after',
142+ 'agent_name',
143+ 'id',
144+ 'level',
145+ 'limit',
146+ 'mac_address',
147+ 'op',
148+ 'zone')
149+
150+ @classmethod
151+ def resource_uri(cls, *args, **kwargs):
152+ return ('events_handler', [])
153+
154+ @operation(idempotent=True)
155+ def query(self, request):
156+ """List Node events, optionally filtered by various criteria via
157+ URL query parameters.
158+
159+ :param hostname: An optional hostname. Only events relating to the node
160+ with the matching hostname will be returned. This can be specified
161+ multiple times to get events relating to more than one node.
162+ :param mac_address: An optional list of MAC addresses. Only
163+ nodes with matching MAC addresses will be returned.
164+ :param id: An optional list of system ids. Only nodes with
165+ matching system ids will be returned.
166+ :param zone: An optional name for a physical zone. Only nodes in the
167+ zone will be returned.
168+ :param agent_name: An optional agent name. Only nodes with
169+ matching agent names will be returned.
170+ :param level: Desired minimum log level of returned events. Returns
171+ this level of events and greater. Choose from: %(log_levels)s.
172+ The default is INFO.
173+ """
174+
175+ # Filter first by optional node id, hostname, or mac
176+ nodes = filtered_nodes_list_from_request(request)
177+ limit = get_optional_param(
178+ request.GET, "limit", DEFAULT_EVENT_LOG_LIMIT, Int)
179+
180+ start_event_id_param = get_optional_param(
181+ request.GET, 'after', None, Int)
182+
183+ log_level = get_optional_param(request.GET, 'level', 'INFO')
184+
185+ if limit > MAX_EVENT_LOG_COUNT:
186+ raise MAASAPIBadRequest((
187+ "Requested number of events %d is greater than"
188+ " limit: %d") % (limit, MAX_EVENT_LOG_COUNT))
189+
190+ if start_event_id_param is not None:
191+ node_events = Event.objects.filter(
192+ id__gte=start_event_id_param,
193+ node=nodes)
194+ start_event_id = start_event_id_param
195+ else:
196+ node_events = Event.objects.filter(node=nodes)
197+ start_event_id = 0
198+
199+ # Filter next by log level >= to 'level', if specified
200+ if log_level is None and log_level != 'NOTSET':
201+ numeric_log_level = logging.NOTSET
202+ elif log_level in LOGGING_LEVELS_BY_NAME:
203+ numeric_log_level = LOGGING_LEVELS_BY_NAME[log_level]
204+ assert isinstance(numeric_log_level, int)
205+ else:
206+ raise MAASAPIBadRequest(
207+ "Unknown log level: %s" % log_level)
208+
209+ if log_level is not None and log_level != 'NOTSET':
210+ node_events = node_events.exclude(
211+ type__level__lt=numeric_log_level)
212+
213+ # Future feature:
214+ # This is where we would filter for events 'since last node deployment'
215+ # using a query param like since_last_deployed=true, but we aren't
216+ # right now because we don't currently record a timestamp of the last
217+ # deployment, and we don't have an event subtype for node status
218+ # changes to filter for the deploying status event.
219+
220+ base_path = reverse('events_handler')
221+
222+ prev_uri_params = get_overridden_query_dict(
223+ request.GET,
224+ {'after': max(0, start_event_id - limit)}, self.all_params)
225+ prev_uri = '%s?%s' % (base_path,
226+ urllib.urlencode(
227+ prev_uri_params,
228+ doseq=True))
229+
230+ next_uri_params = get_overridden_query_dict(
231+ request.GET,
232+ {'after': start_event_id + limit}, self.all_params)
233+ next_uri = '%s?%s' % (base_path,
234+ urllib.urlencode(
235+ next_uri_params,
236+ doseq=True))
237+
238+ node_events = (
239+ node_events.all().order_by('id')
240+ .prefetch_related('type')
241+ .prefetch_related('node'))
242+
243+ # Lastly, order by id and return up to 'limit' events
244+ if start_event_id_param is not None:
245+ # If start_event_id is specified, limit to a window of
246+ # 'limit' events with 'start_event_id' being the id
247+ # of the oldest event
248+ node_events = node_events.order_by('id')
249+ node_events = reversed(node_events[:limit])
250+ else:
251+ # If start_event_id is not specified, limit to most recent
252+ # 'limit' events
253+ node_events = node_events.order_by('-id')
254+ node_events = node_events[:limit]
255+
256+ # We need to load all of these events at some point, so save them
257+ # into a list now so that len() is cheap.
258+ node_events = list(node_events)
259+
260+ displayed_events_count = len(node_events)
261+ events_dict = dict(
262+ count=displayed_events_count,
263+ events=[event_to_dict(event) for event in node_events],
264+ next_uri=next_uri,
265+ prev_uri=prev_uri,
266+ )
267+ return events_dict
268+
269+ query.__doc__ %= {"log_levels": ", ".join(LOGGING_LEVELS_BY_NAME)}
270+
271+
272 class NodeHandler(OperationsHandler):
273 """Manage an individual Node.
274
275@@ -174,7 +375,7 @@
276 old_deployed_status_aliases = [
277 NODE_STATUS.RELEASING, NODE_STATUS.DISK_ERASING,
278 NODE_STATUS.FAILED_RELEASING, NODE_STATUS.FAILED_DISK_ERASING,
279- ]
280+ ]
281 deployed_aliases = (
282 old_allocated_status_aliases + old_deployed_status_aliases)
283 if node.status in deployed_aliases:
284@@ -515,9 +716,9 @@
285 alloc_type=IPADDRESS_TYPE.STICKY, ip=address)
286
287 if len(deallocated_ips) == 0 and address is not None:
288- raise MAASAPIBadRequest(
289- "%s: could not deallocate sticky IP address: %s",
290- node.hostname, address)
291+ raise MAASAPIBadRequest(
292+ "%s: could not deallocate sticky IP address: %s",
293+ node.hostname, address)
294 else:
295 maaslog.info(
296 "%s: Sticky IP address(es) deallocated: %s", node.hostname,
297@@ -909,7 +1110,7 @@
298 'commissioning': NODE_STATUS.COMMISSIONING,
299 'failed_tests': NODE_STATUS.FAILED_COMMISSIONING,
300 'minutes': settings.COMMISSIONING_TIMEOUT
301- }
302+ }
303 query = Node.nodes.raw("""
304 UPDATE maasserver_node
305 SET
306@@ -978,48 +1179,7 @@
307
308 @operation(idempotent=True)
309 def list(self, request):
310- """List Nodes visible to the user, optionally filtered by criteria.
311-
312- :param hostname: An optional list of hostnames. Only nodes with
313- matching hostnames will be returned.
314- :type hostname: iterable
315- :param mac_address: An optional list of MAC addresses. Only
316- nodes with matching MAC addresses will be returned.
317- :type mac_address: iterable
318- :param id: An optional list of system ids. Only nodes with
319- matching system ids will be returned.
320- :type id: iterable
321- :param zone: An optional name for a physical zone. Only nodes in the
322- zone will be returned.
323- :type zone: unicode
324- :param agent_name: An optional agent name. Only nodes with
325- matching agent names will be returned.
326- :type agent_name: unicode
327- """
328- # Get filters from request.
329- match_ids = get_optional_list(request.GET, 'id')
330- match_macs = get_optional_list(request.GET, 'mac_address')
331- if match_macs is not None:
332- invalid_macs = [
333- mac for mac in match_macs if MAC_RE.match(mac) is None]
334- if len(invalid_macs) != 0:
335- raise MAASAPIValidationError(
336- "Invalid MAC address(es): %s" % ", ".join(invalid_macs))
337-
338- # Fetch nodes and apply filters.
339- nodes = Node.nodes.get_nodes(
340- request.user, NODE_PERMISSION.VIEW, ids=match_ids)
341- if match_macs is not None:
342- nodes = nodes.filter(macaddress__mac_address__in=match_macs)
343- match_hostnames = get_optional_list(request.GET, 'hostname')
344- if match_hostnames is not None:
345- nodes = nodes.filter(hostname__in=match_hostnames)
346- match_zone_name = request.GET.get('zone', None)
347- if match_zone_name is not None:
348- nodes = nodes.filter(zone__name=match_zone_name)
349- match_agent_name = request.GET.get('agent_name', None)
350- if match_agent_name is not None:
351- nodes = nodes.filter(agent_name=match_agent_name)
352+ nodes = filtered_nodes_list_from_request(request)
353
354 # Prefetch related objects that are needed for rendering the result.
355 nodes = nodes.prefetch_related('macaddress_set__node')
356
357=== modified file 'src/maasserver/api/tests/test_nodes.py'
358--- src/maasserver/api/tests/test_nodes.py 2015-03-27 06:41:39 +0000
359+++ src/maasserver/api/tests/test_nodes.py 2015-04-03 19:28:00 +0000
360@@ -7,7 +7,7 @@
361 absolute_import,
362 print_function,
363 unicode_literals,
364- )
365+)
366
367 str = None
368
369@@ -15,26 +15,39 @@
370 __all__ = []
371
372 import httplib
373+from itertools import combinations
374 import json
375+import logging
376 import random
377+from random import randint
378+from urlparse import (
379+ parse_qsl,
380+ urlparse,
381+)
382
383 import crochet
384 from django.core.urlresolvers import reverse
385+from django.http import QueryDict
386 from maasserver import forms
387 from maasserver.api import nodes as nodes_module
388+from maasserver.api.utils import get_overridden_query_dict
389 from maasserver.enum import (
390 NODE_STATUS,
391 NODE_STATUS_CHOICES_DICT,
392 NODEGROUP_STATUS,
393 NODEGROUPINTERFACE_MANAGEMENT,
394 )
395-from maasserver.exceptions import ClusterUnavailable
396+from maasserver.exceptions import (
397+ ClusterUnavailable,
398+ MAASAPIValidationError,
399+)
400 from maasserver.fields import MAC
401 from maasserver.models import (
402 Config,
403 Node,
404 NodeGroup,
405 )
406+from maasserver.models.eventtype import LOGGING_LEVELS
407 from maasserver.models.node import RELEASABLE_STATUSES
408 from maasserver.models.user import (
409 create_auth_token,
410@@ -77,7 +90,7 @@
411 scenarios = [
412 ('user', dict(userfactory=factory.make_User)),
413 ('admin', dict(userfactory=factory.make_admin)),
414- ]
415+ ]
416
417 def test_GET_list_returns_fqdn_with_domain_name_from_cluster(self):
418 # If DNS management is enabled, the domain part of a hostname
419@@ -155,6 +168,807 @@
420 return [node.get('system_id') for node in parsed_result]
421
422
423+def extract_system_ids_from_nodes(nodes):
424+ return [node.system_id for node in nodes]
425+
426+
427+def extract_event_desc(parsed_result):
428+ """List the system_ids of the nodes in `parsed_result`'s events."""
429+ return [event.get('description') for event in parsed_result['events']]
430+
431+
432+def extract_event_ids(parsed_result):
433+ """List the system_ids of the nodes in `parsed_result`'s events."""
434+ return [event.get('id') for event in parsed_result['events']]
435+
436+
437+class RequestFixture:
438+ def __init__(self, dict, fields):
439+ self.user = factory.make_User()
440+ self.GET = get_overridden_query_dict(dict, QueryDict(''), fields)
441+
442+
443+class TestFilteredNodesListFromRequest(APITestCase):
444+
445+ def test_node_list_with_id_returns_matching_nodes(self):
446+ # The "list" operation takes optional "id" parameters. Only
447+ # nodes with matching ids will be returned.
448+ ids = [factory.make_Node().system_id for _ in range(3)]
449+ matching_id = ids[0]
450+ query = RequestFixture({'id': [matching_id]}, 'id')
451+ response = nodes_module.filtered_nodes_list_from_request(query)
452+
453+ self.assertItemsEqual(
454+ [matching_id],
455+ extract_system_ids_from_nodes(response))
456+
457+ def test_node_list_with_nonexistent_id_returns_empty_list(self):
458+ # Trying to list a nonexistent node id returns a list containing
459+ # no nodes -- even if other (non-matching) nodes exist.
460+ existing_id = factory.make_Node().system_id
461+ nonexistent_id = existing_id + factory.make_string()
462+ query = RequestFixture({'id': [nonexistent_id]}, 'id')
463+ response = nodes_module.filtered_nodes_list_from_request(query)
464+
465+ self.assertItemsEqual(
466+ [],
467+ extract_system_ids_from_nodes(response))
468+
469+ def test_node_list_with_ids_orders_by_id(self):
470+ # Even when ids are passed to "list," nodes are returned in id
471+ # order, not necessarily in the order of the id arguments.
472+ ids = [factory.make_Node().system_id for _ in range(3)]
473+ random.shuffle(ids)
474+
475+ query = RequestFixture({'id': list(ids)}, 'id')
476+ response = nodes_module.filtered_nodes_list_from_request(query)
477+
478+ self.assertSequenceEqual(
479+ sorted(ids),
480+ extract_system_ids_from_nodes(response))
481+
482+ def test_node_list_with_some_matching_ids_returns_matching_nodes(self):
483+ # If some nodes match the requested ids and some don't, only the
484+ # matching ones are returned.
485+ existing_id = factory.make_Node().system_id
486+ nonexistent_id = existing_id + factory.make_string()
487+
488+ query = RequestFixture({'id': [existing_id, nonexistent_id]}, 'id')
489+ response = nodes_module.filtered_nodes_list_from_request(query)
490+
491+ self.assertItemsEqual(
492+ [existing_id],
493+ extract_system_ids_from_nodes(response))
494+
495+ def test_node_list_with_hostname_returns_matching_nodes(self):
496+ # The list operation takes optional "hostname" parameters. Only nodes
497+ # with matching hostnames will be returned.
498+ nodes = [factory.make_Node() for _ in range(3)]
499+ matching_hostname = nodes[0].hostname
500+ matching_system_id = nodes[0].system_id
501+
502+ query = RequestFixture({'hostname': [matching_hostname]}, 'hostname')
503+ response = nodes_module.filtered_nodes_list_from_request(query)
504+
505+ self.assertItemsEqual(
506+ [matching_system_id],
507+ extract_system_ids_from_nodes(response))
508+
509+ def test_node_list_with_macs_returns_matching_nodes(self):
510+ # The "list" operation takes optional "mac_address" parameters. Only
511+ # nodes with matching MAC addresses will be returned.
512+ macs = [factory.make_MACAddress_with_Node() for _ in range(3)]
513+ matching_mac = unicode(macs[0].mac_address)
514+ matching_system_id = macs[0].node.system_id
515+
516+ query = RequestFixture({'mac_address': [matching_mac]}, 'mac_address')
517+ response = nodes_module.filtered_nodes_list_from_request(query)
518+
519+ self.assertItemsEqual(
520+ [matching_system_id],
521+ extract_system_ids_from_nodes(response))
522+
523+ def test_node_list_with_invalid_macs_returns_sensible_error(self):
524+ # If specifying an invalid MAC, make sure the error that's
525+ # returned is not a crazy stack trace, but something nice to
526+ # humans.
527+ bad_mac1 = '00:E0:81:DD:D1:ZZ' # ZZ is bad.
528+ bad_mac2 = '00:E0:81:DD:D1:XX' # XX is bad.
529+ ok_mac = unicode(factory.make_MACAddress_with_Node())
530+ mac_list = [bad_mac1, bad_mac2, ok_mac]
531+
532+ query = RequestFixture({'mac_address': mac_list}, 'mac_address')
533+ expected_msg = ("[u'Invalid MAC address(es): 00:E0:81:DD:D1:ZZ, "
534+ "00:E0:81:DD:D1:XX']")
535+
536+ ex = self.assertRaises(MAASAPIValidationError,
537+ nodes_module.filtered_nodes_list_from_request,
538+ query)
539+ self.assertEqual(expected_msg, unicode(ex))
540+
541+ def test_node_list_with_agent_name_filters_by_agent_name(self):
542+ non_listed_node = factory.make_Node(
543+ agent_name=factory.make_name('agent_name'))
544+ ignore_unused(non_listed_node)
545+ agent_name = factory.make_name('agent-name')
546+ node = factory.make_Node(agent_name=agent_name)
547+
548+ query = RequestFixture({'agent_name': agent_name}, 'agent_name')
549+ response = nodes_module.filtered_nodes_list_from_request(query)
550+
551+ self.assertSequenceEqual(
552+ [node.system_id],
553+ extract_system_ids_from_nodes(response))
554+
555+ def test_node_list_with_agent_name_filters_with_empty_string(self):
556+ factory.make_Node(agent_name=factory.make_name('agent-name'))
557+ node = factory.make_Node(agent_name='')
558+
559+ query = RequestFixture({'agent_name': ''}, 'agent_name')
560+ response = nodes_module.filtered_nodes_list_from_request(query)
561+
562+ self.assertSequenceEqual(
563+ [node.system_id],
564+ extract_system_ids_from_nodes(response))
565+
566+ def test_node_list_without_agent_name_does_not_filter(self):
567+ nodes = [
568+ factory.make_Node(agent_name=factory.make_name('agent-name'))
569+ for _ in range(3)]
570+
571+ query = RequestFixture({}, '')
572+ response = nodes_module.filtered_nodes_list_from_request(query)
573+
574+ self.assertSequenceEqual(
575+ [node.system_id for node in nodes],
576+ extract_system_ids_from_nodes(response))
577+
578+ def test_node_list_doesnt_list_devices(self):
579+ nodes = [
580+ factory.make_Node(agent_name=factory.make_name('agent-name'))
581+ for _ in range(3)]
582+ # Create devices.
583+ nodes = [
584+ factory.make_Node(installable=False)
585+ for _ in range(3)]
586+
587+ query = RequestFixture({}, '')
588+ response = nodes_module.filtered_nodes_list_from_request(query)
589+
590+ system_ids = extract_system_ids_from_nodes(response)
591+ self.assertEqual(
592+ [],
593+ [node.system_id for node in nodes if node.system_id in system_ids],
594+ "Node listing contains devices.")
595+
596+ def test_node_list_with_zone_filters_by_zone(self):
597+ non_listed_node = factory.make_Node(
598+ zone=factory.make_Zone(name='twilight'))
599+ ignore_unused(non_listed_node)
600+ zone = factory.make_Zone()
601+ node = factory.make_Node(zone=zone)
602+
603+ query = RequestFixture({'zone': zone.name}, 'zone')
604+ response = nodes_module.filtered_nodes_list_from_request(query)
605+
606+ self.assertSequenceEqual(
607+ [node.system_id], extract_system_ids_from_nodes(response))
608+
609+ def test_node_list_without_zone_does_not_filter(self):
610+ nodes = [factory.make_Node(zone=factory.make_Zone())
611+ for _ in range(3)]
612+
613+ query = RequestFixture({}, '')
614+ response = nodes_module.filtered_nodes_list_from_request(query)
615+
616+ self.assertSequenceEqual(
617+ [node.system_id for node in nodes],
618+ extract_system_ids_from_nodes(response))
619+
620+
621+def make_events_with_log_levels(log_levels_dict, events_per_level=2):
622+ return [factory.make_Event(
623+ type=factory.make_EventType(
624+ level=level[1])).id
625+ for level in log_levels_dict
626+ for _ in range(events_per_level)
627+ ]
628+
629+
630+class TestEventsAPI(APITestCase):
631+ """Tests for /api/1.0/events/."""
632+ log_levels = (('CRITICAL', logging.CRITICAL),
633+ ('ERROR', logging.ERROR),
634+ ('WARNING', logging.WARNING),
635+ ('INFO', logging.INFO),
636+ ('DEBUG', logging.DEBUG),
637+ )
638+
639+ def test_handler_path(self):
640+ self.assertEqual(
641+ '/api/1.0/events/', reverse('events_handler'))
642+
643+ def create_nodes_in_group_with_events(
644+ self, nodegroup, number_nodes=2, number_events=2):
645+ for _ in range(number_nodes):
646+ node = factory.make_Node(nodegroup=nodegroup, mac=True)
647+ for _ in range(number_events):
648+ factory.make_Event(node=node)
649+
650+ def test_GET_query_without_events_returns_empty_list(self):
651+ # If there are no nodes to list, the "list" op still works but
652+ # returns an empty list.
653+ response = self.client.get(
654+ reverse('events_handler'), {
655+ 'op': 'query'})
656+
657+ self.assertItemsEqual(
658+ {'count': 0,
659+ 'events': [],
660+ 'prev_uri': '',
661+ 'next_uri': ''},
662+ json.loads(response.content))
663+
664+ def test_GET_query_orders_by_reverse_id(self):
665+ # Events are returned in reverse id order (newest first).
666+ node = factory.make_Node()
667+ event_ids = [factory.make_Event(node=node).id for _ in range(3)]
668+ response = self.client.get(reverse('events_handler'), {
669+ 'op': 'query',
670+ 'level': 'DEBUG'}
671+ )
672+ parsed_result = json.loads(response.content)
673+ self.assertSequenceEqual(
674+ list(reversed(event_ids)), extract_event_ids(parsed_result))
675+ self.assertEqual(parsed_result['count'], len(event_ids))
676+
677+ def test_GET_query_with_id_returns_matching_nodes(self):
678+ # The "list" operation takes optional "id" parameters. Only
679+ # events from nodes with matching ids will be returned.
680+ nodes = [factory.make_Node() for _ in range(3)]
681+ ids = [node.system_id for node in nodes]
682+ events = [factory.make_Event(node=node) for node in nodes]
683+ matching_id = ids[0]
684+ matching_events = events[0].id
685+ response = self.client.get(reverse('events_handler'), {
686+ 'op': 'query',
687+ 'id': [matching_id],
688+ 'level': 'DEBUG'
689+ })
690+ parsed_result = json.loads(response.content)
691+ self.assertItemsEqual(
692+ [matching_events], extract_event_ids(parsed_result))
693+ self.assertEqual(parsed_result['count'], len([matching_events]))
694+
695+ def test_GET_query_with_nonexistent_id_returns_empty_list(self):
696+ # Trying to list events for a nonexistent node id returns a list
697+ # containing no nodes -- even if other (non-matching) nodes exist.
698+ node = factory.make_Node()
699+ [factory.make_Event(node=node) for _ in range(3)]
700+ existing_id = node.system_id
701+ nonexistent_id = existing_id + factory.make_string()
702+ response = self.client.get(reverse('events_handler'), {
703+ 'op': 'query',
704+ 'id': [nonexistent_id],
705+ })
706+ self.assertItemsEqual({'count': 0,
707+ 'events': [],
708+ 'prev_uri': '',
709+ 'next_uri': ''},
710+ json.loads(response.content))
711+
712+ def test_GET_query_with_ids_orders_by_id_reverse(self):
713+ # Even when node ids are passed to "list," events for nodes are
714+ # returned in event id order, not necessarily in the order of the
715+ # node id arguments.
716+ nodes = [factory.make_Node() for _ in range(3)]
717+ ids = [node.system_id for node in nodes]
718+ events = [factory.make_Event(node=node) for node in nodes]
719+ event_ids = [event.id for event in reversed(events)]
720+ response = self.client.get(reverse('events_handler'), {
721+ 'op': 'query',
722+ 'id': list(reversed(ids)),
723+ 'level': 'DEBUG'
724+ })
725+ parsed_result = json.loads(response.content)
726+ self.assertSequenceEqual(
727+ event_ids, extract_event_ids(parsed_result))
728+ self.assertEqual(parsed_result['count'], len(event_ids))
729+ self.assertNumQueries(1)
730+
731+ def test_GET_query_with_some_matching_ids_returns_matching_nodes(self):
732+ # If some nodes match the requested ids and some don't, only the
733+ # events matching nodes specified are returned.
734+ existing_node = factory.make_Node()
735+ existing_id = existing_node.system_id
736+ existing_event_ids = [
737+ factory.make_Event(
738+ node=existing_node).id for _ in range(3)]
739+ nonexistent_id = existing_id + factory.make_string()
740+ # Generate some non-matching events as well
741+ [factory.make_Event().id for counter in range(3)]
742+ response = self.client.get(reverse('events_handler'), {
743+ 'op': 'query',
744+ 'id': [existing_id, nonexistent_id],
745+ 'level': 'DEBUG'
746+ })
747+ parsed_result = json.loads(response.content)
748+ self.assertItemsEqual(
749+ reversed(existing_event_ids), extract_event_ids(parsed_result))
750+ self.assertEqual(parsed_result['count'], len(existing_event_ids))
751+
752+ def test_GET_query_with_hostname_returns_matching_nodes(self):
753+ # The list operation takes optional "hostname" parameters. Only events
754+ # for nodes with matching hostnames will be returned.
755+ nodes = [factory.make_Node() for _ in range(3)]
756+ events = [factory.make_Event(node=node) for node in nodes]
757+
758+ matching_hostname = nodes[0].hostname
759+ matching_event_id = events[0].id
760+ response = self.client.get(reverse('events_handler'), {
761+ 'op': 'query',
762+ 'hostname': [matching_hostname],
763+ 'level': 'DEBUG'
764+ })
765+ parsed_result = json.loads(response.content)
766+ self.assertItemsEqual(
767+ [matching_event_id], extract_event_ids(parsed_result))
768+ self.assertEqual(parsed_result['count'], len([matching_event_id]))
769+
770+ def test_GET_query_with_macs_returns_matching_nodes(self):
771+ # The "list" operation takes optional "mac_address" parameters. Only
772+ # events for nodes with matching MAC addresses will be returned.
773+ macs = [factory.make_MACAddress_with_Node() for _ in range(3)]
774+ events = [factory.make_Event(node=mac.node) for mac in macs]
775+ matching_mac = macs[0].mac_address
776+ matching_event_id = events[0].id
777+ response = self.client.get(reverse('events_handler'), {
778+ 'op': 'query',
779+ 'mac_address': [matching_mac],
780+ 'level': 'DEBUG'
781+ })
782+ parsed_result = json.loads(response.content)
783+ self.assertItemsEqual(
784+ [matching_event_id], extract_event_ids(parsed_result))
785+ self.assertEqual(parsed_result['count'], len([matching_event_id]))
786+
787+ def test_GET_query_with_invalid_macs_returns_sensible_error(self):
788+ # If specifying an invalid MAC, make sure the error that's
789+ # returned is not a crazy stack trace, but something nice to
790+ # humans.
791+ bad_mac1 = '00:E0:81:DD:D1:ZZ' # ZZ is bad.
792+ bad_mac2 = '00:E0:81:DD:D1:XX' # XX is bad.
793+ ok_mac = factory.make_MACAddress_with_Node()
794+ [factory.make_Event(node=ok_mac.node) for _ in range(3)]
795+ response = self.client.get(reverse('events_handler'), {
796+ 'op': 'query',
797+ 'mac_address': [bad_mac1, bad_mac2, ok_mac],
798+ 'level': 'DEBUG'
799+ })
800+ observed = response.status_code, response.content
801+ expected = (
802+ Equals(httplib.BAD_REQUEST),
803+ Contains(
804+ "Invalid MAC address(es): 00:E0:81:DD:D1:ZZ, "
805+ "00:E0:81:DD:D1:XX"),
806+ )
807+ self.assertThat(observed, MatchesListwise(expected))
808+
809+ def test_GET_query_with_agent_name_filters_by_agent_name(self):
810+ non_listed_node = factory.make_Node(
811+ agent_name=factory.make_name('agent_name'))
812+ ignore_unused(non_listed_node)
813+ agent_name = factory.make_name('agent-name')
814+ node = factory.make_Node(agent_name=agent_name)
815+
816+ [factory.make_Event(node=non_listed_node) for _ in range(3)]
817+ matching_event_ids = [
818+ factory.make_Event(
819+ node=node).id for _ in range(3)]
820+
821+ response = self.client.get(reverse('events_handler'), {
822+ 'op': 'query',
823+ 'agent_name': agent_name,
824+ 'level': 'DEBUG',
825+ })
826+
827+ self.assertEqual(httplib.OK, response.status_code)
828+ parsed_result = json.loads(response.content)
829+
830+ matching_event_ids = list(reversed(matching_event_ids))
831+ self.assertSequenceEqual(
832+ matching_event_ids, extract_event_ids(parsed_result))
833+ self.assertEqual(parsed_result['count'], len(matching_event_ids))
834+
835+ def test_GET_query_with_agent_name_filters_with_empty_string(self):
836+ non_listed_node = factory.make_Node(
837+ agent_name=factory.make_name('agent-name'))
838+ node = factory.make_Node(agent_name='')
839+
840+ [factory.make_Event(node=non_listed_node) for _ in range(3)]
841+ matching_event_ids = [
842+ factory.make_Event(
843+ node=node).id for _ in range(3)]
844+
845+ response = self.client.get(reverse('events_handler'), {
846+ 'op': 'query',
847+ 'agent_name': '',
848+ 'level': 'DEBUG'
849+ })
850+ self.assertEqual(httplib.OK, response.status_code)
851+ parsed_result = json.loads(response.content)
852+ matching_event_ids = list(reversed(matching_event_ids))
853+ self.assertSequenceEqual(
854+ matching_event_ids, extract_event_ids(parsed_result))
855+ self.assertEqual(parsed_result['count'], len(matching_event_ids))
856+
857+ def test_GET_query_without_agent_name_does_not_filter(self):
858+ nodes = [
859+ factory.make_Node(agent_name=factory.make_name('agent-name'))
860+ for _ in range(3)]
861+ matching_event_ids = [
862+ factory.make_Event(
863+ node=node).id for node in nodes]
864+ response = self.client.get(reverse('events_handler'), {
865+ 'op': 'query',
866+ 'level': 'DEBUG'
867+ }
868+ )
869+ self.assertEqual(httplib.OK, response.status_code)
870+ parsed_result = json.loads(response.content)
871+ matching_event_ids = list(reversed(matching_event_ids))
872+ self.assertSequenceEqual(
873+ matching_event_ids,
874+ extract_event_ids(parsed_result))
875+ self.assertEqual(parsed_result['count'], len(matching_event_ids))
876+
877+ def test_GET_query_doesnt_list_devices(self):
878+ nodes = [
879+ factory.make_Node(agent_name=factory.make_name('agent-name'))
880+ for _ in range(3)]
881+ [factory.make_Event(node=node) for node in nodes]
882+
883+ # Create devices.
884+ device_nodes = [
885+ factory.make_Node(installable=False)
886+ for _ in range(3)]
887+ [factory.make_Event(node=node) for node in device_nodes]
888+
889+ response = self.client.get(reverse('events_handler'), {
890+ 'op': 'query',
891+ 'level': 'DEBUG',
892+ }
893+ )
894+ self.assertEqual(httplib.OK, response.status_code)
895+ parsed_result = json.loads(response.content)
896+ system_ids = extract_event_ids(parsed_result)
897+ self.assertEqual(
898+ [],
899+ [node.system_id for node in
900+ device_nodes if node.system_id in system_ids],
901+ "Node listing contains devices.")
902+ self.assertEqual(parsed_result['count'], len(nodes))
903+
904+ def test_GET_query_with_zone_filters_by_zone(self):
905+ non_listed_node = factory.make_Node(
906+ zone=factory.make_Zone(name='twilight'))
907+ [factory.make_Event(node=non_listed_node) for _ in range(3)]
908+ zone = factory.make_Zone()
909+ node = factory.make_Node(zone=zone)
910+ matching_event_ids = [
911+ factory.make_Event(
912+ node=node).id for _ in range(3)]
913+ response = self.client.get(reverse('events_handler'), {
914+ 'op': 'query',
915+ 'zone': zone.name,
916+ 'level': 'DEBUG'
917+ })
918+ self.assertEqual(httplib.OK, response.status_code)
919+ parsed_result = json.loads(response.content)
920+ matching_event_ids = list(reversed(matching_event_ids))
921+ self.assertSequenceEqual(
922+ matching_event_ids, extract_event_ids(parsed_result))
923+ self.assertEqual(parsed_result['count'], len(matching_event_ids))
924+
925+ def test_GET_query_with_limit_limits_with_most_recent_events(self):
926+ test_limit = 5
927+ # Events are returned in id order.
928+ event_ids = [factory.make_Event().id for _
929+ in range(test_limit + 1)]
930+ response = self.client.get(reverse('events_handler'), {
931+ 'op': 'query',
932+ 'limit': unicode(test_limit),
933+ 'level': 'DEBUG'
934+ })
935+
936+ parsed_result = json.loads(response.content)
937+
938+ self.assertSequenceEqual(
939+ list(reversed(event_ids))[:test_limit],
940+ extract_event_ids(parsed_result))
941+ self.assertEqual(parsed_result['count'], test_limit)
942+
943+ def test_GET_query_with_limit_over_hard_limit_raises_error_with_msg(self):
944+ artificial_limit = 5
945+ test_limit = artificial_limit + 1
946+ self.patch(nodes_module, 'MAX_EVENT_LOG_COUNT', artificial_limit)
947+
948+ [factory.make_Event().id for _ in range(test_limit)]
949+ response = self.client.get(reverse('events_handler'), {
950+ 'op': 'query',
951+ 'limit': unicode(test_limit),
952+ })
953+
954+ expected_msg = ("Requested number of events %d is greater than"
955+ " limit: %d" % (test_limit,
956+ nodes_module.MAX_EVENT_LOG_COUNT))
957+
958+ observed = response.status_code, response.content
959+ expected = (
960+ Equals(httplib.BAD_REQUEST),
961+ Contains(expected_msg),
962+ )
963+ self.assertThat(observed, MatchesListwise(expected))
964+
965+ def test_GET_query_with_without_limit_limits_to_default_newest(self):
966+ artificial_limit = 5
967+ test_limit = artificial_limit + 1
968+ self.patch(
969+ nodes_module,
970+ 'DEFAULT_EVENT_LOG_LIMIT',
971+ artificial_limit)
972+
973+ # Nodes are returned in id order.
974+ event_ids = [factory.make_Event().id for _ in range(test_limit)]
975+ response = self.client.get(reverse('events_handler'), {
976+ 'op': 'query',
977+ 'level': 'DEBUG'
978+ })
979+
980+ parsed_result = json.loads(response.content)
981+
982+ self.assertSequenceEqual(
983+ list(reversed(event_ids))[:artificial_limit],
984+ extract_event_ids(parsed_result))
985+ self.assertEqual(parsed_result['count'], artificial_limit)
986+
987+ def test_GET_query_with_start_event_id_with_limit(self):
988+ test_limit = 5
989+ (_, _, test_event_3, test_event_4,
990+ test_event_5) = (factory.make_Event(), factory.make_Event(),
991+ factory.make_Event(), factory.make_Event(),
992+ factory.make_Event())
993+ ignore_unused(_)
994+
995+ response = self.client.get(reverse('events_handler'), {
996+ 'op': 'query',
997+ 'after': unicode(test_event_3.id),
998+ 'limit': unicode(test_limit),
999+ 'level': 'DEBUG'
1000+ })
1001+ parsed_result = json.loads(response.content)
1002+ expected_result = list(
1003+ [test_event_5.id, test_event_4.id, test_event_3.id])
1004+ observed_result = extract_event_ids(parsed_result)
1005+
1006+ self.assertSequenceEqual(
1007+ expected_result, observed_result)
1008+ self.assertEqual(parsed_result['count'], len(expected_result))
1009+
1010+ def test_GET_query_with_start_event_id_without_limit(self):
1011+ (_, _, test_event_3, test_event_4,
1012+ test_event_5) = (factory.make_Event(), factory.make_Event(),
1013+ factory.make_Event(), factory.make_Event(),
1014+ factory.make_Event())
1015+ ignore_unused(_)
1016+
1017+ response = self.client.get(reverse('events_handler'), {
1018+ 'op': 'query',
1019+ 'after': unicode(test_event_3.id),
1020+ 'level': 'DEBUG'
1021+ })
1022+ parsed_result = json.loads(response.content)
1023+ expected_result = list(
1024+ [test_event_5.id, test_event_4.id, test_event_3.id])
1025+ observed_result = extract_event_ids(parsed_result)
1026+
1027+ self.assertSequenceEqual(
1028+ expected_result, observed_result)
1029+ self.assertEqual(parsed_result['count'], len(expected_result))
1030+
1031+ def test_GET_query_with_invalid_log_level_raises_error_with_msg(self):
1032+ [factory.make_Event().id for _ in range(3)]
1033+ invalid_level = factory.make_name('invalid_log_level')
1034+ response = self.client.get(reverse('events_handler'), {
1035+ 'op': 'query',
1036+ 'level': invalid_level,
1037+ })
1038+
1039+ expected_msg = ("Unknown log level: %s" % invalid_level)
1040+
1041+ observed = response.status_code, response.content
1042+ expected = (
1043+ Equals(httplib.BAD_REQUEST),
1044+ Contains(expected_msg),
1045+ )
1046+ self.assertThat(observed, MatchesListwise(expected))
1047+
1048+ def test_GET_query_with_log_level_returns_that_level_and_greater(self):
1049+ events_per_level = 2
1050+ event_ids = make_events_with_log_levels(
1051+ self.log_levels,
1052+ events_per_level)
1053+
1054+ for idx, level in enumerate(self.log_levels):
1055+ response = self.client.get(reverse('events_handler'), {
1056+ 'op': 'query',
1057+ 'level': level[0],
1058+ })
1059+
1060+ parsed_result = json.loads(response.content)
1061+
1062+ expected_result = list(
1063+ reversed(event_ids[:(idx + 1) * events_per_level]))
1064+ observed_result = extract_event_ids(parsed_result)
1065+
1066+ self.assertSequenceEqual(
1067+ expected_result, observed_result)
1068+ self.assertEqual(parsed_result['count'], len(expected_result))
1069+
1070+ def test_GET_query_with_default_log_level_is_info(self):
1071+ make_events_with_log_levels(self.log_levels)
1072+
1073+ info_response = self.client.get(reverse('events_handler'), {
1074+ 'op': 'query',
1075+ 'level': 'INFO',
1076+ })
1077+
1078+ default_response = self.client.get(reverse('events_handler'), {
1079+ 'op': 'query',
1080+ })
1081+
1082+ expected_result = json.loads(info_response.content)
1083+ observed_result = json.loads(default_response.content)
1084+ expected_result_ids = extract_event_ids(expected_result)
1085+
1086+ self.assertSequenceEqual(
1087+ expected_result['events'], observed_result['events'])
1088+ self.assertEqual(observed_result['count'],
1089+ len(expected_result_ids))
1090+ # Don't compare the URIs. They will be different.
1091+
1092+ def test_GET_query_prev_next_uris_preserves_query_params(self):
1093+ test_events = 6
1094+ test_params = list(nodes_module.EventsHandler.all_params)
1095+ if 'op' in test_params:
1096+ test_params.remove('op')
1097+ expected_uri_path = reverse('events_handler')
1098+
1099+ # Try all cardinalities of combinations of query parameters
1100+ for r in range(len(test_params) + 1):
1101+ for params in combinations(test_params, r):
1102+ # Generate test values for all params
1103+ test_values = \
1104+ {'after': unicode(randint(1, test_events)),
1105+ 'agent_name': factory.make_string(),
1106+ 'id': factory.make_string(),
1107+ 'level': random.choice(LOGGING_LEVELS.values()),
1108+ 'limit': unicode(randint(1, test_events)),
1109+ 'mac_address':
1110+ unicode(factory.make_MACAddress_with_Node()),
1111+ 'zone': factory.make_string()}
1112+
1113+ # Build a query dictionary for the given combination of params
1114+ expected_params = {}
1115+ for param_name in params:
1116+ expected_params[param_name] = test_values[param_name]
1117+
1118+ # Ensure that op is always included
1119+ expected_params['op'] = 'query'
1120+
1121+ response = self.client.get(
1122+ reverse('events_handler'), expected_params)
1123+
1124+ self.assertEqual(httplib.OK, response.status_code)
1125+
1126+ # Parse the returned JSON and check URI path
1127+ parsed_result = json.loads(response.content)
1128+
1129+ next_uri = urlparse(parsed_result['next_uri'])
1130+ prev_uri = urlparse(parsed_result['prev_uri'])
1131+
1132+ self.assertEqual(
1133+ (expected_uri_path, '', '', '', ''),
1134+ (next_uri.path,
1135+ next_uri.scheme,
1136+ next_uri.netloc,
1137+ next_uri.params,
1138+ next_uri.fragment))
1139+
1140+ self.assertEqual(
1141+ (expected_uri_path, '', '', '', ''),
1142+ (prev_uri.path,
1143+ prev_uri.scheme,
1144+ prev_uri.netloc,
1145+ prev_uri.params,
1146+ prev_uri.fragment))
1147+ self.assertEqual(expected_uri_path, prev_uri.path)
1148+
1149+ # Parse URI query strings
1150+ next_uri_params = dict(parse_qsl(
1151+ next_uri.query,
1152+ keep_blank_values=True))
1153+
1154+ prev_uri_params = dict(parse_qsl(
1155+ prev_uri.query,
1156+ keep_blank_values=True))
1157+
1158+ # Calculate the expected values for limit
1159+ # and start event id
1160+ limit = nodes_module.DEFAULT_EVENT_LOG_LIMIT \
1161+ if not 'limit' in expected_params \
1162+ else int(expected_params['limit'])
1163+
1164+ start_id = 0 if not 'after' in expected_params \
1165+ else int(expected_params['after'])
1166+
1167+ expected_params['after'] = unicode(start_id + limit)
1168+ self.assertDictEqual(expected_params, next_uri_params)
1169+
1170+ expected_params['after'] = \
1171+ unicode(max(start_id - limit, 0))
1172+ self.assertDictEqual(expected_params, prev_uri_params)
1173+
1174+ def test_query_num_queries_is_independent_of_num_nodes_and_events(self):
1175+ # 1 query for select event +
1176+ # 1 query to prefetch eventtype +
1177+ # 1 query to prefetch node details
1178+ expected_queries = 3
1179+ events_per_node = 5
1180+ num_nodes_per_group = 5
1181+ events_per_group = num_nodes_per_group * events_per_node
1182+
1183+ nodegroup_1 = factory.make_NodeGroup()
1184+ nodegroup_2 = factory.make_NodeGroup()
1185+
1186+ self.create_nodes_in_group_with_events(
1187+ nodegroup_1, num_nodes_per_group, events_per_node)
1188+
1189+ handler = nodes_module.EventsHandler()
1190+
1191+ query_1_count, query_1_result = count_queries(
1192+ handler.query, RequestFixture(
1193+ {
1194+ 'op': 'query',
1195+ 'level': 'DEBUG',
1196+ }, ['op', 'level']
1197+ ))
1198+
1199+ self.create_nodes_in_group_with_events(
1200+ nodegroup_2, num_nodes_per_group, events_per_node)
1201+
1202+ query_2_count, query_2_result = count_queries(
1203+ handler.query, RequestFixture(
1204+ {
1205+ 'op': 'query',
1206+ 'level': 'DEBUG',
1207+ }, ['op', 'level']
1208+ ))
1209+
1210+ # This check is to notify the developer that a change was made that
1211+ # affects the number of queries performed when doing an event listing.
1212+ # If this happens, consider your prefetching and adjust accordingly.
1213+ self.assertEquals(events_per_group, int(query_1_result['count']))
1214+ self.assertEquals(
1215+ expected_queries, query_1_count,
1216+ "Number of queries has changed, make sure this is expected.")
1217+
1218+ self.assertEquals(events_per_group * 2, int(query_2_result['count']))
1219+ self.assertEquals(
1220+ expected_queries, query_2_count,
1221+ "Number of queries is not independent to the number of nodes.")
1222+
1223+
1224 class TestNodesAPI(APITestCase):
1225 """Tests for /api/1.0/nodes/."""
1226
1227@@ -169,7 +983,7 @@
1228 macs = {
1229 factory.make_mac_address()
1230 for _ in range(random.randint(1, 2))
1231- }
1232+ }
1233 response = self.client.post(
1234 reverse('nodes_handler'),
1235 {
1236@@ -408,14 +1222,14 @@
1237 response = self.client.get(reverse('nodes_handler'), {
1238 'op': 'list',
1239 'mac_address': [bad_mac1, bad_mac2, ok_mac],
1240- })
1241+ })
1242 observed = response.status_code, response.content
1243 expected = (
1244 Equals(httplib.BAD_REQUEST),
1245 Contains(
1246 "Invalid MAC address(es): 00:E0:81:DD:D1:ZZ, "
1247 "00:E0:81:DD:D1:XX"),
1248- )
1249+ )
1250 self.assertThat(observed, MatchesListwise(expected))
1251
1252 def test_GET_list_with_agent_name_filters_by_agent_name(self):
1253@@ -427,7 +1241,7 @@
1254 response = self.client.get(reverse('nodes_handler'), {
1255 'op': 'list',
1256 'agent_name': agent_name,
1257- })
1258+ })
1259 self.assertEqual(httplib.OK, response.status_code)
1260 parsed_result = json.loads(response.content)
1261 self.assertSequenceEqual(
1262@@ -439,7 +1253,7 @@
1263 response = self.client.get(reverse('nodes_handler'), {
1264 'op': 'list',
1265 'agent_name': '',
1266- })
1267+ })
1268 self.assertEqual(httplib.OK, response.status_code)
1269 parsed_result = json.loads(response.content)
1270 self.assertSequenceEqual(
1271@@ -482,7 +1296,7 @@
1272 response = self.client.get(reverse('nodes_handler'), {
1273 'op': 'list',
1274 'zone': zone.name,
1275- })
1276+ })
1277 self.assertEqual(httplib.OK, response.status_code)
1278 parsed_result = json.loads(response.content)
1279 self.assertSequenceEqual(
1280@@ -619,7 +1433,7 @@
1281 reverse('nodes_handler'), {
1282 'op': 'acquire',
1283 'name': hostname,
1284- })
1285+ })
1286 self.assertEqual(httplib.CONFLICT, response.status_code)
1287 self.assertEqual(
1288 "No available node matches constraints: name=%s" % hostname,
1289@@ -962,7 +1776,7 @@
1290 node=factory.make_Node(status=NODE_STATUS.READY),
1291 networks=[network])
1292 for network in networks
1293- ]
1294+ ]
1295 # We'll make it so that only the node and network at this index will
1296 # match the request.
1297 pick = 2
1298@@ -999,7 +1813,7 @@
1299 nodes = [
1300 factory.make_Node(status=NODE_STATUS.READY, zone=not_in_zone)
1301 for _ in range(5)
1302- ]
1303+ ]
1304 # Pick a node in the middle to avoid false negatives if acquire()
1305 # always tries the oldest, or the newest, node first.
1306 eligible_node = nodes[2]
1307@@ -1057,7 +1871,7 @@
1308 NODE_STATUS.NEW,
1309 NODE_STATUS.COMMISSIONING,
1310 NODE_STATUS.READY,
1311- ])
1312+ ])
1313 unacceptable_states = (
1314 set(map_enum(NODE_STATUS).values()) - acceptable_states)
1315 nodes = {
1316@@ -1068,7 +1882,7 @@
1317 reverse('nodes_handler'), {
1318 'op': 'accept',
1319 'nodes': [node.system_id],
1320- })
1321+ })
1322 for status, node in nodes.items()}
1323 # All of these attempts are rejected with Conflict errors.
1324 self.assertEqual(
1325@@ -1121,7 +1935,7 @@
1326 response = self.client.post(reverse('nodes_handler'), {
1327 'op': 'accept',
1328 'nodes': node_ids,
1329- })
1330+ })
1331 self.assertEqual(httplib.OK, response.status_code)
1332 self.assertEqual(
1333 [target_state] * len(nodes),
1334@@ -1133,13 +1947,13 @@
1335 acceptable_nodes = [
1336 factory.make_Node(status=NODE_STATUS.NEW)
1337 for counter in range(2)
1338- ]
1339+ ]
1340 accepted_node = factory.make_Node(status=NODE_STATUS.READY)
1341 nodes = acceptable_nodes + [accepted_node]
1342 response = self.client.post(reverse('nodes_handler'), {
1343 'op': 'accept',
1344 'nodes': [node.system_id for node in nodes],
1345- })
1346+ })
1347 self.assertEqual(httplib.OK, response.status_code)
1348 accepted_ids = [
1349 node['system_id'] for node in json.loads(response.content)]
1350@@ -1157,12 +1971,12 @@
1351 node_ids = {
1352 factory.make_Node(installable=False).system_id
1353 for _ in xrange(3)
1354- }
1355+ }
1356 response = self.client.post(
1357 reverse('nodes_handler'), {
1358 'op': 'release',
1359 'nodes': node_ids
1360- })
1361+ })
1362 self.assertEqual(httplib.BAD_REQUEST, response.status_code)
1363
1364 def test_POST_release_rejects_request_from_unauthorized_user(self):
1365@@ -1172,7 +1986,7 @@
1366 reverse('nodes_handler'), {
1367 'op': 'release',
1368 'nodes': [node.system_id],
1369- })
1370+ })
1371 self.assertEqual(httplib.FORBIDDEN, response.status_code)
1372 self.assertEqual(NODE_STATUS.ALLOCATED, reload_object(node).status)
1373
1374@@ -1184,7 +1998,7 @@
1375 reverse('nodes_handler'), {
1376 'op': 'release',
1377 'nodes': node_ids
1378- })
1379+ })
1380 # Awkward parsing, but the order may vary and it's not JSON
1381 s = response.content
1382 returned_ids = s[s.find(':') + 2:s.rfind('.')].split(', ')
1383@@ -1199,7 +2013,7 @@
1384 status=NODE_STATUS.ALLOCATED,
1385 owner=self.logged_in_user).system_id
1386 for _ in xrange(3)
1387- }
1388+ }
1389 # And one with no owner
1390 another_node = factory.make_Node(status=NODE_STATUS.RESERVED)
1391 node_ids.add(another_node.system_id)
1392@@ -1207,7 +2021,7 @@
1393 reverse('nodes_handler'), {
1394 'op': 'release',
1395 'nodes': node_ids
1396- })
1397+ })
1398 self.assertEqual(
1399 (httplib.FORBIDDEN,
1400 "You don't have the required permission to release the "
1401@@ -1227,7 +2041,7 @@
1402 reverse('nodes_handler'), {
1403 'op': 'release',
1404 'nodes': [node.system_id for node in nodes],
1405- })
1406+ })
1407 # Awkward parsing again, because a string is returned, not JSON
1408 expected = [
1409 "%s ('%s')" % (node.system_id, node.display_status())
1410@@ -1247,12 +2061,12 @@
1411 nodes = [
1412 factory.make_Node(status=status, owner=owner)
1413 for status in acceptable_states
1414- ]
1415+ ]
1416 response = self.client.post(
1417 reverse('nodes_handler'), {
1418 'op': 'release',
1419 'nodes': [node.system_id for node in nodes],
1420- })
1421+ })
1422 parsed_result = json.loads(response.content)
1423 self.assertEqual(httplib.OK, response.status_code)
1424 # The first node is READY, so shouldn't be touched.
1425@@ -1269,7 +2083,7 @@
1426 reverse('nodes_handler'), {
1427 'op': 'release',
1428 'nodes': [node.system_id],
1429- })
1430+ })
1431 self.assertEqual(httplib.OK, response.status_code, response)
1432 node = reload_object(node)
1433 self.assertEqual(NODE_STATUS.DISK_ERASING, node.status)
1434@@ -1532,7 +2346,7 @@
1435 ('failed_releasing', dict(status=NODE_STATUS.FAILED_RELEASING)),
1436 ('disk_erasing', dict(status=NODE_STATUS.DISK_ERASING)),
1437 ('failed_disk_erasing', dict(status=NODE_STATUS.FAILED_DISK_ERASING)),
1438- ]
1439+ ]
1440
1441 old_allocated_status = 6
1442
1443
1444=== modified file 'src/maasserver/models/eventtype.py'
1445--- src/maasserver/models/eventtype.py 2015-03-25 15:33:23 +0000
1446+++ src/maasserver/models/eventtype.py 2015-04-03 19:28:00 +0000
1447@@ -7,14 +7,14 @@
1448 absolute_import,
1449 print_function,
1450 unicode_literals,
1451- )
1452+)
1453
1454 str = None
1455
1456 __metaclass__ = type
1457 __all__ = [
1458 'EventType',
1459- ]
1460+]
1461
1462
1463 import logging
1464@@ -42,6 +42,14 @@
1465 logging.CRITICAL: 'CRITICAL',
1466 }
1467
1468+LOGGING_LEVELS_BY_NAME = {
1469+ 'DEBUG': logging.DEBUG,
1470+ 'INFO': logging.INFO,
1471+ 'WARNING': logging.WARNING,
1472+ 'ERROR': logging.ERROR,
1473+ 'CRITICAL': logging.CRITICAL,
1474+}
1475+
1476
1477 class EventTypeManager(Manager):
1478 """A utility to manage the collection of Events."""
1479
1480=== modified file 'src/maasserver/urls_api.py'
1481--- src/maasserver/urls_api.py 2015-03-25 15:33:23 +0000
1482+++ src/maasserver/urls_api.py 2015-04-03 19:28:00 +0000
1483@@ -7,7 +7,7 @@
1484 absolute_import,
1485 print_function,
1486 unicode_literals,
1487- )
1488+)
1489
1490 str = None
1491
1492@@ -77,6 +77,7 @@
1493 NodeGroupsHandler,
1494 )
1495 from maasserver.api.nodes import (
1496+ EventsHandler,
1497 NodeHandler,
1498 NodesHandler,
1499 )
1500@@ -117,6 +118,7 @@
1501 BootResourceFileUploadHandler, authentication=api_auth)
1502 boot_resources_handler = RestrictedResource(
1503 BootResourcesHandler, authentication=api_auth)
1504+events_handler = RestrictedResource(EventsHandler, authentication=api_auth)
1505 files_handler = RestrictedResource(FilesHandler, authentication=api_auth)
1506 file_handler = RestrictedResource(FileHandler, authentication=api_auth)
1507 ipaddresses_handler = RestrictedResource(
1508@@ -216,6 +218,7 @@
1509 r'^devices/(?P<system_id>[\w\-]+)/$', device_handler,
1510 name='device_handler'),
1511 url(r'^devices/$', devices_handler, name='devices_handler'),
1512+ url(r'^events/$', events_handler, name='events_handler'),
1513 url(
1514 r'^nodegroups/(?P<uuid>[^/]+)/$',
1515 nodegroup_handler, name='nodegroup_handler'),
1516
1517=== added file 'src/maasserver/views/nodes.py'
1518--- src/maasserver/views/nodes.py 1970-01-01 00:00:00 +0000
1519+++ src/maasserver/views/nodes.py 2015-04-03 19:28:00 +0000
1520@@ -0,0 +1,626 @@
1521+# Copyright 2012-2014 Canonical Ltd. This software is licensed under the
1522+# GNU Affero General Public License version 3 (see the file LICENSE).
1523+
1524+"""Nodes views."""
1525+
1526+from __future__ import (
1527+ absolute_import,
1528+ print_function,
1529+ unicode_literals,
1530+)
1531+
1532+str = None
1533+
1534+__metaclass__ = type
1535+__all__ = [
1536+ 'enlist_preseed_view',
1537+ 'MacAdd',
1538+ 'MacDelete',
1539+ 'NodeDelete',
1540+ 'NodeEventListView',
1541+ 'NodePreseedView',
1542+ 'NodeView',
1543+ 'NodeEdit',
1544+ 'prefetch_nodes_listing',
1545+]
1546+
1547+from cgi import escape
1548+import json
1549+import logging
1550+import re
1551+from textwrap import dedent
1552+
1553+from django.contrib import messages
1554+from django.core.urlresolvers import reverse
1555+from django.http import HttpResponse
1556+from django.shortcuts import (
1557+ get_object_or_404,
1558+ render_to_response,
1559+)
1560+from django.template import (
1561+ loader,
1562+ RequestContext,
1563+)
1564+from django.utils.safestring import mark_safe
1565+from django.views.generic import (
1566+ CreateView,
1567+ DetailView,
1568+ UpdateView,
1569+)
1570+from lxml import etree
1571+from maasserver.clusterrpc.power_parameters import get_power_types
1572+from maasserver.enum import (
1573+ NODE_BOOT,
1574+ NODE_PERMISSION,
1575+ NODE_STATUS,
1576+ NODE_STATUS_CHOICES_DICT,
1577+)
1578+from maasserver.exceptions import MAASAPIException
1579+from maasserver.forms import (
1580+ get_action_form,
1581+ get_node_edit_form,
1582+ MACAddressForm,
1583+)
1584+from maasserver.models import (
1585+ MACAddress,
1586+ Node,
1587+ StaticIPAddress,
1588+ Tag,
1589+)
1590+from maasserver.models.config import Config
1591+from maasserver.models.event import Event
1592+from maasserver.models.nodeprobeddetails import get_single_probed_details
1593+from maasserver.preseed import (
1594+ get_enlist_preseed,
1595+ get_preseed,
1596+ OS_WITH_IPv6_SUPPORT,
1597+)
1598+from maasserver.third_party_drivers import get_third_party_driver
1599+from maasserver.utils.converters import XMLToYAML
1600+from maasserver.views import (
1601+ HelpfulDeleteView,
1602+ PaginatedListView,
1603+)
1604+from metadataserver.enum import RESULT_TYPE
1605+from metadataserver.models import NodeResult
1606+from netaddr import IPAddress
1607+from provisioningserver.tags import merge_details_cleanly
1608+
1609+
1610+def message_from_form_stats(action, done, not_actionable, not_permitted):
1611+ """Return a message suitable for user display from the given stats."""
1612+ action_name = 'The action "%s"' % action.display
1613+ # singular/plural messages.
1614+ done_templates = [
1615+ '%s was successfully performed on %d node.',
1616+ '%s was successfully performed on %d nodes.'
1617+ ]
1618+ not_actionable_templates = [
1619+ ('%s could not be performed on %d node because its '
1620+ 'state does not allow that action.'),
1621+ ('%s could not be performed on %d nodes because their '
1622+ 'state does not allow that action.'),
1623+ ]
1624+ not_permitted_templates = [
1625+ ('%s could not be performed on %d node because that '
1626+ "action is not permitted on that node."),
1627+ ('%s could not be performed on %d nodes because that '
1628+ "action is not permitted on these nodes."),
1629+ ]
1630+ number_message = [
1631+ (done, done_templates),
1632+ (not_actionable, not_actionable_templates),
1633+ (not_permitted, not_permitted_templates)]
1634+ message = []
1635+ for index, (number, message_templates) in enumerate(number_message):
1636+ singular, plural = message_templates
1637+ if number != 0:
1638+ message_template = singular if number == 1 else plural
1639+ message.append(message_template % (action_name, number))
1640+ # Override the action name so that only the first sentence will
1641+ # contain the full name of the action.
1642+ action_name = 'It'
1643+ level = index
1644+ return ' '.join(message), ('info', 'warning', 'error')[level]
1645+
1646+
1647+def prefetch_nodes_listing(nodes_query):
1648+ """Prefetch any data needed to display a given query of nodes.
1649+
1650+ :param nodes_query: A query set of nodes.
1651+ :return: A version of `nodes_query` that prefetches any data needed for
1652+ displaying these nodes as a listing.
1653+ """
1654+ return (
1655+ nodes_query
1656+ .prefetch_related('macaddress_set')
1657+ .select_related('nodegroup')
1658+ .prefetch_related('nodegroup__nodegroupinterface_set')
1659+ .prefetch_related('zone'))
1660+
1661+
1662+def generate_js_power_types(nodegroup=None):
1663+ """Return a JavaScript definition of supported power-type choices.
1664+
1665+ Produces an array of power-type identifiers, starting with the opening
1666+ bracket and ending with the closing bracket, without line breaks on either
1667+ end. Entries are one per line, sorted lexicographically.
1668+ """
1669+ if nodegroup is not None:
1670+ nodegroup = [nodegroup]
1671+ power_types = get_power_types(nodegroup, ignore_errors=True)
1672+ names = ['"%s"' % power_type for power_type in sorted(power_types)]
1673+ return mark_safe("[\n%s\n]" % ',\n'.join(names))
1674+
1675+
1676+def node_to_dict(node, event_log_count=0):
1677+ """Convert `Node` to a dictionary.
1678+
1679+ :param event_log_count: Number of entries from the event log to add to
1680+ the dictionary.
1681+ """
1682+ if node.owner is None:
1683+ owner = ""
1684+ else:
1685+ owner = '%s' % node.owner
1686+ pxe_mac = node.get_pxe_mac()
1687+ node_dict = dict(
1688+ id=node.id,
1689+ system_id=node.system_id,
1690+ url=reverse('node-view', args=[node.system_id]),
1691+ hostname=node.hostname,
1692+ architecture=node.architecture,
1693+ fqdn=node.fqdn,
1694+ status=node.display_status(),
1695+ owner=owner,
1696+ cpu_count=node.cpu_count,
1697+ memory=node.display_memory(),
1698+ storage=node.display_storage(),
1699+ power_state=node.power_state,
1700+ zone=node.zone.name,
1701+ zone_url=reverse('zone-view', args=[node.zone.name]),
1702+ mac=None if pxe_mac is None else pxe_mac.mac_address.get_raw(),
1703+ vendor=node.get_pxe_mac_vendor(),
1704+ macs=[mac.mac_address.get_raw() for mac in node.get_extra_macs()],
1705+ )
1706+ if event_log_count != 0:
1707+ # Add event information to the generated node dictionary. We exclude
1708+ # debug after we calculate the count, so we show the correct total
1709+ # number of events.
1710+ node_events = Event.objects.filter(node=node)
1711+ total_num_events = node_events.count()
1712+ non_debug_events = node_events.exclude(
1713+ type__level=logging.DEBUG).order_by('-id')
1714+ if event_log_count > 0:
1715+ # Limit the number of events.
1716+ events = non_debug_events.all()[:event_log_count]
1717+ displayed_events_count = len(events)
1718+ node_dict['events'] = dict(
1719+ total=total_num_events,
1720+ count=displayed_events_count,
1721+ events=[event_to_dict(event) for event in events],
1722+ more_url=reverse('node-event-list-view', args=[node.system_id]),
1723+ )
1724+ return node_dict
1725+
1726+
1727+def event_to_dict(event):
1728+ """Convert `Event` to a dictionary."""
1729+ return dict(
1730+ node=event.node.system_id,
1731+ hostname=event.node.hostname,
1732+ id=event.id,
1733+ level=event.type.level_str,
1734+ created=event.created.strftime('%a, %d %b. %Y %H:%M:%S'),
1735+ type=event.type.description,
1736+ description=event.description
1737+ )
1738+
1739+
1740+def convert_query_status(value):
1741+ """Convert the given value into a list of status integers."""
1742+ value = value.lower()
1743+ ids = []
1744+ for status_id, status_text in NODE_STATUS_CHOICES_DICT.items():
1745+ status_text = status_text.lower()
1746+ if value in status_text:
1747+ ids.append(status_id)
1748+ if len(ids) == 0:
1749+ return None
1750+ return ids
1751+
1752+
1753+def enlist_preseed_view(request):
1754+ """View method to display the enlistment preseed."""
1755+ warning_message = (
1756+ "The URL mentioned in the following enlistment preseed will "
1757+ "be different depending on which cluster controller is "
1758+ "responsible for the enlisting node. The URL shown here is for "
1759+ "nodes handled by the cluster controller located in the region "
1760+ "controller's network."
1761+ )
1762+ context = RequestContext(request, {'warning_message': warning_message})
1763+ try:
1764+ preseed = get_enlist_preseed()
1765+ except NameError as e:
1766+ preseed = "ERROR RENDERING PRESEED\n" + unicode(e)
1767+ return render_to_response(
1768+ 'maasserver/enlist_preseed.html',
1769+ {'preseed': mark_safe(preseed)},
1770+ context_instance=context)
1771+
1772+
1773+class NodeViewMixin:
1774+ """Mixin class used to fetch a node by system_id.
1775+
1776+ The logged-in user must have View permission to access this page.
1777+ """
1778+
1779+ context_object_name = 'node'
1780+
1781+ def get_object(self):
1782+ system_id = self.kwargs.get('system_id', None)
1783+ node = Node.objects.get_node_or_404(
1784+ system_id=system_id, user=self.request.user,
1785+ perm=NODE_PERMISSION.VIEW)
1786+ return node
1787+
1788+
1789+class NodePreseedView(NodeViewMixin, DetailView):
1790+ """View class to display a node's preseed."""
1791+
1792+ template_name = 'maasserver/node_preseed.html'
1793+
1794+ def get_context_data(self, **kwargs):
1795+ context = super(NodePreseedView, self).get_context_data(**kwargs)
1796+ node = self.get_object()
1797+ # Display the preseed content exactly as generated by
1798+ # `get_preseed`. This will be rendered in a <pre> tag.
1799+ try:
1800+ preseed = get_preseed(node)
1801+ except NameError as e:
1802+ preseed = "ERROR RENDERING PRESEED\n" + unicode(e)
1803+ context['preseed'] = mark_safe(preseed)
1804+ context['is_commissioning'] = (
1805+ node.status == NODE_STATUS.COMMISSIONING)
1806+ return context
1807+
1808+
1809+# Info message displayed on the node page for COMMISSIONING
1810+# or READY nodes.
1811+NODE_BOOT_INFO = mark_safe("""
1812+You can boot this node using an adequately
1813+configured DHCP server. See
1814+<a href="https://maas.ubuntu.com/docs/nodes.html"
1815+>https://maas.ubuntu.com/docs/nodes.html</a> for instructions.
1816+""")
1817+
1818+
1819+NO_POWER_SET = mark_safe("""
1820+This node does not have a power type set and MAAS will be unable to
1821+control it. Click 'Edit node' and set one.
1822+""")
1823+
1824+
1825+THIRD_PARTY_DRIVERS_NOTICE = dedent("""
1826+ Third party drivers may be used when booting or installing nodes.
1827+ These may be proprietary and closed-source.
1828+ """)
1829+
1830+
1831+THIRD_PARTY_DRIVERS_ADMIN_NOTICE = dedent("""
1832+ The installation of third party drivers can be disabled on the <a
1833+ href="%s#third_party_drivers">settings</a> page.
1834+ """)
1835+
1836+UNCONFIGURED_IPS_NOTICE = dedent("""
1837+ Automatic configuration of IPv6 addresses is currently only supported on
1838+ Ubuntu, using the fast installer. To activate the IPv6 address(es) shown
1839+ here, configure them in the installed operating system.
1840+ """)
1841+
1842+
1843+def construct_third_party_drivers_notice(user_is_admin):
1844+ """Build and return the notice about third party drivers.
1845+
1846+ If `user_is_admin` is True, a link to the settings page will be
1847+ included in the message.
1848+
1849+ :param user_is_admin: True if the user is an administrator, False
1850+ otherwise.
1851+ """
1852+ if user_is_admin:
1853+ return mark_safe(
1854+ THIRD_PARTY_DRIVERS_NOTICE +
1855+ THIRD_PARTY_DRIVERS_ADMIN_NOTICE %
1856+ escape(reverse("settings"), quote=True))
1857+ else:
1858+ return mark_safe(THIRD_PARTY_DRIVERS_NOTICE)
1859+
1860+
1861+class NodeView(NodeViewMixin, UpdateView):
1862+ """View class to display a node's information and buttons for the actions
1863+ which can be performed on this node.
1864+ """
1865+
1866+ template_name = 'maasserver/node_view.html'
1867+
1868+ def get_form_class(self):
1869+ return get_action_form(self.request.user, self.request)
1870+
1871+ # The number of events shown on the node view page.
1872+ number_of_events_shown = 5
1873+
1874+ def get(self, request, *args, **kwargs):
1875+ """Handle a GET request."""
1876+ if request.is_ajax():
1877+ return self.handle_ajax_request(request, *args, **kwargs)
1878+ return super(NodeView, self).get(request, *args, **kwargs)
1879+
1880+ def warn_unconfigured_ip_addresses(self, node):
1881+ """Should the UI warn about unconfigured IPv6 addresses on the node?
1882+
1883+ Static IPv6 addresses are configured on the node using Curtin. But
1884+ this is not yet supported for all operating systems and installers.
1885+ If a node has IPv6 addresses assigned but is not being deployed in a
1886+ way that supports configuring them, the node page should show a warning
1887+ to say that the user will need to configure the node to use those
1888+ addresses.
1889+
1890+ :return: Bool: should the UI show this warning?
1891+ """
1892+ supported_os = (node.get_osystem() in OS_WITH_IPv6_SUPPORT)
1893+ if supported_os and node.boot_type == NODE_BOOT.FASTPATH:
1894+ # MAAS knows how to configure IPv6 addresses on an Ubuntu node
1895+ # installed with the fast installer. No warning needed.
1896+ return False
1897+ # For other installs, we need the warning if and only if the node has
1898+ # static IPv6 addresses.
1899+ static_ips = StaticIPAddress.objects.filter(macaddress__node=node)
1900+ return any(
1901+ IPAddress(static_ip.ip).version == 6
1902+ for static_ip in static_ips)
1903+
1904+ def get_context_data(self, **kwargs):
1905+ context = super(NodeView, self).get_context_data(**kwargs)
1906+ node = self.get_object()
1907+ context['can_edit'] = self.request.user.has_perm(
1908+ NODE_PERMISSION.EDIT, node)
1909+ if node.status in (NODE_STATUS.COMMISSIONING, NODE_STATUS.READY):
1910+ messages.info(self.request, NODE_BOOT_INFO)
1911+ if node.power_type == '':
1912+ messages.error(self.request, NO_POWER_SET)
1913+ if self.warn_unconfigured_ip_addresses(node):
1914+ messages.warning(self.request, UNCONFIGURED_IPS_NOTICE)
1915+ context['unconfigured_ips_warning'] = UNCONFIGURED_IPS_NOTICE
1916+
1917+ context['error_text'] = (
1918+ node.error if node.status == NODE_STATUS.FAILED_COMMISSIONING
1919+ else None)
1920+ context['status_text'] = (
1921+ node.error if node.status != NODE_STATUS.FAILED_COMMISSIONING
1922+ else None)
1923+ kernel_opts = node.get_effective_kernel_options()
1924+ context['kernel_opts'] = {
1925+ 'is_global': kernel_opts[0] is None,
1926+ 'is_tag': isinstance(kernel_opts[0], Tag),
1927+ 'tag': kernel_opts[0],
1928+ 'value': kernel_opts[1]
1929+ }
1930+ # Produce a "clean" composite details document.
1931+ probed_details = merge_details_cleanly(
1932+ get_single_probed_details(node.system_id))
1933+ # We check here if there's something to show instead of after
1934+ # the call to get_single_probed_details() because here the
1935+ # details will be guaranteed well-formed.
1936+ if len(probed_details.xpath('/*/*')) == 0:
1937+ context['probed_details_xml'] = None
1938+ context['probed_details_yaml'] = None
1939+ else:
1940+ context['probed_details_xml'] = etree.tostring(
1941+ probed_details, encoding=unicode, pretty_print=True)
1942+ context['probed_details_yaml'] = XMLToYAML(
1943+ etree.tostring(
1944+ probed_details, encoding=unicode,
1945+ pretty_print=True)).convert()
1946+
1947+ commissioning_results = NodeResult.objects.filter(
1948+ node=node, result_type=RESULT_TYPE.COMMISSIONING).count()
1949+ context['nodecommissionresults'] = commissioning_results
1950+
1951+ installation_results = NodeResult.objects.filter(
1952+ node=node, result_type=RESULT_TYPE.INSTALLATION)
1953+ if len(installation_results) > 1:
1954+ for result in installation_results:
1955+ result.name = re.sub('[_.]', ' ', result.name)
1956+ context['nodeinstallresults'] = installation_results
1957+ elif len(installation_results) == 1:
1958+ installation_results[0].name = "install log"
1959+ context['nodeinstallresults'] = installation_results
1960+
1961+ context['third_party_drivers_enabled'] = Config.objects.get_config(
1962+ 'enable_third_party_drivers')
1963+ context['drivers'] = get_third_party_driver(node)
1964+
1965+ event_list = (
1966+ Event.objects.filter(node=self.get_object())
1967+ .exclude(type__level=logging.DEBUG)
1968+ .order_by('-id')[:self.number_of_events_shown])
1969+ context['event_list'] = event_list
1970+ context['event_count'] = Event.objects.filter(
1971+ node=self.get_object()).count()
1972+
1973+ return context
1974+
1975+ def dispatch(self, *args, **kwargs):
1976+ """Override from Django `View`: Handle MAAS exceptions.
1977+
1978+ Node actions may raise exceptions derived from
1979+ :class:`MAASAPIException`. This type of exception contains an
1980+ http status code that we will forward to the client.
1981+ """
1982+ try:
1983+ return super(NodeView, self).dispatch(*args, **kwargs)
1984+ except MAASAPIException as e:
1985+ return e.make_http_response()
1986+
1987+ def get_success_url(self):
1988+ return reverse('node-view', args=[self.get_object().system_id])
1989+
1990+ def render_node_actions(self, request):
1991+ """Render the HTML for all the available node actions."""
1992+ template = loader.get_template('maasserver/node_actions.html')
1993+ self.object = self.get_object()
1994+ context = {
1995+ 'node': self.object,
1996+ 'can_edit': self.request.user.has_perm(
1997+ NODE_PERMISSION.EDIT, self.object),
1998+ 'form': self.get_form(self.get_form_class()),
1999+ }
2000+ return template.render(RequestContext(request, context))
2001+
2002+ def handle_ajax_request(self, request, *args, **kwargs):
2003+ """JSON response to update the node view."""
2004+ node = self.get_object()
2005+ node = node_to_dict(
2006+ node, event_log_count=self.number_of_events_shown)
2007+ node['action_view'] = self.render_node_actions(request)
2008+ return HttpResponse(json.dumps(node), mimetype='application/json')
2009+
2010+
2011+class NodeEventListView(NodeViewMixin, PaginatedListView):
2012+
2013+ context_object_name = "event_list"
2014+
2015+ template_name = "maasserver/node_event_list.html"
2016+
2017+ def get_queryset(self):
2018+ return Event.objects.filter(
2019+ node=self.get_object()).order_by('-id')
2020+
2021+ def get_context_data(self, **kwargs):
2022+ context = super(NodeEventListView, self).get_context_data(**kwargs)
2023+ node = self.get_object()
2024+ context['node'] = node
2025+ return context
2026+
2027+
2028+class NodeEdit(UpdateView):
2029+
2030+ template_name = 'maasserver/node_edit.html'
2031+
2032+ def get_object(self):
2033+ system_id = self.kwargs.get('system_id', None)
2034+ node = Node.objects.get_node_or_404(
2035+ system_id=system_id, user=self.request.user,
2036+ perm=NODE_PERMISSION.EDIT)
2037+ return node
2038+
2039+ def get_form_class(self):
2040+ return get_node_edit_form(self.request.user)
2041+
2042+ def get_has_owner(self):
2043+ node = self.get_object()
2044+ if node is None or node.owner is None:
2045+ return mark_safe("false")
2046+ return mark_safe("true")
2047+
2048+ def get_form_kwargs(self):
2049+ # This is here so the request can be passed to the form. The
2050+ # form needs it because it sets error messages for the UI.
2051+ kwargs = super(NodeEdit, self).get_form_kwargs()
2052+ kwargs['request'] = self.request
2053+ kwargs['ui_submission'] = True
2054+ return kwargs
2055+
2056+ def get_success_url(self):
2057+ return reverse('node-view', args=[self.get_object().system_id])
2058+
2059+ def get_context_data(self, **kwargs):
2060+ context = super(NodeEdit, self).get_context_data(**kwargs)
2061+ context['power_types'] = generate_js_power_types(
2062+ self.get_object().nodegroup)
2063+ # 'os_release' lets us know if we should render the `OS`
2064+ # and `Release` choice fields in the UI.
2065+ context['os_release'] = self.get_has_owner()
2066+ return context
2067+
2068+
2069+class NodeDelete(HelpfulDeleteView):
2070+
2071+ template_name = 'maasserver/node_confirm_delete.html'
2072+ context_object_name = 'node_to_delete'
2073+ model = Node
2074+
2075+ def get_object(self):
2076+ system_id = self.kwargs.get('system_id', None)
2077+ node = Node.objects.get_node_or_404(
2078+ system_id=system_id, user=self.request.user,
2079+ perm=NODE_PERMISSION.ADMIN)
2080+ return node
2081+
2082+ def get_next_url(self):
2083+ return reverse('index') + "#/nodes"
2084+
2085+ def name_object(self, obj):
2086+ """See `HelpfulDeleteView`."""
2087+ return "Node %s" % obj.system_id
2088+
2089+
2090+class MacAdd(CreateView):
2091+ form_class = MACAddressForm
2092+ template_name = 'maasserver/node_add_mac.html'
2093+
2094+ def get_node(self):
2095+ system_id = self.kwargs.get('system_id', None)
2096+ node = Node.objects.get_node_or_404(
2097+ system_id=system_id, user=self.request.user,
2098+ perm=NODE_PERMISSION.EDIT)
2099+ return node
2100+
2101+ def get_form_kwargs(self):
2102+ kwargs = super(MacAdd, self).get_form_kwargs()
2103+ kwargs['node'] = self.get_node()
2104+ return kwargs
2105+
2106+ def form_valid(self, form):
2107+ res = super(MacAdd, self).form_valid(form)
2108+ messages.info(self.request, "MAC address added.")
2109+ return res
2110+
2111+ def get_success_url(self):
2112+ node = self.get_node()
2113+ return reverse('node-edit', args=[node.system_id])
2114+
2115+ def get_context_data(self, **kwargs):
2116+ context = super(MacAdd, self).get_context_data(**kwargs)
2117+ context.update({'node': self.get_node()})
2118+ return context
2119+
2120+
2121+class MacDelete(HelpfulDeleteView):
2122+
2123+ template_name = 'maasserver/mac_confirm_delete.html'
2124+ context_object_name = 'mac_to_delete'
2125+ model = MACAddress
2126+
2127+ def get_node(self):
2128+ system_id = self.kwargs.get('system_id', None)
2129+ node = Node.objects.get_node_or_404(
2130+ system_id=system_id, user=self.request.user,
2131+ perm=NODE_PERMISSION.EDIT)
2132+ return node
2133+
2134+ def get_object(self):
2135+ node = self.get_node()
2136+ mac_address = self.kwargs.get('mac_address', None)
2137+ return get_object_or_404(
2138+ MACAddress, node=node, mac_address=mac_address)
2139+
2140+ def get_next_url(self):
2141+ node = self.get_node()
2142+ return reverse('node-edit', args=[node.system_id])
2143+
2144+ def name_object(self, obj):
2145+ """See `HelpfulDeleteView`."""
2146+ return "MAC address %s" % obj.mac_address