Merge lp:~dimitern/maas/bug-994887-alternate-nodes-list-sorting into lp:~maas-committers/maas/trunk

[Dimiter Naydenov]

Fix bug #994887: Nodes list view now supports sorting by hostname(mac) or status columns.

Implemented sorting for /MAAS/nodes/ list - by hostname or status (asc/desc). This is an alternative, simpler solution, compared to https://code.launchpad.net/~dimitern/maas/bug-994887-nodes-listing-does-not-support-sorting/+merge/130600 - not using third party modules (django-sortable) anymore. UI changes (sorting up/down arrows) were already approved, so just the code changes need approval.

[John A Meinel (jameinel)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/24/2012 02:57 PM, Dimiter Naydenov wrote:
> Dimiter Naydenov has proposed merging
> lp:~dimitern/maas/bug-994887-alternate-nodes-list-sorting into
> lp:maas.
>
> Commit message: Fix bug #994887: Nodes list view now supports
> sorting by hostname(mac) or status columns.
>
> Requested reviews: Launchpad code reviewers (launchpad-reviewers)
> Related bugs: Bug #994887 in MAAS: "Nodes listing does not support
> sorting" https://bugs.launchpad.net/maas/+bug/994887
>
> For more details, see:
> https://code.launchpad.net/~dimitern/maas/bug-994887-alternate-nodes-list-sorting/+merge/131177
>
> Implemented sorting for /MAAS/nodes/ list - by hostname or status
> (asc/desc). This is an alternative, simpler solution, compared to
> https://code.launchpad.net/~dimitern/maas/bug-994887-nodes-listing-does-not-support-sorting/+merge/130600
> - not using third party modules (django-sortable) anymore. UI
> changes (sorting up/down arrows) were already approved, so just the
> code changes need approval.
>

+ # Just creare some nodes to have something in the list

typo 'create'

However, why do we need the nodes to actually get the links? I would
tend to have no more than 1 'make_node()' call.

...
+ node_links = [
+ reverse('node-view', args=[node.system_id])
+
+ for node in sorted_nodes]

The spacing in this test looks wrong. You probably want to run 'make
lint' and handle anything that it brings up as a problem.

+ # Now check the reverse order
+ sorted_nodes = sorted(
+ nodes, key=lambda x: x.status, reverse=True)

I don't think you need to re-do the sort, if anything just do
"sorted_nodes = reversed(sorted_nodes)".
However, I think all you actually need to do is:
node_links = reversed(node_links) and not do anything with re-sorting
the node objects themselves.

That should simplify the test a bit.

+ # Exclude header sorting links
         self.assertEqual([last_node_link],
- - document.xpath("//div[@id='nodes']/table//a/@href"))
+ filter(
+ lambda x: 'sort=' not in x,
+ document.xpath("//div[@id='nodes']/table//a/@href")))

I'm hesitant about using 'filter' in these cases. I'm wondering if
there is a better way to write the tests. Perhaps giving a specific
class id for the node links, or one for the sorting links?

         nodes = Node.objects.get_nodes(
             user=self.request.user, prefetch_mac=True,
- - perm=NODE_PERMISSION.VIEW,).order_by('-created')
+ perm=NODE_PERMISSION.VIEW,).order_by(order_by)

I kind of feel like we want to always have a sort field that is
unique. What I mean, is that if you sort by 'status' then all nodes
with the same status can occur in arbitrary order. (It is probably
reasonably static because the db isn't changing a lot in the
background, but it isn't guaranteed). Certainly an unrelated action
that would otherwise not affect the nodes being shown could cause a
node that was on page 1 to suddenly swap places with a node on page 2.

I'm not sure about the specific order of operations, but I think we
want something li...

review: Approve

[Martin Packman (gz)]

        # Just creare some nodes to have something in the list

Tyop, 'create'. Hm, we should really make a helper for multi-node creation on these views, but that can happen later.

+ sorted_nodes = sorted(nodes, key=lambda x: x.hostname)

Always nice to use operator.attrgetter('hostname') for this.

+ header_links = document.xpath("//div[@id='nodes']/table//th/a/@href")
+ for link in header_links:
+ self.assertIn('page=1', link)
+ self.assertIn('query=maas-tags%3Dshiny', link)

Rather than having multiple string contains assertions like this, I'd use urlparse.parse_qsl and testtools matchers against all the key/values you expect, so a failure has the full context. Something like:

    self.assertThat(urlparse.parse_qsl(link), ContainsAll([('page', '1'), ...]))

- node_links = document.xpath("//div[@id='nodes']/table//a/@href")
+ # Exclude header sorting links
+ node_links = filter(
+ lambda x: 'sort=' not in x,
+ document.xpath("//div[@id='nodes']/table//a/@href"))

For these, I'd adjust the xpath instead, ie, .../table//td//a/...

+ links = {field: self.request.path + '?' for field in fields}

Give relative links rather than including the path in them, the link should just start with the '?', note tests will need changing to expect this too.

review: Needs Fixing

[Dimiter Naydenov (dimitern)]

Thanks for the review! I made the requested modifications and also simplified the tests (fixing a minor issue I found with the initial sort order).

[Martin Packman (gz)]

+ perm=NODE_PERMISSION.VIEW,).order_by(order_by, '-created')

Having the default order_by here being ('-created', '-created') but all the other changes look good.

review: Approve

=== modified file 'src/maasserver/static/css/components/table_list.css'
--- src/maasserver/static/css/components/table_list.css	2012-04-05 06:23:59 +0000
+++ src/maasserver/static/css/components/table_list.css	2012-10-24 16:02:22 +0000
@@ -30,6 +30,17 @@
     border-color: #dd4814;
     cursor: pointer;
     }
+.sort-none, .sort-asc, .sort-desc {
+    background-repeat: no-repeat;
+    background-position: right 0.4em;
+    padding-right: 1.2em;
+    }
+.sort-asc {
+    background-image: url('/static/img/up_arrow_dark.png');
+    }
+.sort-desc {
+    background-image: url('/static/img/down_arrow_dark.png');
+    }
 /* ul list */
 ul.list {
     list-style: none;

=== added file 'src/maasserver/static/img/down_arrow_dark.png'
Binary files src/maasserver/static/img/down_arrow_dark.png	1970-01-01 00:00:00 +0000 and src/maasserver/static/img/down_arrow_dark.png	2012-10-24 16:02:22 +0000 differ
=== added file 'src/maasserver/static/img/up_arrow_dark.png'
Binary files src/maasserver/static/img/up_arrow_dark.png	1970-01-01 00:00:00 +0000 and src/maasserver/static/img/up_arrow_dark.png	2012-10-24 16:02:22 +0000 differ
=== modified file 'src/maasserver/templates/maasserver/node_list.html'
--- src/maasserver/templates/maasserver/node_list.html	2012-10-12 13:33:09 +0000
+++ src/maasserver/templates/maasserver/node_list.html	2012-10-24 16:02:22 +0000
@@ -38,7 +38,7 @@
     {% if input_query_error %}
     <p class="form-errors">{{input_query_error}}</p>
     {% endif %}
-    {% include "maasserver/nodes_listing.html" %}
+    {% include "maasserver/nodes_listing.html" with sorting="true" %}
     {% include "maasserver/pagination.html" %}
     <a id="addnode" href="#" class="button right space-top">+ Add node</a>
     <div class="clear"></div>

=== modified file 'src/maasserver/templates/maasserver/nodes_listing.html'
--- src/maasserver/templates/maasserver/nodes_listing.html	2012-10-10 08:13:01 +0000
+++ src/maasserver/templates/maasserver/nodes_listing.html	2012-10-24 16:02:22 +0000
@@ -2,8 +2,15 @@
   <table class="list">
     <thead>
       <tr>
+	{% if sorting == "true" %}
+        <th><a href="{{ sort_links.hostname }}"
+	       class="{{ sort_classes.hostname }}">MAC</a></th>
+        <th><a href="{{ sort_links.status }}"
+	       class="{{ sort_classes.status }}">Status</a></th>
+	{% else %}
         <th>MAC</th>
         <th>Status</th>
+	{% endif %}
       </tr>
     </thead>
     {% for node in node_list %}

=== modified file 'src/maasserver/templates/maasserver/tag_view.html'
--- src/maasserver/templates/maasserver/tag_view.html	2012-10-18 11:26:02 +0000
+++ src/maasserver/templates/maasserver/tag_view.html	2012-10-24 16:02:22 +0000
@@ -34,7 +34,7 @@
   </ul>
   <div id="nodes" class="block first pad-top">
     <h2 class="block first line-top pad-top">{{ paginator.count }} Node{{ paginator.count|pluralize }}</h2>
-    {% include "maasserver/nodes_listing.html" %}
+    {% include "maasserver/nodes_listing.html" with sorting="false" %}
     {% include "maasserver/pagination.html" %}
   </div>
 {% endblock %}

=== modified file 'src/maasserver/tests/test_views_nodes.py'
--- src/maasserver/tests/test_views_nodes.py	2012-10-15 17:39:31 +0000
+++ src/maasserver/tests/test_views_nodes.py	2012-10-24 16:02:22 +0000
@@ -13,7 +13,12 @@
 __all__ = []
 
 import httplib
+from operator import attrgetter
 from unittest import skip
+from urlparse import (
+    parse_qsl,
+    urlparse,
+    )
 
 from django.conf import settings
 from django.core.urlresolvers import reverse
@@ -79,6 +84,108 @@
         enlist_preseed_link = reverse('enlist-preseed-view')
         self.assertIn(enlist_preseed_link, get_content_links(response))
 
+    def test_node_list_contains_column_sort_links(self):
+        # Just create a node to have something in the list
+        factory.make_node()
+        response = self.client.get(reverse('node-list'))
+        sort_hostname = '?sort=hostname&dir=asc'
+        sort_status = '?sort=status&dir=asc'
+        self.assertIn(sort_hostname, get_content_links(response))
+        self.assertIn(sort_status, get_content_links(response))
+
+    def test_node_list_sorts_by_hostname(self):
+        names = ['zero', 'one', 'five']
+        nodes = [factory.make_node(hostname=n) for n in names]
+
+        # First check the ascending sort order
+        sorted_nodes = sorted(nodes, key=attrgetter('hostname'))
+        response = self.client.get(
+            reverse('node-list'), {
+                'sort': 'hostname',
+                'dir': 'asc'})
+        node_links = [
+             reverse('node-view', args=[node.system_id])
+             for node in sorted_nodes]
+        self.assertEqual(
+            node_links,
+            [link for link in get_content_links(response)
+                if link.startswith('/nodes/node')])
+
+        # Now check the reverse order
+        node_links = list(reversed(node_links))
+        response = self.client.get(
+            reverse('node-list'), {
+                'sort': 'hostname',
+                'dir': 'desc'})
+        self.assertEqual(
+            node_links,
+            [link for link in get_content_links(response)
+                if link.startswith('/nodes/node')])
+
+    def test_node_list_sorts_by_status(self):
+        statuses = {
+            NODE_STATUS.READY,
+            NODE_STATUS.DECLARED,
+            NODE_STATUS.FAILED_TESTS,
+            }
+        nodes = [factory.make_node(status=s) for s in statuses]
+
+        # First check the ascending sort order
+        sorted_nodes = sorted(nodes, key=attrgetter('status'))
+        response = self.client.get(
+            reverse('node-list'), {
+                'sort': 'status',
+                'dir': 'asc'})
+        node_links = [
+             reverse('node-view', args=[node.system_id])
+             for node in sorted_nodes]
+        self.assertEqual(
+            node_links,
+            [link for link in get_content_links(response)
+                if link.startswith('/nodes/node')])
+
+        # Now check the reverse order
+        node_links = list(reversed(node_links))
+        response = self.client.get(
+            reverse('node-list'), {
+                'sort': 'status',
+                'dir': 'desc'})
+        self.assertEqual(
+            node_links,
+            [link for link in get_content_links(response)
+                if link.startswith('/nodes/node')])
+
+    def test_node_list_sort_preserves_other_params(self):
+        # Set a very small page size to save creating lots of nodes
+        page_size = 2
+        self.patch(nodes_views.NodeListView, 'paginate_by', page_size)
+
+        nodes = []
+        tag = factory.make_tag('shiny')
+        for name in ('bbb', 'ccc', 'ddd', 'aaa'):
+            node = factory.make_node(hostname=name)
+            node.tags = [tag]
+            nodes.append(node)
+
+        params = {
+                'sort': 'hostname',
+                'dir': 'asc',
+                'page': '1',
+                'query': 'maas-tags=shiny'}
+        response = self.client.get(reverse('node-list'), params)
+        document = fromstring(response.content)
+        header_links = document.xpath("//div[@id='nodes']/table//th/a/@href")
+        fields = iter(('hostname', 'status'))
+        field_dirs = iter(('desc', 'asc'))
+        for link in header_links:
+            self.assertThat(
+                parse_qsl(urlparse(link).query),
+                ContainsAll([
+                    ('page', '1'),
+                    ('query', 'maas-tags=shiny'),
+                    ('sort', next(fields)),
+                    ('dir', next(field_dirs))]))
+
     def test_node_list_displays_sorted_list_of_nodes(self):
         # Nodes are sorted on the node list page, newest first.
         nodes = [factory.make_node() for i in range(3)]
@@ -421,7 +528,8 @@
             {"query": "maas-tags=shiny cpu=2"})
         node2_link = reverse('node-view', args=[node2.system_id])
         document = fromstring(response.content)
-        node_links = document.xpath("//div[@id='nodes']/table//a/@href")
+        node_links = document.xpath(
+            "//div[@id='nodes']/table//td/a/@href")
         self.assertEqual([node2_link], node_links)
 
     def test_node_list_paginates(self):
@@ -434,7 +542,7 @@
         # Order node links with newest first as the view is expected to
         node_links = [reverse('node-view', args=[node.system_id])
             for node in reversed(nodes)]
-        expr_node_links = XPath("//div[@id='nodes']/table//a/@href")
+        expr_node_links = XPath("//div[@id='nodes']/table//td/a/@href")
         expr_page_anchors = XPath("//div[@class='pagination']//a")
         # Fetch first page, should link newest two nodes and page 2
         response = self.client.get(reverse('node-list'))
@@ -446,7 +554,8 @@
         # Fetch second page, should link next nodes and adjacent pages
         response = self.client.get(reverse('node-list'), {"page": 2})
         page2 = fromstring(response.content)
-        self.assertEqual(node_links[page_size:page_size * 2],
+        self.assertEqual(
+            node_links[page_size:page_size * 2],
             expr_node_links(page2))
         self.assertEqual([("first", "."), ("previous", "."),
                 ("next", "?page=3"), ("last", "?page=3")],
@@ -474,8 +583,9 @@
             {"query": "maas-tags=odd", "page": 3})
         document = fromstring(response.content)
         self.assertIn("5 matching nodes", document.xpath("string(//h1)"))
-        self.assertEqual([last_node_link],
-            document.xpath("//div[@id='nodes']/table//a/@href"))
+        self.assertEqual(
+            [last_node_link],
+            document.xpath("//div[@id='nodes']/table//td/a/@href"))
         self.assertEqual([("first", "?query=maas-tags%3Dodd"),
                 ("previous", "?query=maas-tags%3Dodd&page=2")],
             [(a.text.lower(), a.get("href"))

=== modified file 'src/maasserver/views/nodes.py'
--- src/maasserver/views/nodes.py	2012-10-19 13:55:51 +0000
+++ src/maasserver/views/nodes.py	2012-10-24 16:02:22 +0000
@@ -110,13 +110,26 @@
     def get(self, request, *args, **kwargs):
         self.query = request.GET.get("query")
         self.query_error = None
+        self.sort_by = request.GET.get("sort")
+        self.sort_dir = request.GET.get("dir")
+
         return super(NodeListView, self).get(request, *args, **kwargs)
 
     def get_queryset(self):
-        # Return node list sorted, newest first.
+        # Default sort - newest first, unless sorting params are
+        # present. In addition, to ensure order consistency, when
+        # sorting by non-unique fields (like status), we always
+        # sort by the unique creation date as well
+        if self.sort_by is not None:
+            prefix = '-' if self.sort_dir == 'desc' else ''
+            order_by = (prefix + self.sort_by, '-created')
+        else:
+            order_by = ('-created', )
+
+        # Return the sorted node list
         nodes = Node.objects.get_nodes(
             user=self.request.user, prefetch_mac=True,
-            perm=NODE_PERMISSION.VIEW,).order_by('-created')
+            perm=NODE_PERMISSION.VIEW,).order_by(*order_by)
         if self.query:
             try:
                 return constrain_nodes(nodes, _parse_constraints(self.query))
@@ -125,11 +138,39 @@
                 return Node.objects.none()
         return nodes
 
+    def _prepare_sort_links(self):
+        """Returns 2 dicts, with sort fields as keys and
+        links and CSS classes for the that field.
+        """
+
+        fields = ('hostname', 'status')
+        # Build relative URLs for the links, just with the params
+        links = {field: '?' for field in fields}
+        classes = {field: 'sort-none' for field in fields}
+
+        params = self.request.GET.copy()
+        reverse_dir = 'asc' if self.sort_dir == 'desc' else 'desc'
+
+        for field in fields:
+            params['sort'] = field
+            if field == self.sort_by:
+                params['dir'] = reverse_dir
+                classes[field] = 'sort-%s' % self.sort_dir
+            else:
+                params['dir'] = 'asc'
+
+            links[field] += params.urlencode()
+
+        return links, classes
+
     def get_context_data(self, **kwargs):
         context = super(NodeListView, self).get_context_data(**kwargs)
         context.update(get_longpoll_context())
         context["input_query"] = self.query
         context["input_query_error"] = self.query_error
+        links, classes = self._prepare_sort_links()
+        context["sort_links"] = links
+        context["sort_classes"] = classes
         return context