Merge lp:~widelands-dev/widelands-website/sitemap into lp:widelands-website

[kaputtnik]

Adding a sitemap.xml containing links to:

- Wiki (each article)
- News (news of last2 years)
- Forums (only links to section of forum, e.g. /forum/forum/3/)
- Mainpage
- Changelog
- Encyclopedia (Tribes, Buildings, Workers, Wares)
- Developer documentation (a link to the index)

I decided that each app has his own sitemap.py file containing what links get into sitemap.xml for this app. Added a new file sitemap_urls.py included from urls.py which gathers all sitemap files and serves the sitemap.xml.

Changes to sphinxdoc:

- Removed unused templates
- Updated urls.py to fit with future django versions (other urls.py files had this update already)

Changes to settings:

Same as in anti_spam_4, removed static string 'widelands' to make working with branches easier.

---------

There is a management command 'ping_google' available (works only if the site is registered on google webmaster tools), which could be used in a cron job to inform google of changes in sitemap.xml. I am not sure how often this should be called. See also: https://docs.djangoproject.com/en/1.8/ref/contrib/sitemaps/#pinging-google

[lp:~widelands-dev/widelands-website/sitemap]

437. By kaputtnik on 2016-11-12

provide only news which are not in future and no drafts

438. By kaputtnik on 2016-11-15

resolved merge conflict because of settings.py

439. By kaputtnik on 2016-11-15

merged with trunk

440. By kaputtnik on 2016-11-18

merged with trunk

441. By kaputtnik on 2016-11-18

merged with trunk and fixed wrong resolving of merge conflict

[kaputtnik (franku)]

I have failed with the last merge conflict, causing the missing entry 'django.contrib.sitemaps' in installed apps :-S. But this is fixed now.

review: Needs Resubmitting

[SirVer (sirver)]

code lgtm. not tested.

review: Approve

[lp:~widelands-dev/widelands-website/sitemap]

442. By kaputtnik on 2016-11-22

consistent use of 2 blank lines after imports

[kaputtnik (franku)]

merged and committed.

Thanks for the review :-)

=== modified file 'news/sitemap.py'
--- news/sitemap.py	2009-02-21 18:24:02 +0000
+++ news/sitemap.py	2016-11-22 19:21:51 +0000
@@ -1,5 +1,7 @@
 from django.contrib.sitemaps import Sitemap
-from widelands.news.models import Post
+from .models import Post
+from datetime import datetime
+from datetime import timedelta
 
 
 class NewsSitemap(Sitemap):
@@ -7,7 +9,8 @@
     priority = 0.5
 
     def items(self):
-        return Post.objects.published()
+        start_date = datetime.today() - timedelta(days=365 * 2)
+        return Post.objects.published().filter(publish__gt=start_date)
 
-        def lastmod(self, obj):
-            return obj.publish
+    def lastmod(self, obj):
+        return obj.publish

=== added file 'pybb/sitemap.py'
--- pybb/sitemap.py	1970-01-01 00:00:00 +0000
+++ pybb/sitemap.py	2016-11-22 19:21:51 +0000
@@ -0,0 +1,13 @@
+from django.contrib.sitemaps import Sitemap
+from pybb.models import Forum
+
+
+class ForumSitemap(Sitemap):
+    changefreq = 'monthly'
+    priority = 0.5
+
+    def items(self):
+        return Forum.objects.all()
+
+    def lastmod(self, obj):
+        return obj.updated

=== modified file 'settings.py'
--- settings.py	2016-11-15 08:03:19 +0000
+++ settings.py	2016-11-22 19:21:51 +0000
@@ -76,6 +76,7 @@
     'django.contrib.staticfiles',
     'django.contrib.sites',
     'django.contrib.humanize',
+    'django.contrib.sitemaps',
     'django_comments',
     'nocaptcha_recaptcha',
     # Thirdparty apps, but need preload

=== added file 'sitemap_urls.py'
--- sitemap_urls.py	1970-01-01 00:00:00 +0000
+++ sitemap_urls.py	2016-11-22 19:21:51 +0000
@@ -0,0 +1,29 @@
+from django.conf.urls import *
+
+from mainpage.views import mainpage
+from django.contrib.sitemaps.views import sitemap
+from static_sitemap import StaticViewSitemap
+from wiki.sitemap import *
+from news.sitemap import *
+from pybb.sitemap import *
+from wlhelp.sitemap import *
+from sphinxdoc.sitemap import *
+
+
+sitemaps = {
+    'static': StaticViewSitemap,
+    'docs': DocumentationSitemap,
+    'news': NewsSitemap,
+    'wiki': WikiSitemap,
+    'forum': ForumSitemap,
+    'wlhelptribe': WlHelpTribeSitemap,
+    'wlhelpware': WlHelpWareSitemap,
+    'wlhelpworker': WlHelpWorkerSitemap,
+    'wlhelpbuildings': WlHelpBuildingSitemap,
+    }
+
+urlpatterns = [
+    # Creating a sitemap.xml
+    url(r'^$', sitemap, {'sitemaps': sitemaps},
+        name='django.contrib.sitemaps.views.sitemap')
+    ]

=== added file 'sphinxdoc/sitemap.py'
--- sphinxdoc/sitemap.py	1970-01-01 00:00:00 +0000
+++ sphinxdoc/sitemap.py	2016-11-22 19:21:51 +0000
@@ -0,0 +1,23 @@
+from django.contrib.sitemaps import Sitemap
+from sphinxdoc.models import App
+import datetime
+import os
+
+app = App.objects.get(slug='wl')
+
+
+class DocumentationSitemap(Sitemap):
+    """This is just a dummy class to return the link to docs/wl/genindex."""
+
+    changefreq = 'yearly'
+    priority = 0.5
+
+    def items(self):
+        return ['']
+
+    def location(self, item):
+        return '/docs/wl/genindex/'
+
+    def lastmod(self, item):
+        return datetime.datetime.fromtimestamp(
+            os.path.getmtime(os.path.join(app.path, 'last_build')))

=== removed directory 'sphinxdoc/templates'
=== removed directory 'sphinxdoc/templates/sphinxdoc'
=== removed file 'sphinxdoc/templates/sphinxdoc/app_list.html'
--- sphinxdoc/templates/sphinxdoc/app_list.html	2016-06-13 07:14:59 +0000
+++ sphinxdoc/templates/sphinxdoc/app_list.html	1970-01-01 00:00:00 +0000
@@ -1,14 +0,0 @@
-{% extends 'base.html' %}
-
-{% block title %}{{ block.super }} » Documentation Overview{% endblock %}
-
-{% block content %}
-<div>
-    <h2 class="pagetitle">Documentation Overview</h2>
-    <ul>
-    {% for app in app_list %}
-        <li><a href="{{ app.get_absolute_url }}">{{ app.name }}</a></li>
-    {% endfor %}
-    </ul>
-</div>
-{% endblock content %}

=== removed file 'sphinxdoc/templates/sphinxdoc/documentation.html'
--- sphinxdoc/templates/sphinxdoc/documentation.html	2016-06-13 07:14:59 +0000
+++ sphinxdoc/templates/sphinxdoc/documentation.html	1970-01-01 00:00:00 +0000
@@ -1,65 +0,0 @@
-{% extends 'base.html' %}
-
-{% block title %}{{ block.super }} » {{ app.name }}{% for p in doc.parents %} » {{ p.title|striptags|safe }}{% endfor %} » {{ doc.title|striptags|safe }}{% endblock %}
-
-{% block content %}
-<div class="pagination-top">
-    » <a href="{{ app.get_absolute_url }}">{{ app.name }}</a>
-    {% for p in doc.parents %}
-    » <a href="{{ p.link }}">{{ p.title|safe }}</a>
-    {% endfor %}
-    » {{ doc.title|safe }}
-    {% if doc.prev or doc.next %}
-    <br /><br />
-    <span class="left">
-        {% if doc.prev %}
-          Prev: <a href="{{ doc.prev.link }}">{{ doc.prev.title|safe }}</a>
-        {% endif %}</span><span class="right">
-        {% if doc.next %}
-          Next: <a href="{{ doc.next.link }}">{{ doc.next.title|safe }}</a>
-        {% endif %}</span>
-    {% endif %}
-</div>
-
-<div class="sphinx">
-    {% block doc_body %}
-    {{ doc.body|safe }}    
-    {% endblock %}
-</div>
-
-<div class="pagination-bottom">
-    {% if doc.prev or doc.next %}
-    <span class="left">
-        {% if doc.prev %}
-          Prev: <a href="{{ doc.prev.link }}">{{ doc.prev.title|safe }}</a>
-        {% endif %}</span><span class="right">
-        {% if doc.next %}
-          Next: <a href="{{ doc.next.link }}">{{ doc.next.title|safe }}</a>
-        {% endif %}</span>
-    <br /><br />
-    {% endif %}
-    » <a href="{{ app.get_absolute_url }}">{{ app.name }} documentation</a>
-    {% for p in doc.parents %}
-    » <a href="{{ p.link }}">{{ p.title|safe }}</a>
-    {% endfor %}
-    » {{ doc.title|safe }}
-    <br /><br />
-    Last update: {{ update_date|date:"Y-m-d H:i" }} (<a href="http://www.timeanddate.com/worldclock/city.html?n=37">CET</a>)
-</div>
-{% endblock content %}
-
-{% block sidebar %}
-    {% block doc_toc %}
-<div class="box">
-    <h2>Contents</h2>
-    {{ doc.toc|safe }}
-</div>
-    {% endblock %}
-<div class="box">
-    <h2>Search</h2>
-    <em>Not yet implemented</em>
-    {# {% load docs %} #}
-    {# {% search_form %} #}
-</div>
-    {{ block.super }}
-{% endblock sidebar %}

=== removed file 'sphinxdoc/templates/sphinxdoc/genindex.html'
--- sphinxdoc/templates/sphinxdoc/genindex.html	2016-06-13 07:14:59 +0000
+++ sphinxdoc/templates/sphinxdoc/genindex.html	1970-01-01 00:00:00 +0000
@@ -1,38 +0,0 @@
-{% extends 'sphinxdoc/documentation.html' %}
-
-{% block doc_body %}
-    <h1>General Index</h1>
-    <p class="indexletters">
-    {% for letter, _ in doc.genindexentries %}
-        <a href="#{{ letter }}">{{ letter }}</a> {% if not forloop.last %} •{% endif %}
-    {% endfor %}
-    </p>
-
-    {% for letter, entries in doc.genindexentries %}
-    <br />    
-    <h2 id="{{ letter }}">{{ letter }}</h2>
-    <dl class="index">
-        {% for name, contents in entries %}
-        <dt>
-          {# contents.0 is a list of links for the item #}
-            {% if contents.0 %}
-            <a href="{{ contents.0.0 }}">{{ name }}</a>
-            {% else %}
-            {{ name }}
-            {% endif %}
-        </dt>
-        {# contents.1 is a list of subitems #}
-            {% if contents.1 %}
-                {% for subname, sublinks in contents.1 %}
-        <dd>
-            <a href="{{ sublinks.0 }}">{{ subname }}</a>
-                    {% for link in sublinks|slice:"1:" %}, <a href="{{ link }}">[Link]</a>{% endfor %}
-        </dd>
-                {% endfor %}
-            {% endif %}
-        {% endfor %}
-    </dl>
-    {% endfor %}
-    <br />
-{% endblock doc_body %}
-{% block doc_toc %}{% endblock %}

=== removed file 'sphinxdoc/templates/sphinxdoc/modindex.html'
--- sphinxdoc/templates/sphinxdoc/modindex.html	2016-06-13 07:14:59 +0000
+++ sphinxdoc/templates/sphinxdoc/modindex.html	1970-01-01 00:00:00 +0000
@@ -1,13 +0,0 @@
-{% extends 'sphinxdoc/documentation.html' %}
-
-{% block doc_body %}
-    <h1>Module Index</h1>
-    <dl>
-    {% for modname, collapse, cgroup, indent, fname, synops, pform, dep in doc.modindexentries %}
-        <dt><a href="{{ fname }}"><tt class="literal xref">{{ modname }}</tt></a></dt>
-        <dd>{{ synops }}</dd>
-    {% endfor %}
-    </dl>
-    <br />
-{% endblock doc_body %}
-{% block doc_toc %}{% endblock %}

=== removed file 'sphinxdoc/templates/sphinxdoc/search_form.html'
--- sphinxdoc/templates/sphinxdoc/search_form.html	2016-06-13 07:14:59 +0000
+++ sphinxdoc/templates/sphinxdoc/search_form.html	1970-01-01 00:00:00 +0000
@@ -1,12 +0,0 @@
-<form action="{{ action|escape }}" id="{{ search_form_id|escape }}" class="search">
-    <div>
-        <input type="hidden" name="cx" value="009763561546736975936:e88ek0eurf4" />
-        <input type="hidden" name="cof" value="FORID:11" />
-        <input type="hidden" name="ie" value="UTF-8" />
-        <input type="hidden" name="hl" value="{{ lang|escape }}" />
-        {{ form.q }}
-        <input type="submit" name="sa" class="submit" value="Search" />
-        {{ form.as_q }}
-    </div>
-</form>
-<script type="text/javascript" src="http://www.google.com/coop/cse/brand?form={{ search_form_id|escape }}&amp;lang={{ lang|escape }}"></script>
\ No newline at end of file

=== modified file 'sphinxdoc/urls.py'
--- sphinxdoc/urls.py	2016-06-13 07:14:59 +0000
+++ sphinxdoc/urls.py	2016-11-22 19:21:51 +0000
@@ -2,7 +2,7 @@
 
 from django.conf.urls import *
 from django.views.generic.list import ListView
-
+from sphinxdoc import views
 from sphinxdoc import models
 
 app_info = {
@@ -11,39 +11,18 @@
 }
 
 
-urlpatterns = patterns('sphinxdoc.views',
-    url(
-        r'^$',
-        ListView.as_view(),
-        app_info,
-    ),
-    url(
-        r'^(?P<slug>[\w-]+)/search/$',
-        'search',
-        name='doc-search',
-    ),
-    url(
-        r'^(?P<slug>[\w-]+)/_images/(?P<path>.*)$',
-        'images',
-    ),
-    url(
-        r'^(?P<slug>[\w-]+)/_source/(?P<path>.*)$',
-        'source',
-    ),
-    url(
-        r'^(?P<slug>[\w-]+)/_objects/$',
-        'objects_inventory',
-        name='objects-inv',
-    ),
-    url(
-        r'^(?P<slug>[\w-]+)/$',
-        'documentation',
-        {'url': ''},
-        name='doc-index',
-    ),
-    url(
-        r'^(?P<slug>[\w-]+)/(?P<url>(([\w-]+)/)+)$',
-        'documentation',
-        name='doc-detail',
-    ),
-)
+urlpatterns = [
+    url(r'^$', ListView.as_view(), app_info,),
+    url(r'^(?P<slug>[\w-]+)/search/$',
+        views.search, name='doc-search', ),
+    url(r'^(?P<slug>[\w-]+)/_images/(?P<path>.*)$',
+        views.images, ),
+    url(r'^(?P<slug>[\w-]+)/_source/(?P<path>.*)$',
+        views.source, ),
+    url(r'^(?P<slug>[\w-]+)/_objects/$',
+        views.objects_inventory, name='objects-inv', ),
+    url(r'^(?P<slug>[\w-]+)/$',
+        views.documentation, {'url': ''}, name='doc-index', ),
+    url(r'^(?P<slug>[\w-]+)/(?P<url>(([\w-]+)/)+)$',
+        views.documentation, name='doc-detail', ),
+]

=== modified file 'sphinxdoc/views.py'
--- sphinxdoc/views.py	2016-08-02 19:22:42 +0000
+++ sphinxdoc/views.py	2016-11-22 19:21:51 +0000
@@ -6,7 +6,6 @@
 from django.http import Http404
 from django.shortcuts import get_object_or_404, render_to_response
 from django.template import RequestContext
-#from django.utils import simplejson as json
 import json
 from django.views import static
 
@@ -52,7 +51,7 @@
     }
     if 'title' not in data['doc']:
         data['doc']['title'] = SPECIAL_TITLES[page_name]
-        
+
     return render_to_response(templates, data,
             context_instance=RequestContext(request))
 

=== added file 'static_sitemap.py'
--- static_sitemap.py	1970-01-01 00:00:00 +0000
+++ static_sitemap.py	2016-11-22 19:21:51 +0000
@@ -0,0 +1,13 @@
+from django.contrib.sitemaps import Sitemap
+from django.core.urlresolvers import reverse
+
+
+class StaticViewSitemap(Sitemap):
+    priority = 0.5
+    changefreq = 'yearly'
+
+    def items(self):
+        return ['mainpage', 'changelog']
+
+    def location(self, item):
+        return reverse(item)

=== modified file 'templates/sphinxdoc/documentation.html'
--- templates/sphinxdoc/documentation.html	2016-07-19 21:36:58 +0000
+++ templates/sphinxdoc/documentation.html	2016-11-22 19:21:51 +0000
@@ -30,7 +30,7 @@
 
 	<div class="sphinx">
 		{% block doc_body %}
-		{{ doc.body|safe }}    
+		{{ doc.body|safe }}
 		{% endblock %}
 	</div>
 

=== modified file 'urls.py'
--- urls.py	2016-11-14 21:05:57 +0000
+++ urls.py	2016-11-22 19:21:51 +0000
@@ -5,17 +5,20 @@
 admin.autodiscover()
 
 from mainpage.views import mainpage
-
 from news.feeds import NewsPostsFeed
 from django.views.generic.base import RedirectView
 from django.contrib.syndication.views import Feed
 from registration.backends.hmac.views import RegistrationView
 from mainpage.forms import RegistrationWithCaptchaForm
 
+
 urlpatterns = [
+    # Creating a sitemap.xml
+    url(r'^sitemap\.xml/', include('sitemap_urls')),
+
     # Uncomment the next line to enable the admin:
     url(r'^admin/', admin.site.urls),
-
+    
     # Django builtin / Registration
     # overwrite registration with own implementation
     url (r'^accounts/register/$', RegistrationView.as_view(form_class=RegistrationWithCaptchaForm), name='registration_register'),

=== added file 'wiki/sitemap.py'
--- wiki/sitemap.py	1970-01-01 00:00:00 +0000
+++ wiki/sitemap.py	2016-11-22 19:21:51 +0000
@@ -0,0 +1,13 @@
+from django.contrib.sitemaps import Sitemap
+from wiki.models import Article
+
+
+class WikiSitemap(Sitemap):
+    changefreq = 'yearly'
+    priority = 0.5
+
+    def items(self):
+        return Article.objects.all()
+
+    def lastmod(self, obj):
+        return obj.last_update

=== added file 'wlhelp/sitemap.py'
--- wlhelp/sitemap.py	1970-01-01 00:00:00 +0000
+++ wlhelp/sitemap.py	2016-11-22 19:21:51 +0000
@@ -0,0 +1,46 @@
+from django.contrib.sitemaps import Sitemap
+from wlhelp.models import Tribe, Building, Ware, Worker
+
+
+class WlHelpTribeSitemap(Sitemap):
+    changefreq = 'yearly'
+    priority = 0.5
+
+    def items(self):
+        return Tribe.objects.all()
+
+    def location(self, obj):
+        return '/encyclopedia/%s' % obj.name
+
+
+class WlHelpBuildingSitemap(Sitemap):
+    changefreq = 'yearly'
+    priority = 0.5
+
+    def items(self):
+        return Building.objects.all()
+
+    def location(self, obj):
+        return '/encyclopedia/%s/buildings/%s' % (obj.tribe.name, obj.name)
+
+
+class WlHelpWareSitemap(Sitemap):
+    changefreq = 'yearly'
+    priority = 0.5
+
+    def items(self):
+        return Ware.objects.all()
+
+    def location(self, obj):
+        return '/encyclopedia/%s/wares/%s' % (obj.tribe.name, obj.name)
+
+
+class WlHelpWorkerSitemap(Sitemap):
+    changefreq = 'yearly'
+    priority = 0.5
+
+    def items(self):
+        return Worker.objects.all()
+
+    def location(self, obj):
+        return '/encyclopedia/%s/workers/%s' % (obj.tribe.name, obj.name)