Merge lp:~widelands-dev/widelands-website/wlwebsite_py36 into lp:widelands-website

[kaputtnik]

Adapted code for use with python3.6

Changes need to get the website running with python 3.6. Because python2.7 is EOL in year 2020 i did not spend any time to make the code python2.7 compatible. So installing the website needs a virtualenvironment created with python 3.6.

Main changes (maybe interesting parts are linked):
 - after running the python 2to3 script: https://bazaar.launchpad.net/~widelands-dev/widelands-website/wlwebsite_py36/revision/533
 - changes regarding to django (mostly replace __unicode__ with __str__)
 - changes regarding str objects, which are bytes objects in python3
 - same for replacing StringIo -> BytesIO when working with images
 - added a python3 compatible version of diff_match_patch.py and applied individual coloring like it is right now
 - fixed template filter 'minutes', renamed it to 'elapsed_time' and applied output of day(s), https://bazaar.launchpad.net/~widelands-dev/widelands-website/wlwebsite_py36/revision/550
 - the url /locale/ shows now also if the environment variable DISPLAY is set
 - after installing on the new server the file README got the needed changes

The added text file 'python3.txt' contain my test results and can be removed if we finally merge this changes into trunk.

I want to merge this before i set up the productive website on the new server.

[lp:~widelands-dev/widelands-website/wlwebsite_py36]

562. By kaputtnik on 2019-06-13

fixed a failure for queued notification mails

563. By kaputtnik on 2019-06-14

use base64 instead of codecs

[GunChleoc (gunchleoc)]

Let's have it :)

We should document the test protocol in the Python3.txt file on our wiki too - it will come in useful for any big website changes that need testing on alpha.

review: Approve

[kaputtnik (franku)]

Thanks :-)

=== modified file 'README.txt'
--- README.txt	2019-04-08 05:55:07 +0000
+++ README.txt	2019-06-14 07:22:45 +0000
@@ -1,6 +1,14 @@
 Installing the homepage
 =======================
 
+Used python version
+-------------------
+The website is tested with python 3.6. This README reflects setting up the
+website with this python version.
+
+Install prerequisites
+---------------------
+
 Getting the homepage to run locally is best supported using virtualenv and
 pip. Install those two tools first, either via easy_install or via your local
 package manager. You will also need development tools (gcc or therelike), hg
@@ -11,16 +19,8 @@
 Example:
 On Ubuntu, installing all required tools and dependencies in two commands:
 
-   $ sudo apt-get install python-dev python-virtualenv python-pip mercurial bzr subversion git-core sqlite3
-   $ sudo apt-get build-dep python-numpy
-
-Used python version
--------------------
-
-Currently, the website depends on python 2.7. In case you have python 3 as default (like on arch-linux),
-you have to adjust the python relevant commands to use python 2.7. E.g. 'virtualenv2 wlwebsite' creates
-a virtualenvironment using python 2.7. If the virtualenvironment is activated, python 2.7 will become
-standard for executing python code in this shell.
+   $ sudo apt-get install python3-virtualenv python3-pip bzr libmysqlclient-dev
+   $ sudo apt-get build-dep python3-numpy
 
 Setting up the local environment
 --------------------------------
@@ -33,7 +33,7 @@
 packages from your global site packages. Very important!
 Now, we create and activate our environment:
 
-   $ virtualenv wlwebsite
+   $ virtualenv --python=python3.6 wlwebsite
    $ cd wlwebsite
    $ source bin/activate
 

=== modified file 'check_input/migrations/0001_initial.py'
--- check_input/migrations/0001_initial.py	2017-11-24 10:55:59 +0000
+++ check_input/migrations/0001_initial.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.db import models, migrations
 from django.conf import settings

=== modified file 'check_input/models.py'
--- check_input/models.py	2018-04-26 21:10:19 +0000
+++ check_input/models.py	2019-06-14 07:22:45 +0000
@@ -31,7 +31,7 @@
         ordering = ['content_type_id']
         default_permissions = ('change', 'delete',)
 
-    def __unicode__(self):
+    def __str__(self):
         return self.text
 
     def clean(self):

=== modified file 'documentation/conf.py'
--- documentation/conf.py	2018-11-27 17:01:42 +0000
+++ documentation/conf.py	2019-06-14 07:22:45 +0000
@@ -41,8 +41,8 @@
 master_doc = 'index'
 
 # General information about the project.
-project = u'Widelands'
-copyright = u'The Widelands Development Team'
+project = 'Widelands'
+copyright = 'The Widelands Development Team'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the

=== modified file 'documentation/management/commands/create_docs.py'
--- documentation/management/commands/create_docs.py	2018-05-13 09:05:21 +0000
+++ documentation/management/commands/create_docs.py	2019-06-14 07:22:45 +0000
@@ -10,7 +10,7 @@
 
 """
 
-from __future__ import print_function
+
 from django.core.management.base import BaseCommand, CommandError
 from django.conf import settings
 from subprocess import check_call, CalledProcessError

=== modified file 'local_settings.py.sample'
--- local_settings.py.sample	2019-04-04 06:47:19 +0000
+++ local_settings.py.sample	2019-06-14 07:22:45 +0000
@@ -3,8 +3,6 @@
 # The above leads python2.7 to read this file as utf-8
 # Needs to be directly after the python shebang
 
-# Treat all strings as unicode
-from __future__ import unicode_literals
 import os
 import re
 

=== modified file 'mainpage/admin.py'
--- mainpage/admin.py	2019-01-18 11:10:27 +0000
+++ mainpage/admin.py	2019-06-14 07:22:45 +0000
@@ -28,7 +28,7 @@
         text += ['has perm.']
     value = ', '.join(text)
     return value
-roles.short_description = u'Groups/Permissions'
+roles.short_description = 'Groups/Permissions'
 
 
 def persons(self):
@@ -38,7 +38,7 @@
 
 def deleted(self):
     return '' if self.wlprofile.deleted==False else 'Yes'
-deleted.short_description = u'Deleted himself'
+deleted.short_description = 'Deleted himself'
 
 
 class GroupAdmin(GroupAdmin):

=== modified file 'mainpage/online_users_middleware.py'
--- mainpage/online_users_middleware.py	2019-03-31 11:08:21 +0000
+++ mainpage/online_users_middleware.py	2019-06-14 07:22:45 +0000
@@ -36,7 +36,7 @@
 
         # Perform the multiget on the individual online uid keys
         online_keys = ['online-%s' % (u,) for u in uids]
-        fresh = cache.get_many(online_keys).keys()
+        fresh = list(cache.get_many(online_keys).keys())
         online_now_ids = [int(k.replace('online-', '')) for k in fresh]
 
         # If the user is authenticated, add their id to the list

=== modified file 'mainpage/settings.py'
--- mainpage/settings.py	2019-04-07 10:02:31 +0000
+++ mainpage/settings.py	2019-06-14 07:22:45 +0000
@@ -288,20 +288,20 @@
 ## Allowed tags/attributes for 'bleach' ##
 ## Used for sanitizing user input.      ##
 ##########################################
-BLEACH_ALLOWED_TAGS = [u'a',
-                       u'abbr',
-                       u'acronym',
-                       u'blockquote',
-                       u'br',
-                       u'em',  u'i',  u'strong', u'b',
-                       u'ul',  u'ol', u'li',
-                       u'div', u'p',
-                       u'h1',  u'h2', u'h3', u'h4', u'h5', u'h6',
-                       u'pre', u'code',
-                       u'img',
-                       u'hr',
-                       u'table', u'tbody', u'thead', u'th', u'tr', u'td',
-                       u'sup',
+BLEACH_ALLOWED_TAGS = ['a',
+                       'abbr',
+                       'acronym',
+                       'blockquote',
+                       'br',
+                       'em',  'i',  'strong', 'b',
+                       'ul',  'ol', 'li',
+                       'div', 'p',
+                       'h1',  'h2', 'h3', 'h4', 'h5', 'h6',
+                       'pre', 'code',
+                       'img',
+                       'hr',
+                       'table', 'tbody', 'thead', 'th', 'tr', 'td',
+                       'sup',
                        ]
 
 BLEACH_ALLOWED_ATTRIBUTES = {'img': ['src', 'alt'], 'a': [
@@ -368,6 +368,6 @@
 
 
 try:
-    from local_settings import *
+    from .local_settings import *
 except ImportError:
     pass

=== modified file 'mainpage/sitemap_urls.py'
--- mainpage/sitemap_urls.py	2019-03-31 11:08:21 +0000
+++ mainpage/sitemap_urls.py	2019-06-14 07:22:45 +0000
@@ -1,7 +1,7 @@
 from django.conf.urls import url
 
 from django.contrib.sitemaps.views import sitemap
-from static_sitemap import StaticViewSitemap
+from .static_sitemap import StaticViewSitemap
 from wiki.sitemap import *
 from news.sitemap import *
 from pybb.sitemap import *

=== modified file 'mainpage/templatetags/wl_markdown.py'
--- mainpage/templatetags/wl_markdown.py	2019-03-31 11:08:21 +0000
+++ mainpage/templatetags/wl_markdown.py	2019-06-14 07:22:45 +0000
@@ -11,7 +11,7 @@
 
 from django import template
 from django.conf import settings
-from django.utils.encoding import smart_str, force_unicode
+from django.utils.encoding import smart_bytes, force_text
 from django.utils.safestring import mark_safe
 from django.conf import settings
 from markdownextensions.semanticwikilinks.mdx_semanticwikilinks import SemanticWikiLinkExtension
@@ -19,10 +19,10 @@
 # Try to get a not so fully broken markdown module
 import markdown
 if markdown.version_info[0] < 2:
-    raise ImportError, 'Markdown library to old!'
+    raise ImportError('Markdown library to old!')
 from markdown import markdown
 import re
-import urllib
+import urllib.request, urllib.parse, urllib.error
 import bleach
 
 from bs4 import BeautifulSoup, NavigableString
@@ -138,7 +138,7 @@
 
         # Check for missing wikilink /wiki/PageName[/additionl/stuff]
         # Using href because we need cAsEs here
-        article_name = urllib.unquote(tag['href'][6:].split('/', 1)[0])
+        article_name = urllib.parse.unquote(tag['href'][6:].split('/', 1)[0])
 
         if not len(article_name):  # Wiki root link is not a page
             tag['class'] = 'wrongLink'
@@ -159,7 +159,7 @@
             # get actual title of article
             act_t = Article.objects.get(id=a_id[0]).title
             if article_name != act_t:
-                tag['title'] = 'This is a redirect and points to \"" + act_t + "\"'
+                tag['title'] = 'This is a redirect and points to \"' + act_t + '\"'
                 return
             else:
                 return
@@ -218,7 +218,7 @@
     """Apply wl specific things, like smileys or colored links."""
 
     beautify = keyw.pop('beautify', True)
-    html = smart_str(markdown(value, extensions=md_extensions))
+    html = markdown(value, extensions=md_extensions)
 
     # Sanitize posts from potencial untrusted users (Forum/Wiki/Maps)
     if 'bleachit' in args:
@@ -232,7 +232,7 @@
     soup = BeautifulSoup(html, features='lxml')
     if len(soup.contents) == 0:
         # well, empty soup. Return it
-        return unicode(soup)
+        return str(soup)
 
     if beautify:
         # Insert smileys
@@ -249,7 +249,7 @@
         for tag in soup.find_all('img'):
             _make_clickable_images(tag)
 
-    return unicode(soup)
+    return str(soup)
 
 
 @register.filter

=== modified file 'mainpage/urls.py'
--- mainpage/urls.py	2019-03-31 11:08:21 +0000
+++ mainpage/urls.py	2019-06-14 07:22:45 +0000
@@ -63,7 +63,7 @@
 ]
 
 try:
-    from local_urls import *
+    from .local_urls import *
     urlpatterns += local_urlpatterns
 except ImportError:
     pass

=== modified file 'mainpage/utest/test_wl_markdown.py'
--- mainpage/utest/test_wl_markdown.py	2019-03-31 11:08:21 +0000
+++ mainpage/utest/test_wl_markdown.py	2019-06-14 07:22:45 +0000
@@ -37,222 +37,222 @@
         self.assertEqual(wanted, res)
 
     def test_simple_case__correct_result(self):
-        input = u"Hallo Welt"
-        wanted = u"<p>Hallo Welt</p>"
+        input = "Hallo Welt"
+        wanted = "<p>Hallo Welt</p>"
         self._check(input, wanted)
 
     def test_wikiwords_simple__except_correct_result(self):
-        input = u"Na Du HalloWelt, Du?"
-        wanted = u"""<p>Na Du <a href="/wiki/HalloWelt">HalloWelt</a>, Du?</p>"""
+        input = "Na Du HalloWelt, Du?"
+        wanted = """<p>Na Du <a href="/wiki/HalloWelt">HalloWelt</a>, Du?</p>"""
         self._check(input, wanted)
 
     def test_wikiwords_avoid__except_correct_result(self):
-        input = u"Hi !NotAWikiWord Moretext"
-        wanted = u"""<p>Hi NotAWikiWord Moretext</p>"""
+        input = "Hi !NotAWikiWord Moretext"
+        wanted = """<p>Hi NotAWikiWord Moretext</p>"""
         self._check(input, wanted)
 
     def test_wikiwords_in_link__except_correct_result(self):
-        input = u"""WikiWord [NoWikiWord](/forum/)"""
-        wanted = u"""<p><a href="/wiki/WikiWord">WikiWord</a> <a href="/forum/">NoWikiWord</a></p>"""
+        input = """WikiWord [NoWikiWord](/forum/)"""
+        wanted = """<p><a href="/wiki/WikiWord">WikiWord</a> <a href="/forum/">NoWikiWord</a></p>"""
         self._check(input, wanted)
 
     def test_wikiwords_external_links__except_correct_result(self):
-        input = u"""[NoWikiWord](http://www.sun.com)"""
-        wanted = u"""<p><a href="http://www.sun.com" class="external">NoWikiWord</a></p>"""
+        input = """[NoWikiWord](http://www.sun.com)"""
+        wanted = """<p><a href="http://www.sun.com" class="external">NoWikiWord</a></p>"""
         self._check(input, wanted)
 
     def test_wikiwords_noexternal_links__except_correct_result(self):
-        input = u"""[NoWikiWord](http://%s/blahfasel/wiki)""" % _domain
-        wanted = u"""<p><a href="http://%s/blahfasel/wiki">NoWikiWord</a></p>""" % _domain
+        input = """[NoWikiWord](http://%s/blahfasel/wiki)""" % _domain
+        wanted = """<p><a href="http://%s/blahfasel/wiki">NoWikiWord</a></p>""" % _domain
         self._check(input, wanted)
 
     def test_wikiwords_noclasschangeforimage_links__except_correct_result(self):
-        input =  u"""<a href="http://www.ccc.de"><img src="/blub" /></a>"""
-        wanted = u"""<p><a href="http://www.ccc.de"><img src="/blub" /></a></p>"""
+        input =  """<a href="http://www.ccc.de"><img src="/blub" /></a>"""
+        wanted = """<p><a href="http://www.ccc.de"><img src="/blub" /></a></p>"""
         self._check(input, wanted)
 
     # Existing links
     def test_existing_link_html(self):
-        input = u"""<a href="/wiki/MainPage">this page</a>"""
-        wanted = u"""<p><a href="/wiki/MainPage">this page</a></p>"""
+        input = """<a href="/wiki/MainPage">this page</a>"""
+        wanted = """<p><a href="/wiki/MainPage">this page</a></p>"""
         self._check(input, wanted)
 
     def test_existing_link_markdown(self):
-        input = u"""[this page](/wiki/MainPage)"""
-        wanted = u"""<p><a href="/wiki/MainPage">this page</a></p>"""
+        input = """[this page](/wiki/MainPage)"""
+        wanted = """<p><a href="/wiki/MainPage">this page</a></p>"""
         self._check(input, wanted)
 
     def test_existing_link_wikiword(self):
-        input = u"""MainPage"""
-        wanted = u"""<p><a href="/wiki/MainPage">MainPage</a></p>"""
+        input = """MainPage"""
+        wanted = """<p><a href="/wiki/MainPage">MainPage</a></p>"""
         self._check(input, wanted)
 
     def test_existing_editlink_wikiword(self):
-        input = u"""<a href="/wiki/MainPage/edit/">this page</a>"""
-        wanted = u"""<p><a href="/wiki/MainPage/edit/">this page</a></p>"""
+        input = """<a href="/wiki/MainPage/edit/">this page</a>"""
+        wanted = """<p><a href="/wiki/MainPage/edit/">this page</a></p>"""
         self._check(input, wanted)
 
     # Missing links
     def test_missing_link_html(self):
-        input = u"""<a href="/wiki/MissingPage">this page</a>"""
-        wanted = u"""<p><a href="/wiki/MissingPage" class="missing">this page</a></p>"""
+        input = """<a href="/wiki/MissingPage">this page</a>"""
+        wanted = """<p><a href="/wiki/MissingPage" class="missing">this page</a></p>"""
         self._check(input, wanted)
 
     def test_missing_link_markdown(self):
-        input = u"""[this page](/wiki/MissingPage)"""
-        wanted = u"""<p><a href="/wiki/MissingPage" class="missing">this page</a></p>"""
+        input = """[this page](/wiki/MissingPage)"""
+        wanted = """<p><a href="/wiki/MissingPage" class="missing">this page</a></p>"""
         self._check(input, wanted)
 
     def test_missing_link_wikiword(self):
-        input = u"""BlubMissingPage"""
-        wanted = u"""<p><a href="/wiki/BlubMissingPage" class="missing">BlubMissingPage</a></p>"""
+        input = """BlubMissingPage"""
+        wanted = """<p><a href="/wiki/BlubMissingPage" class="missing">BlubMissingPage</a></p>"""
         res = do_wl_markdown(input)
         # self._check(input,wanted)
 
     def test_missing_editlink_wikiword(self):
-        input = u"""<a href="/wiki/MissingPage/edit/">this page</a>"""
-        wanted = u"""<p><a href="/wiki/MissingPage/edit/" class="missing">this page</a></p>"""
+        input = """<a href="/wiki/MissingPage/edit/">this page</a>"""
+        wanted = """<p><a href="/wiki/MissingPage/edit/" class="missing">this page</a></p>"""
         self._check(input, wanted)
 
     # Check smileys
     def test_smiley_angel(self):
         input = """O:-)"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-angel.png" alt="face-angel.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-angel.png" alt="face-angel.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_crying(self):
         input = """:'-("""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-crying.png" alt="face-crying.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-crying.png" alt="face-crying.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_devilish(self):
         input = """>:-)"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-devilish.png" alt="face-devilish.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-devilish.png" alt="face-devilish.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_glasses(self):
         input = """8-)"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-glasses.png" alt="face-glasses.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-glasses.png" alt="face-glasses.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_kiss(self):
         input = """:-x"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-kiss.png" alt="face-kiss.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-kiss.png" alt="face-kiss.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_plain(self):
         input = """:-|"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-plain.png" alt="face-plain.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-plain.png" alt="face-plain.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_sad(self):
         input = """:-("""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-sad.png" alt="face-sad.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-sad.png" alt="face-sad.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_smilebig(self):
         input = """:))"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-smile-big.png" alt="face-smile-big.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-smile-big.png" alt="face-smile-big.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_smile(self):
         input = """:-)"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-smile.png" alt="face-smile.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-smile.png" alt="face-smile.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_surprise(self):
         input = """:-O"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-surprise.png" alt="face-surprise.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-surprise.png" alt="face-surprise.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_wink(self):
         input = """;-)"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-wink.png" alt="face-wink.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-wink.png" alt="face-wink.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_grin(self):
         input = """:D"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-grin.png" alt="face-grin.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-grin.png" alt="face-grin.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_sad(self):
         input = """:("""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-sad.png" alt="face-sad.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-sad.png" alt="face-sad.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_smile(self):
         input = """:)"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-smile.png" alt="face-smile.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-smile.png" alt="face-smile.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_surprise(self):
         input = """:O"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-surprise.png" alt="face-surprise.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-surprise.png" alt="face-surprise.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_wink(self):
         input = """;)"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-wink.png" alt="face-wink.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-wink.png" alt="face-wink.png" /></p>"""
         self._check(input, wanted)
 
     def test_smiley_monkey(self):
         input = """:(|)"""
-        wanted = u"""<p><img src="/wlmedia/img/smileys/face-monkey.png" alt="face-monkey.png" /></p>"""
+        wanted = """<p><img src="/wlmedia/img/smileys/face-monkey.png" alt="face-monkey.png" /></p>"""
         self._check(input, wanted)
 
     # Occured errors
     def test_wiki_rootlink(self):
-        input = u"""<a href="/wiki">this page</a>"""
-        wanted = u"""<p><a href="/wiki">this page</a></p>"""
+        input = """<a href="/wiki">this page</a>"""
+        wanted = """<p><a href="/wiki">this page</a></p>"""
         self._check(input, wanted)
 
     def test_wiki_rootlink_with_slash(self):
-        input = u"""<a href="/wiki/">this page</a>"""
-        wanted = u"""<p><a href="/wiki/">this page</a></p>"""
+        input = """<a href="/wiki/">this page</a>"""
+        wanted = """<p><a href="/wiki/">this page</a></p>"""
         self._check(input, wanted)
 
     # Special pages
     def test_wiki_specialpage(self):
-        input = u"""<a href="/wiki/list">this page</a>"""
-        wanted = u"""<p><a href="/wiki/list">this page</a></p>"""
+        input = """<a href="/wiki/list">this page</a>"""
+        wanted = """<p><a href="/wiki/list">this page</a></p>"""
         self._check(input, wanted)
 
     def test_wiki_specialpage_markdown(self):
-        input = u"""[list](/wiki/list)"""
-        wanted = u"""<p><a href="/wiki/list">list</a></p>"""
+        input = """[list](/wiki/list)"""
+        wanted = """<p><a href="/wiki/list">list</a></p>"""
         self._check(input, wanted)
 
     # Special problem with emphasis
     def test_markdown_emphasis_problem(self):
-        input = u"""*This is bold*  _This too_\n\n"""
-        wanted = u"""<p><em>This is bold</em> <em>This too</em></p>"""
+        input = """*This is bold*  _This too_\n\n"""
+        wanted = """<p><em>This is bold</em> <em>This too</em></p>"""
         self._check(input, wanted)
 
     # Another markdown problem with alt tag escaping
     def test_markdown_alt_problem(self):
         # {{{ Test strings
-        input = u"""![img_thisisNOTitalicplease_name.png](/wlmedia/blah.png)\n\n"""
-        wanted = u'<p><img alt="img_thisisNOTitalicplease_name.png" src="/wlmedia/blah.png" /></p>'
+        input = """![img_thisisNOTitalicplease_name.png](/wlmedia/blah.png)\n\n"""
+        wanted = '<p><img alt="img_thisisNOTitalicplease_name.png" src="/wlmedia/blah.png" /></p>'
         # }}}
         self._check(input, wanted)
 
     def test_emptystring_problem(self):
         # {{{ Test strings
-        input = u''
-        wanted = u''
+        input = ''
+        wanted = ''
         # }}}
         self._check(input, wanted)
 
     # Damned problem with tables
     def test_markdown_table_problem(self):
         # {{{ Test strings
-        input = u"""
+        input = """
 Header1 | Header 2
 ------- | --------
 Value 1 | Value 2
 Value 3 | Value 4
 """
-        wanted = u"""<table>
+        wanted = """<table>
 <thead>
 <tr>
 <th>Header1</th>
@@ -274,13 +274,13 @@
         self._check(input, wanted)
 
     def test_svnrevision_replacement(self):
-        input = u"- Fixed this bug (bzr:r3222)"
-        wanted = u"""<ul>\n<li>Fixed this bug (<a href="http://bazaar.launchpad.net/%7Ewidelands-dev/widelands/trunk/revision/3222" class="external">r3222</a>)</li>\n</ul>"""
+        input = "- Fixed this bug (bzr:r3222)"
+        wanted = """<ul>\n<li>Fixed this bug (<a href="http://bazaar.launchpad.net/%7Ewidelands-dev/widelands/trunk/revision/3222" class="external">r3222</a>)</li>\n</ul>"""
         self._check(input, wanted)
 
     def test_svnrevision_multiple_replacement(self):
-        input = u"- Fixed this bug (bzr:r3222, bzr:r3424)"
-        wanted = u"""<ul>\n<li>Fixed this bug (<a href="http://bazaar.launchpad.net/%7Ewidelands-dev/widelands/trunk/revision/3222" class="external">r3222</a>, <a href="http://bazaar.launchpad.net/%7Ewidelands-dev/widelands/trunk/revision/3424" class="external">r3424</a>)</li>\n</ul>"""
+        input = "- Fixed this bug (bzr:r3222, bzr:r3424)"
+        wanted = """<ul>\n<li>Fixed this bug (<a href="http://bazaar.launchpad.net/%7Ewidelands-dev/widelands/trunk/revision/3222" class="external">r3222</a>, <a href="http://bazaar.launchpad.net/%7Ewidelands-dev/widelands/trunk/revision/3424" class="external">r3424</a>)</li>\n</ul>"""
         self._check(input, wanted)
 
 

=== modified file 'mainpage/views.py'
--- mainpage/views.py	2019-03-31 11:08:21 +0000
+++ mainpage/views.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 from django.conf import settings
-from templatetags.wl_markdown import do_wl_markdown
+from .templatetags.wl_markdown import do_wl_markdown
 from operator import itemgetter
 from django.core.mail import send_mail
 from mainpage.forms import ContactForm
@@ -119,12 +119,12 @@
 
                 # Add a subheader or/and the member(s)
                 for entry in head['entries']:
-                    if 'subheading' in entry.keys():
+                    if 'subheading' in list(entry.keys()):
                         txt = txt + '###' + entry['subheading'] + '\n'
-                    if 'members' in entry.keys():
+                    if 'members' in list(entry.keys()):
                         for name in entry['members']:
                             txt = txt + '* ' + name + '\n'
-                    if 'translate' in entry.keys():
+                    if 'translate' in list(entry.keys()):
                         for transl in entry['translate']:
                             txt = txt + '* ' + transl + '\n'
 
@@ -160,5 +160,7 @@
     loc_info = 'getlocale: ' + str(locale.getlocale()) + \
         '<br/>getdefaultlocale(): ' + str(locale.getdefaultlocale()) + \
         '<br/>fs_encoding: ' + str(sys.getfilesystemencoding()) + \
-        '<br/>sys default encoding: ' + str(sys.getdefaultencoding())
+        '<br/>sys default encoding: ' + str(sys.getdefaultencoding()) + \
+        '<br><br> Environment variables:' + \
+        '<br>DISPLAY: ' + os.environ.get('DISPLAY', 'Not set')
     return HttpResponse(loc_info)

=== modified file 'mainpage/wlwebsite_wsgi.py'
--- mainpage/wlwebsite_wsgi.py	2019-03-31 11:08:21 +0000
+++ mainpage/wlwebsite_wsgi.py	2019-06-14 07:22:45 +0000
@@ -16,7 +16,7 @@
 if activate_this is None:
     raise RuntimeException('Could not find virtualenv to start up!')
 
-execfile(activate_this, dict(__file__=activate_this))
+exec(compile(open(activate_this, "rb").read(), activate_this, 'exec'), dict(__file__=activate_this))
 
 sys.path.append(parent_dir(code_directory))
 sys.path.append(code_directory)

=== modified file 'news/migrations/0001_initial.py'
--- news/migrations/0001_initial.py	2016-12-13 18:28:51 +0000
+++ news/migrations/0001_initial.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.db import models, migrations
 from django.conf import settings

=== modified file 'news/migrations/0002_auto_20170417_1857.py'
--- news/migrations/0002_auto_20170417_1857.py	2017-04-17 17:08:27 +0000
+++ news/migrations/0002_auto_20170417_1857.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.db import models, migrations
 

=== modified file 'news/models.py'
--- news/models.py	2019-03-31 11:08:21 +0000
+++ news/models.py	2019-06-14 07:22:45 +0000
@@ -29,8 +29,8 @@
         db_table = 'news_categories'
         ordering = ('title',)
 
-    def __unicode__(self):
-        return u'%s' % self.title
+    def __str__(self):
+        return '%s' % self.title
 
     def get_absolute_url(self):
         return reverse('category_posts', args=(self.slug,))
@@ -65,8 +65,8 @@
         ordering = ('-publish',)
         get_latest_by = 'publish'
 
-    def __unicode__(self):
-        return u'%s' % self.title
+    def __str__(self):
+        return '%s' % self.title
 
     #########
     # IMAGE #

=== modified file 'news/templatetags/news_extras.py'
--- news/templatetags/news_extras.py	2016-12-13 18:28:51 +0000
+++ news/templatetags/news_extras.py	2019-06-14 07:22:45 +0000
@@ -42,9 +42,9 @@
     try:
         tag_name, arg = token.contents.split(None, 1)
     except ValueError:
-        raise template.TemplateSyntaxError, '%s tag requires arguments' % token.contents.split()[0]
+        raise template.TemplateSyntaxError('%s tag requires arguments' % token.contents.split()[0])
     m = re.search(r'(.*?) as (\w+)', arg)
     if not m:
-        raise template.TemplateSyntaxError, '%s tag had invalid arguments' % tag_name
+        raise template.TemplateSyntaxError('%s tag had invalid arguments' % tag_name)
     format_string, var_name = m.groups()
     return LatestPosts(format_string, var_name)

=== modified file 'notification/engine.py'
--- notification/engine.py	2017-04-29 19:52:28 +0000
+++ notification/engine.py	2019-06-14 07:22:45 +0000
@@ -3,9 +3,10 @@
 import time
 import logging
 import traceback
+import base64
 
 try:
-    import cPickle as pickle
+    import pickle as pickle
 except ImportError:
     import pickle
 
@@ -14,7 +15,7 @@
 from django.contrib.auth.models import User
 from django.contrib.sites.models import Site
 
-from lockfile import FileLock, AlreadyLocked, LockTimeout
+from .lockfile import FileLock, AlreadyLocked, LockTimeout
 
 from notification.models import NoticeQueueBatch
 from notification import models as notification
@@ -46,7 +47,8 @@
         try:
             for queued_batch in NoticeQueueBatch.objects.all():
                 notices = pickle.loads(
-                    str(queued_batch.pickled_data).decode('base64'))
+                    base64.b64decode(queued_batch.pickled_data)
+                    )
                 for user, label, extra_context, on_site in notices:
                     user = User.objects.get(pk=user)
                     # FrankU: commented, because not all users get e-mailed

=== modified file 'notification/lockfile.py'
--- notification/lockfile.py	2016-12-13 18:28:51 +0000
+++ notification/lockfile.py	2019-06-14 07:22:45 +0000
@@ -48,7 +48,7 @@
             NotMyLock - File was locked but not by the current thread/process
 """
 
-from __future__ import division
+
 
 import sys
 import socket
@@ -385,8 +385,8 @@
 
     def __init__(self, path, threaded=True):
         LockBase.__init__(self, path, threaded)
-        self.lock_file = unicode(self.lock_file)
-        self.unique_name = unicode(self.unique_name)
+        self.lock_file = str(self.lock_file)
+        self.unique_name = str(self.unique_name)
 
         import sqlite3
         self.connection = sqlite3.connect(SQLiteFileLock.testdb)

=== modified file 'notification/migrations/0001_initial.py'
--- notification/migrations/0001_initial.py	2016-12-13 18:28:51 +0000
+++ notification/migrations/0001_initial.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.db import models, migrations
 import datetime

=== modified file 'notification/migrations/0002_auto_20170417_1857.py'
--- notification/migrations/0002_auto_20170417_1857.py	2017-04-17 17:08:27 +0000
+++ notification/migrations/0002_auto_20170417_1857.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.db import models, migrations
 

=== modified file 'notification/migrations/0003_auto_20190409_0924.py'
--- notification/migrations/0003_auto_20190409_0924.py	2019-04-10 16:01:43 +0000
+++ notification/migrations/0003_auto_20190409_0924.py	2019-06-14 07:22:45 +0000
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 # Generated by Django 1.11.20 on 2019-04-09 09:24
-from __future__ import unicode_literals
+
 
 from django.db import migrations, models
 

=== modified file 'notification/models.py'
--- notification/models.py	2018-04-19 17:48:31 +0000
+++ notification/models.py	2019-06-14 07:22:45 +0000
@@ -1,7 +1,8 @@
 import datetime
+import base64
 
 try:
-    import cPickle as pickle
+    import pickle as pickle
 except ImportError:
     import pickle
 
@@ -48,7 +49,7 @@
     # number
     default = models.IntegerField(_('default'))
 
-    def __unicode__(self):
+    def __str__(self):
         return self.label
 
     class Meta:
@@ -150,12 +151,12 @@
         if updated:
             notice_type.save()
             if verbosity > 1:
-                print 'Updated %s NoticeType' % label
+                print('Updated %s NoticeType' % label)
     except NoticeType.DoesNotExist:
         NoticeType(label=label, display=display,
                    description=description, default=default).save()
         if verbosity > 1:
-            print 'Created %s NoticeType' % label
+            print('Created %s NoticeType' % label)
 
 
 def get_notification_language(user):
@@ -222,8 +223,8 @@
         notice_type = NoticeType.objects.get(label=label)
 
         current_site = Site.objects.get_current()
-        notices_url = u"http://%s%s" % (
-            unicode(current_site),
+        notices_url = "http://%s%s" % (
+            str(current_site),
             reverse('notification_notices'),
         )
 
@@ -257,12 +258,12 @@
 
             # get prerendered format messages and subjects
             messages = get_formatted_messages(formats, label, context)
-            
+
             # Create the subject
             # Use 'email_subject.txt' to add Strings in every emails subject
             subject = render_to_string('notification/email_subject.txt',
                                        {'message': messages['short.txt'],}).replace('\n', '')
-            
+
             # Strip leading newlines. Make writing the email templates easier:
             # Each linebreak in the templates results in a linebreak in the emails
             # If the first line in a template contains only template tags the 
@@ -275,8 +276,8 @@
             if should_send(user, notice_type, '1') and user.email:  # Email
                 recipients.append(user.email)
 
-            send_mail(subject, body, settings.DEFAULT_FROM_EMAIL, recipients)
-        
+            send_mail(subject, body, settings.DEFAULT_FROM_EMAIL, recipients, fail_silently=True)
+
         # reset environment to original language
         activate(current_language)
     except NoticeType.DoesNotExist:
@@ -322,8 +323,9 @@
     notices = []
     for user in users:
         notices.append((user, label, extra_context, on_site))
-    NoticeQueueBatch(pickled_data=pickle.dumps(
-        notices).encode('base64')).save()
+
+    NoticeQueueBatch(pickled_data=base64.b64encode(
+    	pickle.dumps(notices))).save()
 
 
 class ObservedItemManager(models.Manager):

=== modified file 'notification/views.py'
--- notification/views.py	2018-04-03 05:18:03 +0000
+++ notification/views.py	2019-06-14 07:22:45 +0000
@@ -29,5 +29,5 @@
 
     return render(request, 'notification/notice_settings.html', {
         'column_headers': [medium_display for medium_id, medium_display in NOTICE_MEDIA],
-        'app_tables': OrderedDict(sorted(app_tables.items(), key=lambda t: t[0]))
+        'app_tables': OrderedDict(sorted(list(app_tables.items()), key=lambda t: t[0]))
     })

=== modified file 'privacy_policy/admin.py'
--- privacy_policy/admin.py	2018-10-08 17:51:36 +0000
+++ privacy_policy/admin.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.contrib import admin
 from privacy_policy.models import PrivacyPolicy

=== modified file 'privacy_policy/apps.py'
--- privacy_policy/apps.py	2018-10-02 16:55:42 +0000
+++ privacy_policy/apps.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.apps import AppConfig
 

=== modified file 'privacy_policy/migrations/0001_initial.py'
--- privacy_policy/migrations/0001_initial.py	2018-10-08 17:51:36 +0000
+++ privacy_policy/migrations/0001_initial.py	2019-06-14 07:22:45 +0000
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 # Generated by Django 1.11.12 on 2018-10-08 18:23
-from __future__ import unicode_literals
+
 
 from django.db import migrations, models
 

=== modified file 'privacy_policy/models.py'
--- privacy_policy/models.py	2018-10-14 07:58:31 +0000
+++ privacy_policy/models.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.db import models
 from django.urls import reverse
@@ -17,7 +17,7 @@
     class Meta:
         ordering = ['language']
 
-    def __unicode__(self):
+    def __str__(self):
         return self.language
     
     def get_absolute_url(self):

=== modified file 'privacy_policy/views.py'
--- privacy_policy/views.py	2019-04-07 09:01:06 +0000
+++ privacy_policy/views.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.shortcuts import render
 from django.core.exceptions import ObjectDoesNotExist

=== modified file 'pybb/lib/phpserialize.py'
--- pybb/lib/phpserialize.py	2009-02-25 16:55:36 +0000
+++ pybb/lib/phpserialize.py	2019-06-14 07:22:45 +0000
@@ -96,7 +96,7 @@
     :copyright: 2007-2008 by Armin Ronacher.
     license: BSD
 """
-from StringIO import StringIO
+from io import StringIO
 
 __author__ = 'Armin Ronacher <[e-mail]>'
 __version__ = '1.1'
@@ -108,10 +108,10 @@
     """
     def _serialize(obj, keypos):
         if keypos:
-            if isinstance(obj, (int, long, float, bool)):
+            if isinstance(obj, (int, float, bool)):
                 return 'i:%i;' % obj
-            if isinstance(obj, basestring):
-                if isinstance(obj, unicode):
+            if isinstance(obj, str):
+                if isinstance(obj, str):
                     obj = obj.encode(charset, errors)
                 return 's:%i:"%s";' % (len(obj), obj)
             if obj is None:
@@ -122,18 +122,18 @@
                 return 'N;'
             if isinstance(obj, bool):
                 return 'b:%i;' % obj
-            if isinstance(obj, (int, long)):
+            if isinstance(obj, int):
                 return 'i:%s;' % obj
             if isinstance(obj, float):
                 return 'd:%s;' % obj
-            if isinstance(obj, basestring):
-                if isinstance(obj, unicode):
+            if isinstance(obj, str):
+                if isinstance(obj, str):
                     obj = obj.encode(charset, errors)
                 return 's:%i:"%s";' % (len(obj), obj)
             if isinstance(obj, (list, tuple, dict)):
                 out = []
                 if isinstance(obj, dict):
-                    iterable = obj.iteritems()
+                    iterable = iter(obj.items())
                 else:
                     iterable = enumerate(obj)
                 for key, value in iterable:
@@ -202,7 +202,7 @@
             _expect('{')
             result = {}
             last_item = Ellipsis
-            for idx in xrange(items):
+            for idx in range(items):
                 item = _unserialize()
                 if last_item is Ellipsis:
                     last_item = item
@@ -238,7 +238,7 @@
 def dict_to_list(d):
     """Converts an ordered dict into a list."""
     try:
-        return [d[x] for x in xrange(len(d))]
+        return [d[x] for x in range(len(d))]
     except KeyError:
         raise ValueError('dict is not a sequence')
 

=== modified file 'pybb/management/__init__.py'
--- pybb/management/__init__.py	2010-06-08 19:11:54 +0000
+++ pybb/management/__init__.py	2019-06-14 07:22:45 +0000
@@ -1,1 +1,1 @@
-import pybb_notifications
+from . import pybb_notifications

=== modified file 'pybb/management/commands/pybb_resave_post.py'
--- pybb/management/commands/pybb_resave_post.py	2018-04-08 15:53:05 +0000
+++ pybb/management/commands/pybb_resave_post.py	2019-06-14 07:22:45 +0000
@@ -14,5 +14,5 @@
 
         for count, post in enumerate(Post.objects.all()):
             if count and not count % 1000:
-                print count
+                print(count)
             post.save()

=== modified file 'pybb/management/pybb_notifications.py'
--- pybb/management/pybb_notifications.py	2018-04-03 18:57:40 +0000
+++ pybb/management/pybb_notifications.py	2019-06-14 07:22:45 +0000
@@ -14,4 +14,4 @@
                                         _('Forum New Post'),
                                         _('a new comment has been posted to a topic you observe'))
 except ImportError:
-    print 'Skipping creation of NoticeTypes as notification app not found'
+    print('Skipping creation of NoticeTypes as notification app not found')

=== modified file 'pybb/markups/postmarkup.py'
--- pybb/markups/postmarkup.py	2016-12-13 18:28:51 +0000
+++ pybb/markups/postmarkup.py	2019-06-14 07:22:45 +0000
@@ -8,8 +8,8 @@
 __version__ = '1.1.3'
 
 import re
-from urllib import quote, unquote, quote_plus
-from urlparse import urlparse, urlunparse
+from urllib.parse import quote, unquote, quote_plus
+from urllib.parse import urlparse, urlunparse
 
 pygments_available = True
 try:
@@ -28,7 +28,7 @@
     domain -- Domain parsed from url
 
     """
-    return u" [%s]" % _escape(domain)
+    return " [%s]" % _escape(domain)
 
 
 re_url = re.compile(
@@ -62,7 +62,7 @@
     if match is None:
         return ''
     excerpt = match.group(0)
-    excerpt = excerpt.replace(u'\n', u"<br/>")
+    excerpt = excerpt.replace('\n', "<br/>")
     return remove_markup(excerpt)
 
 
@@ -73,7 +73,7 @@
 
     """
 
-    return u"".join([t[1] for t in PostMarkup.tokenize(bbcode) if t[0] == PostMarkup.TOKEN_TEXT])
+    return "".join([t[1] for t in PostMarkup.tokenize(bbcode) if t[0] == PostMarkup.TOKEN_TEXT])
 
 
 def create(include=None, exclude=None, use_pygments=True, **kwargs):
@@ -107,28 +107,28 @@
 
     add_tag(QuoteTag, 'quote')
 
-    add_tag(SearchTag, u'wiki',
-            u"http://en.wikipedia.org/wiki/Special:Search?search=%s", u'wikipedia.com', **kwargs)
-    add_tag(SearchTag, u'google',
-            u"http://www.google.com/search?hl=en&q=%s&btnG=Google+Search", u'google.com', **kwargs)
-    add_tag(SearchTag, u'dictionary',
-            u"http://dictionary.reference.com/browse/%s", u'dictionary.com', **kwargs)
-    add_tag(SearchTag, u'dict',
-            u"http://dictionary.reference.com/browse/%s", u'dictionary.com', **kwargs)
-
-    add_tag(ImgTag, u'img')
-    add_tag(ListTag, u'list')
-    add_tag(ListItemTag, u'*')
-
-    add_tag(SizeTag, u"size")
-    add_tag(ColorTag, u"color")
-    add_tag(CenterTag, u"center")
+    add_tag(SearchTag, 'wiki',
+            "http://en.wikipedia.org/wiki/Special:Search?search=%s", 'wikipedia.com', **kwargs)
+    add_tag(SearchTag, 'google',
+            "http://www.google.com/search?hl=en&q=%s&btnG=Google+Search", 'google.com', **kwargs)
+    add_tag(SearchTag, 'dictionary',
+            "http://dictionary.reference.com/browse/%s", 'dictionary.com', **kwargs)
+    add_tag(SearchTag, 'dict',
+            "http://dictionary.reference.com/browse/%s", 'dictionary.com', **kwargs)
+
+    add_tag(ImgTag, 'img')
+    add_tag(ListTag, 'list')
+    add_tag(ListItemTag, '*')
+
+    add_tag(SizeTag, "size")
+    add_tag(ColorTag, "color")
+    add_tag(CenterTag, "center")
 
     if use_pygments:
         assert pygments_available, 'Install Pygments (http://pygments.org/) or call create with use_pygments=False'
-        add_tag(PygmentsCodeTag, u'code', **kwargs)
+        add_tag(PygmentsCodeTag, 'code', **kwargs)
     else:
-        add_tag(CodeTag, u'code', **kwargs)
+        add_tag(CodeTag, 'code', **kwargs)
 
     return postmarkup
 
@@ -203,7 +203,7 @@
     def get_contents_text(self, parser):
         """Returns the string between the the open and close tag, minus bbcode
         tags."""
-        return u"".join(parser.get_text_nodes(self.open_node_index, self.close_node_index))
+        return "".join(parser.get_text_nodes(self.open_node_index, self.close_node_index))
 
     def skip_contents(self, parser):
         """Skips the contents of a tag while rendering."""
@@ -223,10 +223,10 @@
         self.html_name = html_name
 
     def render_open(self, parser, node_index):
-        return u"<%s>" % self.html_name
+        return "<%s>" % self.html_name
 
     def render_close(self, parser, node_index):
-        return u"</%s>" % self.html_name
+        return "</%s>" % self.html_name
 
 
 class DivStyleTag(TagBase):
@@ -239,10 +239,10 @@
         self.value = value
 
     def render_open(self, parser, node_index):
-        return u'<div style="%s:%s;">' % (self.style, self.value)
+        return '<div style="%s:%s;">' % (self.style, self.value)
 
     def render_close(self, parser, node_index):
-        return u'</div>'
+        return '</div>'
 
 
 class LinkTag(TagBase):
@@ -254,13 +254,13 @@
 
     def render_open(self, parser, node_index):
 
-        self.domain = u''
+        self.domain = ''
         tag_data = parser.tag_data
         nest_level = tag_data['link_nest_level'] = tag_data.setdefault(
             'link_nest_level', 0) + 1
 
         if nest_level > 1:
-            return u""
+            return ""
 
         if self.params:
             url = self.params.strip()
@@ -272,12 +272,12 @@
         self.url = unquote(url)
 
         # Disallow javascript links
-        if u"javascript:" in self.url.lower():
+        if "javascript:" in self.url.lower():
             return ''
 
         # Disallow non http: links
         url_parsed = urlparse(self.url)
-        if url_parsed[0] and not url_parsed[0].lower().startswith(u'http'):
+        if url_parsed[0] and not url_parsed[0].lower().startswith('http'):
             return ''
 
         # Prepend http: if it is not present
@@ -289,21 +289,21 @@
         self.domain = url_parsed[1].lower()
 
         # Remove www for brevity
-        if self.domain.startswith(u'www.'):
+        if self.domain.startswith('www.'):
             self.domain = self.domain[4:]
 
         # Quote the url
         #self.url="http:"+urlunparse( map(quote, (u"",)+url_parsed[1:]) )
-        self.url = unicode(urlunparse(
+        self.url = str(urlunparse(
             [quote(component.encode('utf-8'), safe='/=&?:+') for component in url_parsed]))
 
         if not self.url:
-            return u""
+            return ""
 
         if self.domain:
-            return u'<a href="%s">' % self.url
+            return '<a href="%s">' % self.url
         else:
-            return u""
+            return ""
 
     def render_close(self, parser, node_index):
 
@@ -311,19 +311,19 @@
         tag_data['link_nest_level'] -= 1
 
         if tag_data['link_nest_level'] > 0:
-            return u''
+            return ''
 
         if self.domain:
-            return u'</a>' + self.annotate_link(self.domain)
+            return '</a>' + self.annotate_link(self.domain)
         else:
-            return u''
+            return ''
 
     def annotate_link(self, domain=None):
 
         if domain and self.annotate_links:
             return annotate_link(domain)
         else:
-            return u""
+            return ""
 
 
 class QuoteTag(TagBase):
@@ -339,12 +339,12 @@
 
     def render_open(self, parser, node_index):
         if self.params:
-            return u'<blockquote><em>%s</em><br/>' % (PostMarkup.standard_replace(self.params))
+            return '<blockquote><em>%s</em><br/>' % (PostMarkup.standard_replace(self.params))
         else:
-            return u'<blockquote>'
+            return '<blockquote>'
 
     def render_close(self, parser, node_index):
-        return u"</blockquote>"
+        return "</blockquote>"
 
 
 class SearchTag(TagBase):
@@ -361,8 +361,8 @@
             search = self.params
         else:
             search = self.get_contents(parser)
-        link = u'<a href="%s">' % self.url
-        if u'%' in link:
+        link = '<a href="%s">' % self.url
+        if '%' in link:
             return link % quote_plus(search.encode('UTF-8'))
         else:
             return link
@@ -370,12 +370,12 @@
     def render_close(self, parser, node_index):
 
         if self.label:
-            ret = u'</a>'
+            ret = '</a>'
             if self.annotate_links:
                 ret += annotate_link(self.label)
             return ret
         else:
-            return u''
+            return ''
 
 
 class PygmentsCodeTag(TagBase):
@@ -421,9 +421,9 @@
         contents = self.get_contents(parser)
         self.skip_contents(parser)
 
-        contents = strip_bbcode(contents).replace(u'"', '%22')
+        contents = strip_bbcode(contents).replace('"', '%22')
 
-        return u'<img src="%s"></img>' % contents
+        return '<img src="%s"></img>' % contents
 
 
 class ListTag(TagBase):
@@ -439,30 +439,30 @@
 
     def render_open(self, parser, node_index):
 
-        self.close_tag = u""
+        self.close_tag = ""
 
         tag_data = parser.tag_data
         tag_data.setdefault('ListTag.count', 0)
 
         if tag_data['ListTag.count']:
-            return u""
+            return ""
 
         tag_data['ListTag.count'] += 1
 
         tag_data['ListItemTag.initial_item'] = True
 
         if self.params == '1':
-            self.close_tag = u"</li></ol>"
-            return u"<ol><li>"
+            self.close_tag = "</li></ol>"
+            return "<ol><li>"
         elif self.params == 'a':
-            self.close_tag = u"</li></ol>"
-            return u'<ol style="list-style-type: lower-alpha;"><li>'
+            self.close_tag = "</li></ol>"
+            return '<ol style="list-style-type: lower-alpha;"><li>'
         elif self.params == 'A':
-            self.close_tag = u"</li></ol>"
-            return u'<ol style="list-style-type: upper-alpha;"><li>'
+            self.close_tag = "</li></ol>"
+            return '<ol style="list-style-type: upper-alpha;"><li>'
         else:
-            self.close_tag = u"</li></ul>"
-            return u"<ul><li>"
+            self.close_tag = "</li></ul>"
+            return "<ul><li>"
 
     def render_close(self, parser, node_index):
 
@@ -482,13 +482,13 @@
 
         tag_data = parser.tag_data
         if not tag_data.setdefault('ListTag.count', 0):
-            return u""
+            return ""
 
         if tag_data['ListItemTag.initial_item']:
             tag_data['ListItemTag.initial_item'] = False
             return
 
-        return u"</li><li>"
+        return "</li><li>"
 
 
 class SizeTag(TagBase):
@@ -507,18 +507,18 @@
             self.size = None
 
         if self.size is None:
-            return u""
+            return ""
 
         self.size = self.validate_size(self.size)
 
-        return u'<span style="font-size:%spx">' % self.size
+        return '<span style="font-size:%spx">' % self.size
 
     def render_close(self, parser, node_index):
 
         if self.size is None:
-            return u""
+            return ""
 
-        return u'</span>'
+        return '</span>'
 
     def validate_size(self, size):
 
@@ -541,26 +541,26 @@
         self.color = ''.join([c for c in color if c in valid_chars])
 
         if not self.color:
-            return u""
+            return ""
 
-        return u'<span style="color:%s">' % self.color
+        return '<span style="color:%s">' % self.color
 
     def render_close(self, parser, node_index):
 
         if not self.color:
-            return u''
-        return u'</span>'
+            return ''
+        return '</span>'
 
 
 class CenterTag(TagBase):
 
     def render_open(self, parser, node_index, **kwargs):
 
-        return u'<div style="text-align:center">'
+        return '<div style="text-align:center">'
 
     def render_close(self, parser, node_index):
 
-        return u'</div>'
+        return '</div>'
 
 # http://effbot.org/zone/python-replace.htm
 
@@ -570,10 +570,10 @@
     def __init__(self, repl_dict):
 
         # string to string mapping; use a regular expression
-        keys = repl_dict.keys()
+        keys = list(repl_dict.keys())
         keys.sort()  # lexical order
         keys.reverse()  # use longest match first
-        pattern = u"|".join([re.escape(key) for key in keys])
+        pattern = "|".join([re.escape(key) for key in keys])
         self.pattern = re.compile(pattern)
         self.dict = repl_dict
 
@@ -673,16 +673,16 @@
 
 class PostMarkup(object):
 
-    standard_replace = MultiReplace({u'<': u'&lt;',
-                                     u'>': u'&gt;',
-                                     u'&': u'&amp;',
-                                     u'\n': u'<br/>'})
-
-    standard_replace_no_break = MultiReplace({u'<': u'&lt;',
-                                              u'>': u'&gt;',
-                                              u'&': u'&amp;', })
-
-    TOKEN_TAG, TOKEN_PTAG, TOKEN_TEXT = range(3)
+    standard_replace = MultiReplace({'<': '&lt;',
+                                     '>': '&gt;',
+                                     '&': '&amp;',
+                                     '\n': '<br/>'})
+
+    standard_replace_no_break = MultiReplace({'<': '&lt;',
+                                              '>': '&gt;',
+                                              '&': '&amp;', })
+
+    TOKEN_TAG, TOKEN_PTAG, TOKEN_TEXT = list(range(3))
 
     # I tried to use RE's. Really I did.
     @classmethod
@@ -702,7 +702,7 @@
 
         while True:
 
-            brace_pos = post.find(u'[', pos)
+            brace_pos = post.find('[', pos)
             if brace_pos == -1:
                 if pos < len(post):
                     yield PostMarkup.TOKEN_TEXT, post[pos:], pos, len(post)
@@ -713,8 +713,8 @@
             pos = brace_pos
             end_pos = pos + 1
 
-            open_tag_pos = post.find(u'[', end_pos)
-            end_pos = find_first(post, end_pos, u']=')
+            open_tag_pos = post.find('[', end_pos)
+            end_pos = find_first(post, end_pos, ']=')
             if end_pos == -1:
                 yield PostMarkup.TOKEN_TEXT, post[pos:], pos, len(post)
                 return
@@ -736,20 +736,20 @@
                     while post[end_pos] == ' ':
                         end_pos += 1
                     if post[end_pos] != '"':
-                        end_pos = post.find(u']', end_pos + 1)
+                        end_pos = post.find(']', end_pos + 1)
                         if end_pos == -1:
                             return
                         yield PostMarkup.TOKEN_TAG, post[pos:end_pos + 1], pos, end_pos + 1
                     else:
-                        end_pos = find_first(post, end_pos, u'"]')
+                        end_pos = find_first(post, end_pos, '"]')
 
                         if end_pos == -1:
                             return
                         if post[end_pos] == '"':
-                            end_pos = post.find(u'"', end_pos + 1)
+                            end_pos = post.find('"', end_pos + 1)
                             if end_pos == -1:
                                 return
-                            end_pos = post.find(u']', end_pos + 1)
+                            end_pos = post.find(']', end_pos + 1)
                             if end_pos == -1:
                                 return
                             yield PostMarkup.TOKEN_PTAG, post[pos:end_pos + 1], pos, end_pos + 1
@@ -763,7 +763,7 @@
         """Surrounds urls with url bbcode tags."""
 
         def repl(match):
-            return u'[url]%s[/url]' % match.group(0)
+            return '[url]%s[/url]' % match.group(0)
 
         text_tokens = []
         for tag_type, tag_token, start_pos, end_pos in self.tokenize(postmarkup):
@@ -773,7 +773,7 @@
             else:
                 text_tokens.append(tag_token)
 
-        return u"".join(text_tokens)
+        return "".join(text_tokens)
 
     def __init__(self, tag_factory=None):
 
@@ -784,10 +784,10 @@
 
         add_tag = self.tag_factory.add_tag
 
-        add_tag(SimpleTag, u'b', u'strong')
-        add_tag(SimpleTag, u'i', u'em')
-        add_tag(SimpleTag, u'u', u'u')
-        add_tag(SimpleTag, u's', u's')
+        add_tag(SimpleTag, 'b', 'strong')
+        add_tag(SimpleTag, 'i', 'em')
+        add_tag(SimpleTag, 'u', 'u')
+        add_tag(SimpleTag, 's', 's')
 
     def get_supported_tags(self):
         """Returns a list of the supported tags."""
@@ -808,8 +808,8 @@
 
         """
 
-        if not isinstance(post_markup, unicode):
-            post_markup = unicode(post_markup, encoding, 'replace')
+        if not isinstance(post_markup, str):
+            post_markup = str(post_markup, encoding, 'replace')
 
         if auto_urls:
             post_markup = self.tagify_urls(post_markup)
@@ -901,24 +901,24 @@
             elif tag_type == PostMarkup.TOKEN_TAG:
                 tag_token = tag_token[1:-1].lstrip()
                 if ' ' in tag_token:
-                    tag_name, tag_attribs = tag_token.split(u' ', 1)
+                    tag_name, tag_attribs = tag_token.split(' ', 1)
                     tag_attribs = tag_attribs.strip()
                 else:
                     if '=' in tag_token:
-                        tag_name, tag_attribs = tag_token.split(u'=', 1)
+                        tag_name, tag_attribs = tag_token.split('=', 1)
                         tag_attribs = tag_attribs.strip()
                     else:
                         tag_name = tag_token
-                        tag_attribs = u""
+                        tag_attribs = ""
             else:
                 tag_token = tag_token[1:-1].lstrip()
-                tag_name, tag_attribs = tag_token.split(u'=', 1)
+                tag_name, tag_attribs = tag_token.split('=', 1)
                 tag_attribs = tag_attribs.strip()[1:-1]
 
             tag_name = tag_name.strip().lower()
 
             end_tag = False
-            if tag_name.startswith(u'/'):
+            if tag_name.startswith('/'):
                 end_tag = True
                 tag_name = tag_name[1:]
 
@@ -996,7 +996,7 @@
                 text.append(node_text)
             parser.render_node_index += 1
 
-        return u"".join(text)
+        return "".join(text)
 
     __call__ = render_to_html
 
@@ -1009,7 +1009,7 @@
     post_markup = create(use_pygments=True)
 
     tests = []
-    print """<link rel="stylesheet" href="code.css" type="text/css" />\n"""
+    print("""<link rel="stylesheet" href="code.css" type="text/css" />\n""")
 
     tests.append(']')
     tests.append('[')
@@ -1020,8 +1020,8 @@
     tests.append('[link http://www.willmcgugan.com]My homepage[/link]')
     tests.append('[link]http://www.willmcgugan.com[/link]')
 
-    tests.append(u"[b]Hello AndrУЉ[/b]")
-    tests.append(u"[google]AndrУЉ[/google]")
+    tests.append("[b]Hello AndrУЉ[/b]")
+    tests.append("[google]AndrУЉ[/google]")
     tests.append('[s]Strike through[/s]')
     tests.append('[b]bold [i]bold and italic[/b] italic[/i]')
     tests.append('[google]Will McGugan[/google]')
@@ -1042,7 +1042,7 @@
         return self.__str__()
 [/code]""")
 
-    tests.append(u"[img]http://upload.wikimedia.org/wikipedia/commons"
+    tests.append("[img]http://upload.wikimedia.org/wikipedia/commons"
                  '/6/61/Triops_longicaudatus.jpg[/img]')
 
     tests.append('[list][*]Apples[*]Oranges[*]Pears[/list]')
@@ -1081,13 +1081,13 @@
     tests.append(
         'Nested urls, i.e. [url][url]www.becontrary.com[/url][/url], are condensed in to a single tag.')
 
-    tests.append(u'[google]ЩИЮВfvЮИУАsz[/google]')
-
-    tests.append(u'[size 30]Hello, World![/size]')
-
-    tests.append(u'[color red]This should be red[/color]')
-    tests.append(u'[color #0f0]This should be green[/color]')
-    tests.append(u"[center]This should be in the center!")
+    tests.append('[google]ЩИЮВfvЮИУАsz[/google]')
+
+    tests.append('[size 30]Hello, World![/size]')
+
+    tests.append('[color red]This should be red[/color]')
+    tests.append('[color #0f0]This should be green[/color]')
+    tests.append("[center]This should be in the center!")
 
     tests.append(
         'Nested urls, i.e. [url][url]www.becontrary.com[/url][/url], are condensed in to a single tag.')
@@ -1125,17 +1125,17 @@
     # tests=["""[b]b[i]i[/b][/i]"""]
 
     for test in tests:
-        print u"<pre>%s</pre>" % str(test.encode('ascii', 'xmlcharrefreplace'))
-        print u"<p>%s</p>" % str(post_markup(test).encode('ascii', 'xmlcharrefreplace'))
-        print u"<hr/>"
-        print
-
-    print repr(post_markup('[url=<script>Attack</script>]Attack[/url]'))
-
-    print repr(post_markup('http://www.google.com/search?as_q=bbcode&btnG=%D0%9F%D0%BE%D0%B8%D1%81%D0%BA'))
+        print("<pre>%s</pre>" % str(test.encode('ascii', 'xmlcharrefreplace')))
+        print("<p>%s</p>" % str(post_markup(test).encode('ascii', 'xmlcharrefreplace')))
+        print("<hr/>")
+        print()
+
+    print(repr(post_markup('[url=<script>Attack</script>]Attack[/url]')))
+
+    print(repr(post_markup('http://www.google.com/search?as_q=bbcode&btnG=%D0%9F%D0%BE%D0%B8%D1%81%D0%BA')))
 
     p = create(use_pygments=False)
-    print(p('[code]foo\nbar[/code]'))
+    print((p('[code]foo\nbar[/code]')))
 
     # print render_bbcode("[b]For the lazy, use the http://www.willmcgugan.com
     # render_bbcode function.[/b]")

=== modified file 'pybb/migrations/0001_initial.py'
--- pybb/migrations/0001_initial.py	2016-12-13 18:28:51 +0000
+++ pybb/migrations/0001_initial.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.db import models, migrations
 from django.conf import settings

=== modified file 'pybb/migrations/0002_auto_20161001_2046.py'
--- pybb/migrations/0002_auto_20161001_2046.py	2016-10-08 09:30:34 +0000
+++ pybb/migrations/0002_auto_20161001_2046.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.db import models, migrations
 

=== modified file 'pybb/migrations/0003_remove_post_user_ip.py'
--- pybb/migrations/0003_remove_post_user_ip.py	2018-10-03 09:01:09 +0000
+++ pybb/migrations/0003_remove_post_user_ip.py	2019-06-14 07:22:45 +0000
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 # Generated by Django 1.11.12 on 2018-10-02 19:31
-from __future__ import unicode_literals
+
 
 from django.db import migrations
 

=== modified file 'pybb/migrations/0004_auto_20181209_1334.py'
--- pybb/migrations/0004_auto_20181209_1334.py	2018-12-10 16:37:12 +0000
+++ pybb/migrations/0004_auto_20181209_1334.py	2019-06-14 07:22:45 +0000
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 # Generated by Django 1.11.12 on 2018-12-09 13:34
-from __future__ import unicode_literals
+
 
 from django.db import migrations, models
 import django.db.models.deletion

=== modified file 'pybb/migrations/0005_auto_20181221_1047.py'
--- pybb/migrations/0005_auto_20181221_1047.py	2018-12-21 09:50:32 +0000
+++ pybb/migrations/0005_auto_20181221_1047.py	2019-06-14 07:22:45 +0000
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 # Generated by Django 1.11.12 on 2018-12-21 10:47
-from __future__ import unicode_literals
+
 
 from django.db import migrations
 

=== modified file 'pybb/models.py'
--- pybb/models.py	2019-03-24 08:28:15 +0000
+++ pybb/models.py	2019-06-14 07:22:45 +0000
@@ -61,7 +61,7 @@
         # See also settings.INTERNAL_PERM
         permissions = (("can_access_internal", "Can access Internal Forums"),)
 
-    def __unicode__(self):
+    def __str__(self):
         return self.name
 
     def forum_count(self):
@@ -100,7 +100,7 @@
         verbose_name = _('Forum')
         verbose_name_plural = _('Forums')
 
-    def __unicode__(self):
+    def __str__(self):
         return self.name
 
     def topic_count(self):
@@ -154,7 +154,7 @@
         verbose_name = _('Topic')
         verbose_name_plural = _('Topics')
 
-    def __unicode__(self):
+    def __str__(self):
         return self.name
 
     @property
@@ -215,7 +215,7 @@
         if self.markup == 'bbcode':
             self.body_html = mypostmarkup.markup(self.body, auto_urls=False)
         elif self.markup == 'markdown':
-            self.body_html = unicode(do_wl_markdown(
+            self.body_html = str(do_wl_markdown(
                 self.body, 'bleachit'))
         else:
             raise Exception('Invalid markup property: %s' % self.markup)
@@ -308,7 +308,7 @@
         tail = len(self.body) > LIMIT and '...' or ''
         return self.body[:LIMIT] + tail
 
-    __unicode__ = summary
+    __str__ = summary
 
     def save(self, *args, **kwargs):
         if self.created is None:
@@ -380,8 +380,8 @@
             self.time = datetime.now()
         super(Read, self).save(*args, **kwargs)
 
-    def __unicode__(self):
-        return u'T[%d], U[%d]: %s' % (self.topic.id, self.user.id, unicode(self.time))
+    def __str__(self):
+        return 'T[%d], U[%d]: %s' % (self.topic.id, self.user.id, str(self.time))
 
 
 class Attachment(models.Model):
@@ -401,7 +401,7 @@
                 str(self.id) + settings.SECRET_KEY).hexdigest()
         super(Attachment, self).save(*args, **kwargs)
 
-    def __unicode__(self):
+    def __str__(self):
         return self.name
 
     def get_absolute_url(self):

=== modified file 'pybb/orm.py'
--- pybb/orm.py	2016-12-13 18:28:51 +0000
+++ pybb/orm.py	2019-06-14 07:22:45 +0000
@@ -6,12 +6,12 @@
     rel_field = rel_qs.model._meta.get_field(rel_field_name)
     cache_field_name = '%s_cache' % rel_qs.model.__name__.lower()
 
-    rel_objects = rel_qs.filter(**{'%s__in' % rel_field.name: obj_map.keys()})
+    rel_objects = rel_qs.filter(**{'%s__in' % rel_field.name: list(obj_map.keys())})
 
     temp_map = {}
     for rel_obj in rel_objects:
         pk = getattr(rel_obj, rel_field.attname)
         temp_map.setdefault(pk, []).append(rel_obj)
 
-    for pk, rel_list in temp_map.iteritems():
+    for pk, rel_list in temp_map.items():
         setattr(obj_map[pk], cache_field_name, rel_list)

=== modified file 'pybb/templates/pybb/last_posts.html'
--- pybb/templates/pybb/last_posts.html	2019-03-16 20:10:31 +0000
+++ pybb/templates/pybb/last_posts.html	2019-06-14 07:22:45 +0000
@@ -12,7 +12,7 @@
 				<li>
 					{{ post.topic.forum.name }}<br />
 					<a href="{{ post.get_absolute_url }}" title="{{ post.topic.name }}">{{ post.topic.name|truncatechars:30 }}</a><br />
-					by {{ post.user|user_link }} {{ post.created|minutes }} ago
+					by {{ post.user|user_link }} {{ post.created|elapsed_time }} ago
 				</li>
 			{% endfor %}
 			<li class="small">

=== modified file 'pybb/templatetags/pybb_extras.py'
--- pybb/templatetags/pybb_extras.py	2019-03-23 09:00:44 +0000
+++ pybb/templatetags/pybb_extras.py	2019-06-14 07:22:45 +0000
@@ -7,7 +7,7 @@
 from django import template
 from django.utils.safestring import mark_safe
 from django.template.defaultfilters import stringfilter
-from django.utils.encoding import smart_unicode
+from django.utils.encoding import smart_text
 from django.utils.html import escape
 
 from pybb.models import Post, Forum, Topic, Read
@@ -42,12 +42,12 @@
 
 
 @register.simple_tag
-def pybb_link(object, anchor=u''):
+def pybb_link(object, anchor=''):
     """Return A tag with link to object."""
 
     url = hasattr(
         object, 'get_absolute_url') and object.get_absolute_url() or None
-    anchor = anchor or smart_unicode(object)
+    anchor = anchor or smart_text(object)
     return mark_safe('<a href="%s">%s</a>' % (url, escape(anchor)))
 
 

=== modified file 'pybb/util.py'
--- pybb/util.py	2018-12-21 11:14:06 +0000
+++ pybb/util.py	2019-06-14 07:22:45 +0000
@@ -10,7 +10,7 @@
 from django.http import HttpResponse
 from django.utils.functional import Promise
 from django.utils.translation import check_for_language
-from django.utils.encoding import force_unicode
+from django.utils.encoding import force_text
 from django import forms
 from django.core.paginator import Paginator, EmptyPage, InvalidPage
 from django.conf import settings
@@ -59,7 +59,7 @@
         if request.method == 'POST':
             try:
                 response = func(request, *args, **kwargs)
-            except Exception, ex:
+            except Exception as ex:
                 response = {'error': traceback.format_exc()}
         else:
             response = {'error': {'type': 403,
@@ -76,7 +76,7 @@
 
     def default(self, o):
         if isinstance(o, Promise):
-            return force_unicode(o)
+            return force_text(o)
         else:
             return super(LazyJSONEncoder, self).default(o)
 
@@ -144,7 +144,7 @@
         # Apply the new content
         found_string.parent.contents = new_content
 
-    return unicode(soup)
+    return str(soup)
 
 
 def quote_text(text, user, markup):

=== modified file 'pybb/views.py'
--- pybb/views.py	2019-05-27 15:46:36 +0000
+++ pybb/views.py	2019-06-14 07:22:45 +0000
@@ -365,7 +365,7 @@
     content = request.POST.get('content')
     markup = request.POST.get('markup')
 
-    if not markup in dict(MARKUP_CHOICES).keys():
+    if not markup in list(dict(MARKUP_CHOICES).keys()):
         return {'error': 'Invalid markup'}
 
     if not content:
@@ -374,7 +374,7 @@
     if markup == 'bbcode':
         html = mypostmarkup.markup(content, auto_urls=False)
     elif markup == 'markdown':
-        html = unicode(do_wl_markdown(content, 'bleachit'))
+        html = str(do_wl_markdown(content, 'bleachit'))
 
     html = urlize(html)
     return {'content': html}

=== added file 'python3.txt'
--- python3.txt	1970-01-01 00:00:00 +0000
+++ python3.txt	2019-06-14 07:22:45 +0000
@@ -0,0 +1,104 @@
+Anmerkungen zum update auf python3:
+
+Der code für die implementierung vom gravatar-service ist ungetestet, da es nie
+implementiert wurde.
+
+virtualenvironment
+------------------
+
+Installiere die richtige python Version (hier python3.6).
+Erstellen des virtualenvironments:
+    
+    $:> virtualenv --python=python3.6 wlwebsite
+
+Getestet
+========
+
+Wiki
+----
+- editieren, Vorschau, speichern
+- upload von Bildern
+- wiki-syntax
+- History: Diff anzeigen, Artikel wieder herstellen (Revert)
+- Backlinks
+- Artikel beobachten
+- Atom feed
+
+Registrierung
+-------------
+- Registrierung, Anmeldung
+- Passwortänderung
+- Passwort vergessen
+- Logout
+
+Maps
+----
+- Upload maps with comment
+- Edit uploader comment, also with polish characters
+- Rate a map
+- Make a threadedcomment on a map
+- Delete map
+
+WlHelp (Encyclopedia)
+---------------------
+- Run ./manage.py update_help
+- Run ./manage.py update_help_pdf
+- views
+
+Screenshots
+-----------
+- upload a new screenshot
+- delete screenshot
+- delete category
+
+Search
+------
+- update index
+- searching
+
+News
+----
+- create new News
+- public in future (not show)
+- edit News
+- deletion
+- commenting
+
+Polls
+-----
+- create one
+- edit
+- deletion
+
+Webchat
+-------
+- connect
+
+Documentation
+-------------
+- run ./manage.py create_docs
+- views
+
+wlprofile
+---------
+- editing (Website/signature)
+
+pybb
+----
+- create topic
+- create post
+- post with markdown/bbcode
+- quoting a post
+- edit own post
+- stick/unstick topic
+- close/open topic
+- subscribe to topic (emailing works, also then)
+- edit foreign posts by forum mod
+- delete post by forum mod
+- delete post by posts author
+
+
+Others
+------
+- changelog page
+- developers page

=== modified file 'threadedcomments/admin.py'
--- threadedcomments/admin.py	2018-10-04 16:00:34 +0000
+++ threadedcomments/admin.py	2019-06-14 07:22:45 +0000
@@ -12,7 +12,7 @@
                                 'date_modified', 'date_approved', 'is_approved')}),
     )
     list_display = ('user', 'date_submitted', 'content_type',
-                    'get_content_object', 'parent', '__unicode__')
+                    'get_content_object', 'parent', '__str__')
     list_filter = ('date_submitted',)
     date_hierarchy = 'date_submitted'
     search_fields = ('comment', 'user__username')

=== modified file 'threadedcomments/migrations/0001_initial.py'
--- threadedcomments/migrations/0001_initial.py	2016-12-13 18:28:51 +0000
+++ threadedcomments/migrations/0001_initial.py	2019-06-14 07:22:45 +0000
@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-from __future__ import unicode_literals
+
 
 from django.db import models, migrations
 import datetime

=== modified file 'threadedcomments/migrations/0002_auto_20181003_1238.py'
--- threadedcomments/migrations/0002_auto_20181003_1238.py	2018-10-03 10:53:41 +0000
+++ threadedcomments/migrations/0002_auto_20181003_1238.py	2019-06-14 07:22:45 +0000
@@ -1,6 +1,6 @@
 # -*- coding: utf-8 -*-
 # Generated by Django 1.11.12 on 2018-10-03 12:38
-from __future__ import unicode_literals
+
 
 from django.db import migrations
 

=== modified file 'threadedcomments/models.py'
--- threadedcomments/models.py	2018-10-04 16:00:34 +0000
+++ threadedcomments/models.py	2019-06-14 07:22:45 +0000
@@ -6,7 +6,7 @@
 from django.db.models import Q
 from django.utils.translation import ugettext_lazy as _
 from django.conf import settings
-from django.utils.encoding import force_unicode
+from django.utils.encoding import force_text
 
 DEFAULT_MAX_COMMENT_LENGTH = getattr(
     settings, 'DEFAULT_MAX_COMMENT_LENGTH', 1000)
@@ -178,7 +178,7 @@
     objects = ThreadedCommentManager()
     public = PublicThreadedCommentManager()
 
-    def __unicode__(self):
+    def __str__(self):
         if len(self.comment) > 50:
             return self.comment[:50] + '...'
         return self.comment[:50]

=== modified file 'threadedcomments/templatetags/gravatar.py'
--- threadedcomments/templatetags/gravatar.py	2018-04-03 05:18:03 +0000
+++ threadedcomments/templatetags/gravatar.py	2019-06-14 07:22:45 +0000
@@ -1,16 +1,16 @@
 from django import template
 from django.conf import settings
 from django.template.defaultfilters import stringfilter
-from django.utils.encoding import smart_str
+from django.utils.encoding import smart_bytes
 from django.utils.safestring import mark_safe
 from hashlib import md5 as md5_constructor
-import urllib
+import urllib.request, urllib.parse, urllib.error
 
 GRAVATAR_MAX_RATING = getattr(settings, 'GRAVATAR_MAX_RATING', 'R')
 GRAVATAR_DEFAULT_IMG = getattr(settings, 'GRAVATAR_DEFAULT_IMG', 'img:blank')
 GRAVATAR_SIZE = getattr(settings, 'GRAVATAR_SIZE', 80)
 
-GRAVATAR_URL = u'http://www.gravatar.com/avatar.php?gravatar_id=%(hash)s&rating=%(rating)s&size=%(size)s&default=%(default)s'
+GRAVATAR_URL = 'http://www.gravatar.com/avatar.php?gravatar_id=%(hash)s&rating=%(rating)s&size=%(size)s&default=%(default)s'
 
 
 def get_gravatar_url(parser, token):
@@ -34,12 +34,12 @@
     words = token.contents.split()
     tagname = words.pop(0)
     if len(words) < 2:
-        raise template.TemplateSyntaxError, '%r tag: At least one argument should be provided.' % tagname
+        raise template.TemplateSyntaxError('%r tag: At least one argument should be provided.' % tagname)
     if words.pop(0) != 'for':
-        raise template.TemplateSyntaxError, '%r tag: Syntax is {% get_gravatar_url for myemailvar rating 'R" size 80 default img:blank as gravatar_url %}, where everything after myemailvar is optional."
+        raise template.TemplateSyntaxError('%r tag: Syntax is {% get_gravatar_url for myemailvar rating 'R" size 80 default img:blank as gravatar_url %}, where everything after myemailvar is optional.")
     email = words.pop(0)
     if len(words) % 2 != 0:
-        raise template.TemplateSyntaxError, '%r tag: Imbalanced number of arguments.' % tagname
+        raise template.TemplateSyntaxError('%r tag: Imbalanced number of arguments.' % tagname)
     args = {
         'email': email,
         'rating': GRAVATAR_MAX_RATING,
@@ -49,8 +49,8 @@
     for name, value in zip(words[::2], words[1::2]):
         name = name.lower()
         if name not in ('rating', 'size', 'default', 'as'):
-            raise template.TemplateSyntaxError, '%r tag: Invalid argument %r.' % tagname, name
-        args[smart_str(name)] = value
+            raise template.TemplateSyntaxError('%r tag: Invalid argument %r.' % tagname).with_traceback(name)
+        args[smart_bytes(name)] = value
     return GravatarUrlNode(**args)
 
 
@@ -93,7 +93,7 @@
             'hash': md5_constructor(email).hexdigest(),
             'rating': rating,
             'size': size,
-            'default': urllib.quote_plus(default),
+            'default': urllib.parse.quote_plus(default),
         }
         url = GRAVATAR_URL % gravatargs
         if 'as' in self.other_kwargs:
@@ -112,7 +112,7 @@
         'hash': hashed_email,
         'rating': GRAVATAR_MAX_RATING,
         'size': GRAVATAR_SIZE,
-        'default': urllib.quote_plus(GRAVATAR_DEFAULT_IMG),
+        'default': urllib.parse.quote_plus(GRAVATAR_DEFAULT_IMG),
     })
 gravatar = stringfilter(gravatar)
 

=== modified file 'threadedcomments/templatetags/threadedcommentstags.py'
--- threadedcomments/templatetags/threadedcommentstags.py	2018-10-04 06:15:32 +0000
+++ threadedcomments/templatetags/threadedcommentstags.py	2019-06-14 07:22:45 +0000
@@ -2,7 +2,7 @@
 from django import template
 from django.contrib.contenttypes.models import ContentType
 from django.urls import reverse
-from django.utils.encoding import force_unicode
+from django.utils.encoding import force_text
 from django.utils.safestring import mark_safe
 from threadedcomments.models import ThreadedComment
 from threadedcomments.forms import ThreadedCommentForm
@@ -28,7 +28,7 @@
     kwargs = get_contenttype_kwargs(content_object)
     if parent:
         if not isinstance(parent, ThreadedComment):
-            raise template.TemplateSyntaxError, 'get_comment_url requires its parent object to be of type ThreadedComment'
+            raise template.TemplateSyntaxError('get_comment_url requires its parent object to be of type ThreadedComment')
         kwargs.update({'parent_id': getattr(
             parent, 'pk', getattr(parent, 'id'))})
         return reverse('tc_comment_parent', kwargs=kwargs)
@@ -48,7 +48,7 @@
     kwargs.update({'ajax': ajax_type})
     if parent:
         if not isinstance(parent, ThreadedComment):
-            raise template.TemplateSyntaxError, 'get_comment_url_ajax requires its parent object to be of type ThreadedComment'
+            raise template.TemplateSyntaxError('get_comment_url_ajax requires its parent object to be of type ThreadedComment')
         kwargs.update({'parent_id': getattr(
             parent, 'pk', getattr(parent, 'id'))})
         return reverse('tc_comment_parent_ajax', kwargs=kwargs)
@@ -63,7 +63,7 @@
     try:
         return get_comment_url_ajax(content_object, parent, ajax_type='json')
     except template.TemplateSyntaxError:
-        raise template.TemplateSyntaxError, 'get_comment_url_json requires its parent object to be of type ThreadedComment'
+        raise template.TemplateSyntaxError('get_comment_url_json requires its parent object to be of type ThreadedComment')
     return ''
 
 
@@ -74,7 +74,7 @@
     try:
         return get_comment_url_ajax(content_object, parent, ajax_type='xml')
     except template.TemplateSyntaxError:
-        raise template.TemplateSyntaxError, 'get_comment_url_xml requires its parent object to be of type ThreadedComment'
+        raise template.TemplateSyntaxError('get_comment_url_xml requires its parent object to be of type ThreadedComment')
     return ''
 
 
@@ -92,13 +92,13 @@
     try:
         split = token.split_contents()
     except ValueError:
-        raise template.TemplateSyntaxError, '%r tag must be of format {%% %r COMMENT %%} or of format {%% %r COMMENT as CONTEXT_VARIABLE %%}' % (token.contents.split()[0], token.contents.split()[0], token.contents.split()[0])
+        raise template.TemplateSyntaxError('%r tag must be of format {%% %r COMMENT %%} or of format {%% %r COMMENT as CONTEXT_VARIABLE %%}' % (token.contents.split()[0], token.contents.split()[0], token.contents.split()[0]))
     if len(split) == 2:
         return AutoTransformMarkupNode(split[1])
     elif len(split) == 4:
         return AutoTransformMarkupNode(split[1], context_name=split[3])
     else:
-        raise template.TemplateSyntaxError, 'Invalid number of arguments for tag %r' % split[0]
+        raise template.TemplateSyntaxError('Invalid number of arguments for tag %r' % split[0])
 
 
 class AutoTransformMarkupNode(template.Node):
@@ -167,9 +167,9 @@
     try:
         split = token.split_contents()
     except ValueError:
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
     if split[1] != 'for' or split[3] != 'as':
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
     return ThreadedCommentCountNode(split[2], split[4])
 
 
@@ -202,11 +202,11 @@
     try:
         split = token.split_contents()
     except ValueError:
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
     if split[1] != 'as':
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
     if len(split) != 3:
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
     
     return ThreadedCommentFormNode(split[2])
 
@@ -230,11 +230,11 @@
     try:
         split = token.split_contents()
     except ValueError:
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
     if len(split) != 4:
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
     if split[2] != 'as':
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
 
     return LatestCommentsNode(split[1], split[3])
 
@@ -259,9 +259,9 @@
     try:
         split = token.split_contents()
     except ValueError:
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
     if len(split) != 5:
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
 
     return UserCommentsNode(split[2], split[4])
 
@@ -285,9 +285,9 @@
     try:
         split = token.split_contents()
     except ValueError:
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
     if len(split) != 5:
-        raise template.TemplateSyntaxError, error_message
+        raise template.TemplateSyntaxError(error_message)
 
     return UserCommentCountNode(split[2], split[4])
 

=== modified file 'threadedcomments/utils.py'
--- threadedcomments/utils.py	2016-12-13 18:28:51 +0000
+++ threadedcomments/utils.py	2019-06-14 07:22:45 +0000
@@ -1,7 +1,7 @@
 from django.core.serializers import serialize
 from django.http import HttpResponse
 from django.utils.functional import Promise
-from django.utils.encoding import force_unicode
+from django.utils.encoding import force_text
 import json as simplejson
 
 
@@ -9,7 +9,7 @@
 
     def default(self, obj):
         if isinstance(obj, Promise):
-            return force_unicode(obj)
+            return force_text(obj)
         return obj
 
 

=== modified file 'threadedcomments/views.py'
--- threadedcomments/views.py	2019-03-31 11:08:21 +0000
+++ threadedcomments/views.py	2019-06-14 07:22:45 +0000
@@ -121,7 +121,7 @@
 </errorlist>
         """
         response_str = Template(template_str).render(
-            Context({'errors': zip(form.errors.values(), form.errors.keys())}))
+            Context({'errors': list(zip(list(form.errors.values()), list(form.errors.keys())))}))
         return XMLResponse(response_str, is_iterable=False)
     else:
         return _preview(request, context_processors, extra_context, form_class=form_class)

=== modified file 'widelandslib/make_flow_diagram.py'
--- widelandslib/make_flow_diagram.py	2019-03-31 11:08:21 +0000
+++ widelandslib/make_flow_diagram.py	2019-06-14 07:22:45 +0000
@@ -68,7 +68,7 @@
 def add_building(g, b, limit_inputs=None, limit_outputs=None, limit_buildings=None, link_workers=True, limit_recruits=None):
     # Add the nice node
     workers = ''
-    if isinstance(b, (ProductionSite,)):
+    if isinstance(b, ProductionSite):
         workers = r"""<table border="0px" cellspacing="0">"""
         for worker in b.workers:
             wo = b.tribe.workers[worker]
@@ -86,7 +86,7 @@
                 g.add_edge(Edge(b.name, wo.name, color='darkgreen'))
         workers += r"""</table>"""
 
-    if isinstance(b, (MilitarySite,)):
+    if isinstance(b, MilitarySite):
         workers = r"""<table border="0px" cellspacing="0">"""
         workers += (r"""<tr><td border="0px">Keeps %s Soldiers</td></tr>"""
                     r"""<tr><td border="0px">Conquers %s fields</td></tr>"""
@@ -94,7 +94,7 @@
         workers += r"""</table>"""
 
     costs = r"""<tr><td colspan="2"><table border="0px" cellspacing="0">"""
-    for ware, count in b.buildcost.items():
+    for ware, count in list(b.buildcost.items()):
         w = b.tribe.wares[ware]
         costs += ('<tr><td border="0px">%s x </td><td border="0px"><img src="%s"/></td><td border="0px">%s</td></tr>' %
                   (count, w.image, w.descname))
@@ -133,7 +133,7 @@
     else:
         g.add_node(n)
 
-    if isinstance(b, (ProductionSite,)):
+    if isinstance(b, ProductionSite):
         # for worker,c in b.workers:
         #     g.add_edge(Edge(worker, name, color="orange"))
 
@@ -181,24 +181,23 @@
 def make_graph(tribe_name):
     global tdir
     tdir = mkdtemp(prefix='widelands-help')
-
     json_directory = path.normpath(settings.MEDIA_ROOT + '/map_object_info')
-    tribeinfo_file = open(path.normpath(
-        json_directory + '/tribe_' + tribe_name + '.json'), 'r')
-    tribeinfo = json.load(tribeinfo_file)
+    with open(path.normpath(
+            json_directory + '/tribe_' + tribe_name + '.json'), 'r') as tribeinfo_file:
+        tribeinfo = json.load(tribeinfo_file)
 
     t = Tribe(tribeinfo, json_directory)
 
     g = CleanedDot(concentrate='false', style='filled', bgcolor='white',
                    overlap='false', splines='true', rankdir='LR')
 
-    for name, w in t.wares.items():
+    for name, w in list(t.wares.items()):
         add_ware(g, w)
     #
     # for name,w in t.workers.items():
     #     add_worker(g, w)
 
-    for name, b in t.buildings.items():
+    for name, b in list(t.buildings.items()):
         add_building(g, b, link_workers=False)
 
     g.write_pdf(path.join(tdir, '%s.pdf' % tribe_name))
@@ -211,7 +210,7 @@
 
 
 def make_building_graph(t, building_name):
-    if isinstance(t, basestring):
+    if isinstance(t, str):
         t = Tribe(t)
 
     b = t.buildings[building_name]
@@ -219,7 +218,7 @@
     g = CleanedDot(concentrate='false', bgcolor='transparent',
                    overlap='false', splines='true', rankdir='LR')
 
-    if not isinstance(b, (ProductionSite,)):
+    if not isinstance(b, ProductionSite):
         inputs, outputs = [], []
     else:
         # TODO: prepare for tribes having buildings with a ware as both input
@@ -254,7 +253,7 @@
 
 
 def make_worker_graph(t, worker_name):
-    if isinstance(t, basestring):
+    if isinstance(t, str):
         t = Tribe(t)
 
     w = t.workers[worker_name]
@@ -262,7 +261,7 @@
     g = CleanedDot(concentrate='false', bgcolor='transparent',
                    overlap='false', splines='true', rankdir='LR')
 
-    buildings = [bld for bld in t.buildings.values() if
+    buildings = [bld for bld in list(t.buildings.values()) if
                  isinstance(bld, ProductionSite) and
                  (w.name in bld.workers or w.name in bld.recruits)]
 
@@ -275,7 +274,7 @@
     sg = Subgraph('%s_enhancements' % w.name,
                   ordering='out', rankdir='TB', rank='same')
     # find exactly one level of enhancement
-    for other in t.workers.values():
+    for other in list(t.workers.values()):
         if other.becomes == w.name:
             add_worker(sg, other)
             g.add_edge(Edge(other.name, w.name, color='blue'))
@@ -294,15 +293,15 @@
 
 
 def make_ware_graph(t, ware_name):
-    if isinstance(t, basestring):
+    if isinstance(t, str):
         t = Tribe(t)
     w = t.wares[ware_name]
 
     g = CleanedDot(concentrate='false', bgcolor='transparent',
                    overlap='false', splines='true', rankdir='LR')
 
-    buildings = [bld for bld in t.buildings.values() if isinstance(
-        bld, (ProductionSite, )) and (w.name in bld.inputs or w.name in bld.outputs)]
+    buildings = [bld for bld in list(t.buildings.values()) if isinstance(
+        bld, ProductionSite) and (w.name in bld.inputs or w.name in bld.outputs)]
     [add_building(g, bld, limit_inputs=[w.name], limit_outputs=[w.name], limit_buildings=[
                   b.name for b in buildings], link_workers=False) for bld in buildings]
 
@@ -326,28 +325,28 @@
 def make_all_subgraphs(t):
     global tdir
     tdir = mkdtemp(prefix='widelands-help')
-    if isinstance(t, basestring):
+    if isinstance(t, str):
         t = Tribe(t)
-    print 'making all subgraphs for tribe', t.name, 'in', tdir
+    print('making all subgraphs for tribe', t.name, 'in', tdir)
 
-    print '  making wares'
+    print('  making wares')
 
     for w in t.wares:
-        print '    ' + w
+        print('    ' + w)
         make_ware_graph(t, w)
         process_dotfile(path.join(tdir, 'help/%s/wares/%s/' % (t.name, w)))
 
-    print '  making workers'
+    print('  making workers')
 
     for w in t.workers:
-        print '    ' + w
+        print('    ' + w)
         make_worker_graph(t, w)
         process_dotfile(path.join(tdir, 'help/%s/workers/%s/' % (t.name, w)))
 
-    print '  making buildings'
+    print('  making buildings')
 
     for b in t.buildings:
-        print '    ' + b
+        print('    ' + b)
         make_building_graph(t, b)
         process_dotfile(path.join(tdir, 'help/%s/buildings/%s/' % (t.name, b)))
 
@@ -359,5 +358,6 @@
     if b.enhanced_building:
         add_building()
 
+
 if __name__ == '__main__':
     make_all_subgraphs()

=== modified file 'widelandslib/test/test_conf.py'
--- widelandslib/test/test_conf.py	2016-12-13 18:28:51 +0000
+++ widelandslib/test/test_conf.py	2019-06-14 07:22:45 +0000
@@ -13,7 +13,7 @@
 sys.path.append('..')
 
 import unittest
-from cStringIO import StringIO
+from io import StringIO
 
 from conf import WidelandsConfigParser
 

=== modified file 'widelandslib/test/test_map.py'
--- widelandslib/test/test_map.py	2016-12-13 18:28:51 +0000
+++ widelandslib/test/test_map.py	2019-06-14 07:22:45 +0000
@@ -8,7 +8,7 @@
 from numpy import *
 import unittest
 import base64
-from cStringIO import StringIO
+from io import StringIO
 from itertools import *
 
 from map import *
@@ -186,12 +186,12 @@
     # }}}
 
     def test_R(self):
-        r = [i.name == j for i, j in izip(
+        r = [i.name == j for i, j in zip(
             self.m.ter_r.flat, self.ter_r_names.flat)]
         self.assertTrue(all(r))
 
     def test_D(self):
-        d = [i.name == j for i, j in izip(
+        d = [i.name == j for i, j in zip(
             self.m.ter_d.flat, self.ter_d_names.flat)]
         self.assertTrue(all(d))
 

=== modified file 'widelandslib/tribe.py'
--- widelandslib/tribe.py	2019-03-31 11:08:21 +0000
+++ widelandslib/tribe.py	2019-06-14 07:22:45 +0000
@@ -55,7 +55,7 @@
     def base_building(self):
         if not self.enhanced_building:
             return None
-        bases = [b for b in self.tribe.buildings.values()
+        bases = [b for b in list(self.tribe.buildings.values())
                  if b.enhancement == self.name]
         if len(bases) == 0 and self.enhanced_building:
             raise Exception('Building %s has no bases in tribe %s' %
@@ -152,30 +152,29 @@
     def __init__(self, tribeinfo, json_directory):
         self.name = tribeinfo['name']
 
-        wares_file = open(p.normpath(json_directory + '/' +
-                                     self.name + '_wares.json'), 'r')
-        waresinfo = json.load(wares_file)
+        with open(p.normpath(json_directory + '/' +
+                             self.name + '_wares.json'), 'r') as wares_file:
+            waresinfo = json.load(wares_file)
         self.wares = dict()
         for ware in waresinfo['wares']:
-            descname = ware['descname'].encode('ascii', 'xmlcharrefreplace')
+            descname = ware['descname']
             self.wares[ware['name']] = Ware(self, ware['name'], descname, ware)
 
-        workers_file = open(p.normpath(
-            json_directory + '/' + self.name + '_workers.json'), 'r')
-        workersinfo = json.load(workers_file)
+        with open(p.normpath(
+                json_directory + '/' + self.name + '_workers.json'), 'r') as workers_file:
+            workersinfo = json.load(workers_file)
         self.workers = dict()
         for worker in workersinfo['workers']:
-            descname = worker['descname'].encode('ascii', 'xmlcharrefreplace')
+            descname = worker['descname']
             self.workers[worker['name']] = Worker(
                 self, worker['name'], descname, worker)
 
-        buildings_file = open(p.normpath(
-            json_directory + '/' + self.name + '_buildings.json'), 'r')
-        buildingsinfo = json.load(buildings_file)
+        with open(p.normpath(
+                json_directory + '/' + self.name + '_buildings.json'), 'r') as buildings_file:
+            buildingsinfo = json.load(buildings_file)
         self.buildings = dict()
         for building in buildingsinfo['buildings']:
-            descname = building['descname'].encode(
-                'ascii', 'xmlcharrefreplace')
+            descname = building['descname']
             if building['type'] == 'productionsite':
                 self.buildings[building['name']] = ProductionSite(
                     self, building['name'], descname, building)

=== modified file 'wiki/diff_match_patch.py'
--- wiki/diff_match_patch.py	2019-03-31 11:08:21 +0000
+++ wiki/diff_match_patch.py	2019-06-14 07:22:45 +0000
@@ -1,11 +1,11 @@
-#!/usr/bin/python2.4
-
-# Taken from google because they do not provide a setup.py file. Thanks anyway
-
-"""Diff Match and Patch.
-
-Copyright 2006 Google Inc.
-http://code.google.com/p/google-diff-match-patch/
+#!/usr/bin/python3
+
+# Taken from https://github.com/google/diff-match-patch/blob/master/python3/diff_match_patch.py
+# Applied class attribute in diff_prettyHtml() to get variable coloring
+
+"""Diff Match and Patch
+Copyright 2018 The diff-match-patch Authors.
+https://github.com/google/diff-match-patch
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
@@ -18,7 +18,6 @@
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-
 """
 
 """Functions for diff, match and patch.
@@ -29,1901 +28,1883 @@
 
 __author__ = '[e-mail] (Neil Fraser)'
 
-import math
+import re
+import sys
 import time
-import urllib
-import re
+import urllib.parse
 
 
 class diff_match_patch:
-    """Class containing the diff, match and patch methods.
-
-    Also contains the behaviour settings.
-
-    """
-
-    def __init__(self):
-        """Inits a diff_match_patch object with default settings.
-
-        Redefine these in your program to override the defaults.
-
-        """
-
-        # Number of seconds to map a diff before giving up (0 for infinity).
-        self.Diff_Timeout = 1.0
-        # Cost of an empty edit operation in terms of edit characters.
-        self.Diff_EditCost = 4
-        # The size beyond which the double-ended diff activates.
-        # Double-ending is twice as fast, but less accurate.
-        self.Diff_DualThreshold = 32
-        # At what point is no match declared (0.0 = perfection, 1.0 = very
-        # loose).
-        self.Match_Threshold = 0.5
-        # How far to search for a match (0 = exact location, 1000+ = broad match).
-        # A match this many characters away from the expected location will add
-        # 1.0 to the score (0.0 is a perfect match).
-        self.Match_Distance = 1000
-        # When deleting a large block of text (over ~64 characters), how close does
-        # the contents have to match the expected contents. (0.0 = perfection,
-        # 1.0 = very loose).  Note that Match_Threshold controls how closely the
-        # end points of a delete need to match.
-        self.Patch_DeleteThreshold = 0.5
-        # Chunk size for context length.
-        self.Patch_Margin = 4
-
-        # How many bits in a number?
-        # Python has no maximum, thus to disable patch splitting set to 0.
-        # However to avoid long patches in certain pathological cases, use 32.
-        # Multiple short patches (using native ints) are much faster than long
-        # ones.
-        self.Match_MaxBits = 32
-
-    #  DIFF FUNCTIONS
-
-    # The data structure representing a diff is an array of tuples:
-    # [(DIFF_DELETE, "Hello"), (DIFF_INSERT, "Goodbye"), (DIFF_EQUAL, " world.")]
-    # which means: delete "Hello", add "Goodbye" and keep " world."
-    DIFF_DELETE = -1
-    DIFF_INSERT = 1
-    DIFF_EQUAL = 0
-
-    def diff_main(self, text1, text2, checklines=True):
-        """Find the differences between two texts.  Simplifies the problem by
-        stripping any common prefix or suffix off the texts before diffing.
-
-        Args:
-          text1: Old string to be diffed.
-          text2: New string to be diffed.
-          checklines: Optional speedup flag.  If present and false, then don't run
-            a line-level diff first to identify the changed areas.
-            Defaults to true, which does a faster, slightly less optimal diff.
-
-        Returns:
-          Array of changes.
-
-        """
-
-        # Check for equality (speedup)
-        if text1 == text2:
-            return [(self.DIFF_EQUAL, text1)]
-
-        # Trim off common prefix (speedup)
-        commonlength = self.diff_commonPrefix(text1, text2)
-        commonprefix = text1[:commonlength]
-        text1 = text1[commonlength:]
-        text2 = text2[commonlength:]
-
-        # Trim off common suffix (speedup)
-        commonlength = self.diff_commonSuffix(text1, text2)
-        if commonlength == 0:
-            commonsuffix = ''
-        else:
-            commonsuffix = text1[-commonlength:]
-            text1 = text1[:-commonlength]
-            text2 = text2[:-commonlength]
-
-        # Compute the diff on the middle block
-        diffs = self.diff_compute(text1, text2, checklines)
-
-        # Restore the prefix and suffix
-        if commonprefix:
-            diffs[:0] = [(self.DIFF_EQUAL, commonprefix)]
-        if commonsuffix:
-            diffs.append((self.DIFF_EQUAL, commonsuffix))
-        self.diff_cleanupMerge(diffs)
-        return diffs
-
-    def diff_compute(self, text1, text2, checklines):
-        """Find the differences between two texts.  Assumes that the texts do
-        not have any common prefix or suffix.
-
-        Args:
-          text1: Old string to be diffed.
-          text2: New string to be diffed.
-          checklines: Speedup flag.  If false, then don't run a line-level diff
-            first to identify the changed areas.
-            If true, then run a faster, slightly less optimal diff.
-
-        Returns:
-          Array of changes.
-
-        """
-        if not text1:
-            # Just add some text (speedup)
-            return [(self.DIFF_INSERT, text2)]
-
-        if not text2:
-            # Just delete some text (speedup)
-            return [(self.DIFF_DELETE, text1)]
-
-        if len(text1) > len(text2):
-            (longtext, shorttext) = (text1, text2)
-        else:
-            (shorttext, longtext) = (text1, text2)
-        i = longtext.find(shorttext)
-        if i != -1:
-            # Shorter text is inside the longer text (speedup)
-            diffs = [(self.DIFF_INSERT, longtext[:i]), (self.DIFF_EQUAL, shorttext),
-                     (self.DIFF_INSERT, longtext[i + len(shorttext):])]
-            # Swap insertions for deletions if diff is reversed.
-            if len(text1) > len(text2):
-                diffs[0] = (self.DIFF_DELETE, diffs[0][1])
-                diffs[2] = (self.DIFF_DELETE, diffs[2][1])
-            return diffs
-        longtext = shorttext = None  # Garbage collect.
-
-        # Check to see if the problem can be split in two.
-        hm = self.diff_halfMatch(text1, text2)
-        if hm:
-            # A half-match was found, sort out the return data.
-            (text1_a, text1_b, text2_a, text2_b, mid_common) = hm
-            # Send both pairs off for separate processing.
-            diffs_a = self.diff_main(text1_a, text2_a, checklines)
-            diffs_b = self.diff_main(text1_b, text2_b, checklines)
-            # Merge the results.
-            return diffs_a + [(self.DIFF_EQUAL, mid_common)] + diffs_b
-
-        # Perform a real diff.
-        if checklines and (len(text1) < 100 or len(text2) < 100):
-            checklines = False  # Too trivial for the overhead.
-        if checklines:
-            # Scan the text on a line-by-line basis first.
-            (text1, text2, linearray) = self.diff_linesToChars(text1, text2)
-
-        diffs = self.diff_map(text1, text2)
-        if not diffs:  # No acceptable result.
-            diffs = [(self.DIFF_DELETE, text1), (self.DIFF_INSERT, text2)]
-        if checklines:
-            # Convert the diff back to original text.
-            self.diff_charsToLines(diffs, linearray)
-            # Eliminate freak matches (e.g. blank lines)
-            self.diff_cleanupSemantic(diffs)
-
-            # Rediff any replacement blocks, this time character-by-character.
-            # Add a dummy entry at the end.
-            diffs.append((self.DIFF_EQUAL, ''))
-            pointer = 0
-            count_delete = 0
-            count_insert = 0
-            text_delete = ''
-            text_insert = ''
-            while pointer < len(diffs):
-                if diffs[pointer][0] == self.DIFF_INSERT:
-                    count_insert += 1
-                    text_insert += diffs[pointer][1]
-                elif diffs[pointer][0] == self.DIFF_DELETE:
-                    count_delete += 1
-                    text_delete += diffs[pointer][1]
-                elif diffs[pointer][0] == self.DIFF_EQUAL:
-                    # Upon reaching an equality, check for prior redundancies.
-                    if count_delete >= 1 and count_insert >= 1:
-                        # Delete the offending records and add the merged ones.
-                        a = self.diff_main(text_delete, text_insert, False)
-                        diffs[pointer - count_delete -
-                              count_insert: pointer] = a
-                        pointer = pointer - count_delete - \
-                            count_insert + len(a)
-                    count_insert = 0
-                    count_delete = 0
-                    text_delete = ''
-                    text_insert = ''
-
-                pointer += 1
-
-            diffs.pop()  # Remove the dummy entry at the end.
-        return diffs
-
-    def diff_linesToChars(self, text1, text2):
-        """Split two texts into an array of strings.  Reduce the texts to a
-        string of hashes where each Unicode character represents one line.
-
-        Args:
-          text1: First string.
-          text2: Second string.
-
-        Returns:
-          Three element tuple, containing the encoded text1, the encoded text2 and
-          the array of unique strings.  The zeroth element of the array of unique
-          strings is intentionally blank.
-
-        """
-        lineArray = []  # e.g. lineArray[4] == "Hello\n"
-        lineHash = {}   # e.g. lineHash["Hello\n"] == 4
-
-        # "\x00" is a valid character, but various debuggers don't like it.
-        # So we'll insert a junk entry to avoid generating a null character.
-        lineArray.append('')
-
-        def diff_linesToCharsMunge(text):
-            """Split a text into an array of strings.  Reduce the texts to a
-            string of hashes where each Unicode character represents one line.
-            Modifies linearray and linehash through being a closure.
-
-            Args:
-              text: String to encode.
-
-            Returns:
-              Encoded string.
-
-            """
-            chars = []
-            # Walk the text, pulling out a substring for each line.
-            # text.split('\n') would would temporarily double our memory footprint.
-            # Modifying text would create many large strings to garbage
-            # collect.
-            lineStart = 0
-            lineEnd = -1
-            while lineEnd < len(text) - 1:
-                lineEnd = text.find('\n', lineStart)
-                if lineEnd == -1:
-                    lineEnd = len(text) - 1
-                line = text[lineStart:lineEnd + 1]
-                lineStart = lineEnd + 1
-
-                if line in lineHash:
-                    chars.append(unichr(lineHash[line]))
-                else:
-                    lineArray.append(line)
-                    lineHash[line] = len(lineArray) - 1
-                    chars.append(unichr(len(lineArray) - 1))
-            return ''.join(chars)
-
-        chars1 = diff_linesToCharsMunge(text1)
-        chars2 = diff_linesToCharsMunge(text2)
-        return (chars1, chars2, lineArray)
-
-    def diff_charsToLines(self, diffs, lineArray):
-        """Rehydrate the text in a diff from a string of line hashes to real
-        lines of text.
-
-        Args:
-          diffs: Array of diff tuples.
-          lineArray: Array of unique strings.
-
-        """
-        for x in xrange(len(diffs)):
-            text = []
-            for char in diffs[x][1]:
-                text.append(lineArray[ord(char)])
-            diffs[x] = (diffs[x][0], ''.join(text))
-
-    def diff_map(self, text1, text2):
-        """Explore the intersection points between the two texts.
-
-        Args:
-          text1: Old string to be diffed.
-          text2: New string to be diffed.
-
-        Returns:
-          Array of diff tuples or None if no diff available.
-
-        """
-
-        # Unlike in most languages, Python counts time in seconds.
-        s_end = time.time() + self.Diff_Timeout  # Don't run for too long.
-        # Cache the text lengths to prevent multiple calls.
-        text1_length = len(text1)
-        text2_length = len(text2)
-        max_d = text1_length + text2_length - 1
-        doubleEnd = self.Diff_DualThreshold * 2 < max_d
-        v_map1 = []
-        v_map2 = []
-        v1 = {}
-        v2 = {}
-        v1[1] = 0
-        v2[1] = 0
-        footsteps = {}
-        done = False
-        # If the total number of characters is odd, then the front path will
-        # collide with the reverse path.
-        front = (text1_length + text2_length) % 2
-        for d in xrange(max_d):
-            # Bail out if timeout reached.
-            if self.Diff_Timeout > 0 and time.time() > s_end:
-                return None
-
-            # Walk the front path one step.
-            v_map1.append({})
-            for k in xrange(-d, d + 1, 2):
-                if k == -d or k != d and v1[k - 1] < v1[k + 1]:
-                    x = v1[k + 1]
-                else:
-                    x = v1[k - 1] + 1
-                y = x - k
-                if doubleEnd:
-                    footstep = (x, y)
-                    if front and footstep in footsteps:
-                        done = True
-                    if not front:
-                        footsteps[footstep] = d
-
-                while (not done and x < text1_length and y < text2_length and
-                       text1[x] == text2[y]):
-                    x += 1
-                    y += 1
-                    if doubleEnd:
-                        footstep = (x, y)
-                        if front and footstep in footsteps:
-                            done = True
-                        if not front:
-                            footsteps[footstep] = d
-
-                v1[k] = x
-                v_map1[d][(x, y)] = True
-                if x == text1_length and y == text2_length:
-                    # Reached the end in single-path mode.
-                    return self.diff_path1(v_map1, text1, text2)
-                elif done:
-                    # Front path ran over reverse path.
-                    v_map2 = v_map2[:footsteps[footstep] + 1]
-                    a = self.diff_path1(v_map1, text1[:x], text2[:y])
-                    b = self.diff_path2(v_map2, text1[x:], text2[y:])
-                    return a + b
-
-            if doubleEnd:
-                # Walk the reverse path one step.
-                v_map2.append({})
-                for k in xrange(-d, d + 1, 2):
-                    if k == -d or k != d and v2[k - 1] < v2[k + 1]:
-                        x = v2[k + 1]
-                    else:
-                        x = v2[k - 1] + 1
-                    y = x - k
-                    footstep = (text1_length - x, text2_length - y)
-                    if not front and footstep in footsteps:
-                        done = True
-                    if front:
-                        footsteps[footstep] = d
-                    while (not done and x < text1_length and y < text2_length and
-                           text1[-x - 1] == text2[-y - 1]):
-                        x += 1
-                        y += 1
-                        footstep = (text1_length - x, text2_length - y)
-                        if not front and footstep in footsteps:
-                            done = True
-                        if front:
-                            footsteps[footstep] = d
-
-                    v2[k] = x
-                    v_map2[d][(x, y)] = True
-                    if done:
-                        # Reverse path ran over front path.
-                        v_map1 = v_map1[:footsteps[footstep] + 1]
-                        a = self.diff_path1(v_map1, text1[:text1_length - x],
-                                            text2[:text2_length - y])
-                        b = self.diff_path2(v_map2, text1[text1_length - x:],
-                                            text2[text2_length - y:])
-                        return a + b
-
-        # Number of diffs equals number of characters, no commonality at all.
+  """Class containing the diff, match and patch methods.
+
+  Also contains the behaviour settings.
+  """
+
+  def __init__(self):
+    """Inits a diff_match_patch object with default settings.
+    Redefine these in your program to override the defaults.
+    """
+
+    # Number of seconds to map a diff before giving up (0 for infinity).
+    self.Diff_Timeout = 1.0
+    # Cost of an empty edit operation in terms of edit characters.
+    self.Diff_EditCost = 4
+    # At what point is no match declared (0.0 = perfection, 1.0 = very loose).
+    self.Match_Threshold = 0.5
+    # How far to search for a match (0 = exact location, 1000+ = broad match).
+    # A match this many characters away from the expected location will add
+    # 1.0 to the score (0.0 is a perfect match).
+    self.Match_Distance = 1000
+    # When deleting a large block of text (over ~64 characters), how close do
+    # the contents have to be to match the expected contents. (0.0 = perfection,
+    # 1.0 = very loose).  Note that Match_Threshold controls how closely the
+    # end points of a delete need to match.
+    self.Patch_DeleteThreshold = 0.5
+    # Chunk size for context length.
+    self.Patch_Margin = 4
+
+    # The number of bits in an int.
+    # Python has no maximum, thus to disable patch splitting set to 0.
+    # However to avoid long patches in certain pathological cases, use 32.
+    # Multiple short patches (using native ints) are much faster than long ones.
+    self.Match_MaxBits = 32
+
+  #  DIFF FUNCTIONS
+
+  # The data structure representing a diff is an array of tuples:
+  # [(DIFF_DELETE, "Hello"), (DIFF_INSERT, "Goodbye"), (DIFF_EQUAL, " world.")]
+  # which means: delete "Hello", add "Goodbye" and keep " world."
+  DIFF_DELETE = -1
+  DIFF_INSERT = 1
+  DIFF_EQUAL = 0
+
+  def diff_main(self, text1, text2, checklines=True, deadline=None):
+    """Find the differences between two texts.  Simplifies the problem by
+      stripping any common prefix or suffix off the texts before diffing.
+
+    Args:
+      text1: Old string to be diffed.
+      text2: New string to be diffed.
+      checklines: Optional speedup flag.  If present and false, then don't run
+        a line-level diff first to identify the changed areas.
+        Defaults to true, which does a faster, slightly less optimal diff.
+      deadline: Optional time when the diff should be complete by.  Used
+        internally for recursive calls.  Users should set DiffTimeout instead.
+
+    Returns:
+      Array of changes.
+    """
+    # Set a deadline by which time the diff must be complete.
+    if deadline == None:
+      # Unlike in most languages, Python counts time in seconds.
+      if self.Diff_Timeout <= 0:
+        deadline = sys.maxsize
+      else:
+        deadline = time.time() + self.Diff_Timeout
+
+    # Check for null inputs.
+    if text1 == None or text2 == None:
+      raise ValueError("Null inputs. (diff_main)")
+
+    # Check for equality (speedup).
+    if text1 == text2:
+      if text1:
+        return [(self.DIFF_EQUAL, text1)]
+      return []
+
+    # Trim off common prefix (speedup).
+    commonlength = self.diff_commonPrefix(text1, text2)
+    commonprefix = text1[:commonlength]
+    text1 = text1[commonlength:]
+    text2 = text2[commonlength:]
+
+    # Trim off common suffix (speedup).
+    commonlength = self.diff_commonSuffix(text1, text2)
+    if commonlength == 0:
+      commonsuffix = ''
+    else:
+      commonsuffix = text1[-commonlength:]
+      text1 = text1[:-commonlength]
+      text2 = text2[:-commonlength]
+
+    # Compute the diff on the middle block.
+    diffs = self.diff_compute(text1, text2, checklines, deadline)
+
+    # Restore the prefix and suffix.
+    if commonprefix:
+      diffs[:0] = [(self.DIFF_EQUAL, commonprefix)]
+    if commonsuffix:
+      diffs.append((self.DIFF_EQUAL, commonsuffix))
+    self.diff_cleanupMerge(diffs)
+    return diffs
+
+  def diff_compute(self, text1, text2, checklines, deadline):
+    """Find the differences between two texts.  Assumes that the texts do not
+      have any common prefix or suffix.
+
+    Args:
+      text1: Old string to be diffed.
+      text2: New string to be diffed.
+      checklines: Speedup flag.  If false, then don't run a line-level diff
+        first to identify the changed areas.
+        If true, then run a faster, slightly less optimal diff.
+      deadline: Time when the diff should be complete by.
+
+    Returns:
+      Array of changes.
+    """
+    if not text1:
+      # Just add some text (speedup).
+      return [(self.DIFF_INSERT, text2)]
+
+    if not text2:
+      # Just delete some text (speedup).
+      return [(self.DIFF_DELETE, text1)]
+
+    if len(text1) > len(text2):
+      (longtext, shorttext) = (text1, text2)
+    else:
+      (shorttext, longtext) = (text1, text2)
+    i = longtext.find(shorttext)
+    if i != -1:
+      # Shorter text is inside the longer text (speedup).
+      diffs = [(self.DIFF_INSERT, longtext[:i]), (self.DIFF_EQUAL, shorttext),
+               (self.DIFF_INSERT, longtext[i + len(shorttext):])]
+      # Swap insertions for deletions if diff is reversed.
+      if len(text1) > len(text2):
+        diffs[0] = (self.DIFF_DELETE, diffs[0][1])
+        diffs[2] = (self.DIFF_DELETE, diffs[2][1])
+      return diffs
+
+    if len(shorttext) == 1:
+      # Single character string.
+      # After the previous speedup, the character can't be an equality.
+      return [(self.DIFF_DELETE, text1), (self.DIFF_INSERT, text2)]
+
+    # Check to see if the problem can be split in two.
+    hm = self.diff_halfMatch(text1, text2)
+    if hm:
+      # A half-match was found, sort out the return data.
+      (text1_a, text1_b, text2_a, text2_b, mid_common) = hm
+      # Send both pairs off for separate processing.
+      diffs_a = self.diff_main(text1_a, text2_a, checklines, deadline)
+      diffs_b = self.diff_main(text1_b, text2_b, checklines, deadline)
+      # Merge the results.
+      return diffs_a + [(self.DIFF_EQUAL, mid_common)] + diffs_b
+
+    if checklines and len(text1) > 100 and len(text2) > 100:
+      return self.diff_lineMode(text1, text2, deadline)
+
+    return self.diff_bisect(text1, text2, deadline)
+
+  def diff_lineMode(self, text1, text2, deadline):
+    """Do a quick line-level diff on both strings, then rediff the parts for
+      greater accuracy.
+      This speedup can produce non-minimal diffs.
+
+    Args:
+      text1: Old string to be diffed.
+      text2: New string to be diffed.
+      deadline: Time when the diff should be complete by.
+
+    Returns:
+      Array of changes.
+    """
+
+    # Scan the text on a line-by-line basis first.
+    (text1, text2, linearray) = self.diff_linesToChars(text1, text2)
+
+    diffs = self.diff_main(text1, text2, False, deadline)
+
+    # Convert the diff back to original text.
+    self.diff_charsToLines(diffs, linearray)
+    # Eliminate freak matches (e.g. blank lines)
+    self.diff_cleanupSemantic(diffs)
+
+    # Rediff any replacement blocks, this time character-by-character.
+    # Add a dummy entry at the end.
+    diffs.append((self.DIFF_EQUAL, ''))
+    pointer = 0
+    count_delete = 0
+    count_insert = 0
+    text_delete = ''
+    text_insert = ''
+    while pointer < len(diffs):
+      if diffs[pointer][0] == self.DIFF_INSERT:
+        count_insert += 1
+        text_insert += diffs[pointer][1]
+      elif diffs[pointer][0] == self.DIFF_DELETE:
+        count_delete += 1
+        text_delete += diffs[pointer][1]
+      elif diffs[pointer][0] == self.DIFF_EQUAL:
+        # Upon reaching an equality, check for prior redundancies.
+        if count_delete >= 1 and count_insert >= 1:
+          # Delete the offending records and add the merged ones.
+          subDiff = self.diff_main(text_delete, text_insert, False, deadline)
+          diffs[pointer - count_delete - count_insert : pointer] = subDiff
+          pointer = pointer - count_delete - count_insert + len(subDiff)
+        count_insert = 0
+        count_delete = 0
+        text_delete = ''
+        text_insert = ''
+
+      pointer += 1
+
+    diffs.pop()  # Remove the dummy entry at the end.
+
+    return diffs
+
+  def diff_bisect(self, text1, text2, deadline):
+    """Find the 'middle snake' of a diff, split the problem in two
+      and return the recursively constructed diff.
+      See Myers 1986 paper: An O(ND) Difference Algorithm and Its Variations.
+
+    Args:
+      text1: Old string to be diffed.
+      text2: New string to be diffed.
+      deadline: Time at which to bail if not yet complete.
+
+    Returns:
+      Array of diff tuples.
+    """
+
+    # Cache the text lengths to prevent multiple calls.
+    text1_length = len(text1)
+    text2_length = len(text2)
+    max_d = (text1_length + text2_length + 1) // 2
+    v_offset = max_d
+    v_length = 2 * max_d
+    v1 = [-1] * v_length
+    v1[v_offset + 1] = 0
+    v2 = v1[:]
+    delta = text1_length - text2_length
+    # If the total number of characters is odd, then the front path will
+    # collide with the reverse path.
+    front = (delta % 2 != 0)
+    # Offsets for start and end of k loop.
+    # Prevents mapping of space beyond the grid.
+    k1start = 0
+    k1end = 0
+    k2start = 0
+    k2end = 0
+    for d in range(max_d):
+      # Bail out if deadline is reached.
+      if time.time() > deadline:
+        break
+
+      # Walk the front path one step.
+      for k1 in range(-d + k1start, d + 1 - k1end, 2):
+        k1_offset = v_offset + k1
+        if k1 == -d or (k1 != d and
+            v1[k1_offset - 1] < v1[k1_offset + 1]):
+          x1 = v1[k1_offset + 1]
+        else:
+          x1 = v1[k1_offset - 1] + 1
+        y1 = x1 - k1
+        while (x1 < text1_length and y1 < text2_length and
+               text1[x1] == text2[y1]):
+          x1 += 1
+          y1 += 1
+        v1[k1_offset] = x1
+        if x1 > text1_length:
+          # Ran off the right of the graph.
+          k1end += 2
+        elif y1 > text2_length:
+          # Ran off the bottom of the graph.
+          k1start += 2
+        elif front:
+          k2_offset = v_offset + delta - k1
+          if k2_offset >= 0 and k2_offset < v_length and v2[k2_offset] != -1:
+            # Mirror x2 onto top-left coordinate system.
+            x2 = text1_length - v2[k2_offset]
+            if x1 >= x2:
+              # Overlap detected.
+              return self.diff_bisectSplit(text1, text2, x1, y1, deadline)
+
+      # Walk the reverse path one step.
+      for k2 in range(-d + k2start, d + 1 - k2end, 2):
+        k2_offset = v_offset + k2
+        if k2 == -d or (k2 != d and
+            v2[k2_offset - 1] < v2[k2_offset + 1]):
+          x2 = v2[k2_offset + 1]
+        else:
+          x2 = v2[k2_offset - 1] + 1
+        y2 = x2 - k2
+        while (x2 < text1_length and y2 < text2_length and
+               text1[-x2 - 1] == text2[-y2 - 1]):
+          x2 += 1
+          y2 += 1
+        v2[k2_offset] = x2
+        if x2 > text1_length:
+          # Ran off the left of the graph.
+          k2end += 2
+        elif y2 > text2_length:
+          # Ran off the top of the graph.
+          k2start += 2
+        elif not front:
+          k1_offset = v_offset + delta - k2
+          if k1_offset >= 0 and k1_offset < v_length and v1[k1_offset] != -1:
+            x1 = v1[k1_offset]
+            y1 = v_offset + x1 - k1_offset
+            # Mirror x2 onto top-left coordinate system.
+            x2 = text1_length - x2
+            if x1 >= x2:
+              # Overlap detected.
+              return self.diff_bisectSplit(text1, text2, x1, y1, deadline)
+
+    # Diff took too long and hit the deadline or
+    # number of diffs equals number of characters, no commonality at all.
+    return [(self.DIFF_DELETE, text1), (self.DIFF_INSERT, text2)]
+
+  def diff_bisectSplit(self, text1, text2, x, y, deadline):
+    """Given the location of the 'middle snake', split the diff in two parts
+    and recurse.
+
+    Args:
+      text1: Old string to be diffed.
+      text2: New string to be diffed.
+      x: Index of split point in text1.
+      y: Index of split point in text2.
+      deadline: Time at which to bail if not yet complete.
+
+    Returns:
+      Array of diff tuples.
+    """
+    text1a = text1[:x]
+    text2a = text2[:y]
+    text1b = text1[x:]
+    text2b = text2[y:]
+
+    # Compute both diffs serially.
+    diffs = self.diff_main(text1a, text2a, False, deadline)
+    diffsb = self.diff_main(text1b, text2b, False, deadline)
+
+    return diffs + diffsb
+
+  def diff_linesToChars(self, text1, text2):
+    """Split two texts into an array of strings.  Reduce the texts to a string
+    of hashes where each Unicode character represents one line.
+
+    Args:
+      text1: First string.
+      text2: Second string.
+
+    Returns:
+      Three element tuple, containing the encoded text1, the encoded text2 and
+      the array of unique strings.  The zeroth element of the array of unique
+      strings is intentionally blank.
+    """
+    lineArray = []  # e.g. lineArray[4] == "Hello\n"
+    lineHash = {}   # e.g. lineHash["Hello\n"] == 4
+
+    # "\x00" is a valid character, but various debuggers don't like it.
+    # So we'll insert a junk entry to avoid generating a null character.
+    lineArray.append('')
+
+    def diff_linesToCharsMunge(text):
+      """Split a text into an array of strings.  Reduce the texts to a string
+      of hashes where each Unicode character represents one line.
+      Modifies linearray and linehash through being a closure.
+
+      Args:
+        text: String to encode.
+
+      Returns:
+        Encoded string.
+      """
+      chars = []
+      # Walk the text, pulling out a substring for each line.
+      # text.split('\n') would would temporarily double our memory footprint.
+      # Modifying text would create many large strings to garbage collect.
+      lineStart = 0
+      lineEnd = -1
+      while lineEnd < len(text) - 1:
+        lineEnd = text.find('\n', lineStart)
+        if lineEnd == -1:
+          lineEnd = len(text) - 1
+        line = text[lineStart:lineEnd + 1]
+
+        if line in lineHash:
+          chars.append(chr(lineHash[line]))
+        else:
+          if len(lineArray) == maxLines:
+            # Bail out at 1114111 because chr(1114112) throws.
+            line = text[lineStart:]
+            lineEnd = len(text)
+          lineArray.append(line)
+          lineHash[line] = len(lineArray) - 1
+          chars.append(chr(len(lineArray) - 1))
+        lineStart = lineEnd + 1
+      return "".join(chars)
+
+    # Allocate 2/3rds of the space for text1, the rest for text2.
+    maxLines = 666666
+    chars1 = diff_linesToCharsMunge(text1)
+    maxLines = 1114111
+    chars2 = diff_linesToCharsMunge(text2)
+    return (chars1, chars2, lineArray)
+
+  def diff_charsToLines(self, diffs, lineArray):
+    """Rehydrate the text in a diff from a string of line hashes to real lines
+    of text.
+
+    Args:
+      diffs: Array of diff tuples.
+      lineArray: Array of unique strings.
+    """
+    for i in range(len(diffs)):
+      text = []
+      for char in diffs[i][1]:
+        text.append(lineArray[ord(char)])
+      diffs[i] = (diffs[i][0], "".join(text))
+
+  def diff_commonPrefix(self, text1, text2):
+    """Determine the common prefix of two strings.
+
+    Args:
+      text1: First string.
+      text2: Second string.
+
+    Returns:
+      The number of characters common to the start of each string.
+    """
+    # Quick check for common null cases.
+    if not text1 or not text2 or text1[0] != text2[0]:
+      return 0
+    # Binary search.
+    # Performance analysis: https://neil.fraser.name/news/2007/10/09/
+    pointermin = 0
+    pointermax = min(len(text1), len(text2))
+    pointermid = pointermax
+    pointerstart = 0
+    while pointermin < pointermid:
+      if text1[pointerstart:pointermid] == text2[pointerstart:pointermid]:
+        pointermin = pointermid
+        pointerstart = pointermin
+      else:
+        pointermax = pointermid
+      pointermid = (pointermax - pointermin) // 2 + pointermin
+    return pointermid
+
+  def diff_commonSuffix(self, text1, text2):
+    """Determine the common suffix of two strings.
+
+    Args:
+      text1: First string.
+      text2: Second string.
+
+    Returns:
+      The number of characters common to the end of each string.
+    """
+    # Quick check for common null cases.
+    if not text1 or not text2 or text1[-1] != text2[-1]:
+      return 0
+    # Binary search.
+    # Performance analysis: https://neil.fraser.name/news/2007/10/09/
+    pointermin = 0
+    pointermax = min(len(text1), len(text2))
+    pointermid = pointermax
+    pointerend = 0
+    while pointermin < pointermid:
+      if (text1[-pointermid:len(text1) - pointerend] ==
+          text2[-pointermid:len(text2) - pointerend]):
+        pointermin = pointermid
+        pointerend = pointermin
+      else:
+        pointermax = pointermid
+      pointermid = (pointermax - pointermin) // 2 + pointermin
+    return pointermid
+
+  def diff_commonOverlap(self, text1, text2):
+    """Determine if the suffix of one string is the prefix of another.
+
+    Args:
+      text1 First string.
+      text2 Second string.
+
+    Returns:
+      The number of characters common to the end of the first
+      string and the start of the second string.
+    """
+    # Cache the text lengths to prevent multiple calls.
+    text1_length = len(text1)
+    text2_length = len(text2)
+    # Eliminate the null case.
+    if text1_length == 0 or text2_length == 0:
+      return 0
+    # Truncate the longer string.
+    if text1_length > text2_length:
+      text1 = text1[-text2_length:]
+    elif text1_length < text2_length:
+      text2 = text2[:text1_length]
+    text_length = min(text1_length, text2_length)
+    # Quick check for the worst case.
+    if text1 == text2:
+      return text_length
+
+    # Start by looking for a single character match
+    # and increase length until no match is found.
+    # Performance analysis: https://neil.fraser.name/news/2010/11/04/
+    best = 0
+    length = 1
+    while True:
+      pattern = text1[-length:]
+      found = text2.find(pattern)
+      if found == -1:
+        return best
+      length += found
+      if found == 0 or text1[-length:] == text2[:length]:
+        best = length
+        length += 1
+
+  def diff_halfMatch(self, text1, text2):
+    """Do the two texts share a substring which is at least half the length of
+    the longer text?
+    This speedup can produce non-minimal diffs.
+
+    Args:
+      text1: First string.
+      text2: Second string.
+
+    Returns:
+      Five element Array, containing the prefix of text1, the suffix of text1,
+      the prefix of text2, the suffix of text2 and the common middle.  Or None
+      if there was no match.
+    """
+    if self.Diff_Timeout <= 0:
+      # Don't risk returning a non-optimal diff if we have unlimited time.
+      return None
+    if len(text1) > len(text2):
+      (longtext, shorttext) = (text1, text2)
+    else:
+      (shorttext, longtext) = (text1, text2)
+    if len(longtext) < 4 or len(shorttext) * 2 < len(longtext):
+      return None  # Pointless.
+
+    def diff_halfMatchI(longtext, shorttext, i):
+      """Does a substring of shorttext exist within longtext such that the
+      substring is at least half the length of longtext?
+      Closure, but does not reference any external variables.
+
+      Args:
+        longtext: Longer string.
+        shorttext: Shorter string.
+        i: Start index of quarter length substring within longtext.
+
+      Returns:
+        Five element Array, containing the prefix of longtext, the suffix of
+        longtext, the prefix of shorttext, the suffix of shorttext and the
+        common middle.  Or None if there was no match.
+      """
+      seed = longtext[i:i + len(longtext) // 4]
+      best_common = ''
+      j = shorttext.find(seed)
+      while j != -1:
+        prefixLength = self.diff_commonPrefix(longtext[i:], shorttext[j:])
+        suffixLength = self.diff_commonSuffix(longtext[:i], shorttext[:j])
+        if len(best_common) < suffixLength + prefixLength:
+          best_common = (shorttext[j - suffixLength:j] +
+              shorttext[j:j + prefixLength])
+          best_longtext_a = longtext[:i - suffixLength]
+          best_longtext_b = longtext[i + prefixLength:]
+          best_shorttext_a = shorttext[:j - suffixLength]
+          best_shorttext_b = shorttext[j + prefixLength:]
+        j = shorttext.find(seed, j + 1)
+
+      if len(best_common) * 2 >= len(longtext):
+        return (best_longtext_a, best_longtext_b,
+                best_shorttext_a, best_shorttext_b, best_common)
+      else:
         return None
 
-    def diff_path1(self, v_map, text1, text2):
-        """Work from the middle back to the start to determine the path.
-
-        Args:
-          v_map: Array of paths.
-          text1: Old string fragment to be diffed.
-          text2: New string fragment to be diffed.
-
-        Returns:
-          Array of diff tuples.
-
-        """
-        path = []
-        x = len(text1)
-        y = len(text2)
-        last_op = None
-        for d in xrange(len(v_map) - 2, -1, -1):
-            while True:
-                if (x - 1, y) in v_map[d]:
-                    x -= 1
-                    if last_op == self.DIFF_DELETE:
-                        path[0] = (self.DIFF_DELETE, text1[x] + path[0][1])
-                    else:
-                        path[:0] = [(self.DIFF_DELETE, text1[x])]
-                    last_op = self.DIFF_DELETE
-                    break
-                elif (x, y - 1) in v_map[d]:
-                    y -= 1
-                    if last_op == self.DIFF_INSERT:
-                        path[0] = (self.DIFF_INSERT, text2[y] + path[0][1])
-                    else:
-                        path[:0] = [(self.DIFF_INSERT, text2[y])]
-                    last_op = self.DIFF_INSERT
-                    break
-                else:
-                    x -= 1
-                    y -= 1
-                    assert text1[x] == text2[y], ('No diagonal.  ' +
-                                                  "Can't happen. (diff_path1)")
-                    if last_op == self.DIFF_EQUAL:
-                        path[0] = (self.DIFF_EQUAL, text1[x] + path[0][1])
-                    else:
-                        path[:0] = [(self.DIFF_EQUAL, text1[x])]
-                    last_op = self.DIFF_EQUAL
-        return path
-
-    def diff_path2(self, v_map, text1, text2):
-        """Work from the middle back to the end to determine the path.
-
-        Args:
-          v_map: Array of paths.
-          text1: Old string fragment to be diffed.
-          text2: New string fragment to be diffed.
-
-        Returns:
-          Array of diff tuples.
-
-        """
-        path = []
-        x = len(text1)
-        y = len(text2)
-        last_op = None
-        for d in xrange(len(v_map) - 2, -1, -1):
-            while True:
-                if (x - 1, y) in v_map[d]:
-                    x -= 1
-                    if last_op == self.DIFF_DELETE:
-                        path[-1] = (self.DIFF_DELETE, path[-1]
-                                    [1] + text1[-x - 1])
-                    else:
-                        path.append((self.DIFF_DELETE, text1[-x - 1]))
-                    last_op = self.DIFF_DELETE
-                    break
-                elif (x, y - 1) in v_map[d]:
-                    y -= 1
-                    if last_op == self.DIFF_INSERT:
-                        path[-1] = (self.DIFF_INSERT, path[-1]
-                                    [1] + text2[-y - 1])
-                    else:
-                        path.append((self.DIFF_INSERT, text2[-y - 1]))
-                    last_op = self.DIFF_INSERT
-                    break
-                else:
-                    x -= 1
-                    y -= 1
-                    assert text1[-x - 1] == text2[-y - 1], ('No diagonal.  ' +
-                                                            "Can't happen. (diff_path2)")
-                    if last_op == self.DIFF_EQUAL:
-                        path[-1] = (self.DIFF_EQUAL, path[-1]
-                                    [1] + text1[-x - 1])
-                    else:
-                        path.append((self.DIFF_EQUAL, text1[-x - 1]))
-                    last_op = self.DIFF_EQUAL
-        return path
-
-    def diff_commonPrefix(self, text1, text2):
-        """Determine the common prefix of two strings.
-
-        Args:
-          text1: First string.
-          text2: Second string.
-
-        Returns:
-          The number of characters common to the start of each string.
-
-        """
-        # Quick check for common null cases.
-        if not text1 or not text2 or text1[0] != text2[0]:
-            return 0
-        # Binary search.
-        # Performance analysis: http://neil.fraser.name/news/2007/10/09/
-        pointermin = 0
-        pointermax = min(len(text1), len(text2))
-        pointermid = pointermax
-        pointerstart = 0
-        while pointermin < pointermid:
-            if text1[pointerstart:pointermid] == text2[pointerstart:pointermid]:
-                pointermin = pointermid
-                pointerstart = pointermin
-            else:
-                pointermax = pointermid
-            pointermid = int((pointermax - pointermin) / 2 + pointermin)
-        return pointermid
-
-    def diff_commonSuffix(self, text1, text2):
-        """Determine the common suffix of two strings.
-
-        Args:
-          text1: First string.
-          text2: Second string.
-
-        Returns:
-          The number of characters common to the end of each string.
-
-        """
-        # Quick check for common null cases.
-        if not text1 or not text2 or text1[-1] != text2[-1]:
-            return 0
-        # Binary search.
-        # Performance analysis: http://neil.fraser.name/news/2007/10/09/
-        pointermin = 0
-        pointermax = min(len(text1), len(text2))
-        pointermid = pointermax
-        pointerend = 0
-        while pointermin < pointermid:
-            if (text1[-pointermid:len(text1) - pointerend] ==
-                    text2[-pointermid:len(text2) - pointerend]):
-                pointermin = pointermid
-                pointerend = pointermin
-            else:
-                pointermax = pointermid
-            pointermid = int((pointermax - pointermin) / 2 + pointermin)
-        return pointermid
-
-    def diff_halfMatch(self, text1, text2):
-        """Do the two texts share a substring which is at least half the length
-        of the longer text?
-
-        Args:
-          text1: First string.
-          text2: Second string.
-
-        Returns:
-          Five element Array, containing the prefix of text1, the suffix of text1,
-          the prefix of text2, the suffix of text2 and the common middle.  Or None
-          if there was no match.
-
-        """
-        if len(text1) > len(text2):
-            (longtext, shorttext) = (text1, text2)
-        else:
-            (shorttext, longtext) = (text1, text2)
-        if len(longtext) < 10 or len(shorttext) < 1:
-            return None  # Pointless.
-
-        def diff_halfMatchI(longtext, shorttext, i):
-            """Does a substring of shorttext exist within longtext such that
-            the substring is at least half the length of longtext? Closure, but
-            does not reference any external variables.
-
-            Args:
-              longtext: Longer string.
-              shorttext: Shorter string.
-              i: Start index of quarter length substring within longtext.
-
-            Returns:
-              Five element Array, containing the prefix of longtext, the suffix of
-              longtext, the prefix of shorttext, the suffix of shorttext and the
-              common middle.  Or None if there was no match.
-
-            """
-            seed = longtext[i:i + len(longtext) / 4]
-            best_common = ''
-            j = shorttext.find(seed)
-            while j != -1:
-                prefixLength = self.diff_commonPrefix(
-                    longtext[i:], shorttext[j:])
-                suffixLength = self.diff_commonSuffix(
-                    longtext[:i], shorttext[:j])
-                if len(best_common) < suffixLength + prefixLength:
-                    best_common = (shorttext[j - suffixLength:j] +
-                                   shorttext[j:j + prefixLength])
-                    best_longtext_a = longtext[:i - suffixLength]
-                    best_longtext_b = longtext[i + prefixLength:]
-                    best_shorttext_a = shorttext[:j - suffixLength]
-                    best_shorttext_b = shorttext[j + prefixLength:]
-                j = shorttext.find(seed, j + 1)
-
-            if len(best_common) >= len(longtext) / 2:
-                return (best_longtext_a, best_longtext_b,
-                        best_shorttext_a, best_shorttext_b, best_common)
-            else:
-                return None
-
-        # First check if the second quarter is the seed for a half-match.
-        hm1 = diff_halfMatchI(longtext, shorttext, (len(longtext) + 3) / 4)
-        # Check again based on the third quarter.
-        hm2 = diff_halfMatchI(longtext, shorttext, (len(longtext) + 1) / 2)
-        if not hm1 and not hm2:
-            return None
-        elif not hm2:
-            hm = hm1
-        elif not hm1:
-            hm = hm2
-        else:
-            # Both matched.  Select the longest.
-            if len(hm1[4]) > len(hm2[4]):
-                hm = hm1
-            else:
-                hm = hm2
-
-        # A half-match was found, sort out the return data.
-        if len(text1) > len(text2):
-            (text1_a, text1_b, text2_a, text2_b, mid_common) = hm
-        else:
-            (text2_a, text2_b, text1_a, text1_b, mid_common) = hm
-        return (text1_a, text1_b, text2_a, text2_b, mid_common)
-
-    def diff_cleanupSemantic(self, diffs):
-        """Reduce the number of edits by eliminating semantically trivial
-        equalities.
-
-        Args:
-          diffs: Array of diff tuples.
-
-        """
-        changes = False
-        equalities = []  # Stack of indices where equalities are found.
-        lastequality = None  # Always equal to equalities[-1][1]
-        pointer = 0  # Index of current position.
-        # Number of chars that changed prior to the equality.
-        length_changes1 = 0
-        length_changes2 = 0  # Number of chars that changed after the equality.
-        while pointer < len(diffs):
-            if diffs[pointer][0] == self.DIFF_EQUAL:  # equality found
-                equalities.append(pointer)
-                length_changes1 = length_changes2
-                length_changes2 = 0
-                lastequality = diffs[pointer][1]
-            else:  # an insertion or deletion
-                length_changes2 += len(diffs[pointer][1])
-                if (lastequality != None and (len(lastequality) <= length_changes1) and
-                        (len(lastequality) <= length_changes2)):
-                    # Duplicate record
-                    diffs.insert(equalities[-1],
-                                 (self.DIFF_DELETE, lastequality))
-                    # Change second copy to insert.
-                    diffs[equalities[-1] + 1] = (self.DIFF_INSERT,
-                                                 diffs[equalities[-1] + 1][1])
-                    # Throw away the equality we just deleted.
-                    equalities.pop()
-                    # Throw away the previous equality (it needs to be
-                    # reevaluated).
-                    if len(equalities) != 0:
-                        equalities.pop()
-                    if len(equalities):
-                        pointer = equalities[-1]
-                    else:
-                        pointer = -1
-                    length_changes1 = 0  # Reset the counters.
-                    length_changes2 = 0
-                    lastequality = None
-                    changes = True
-            pointer += 1
-
-        if changes:
-            self.diff_cleanupMerge(diffs)
-
-        self.diff_cleanupSemanticLossless(diffs)
-
-    def diff_cleanupSemanticLossless(self, diffs):
-        """Look for single edits surrounded on both sides by equalities
-        which can be shifted sideways to align the edit to a word boundary.
-        e.g: The c<ins>at c</ins>ame. -> The <ins>cat </ins>came.
-
-        Args:
-          diffs: Array of diff tuples.
-        """
-
-        def diff_cleanupSemanticScore(one, two):
-            """Given two strings, compute a score representing whether the
-            internal boundary falls on logical boundaries. Scores range from 5
-            (best) to 0 (worst). Closure, but does not reference any external
-            variables.
-
-            Args:
-              one: First string.
-              two: Second string.
-
-            Returns:
-              The score.
-
-            """
-            if not one or not two:
-                # Edges are the best.
-                return 5
-
-            # Each port of this function behaves slightly differently due to
-            # subtle differences in each language's definition of things like
-            # 'whitespace'.  Since this function's purpose is largely cosmetic,
-            # the choice has been made to use each language's native features
-            # rather than force total conformity.
-            score = 0
-            # One point for non-alphanumeric.
-            if not one[-1].isalnum() or not two[0].isalnum():
-                score += 1
-                # Two points for whitespace.
-                if one[-1].isspace() or two[0].isspace():
-                    score += 1
-                    # Three points for line breaks.
-                    if (one[-1] == '\r' or one[-1] == '\n' or
-                            two[0] == '\r' or two[0] == '\n'):
-                        score += 1
-                        # Four points for blank lines.
-                        if (re.search('\\n\\r?\\n$', one) or
-                                re.match('^\\r?\\n\\r?\\n', two)):
-                            score += 1
-            return score
-
-        pointer = 1
-        # Intentionally ignore the first and last element (don't need
-        # checking).
-        while pointer < len(diffs) - 1:
-            if (diffs[pointer - 1][0] == self.DIFF_EQUAL and
-                    diffs[pointer + 1][0] == self.DIFF_EQUAL):
-                # This is a single edit surrounded by equalities.
-                equality1 = diffs[pointer - 1][1]
-                edit = diffs[pointer][1]
-                equality2 = diffs[pointer + 1][1]
-
-                # First, shift the edit as far left as possible.
-                commonOffset = self.diff_commonSuffix(equality1, edit)
-                if commonOffset:
-                    commonString = edit[-commonOffset:]
-                    equality1 = equality1[:-commonOffset]
-                    edit = commonString + edit[:-commonOffset]
-                    equality2 = commonString + equality2
-
-                # Second, step character by character right, looking for the
-                # best fit.
-                bestEquality1 = equality1
-                bestEdit = edit
-                bestEquality2 = equality2
-                bestScore = (diff_cleanupSemanticScore(equality1, edit) +
-                             diff_cleanupSemanticScore(edit, equality2))
-                while edit and equality2 and edit[0] == equality2[0]:
-                    equality1 += edit[0]
-                    edit = edit[1:] + equality2[0]
-                    equality2 = equality2[1:]
-                    score = (diff_cleanupSemanticScore(equality1, edit) +
-                             diff_cleanupSemanticScore(edit, equality2))
-                    # The >= encourages trailing rather than leading whitespace
-                    # on edits.
-                    if score >= bestScore:
-                        bestScore = score
-                        bestEquality1 = equality1
-                        bestEdit = edit
-                        bestEquality2 = equality2
-
-                if diffs[pointer - 1][1] != bestEquality1:
-                    # We have an improvement, save it back to the diff.
-                    if bestEquality1:
-                        diffs[pointer - 1] = (diffs[pointer - 1]
-                                              [0], bestEquality1)
-                    else:
-                        del diffs[pointer - 1]
-                        pointer -= 1
-                    diffs[pointer] = (diffs[pointer][0], bestEdit)
-                    if bestEquality2:
-                        diffs[pointer + 1] = (diffs[pointer + 1]
-                                              [0], bestEquality2)
-                    else:
-                        del diffs[pointer + 1]
-                        pointer -= 1
-            pointer += 1
-
-    def diff_cleanupEfficiency(self, diffs):
-        """Reduce the number of edits by eliminating operationally trivial
-        equalities.
-
-        Args:
-          diffs: Array of diff tuples.
-
-        """
-        changes = False
-        equalities = []  # Stack of indices where equalities are found.
-        lastequality = ''  # Always equal to equalities[-1][1]
-        pointer = 0  # Index of current position.
-        # Is there an insertion operation before the last equality.
-        pre_ins = False
-        # Is there a deletion operation before the last equality.
-        pre_del = False
-        # Is there an insertion operation after the last equality.
-        post_ins = False
-        # Is there a deletion operation after the last equality.
-        post_del = False
-        while pointer < len(diffs):
-            if diffs[pointer][0] == self.DIFF_EQUAL:  # equality found
-                if (len(diffs[pointer][1]) < self.Diff_EditCost and
-                        (post_ins or post_del)):
-                    # Candidate found.
-                    equalities.append(pointer)
-                    pre_ins = post_ins
-                    pre_del = post_del
-                    lastequality = diffs[pointer][1]
-                else:
-                    # Not a candidate, and can never become one.
-                    equalities = []
-                    lastequality = ''
-
-                post_ins = post_del = False
-            else:  # an insertion or deletion
-                if diffs[pointer][0] == self.DIFF_DELETE:
-                    post_del = True
-                else:
-                    post_ins = True
-
-                # Five types to be split:
-                # <ins>A</ins><del>B</del>XY<ins>C</ins><del>D</del>
-                # <ins>A</ins>X<ins>C</ins><del>D</del>
-                # <ins>A</ins><del>B</del>X<ins>C</ins>
-                # <ins>A</del>X<ins>C</ins><del>D</del>
-                # <ins>A</ins><del>B</del>X<del>C</del>
-
-                if lastequality and ((pre_ins and pre_del and post_ins and post_del) or
-                                     ((len(lastequality) < self.Diff_EditCost / 2) and
-                                      (pre_ins + pre_del + post_ins + post_del) == 3)):
-                    # Duplicate record
-                    diffs.insert(equalities[-1],
-                                 (self.DIFF_DELETE, lastequality))
-                    # Change second copy to insert.
-                    diffs[equalities[-1] + 1] = (self.DIFF_INSERT,
-                                                 diffs[equalities[-1] + 1][1])
-                    equalities.pop()  # Throw away the equality we just deleted
-                    lastequality = ''
-                    if pre_ins and pre_del:
-                        # No changes made which could affect previous entry,
-                        # keep going.
-                        post_ins = post_del = True
-                        equalities = []
-                    else:
-                        if len(equalities):
-                            equalities.pop()  # Throw away the previous equality
-                        if len(equalities):
-                            pointer = equalities[-1]
-                        else:
-                            pointer = -1
-                        post_ins = post_del = False
-                    changes = True
-            pointer += 1
-
-        if changes:
-            self.diff_cleanupMerge(diffs)
-
-    def diff_cleanupMerge(self, diffs):
-        """Reorder and merge like edit sections.  Merge equalities. Any edit
-        section can move as long as it doesn't cross an equality.
-
-        Args:
-          diffs: Array of diff tuples.
-
-        """
-        diffs.append((self.DIFF_EQUAL, ''))  # Add a dummy entry at the end.
-        pointer = 0
+    # First check if the second quarter is the seed for a half-match.
+    hm1 = diff_halfMatchI(longtext, shorttext, (len(longtext) + 3) // 4)
+    # Check again based on the third quarter.
+    hm2 = diff_halfMatchI(longtext, shorttext, (len(longtext) + 1) // 2)
+    if not hm1 and not hm2:
+      return None
+    elif not hm2:
+      hm = hm1
+    elif not hm1:
+      hm = hm2
+    else:
+      # Both matched.  Select the longest.
+      if len(hm1[4]) > len(hm2[4]):
+        hm = hm1
+      else:
+        hm = hm2
+
+    # A half-match was found, sort out the return data.
+    if len(text1) > len(text2):
+      (text1_a, text1_b, text2_a, text2_b, mid_common) = hm
+    else:
+      (text2_a, text2_b, text1_a, text1_b, mid_common) = hm
+    return (text1_a, text1_b, text2_a, text2_b, mid_common)
+
+  def diff_cleanupSemantic(self, diffs):
+    """Reduce the number of edits by eliminating semantically trivial
+    equalities.
+
+    Args:
+      diffs: Array of diff tuples.
+    """
+    changes = False
+    equalities = []  # Stack of indices where equalities are found.
+    lastEquality = None  # Always equal to diffs[equalities[-1]][1]
+    pointer = 0  # Index of current position.
+    # Number of chars that changed prior to the equality.
+    length_insertions1, length_deletions1 = 0, 0
+    # Number of chars that changed after the equality.
+    length_insertions2, length_deletions2 = 0, 0
+    while pointer < len(diffs):
+      if diffs[pointer][0] == self.DIFF_EQUAL:  # Equality found.
+        equalities.append(pointer)
+        length_insertions1, length_insertions2 = length_insertions2, 0
+        length_deletions1, length_deletions2 = length_deletions2, 0
+        lastEquality = diffs[pointer][1]
+      else:  # An insertion or deletion.
+        if diffs[pointer][0] == self.DIFF_INSERT:
+          length_insertions2 += len(diffs[pointer][1])
+        else:
+          length_deletions2 += len(diffs[pointer][1])
+        # Eliminate an equality that is smaller or equal to the edits on both
+        # sides of it.
+        if (lastEquality and (len(lastEquality) <=
+            max(length_insertions1, length_deletions1)) and
+            (len(lastEquality) <= max(length_insertions2, length_deletions2))):
+          # Duplicate record.
+          diffs.insert(equalities[-1], (self.DIFF_DELETE, lastEquality))
+          # Change second copy to insert.
+          diffs[equalities[-1] + 1] = (self.DIFF_INSERT,
+              diffs[equalities[-1] + 1][1])
+          # Throw away the equality we just deleted.
+          equalities.pop()
+          # Throw away the previous equality (it needs to be reevaluated).
+          if len(equalities):
+            equalities.pop()
+          if len(equalities):
+            pointer = equalities[-1]
+          else:
+            pointer = -1
+          # Reset the counters.
+          length_insertions1, length_deletions1 = 0, 0
+          length_insertions2, length_deletions2 = 0, 0
+          lastEquality = None
+          changes = True
+      pointer += 1
+
+    # Normalize the diff.
+    if changes:
+      self.diff_cleanupMerge(diffs)
+    self.diff_cleanupSemanticLossless(diffs)
+
+    # Find any overlaps between deletions and insertions.
+    # e.g: <del>abcxxx</del><ins>xxxdef</ins>
+    #   -> <del>abc</del>xxx<ins>def</ins>
+    # e.g: <del>xxxabc</del><ins>defxxx</ins>
+    #   -> <ins>def</ins>xxx<del>abc</del>
+    # Only extract an overlap if it is as big as the edit ahead or behind it.
+    pointer = 1
+    while pointer < len(diffs):
+      if (diffs[pointer - 1][0] == self.DIFF_DELETE and
+          diffs[pointer][0] == self.DIFF_INSERT):
+        deletion = diffs[pointer - 1][1]
+        insertion = diffs[pointer][1]
+        overlap_length1 = self.diff_commonOverlap(deletion, insertion)
+        overlap_length2 = self.diff_commonOverlap(insertion, deletion)
+        if overlap_length1 >= overlap_length2:
+          if (overlap_length1 >= len(deletion) / 2.0 or
+              overlap_length1 >= len(insertion) / 2.0):
+            # Overlap found.  Insert an equality and trim the surrounding edits.
+            diffs.insert(pointer, (self.DIFF_EQUAL,
+                                   insertion[:overlap_length1]))
+            diffs[pointer - 1] = (self.DIFF_DELETE,
+                                  deletion[:len(deletion) - overlap_length1])
+            diffs[pointer + 1] = (self.DIFF_INSERT,
+                                  insertion[overlap_length1:])
+            pointer += 1
+        else:
+          if (overlap_length2 >= len(deletion) / 2.0 or
+              overlap_length2 >= len(insertion) / 2.0):
+            # Reverse overlap found.
+            # Insert an equality and swap and trim the surrounding edits.
+            diffs.insert(pointer, (self.DIFF_EQUAL, deletion[:overlap_length2]))
+            diffs[pointer - 1] = (self.DIFF_INSERT,
+                                  insertion[:len(insertion) - overlap_length2])
+            diffs[pointer + 1] = (self.DIFF_DELETE, deletion[overlap_length2:])
+            pointer += 1
+        pointer += 1
+      pointer += 1
+
+  def diff_cleanupSemanticLossless(self, diffs):
+    """Look for single edits surrounded on both sides by equalities
+    which can be shifted sideways to align the edit to a word boundary.
+    e.g: The c<ins>at c</ins>ame. -> The <ins>cat </ins>came.
+
+    Args:
+      diffs: Array of diff tuples.
+    """
+
+    def diff_cleanupSemanticScore(one, two):
+      """Given two strings, compute a score representing whether the
+      internal boundary falls on logical boundaries.
+      Scores range from 6 (best) to 0 (worst).
+      Closure, but does not reference any external variables.
+
+      Args:
+        one: First string.
+        two: Second string.
+
+      Returns:
+        The score.
+      """
+      if not one or not two:
+        # Edges are the best.
+        return 6
+
+      # Each port of this function behaves slightly differently due to
+      # subtle differences in each language's definition of things like
+      # 'whitespace'.  Since this function's purpose is largely cosmetic,
+      # the choice has been made to use each language's native features
+      # rather than force total conformity.
+      char1 = one[-1]
+      char2 = two[0]
+      nonAlphaNumeric1 = not char1.isalnum()
+      nonAlphaNumeric2 = not char2.isalnum()
+      whitespace1 = nonAlphaNumeric1 and char1.isspace()
+      whitespace2 = nonAlphaNumeric2 and char2.isspace()
+      lineBreak1 = whitespace1 and (char1 == "\r" or char1 == "\n")
+      lineBreak2 = whitespace2 and (char2 == "\r" or char2 == "\n")
+      blankLine1 = lineBreak1 and self.BLANKLINEEND.search(one)
+      blankLine2 = lineBreak2 and self.BLANKLINESTART.match(two)
+
+      if blankLine1 or blankLine2:
+        # Five points for blank lines.
+        return 5
+      elif lineBreak1 or lineBreak2:
+        # Four points for line breaks.
+        return 4
+      elif nonAlphaNumeric1 and not whitespace1 and whitespace2:
+        # Three points for end of sentences.
+        return 3
+      elif whitespace1 or whitespace2:
+        # Two points for whitespace.
+        return 2
+      elif nonAlphaNumeric1 or nonAlphaNumeric2:
+        # One point for non-alphanumeric.
+        return 1
+      return 0
+
+    pointer = 1
+    # Intentionally ignore the first and last element (don't need checking).
+    while pointer < len(diffs) - 1:
+      if (diffs[pointer - 1][0] == self.DIFF_EQUAL and
+          diffs[pointer + 1][0] == self.DIFF_EQUAL):
+        # This is a single edit surrounded by equalities.
+        equality1 = diffs[pointer - 1][1]
+        edit = diffs[pointer][1]
+        equality2 = diffs[pointer + 1][1]
+
+        # First, shift the edit as far left as possible.
+        commonOffset = self.diff_commonSuffix(equality1, edit)
+        if commonOffset:
+          commonString = edit[-commonOffset:]
+          equality1 = equality1[:-commonOffset]
+          edit = commonString + edit[:-commonOffset]
+          equality2 = commonString + equality2
+
+        # Second, step character by character right, looking for the best fit.
+        bestEquality1 = equality1
+        bestEdit = edit
+        bestEquality2 = equality2
+        bestScore = (diff_cleanupSemanticScore(equality1, edit) +
+            diff_cleanupSemanticScore(edit, equality2))
+        while edit and equality2 and edit[0] == equality2[0]:
+          equality1 += edit[0]
+          edit = edit[1:] + equality2[0]
+          equality2 = equality2[1:]
+          score = (diff_cleanupSemanticScore(equality1, edit) +
+              diff_cleanupSemanticScore(edit, equality2))
+          # The >= encourages trailing rather than leading whitespace on edits.
+          if score >= bestScore:
+            bestScore = score
+            bestEquality1 = equality1
+            bestEdit = edit
+            bestEquality2 = equality2
+
+        if diffs[pointer - 1][1] != bestEquality1:
+          # We have an improvement, save it back to the diff.
+          if bestEquality1:
+            diffs[pointer - 1] = (diffs[pointer - 1][0], bestEquality1)
+          else:
+            del diffs[pointer - 1]
+            pointer -= 1
+          diffs[pointer] = (diffs[pointer][0], bestEdit)
+          if bestEquality2:
+            diffs[pointer + 1] = (diffs[pointer + 1][0], bestEquality2)
+          else:
+            del diffs[pointer + 1]
+            pointer -= 1
+      pointer += 1
+
+  # Define some regex patterns for matching boundaries.
+  BLANKLINEEND = re.compile(r"\n\r?\n$")
+  BLANKLINESTART = re.compile(r"^\r?\n\r?\n")
+
+  def diff_cleanupEfficiency(self, diffs):
+    """Reduce the number of edits by eliminating operationally trivial
+    equalities.
+
+    Args:
+      diffs: Array of diff tuples.
+    """
+    changes = False
+    equalities = []  # Stack of indices where equalities are found.
+    lastEquality = None  # Always equal to diffs[equalities[-1]][1]
+    pointer = 0  # Index of current position.
+    pre_ins = False  # Is there an insertion operation before the last equality.
+    pre_del = False  # Is there a deletion operation before the last equality.
+    post_ins = False  # Is there an insertion operation after the last equality.
+    post_del = False  # Is there a deletion operation after the last equality.
+    while pointer < len(diffs):
+      if diffs[pointer][0] == self.DIFF_EQUAL:  # Equality found.
+        if (len(diffs[pointer][1]) < self.Diff_EditCost and
+            (post_ins or post_del)):
+          # Candidate found.
+          equalities.append(pointer)
+          pre_ins = post_ins
+          pre_del = post_del
+          lastEquality = diffs[pointer][1]
+        else:
+          # Not a candidate, and can never become one.
+          equalities = []
+          lastEquality = None
+
+        post_ins = post_del = False
+      else:  # An insertion or deletion.
+        if diffs[pointer][0] == self.DIFF_DELETE:
+          post_del = True
+        else:
+          post_ins = True
+
+        # Five types to be split:
+        # <ins>A</ins><del>B</del>XY<ins>C</ins><del>D</del>
+        # <ins>A</ins>X<ins>C</ins><del>D</del>
+        # <ins>A</ins><del>B</del>X<ins>C</ins>
+        # <ins>A</del>X<ins>C</ins><del>D</del>
+        # <ins>A</ins><del>B</del>X<del>C</del>
+
+        if lastEquality and ((pre_ins and pre_del and post_ins and post_del) or
+                             ((len(lastEquality) < self.Diff_EditCost / 2) and
+                              (pre_ins + pre_del + post_ins + post_del) == 3)):
+          # Duplicate record.
+          diffs.insert(equalities[-1], (self.DIFF_DELETE, lastEquality))
+          # Change second copy to insert.
+          diffs[equalities[-1] + 1] = (self.DIFF_INSERT,
+              diffs[equalities[-1] + 1][1])
+          equalities.pop()  # Throw away the equality we just deleted.
+          lastEquality = None
+          if pre_ins and pre_del:
+            # No changes made which could affect previous entry, keep going.
+            post_ins = post_del = True
+            equalities = []
+          else:
+            if len(equalities):
+              equalities.pop()  # Throw away the previous equality.
+            if len(equalities):
+              pointer = equalities[-1]
+            else:
+              pointer = -1
+            post_ins = post_del = False
+          changes = True
+      pointer += 1
+
+    if changes:
+      self.diff_cleanupMerge(diffs)
+
+  def diff_cleanupMerge(self, diffs):
+    """Reorder and merge like edit sections.  Merge equalities.
+    Any edit section can move as long as it doesn't cross an equality.
+
+    Args:
+      diffs: Array of diff tuples.
+    """
+    diffs.append((self.DIFF_EQUAL, ''))  # Add a dummy entry at the end.
+    pointer = 0
+    count_delete = 0
+    count_insert = 0
+    text_delete = ''
+    text_insert = ''
+    while pointer < len(diffs):
+      if diffs[pointer][0] == self.DIFF_INSERT:
+        count_insert += 1
+        text_insert += diffs[pointer][1]
+        pointer += 1
+      elif diffs[pointer][0] == self.DIFF_DELETE:
+        count_delete += 1
+        text_delete += diffs[pointer][1]
+        pointer += 1
+      elif diffs[pointer][0] == self.DIFF_EQUAL:
+        # Upon reaching an equality, check for prior redundancies.
+        if count_delete + count_insert > 1:
+          if count_delete != 0 and count_insert != 0:
+            # Factor out any common prefixies.
+            commonlength = self.diff_commonPrefix(text_insert, text_delete)
+            if commonlength != 0:
+              x = pointer - count_delete - count_insert - 1
+              if x >= 0 and diffs[x][0] == self.DIFF_EQUAL:
+                diffs[x] = (diffs[x][0], diffs[x][1] +
+                            text_insert[:commonlength])
+              else:
+                diffs.insert(0, (self.DIFF_EQUAL, text_insert[:commonlength]))
+                pointer += 1
+              text_insert = text_insert[commonlength:]
+              text_delete = text_delete[commonlength:]
+            # Factor out any common suffixies.
+            commonlength = self.diff_commonSuffix(text_insert, text_delete)
+            if commonlength != 0:
+              diffs[pointer] = (diffs[pointer][0], text_insert[-commonlength:] +
+                  diffs[pointer][1])
+              text_insert = text_insert[:-commonlength]
+              text_delete = text_delete[:-commonlength]
+          # Delete the offending records and add the merged ones.
+          new_ops = []
+          if len(text_delete) != 0:
+            new_ops.append((self.DIFF_DELETE, text_delete))
+          if len(text_insert) != 0:
+            new_ops.append((self.DIFF_INSERT, text_insert))
+          pointer -= count_delete + count_insert
+          diffs[pointer : pointer + count_delete + count_insert] = new_ops
+          pointer += len(new_ops) + 1
+        elif pointer != 0 and diffs[pointer - 1][0] == self.DIFF_EQUAL:
+          # Merge this equality with the previous one.
+          diffs[pointer - 1] = (diffs[pointer - 1][0],
+                                diffs[pointer - 1][1] + diffs[pointer][1])
+          del diffs[pointer]
+        else:
+          pointer += 1
+
+        count_insert = 0
         count_delete = 0
-        count_insert = 0
         text_delete = ''
         text_insert = ''
-        while pointer < len(diffs):
-            if diffs[pointer][0] == self.DIFF_INSERT:
-                count_insert += 1
-                text_insert += diffs[pointer][1]
-                pointer += 1
-            elif diffs[pointer][0] == self.DIFF_DELETE:
-                count_delete += 1
-                text_delete += diffs[pointer][1]
-                pointer += 1
-            elif diffs[pointer][0] == self.DIFF_EQUAL:
-                # Upon reaching an equality, check for prior redundancies.
-                if count_delete != 0 or count_insert != 0:
-                    if count_delete != 0 and count_insert != 0:
-                        # Factor out any common prefixies.
-                        commonlength = self.diff_commonPrefix(
-                            text_insert, text_delete)
-                        if commonlength != 0:
-                            x = pointer - count_delete - count_insert - 1
-                            if x >= 0 and diffs[x][0] == self.DIFF_EQUAL:
-                                diffs[x] = (diffs[x][0], diffs[x][1] +
-                                            text_insert[:commonlength])
-                            else:
-                                diffs.insert(
-                                    0, (self.DIFF_EQUAL, text_insert[:commonlength]))
-                                pointer += 1
-                            text_insert = text_insert[commonlength:]
-                            text_delete = text_delete[commonlength:]
-                        # Factor out any common suffixies.
-                        commonlength = self.diff_commonSuffix(
-                            text_insert, text_delete)
-                        if commonlength != 0:
-                            diffs[pointer] = (diffs[pointer][0], text_insert[-commonlength:] +
-                                              diffs[pointer][1])
-                            text_insert = text_insert[:-commonlength]
-                            text_delete = text_delete[:-commonlength]
-                    # Delete the offending records and add the merged ones.
-                    if count_delete == 0:
-                        diffs[pointer - count_insert: pointer] = [
-                            (self.DIFF_INSERT, text_insert)]
-                    elif count_insert == 0:
-                        diffs[pointer - count_delete: pointer] = [
-                            (self.DIFF_DELETE, text_delete)]
-                    else:
-                        diffs[pointer - count_delete - count_insert: pointer] = [
-                            (self.DIFF_DELETE, text_delete),
-                            (self.DIFF_INSERT, text_insert)]
-                    pointer = pointer - count_delete - count_insert + 1
-                    if count_delete != 0:
-                        pointer += 1
-                    if count_insert != 0:
-                        pointer += 1
-                elif pointer != 0 and diffs[pointer - 1][0] == self.DIFF_EQUAL:
-                    # Merge this equality with the previous one.
-                    diffs[pointer - 1] = (diffs[pointer - 1][0],
-                                          diffs[pointer - 1][1] + diffs[pointer][1])
-                    del diffs[pointer]
-                else:
-                    pointer += 1
-
-                count_insert = 0
-                count_delete = 0
-                text_delete = ''
-                text_insert = ''
-
-        if diffs[-1][1] == '':
-            diffs.pop()  # Remove the dummy entry at the end.
-
-        # Second pass: look for single edits surrounded on both sides by equalities
-        # which can be shifted sideways to eliminate an equality.
-        # e.g: A<ins>BA</ins>C -> <ins>AB</ins>AC
-        changes = False
-        pointer = 1
-        # Intentionally ignore the first and last element (don't need
-        # checking).
-        while pointer < len(diffs) - 1:
-            if (diffs[pointer - 1][0] == self.DIFF_EQUAL and
-                    diffs[pointer + 1][0] == self.DIFF_EQUAL):
-                # This is a single edit surrounded by equalities.
-                if diffs[pointer][1].endswith(diffs[pointer - 1][1]):
-                    # Shift the edit over the previous equality.
-                    diffs[pointer] = (diffs[pointer][0],
-                                      diffs[pointer - 1][1] +
-                                      diffs[pointer][1][:-len(diffs[pointer - 1][1])])
-                    diffs[pointer + 1] = (diffs[pointer + 1][0],
-                                          diffs[pointer - 1][1] + diffs[pointer + 1][1])
-                    del diffs[pointer - 1]
-                    changes = True
-                elif diffs[pointer][1].startswith(diffs[pointer + 1][1]):
-                    # Shift the edit over the next equality.
-                    diffs[pointer - 1] = (diffs[pointer - 1][0],
-                                          diffs[pointer - 1][1] + diffs[pointer + 1][1])
-                    diffs[pointer] = (diffs[pointer][0],
-                                      diffs[pointer][1][len(diffs[pointer + 1][1]):] +
-                                      diffs[pointer + 1][1])
-                    del diffs[pointer + 1]
-                    changes = True
-            pointer += 1
-
-        # If shifts were made, the diff needs reordering and another shift
-        # sweep.
-        if changes:
-            self.diff_cleanupMerge(diffs)
-
-    def diff_xIndex(self, diffs, loc):
-        """loc is a location in text1, compute and return the equivalent location
-        in text2.  e.g. "The cat" vs "The big cat", 1->1, 5->8
-
-        Args:
-          diffs: Array of diff tuples.
-          loc: Location within text1.
-
-        Returns:
-          Location within text2.
-        """
-        chars1 = 0
-        chars2 = 0
-        last_chars1 = 0
-        last_chars2 = 0
-        for x in xrange(len(diffs)):
-            (op, text) = diffs[x]
-            if op != self.DIFF_INSERT:  # Equality or deletion.
-                chars1 += len(text)
-            if op != self.DIFF_DELETE:  # Equality or insertion.
-                chars2 += len(text)
-            if chars1 > loc:  # Overshot the location.
-                break
-            last_chars1 = chars1
-            last_chars2 = chars2
-
-        if len(diffs) != x and diffs[x][0] == self.DIFF_DELETE:
-            # The location was deleted.
-            return last_chars2
-        # Add the remaining len(character).
-        return last_chars2 + (loc - last_chars1)
-
-    def diff_prettyHtml(self, diffs):
-        """Convert a diff array into a pretty HTML report.
-
-        Args:
-          diffs: Array of diff tuples.
-
-        Returns:
-          HTML representation.
-
-        """
-        html = []
-        i = 0
-        for (op, data) in diffs:
-            text = (data.replace('&', '&amp;').replace('<', '&lt;')
-                    .replace('>', '&gt;').replace('\n', '&para;<BR>'))
-            if op == self.DIFF_INSERT:
-                html.append("<SPAN STYLE=\"color: #00FF00;\" TITLE=\"i=%i\">%s</SPAN>"
-                            % (i, text))
-            elif op == self.DIFF_DELETE:
-                html.append("<SPAN STYLE=\"color: #FF0000;\" TITLE=\"i=%i\">%s</SPAN>"
-                            % (i, text))
-            elif op == self.DIFF_EQUAL:
-                html.append("<SPAN TITLE=\"i=%i\">%s</SPAN>" % (i, text))
-            if op != self.DIFF_DELETE:
-                i += len(data)
-        return ''.join(html)
-
-    def diff_text1(self, diffs):
-        """Compute and return the source text (all equalities and deletions).
-
-        Args:
-          diffs: Array of diff tuples.
-
-        Returns:
-          Source text.
-
-        """
-        text = []
-        for (op, data) in diffs:
-            if op != self.DIFF_INSERT:
-                text.append(data)
-        return ''.join(text)
-
-    def diff_text2(self, diffs):
-        """Compute and return the destination text (all equalities and
-        insertions).
-
-        Args:
-          diffs: Array of diff tuples.
-
-        Returns:
-          Destination text.
-
-        """
-        text = []
-        for (op, data) in diffs:
-            if op != self.DIFF_DELETE:
-                text.append(data)
-        return ''.join(text)
-
-    def diff_levenshtein(self, diffs):
-        """Compute the Levenshtein distance; the number of inserted, deleted or
-        substituted characters.
-
-        Args:
-          diffs: Array of diff tuples.
-
-        Returns:
-          Number of changes.
-
-        """
-        levenshtein = 0
+
+    if diffs[-1][1] == '':
+      diffs.pop()  # Remove the dummy entry at the end.
+
+    # Second pass: look for single edits surrounded on both sides by equalities
+    # which can be shifted sideways to eliminate an equality.
+    # e.g: A<ins>BA</ins>C -> <ins>AB</ins>AC
+    changes = False
+    pointer = 1
+    # Intentionally ignore the first and last element (don't need checking).
+    while pointer < len(diffs) - 1:
+      if (diffs[pointer - 1][0] == self.DIFF_EQUAL and
+          diffs[pointer + 1][0] == self.DIFF_EQUAL):
+        # This is a single edit surrounded by equalities.
+        if diffs[pointer][1].endswith(diffs[pointer - 1][1]):
+          # Shift the edit over the previous equality.
+          if diffs[pointer - 1][1] != "":
+            diffs[pointer] = (diffs[pointer][0],
+                diffs[pointer - 1][1] +
+                diffs[pointer][1][:-len(diffs[pointer - 1][1])])
+            diffs[pointer + 1] = (diffs[pointer + 1][0],
+                                  diffs[pointer - 1][1] + diffs[pointer + 1][1])
+          del diffs[pointer - 1]
+          changes = True
+        elif diffs[pointer][1].startswith(diffs[pointer + 1][1]):
+          # Shift the edit over the next equality.
+          diffs[pointer - 1] = (diffs[pointer - 1][0],
+                                diffs[pointer - 1][1] + diffs[pointer + 1][1])
+          diffs[pointer] = (diffs[pointer][0],
+              diffs[pointer][1][len(diffs[pointer + 1][1]):] +
+              diffs[pointer + 1][1])
+          del diffs[pointer + 1]
+          changes = True
+      pointer += 1
+
+    # If shifts were made, the diff needs reordering and another shift sweep.
+    if changes:
+      self.diff_cleanupMerge(diffs)
+
+  def diff_xIndex(self, diffs, loc):
+    """loc is a location in text1, compute and return the equivalent location
+    in text2.  e.g. "The cat" vs "The big cat", 1->1, 5->8
+
+    Args:
+      diffs: Array of diff tuples.
+      loc: Location within text1.
+
+    Returns:
+      Location within text2.
+    """
+    chars1 = 0
+    chars2 = 0
+    last_chars1 = 0
+    last_chars2 = 0
+    for x in range(len(diffs)):
+      (op, text) = diffs[x]
+      if op != self.DIFF_INSERT:  # Equality or deletion.
+        chars1 += len(text)
+      if op != self.DIFF_DELETE:  # Equality or insertion.
+        chars2 += len(text)
+      if chars1 > loc:  # Overshot the location.
+        break
+      last_chars1 = chars1
+      last_chars2 = chars2
+
+    if len(diffs) != x and diffs[x][0] == self.DIFF_DELETE:
+      # The location was deleted.
+      return last_chars2
+    # Add the remaining len(character).
+    return last_chars2 + (loc - last_chars1)
+
+  def diff_prettyHtml(self, diffs):
+    """Convert a diff array into a pretty HTML report.
+
+    Args:
+      diffs: Array of diff tuples.
+
+    Returns:
+      HTML representation.
+    """
+    html = []
+    for (op, data) in diffs:
+      text = (data.replace("&", "&amp;").replace("<", "&lt;")
+                 .replace(">", "&gt;").replace("\n", "&para;<br>"))
+      if op == self.DIFF_INSERT:
+        html.append("<ins class=\"inserted\">%s</ins>" % text)
+      elif op == self.DIFF_DELETE:
+        html.append("<del class=\"removed\">%s</del>" % text)
+      elif op == self.DIFF_EQUAL:
+        html.append("<span>%s</span>" % text)
+    return "".join(html)
+
+  def diff_text1(self, diffs):
+    """Compute and return the source text (all equalities and deletions).
+
+    Args:
+      diffs: Array of diff tuples.
+
+    Returns:
+      Source text.
+    """
+    text = []
+    for (op, data) in diffs:
+      if op != self.DIFF_INSERT:
+        text.append(data)
+    return "".join(text)
+
+  def diff_text2(self, diffs):
+    """Compute and return the destination text (all equalities and insertions).
+
+    Args:
+      diffs: Array of diff tuples.
+
+    Returns:
+      Destination text.
+    """
+    text = []
+    for (op, data) in diffs:
+      if op != self.DIFF_DELETE:
+        text.append(data)
+    return "".join(text)
+
+  def diff_levenshtein(self, diffs):
+    """Compute the Levenshtein distance; the number of inserted, deleted or
+    substituted characters.
+
+    Args:
+      diffs: Array of diff tuples.
+
+    Returns:
+      Number of changes.
+    """
+    levenshtein = 0
+    insertions = 0
+    deletions = 0
+    for (op, data) in diffs:
+      if op == self.DIFF_INSERT:
+        insertions += len(data)
+      elif op == self.DIFF_DELETE:
+        deletions += len(data)
+      elif op == self.DIFF_EQUAL:
+        # A deletion and an insertion is one substitution.
+        levenshtein += max(insertions, deletions)
         insertions = 0
         deletions = 0
-        for (op, data) in diffs:
-            if op == self.DIFF_INSERT:
-                insertions += len(data)
-            elif op == self.DIFF_DELETE:
-                deletions += len(data)
-            elif op == self.DIFF_EQUAL:
-                # A deletion and an insertion is one substitution.
-                levenshtein += max(insertions, deletions)
-                insertions = 0
-                deletions = 0
-        levenshtein += max(insertions, deletions)
-        return levenshtein
-
-    def diff_toDelta(self, diffs):
-        """Crush the diff into an encoded string which describes the operations
-        required to transform text1 into text2.
-        E.g. =3\t-2\t+ing  -> Keep 3 chars, delete 2 chars, insert 'ing'.
-        Operations are tab-separated.  Inserted text is escaped using %xx notation.
-
-        Args:
-          diffs: Array of diff tuples.
-
-        Returns:
-          Delta text.
-        """
-        text = []
-        for (op, data) in diffs:
-            if op == self.DIFF_INSERT:
-                # High ascii will raise UnicodeDecodeError.  Use Unicode
-                # instead.
-                data = data.encode('utf-8')
-                text.append('+' + urllib.quote(data, "!~*'();/?:@&=+$,# "))
-            elif op == self.DIFF_DELETE:
-                text.append('-%d' % len(data))
-            elif op == self.DIFF_EQUAL:
-                text.append('=%d' % len(data))
-        return '\t'.join(text)
-
-    def diff_fromDelta(self, text1, delta):
-        """Given the original text1, and an encoded string which describes the
-        operations required to transform text1 into text2, compute the full
-        diff.
-
-        Args:
-          text1: Source string for the diff.
-          delta: Delta text.
-
-        Returns:
-          Array of diff tuples.
-
-        Raises:
-          ValueError: If invalid input.
-
-        """
-        if type(delta) == unicode:
-            # Deltas should be composed of a subset of ascii chars, Unicode not
-            # required.  If this encode raises UnicodeEncodeError, delta is
-            # invalid.
-            delta = delta.encode('ascii')
-        diffs = []
-        pointer = 0  # Cursor in text1
-        tokens = delta.split('\t')
-        for token in tokens:
-            if token == '':
-                # Blank tokens are ok (from a trailing \t).
-                continue
-            # Each token begins with a one character parameter which specifies the
-            # operation of this token (delete, insert, equality).
-            param = token[1:]
-            if token[0] == '+':
-                param = urllib.unquote(param).decode('utf-8')
-                diffs.append((self.DIFF_INSERT, param))
-            elif token[0] == '-' or token[0] == '=':
-                try:
-                    n = int(param)
-                except ValueError:
-                    raise ValueError, 'Invalid number in diff_fromDelta: ' + param
-                if n < 0:
-                    raise ValueError, 'Negative number in diff_fromDelta: ' + param
-                text = text1[pointer: pointer + n]
-                pointer += n
-                if token[0] == '=':
-                    diffs.append((self.DIFF_EQUAL, text))
-                else:
-                    diffs.append((self.DIFF_DELETE, text))
+    levenshtein += max(insertions, deletions)
+    return levenshtein
+
+  def diff_toDelta(self, diffs):
+    """Crush the diff into an encoded string which describes the operations
+    required to transform text1 into text2.
+    E.g. =3\t-2\t+ing  -> Keep 3 chars, delete 2 chars, insert 'ing'.
+    Operations are tab-separated.  Inserted text is escaped using %xx notation.
+
+    Args:
+      diffs: Array of diff tuples.
+
+    Returns:
+      Delta text.
+    """
+    text = []
+    for (op, data) in diffs:
+      if op == self.DIFF_INSERT:
+        # High ascii will raise UnicodeDecodeError.  Use Unicode instead.
+        data = data.encode("utf-8")
+        text.append("+" + urllib.parse.quote(data, "!~*'();/?:@&=+$,# "))
+      elif op == self.DIFF_DELETE:
+        text.append("-%d" % len(data))
+      elif op == self.DIFF_EQUAL:
+        text.append("=%d" % len(data))
+    return "\t".join(text)
+
+  def diff_fromDelta(self, text1, delta):
+    """Given the original text1, and an encoded string which describes the
+    operations required to transform text1 into text2, compute the full diff.
+
+    Args:
+      text1: Source string for the diff.
+      delta: Delta text.
+
+    Returns:
+      Array of diff tuples.
+
+    Raises:
+      ValueError: If invalid input.
+    """
+    diffs = []
+    pointer = 0  # Cursor in text1
+    tokens = delta.split("\t")
+    for token in tokens:
+      if token == "":
+        # Blank tokens are ok (from a trailing \t).
+        continue
+      # Each token begins with a one character parameter which specifies the
+      # operation of this token (delete, insert, equality).
+      param = token[1:]
+      if token[0] == "+":
+        param = urllib.parse.unquote(param)
+        diffs.append((self.DIFF_INSERT, param))
+      elif token[0] == "-" or token[0] == "=":
+        try:
+          n = int(param)
+        except ValueError:
+          raise ValueError("Invalid number in diff_fromDelta: " + param)
+        if n < 0:
+          raise ValueError("Negative number in diff_fromDelta: " + param)
+        text = text1[pointer : pointer + n]
+        pointer += n
+        if token[0] == "=":
+          diffs.append((self.DIFF_EQUAL, text))
+        else:
+          diffs.append((self.DIFF_DELETE, text))
+      else:
+        # Anything else is an error.
+        raise ValueError("Invalid diff operation in diff_fromDelta: " +
+            token[0])
+    if pointer != len(text1):
+      raise ValueError(
+          "Delta length (%d) does not equal source text length (%d)." %
+         (pointer, len(text1)))
+    return diffs
+
+  #  MATCH FUNCTIONS
+
+  def match_main(self, text, pattern, loc):
+    """Locate the best instance of 'pattern' in 'text' near 'loc'.
+
+    Args:
+      text: The text to search.
+      pattern: The pattern to search for.
+      loc: The location to search around.
+
+    Returns:
+      Best match index or -1.
+    """
+    # Check for null inputs.
+    if text == None or pattern == None: