Merge lp:~stefanor/ibid/releasenotes-0.1.1 into lp:~ibid-core/ibid/old-release-0.1-1.6

[Stefano Rivera]

Added CHANGES and tool for generating changelog entries.
Set version to 0.1.1

[Jonathan Hitchcock (vhata)]

review: Approve

[marcog (marco-gallotta)]

review: Approve

[Ibid Branch Auto-Lander (ibid-tarmac)]

Voting does not meet specified criteria. Required: Approve >= 2, Disapprove == 0. Got: 1 Approve.

[Max Rabkin (max-rabkin)]

review: Approve

[Keegan Carruthers-Smith (keegan-csmith)]

review: Approve

=== added file 'CHANGES'
--- CHANGES	1970-01-01 00:00:00 +0000
+++ CHANGES	2011-02-20 21:52:17 +0000
@@ -0,0 +1,435 @@
+Release 0.1.1 (2011-02-24)
+==============================
+
+Bug fix release, including a couple of security issues.
+
+Several plugins that consume Web services or scrape Web sites have been
+updated to cope with changes since the last release.
+
+There were no DB schema changes between 0.1.0 and 0.1.1.
+
+Resolved Security Issues
+------------------------
+
+Remote Execution
+^^^^^^^^^^^^^^^^
+
+:bug:`705860`:
+   Permissions were ignored for handlers not using :func:`@match
+   <ibid.plugins.match>`.
+   This allowed users to perform actions they were not authorised to.
+
+   However, no included plugins were exposed by this, all
+   access-restricted handlers had match patterns.
+
+Information Disclosure
+^^^^^^^^^^^^^^^^^^^^^^
+
+:bug:`567576`:
+   Occasionally insecure permissions on log files.
+   When the bot spoke first (creating a new log file), the log file
+   would be publicly readable, even if the message was sent in private.
+
+   Example:
+   If the bot delivered a *privmsg* memo to a user at the beginning of
+   the month, it would create the logfile with public readable
+   permissions.
+   If the logfile directory was published by a web server, this would
+   make this private conversation log accessible to the public.
+
+   Resolution: Now channels must be explicitly configured to have
+   publicly readable logs.
+
+:bug:`649383`:
+   If someone received a private message from the bot
+   during a public meeting, the message could appear in the meeting
+   minutes.
+
+   Example: a *privmsg* memo received during a meeting would appear in
+   the minutes.
+
+Major User Visible Changes
+--------------------------
+
+*  New configuration option ``plugins.log.public_logs``, a list of
+   ``source:channel`` globs of channels to log in files with publically
+   readable permissions.
+
+*  New configuration option ``plugins.ascii.preferred_fonts``, a list of
+   figlet fonts, the first one found is the default.
+
+*  Currency exchange now uses Yahoo instead of XE.com.
+
+API Changes
+-----------
+
+*  New Function: :func:`ibid.utils.parse_timestamp` for parsing
+   well-formatted timestamps.
+
+*  New Function: :func:`ibid.utils.generic_webservice` for retrieving
+   arbitrary data from a web-service.
+
+*  New Function: :func:`ibid.db.get_regexp_op` which returns a REGEXP
+   SqlAlchemy operator for the DBMS in use.
+
+All Changes
+-----------
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Filter out empty definitions in gdefine.
+
+   Fixes: :bug:`719851`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   We don't support SQLAlchemy 0.6 yet.
+
+   Fixes: :bug:`651992`.
+
[phone]  Marco Gallotta  <[e-mail]>
+
+   Only append .com for url's like "example".
+
+   Fixes: :bug:`702062`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Use escape=# for LIKEs.
+   Perform literal queries on all non-get Factoid operations. Return
+   useful error if start index is too high.
+   Substitute $arg for _% in search.
+
+   Fixes: :bug:`544493`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   HTTPErrors should result in using url as title, not abandoning the
+   grab.
+
+   Fixes: :bug:`702798`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Catch ImportErrors for packages we don't require in setup.py.
+
+   Fixes: :bug:`651990`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   pysqlite is only necessary on ancient Pythons.
+
+   Fixes: :bug:`708302`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Add function get_regexp_op to ibid.db that returns a REGEXP op that
+   works on Postgres too.
+
+   Fixes: :bug:`595423`.
+
[phone]  Keegan Carruthers-Smith  <[e-mail]>
+
+   Use correct plurality in pending memos message.
+
+   Fixes :bug:`634257`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Add parse_timestamp function to ibid.utils, use for parsing
+   timestamps from Twitter.
+
+   Fixes :bug:`702815`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   URLErrors have reasons, but there are other HTTPErrors
+
+   Fixes :bug:`670855`.
+
[phone]  Max Rabkin  <[e-mail]>
+
+   Enforce permissions on non-@match handlers.
+
+   Fixes :bug:`705860`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Handle non-500 error codes from twitter.
+
+   Fixes :bug:`670855`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Strip tags from gcalc response.
+
+   Fixes :bug:`702371`.
+
[phone]  Max Rabkin  <[e-mail]>
+
+   Check content_type is set before checking its value.
+
+   Fixes :bug:`701900`.
+
[phone]  Max Rabkin  <[e-mail]>
+
+   Catch exceptions when polling feeds so that one broken feed doesn't
+   stop all feeds.
+
+   Fixes :bug:`578396`.
+
[phone]  Max Rabkin  <[e-mail]>
+
+   Use new OEIS API at oeis.org
+
+   Fixes :bug:`700475`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Fix for the logging open file cache: Logs in logs might not be in
+   recent_logs.
+
+   Fixes :bug:`655645`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Support toilet fonts, correctly decode utf-8 figlet output, handle
+   font choice edge cases.
+
+   Fixes :bug:`607743`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Follow redirects in "is it up"
+
+   Fixes :bug:`599410`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Rework nickserv auth to allow simultaneous authentications for the
+   same nick (although Nickserv will only be bothered once).
+
+   Fixes :bug:`655647`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Use absolute imports to import SILC correctly
+
+   Fixes :bug:`654202`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Country Code list parsing: Check for ; in a line before splitting by
+   semi-colons.
+
+   Fixes :bug:`692347`.
+
[phone]  Max Rabkin  <[e-mail]>
+
+   Don't treat feeds with no messages as errors.
+
+   Fixes :bug:`661187`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Correctly handle state events that have no channel.
+
+   Fixes :bug:`656349`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Port google scrape search to get_html_parse_tree + etree. Handle
+   superscript in gcalc.
+
+   Fixes :bug:`580696`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Put periodic lock-using code in a try...finally block.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Support twitter's new AJAX URLs.
+
+   Fixes :bug:`654535`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Always respond to memo sending with confirmation of recipient. Allow
+   memos to begin with "on ..." when not naming a known source.
+
+   Fixes :bug:`634253`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Disallow empty factoid names.
+
+   Fixes :bug:`606065`.
+
[phone]  Guy Halse
+
+   Allow bot to identify with zanet.net's NickServ.
+
+   Fixes :bug:`652000`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   [SECURITY] Add a configuration glob-list of channels which should
+   have public logs, rather than attempting to guess.
+
+   Fixes :bug:`567576`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Docs: Be clear that ibid is in Debian & Ubuntu.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Handle 0 tweets in Twitter Atom feed parsing, correctly handle it
+   elsewhere instead of treating it as no such twit.
+
+   Fixes :bug:`646989`.
+
[phone]  Max Rabkin  <[e-mail]>
+
+   [SECURITY] Don't leak private messages to meeting logs.
+
+   Fixes :bug:`649383`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Don't try to process() events without a message in meeting.
+
+   Fixes :bug:`598094`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Correct abbreviated cross-ref format, shown up by Sphinx 1.0b1.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Display latest tweets from retweeting-twits instead of thinking they
+   don't exist.
+
+   Fixes :bug:`554906`.
+
[phone]  Michael Gorven  <[e-mail]>
+
+   Fix real JID detection when more than one 'x' element is received.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   NickServ support for rizon.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Switch from XE.com to Yahoo for currency conversions.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Limit the size of the file-descriptor pool in log.
+
+   Fixes :bug:`567571`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Google is also a calculator.
+
+   Fixes :bug:`574300`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Use explicit lower() on each side of LIKE so factoids with arguments
+   can be case-insensitive on Postgres.
+
+   Fixes :bug:`574427`.
+
[phone]  Max Rabkin  <[e-mail]>
+
+   Escape query url in google scrape.
+
+   Fixes :bug:`572308`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Incorrect substitution in SQLite indexing warning.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Change administrative user & identity linking syntax to be less
+   problematically broad.
+
+   Fixes :bug:`567510`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Increase default HTTP GET size from 500 bytes to 2kiB.
+
+   Fixes :bug:`563928`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Update youtube plugin to cope with site redesign.
+
+   Fixes :bug:`561684`.
+
[phone]  Max Rabkin  <[e-mail]>
+
+   Allow digits in Unicode character names.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   HTTP GET: Don't assume everything is utf-8, decode according to
+   provided charset, fall back to utf-8 for text, and guess with chardet
+   if either of those was wrong.
+
+   Fixes :bug:`560973`.
+
[phone]  Max Rabkin  <[e-mail]>
+
+   Use unicode case-insensitive matching in factoid.
+
+   Fixes :bug:`542707`.
+
[phone]  Michael Gorven  <[e-mail]>
+
+   Treat the Processor's first feature as the primary feature in RPC.
+
+   Fixes :bug:`545168`.
+
[phone]  Max Rabkin  <[e-mail]>
+
+   Respond with unicode in bible error handlers.
+
+   Fixes :bug:`544260`.
+
[phone]  Marco Gallotta  <[e-mail]>
+
+   Allow trailing punctuation in tea-style addressing.
+
+   Fixes :bug:`545186`.
+
[phone]  Marco Gallotta  <[e-mail]>
+
+   Add username=ibid parameter to geonames API calls. Some calls now
+   require it.
+
+   Fixes :bug:`543989`.
+
[phone]  Max Rabkin  <[e-mail]>
+
+   Python 2.5 compatibility update for unicode lookup. Exception for
+   unknown character changed in 2.6.
+
+   Fixes :bug:`542593`.
+
[phone]  Stefano Rivera  <[e-mail]>
+
+   Add placeholder to force ibid/static to be distributed.
+
+Release 0.1.0 (2010-03-10)
+==========================
+
+*  First public release.
+
+.. vi: set ft=rst et sta sw=3 ts=3:

=== added directory 'docs/_extensions'
=== added file 'docs/_extensions/extlinks.py'
--- docs/_extensions/extlinks.py	1970-01-01 00:00:00 +0000
+++ docs/_extensions/extlinks.py	2011-02-20 21:52:17 +0000
@@ -0,0 +1,67 @@
+# -*- coding: utf-8 -*-
+# Backport of sphinx.ext.extlinks to Sphinx 0.6
+"""
+    sphinx.ext.extlinks
+    ~~~~~~~~~~~~~~~~~~~
+
+    Extension to save typing and prevent hard-coding of base URLs in the reST
+    files.
+
+    This adds a new config value called ``extlinks`` that is created like this::
+
+       extlinks = {'exmpl': ('http://example.com/%s.html', prefix), ...}
+
+    Now you can use e.g. :exmpl:`foo` in your documents.  This will create a
+    link to ``http://example.com/foo.html``.  The link caption depends on the
+    *prefix* value given:
+
+    - If it is ``None``, the caption will be the full URL.
+    - If it is a string (empty or not), the caption will be the prefix prepended
+      to the role content.
+
+    You can also give an explicit caption, e.g. :exmpl:`Foo <foo>`.
+
+    :copyright: Copyright 2007-2011 by the Sphinx team, see AUTHORS.
+    :license: BSD, see LICENSE for details.
+"""
+
+import re
+
+from docutils import nodes, utils
+
+explicit_title_re = re.compile(r'^(.+?)\s*(?<!\x00)<(.*?)>$', re.DOTALL)
+def split_explicit_title(text):
+    """Split role content into title and target, if given."""
+    match = explicit_title_re.match(text)
+    if match:
+        return True, match.group(1), match.group(2)
+    return False, text, text
+
+def make_link_role(base_url, prefix):
+    def role(typ, rawtext, text, lineno, inliner, options={}, content=[]):
+        text = utils.unescape(text)
+        has_explicit_title, title, part = split_explicit_title(text)
+        try:
+            full_url = base_url % part
+        except (TypeError, ValueError):
+            env = inliner.document.settings.env
+            env.warn(env.docname, 'unable to expand %s extlink with base '
+                     'URL %r, please make sure the base contains \'%%s\' '
+                     'exactly once' % (typ, base_url))
+            full_url = base_url + part
+        if not has_explicit_title:
+            if prefix is None:
+                title = full_url
+            else:
+                title = prefix + part
+        pnode = nodes.reference(title, title, refuri=full_url)
+        return [pnode], []
+    return role
+
+def setup_link_roles(app):
+    for name, (base_url, prefix) in app.config.extlinks.iteritems():
+        app.add_role(name, make_link_role(base_url, prefix))
+
+def setup(app):
+    app.add_config_value('extlinks', {}, 'env')
+    app.connect('builder-inited', setup_link_roles)

=== added file 'docs/changes.rst'
--- docs/changes.rst	1970-01-01 00:00:00 +0000
+++ docs/changes.rst	2011-02-20 21:52:17 +0000
@@ -0,0 +1,8 @@
+:tocdepth: 2
+
+Changes in Ibid
+***************
+
+.. include:: ../CHANGES
+
+.. vi: set et sta sw=3 ts=3:

=== modified file 'docs/conf.py'
--- docs/conf.py	2011-01-15 08:51:05 +0000
+++ docs/conf.py	2011-02-20 21:52:17 +0000
@@ -13,6 +13,8 @@
 
 import sys, os
 
+import sphinx
+
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
@@ -22,7 +24,11 @@
 
 # Add any Sphinx extension module names here, as strings. They can be extensions
 # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = []
+sys.path.append(os.path.abspath('_extensions'))
+if sphinx.__version__ > '1.0':
+    extensions = ['sphinx.ext.extlinks']
+else:
+    extensions = ['extlinks']
 
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
@@ -47,7 +53,7 @@
 # The short X.Y version.
 version = '0.1'
 # The full version, including alpha/beta/rc tags.
-release = '0.1.0'
+release = '0.1.1'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
@@ -192,3 +198,8 @@
 
 # If false, no module index is generated.
 #latex_use_modindex = True
+
+# -- Options for extensions ----------------------------------------------------
+
+# pre-1.0 mode intersphinx:
+extlinks = {'bug': ('https://bugs.launchpad.net/bugs/%s', 'LP: #')}

=== modified file 'docs/configuration.rst'
--- docs/configuration.rst	2010-02-26 08:29:31 +0000
+++ docs/configuration.rst	2011-02-20 21:52:17 +0000
@@ -170,9 +170,8 @@
    **Reqired**
    String: The hostname of the IRC server to connect to.
 
-   Ibid `does not currently support
-   <https://bugs.launchpad.net/bugs/363466>`_ falling back to alternate
-   servers, so you may want to use a round-robin hostname.
+   Ibid :bug:`does not currently support <363466>` falling back to
+   alternate servers, so you may want to use a round-robin hostname.
 
 .. describe:: port:
 

=== modified file 'docs/index.rst'
--- docs/index.rst	2010-01-12 16:29:15 +0000
+++ docs/index.rst	2011-02-20 21:52:17 +0000
@@ -11,6 +11,7 @@
    tutorial
    contributing
    api/index
+   changes
 
 
 Indices and tables

=== modified file 'setup.py'
--- setup.py	2011-02-20 20:34:20 +0000
+++ setup.py	2011-02-20 21:52:17 +0000
@@ -35,7 +35,7 @@
 
 setup(
     name='Ibid',
-    version='0.1.0',
+    version='0.1.1',
     description='Multi-protocol general purpose chat bot',
     url='http://ibid.omnia.za.net/',
     keywords='chat bot irc jabber twisted messaging',

=== added directory 'tools'
=== added file 'tools/format-changelog'
--- tools/format-changelog	1970-01-01 00:00:00 +0000
+++ tools/format-changelog	2011-02-20 21:52:17 +0000
@@ -0,0 +1,70 @@
+#!/usr/bin/env python
+
+import optparse
+import re
+import subprocess
+import textwrap
+
+def process_block(block):
+    "Reformat a block into nice rst"
+    block = ''.join(block).decode('utf-8').strip()
+
+    template_re = re.compile(r'^\tAuthor: .*\n'
+                             r'(?:\tMerge Request: .*\n)?'
+                             r'\tApproved by: .*\n'
+                             r'\tFixes LP: #(.*)$',
+                             re.MULTILINE | re.UNICODE)
+    def repl(m):
+        bugs = [int(bug) for bug in m.group(1).split(',')]
+        if bugs:
+            bugs = [u':bug:`%i`' % bug for bug in bugs]
+            return u'\n\tFixes: %s.' % u', '.join(bugs)
+        return u''
+    block = template_re.sub(repl, block)
+
+    block = block.splitlines()
+
+    wrapper = textwrap.TextWrapper(subsequent_indent=u'   ')
+    for i, line in enumerate(block):
+        if not line.strip():
+            line = u''
+
+        if line.startswith(u'\t'):
+            line = u'   ' + line[1:]
+            if i + 1 == len(block) or not block[i+1].strip():
+                if not line.endswith(u'.'):
+                    line += u'.'
+
+            line = u'\n'.join(wrapper.wrap(line))
+
+        block[i] = line
+
+    block.append(u'')
+    block = u'\n'.join(block)
+    print block.encode('utf-8')
+
+def main():
+    parser = optparse.OptionParser()
+    parser.add_option('-r', '--revision', metavar='REVSPEC',
+                      help='Passed to bzr')
+    opts, args = parser.parse_args()
+
+    cmd = ['bzr', 'log', '--gnu', '-n1']
+    if opts.revision:
+        cmd.append('-r' + opts.revision)
+    p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
+
+    head_re = re.compile(r'^\d{4}-\d{2}-\d{2}  .+  <.+>$')
+    block = []
+    for line in p.stdout:
+        if head_re.match(line):
+            process_block(block)
+            block = []
+        block.append(line)
+
+    process_block(block)
+
+    assert p.wait() == 0
+
+if __name__ == '__main__':
+    main()