Merge lp:~wking-drexel/pybtex/wtk into lp:pybtex

[Trevor King]

Document a bug with whitespace normalization in concatenated BibTeX strings. Add workaround for AUthor #' and '# OTher case.

[lp:~wking-drexel/pybtex/wtk]

680. By Trevor King on 2010-10-15

Don't check end brace level of empty strings in database.output.bibtex.

681. By Trevor King on 2010-10-15

FormattedEntry should subclass object.

Old-style classes are evil!

682. By Trevor King on 2010-10-15

Add optional automatic key generator to increase database consistency.

683. By Trevor King on 2010-10-15

Store re-keyed entries in the automatic key generator ;).

684. By Trevor King on 2010-10-15

Strip non-letter characters from author lastnames in auto-generated keys.

685. By Trevor King on 2010-10-15

Give database authors a mechanism for avoiding auto-generated keys.

686. By Trevor King on 2010-10-16

Also allow '-' in auto-generated keys.

687. By Trevor King on 2010-10-16

Add Entry.ordered_field_items() for consistent database output.

688. By Trevor King on 2010-10-16

Raise PybtexError if a duplicate BibTeX field would clobber a previous value.

689. By Trevor King on 2010-10-16

Improve key auto-generation for von-containing names.

690. By Trevor King on 2010-10-16

Add doctest showing that comments are parsed in BibTeX input.

691. By Trevor King on 2010-10-16

Simplify comment parsing doctest and add Parser.raw_macros (preserving case).

692. By Trevor King on 2010-10-16

Handle simple comments in the bibtex parser.

693. By Trevor King on 2010-10-16

.parse_stream() should return .data to match .parse_file().

694. By Trevor King on 2010-10-16

Change BibTeX comment regexp to ignore indented comments.

This is at the expense of possibly handling escaped percent signs
(\%), which are not handled correctly at the moment (untested).
Luckily, they probably don't show up too often in BibTeX files.

695. By Trevor King on 2010-10-16

Demonstrate that the BibTeX parser doesn't treat %s in strings as comments.

696. By Trevor King on 2010-10-16

Make (raw_macro,value) dict generation a method: .get_raw_macros().

697. By Trevor King on 2010-10-16

Add database.output.clean_bibtex for generating easy-to-maintain output BibTeX.

Example usage to clean up an inconsistent BibTeX file:
    import sys
    from pybtex.database.input.bibtex import Parser
    from pybtex.database.output.clean_bibtex import Writer
    p = Parser()
    d = p.parse_file("/path/to/old/file.bib")
    w = Writer()
    w.write_stream(d, sys.stdout, p.get_raw_macros())
Information about new macros and key changes will be logged to stderr.

698. By Trevor King on 2010-10-16

Pass extra Writer.write_file() arguments on to Writer.write_stream().

This allows us to pass in a list of macros for the clean_bibtex.Writer.

699. By Trevor King on 2010-10-16

Fix unicode/string TypeError in clean_bibtex.Writer.write_file().

This avoids:

Traceback (most recent call last):
  ...
  File "pybtex/database/output/clean_bibtex.py", line 187, in write_stream
    stream.write('\n')
  File "/usr/lib/python2.6/io.py", line 1491, in write
    s.__class__.__name__)
TypeError: can't write str to text stream

700. By Trevor King on 2010-10-16

Be more flexible in finding a person for KeyGenerator.generate_key().

701. By Trevor King on 2010-10-16

Add crossref and booktitle to Entry.field_order.

702. By Trevor King on 2010-10-16

Add clean_bibtex doctest and fix some errors it turned up.

703. By Trevor King on 2010-10-16

Make "and" avoidance in whitespace normalization immune to endlines.

This works around problems like:
  author = AUthor #" and
           "# OTher,

Unmerged revisions

703. By Trevor King on 2010-10-16

Make "and" avoidance in whitespace normalization immune to endlines.

This works around problems like:
  author = AUthor #" and
           "# OTher,

702. By Trevor King on 2010-10-16

Add clean_bibtex doctest and fix some errors it turned up.

701. By Trevor King on 2010-10-16

Add crossref and booktitle to Entry.field_order.

700. By Trevor King on 2010-10-16

Be more flexible in finding a person for KeyGenerator.generate_key().

699. By Trevor King on 2010-10-16

Fix unicode/string TypeError in clean_bibtex.Writer.write_file().

This avoids:

Traceback (most recent call last):
  ...
  File "pybtex/database/output/clean_bibtex.py", line 187, in write_stream
    stream.write('\n')
  File "/usr/lib/python2.6/io.py", line 1491, in write
    s.__class__.__name__)
TypeError: can't write str to text stream

698. By Trevor King on 2010-10-16

Pass extra Writer.write_file() arguments on to Writer.write_stream().

This allows us to pass in a list of macros for the clean_bibtex.Writer.

697. By Trevor King on 2010-10-16

Add database.output.clean_bibtex for generating easy-to-maintain output BibTeX.

Example usage to clean up an inconsistent BibTeX file:
    import sys
    from pybtex.database.input.bibtex import Parser
    from pybtex.database.output.clean_bibtex import Writer
    p = Parser()
    d = p.parse_file("/path/to/old/file.bib")
    w = Writer()
    w.write_stream(d, sys.stdout, p.get_raw_macros())
Information about new macros and key changes will be logged to stderr.

696. By Trevor King on 2010-10-16

Make (raw_macro,value) dict generation a method: .get_raw_macros().

695. By Trevor King on 2010-10-16

Demonstrate that the BibTeX parser doesn't treat %s in strings as comments.

694. By Trevor King on 2010-10-16

Change BibTeX comment regexp to ignore indented comments.

This is at the expense of possibly handling escaped percent signs
(\%), which are not handled correctly at the moment (untested).
Luckily, they probably don't show up too often in BibTeX files.

=== modified file 'pybtex/core.py'
--- pybtex/core.py	2009-09-06 18:24:24 +0000
+++ pybtex/core.py	2010-10-17 00:56:45 +0000
@@ -17,11 +17,13 @@
 """
 
 import re
+import sys
+import time
 
 from pybtex.exceptions import PybtexError
 from pybtex.bibtex.utils import split_tex_string
 
-class FormattedEntry:
+class FormattedEntry(object):
     """Formatted bibliography entry. Consists of
     - key (which is used for sorting);
     - label (which appears in the resulting bibliography)
@@ -51,6 +53,16 @@
     """Bibliography entry. Important members are:
     - persons (a dict of Person objects)
     - fields (all dict of string)
+
+    >>> e = Entry(type_='article',
+    ...           fields=FieldDict(parent=None, year='1234', month='February'),
+    ...           persons={'author': [Person('A. Uthor'), Person('O. Ther')]})
+    >>> e.year()
+    1234
+    >>> e.month()
+    2
+    >>> e.day()
+    0
     """
 
     def __init__(self, type_, fields=None, persons=None, collection=None):
@@ -65,6 +77,43 @@
 
         # for BibTeX interpreter
         self.vars = {}
+        # for output writers
+        self.field_order = [
+            'key',
+            'title',
+            'crossref',
+            'booktitle',
+            'affiliation',
+            'collaboration',
+            'year',
+            'season',
+            'month',
+            'day',
+            'version',
+            'journal',
+            'edition',
+            'volume',
+            'number',
+            'pages',
+            'xxpages',
+            'numpages',
+            'isbn',
+            'isbn-13',
+            'issn',
+            'lccn',
+            'publisher',
+            'address',
+            'bibdate',
+            'copyright',
+            'eid',
+            'doi',
+            'eprint',
+            'url',
+            'keywords',
+            'abstract',
+            'note',
+            'project',
+            ]
 
     def get_crossref(self):
         return self.collection.entries[self.fields['crossref']]
@@ -80,6 +129,43 @@
     def add_person(self, person, role):
         self.persons.setdefault(role, []).append(person)
 
+    def ordered_field_items(self):
+        def field_index(field_value):
+            field,value = field_value
+            try:
+                return self.field_order.index(field)
+            except ValueError:
+                sys.stderr.write('Unordered field: %s\n' % field)
+                return len(self.field_order)
+        return sorted(self.fields.items(),
+                      key=lambda field_value: field_index(field_value))
+
+    def year(self):
+        return int(self.fields.get('year', '0'))
+
+    def month(self):
+        month = self.fields.get('month', '0').lower()
+        if month == '0':
+            month = 0
+        else:
+            for format in ['%B', '%b', '%m']:
+                try:
+                    t = time.strptime(month, format)
+                    month = t.tm_mon
+                    break
+                except ValueError:
+                    continue
+            if not isinstance(month, int):
+                month = 0  # give up parsing month
+        assert month >= 0 and month <= 12, month
+        return month
+
+    def day(self):
+        day = int(self.fields.get('day', '0'))
+        assert day >= 0 and day <= 31, day
+        return day
+
+
 class Person(object):
     """Represents a person (usually human).
 

=== modified file 'pybtex/database/__init__.py'
--- pybtex/database/__init__.py	2010-01-20 19:34:11 +0000
+++ pybtex/database/__init__.py	2010-10-17 00:56:45 +0000
@@ -13,6 +13,8 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+import re
+
 from collections import defaultdict
 
 from pybtex.exceptions import PybtexError
@@ -66,3 +68,138 @@
             if crossref_count >= min_crossrefs
         )
         return sorted(citations)
+
+
+class KeyGenerator(object):
+    """
+    >>> from pybtex.core import FieldDict, Entry, Person
+    >>> bib_data = BibliographyData(entries={
+    ...     'old_key_A': Entry(type_='article',
+    ...           fields=FieldDict(parent=None, year='1234', month='February'),
+    ...           persons={'author': [Person('A. Uthor'), Person('O. Ther')]}),
+    ...     'old_key_B': Entry(type_='article',
+    ...           fields=FieldDict(parent=None, year='1234', month='January'),
+    ...           persons={'author': [Person('A. Uthor'), Person('O. Ther')]}),
+    ...     })
+    >>> kg = KeyGenerator()
+    >>> kg.rekey(bib_data)
+    >>> print sorted(kg.changed_keys.items())
+    [('old_key_A', 'uthor34b'), ('old_key_B', 'uthor34a')]
+    >>> print sorted(bib_data.entries.items())  # doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE
+    [('uthor34a', <pybtex.core.Entry object at 0x...>),
+     ('uthor34b', <pybtex.core.Entry object at 0x...>)]
+    """
+    def __init__(self):
+        self.changed_keys = {}   # changed_keys[old] = new
+        self.original_keys = {}  # original_keys[id(entry)] = original_key
+
+    def rekey(self, bib_data):
+        new_keys = {}
+        for key, entry in bib_data.entries.iteritems():
+            self.original_keys[id(entry)] = key
+            new_key = self.generate_key(key, entry)
+            new_keys.setdefault(new_key, []).append(entry)
+
+        collision = True
+        while collision:
+            collision = False
+            for key,entries in new_keys.items():
+                if len(entries) > 1:
+                    entries = new_keys.pop(key)
+                    _new_keys = self.handle_key_collision(key, entries)
+                    for entry,new_key in _new_keys.iteritems():
+                        new_keys.setdefault(new_key, []).append(entry)
+                        if len(new_keys[new_key]) > 1:
+                            collision = True
+                            # we've messed with new_keys, restart iteration
+                            break
+
+        for key,entries in new_keys.items():
+            entry = entries[0]
+            new_keys[key] = entry
+            if key != self.original_keys[id(entry)]:
+                old_key = self.original_keys[id(entry)]
+                self.changed_keys[old_key] = key
+
+        bib_data.entries = new_keys
+
+    def generate_key(self, old_key, entry):
+        """
+        Generate keys that look like
+        `<first-author-lowercase-lastname><two-digit-publication-year>`.
+
+        >>> from pybtex.core import FieldDict, Entry, Person
+        >>> kg = KeyGenerator()
+        >>> e = Entry(type_='article',
+        ...           fields=FieldDict(parent=None, year='1234'),
+        ...           persons={'author': [Person('A. Uthor')]})
+        >>> print kg.generate_key(old_key=None, entry=e)
+        uthor34
+
+        Non-letter characters are stripped from the lastname:
+
+        >>> for name in ['van Kampen, N.G.', 'H\"anggi, Peter']:
+        ...     e.persons['author'][0] = Person(name)
+        ...     print kg.generate_key(old_key=None, entry=e)
+        vanKampen34
+        hanggi34
+
+        If you want to avoid the auto-generated key, you can set the
+        key explicitly via the `key` field.
+
+        >>> e.fields['key'] = 'custom_key'
+        >>> print kg.generate_key(old_key=None, entry=e)
+        custom_key
+        """
+        if 'key' in entry.fields:
+            return entry.fields['key']
+        for role in ['author', 'editor']:
+            if role in entry.persons:
+                break
+        if role not in entry.persons:
+            raise PybtexError('No person fields in %s' % old_key)
+        first_author = entry.persons[role][0]
+        last = first_author.get_part_as_text('last')
+        von = first_author.get_part_as_text('prelast')
+        if len(von) > 0:
+            last = ''.join([von, last])
+        else:
+            last = last.lower()
+        last = re.sub('[^-a-zA-Z]', '', last)
+        return '%s%s' % (last, entry.fields['year'][-2:])
+
+    def handle_key_collision(self, old_key, entries):
+        entries.sort(key=self._collision_sort_key, )
+        return dict([(entry, old_key+self._hexavigesimal(i))
+                     for i,entry in enumerate(entries)])
+
+    def _collision_sort_key(self, entry):
+        tiebreaker = float('0.%d' % id(entry))
+        return entry.year()*1e4 + entry.month()*1e2 + entry.day() + tiebreaker
+
+    def _hexavigesimal(self, n):
+        """
+        Based on
+          http://www.python-forum.org/pythonforum/viewtopic.php?f=3&t=16144&start=0
+
+        >>> kg = KeyGenerator()
+        >>> for n in [0,1,2,25,26,27,51,52,53,678]:
+        ...   print "%4d == %s" % (n, kg._hexavigesimal(n))
+        ...
+           0 == a
+           1 == b
+           2 == c
+          25 == z
+          26 == ba
+          27 == bb
+          51 == bz
+          52 == ca
+          53 == cb
+         678 == bac
+        """
+        h = []
+        while True:
+            n,r = divmod(n,26)
+            h[0:0] = chr(ord('a')+r)
+            if n == 0:
+                return ''.join(h)

=== modified file 'pybtex/database/input/bibtex.py'
--- pybtex/database/input/bibtex.py	2010-09-02 21:18:15 +0000
+++ pybtex/database/input/bibtex.py	2010-10-17 00:56:45 +0000
@@ -13,10 +13,76 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-"""BibTeX parser"""
+"""BibTeX parser
+
+>>> import StringIO
+>>> parser = Parser()
+>>> bib_data = parser.parse_stream(StringIO.StringIO('''
+... %@String{COMMENT = "This macro is commented out."}
+...   %@String{INDENTED_COMMENT = "This macro is also commented out."}
+... @String{SCI = "Science"}
+... 
+... @String{JFernandez = "Fernandez, Julio M."}
+... @String{HGaub = "Gaub, Hermann E."}
+... @String{MGautel = "Gautel, Mathias"}
+... @String{FOesterhelt = "Oesterhelt, Filipp"}
+... @String{MRief = "Rief, Matthias"}
+... 
+... @Article{rief97,
+...   author =       MRief #" and "# MGautel #" and "# FOesterhelt
+...                  #" and "# JFernandez #" and "# HGaub,
+...   title =        "Reversible Unfolding of Individual Titin
+...                  Immunoglobulin Domains by {AFM}",
+...   journal =      SCI,
+...   volume =       276,
+...   number =       5315,
+...   pages =        "1109--1112",
+...   year =         1997,
+...   doi =          "10.1126/science.276.5315.1109",
+...   URL =          "http://www.sciencemag.org/cgi/content/abstract/276/5315/1109",
+...   eprint =       "http://www.sciencemag.org/cgi/reprint/276/5315/1109.pdf",
+...   note = "Percent signs in strings (%) are not treated as comments",
+... }
+... '''))
+
+Check that the commented out macro `COMMENT` was not parsed.
+
+>>> print sorted(parser.macros.keys())  # doctest: +NORMALIZE_WHITESPACE
+['apr', 'aug', 'dec', 'feb', 'foesterhelt', 'hgaub', 'jan', 'jfernandez',
+ 'jul', 'jun', 'mar', 'may', 'mgautel', 'mrief', 'nov', 'oct', 'sci', 'sep']
+
+Check that percent signs in strings are not treated as comments.
+
+>>> print bib_data.entries['rief97'].fields['note']
+Percent signs in strings (%) are not treated as comments
+
+To access the macros by their original (not lowercased) name, use
+
+>>> print '\\n'.join([str(item) for item
+...                  in sorted(parser.get_raw_macros().iteritems())])
+... # doctest: +REPORT_UDIFF
+('FOesterhelt', 'Oesterhelt, Filipp')
+('HGaub', 'Gaub, Hermann E.')
+('JFernandez', 'Fernandez, Julio M.')
+('MGautel', 'Gautel, Mathias')
+('MRief', 'Rief, Matthias')
+('SCI', 'Science')
+('apr', 'April')
+('aug', 'August')
+('dec', 'December')
+('feb', 'February')
+('jan', 'January')
+('jul', 'July')
+('jun', 'June')
+('mar', 'March')
+('may', 'May')
+('nov', 'November')
+('oct', 'October')
+('sep', 'September')
+"""
 
 from pyparsing import (
-        Word, Literal, CaselessLiteral, CharsNotIn,
+        Word, Literal, CaselessLiteral, CharsNotIn, Regex,
         nums, alphas, alphanums, printables, delimitedList, downcaseTokens,
         Suppress, Combine, Group, Dict,
         Forward, ZeroOrMore, Optional,
@@ -46,9 +112,13 @@
 
 file_extension = 'bib'
 
+def _normalize_whitespace(tok):
+    if tok.strip() == 'and':
+        return ' and '
+    return textutils.normalize_whitespace(tok)
 
 def normalize_whitespace(s, loc, toks):
-    return [textutils.normalize_whitespace(tok) for tok in toks]
+    return [_normalize_whitespace(tok) for tok in toks]
 
 
 class Parser(ParserBase):
@@ -60,6 +130,9 @@
         self.default_macros = dict(macros)
         self.person_fields = person_fields
 
+        comment = Regex(r'%.*')
+        comment.setName("LaTeX style comment")
+
         lbrace = Suppress('{')
         rbrace = Suppress('}')
         def bibtexGroup(s):
@@ -78,14 +151,17 @@
         name_chars = alphanums + '!$&*+-./:;<>?[\\]^_`|~\x7f'
         macro_substitution = Word(name_chars).setParseAction(self.substitute_macro)
         name = Word(name_chars).setParseAction(downcaseTokens)
+        raw_name = Word(name_chars)
         value = Combine(delimitedList(bibTeXString | Word(nums) | macro_substitution, delim='#'), adjacent=False)
 
         #fields
         field = Group(name + Suppress('=') + value)
+        raw_field = Group(raw_name + Suppress('=') + value)
         fields = Dict(delimitedList(field))
+        raw_fields = Dict(delimitedList(raw_field))
 
         #String (aka macro)
-        string_body = bibtexGroup(fields)
+        string_body = bibtexGroup(raw_fields)
         string = at + CaselessLiteral('STRING').suppress() + string_body
         string.setParseAction(self.process_macro)
 
@@ -105,6 +181,7 @@
         entry.setParseAction(lambda toks: self.process_entry(toks))
 
         self.BibTeX_entry = string | preamble | entry
+        self.BibTeX_entry.ignore(comment)
 
     def process_preamble(self, toks):
         self.data.add_to_preamble(toks[0])
@@ -123,6 +200,8 @@
                 for name in split_name_list(v):
                     entry.add_person(Person(name), k)
             else:
+                if k in entry.fields:
+                    raise PybtexError('Duplicate fields (%s) in %s.' % (k, key))
                 entry.fields[k] = v
 #        return (key, entry)
         self.data.add_entry(key, entry)
@@ -136,11 +215,13 @@
 
     def process_macro(self, s, loc, toks):
         self.macros[toks[0][0].lower()] = toks[0][1]
+        self.raw_macros[toks[0][0].lower()] = toks[0][0]
 
     def parse_stream(self, stream):
         self.unnamed_entry_counter = 1
 
         self.macros = dict(self.default_macros)
+        self.raw_macros = dict([(m,m) for m in self.default_macros.keys()])
         try:
 #            entries = dict(entry[0][0] for entry in self.BibTeX_entry.scanString(s))
             self.BibTeX_entry.searchString(stream.read())
@@ -149,3 +230,8 @@
             print e
             import sys
             sys.exit(1)
+        return self.data
+
+    def get_raw_macros(self):
+        return dict([(self.raw_macros[macro], value)
+                     for macro,value in self.macros.iteritems()])

=== modified file 'pybtex/database/input/bibtexml.py'
--- pybtex/database/input/bibtexml.py	2009-08-19 18:35:32 +0000
+++ pybtex/database/input/bibtexml.py	2010-10-17 00:56:45 +0000
@@ -35,6 +35,7 @@
         t = ET.parse(stream)
         entries = t.findall(bibtexns + 'entry')
         self.data.add_entries(self.process_entry(entry) for entry in entries)
+        return self.data
 
     def process_entry(self, entry):
         def process_person(person_entry, role):

=== modified file 'pybtex/database/input/bibyaml.py'
--- pybtex/database/input/bibyaml.py	2009-08-19 18:35:32 +0000
+++ pybtex/database/input/bibyaml.py	2010-10-17 00:56:45 +0000
@@ -32,6 +32,7 @@
             pass
 
         self.data.add_entries(entries)
+        return self.data
 
     def process_entry(self, entry):
         e = Entry(entry['type']) 

=== modified file 'pybtex/database/output/__init__.py'
--- pybtex/database/output/__init__.py	2010-09-05 14:50:17 +0000
+++ pybtex/database/output/__init__.py	2010-10-17 00:56:45 +0000
@@ -27,11 +27,11 @@
     def __init__(self, encoding=None):
         self.encoding = encoding
 
-    def write_file(self, bib_data, filename):
+    def write_file(self, bib_data, filename, *args, **kwargs):
         open_file = pybtex.io.open_unicode if self.unicode_io else pybtex.io.open_raw
         mode = 'w' if self.unicode_io else 'wb'
         with open_file(filename, mode, encoding=self.encoding) as stream:
-            self.write_stream(bib_data, stream)
+            self.write_stream(bib_data, stream, *args, **kwargs)
 
     def write_stream(self, bib_data, stream):
         raise NotImplementedError

=== modified file 'pybtex/database/output/bibtex.py'
--- pybtex/database/output/bibtex.py	2010-09-20 07:17:44 +0000
+++ pybtex/database/output/bibtex.py	2010-10-17 00:56:45 +0000
@@ -49,9 +49,10 @@
             return '{%s}' % s
 
     def check_braces(self, s):
-        end_brace_level = list(scan_bibtex_string(s))[-1][1]
-        if end_brace_level != 0:
-            raise BibTeXError('String has unmatched braces: %s' % s)
+        if len(s) > 0:
+            end_brace_level = list(scan_bibtex_string(s))[-1][1]
+            if end_brace_level != 0:
+                raise BibTeXError('String has unmatched braces: %s' % s)
 
     def write_stream(self, bib_data, stream):
         def write_field(type, value):

=== added file 'pybtex/database/output/clean_bibtex.py'
--- pybtex/database/output/clean_bibtex.py	1970-01-01 00:00:00 +0000
+++ pybtex/database/output/clean_bibtex.py	2010-10-17 00:56:45 +0000
@@ -0,0 +1,262 @@
+# Copyright (C) 2006, 2007, 2008, 2009  Andrey Golovizin
+#               2010  W.Trevor King
+# 
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import re
+import sys
+import textwrap
+
+import pybtex.io
+from pybtex.bibtex.exceptions import BibTeXError
+from pybtex.database import KeyGenerator
+from pybtex.database.input.bibtex import Parser
+from pybtex.database.output.bibtex import Writer as BibtexWriter
+
+
+class Writer(BibtexWriter):
+    """Outputs BibTeX markup
+
+    Differences from the `.bibtex` module:
+
+    * Citation keys are generated on the fly via KeyGenerator.rekey().
+ check
+    * Entries are alphabetized by citation key.
+ check
+    * Citation keys are listed on the same line as the entry declaration.
+ check
+    * Fields within an entry are ordered via Entry.ordered_field_items().
+ check
+    * Values that do not need to be surrounded by quotes are not.
+ check
+    * Month macros (`jan`, `feb`, ...) are used when possible.
+ check
+    * Names of people, journals, and publishers are broken out into
+      `@String` definitions for consistency.
+    * Long values are wrapped at a configurable character width.
+ check
+
+    >>> import StringIO
+    >>> import sys
+    >>> parser = Parser()
+    >>> bib_data = parser.parse_stream(StringIO.StringIO('''
+    ... @String{SCI = "Science"}
+    ...
+    ... @Article{ crazy-key,
+    ...   author = "Matthias Rief and Mathias Gautel and Filipp
+    ...            Oesterhelt and Julio M. Fernandez and Hermann E. Gaub",
+    ...   title =  "Reversible Unfolding of Individual Titin
+    ...            Immunoglobulin Domains by {AFM}",
+    ...   journal = SCI,
+    ...   volume = 276,
+    ...   number = 5315,
+    ...   pages = "1109--1112",
+    ...   year = 1997,
+    ...   doi = "10.1126/science.276.5315.1109",
+    ...   url = "http://www.sciencemag.org/cgi/content/abstract/276/5315/1109",
+    ...   eprint = "http://www.sciencemag.org/cgi/reprint/276/5315/1109.pdf",
+    ... }
+    ... '''))
+    >>> writer = Writer()
+    >>> writer.width = 70
+    >>> stderr = sys.stderr
+    >>> sys.stderr = StringIO.StringIO()
+    >>> writer.write_stream(bib_data, sys.stdout, parser.get_raw_macros())
+    ... # doctest: +REPORT_UDIFF
+    @string{JMFernandez = "Fernandez, Julio M."}
+    @string{HEGaub = "Gaub, Hermann E."}
+    @string{MGautel = "Gautel, Mathias"}
+    @string{FOesterhelt = "Oesterhelt, Filipp"}
+    @string{MRief = "Rief, Matthias"}
+    @string{SCI = "Science"}
+    <BLANKLINE>
+    @article { rief97,
+        author = MRief #" and "# MGautel #" and "# FOesterhelt #" and "#
+            JMFernandez #" and "# HEGaub,
+        title = "Reversible Unfolding of Individual Titin Immunoglobulin
+            Domains by {AFM}",
+        year = 1997,
+        journal = SCI,
+        volume = 276,
+        number = 5315,
+        pages = "1109--1112",
+        doi = "10.1126/science.276.5315.1109",
+        eprint = "http://www.sciencemag.org/cgi/reprint/276/5315/1109.pdf",
+        url =
+            "http://www.sciencemag.org/cgi/content/abstract/276/5315/1109"
+    }
+    <BLANKLINE>
+    >>> print sys.stderr.getvalue()  # doctest: +REPORT_UDIFF
+    New macros:
+      FOesterhelt = Oesterhelt, Filipp
+      HEGaub = Gaub, Hermann E.
+      JMFernandez = Fernandez, Julio M.
+      MGautel = Gautel, Mathias
+      MRief = Rief, Matthias
+    Changed keys:
+      "crazy-key" -> "rief97"
+    <BLANKLINE>
+    >>> sys.stderr = stderr
+    """
+
+    width = 79
+
+    def quote(self, s):
+        """
+        >>> w = Writer()
+        >>> print w.quote('The World')
+        "The World"
+        >>> print w.quote(r'The \emph{World}')
+        "The \emph{World}"
+        >>> print w.quote(r'The "World"')
+        {The "World"}
+        >>> try:
+        ...     print w.quote(r'The {World')
+        ... except BibTeXError, error:
+        ...     print error
+        String has unmatched braces: The {World
+        >>> print w.quote('1234')
+        1234
+        """
+        try:
+            int(s)
+            return s  # no need to quote numbers
+        except ValueError:
+            return BibtexWriter.quote(self, s)
+
+    def format_name(self, person):
+        def join(l):
+            return ' '.join([name for name in l if name])
+        first = person.get_part_as_text('first')
+        middle = person.get_part_as_text('middle')
+        prelast = person.get_part_as_text('prelast')
+        last = person.get_part_as_text('last')
+        lineage = person.get_part_as_text('lineage')
+        s = '' 
+        if last:
+            s += join([prelast, last])
+        if lineage:
+            s += ', %s' % lineage
+        if first or middle:
+            s += ', '
+            s += join([first, middle])
+        return s
+
+    def generate_person_macro(self, person):
+        """
+        Generate macros that look like
+        `<first-caps><middle-caps><prelast><last>'
+
+        >>> from pybtex.core import Person
+        >>> w = Writer()
+        >>> for name in ['Gaub, Hermann E.', 'van Kampen, N.G.',
+        ...              'H\"anggi, Peter']:
+        ...     p = Person(name)
+        ...     print w.generate_person_macro(p)
+        HEGaub
+        NGvanKampen
+        PHanggi
+        """
+        first = person.get_part_as_text('first')
+        middle = person.get_part_as_text('middle')
+        von = person.get_part_as_text('prelast')
+        last = person.get_part_as_text('last')
+        first = re.sub('[^A-Z]', '', first)
+        middle = re.sub('[^A-Z]', '', middle)
+        macro = ''.join([first, middle, von, last])
+        return re.sub('[^-a-zA-Z]', '', macro)
+
+    def generate_string_macro(self, string):
+        return re.sub('[^-a-zA-Z]', '', string)
+
+    def generate_inverse_macros(self, bib_data, macros):
+        new_macros = {}
+        inverse_macros = dict(
+            [(value,macro) for macro,value in macros.iteritems()])
+        def add_macro(macro, value):
+            if macro in macros:
+                while macro in macros and macros[macro] != value:
+                    macro += 'X'
+            new_macros[macro] = value
+            macros[macro] = value
+            inverse_macros[value] = macro
+        for key,entry in bib_data.entries.iteritems():
+            for role,persons in entry.persons.iteritems():
+                for person in persons:
+                    name = self.format_name(person)
+                    if name not in inverse_macros:
+                        macro = self.generate_person_macro(person)
+                        add_macro(macro, name)
+            for field in ['journal', 'publisher']:
+                if field in entry.fields:
+                    value = entry.fields[field]
+                    if value not in inverse_macros:
+                        macro = self.generate_string_macro(value)
+                        add_macro(macro, value)
+        return inverse_macros, new_macros
+
+    def write_stream(self, bib_data, stream, macros={}):
+        inverse_macros,new_macros = self.generate_inverse_macros(
+            bib_data, macros)
+        sys.stderr.write(
+            'New macros:\n  %s\n'
+            % '\n  '.join(['%s = %s' % (macro,value) for macro,value
+                           in sorted(new_macros.items())]))
+        def write_field(type, value, raw=False):
+            if raw == True:
+                pass
+            elif value in inverse_macros:
+                value = inverse_macros[value]
+            else:
+                value = self.quote(value)
+            stream.write(u',\n%s' % textwrap.fill(
+                    '%s = %s' % (type, value),
+                    width=self.width,
+                    initial_indent=' '*4,
+                    subsequent_indent=' '*8))
+        def write_persons(persons, role):
+            if persons:
+                write_field(role, u' #" and "# '.join(
+                        [inverse_macros[self.format_name(person)]
+                         for person in persons]), raw=True)
+        def write_preamble(preamble):
+            if preamble:
+                stream.write(u'@preamble{%s}\n\n' % self.quote(preamble))
+
+        kg = KeyGenerator()
+        kg.rekey(bib_data)
+        sys.stderr.write(
+            'Changed keys:\n  %s\n'
+            % '\n  '.join(['"%s" -> "%s"' % (old,new) for old,new
+                           in sorted(kg.changed_keys.items())]))
+
+        need_blank_line = False
+        parser = Parser()
+        for value, macro in sorted(inverse_macros.iteritems()):
+            if macro not in parser.default_macros:
+                need_blank_line = True
+                stream.write(u'@string{%s = %s}\n'
+                             % (macro, self.quote(value)))
+        if need_blank_line == True:
+            stream.write(u'\n')
+
+        write_preamble(bib_data.preamble())
+
+        for key, entry in sorted(bib_data.entries.items()):
+            stream.write(u'@%s { %s' % (entry.type, key))
+            for role, persons in entry.persons.iteritems():
+                write_persons(persons, role)
+            for type, value in entry.ordered_field_items():
+                write_field(type, value)
+            stream.write(u'\n}\n\n')