Merge lp:~brian-murray/ubuntu/quantal/apport/string-bytes into lp:ubuntu/quantal/apport

[Brian Murray]

I discovered that regular expressions in the ubuntu general hook were failing when checking DpkgTerminalLog files. I've dealt with this by treating DpkgTerminalLog as a bytes object in trim_log and converting it to a string object so the regular expressions don't have to be modified and continue to be readable. However, I think its worth a review by someone else.

[Martin Pitt (pitti)]

Looks good to me, if you tested this. Thanks!

Merged into packaging branch, but closing manually since this MP specifies the wrong merge target.

review: Approve

=== added directory '.bzr-builddeb'
=== added file '.bzr-builddeb/default.conf'
--- .bzr-builddeb/default.conf	1970-01-01 00:00:00 +0000
+++ .bzr-builddeb/default.conf	2012-07-10 22:38:19 +0000
@@ -0,0 +1,2 @@
+[BUILDDEB]
+merge = True

=== added file '.bzrignore'
--- .bzrignore	1970-01-01 00:00:00 +0000
+++ .bzrignore	2012-07-10 22:38:19 +0000
@@ -0,0 +1,9 @@
+apport/packaging_impl.py
+debhelper/dh_apport.1
+doc/*.aux
+doc/*.log
+doc/*.pdf
+doc/*.toc
+build
+dist
+MANIFEST

=== added file 'AUTHORS'
--- AUTHORS	1970-01-01 00:00:00 +0000
+++ AUTHORS	2012-07-10 22:38:19 +0000
@@ -0,0 +1,34 @@
+Copyright:
+---------
+General:
+  Copyright (C) 2006 - 2011 Canonical Ltd.
+
+backends/packaging_rpm.py:
+  Copyright (C) 2007 Red Hat Inc.
+
+Authors and Contributors:
+-------------------------
+Martin Pitt <[e-mail]>:
+  Lead developer, design, backend, GTK frontend development,
+  maintenance of other frontends
+
+Michael Hofmann <[e-mail]>:
+  Creation of Qt4 and CLI frontends
+
+Richard A. Johnson <[e-mail]>:
+  Changed Qt4 frontend to KDE frontend
+
+Robert Collins <[e-mail]>:
+  Python crash hook
+
+Will Woods <[e-mail]>:
+  RPM packaging backend
+
+Matt Zimmerman <[e-mail]>:
+  Convenience function library for hooks (apport/hookutils.py)
+
+Troy James Sobotka <[e-mail]>:
+  Apport icon (apport/apport.svg)
+
+Kees Cook <[e-mail]>:
+  Various fixes, additional GDB output, SEGV parser.

=== renamed file 'AUTHORS' => 'AUTHORS.moved'
=== added file 'COPYING'
--- COPYING	1970-01-01 00:00:00 +0000
+++ COPYING	2012-07-10 22:38:19 +0000
@@ -0,0 +1,339 @@
+		    GNU GENERAL PUBLIC LICENSE
+		       Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+			    Preamble
+
+  The licenses for most software are designed to take away your
+freedom to share and change it.  By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users.  This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it.  (Some other Free Software Foundation software is covered by
+the GNU Lesser General Public License instead.)  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+  To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have.  You must make sure that they, too, receive or can get the
+source code.  And you must show them these terms so they know their
+rights.
+
+  We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+  Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software.  If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+  Finally, any free program is threatened constantly by software
+patents.  We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary.  To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+		    GNU GENERAL PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License.  The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language.  (Hereinafter, translation is included without limitation in
+the term "modification".)  Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope.  The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+  1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+  2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+    a) You must cause the modified files to carry prominent notices
+    stating that you changed the files and the date of any change.
+
+    b) You must cause any work that you distribute or publish, that in
+    whole or in part contains or is derived from the Program or any
+    part thereof, to be licensed as a whole at no charge to all third
+    parties under the terms of this License.
+
+    c) If the modified program normally reads commands interactively
+    when run, you must cause it, when started running for such
+    interactive use in the most ordinary way, to print or display an
+    announcement including an appropriate copyright notice and a
+    notice that there is no warranty (or else, saying that you provide
+    a warranty) and that users may redistribute the program under
+    these conditions, and telling the user how to view a copy of this
+    License.  (Exception: if the Program itself is interactive but
+    does not normally print such an announcement, your work based on
+    the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole.  If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works.  But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+  3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+    a) Accompany it with the complete corresponding machine-readable
+    source code, which must be distributed under the terms of Sections
+    1 and 2 above on a medium customarily used for software interchange; or,
+
+    b) Accompany it with a written offer, valid for at least three
+    years, to give any third party, for a charge no more than your
+    cost of physically performing source distribution, a complete
+    machine-readable copy of the corresponding source code, to be
+    distributed under the terms of Sections 1 and 2 above on a medium
+    customarily used for software interchange; or,
+
+    c) Accompany it with the information you received as to the offer
+    to distribute corresponding source code.  (This alternative is
+    allowed only for noncommercial distribution and only if you
+    received the program in object code or executable form with such
+    an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it.  For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable.  However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+  4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License.  Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+  5. You are not required to accept this License, since you have not
+signed it.  However, nothing else grants you permission to modify or
+distribute the Program or its derivative works.  These actions are
+prohibited by law if you do not accept this License.  Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+  6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions.  You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+  7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all.  For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices.  Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+  8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded.  In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+  9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number.  If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation.  If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+  10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission.  For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this.  Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+			    NO WARRANTY
+
+  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+		     END OF TERMS AND CONDITIONS
+
+	    How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software; you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 2 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License along
+    with this program; if not, write to the Free Software Foundation, Inc.,
+    51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+    Gnomovision version 69, Copyright (C) year name of author
+    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary.  Here is a sample; alter the names:
+
+  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+  `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+  <signature of Ty Coon>, 1 April 1989
+  Ty Coon, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs.  If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.

=== renamed file 'COPYING' => 'COPYING.moved'
=== added file 'NEWS'
--- NEWS	1970-01-01 00:00:00 +0000
+++ NEWS	2012-07-10 22:38:19 +0000
@@ -0,0 +1,1564 @@
+This file summarizes the major and interesting changes for each release. For a
+detailled list of changes, please see ChangeLog.
+
+2.4 (UNRELEASED):
+-----------------
+Improvements:
+ * apport_python_hook.py: For org.freedesktop.DBus.Error.ServiceUnknown
+   exceptions, add a 'DbusErrorAnalysis' field to the report which points out
+   whether any .service file provides the service it tried to talk to, and
+   whether the processes for those are running. This helps to determine the
+   root cause for such errors (missing dependencies, broken .service files,
+   talking to the wrong bus, etc.) (LP: #1020572)
+ * hookutils.py, attach_alsa(): Use alsa-info.sh when available. Thanks David
+   Henningson.
+
+Bug fixes:
+ * ui tests, test_wait_for_pid(): Fix eternal hang when running as root.
+ * testsuite: Fix ResourceWarnings when running with Python 3.
+ * test_python_crashes.py: Fix race condition in timeout test.
+
+2.3 (2012-07-09):
+-----------------
+Improvements:
+ * launchpad.py: Rework test suite to not use Launchpad's +storeblob facility
+   at all any more. It almost never works on staging and is horribly slow. Fake
+   the bug creation from a blob by manually creating the comment and
+   attachments ourselves, and just assume that storeblob works on production.
+   Also change the structure to allow running every test individually.
+ * crash-digger: Add --crash-db option to specify a non-default crash databae
+   name. (LP: #1003506)
+ * apport-gtk: Add --hanging option to specify the process ID of a hanging
+   application. If the user chooses to report this error, apport will terminate
+   the pid with SIGABRT, otherwise it will send SIGKILL. The normal core pipe
+   handler will be used to process the resulting report file, with a .hanging
+   file in /var/crash to separate these from regular crashes.
+
+Bug fixes:
+ * apport: Also treat a binary as modified if the /proc/pid/exe symlink does
+   not point to an existing file any more. (LP: #984944)
+ * Fix PEP-8 violations picked up by latest pep8 checker.
+ * ui.py: Do not ignore certain exceptions during upload which are not likely
+   to be a network error.
+ * launchpad.py: Recongize Launchpad projects for bug query and marking
+   operations. (LP: #1003506)
+ * packaging-apt-dpkg.py: Fix get_source_tree() to work with apt sandboxes.
+ * apport-retrace: Turn StacktraceSource generation back on, now that it works
+   with the current sandboxing.
+ * launchpad.py: Ensure that upload chunk size does not underrun. (LP: #1013334)
+ * apport_python_hook: Fix UnicodeEncodeError crash with Python 2 for
+   exceptions with non-ASCII characters. (LP: #972436)
+ * test_ui_kde.py: Fix occasional test failure in test_1_crash_details if the
+   application ends before the "is progress bar visible" check is done.
+
+2.2.5 (2012-06-21):
+-------------------
+ * launchpad.py: Fix str vs. bytes crash for already known bugs, take 2. (LP: #1015788)
+ * apport/ui.py, get_desktop_entry(): Disable interpolation, to correctly read
+   desktop files with % signs. (LP: #1014341)
+ * apport/ui.py: Fix rare crash if a report is already being updated in the
+   background when the UI tries to update a previous version. (LP: #949196)
+ * GTK and KDE UI tests: Avoid eternal hangs due to "this is not a distro
+   package" error messages.
+
+2.2.4 (2012-06-21):
+--------------------
+Bug fixes:
+ * test_apport_unpack.py: Fix test_unpack_python() test when running the
+   system-installed tests.
+ * data/java_uncaught_exception: Fix for Python 3.
+ * test_signal_crashes.py: Show crash reports in /var/crash/.
+ * test_crash_digger.py: Do not write crash reports of crash-digger into system
+   /var/crash, use a temporary directory.
+ * test/run: Wait for a previous xvfb server to finish before trying to start
+   one. This fixes a race condition in the KDE UI tests which often failed to
+   start up xvfb.
+ * apport-cli: Unbreak "keep" option. (LP: #1007826)
+ * launchpad.py: Fix str vs. bytes crash for already known bugs. (LP: #1015788)
+
+2.2.3 (2012-06-15):
+-------------------
+Bug fixes:
+ * test/run: Do not run pep8 and pyflakes when running against the sytem
+   installed Apport.
+ * test_backend_apt_dpkg.py: For the "are we online" check, verify that we can
+   download from http://ddebs.ubuntu.com/, not just whether we have a default
+   route. The latter is not sufficient for e. g. buildd environments which are
+   online, but are restricted by proxies or firewalls.
+ * test_report.py: Call "sync" after test script write core dumps, to ensure
+   that subsequent operations have a complete one.
+ * test_signal_crashes.py: Drop the broken and obsolete test_local_python()
+   test. Instead, add two tests which check proper logging.
+ * launchpad.py: Fix urlopen() for Python3. Thanks Steve Langasek.
+ * test/run: Run the tests under LC_MESSAGES=C, to avoid failing tests on
+   translated strings.
+
+2.2.2 (2012-06-13):
+-------------------
+Improvements:
+ * testsuite: Run with Python 3 by default. To test with Python 2, run
+   "PYTHON=python2 test/run".
+
+Bug fixes:
+ * apport: Redefine sys.std{out,err} when redirecting output, as they are None
+   in Python 3 when being called from the kernel.
+ * test/test_signal_crashes.py: Clean up unexpected core dumps on failed test
+   cases.
+ * apport-gtk: Fix crash when closing the crash dialog while the information is
+   being collected.
+ * hookutils.py, xsession_errors(): Fix crash when running under a non-UTF8 locale.
+ * data/apport: Do not use sys.stdin.fileno(), it is invalid when being called
+   from the kernel with Python 3.
+ * data/apport: When core dumps are enabled, read them from the written report
+   instead of directly from stdin (and then reading the written core file into
+   the .crash report). If the core file size is limited, we otherwise stop
+   reading the core dump from the kernel in the middle and have no (or a
+   broken) core dump to be put into the report.
+ * data/apport: Properly close the written crash report before changing its
+   permissions to be readable. This prevents having crash reporting UI from
+   looking at incomplete .crash files.
+
+2.2.1 (2012-06-11)
+------------------
+Bug fixes:
+ * apport-cli: Port to work with Python 3.
+ * setup.py: When fixing hashbang lines of installed scripts, only include the
+   major Python version.
+ * hookutils.py, read_file, attach_file(), attach_file_if_exists(): Convert
+   file contents to unicode if the contents is UTF-8, or the newly added
+   force_unicode argument is True.
+ * hooktuils, command_output(): Convert output to unicode by default, and add
+   a "decode_utf8" parameter to disable this.
+ * hookutils.py, recent_logfile(): Fix fd leak.
+ * data/apport: Do not assume that sys.stdout and sys.stderr always have a
+   name; they can be None in Python 3.
+ * data/dump_acpi_tables.py: Fix for Python 3.
+
+2.2 (2012-06-11)
+----------------
+Improvements:
+ * Clean up module imports.
+ * test/run: Run pyflakes, if available.
+ * package_hook: Add --tags option. Thanks to Brian Murray.
+ * launchpad.py: Drop the external multipartpost_handler.py (which is not
+   portable to Python 3) and replace it with using the standard email module.
+ * launchpad.py: Also work with Python 3. Deal gracefully with a missing
+   "launchpadlib" module; this is not yet available for Python 3, but not
+   required for client-side reporting.
+ * apport-kde: Port to work with Python 3.
+
+Bug fixes:
+ * apport-retrace: Fix crash when using the --procmaps option.
+ * setup.py: Update hashbang lines of installed scripts in data directory to
+   the python executable setup.py was run with, similar to what already happens
+   to scripts installed to ../bin/.
+
+2.1.1 (2012-05-30)
+------------------
+Improvements:
+ * launchpad.py: When closing a bug as a duplicate, copy some well-known tags
+   to the master bug. Thanks Brian Murray.
+ * launchpad.py: Set importance of Python crash reports to "Medium" by default,
+   similar to signal crashes. Thanks Brian Murray.
+ * hookutils.py: Add attach_default_grub() convenience function from the grub2
+   package hook so it can be used by other packages. Thanks Brian Murray.
+ * launchpad.py: Make Launchpad bug subscription user/team configurable: The
+   initial subscriber after filing a bug can be set with the
+   "initial_subscriber" crashdb option, and the team which gets subscribed
+   after retracing with "triaging_team". (LP: #980726)
+
+Bug fixes:
+ * report.py: Do not change the SourcePackage: field if the binary package is
+   not installed and does not exist. This fixes source package hooks to
+   actually work in some cases where source and binary package names overlap.
+   (part of LP: #993810)
+ * apport-gtk, apport-kde: Avoid collecting information twice in "bug update"
+   mode. This caused a crash in cases where the source package in a bug report
+   does not correspond to an installed binary package. (LP: #993810)
+
+2.1 (2012-05-18)
+----------------
+Improvements:
+ * packaging.py, install_packages(): Add permanent_rootdir flag and if set,
+   only unpack newly downloaded packages. Implement it for the apt/dpkg
+   backend. Thanks Evan Dandrea.
+ * apport-retrace: Add --sandbox-dir option for keeping a permanent sandbox
+   (unpacked packages). This provides a considerable speedup. Thanks Evan
+   Dandrea.
+ * crash-digger: Add --sandbox-dir option and pass it to apport-retrace.
+ * Fix the whole code to be PEP-8 compatible, and enforce this in test/run by
+   running the "pep8" tool.
+ * GTK UI tests: Ensure that there are no GLib/GTK warnings or criticals.
+ * Support Python 3. Everything except the launchpad crashdb backend now works
+   with both Python 2 and 3. An important change is that the load(),
+   write(), and write_mime() methods of a ProblemReport and apport.Report
+   object now require the file stream to be opened in binary mode.
+ * data/apport: Ignore a crash if the executable was modified after the process
+   started. This often happens if the package is upgraded and a long-running
+   process is not stopped before. (LP: #984944)
+ * Add test cases for apport-unpack.
+ * apport-retrace: Add information about outdated packages to the
+   "RetraceOutdatedPackages" field.
+ * ui.py: Drop python-xdg dependency, use ConfigParser to read the .desktop
+   files.
+
+Bug fixes:
+ * apport-gtk: Work around GTK crash when trying to set pixmap on an already
+   destroyed parent window. (LP: #938090)
+ * data/dump_acpi_tables.py: Fix crash on undefined variable with non-standard
+   tables. (LP: #982267)
+ * backends/packaging-apt-dpkg.py: Fix crash if a package is installed, but has
+   no candidates in apt. (LP: #980094)
+ * data/general-hooks/generic.py: Bump minimum free space requirement from 10
+   to 50 MB. 10 is not nearly enough particularly for /tmp. (LP: #979928)
+ * hookutils.py, recent_logfile(): Use a default limit of 10000 lines and call
+   "tail" instead of reading the whole file. This protects against using up all
+   memory when there are massive repeated log messages. (LP: #984256)
+ * apport-gtk: Do not assume that an icon requested for size 42 actually
+   delivers size 42; some themes do not have this available and deliver a
+   smaller one instead, causing overflows. Also, copy the image as
+   gtk_icon_theme_load_icon() returns a readonly result which we must not
+   modify. (LP: #937249)
+ * ui.py: Don't show the duplicate warning when the crash database does not
+   accept the problem type, and they are just being sent to whoopsie. Thanks
+   Evan Dandrea. (LP: #989779)
+ * report.py: Correctly escape the file path passed to gdb.
+ * apport-gtk, apport-kde: Do not show the information collection progress
+   dialog if the crash database does not accept this kind of report. In that
+   case whoopsie will upload it in the background and the dialog is not
+   necessary.  (LP: #989698)
+
+2.0.1 (2012-04-10)
+------------------
+Bug fixes:
+ * test_ui_gtk.py: Disable package hooks for the tests, as they might ask for
+   sudo passwords and other interactive bits, and thus make the tests hang.
+ * test_backend_apt_dpkg.py: Fix checks for the installation of -dbgsym
+   packages. This should always happen, as the sandboxes have a ddeb apt
+   source. Only make it conditional on the system apt sources in the "use
+   system config" test.
+ * test_report.py: Sleep a bit after calling our test crash script, to ensure
+   the kernel has time to finish writing the core file.
+ * generic package hook: Also check /tmp for enough space. Thanks Brian Murray.
+   (LP: #972933)
+ * problem_report.py, write_mime(): Fix regression from version 1.95: Add a
+   value as attachment if it is bigger than 1000 bytes, not if it is bigger
+   than 100. (LP: #977882)
+
+Improvements:
+ * packaging-apt-dpkg.py: Avoid constructing and updating the apt.Cache()
+   objects multiple times, to speed up retracing. Thanks Evan Dandrea.
+   (LP: #973494)
+
+2.0 (2012-03-30)
+----------------
+This is the final 2.0 release, featuring the overhauled and simplified GUI,
+support for whoopsie-daemon, and client-side duplicate checking.
+
+Bug fixes:
+ - report.py, anonymize(): Only replace whole words, not substrings.
+   (LP: #966562)
+ - apport_python_hook.py: Fix filtering of org.freedesktop.DBus.Error.NoReply
+   exceptions. (LP: #958575)
+ - crashdb.py: When publishing the crash database, cut hash file names after
+   quoting, to avoid that the quoting causes them to become too long.
+   (LP: #968070) This also uncovered that known() did not actually find any
+   signature which contained an URL-quoted character, therefore breaking
+   client-side duplicate checking in a lot of cases. Double-quote the file name
+   now, as urlopen() unquotes it.
+ - Add a new crash database option "problem_types" and a CrashDatabase method
+   "accepts(report)". This can be used to stop uploading particular problem
+   report types to that database. E. g. a distribution might decide to not get
+   "Crash" reports any more after release.  Document the new option in
+   doc/crashdb-conf.txt.
+ - ui.py: Do not upload a report if the crash database does not accept the
+   report's type. This behaviour is not really correct, but necessary as long
+   as we only support a single crashdb and have whoopsie hardcoded.  Once we
+   have multiple crash dbs, we need to not even present the data if none of the
+   DBs wants the report. See LP #957177 for details. (LP: #968121)
+ - ui.py: Do not short-circuit information collection if report already has a
+   "DistroRelease" field, as the GUIs add that in some cases. Check for
+   "Dependencies" instead. This fixes information collection for kernel
+   problems (which now has a full GTK GUI test case). (LP: #968488)
+
+1.95 (2012-03-22)
+-----------------
+Bug fixes:
+ - ui.py: Ensure that the report file is readable by the crash reporting daemon
+   after running through collect_info(). Thanks Evan Dandrea.
+ - apport-gtk, apport-kde: Set the window title to the distribution name, as
+   per http://wiki.ubuntu.com/ErrorTracker#error . Thanks Evan Dandrea.
+   (LP: #948015)
+ - test/run: Ignore obsolete packages on the system, to avoid breaking the GUI
+   tests due to them.
+ - apport-gtk, apport-kde: When reporting a "system crash", don't say "... of
+   this program version", but "...of this type", as we don't show a program
+   version in the initial dialog (https://wiki.ubuntu.com/ErrorTracker#error)
+   (LP: #961065)
+ - problem_report.py, write_mime(): Do not put a key inline if it is bigger
+   than 1 kB, to guard against very long lines. (LP: #957326)
+ - etc/cron.daily/apport: Do not remove whoopsie's *.upload* stamps every day,
+   only if they are older than a week. whoopsie comes with its own cron job
+   which deals with them. Thanks Steve Langasek. (LP: #957102)
+ - report.py, mark_ignore(): Fix crash if executable went away underneath us.
+   (LP: #961410)
+ - apport-gtk: Do not compare current continue button label against a
+   translated string.  Instead just remember whether or not we can restart the
+   application. (LP: #960439)
+ - hookutils.py, command_output(): Add option to keep the locale instead of
+   disabling it.
+ - hookutils.py, command_output(): Actually make the "input" parameter work,
+   instead of causing an eternal hang. Add tests for all possible modes of
+   operation.
+ - hooktuils.py: Change root_command_output() and attach_root_command_outputs()
+   to disable translated messages (LC_MESSAGES=C) only as part of the command
+   to be run, not already for the root prefix command. This will keep the
+   latter (gksu, kdesudo, etc.) translated. (LP: #961659)
+ - apport-gtk: Cut off text values after 4000 characters, as Gtk's TreeView
+   does not get along well with huge values. KDE's copes fine, so continue to
+   display the complete value there. (LP: #957062)
+ - apport-gtk: Make details window resizable in bug reporting mode.
+ - crashdb.py, known(): Check the address signature duplicate database if the
+   symbolic signature exists, but did not find any result. (LP: #103083)
+ - ui.py: Run anonymization after checking for duplicates, to prevent host or
+   user names which look like hex numbers to corrupt the stack trace.
+   (LP: #953104)
+ - apport-gtk: Require an application to both have TERM and SHELL in its
+   environment to consider it a command line application that was started by
+   the user. (LP: #962130)
+ - backends/packaging-apt-dpkg.py, _check_files_md5(): Fix double encoding,
+   which caused UnicodeDecodeErrors on non-ASCII characters in an md5sum file.
+   (LP: #953682)
+ - apport-kde, apport-gtk: Only show "Relaunch" if the report has a
+   ProcCmdline, otherwise we cannot restart it. (LP: #956173)
+
+Improvements:
+ - hookutils.py, attach_alsa(): Add the full "pacmd list" output instead of
+   just sinks and sources. Thanks David Henningsson.
+ - apport-gtk, apport-kde: Show the ExecutablePath while we're collecting data
+   for the crash report. Thanks Evan Dandrea. (LP: #938707).
+
+1.94.1 (2012-03-07)
+-------------------
+Bug fixes:
+ - test_ui_kde.py: Re-enable inadvertently disabled "bug report for uninstalled
+   package" test.
+ - ui.py, collect_info(): Do not assume that reports have a "ProblemType"
+   field. This is not the case when updating a bug. (LP: #947519)
+ - apport-cli: Consistently handle unicode vs. byte arrays. (LP: #946207)
+ - report.py, anonymize(): Fix crash when the hostname or user name contain
+   non-ASCII characters. (LP: #945230)
+ - packaging-apt-dpkg.py: Fix UnicodeDecodeError on unexpected md5sum output.
+   (LP: #921037)
+ - apport-gtk: Fix handling of non-ASCII strings in message dialogs.
+   (LP: #865394)
+
+1.94 (2012-03-02)
+-----------------
+Bug fixes:
+ - apport: Set the group of written reports to "whoopsie" if that group exists.
+ - Fix tests to run properly against the system-installed modules and binaries.
+ - test/run: Run under LC_MESSAGES=C to avoid test failures due to translated
+   strings.
+ - general-hooks/generic.py: Also attach xsession-errors for programs that link
+   to libgtk-3.
+ - launchpad.py: Properly handle "Expired" status, to avoid marking new bugs as
+   duplicates of expired ones. (LP: #941854)
+ - apport: Fix crash if the "whoopsie" group does not exist. (LP: #942326)
+ - report.py, crash_signature(): Do not put "<module>" frames into Python crash
+   signatures that happen outside of function/method calls. Fall back to the
+   file/line number as a frame description instead. This will do a much better
+   job at disambiguating e. g. different ImportError crashes. (LP: #920403)
+ - Make "binary changed since the time of the crash" error message more
+   comprehensible, thanks Paolo Rotolo. (LP: #942830)
+ - crashdb.py, check_duplicate(): It can happen that a bug gets identified as
+   being a duplicate of bug S by symbolic signatures and a duplicate of bug A
+   by address signatures. Empirical evidence shows that this is due to the
+   unavoidable jitter in stack traces (A and S not being identified as
+   duplicates as their signatures differ slightly) and not a logic error. So
+   instead of erroring out, duplicate all three bugs and keep the lowest number
+   as the master ID. (LP: #943117)
+ - Revert the usage of multiple nested threads during data collection, and
+   switch back to only using one UI thread. The UI implementations can, and now
+   do, decide between showing a spinner and showing a progress dialog in the
+   ui_*_info_collection_progress() methods. This fixes libX11 crashes when
+   multiple UI threads do changes concurrently (LP: #901675), and also avoids
+   multi-thread induced crashes in Pango (LP: #943661). The removal of the
+   collect() method also fixes the new crashes in it. (LP: #942098, #939803)
+ - ui.py, get_desktop_entry(): Fix crash on uninstalled package. (LP: #940984)
+ - data/unkillable_shutdown: Fix crash on race condition when PID goes away
+   while the report is created. (LP: #546369)
+ - apport/hookutils.py, pci_devices(): Fix crash on unexpected lines from
+   lspci. (LP: #904489)
+ - Drop hardcoded "Ubuntu" words again which crept in with the whoopsie support
+   merge. Use the DistroRelease: field.
+ - apport-kde: Fix Home page URL in KApplication metadata.
+ - apport-gtk: Fix resizability and size after hiding details. (LP: #405418)
+
+Improvements:
+ - test/run: Drop "local" argument. This now tests against the source tree when
+   run in the source tree root, and against the system libraries/programs when
+   run from anywhere else.
+ - test/run: Consider command line arguments as test names and only run those
+   when given. Also support just running a single test.
+ - testsuite: Force the skipping of online tests when $SKIP_ONLINE_TESTS is
+   set.
+ - hookutils.py, xsession_errors(): Add a reasonable default pattern which
+   matches glib-style warnings, errors, criticals etc. and X window errors.
+   In data/general-hooks/generic.py, call it with that default instead of the
+   rather incomplete custom pattern. (LP: #932660)
+ - packaging.py: Add get_package_origin() method, and implement it for
+   apt-dpkg.
+ - report.py, add_package_info(): Add "[origin: ...]" tag to "Package" and
+   "Dependencies" fields for any package which is not native to the
+   distribution. If any such package is present, tag the report with
+   "third-party-packages" in data/general-hooks/generic.py. (LP: #927912)
+ - apport/packaging.py: Add get_uninstalled_package() method as a helper method
+   for the test suite. Use it instead of a hardcoded Debian/Ubuntu specific
+   name in test/test_hooks.py.
+ - test/test_ui_{gtk,kde}.py: Add test cases for complete UI workflow runs for
+   reporting a bug against an installed/uninstalled package, and reporting a
+   crash with and without showing details. This reproduces the recent crashes
+   like LP #901675 or LP #943661.
+ - test_ui.py: Add a test case for reporting a complete report on uninstalled
+   package. This happens when reporting a problem from a different machine
+   through copying a .crash file.
+ - test/run: Add a test that there are no hardcoded "Ubuntu" words in the
+   source. The code should use the DistroRelease: field or lsb_release.
+
+1.93 (2012-02-23):
+------------------
+Bug fixes:
+ - apport-gtk: Fix crash on nonexisting icon. Thanks Evan Dandrea.
+   (LP: #937354)
+ - ui.py, open_url(): Revert back to calling sudo instead of dropping
+   privileges ourselves; with the latter, calling firefox as the sudo'ing user
+   fails. (LP: #916810, #938128)
+ - ui.py: Fix aborting with "AssertionError" if the report is already known,
+   but without an URL. (LP: #938778)
+ - launchpad.py: If a bug is already known, but the report is private, do not
+   send the report. There is little sense piling up lots of duplicates.
+   (LP: #938700)
+ - test/crash: Fix regression of test_crash_apport(), consider $TERM a
+   non-sensitive variable.
+ - ui.py: Fix test failures for data collection progress, they are not expected
+   to happen for "ProblemType: Crash" any more (happens in the background
+   during sending, or if user clicks on "Show Details").
+ - test/hooks: Use a package from Debian/Ubuntu main, so that this works better
+   during package builds on build servers.
+ - test/python: Do not assume that /var/crash/ exists. Use /var/tmp/ for the
+   fake binaries instead.
+ - data/general-hooks/parse_segv.py: Fix test case name.
+ - ui.py: Fix crash on invalid core dumps. (LP: #937215)
+ - launchpad.py: Fix crash on unicode report titles. (LP: #896626)
+
+Improvements:
+ - apport-gtk: Show the most interesting fields first in the details view.
+ - do-release: Call pyflakes and abort on errors other than unused imports.
+ - Move all test suites out of the code modules into test/test_<module>.py.
+   This avoids having to load it every time the program runs, and also allows
+   running the tests against the installed version of Apport.
+ - Clean up the other executable test script in test/* and change them to the
+   same structure as the module tests.
+
+1.92 (2012-02-20):
+------------------
+Bug fixes:
+ - ui.py: Fix wrong creation of "~" folder instead of expanding it to home
+   directory when using "Examine locally". Thanks Jason Conti! (LP: #909149)
+ - Replace file() calls with open() for Python 3 compatibility. Thanks Colin
+   Watson!
+ - launchpad.py: Avoid sending tag names with upper case. (LP: #924181)
+ - report.py, crash_signature_addresses(): Fix crash if report does not have
+   "Signal".
+ - apport-gtk: Fix resize handling of expander in details window. Thanks Thomas
+   Bechtold! (LP: #930562)
+ - Clean up unnecessary imports. Thanks Evan Dandrea!
+
+Improvements:
+ - man/apport-bug.1: Mention where crash files are stored. Thanks David
+   Kastrup.
+ - hookutils.py, attach_hardware(): Sort ProcModules, thanks Brian Murray.
+ - launchpad.py: Keep "Dependencies" attachment in duplicates. Thanks Brian
+   Murray.
+ - Reorganize the GNOME and KDE user interface to do the crash notifications
+   and detail browser in a single dialog. Add test/gtk and test/kde tests to
+   check expected dialog layout for different cases. Thanks Evan Dandrea!
+ - Add support for the whoopsie-daisy crash reporting daemon by creating
+   zero-byte .upload file stamps for crash reports. Thanks Evan Dandrea!
+
+1.91 (2012-01-18):
+------------------
+Bug fixes:
+ - crashdb.py, check_duplicate(): If a crash has a signature but no existing
+   duplicate in the DB, also check for an existing address signature duplicate
+   in the DB.
+ - apport-retrace: Use DistroRelease specific subdirectory of the cache dir for
+   mapping a file to a package, as these maps are release specific.
+ - packaging-apt-dpkg.py: Refresh Contents.gz cache if it is older than one
+   day.
+ - crashdb.py: Ensure that address_signature duplicate db table does not have
+   multiple identical signatures by making it a primary key. Bump the db format
+   to "3". Existing databases need to be migrated manually as SQLite does not
+   allow adding a "PRIMARY KEY" constraint to existing tables.
+ - crashdb.py: Do not add a new address signature entry if one already exists.
+ - apport-cli: Fix UnicodeDecodeError on unicode report values. (LP: #275972)
+ - launchpad.py: Only set bug task importance if it is undecided.
+ - apport-retrace: Fix "an useful" typo. (LP: #911437)
+ - report.py: Filter out frames which are internal kernel/glibc implementation
+   details and not stable across duplicates. In particular, filter out
+   __kernel-syscall() and the SSE stubs.
+ - crashdb.py: Remove debugging leftover which completely disabled bug pattern
+   checking.
+ - report.py: Update reading AssertionMessage. Current (e)glibc turned
+   __abort_msg from a simple static string into a struct.
+
+Improvements:
+ - Change permissions of .crash files from 0600 to 0640, so that /var/crash can
+   be made g+s and crash handling daemons can access those.
+ - Python exceptions: Blacklist DBus.Error.NoReply. It does not help to get
+   these traces from the client-side application, you need the actual exception
+   in the D-Bus server backend instead. (LP: #914220)
+ - Support /etc/apport/whitelist.d/ similarly to /etc/apport/blacklist.d/, for
+   cases like installer environments where only crashes of a few selected
+   programs should be reported.
+
+1.90 (2011-11-24):
+------------------
+First beta release of 2.0 which introduces client-side duplicate checking.
+
+Bug fixes:
+ - backends/packaging-apt-dpkg.py: Fix another test case failure when ddeb
+   repository is not enabled.
+ - backends/packaging-apt-dpkg.py: Fix handling of explicit cache directory
+   name when it is a relative path.
+ - launchpad.py: Only query for bugs after 2011-08-01, to avoid timeouts.
+ - ui.py: Also anonymize standard bug title. (LP: #893863)
+ - launchpad.py: Current Launchpad cannot have private bugs which affect
+   multiple projects. Fix test suite accordingly.
+
+Improvements:
+ - report.py: Break out new method stacktrace_top_function() from
+   standard_title(), so that other parts of the code can use this as well.
+ - launchpad.net: When sending retraced results back to the bug report, update
+   the topmost function in the bug title. (LP: #869970)
+ - report.py, add_gdb_info(): Add a new field "StacktraceAddressSignature"
+   which is a heuristic signature for signal crashes. This should be used if
+   crash_signature() fails, i. e. the Stacktrace field does not have enough
+   symbols. This can be used to check for duplicates on the client side,
+   provided that the crash database server supports querying for these.
+   Do not expose this field when uploading to crash databases though, as it can
+   be recomputed from the already existing information (ProcMaps and
+   Stacktrace) and thus would just clutter the reports.
+ - crashdb.py: Add a table "version" with the database format version. Add
+   automatic upgrading to the most current format.
+ - crashdb.py: Put address signatures from reports checked with
+   check_duplicate() into the duplicate database, so that implementations of
+   known() can check for these.
+ - dupdb-admin: Add "publish" dupdb-admin command which exports the
+   duplicate database into a set of text files suitable for WWW publishing.
+ - crashdb.py: Add new method "known(report)" which can be implemented to check
+   if the crash db already knows about the crash signature. If so, the report
+   will not be uploaded, and instead the user will be directed to the existing
+   report URL (if available), similar to bug patterns. The default
+   implementation checks this format, if the crash database is initialized with
+   a "dupdb_url" option pointing to the exported database.
+ - launchpad.py: Override known() to check if the master bug is actually
+   accessible by the reporter, and is not tagged with "apport-failed-retrace"
+   or "apport-request-retrace"; otherwise file it anyway.
+ - crash-digger: Add --publish-db option to conveniently integrate duplicate DB
+   publication (similar to dupdb-admin publish) into retracer setups.
+ - launchpad.py: Attach updated stack traces from a duplicate to the master bug
+   if it failed retracing previously or has an "apport-request-retrace" tag.
+   (LP: #869982)
+ - apport-kde, apport-gtk: Support the "Annotation" field for custom dialog
+   titles for "Crash" and "Package" problem types as well, not just for
+   "Kernel". (LP: #664378)
+
+1.26 (2011-11-11):
+------------------
+Bug fixes:
+ - backends/packaging-apt-dpkg.py: Port to current python-apt API.
+ - hookutils.py: Fix path_to_key() to also work with unicode arguments.
+ - test/crash: Exit successfully if apport is not enabled in the system. This
+   allows packages to run the test suite during build.
+ - report.py, add_proc_info(): Correctly handle "python -m <modulename>"
+   programs as being interpreted and determine the appropriate module path.
+ - Fix some import statements to also work for the system-installed test suite.
+ - test/run: Fix testing data/general-hooks/parse_segv.py when called in
+   system-installed mode.
+ - apport/ui.py: Clean up test .crash file after test cases.
+ - Fix tests when running as root.
+ - setup.py: Fix crash when "javac -version" fails.
+ - README: Update command for one-time enablement.
+ - backends/packaging-apt-dpkg.py: Fix interleaving usage of install_packages()
+   with other operations such as get_version(), by resetting the apt status
+   after building and using the sandbox.
+ - report.py test suite: Remove requirement that $USER is set, which makes it
+   easier to run this from package build environments.
+ - apport/ui.py, test/crash: Use "yes" as test process instead of "cat". The
+   former is less likely to run already, and does not depend on having a stdin,
+   so it runs better in test environments like autopkgtest.
+ - backends/packaging-apt-dpkg.py: Fix tests if system does not have a dbgsym
+   apt source.
+
+Improvements:
+ - Ignore a crash if gnome-session is running and says that the session is
+   being shut down. These often die because X.org or other services are going
+   away, are usually harmless, and just cause a lot of clutter in bug trackers.
+   (LP: #460932)
+ - test/crash: Rewrite using Python's unittest, to be in line with other tests,
+   and be easier to maintain and extend.
+
+1.25 (2011-11-02):
+------------------
+Improvements:
+ - Add new response "Examine locally" to presenting the report details, which
+   runs apport-retrace in the chosen mode in a terminal. This should be made
+   available for crash reports if apport-retrace and a Terminal application are
+   installed; add an abstrace UI method for this. (LP: #75901)
+ - apport-gtk: Add "Examine locally..." button, and implement
+   ui_run_terminal().
+ - apport-cli: Add "Examine locally..." responses, and implement
+   ui_run_terminal().
+ - apport-cli: Greatly speed up displaying large reports. This also changes the
+   format to avoid indenting each line with a space, and visually set apart the
+   keys in a better way.
+ - apport_python_hook.py: Move tests out of this file into test/python, to
+   avoid having to parse the unit tests at each Python startup.
+ - test/python: Also make tests work if Python hook is not installed in
+   system's sitecustomize.py.
+ - packaging.py: Add get_modified_conffiles() API, and implement it in
+   packaging-apt-dpkg.py.
+ - hookutils.py: Add attach_conffiles().
+ - hookutils.py: Add attach_upstart_overrides().
+
+Bug fixes: 
+ - launchpad.py: Remove "Ubuntu" in bug response, replace with "this software".
+   (LP: #883234)
+ - apport-kde: Rearrange order of imports to get intended error message if
+   PyKDE is not installed.
+ - packaging-apt-dpkg.py: Ignore hardening-wrapper diversions, to make
+   gcc_ice_hook work if hardening-wrapper is installed.
+ - apport_python_hook: Respect $APPORT_REPORT_DIR.
+ - apport_python_hook: Limit successive crashes per program and user to 3 per
+   day, just like signal crashes. (LP: #603503)
+ - packaging-apt-dpkg.py: Skip online tests when there is no default route.
+ - ui.py: Fix test suite to not fail if system has some obsolete or non-distro
+   packages.
+
+1.24 (2011-10-19):
+------------------
+Bug fixes:
+ - backends/packaging-apt-dpkg.py, install_packages(): Also copy
+   apt/sources.list.d/ into sandbox.
+ - backends/packaging-apt-dpkg.py, install_packages(): Install apt keyrings
+   from config dir or from system into sandbox. (LP: #856216)
+ - packaging.py, backends/packaging-apt-dpkg.py: Define that install_packages()
+   should return a SystemError for broken configs/unreachable servers etc., and
+   fix the apt/dpkg implementation accordingly.
+ - apport-retrace: Don't crash, just give a proper error message if servers are
+   unreachable, or configuration files are broken. (LP: #859248)
+ - backends/packaging-apt-dpkg.py: Fix crash when /etc/apport/native-origins.d
+   contains any files. (LP: #865199)
+ - hookutils, recent_logfile(): Fix invalid return value if log file is not
+   readable. (LP: #819357)
+ - test/crash: Fix race condition in the "second instance terminates
+   immediately" check.
+ - hookutils.py: Replace attach_gconf() with a no-op stub. It used static
+   python modules like "gconf" which broke the PyGI GTK user interface, and
+   gconf is rather obsolete these days.
+ - ui.py, open_url(): Greatly simply and robustify by just using xdg-open. This
+   already does the right thing wrt. reading the default browser from GNOME,
+   KDE, XCE, and other desktops. (LP: #198449)
+ - data/general-hooks/generic.py: Only attach ~/.xsession_errors if the bug is
+   reported in the same XDG session as the crash happened. (LP: #869974)
+ - Ignore crashes for programs which got updated in between the crash and
+   reporting. (LP: #132904)
+ - Special-case crashes of 'twistd': Try to determine the client program and
+   assign the report to that, or fail with an UnreportableReason. (LP: #755025)
+ - apport-gtk: In bug update mode, make details dialog resizable and fix
+   default size. (LP: #865754)
+ - apport-gtk: Fix crash if report does not have ProcCmdline. (LP: #854452)
+ - hookutils.py, attach_wifi(): Anonymize ESSID and AP MAC from "iwconfig"
+   output. (LP: #746900)
+ - test/crash: Fix test failure if user is not in any system groups.
+ - test/crash: Change to /tmp/ for test crash process, to fix failure if the
+   user that runs the test suite cannot write into the current directory.
+   (LP: #868695)
+ - ui.py: Improve error message if package is not a genuine distro package.
+   Thanks to Ronan Jouchet. (LP: #559345)
+
+Improvements:
+ - apport-retrace: Add --timestamp option to prepend a timestamp to log
+   messages. This is useful for batch operations.
+ - crash-digger: Call apport-retrace with --timestamps, to get consistent
+   timestamps in log output.
+ - hookutils.py: Add two new functions attach_gsettings_package() and
+   attach_gsettings_schema() for adding user-modified gsettings keys to a
+   report. (LP: #836489)
+ - hookutils.py: Add new function in_session_of_problem() which returns whether
+   the given report happened in the currently running XDG session. This can be
+   used to determine if e. g. ~/.xsession-errors is relevant and should be
+   attached.
+
+1.23.1 (2011-09-29)
+-------------------
+Bug fixes:
+ - apport/crashdb.py: Ensure that duplicate table only has one entry per report
+   ID.
+ - apport-retrace: Pass correct executable path to gdb in --gdb with --sandbox
+   mode.
+ - apport-retrace: Do not leave behind temporary directories on errors.
+ - apport-retrace: Drop assertion failure for existance of "Stacktrace". This
+   isn't present in the case of gdb crashing, and there is not much we can do
+   about it. This should not break the retracer.
+ - apport/report.py: Unwind XError() from stack traces for the "StacktraceTop"
+   field, as they take a significant part of the trace. This causes bugs to be
+   duplicated which really have different causes.
+
+1.23 (2011-09-14)
+-----------------
+Improvements:
+ - crashdb.py, crash-digger, dupdb-admin: Drop the concept of "duplicate DB
+   consolidation". Such massive queries cause timeouts with e. g. Launchpad.
+   Instead, update the status of potential master bugs in the crash DB whenever
+   check_duplicate() is called.
+
+Bug fixes:
+ - launchpad.py: Fix crash in close_duplicate() if master bug was already
+   marked as a duplicate of the examined bug.
+ - problem_report.py, load(): Fix missing last character if the last line in a
+   multi-line field is not terminated with a newline.
+ - launchpad.py: Fix test_marking_python_task_mangle() check to work with
+   current Launchpad.
+ - apport-retrace: If the user did not specify a --cache directory, create a
+   shared one instead of letting the two install_packages() calls create their
+   own. This ensures that the apt and dpkg status is up to date, and avoids
+   downloading the package indexes multiple times. (LP: #847951)
+ - apport-retrace: Give proper error mesage instead of AssertionError crash if
+   a report does not contain standard Apport format data. (LP: #843221)
+ - fileutils.py, get_new_reports(): Fix crash if report file disappears in the
+   middle of the operation. (LP: #640216)
+ - apport/ui.py, load_report(): Intercept another case of broken report files.
+   (LP: #445142)
+ - apport/report.py, standard_title(): Escape regular expression control
+   characters in custom exception names. (LP: #762998)
+
+1.22.1 (2011-09-06)
+-------------------
+Improvements:
+ - dupdb-admin: Add "removeid" command.
+
+Bug fixes:
+ - dupdb-admin: Use the in-memory CrashDB implementation for simple operations
+   like dump or changeid, which do not require an actual backend. This makes
+   the command work in checkouts without a /etc/apport/crashdb.conf.
+ - dupdb-admin: Fix UnicodeEncodeError crash.
+ - launchpad.py: Fix crash if a crash report does not have a DistroRelease.
+ - Set the default "Apport" title for choice dialogs instead of the default
+   apport-gtk title. Thanks Robert Roth. (LP: #608222)
+ - apport-gtk: Update markup_escape_text() call to current glib. (LP: #829635)
+
+1.22 (2011-08-25)
+-----------------
+Improvements:
+ - Completely rework apport-retrace to use gdb's "debug-file-directory" and
+   "solib-absolute-prefix" settings and only unpack the necessary packages in a
+   temporary directory. This makes it possible to use it in a running system
+   without actually touching installed packages, does not need any root
+   privileges, and stops the requirement of using chroots with fakechroot and
+   fakeroot. This is a lot easier to maintain and use, and a lot faster, too.
+   As a consequence, drop the chroot module, and update crash-digger
+   accordingly. See "man apport-retrace" for the new usage.
+   It is now also easier to port to other packaging backends, as a lot of the
+   common logic moved out of the packaging API;
+   packaging.install_retracing_packages() got dropped in favor of the simpler
+   packaging.install_packages().
+ - crash-digger: Show how many bugs are left in the pool with each new retrace.
+
+Bug fixes:
+ - apport-gtk: Fix crash in GLib.markup_escape_text() call, regression from
+   1.21.3. (LP: #828010)
+ - launchpad.py: When searchTasks() times out, exit with 99 as this is a
+   transient error.
+ - crash-digger: Intercept OverflowError from downloaded compressed
+   attachments.
+
+1.21.3 (2011-08-17)
+-------------------
+Bug fixes:
+ - gtk/apport-gtk.desktop.in: Also show in Unity. (LP: #803519)
+ - apport-unpack: Fix crash on file errors.
+ - Add apport.packaging.get_library_paths() interface and implement it for
+   backends/packaging-apt-dpkg.py using dpkg multiarch directories. Use it in
+   chroot.py.
+ - hookutils.py: Don't attach empty values. Thanks Bryce Harrington.
+   (LP: #813798)
+ - apport-gtk: Correctly pass message dialog type.
+ - apport-gtk: Fix GLib and GObject imports to be compatible with the future
+   pygobject 3.0.
+
+Improvements:
+ - hookutils.py: Add attach_mac_events() for reporting logs of MAC systems.
+   Looks for AppArmor messages for now. Thanks Marc Deslauriers!
+ - hookutils.py, attach_alsa(): Get a list of outputs/inputs that PulseAudio
+   knows about, which also shows the currently selected output/input, as well
+   as volumes. This should help with "no sound" bug troubleshooting. Thanks
+   Luke Yelavich.
+
+1.21.2 (2011-07-01)
+-------------------
+Improvements:
+ - test/run: Check $PYTHON for using a different Python interpreter (such as
+   "python3") for the tests.
+ - generic hook: Don't report package installation failures due to segfaulting
+   maintainer scripts. We want the actual crash report only. Thanks Brian
+   Murray.
+ - hookutils.py, attach_wifi(): Also include wpasupplicant logs. Thanks Mathieu
+   Trudel-Lapierre!
+
+Bug fixes:
+ - backends/packaging-apt-dpkg.py: Fix crash introduced in 1.21.1's multiarch
+   fixes.
+ - report.py: Fix bug patterns to correctly match against compressed report
+   fields.
+
+1.21.1 (2011-06-20)
+-------------------
+Improvements:
+ - data/general-hooks/generic.py: Also check for low space on /var. Thanks
+   Brian Murray.
+ - hookutils.py, attach_file() and attach_file_if_exists(): Add a new
+   "overwrite" flag option. If not given, now default to overwriting an
+   existing key, as this is usually what you need when attaching files
+   (instead of attaching it several times with '_' appended to the keys). You
+   can get the old behaviour by setting overwrite=False.
+
+Bug fixes:
+ - When showing the size of the full report, take the compressed size of binary
+   values instead of their uncompressed size, as the crash db upload will use
+   the compressed values.
+ - backends/packaging-apt-dpkg.py: Fix for current dpkg with multiarch support.
+ - test/run: Fix the test suite to run against the system installed libraries
+   with current Python versions (2.6, 2.7) where __file__ does not work any
+   more with imports.
+
+1.21 (2011-06-08)
+-----------------
+Improvements:
+ - Supply --desktop option to kdesudo to improve the description which program
+   is requesting administrative privileges.
+ - apport-checkreports: Exit with status 2 if there are new reports, but apport
+   is disabled. This helps crash notification GUIs to not display new crash
+   reports in that case. Thanks to Michael Vogt for the original patch.
+ - Add data/is-enabled: Shell script to check if apport is enabled. Non-Python
+   programs (which can't use apport.packaging.enabled() ) can call this instead
+   of having to parse /etc/default/apport themselves, and just check the exit
+   code. Inspired by original patch from Michael Vogt, thanks!
+
+Bug fixes:
+ - apport-gtk: HTML-escape text for dialogs with URLs. (LP: #750870)
+ - dump_acpi_tables.py: Check to see if acpi/tables dir is mounted first.
+   Thanks Brian Murray. (LP: #729622)
+ - man/apport-cli.1: Document recently added -w/--window option. Thanks Abhinav
+   Upadhyay! (LP: #765600)
+ - Use kde-open instead of kfmclient to open URLs under KDE. Thanks Philip
+   Muškovac. (LP: #765808)
+
+1.20.1 (2011-03-31)
+-------------------
+Bug fixes:
+ - Add bash completion support for new -w/--window option that was introduced
+   in 1.20. Thanks Philip Muškovac.
+ - apport-unpack: Fix crash if target directory already exists.
+ - Fix crash if UnreportableReason is a non-ASCII string. (LP: #738632)
+ - Fix crash if application from desktop name is a non-ASCII string.
+   (LP: #737799)
+ - unkillable_shutdown: Fix rare crash if ExecutablePath does not exist (any
+   more). (LP: #537904)
+ - kernel_crashdump: Fix crash if the vmcore file disappeared underneath us.
+   (LP: #450295)
+ - unkillable_shutdown: Fix crash if the checked process terminated underneath
+   us. (LP: #540436)
+ - ui.py: Properly raise exceptions from the upload thread that happen at its
+   very end. (LP: #469943)
+
+1.20 (2011-03-17)
+-----------------
+Improvements:
+ - Add support for -w/--window option which will enable user to select a
+   window as a target for filing a problem report. Thanks Abhinav Upadhyay for
+   the patch! (LP: #357847)
+ - Disable the filtering on SIGABRT without assertion messages. Turns out that
+   developers want these crash reports after all. (LP: #729223)
+ - Add support for a "DuplicateSignature" report fields. This allows package
+   hooks to implement custom duplicate problem handling which doesn't need to
+   be hardcoded in Apport itself. Update the launchpad backend to tag such bugs
+   as "need-duplicate-check".
+
+Bug fixes:
+ - report.py, add_hooks_info(): Properly report TypeErrors from hooks.
+ - apport-retrace: Intercept SystemErrors from ill-formed gzip attachments as
+   well.
+ - Fix crash if crash database configuration does not specify a
+   bug_pattern_url. Just assume None. (LP: #731526)
+ - If a custom crash database does not specify a bug_pattern_url, fall back to
+   using the default database's. (LP: #731526)
+ - hookutils.py Update WifiSyslog regex to correctly catch application log
+   messages in syslog. Thanks Mathieu Trudel-Lapierre. (LP: #732917)
+ - hookutils.py, attach_hardware(): Avoid error message if machine does not
+   have a PCI bus. Thanks Marcin Juszkiewicz! (LP: #608449)
+ - backends/packaging-apt-dpkg.py: Replace deprecated getChanges() call with
+   get_changes().
+ - apport-gtk: Fix broken dialog heading if the name of the crashed program
+   contains an & or other markup specific characters.
+ - apport-gtk: Don't crash if GTK cannot be initialized. This usually happens
+   without a $DISPLAY or when the session is being shut down. Just print an
+   error message. If there are pending crashes, they will be shown again the
+   next time a session starts. (LP: #730569)
+
+1.19 (2011-02-28)
+-----------------
+Bug fixes:
+ - Update stack unwind patterns for current glib (slightly changed function
+   names), and also ignore a preceding '*'. (LP: #716251)
+ - Fix crash_signature() to fail if there is an empty or too short
+   StacktraceTop.
+ - apt backend: Do not generate a warning if the opportunistically added -dbg
+   package does not exist.
+ - apt backend: Only add -dbg in --no-pkg mode, as there will be conflicts in
+   normal package mode.
+ - apt backend: Call tar with target cwd instead of using -C; the latter causes
+   an extra openat() call which breaks with current fakechroot.
+ - launchpad.py: Fix retracer crash if DistroRelease field does not exist.
+ - Convert deprecated failIf()/assert_() TestCase method calls to
+   assertFalse()/assertTrue().
+
+Improvements:
+ - In apport-bug, if the user specifies a PID referring to a kernel thread,
+   do the right thing and file the bug against the kernel
+ - In hookutils.attach_dmesg, skip over an initial truncated message if one
+   is present (this happens when the ring buffer overflows)
+ - Change bug patterns to just use one central file instead of per-package
+   files. This allows bug patterns to be written which are not package
+   specific, and is easier to maintain as well. IMPORTANT: This changed the
+   format of crashdb.conf: bug_pattern_base is now obsolete, and the new
+   attribute bug_pattern_url now points to the full URL/path of the patterns
+   file. Thanks to Matt Zimmerman!
+
+1.18 (2011-02-16)
+-----------------
+Bug fixes:
+ - Ensure that symptom scripts define a run() function, and don't show them if
+   not.
+ - Do not show symptom scripts which start with an underscore. These can be
+   used for private libraries for the actual symptom scripts.
+ - Update bash completion. Thanks Philip Muškovac.
+ - etc/default/apport: Remove obsolete "maxsize" setting. (LP: #719564)
+
+Improvements:
+ - Remove explicit handling of KDE *.ui files in setup.py, as
+   python-distutils-extra 2.24 fixes this. Bump version check.
+ - hookutils.py: Add attach_root_command_outputs() to run several commands
+   at once. This avoids asking for the password several times. (LP: #716595)
+
+1.17.2 (2011-02-04)
+-------------------
+Improvements:
+ - Be more Python 3 compatible (not fully working with Python 3 yet, though).
+ - apt/dpkg backend: Drop support for pre-0.7.9 python-apt API.
+ - Add --tag option to add extra tags to reports. (LP: #572504)
+
+Bug fixes:
+ - hookutils.py, attach_dmesg(): Do not overwrite already existing dmesg.
+ - hookutils.py: Be more robust against file permission errors. (LP: #444678)
+ - ui.py: Do not show all the options in --help when invoked as *-bug.
+   (LP: #665953)
+ - launchpad.py: Adapt test cases to current standard_title() behaviour.
+
+1.17.1 (2011-01-10)
+-------------------
+Bug fixes:
+ - Make the GTK frontend work with GTK 2.0 as well, and drop "3.0" requirement.
+
+1.17 (2010-12-31)
+-----------------
+Improvements:
+ - Better standard bug titles for Python crashes. Thanks Matt Zimmerman!
+   (LP: #681574)
+ - Add handler for uncaught Java exceptions. There is no integration for
+   automatically intercepting all Java crashes yet, see java/README.
+   Thanks Matt Zimmerman! (LP: #548877)
+
+Bug fixes:
+ - GTK frontend: Require GTK 3.0.
+ - launchpad.py: Default to "production" instance, not "edge", since edge is
+   obsolete now.
+ - hookutils.py, attach_alsa(): Fix crash if /proc/asound/cards does not exist.
+   (LP: #626215)
+ - ui.py, format_filesize(): Fix to work with stricter locale.format() in
+   Python 2.7. (LP: #688535). While we are at it, also change it to use base-10
+   units.
+ - hookutils.py, package_versions(): Always include all requested package names
+   even if they're unknown to us. Thanks Matt Zimmerman! (LP: #695188)
+ - launchpad.py: When updating a bug, also add new tags. Thanks Brian Murray!
+
+1.16 (2010-11-19)
+-----------------
+New features:
+ - Port GTK frontend from pygtk2 to GTK+3.0 and gobject-introspection.
+
+Bug fixes:
+ - Fix symptoms again. Version 1.15 broke the default symptom directory.
+ - Fix memory test case to work with current Python versions, where the SQLite
+   integrity check throws a different exception.
+
+1.15 (2010-11-11)
+-----------------
+New features:
+ - Add dump_acpi_tables.py script. This can be called by package hooks which
+   need ACPI table information (in particular, kernel bug reports). Thanks to
+   Brad Figg for the script!
+ - Order symptom descriptions alphabetically. Thanks to Javier Collado.
+ - Check $APPORT_SYMPTOMS_DIR environment variable for overriding the system
+   default path. Thanks to Javier Collado.
+
+Bug fixes:
+ - testsuite: Check that crashdb.conf can have dynamic code to determine DB
+   names and options.
+ - ui.py test suite: Rewrite _gen_test_crash() to have the test process core
+   dump itself, instead of using gdb to do it. The latter fails in ptrace
+   restricted environments, such as Ubuntu 10.10.
+ - packaging-apt-dpkg.py: Fix handling of /etc/apport/native-origins.d to
+   actually work. Thanks Steve Langasek. (LP: #627777)
+ - apport-kde: Load correct translation catalogue. Thanks Jonathan Riddell.
+   (LP: #633483)
+ - launchpad.py: Use launchpadlib to file a bug instead of screen scraping.
+   The latter was completely broken with current Launchpad, so this makes the
+   test suite actually work again. Thanks to Diogo Matsubara!
+ - launchpad.py: Change $APPORT_STAGING to $APPORT_LAUNCHPAD_INSTANCE, so that
+   you can now specify "staging", "edge", or "dev" (for a local
+   http://launchpad.dev installation). Thanks to Diogo Matsubara!
+ - backends/packaging-apt-dpkg.py: Fix crash on empty lines in ProcMaps
+   attachment.
+ - doc/symptoms.txt: Fix typo, thanks Philip Muskovac. (LP: #590521)
+ - apport/hookutils.py: rename ProcCmdLine to ProcKernelCmdLine to not wipe
+   wipe out /proc/$pid/cmdline information. (LP: #657091)
+ - apport/hookutils.py: attach_file() will not overwrite existing report
+   keys, instead appending "_" until the key is unique.
+ - Fix --save option to recognise ~, thanks Philip Muškovac. (LP: #657278)
+ - Remove escalation_subscription from Ubuntu bug DB definition, turned out to
+   not be useful; thanks Brian Murray.
+ - launchpad.py: Fix APPORT_LAUNCHPAD_INSTANCE values with a https:// prefix.
+ - apt backend: Opportunistically try to install a -dbg package in addition to
+   -dbgsym, to increase the chance that at least one of it exists. Thanks
+   Daniel J Blueman!
+
+1.14.1 (2010-06-24)
+-------------------
+Bug fixes:
+ - hookutils.py, attach_drm_info(): Sanitize connector names. Thanks Chris
+   Halse Rogers! (LP: #597558)
+ - bash completion: Complete all path names, apport-bug can be invoked with a
+   path to a program. Thanks Philip Muskovac.
+
+1.14 (2010-06-16)
+-----------------
+New features:
+ - hookutils.py: Add new method attach_drm_info() to read and format
+   /sys/class/drm/*.
+
+Bug fixes:
+ - packaging-apt-dpkg.py: Fix deprecated python-apt variables, thanks David
+   Stansby. (LP: #591695)
+ - launchpad.py: Fix crash on attachments which are named *.gz, but
+   uncompressed. (LP: #574360)
+ - hookutils.py, attach_gconf(): Fix defaults parsing for boolean keys.
+   (LP: #583109)
+
+1.13.4 (2010-05-04)
+-------------------
+ - bash completion: Fix error message if /usr/share/apport/symptoms does not
+   exist. Thanks Philip Muškovac! (LP: #562118)
+ - general-hooks/parse_segv.py: Report stack exhaustion more clearly and
+   correctly handle register dereferencing calls.
+ - Save/restore environment when calling hooks, in case they change the locale,
+   etc. (LP: #564422)
+ - hookutils.py, command_output(): Do not set $LC_MESSAGES for the calling
+   process/hook, just for the command to be called.
+ - ui.py: When displaying strings from system exceptions, decode them into an
+   unicode string, to avoid crashing the KDE UI. (LP: #567253)
+ - apport-retrace: Fix crash for retracing kernel vmcores, which do not have an
+   ExecutablePath.
+ - apport-bug manpage: Clarify when apport-collect may be used. Thanks Brian
+   Murray! (LP: #537273)
+ - generic hook: Check ProcMaps for unpackaged libraries, and ask the user if
+   he really wants to continue. If he does, tag the report as "local-libs" and
+   add a "LocalLibraries" field to the report with a list of them.
+   (LP: #545227)
+
+1.13.3 (2010-04-14)
+-------------------
+ - data/general-hooks/parse_segv.py: suggest segv-in-kernel possibility.
+ - ui.py: When running as root, only show system crash reports, to avoid
+   restarting user programs as root. (LP: #445017)
+
+1.13.2 (2010-03-31)
+-------------------
+ - problem_report.py, write_mime(): Add new optional argument "priority_fields"
+   for ordering report keys. Patch by Brian Murray, thanks!
+ - launchpad.py: Put some interesting fields first in the report, with the new
+   priority_fields argument. Patch by Brian Murray, thanks!
+ - packaging-apt-dpkg.py, _install_debug_kernel(): Do not crash on an outdated
+   kernel, just return that it is outdated. (LP: #532923)
+ - launchpad.py test suite: Add "Referer" HTTP header, now required by
+   launchpad.
+ - launchpad.py: Fix crash if configuration does not have an "escalated_tag"
+   option.
+ - launchpad.py: Port to launchpadlib 1.0 API, thanks Michael Bienia for the
+   initial patch! (LP: #545009)
+ - gtk/apport-gtk-mime.desktop.in, kde/apport-kde-mime.desktop.in: Change
+   categories so that these do not ever appear in menu editors. (LP: #449215)
+ - launchpad.py: Some LP bugs have broken attachments (this is a bug in
+   Launchpad itself). Ignore those instead of crashing.
+ - apport-gtk: Turn http:// and https:// links into clickable hyperlinks in
+   information and error dialogs. (LP: #516323)
+ - apport-retrace: Fix crash when trying to rebuild package info for reports
+   without an ExecutablePath. (LP: #436157)
+ - ui.py: Fix crash when package information cannot be determined due to broken
+   apt status. (LP: #362743)
+ - ui.py: Fix crash when /etc/apport/crashdb.conf is damaged; print an
+   appropriate error message instead. (LP: #528327)
+ - data/kernel_crashdump: Fix crash if log file disappeared underneath us.
+   (LP: #510327)
+ - data/apport: Fix IOError when apport is called with invalid number of
+   arguments, and stderr is not a valid fd. (LP: #467363)
+ - hookutils.py: Factor out the DMI collection code from attach_hardware()
+   into attach_dmi(), and call that in attach_alsa() as well. Thanks to Brad
+   Figg for the patch! (LP: #552091)
+ - apport/ui.py: Fix the help output if Apport is invoked under an alternative
+   name (like apport-collect). (LP: #539427)
+
+1.13.1 (2010-03-20)
+-------------------
+Bug fixes:
+ - Update parse-segv to handle gdb 7.1 output.
+ - Enhance test suite to work with gdb 7.1 as well, and catch future outputs.
+ - UI: Add exception string to the "network error" dialog, to better tell what
+   the problem is.
+ - UI: Add back -p option to apport-collect/apport-update-bug (regression from
+   1.13). (LP: #538944)
+ - launchpad.py: Add yet another workaround for LP#336866. (LP: #516381)
+ - launchpad.py, download(): Ignore attachments with invalid key names.
+ - Fix regression from 1.10 which made it impossible for a package hook to set
+   a third-party crash database for non-native packages. (LP: #517272)
+ - apport-cli: Create the 'details' string only if user wants to view details,
+   and do not show files larger than 1MB. Thanks Scott Moser! (LP: #486122)
+ - packaging-apt-dpkg.py: Silence apt.Cache() spewage to stdout with newer
+   python-apt versions. (LP: #531518)
+
+Improvements:
+ - unkillable_shutdown: Add list of running processes and blacklisted pids to
+   report. (LP: #537262)
+ - Sort the report by key in the details view. (LP: #519416)
+
+1.13 (2010-03-10)
+-----------------
+New features:
+ - Add "unkillable_shutdown" script to collect information about processes
+   which are still running after sending SIGTERM to them. This can be hooked
+   into e. g. /etc/init.d/sendsigs on Debian/Ubuntu systems.
+
+Improvements:
+ - apport_python_hook.py: Directly check /etc/default/apport instead of
+   querying packaging.enabled(), to avoid importing lots of modules for
+   non-packaged scripts. Thanks Stuart Colville! (LP: #528355)
+
+Bug fixes:
+ - Fix SegV parser to notice walking off the stack during "call" or "ret"
+   (LP: #531672).
+ - Fix --help output for bug updating mode (invocation as apport-collect or
+   apport-update-bug). (LP: #504116)
+ - Fix bug escalation tagging, thanks to Brian Murray.
+ - Fix option processing when being invoked as apport-bug. Thanks to Daniel
+   Hahler for the patch! (LP: #532944)
+
+1.12.1 (2010-02-22)
+-------------------
+Bug fixes:
+ - launchpad.py: Do not keep escalating bugs, just escalate at the 10th
+   duplicate.
+ - Improve error message if a symptom script did not determine a package name.
+   (LP: #503834)
+ - general-hooks/generic.py: Fix crash on libGL check with empty StacktraceTop.
+ - Review and clean up usage of chmod(). This fixes a small race condition in the
+   Python exception hook where a local attacker could read the information from
+   another user's crash report. (LP: #516029)
+ - hookutils, package_versions(): Ignore "None" packages, for more robust
+   package hooks. (LP: #518295)
+
+1.12 (2010-01-20)
+-----------------
+Improvements:
+ - launchpad.py: Add options 'escalation_subscription' and 'escalation_tag' for
+   handling bugs with more than 10 duplicates.
+ - crashdb.conf: For Ubuntu, escalate bugs with >= 10 duplicates to
+   "ubuntu-bugcontrol" and tag them with "bugpattern-needed". (LP: #487900)
+ - general-hooks/generic.py: Filter out crashes on missing GLX (LP: #327673)
+ - Add bash completion script. Thanks to Philip Muškovac. (LP: #218933)
+
+Bug fixes:
+ - launchpad.py: Drop APPORT_FILES whitelist for download() and instead just
+   filter out file extensions that we know about (*.txt and *.gz).
+   (LP: #444975)
+ - launchpad.py: Do not put the Tags: field into the bug description, since
+   they are already proper tags. In download(), convert the real tags back to
+   the Tags: field. (LP: #505671)
+ - test/crash: Update expected core dump flags for changed rlimit behaviour in
+   Linux 2.6.32.
+ - launchpad.py: Fix marking of 'checked for duplicate' for bugs with upstream
+   tasks.
+ - launchpad.py, get_fixed_version(): Do not consider a bug as invalid just
+   because it has any invalid distro package task.
+
+1.11 (2009-12-23)
+-----------------
+Improvements:
+ - Add "--save" UI option to store the collected information into an .apport
+   file instead of sending it right away. The file can then later be sent
+   through apport-bug. Update manpages accordingly.
+ - Update all copyright and description headers and consistently format them.
+ - Rename all TestCase classes to "_T", which makes it much easier to run
+   individual tests from the command line.
+ - Testsuite: Verify that report details are/are not shown. This uncovered that
+   details about package installation failures were not shown before sending
+   them, which is fixed now.
+
+Bug fixes:
+ - test/hooks: Do not try to add hook information to kernel_crashdump test
+   case, since we do not have an UI here. This test case broke when the system
+   had an interactive package hook for the kernel.
+ - When reporting a bug from a saved .apport file, let the user review/confirm
+   the content before sending.
+
+1.10.1 (2009-12-23)
+-------------------
+Improvements:
+ - Install apport-collect symlink.
+ - Update translations from Launchpad.
+
+Bug fixes:
+ - Move all remaining option/argument parsing from apport-bug into ui.py. This
+   allows the user to add options to apport-bug/apport-collect, and also avoids
+   unwieldy dissection of options/arguments in shell.
+
+1.10 (2009-12-19)
+-----------------
+New features:
+ - Add a mode for updating an existing problem report to ui.py (-u/--update).
+   This is similar to the Ubuntu specific "apport-collect" tool, but
+   implemented the right way now: In particular, this has access to the UI and
+   thus can use interactive hooks (LP: #385811) and show you what is being sent
+   for confirmation/cancelling (LP: #371827)
+ - apport-bug: If invoked as "apport-collect" or "apport-update-bug" (i. e.
+   through a symlink), run apport in update mode (-u <number>). This provides a
+   convenient no-options command line program. Please note that setup.py does
+   not currently install such a symlink. Update the apport-bug manpage
+   accordingly.
+
+Improvements:
+ - launchpad.py: Use new login_with() to clean up code, and specify allowed
+   access levels (WRITE_PRIVATE is the only sensible one anyway). (LP: #410205)
+ - New hookutils functions:
+   xsession_errors (match lines from ~/.xsession-errors)
+   shared_libraries (determine which libraries a binary links with)
+   links_with_shared_library (test if a binary links with a particular library)
+ - New CrashDatabase API: get_affected_packages(), can_update(), is_reporter()
+ - Rename CrashDatabase.update() to update_traces().
+ - Add CrashDatabase.update() for adding all new fields of a report. This is
+   primarily useful for collecting local standard and package hook data for an
+   already existing bug report which was not filed through Apport. This checks
+   can_update()/is_reporter() if the user is eligible for updating that
+   particular bug. (LP: #485880)
+
+Bug fixes:
+ - Ignore SIGXCPU and SIGXFSZ; thanks to Kees Cook. (LP: #498074)
+ - launchpad.py: Do not mark non-Ubuntu bugs as needs-retrace, since there is
+   no retracer right now. (LP: #489794)
+ - packaging-apt-dpkg.py, install_retracing_packages(): Do not crash on
+   malformed Dependencies.txt lines. (LP: #441709)
+ - use-local: Fix for new source tree location of "apport" binary.
+
+1.9.6 (2009-12-01)
+------------------
+Improvements:
+ - Add pm-utils hook to record current operation, so that apportcheckresume can
+   check it. Before this was kept in Ubuntu's pm-utils package.
+ - general-hooks/generic.py: Check if using ecryptfs, and which directory.
+   (LP: #444656)
+
+Bug fixes:
+ - launchpad.py: Ensure that text attachments on initial bug filing are valid
+   UTF-8. (LP: #453203)
+ - man/apport-retrace.1: Document -R option.
+
+1.9.5 (2009-11-20)
+------------------
+Bug fixes:
+ - apport-retrace: Fix crash if InterpreterPath/ExecutablePath do not exist.
+ - hookutils.py, attach_alsa(): Attach /proc/cpuinfo too, for CPU flags.
+ - Fix crash if InterpreterPath does not exist any more at the time of
+   reporting. (LP: #428289)
+ - apport-gtk: Connect signals properly, to repair cancel/window close buttons.
+   (LP: #427814)
+ - Update German translations and fix "konnre" typo. (LP: #484119)
+
+1.9.4 (2009-11-06)
+------------------
+Bug fixes:
+ - Fix crash when ExecutablePath isn't part of a package. (LP: #424965)
+ - hookutils.py, attach_hardware(): Anonymize disk labels. Thanks to Marco
+   Rodrigues. (LP: #394411)
+ - hookutils.py, attach_wifi(): Anonymize encryption key (which appeared in hex
+   when being called as root). Thanks to Marco Rodrigues. (LP: #446299)
+ - launchpad.py: If unset, set bug task source package also for interpreter
+   crashes.
+ - apport-gtk: Give details window a minimize/maximize button, which were
+   missing in some window managers. Thanks to Marien Zwart. (LP: #447749)
+ - apport-kde: Properly terminate program after closing the last dialog.
+   (LP: #458662)
+ - hookutils.py, attach_alsa(): Attach /proc/asound/version. (LP: #467233)
+ - general-hooks/generic.py: Only collect ~/.xsession-errors bits when we have
+   an ExecutablePath linked to libgtk.
+
+1.9.3 (2009-10-14)
+------------------
+Changes:
+ - Drop handling of the APPORT_REPORT_THIRDPARTY environment variable and
+   "thirdparty" configuration file option. This has never been documented, and
+   conceptually does not work. There is a proper mechanism for this in place
+   now, e. g. launchpad.py's "project" option.
+
+Bug fixes:
+ - hookutils.py: Fix error codes from "comm", thanks to Brian Murray.
+ - general-hooks/generic.py: Catch xkbcomp error messages. (LP: #431807)
+ - launchpad.py: Assert that we have exactly one of "distro" or "project"
+   option.
+ - doc/crashdb-conf.txt: Improve documentation of crash database options.
+ - apport-gtk: Make Cancel/Send buttons focusable. Thanks to Marco Rodrigues.
+   (LP: #447780)
+
+1.9.2 (2009-10-02)
+------------------
+Improvements:
+ - apport-cli: Print the URL and ask whether to open a browser. In many
+   situations (such as usage on a server through ssh), it's preferable to not
+   open the browser on the reporting computer. Thanks to Matt Zimmerman for the
+   initial patch! (LP: #286415)
+ - general-hooks/generic.py: Collect important glib errors/assertions (which
+   should not have private data) from ~/.xsession-errors (LP: #431807)
+ - launchpad.py: Link hardware data submission key if it exists. (LP: #424382)
+
+Bug fixes:
+ - apport-cli: Fix crash with non-ASCII characters in prompts.
+ - Fix "apport-bug symptomname" to actually work.
+ - launchpad.py: Fix crash on invalid credentials file. Thanks to Marco
+   Rodrigues for the initial patch! (LP: #414055)
+
+1.9.1 (2009-09-22)
+------------------
+Bug fixes:
+ - hookutils.py, attach_hardware(): Do not attach empty Pccardctl*.
+ - apport/report.py, add_gdb_info(): Do not throw away stderr from gdb.
+ - data/general-hooks/parse_segv.py:
+   + Handle arithmetic wrapping correctly.
+   + Handle empty base, scale, or index registers in disassembly.
+   + Handle in/out ioport faults.
+ - Various improvements to user-visible strings, thanks to Marco Rodrigues!
+   (LP: #178507)
+ - Various apport-retrace robustifications.
+ - setup.py: Fix DistUtilsExtra version check. (LP: #428337)
+ - hookutils.py, attach_gconf(): Do not overwrite previous values from other
+   packages, thanks Loïc Minier!
+ - hookutils.py, attach_gconf(): Fix crash with nonexisting <applyto> tags.
+
+1.9 (2009-09-08)
+----------------
+New features:
+ - Add "do what I mean" mode to command line argument parsing (applies to all
+   interfaces: -cli, -gtk, -kde). When giving a single argument and no options,
+   determine the most likely mode, like reporting a bug against a symptom,
+   package, executable name, or PID.
+ - Add program "apport-bug" which determines the most appropriate user
+   interface (GTK, KDE, CLI) and files a bug through it, using the single
+   argument "do what I mean" mode. This is an improved version of Ubuntu's
+   "ubuntu-bug" script.
+
+Bug fixes:
+ - Update apport-cli manpage to current set of options and behaviour. Also
+   point out that apport-gtk and apport-kde share the same CLI.
+ - setup.py now installs apport-{gtk,kde} into $prefix/share/apport/, they are
+   not supposed to be called directly. This also reflects the path which the
+   .desktop files expect.
+ - setup.py now installs the internal helper scripts like "kernel_crashdump",
+   "apport", or "apportcheckresume" into $prefix/share/apport instead of
+   $prefix/bin.
+ - Update usage of gettext to work around Python bug of gettext() not returning
+   unicodes, but str. Fixes UnicodeDecodeErrors on translated --help output.
+
+1.8.2 (2009-09-05)
+------------------
+Bug fixes.
+
+1.8.1 (2009-09-03)
+------------------
+Lots of bug fixes.
+
+1.8 (2009-08-26)
+----------------
+New features:
+ - Do not generally ignore SIGABRT any more. Try to extract the assertion
+   message from the core dump, and add it as "AssertionMessage" field. Mark
+   reports as unreportable if they do not have an assertion message and crashed
+   with SIGABRT. This requires your glibc to have this patch:
+   http://sourceware.org/git/?p=glibc.git;a=commitdiff;h=48dcd0ba
+ - report.py, add_hooks_info(): Add optional package/srcpackage argument. Hooks
+   can use that to change the affected package or call hooks from different
+   packages.
+ - KDE frontend implementation of ui_question_userpass(), for crash databases
+   which need to ask for credentials.
+ - hookutils.py: New funtion attach_wifi() to add wireless network related
+   information to reports.
+
+Important bug fixes:
+ - Fix the test suite on current kernels; test/crash previously often failed
+   with python segfaults, since it killed the test processes too early.
+
+1.7 (2009-08-05):
+-----------------
+New features:
+ - Support for "symptom" scripts, which figure out the package for a bug report
+   based on interactive questions.
+
+1.6 (2009-07-15)
+----------------
+New features:
+ - Integrate analysis and retracing of kernel vmcore crashes with the "crash"
+   tool. Courtesy of Michael Vogt.
+
+Various little bug fixes.
+
+1.5 (2009-06-29)
+----------------
+New features:
+ - Drop all Makefiles, po/POTFILES.in, and most code from setup.py, and use
+   DistUtilsExtras.auto which "just does the right thing" for most build system
+   tasks. This requires python-distutils-extra >= 2.2, see
+   https://launchpad.net/python-distutils-extra
+
+Cleanup:
+ - Move all test scripts into test/, to unclutter source tree.
+ - setup.py now auto-detects the required packaging backend if
+   apport/packaging_impl.py is not manually installed.
+
+1.4 (2009-06-26)
+----------------
+New features:
+ - Replaced Qt4 frontend with a KDE frontend for better KDE integration.
+
+Major bug fixes:
+ - packaging-apt-dpkg.py: Add backwards compatibility code for python-apt <
+   0.7.9 to not break backportability.
+
+1.3 (2009-06-10)
+----------------
+New features:
+- Interactive package hooks:
+  * Add apport.ui.HookUI class which provides GUI functionality such as yes/no
+    questions or file dialogs to hooks.
+  * add_info() in package hooks now can (optionally) take a second argument which
+    is the HookUI instance.
+  * See doc/package-hooks.txt for details.
+- New function apport.hookutils.root_command_output() to run a command as root,
+  through gksu/kdesudo/sudo, depending on the desktop environment.
+- Add general hook for analyzing reason of a segfault.
+
+Bug fixes:
+- Drop "UnsupportableReason" field, it is too similar to UnreportableReason and
+  just confusing.
+- Report key names can now contain dashes ('-') and underscores ('_').
+  (LP #380811)
+
+1.2.1 (2009-05-15)
+------------------
+Bug fixes:
+- Fix setup.py and PO file merging for recent .glade -> .ui renaming.
+
+Translations:
+- Update German translations.
+
+1.2.0 (2009-05-15)
+------------------
+Moving away from deprecated APIs:
+- packaging-apt-dpkg.py: Use python-apt >= 0.7.9 official API and drop usage of
+  internal symbols.
+- hookutils.py: Drop hal related functions and queries, replace with udev
+  database, udev log file, and DMI information from sysfs.
+- gtk UI: Convert from libglade to gtk.Builder.
+
+Bug fixes:
+- hookutils.py: Drop /proc/version_signature collection, it is Ubuntu specific.
+- apportcheckresume: Fix log collection from pm-utils.
+- Fix various crashes and report properties for reporting against uninstalled
+  packages.
+
+1.1.1 (2009-04-30)
+------------------
+Security fix:
+- etc/cron.daily/apport: Only attempt to remove files and symlinks, do not
+  descend into subdirectories of /var/crash/. Doing so might be exploited by a
+  race condition between find traversing a huge directory tree, changing an
+  existing subdir into a symlink to e. g. /etc/, and finally getting that piped
+  to rm. This also changes the find command to not use GNU extensions.  Thanks
+  to Stephane Chazelas for discovering this! (LP #357024, CVE-2009-1295)
+
+Bug fixes:
+- launchpad.py: Send and read Date: field again, reverting r1128; it is useful
+  after all. (LP #349139)
+- Only add ProcAttrCurrent to reports if it is not "unconfined", to remove some
+  noise from reports.
+- Detect invalid PIDs in the UI (such as for kernel processes) and give a
+  friendly error message instead of silently doing nothing. (LP #360608)
+- Always run common hooks, and run source package hooks if we do not have a
+  binary package name. (LP #350131)
+- launchpad.py: Consider socket errors when connecting as transient, so
+  that crash-digger doesn't stop completely on them.
+
+1.1 (2009-04-20)
+----------------
+New features:
+- Add hookutils methods for attaching relevant packages, greatly improve
+  attach_alsa() for sound problem debugging. 
+- Move launchpad crash database implementation from ever-breaking
+  python-launchpad-bugs (screenscraping) to launchpadlib (official and stable
+  Launchpad API).
+
+Bug fixes:
+- Drop some remaining distro specific pieces of code.
+- Add new field Report.pid which gets set on add_proc_info() and can be used by
+  hooks.
+- setup.py: Properly clean up all generated files, install missing
+  mimetypes/text-x-apport.svg icon symlink.
+- Add README file.
+- Add translations from Launchpad.
+- Remove preloadlib/*; it's undermaintained, and not really useful any more
+  these days.
+- Various bug fixes; most visible being the misnamed etc/default/apport.default
+  file (which should just be etc/default/apport).
+
+1.0 (2009-04-06)
+----------------
+First upstream release, based on Ubuntu packaging branch; that had been the
+de-facto trunk for many years, but this becomes unpractical with several
+distributions using it now.

=== renamed file 'NEWS' => 'NEWS.moved'
=== added file 'README'
--- README	1970-01-01 00:00:00 +0000
+++ README	2012-07-10 22:38:19 +0000
@@ -0,0 +1,87 @@
+Apport crash detection/reporting
+================================
+
+Apport intercepts Program crashes, collects debugging information about the
+crash and the operating system environment, and sends it to bug trackers in a
+standardized form. It also offers the user to report a bug about a package,
+with again collecting as much information about it as possible.
+
+It currently supports
+
+ - Crashes from standard signals (SIGSEGV, SIGILL, etc.) through the kernel
+   coredump handler (in piping mode)
+ - Unhandled Python exceptions
+ - GTK, KDE, and command line user interfaces
+ - Packages can ship hooks for collecting specific data (such as
+   /var/log/Xorg.0.log for X.org, or modified gconf settings for GNOME
+   programs)
+ - apt/dpkg and rpm backend (in production use in Ubuntu and OpenSUSE)
+ - Reprocessing a core dump and debug symbols for post-mortem (and preferably
+   server-side) generation of fully symbolic stack traces (apport-retrace)
+ - Reporting bugs to Launchpad (more backends can be easily added)
+
+Please see https://wiki.ubuntu.com/Apport for more details and further links.
+The files in doc/ document particular details such as package hooks, crash
+database configuration, or the internal data format.
+
+Temporarily enabling apport
+===========================
+
+The automatic crash interception component of apport is disabled by default in
+stable releases for a number of reasons [1].  To enable it just for the current
+session, do
+
+  sudo service apport start force_start=1
+
+Then you can simply trigger the crash again, and Apport's dialog will show up
+with instructions to report a bug with traces. Apport will be automatically
+disabled on next start.
+
+If you are triaging bugs, this is the best way to get traces from bug reporters
+that didn't use Apport in the first place.
+
+To enable it permanently, do:
+
+  sudo nano /etc/default/apport
+
+and change enabled from "0" to "1".
+
+[1] https://wiki.ubuntu.com/Apport#How%20to%20enable%20apport
+
+Crash notification on servers
+=============================
+
+You can add
+
+if [ -x /usr/bin/apport-cli ]; then 
+    if groups | grep -qw admin && /usr/share/apport/apport-checkreports -s; then
+	cat <<-EOF
+	You have new problem reports waiting in /var/crash. 
+	To take a look at them, run "sudo apport-cli".
+
+	EOF
+    elif /usr/share/apport/apport-checkreports; then
+	cat <<-EOF
+	You have new problem reports waiting in /var/crash. 
+	To take a look at them, run "apport-cli".
+
+	EOF
+    fi
+fi
+
+to your ~/.bashrc to get automatic notification of problem reports.
+
+Contributing
+============
+
+Please visit Apport's Launchpad homepage for links to the source code revision
+control, the bug tracker, translations, downloads, etc.:
+
+  https://launchpad.net/apport
+
+The preferred mode of operation for Linux distribution packagers is to create
+their own branch from 'trunk' and add the distro specific packaging and patches
+to it. Please send patches which are applicable to trunk as merge requests or
+bug reports, so that (1) other distributions can benefit from them as well, and
+(2) you reduce the code delta to upstream.
+

=== renamed file 'README' => 'README.moved'
=== added file 'TODO'
--- TODO	1970-01-01 00:00:00 +0000
+++ TODO	2012-07-10 22:38:19 +0000
@@ -0,0 +1,21 @@
+apport:
+ - check crashes of root processes with dropped privs in test suite
+
+dup detection:
+ - add merging of two databases -> needs time stamp of last change
+
+GUI:
+ - point out bug privacy and to leave it private by default
+
+hooks:
+ - add hooks which run during program crash, to collect more runtime data
+
+hookutils:
+- run hooks for related packages in attach_related_packages
+
+apt-dpkg backend:
+- use python-apt's Version.get_source() instead of apt-get source
+
+apport-retrace:
+- add "gdb" mode: apport-gdb /usr/bin/myprog -> imply -S system, throw you into
+  gdb session with debug symbols

=== renamed file 'TODO' => 'TODO.moved'
=== added directory 'apport'
=== renamed directory 'apport' => 'apport.moved'
=== added file 'apport/REThread.py'
--- apport/REThread.py	1970-01-01 00:00:00 +0000
+++ apport/REThread.py	2012-07-10 22:38:19 +0000
@@ -0,0 +1,66 @@
+'''Enhanced Thread with support for return values and exception propagation.'''
+
+# Copyright (C) 2007 Canonical Ltd.
+# Author: Martin Pitt <[e-mail]>
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 2 of the License, or (at your
+# option) any later version.  See http://www.gnu.org/copyleft/gpl.html for
+# the full text of the license.
+
+import threading, sys
+
+
+class REThread(threading.Thread):
+    '''Thread with return values and exception propagation.'''
+
+    def __init__(self, group=None, target=None, name=None, args=(), kwargs={},
+                 verbose=None):
+        '''Initialize Thread, identical to threading.Thread.__init__().'''
+
+        threading.Thread.__init__(self, group, target, name, args, kwargs, verbose)
+        self.__target = target
+        self.__args = args
+        self.__kwargs = kwargs
+        self._retval = None
+        self._exception = None
+
+    def run(self):
+        '''Run target function, identical to threading.Thread.run().'''
+
+        if self.__target:
+            try:
+                self._retval = self.__target(*self.__args, **self.__kwargs)
+            except:
+                if sys:
+                    self._exception = sys.exc_info()
+
+    def return_value(self):
+        '''Return value from target function.
+
+        This can only be called after the thread has finished, i. e. when
+        isAlive() is False and did not terminate with an exception.
+        '''
+        assert not self.isAlive()
+        assert not self._exception
+        return self._retval
+
+    def exc_info(self):
+        '''Return (type, value, traceback) of the exception caught in run().'''
+
+        return self._exception
+
+    def exc_raise(self):
+        '''Raise the exception caught in the thread.
+
+        Do nothing if no exception was caught.
+        '''
+        if self._exception:
+            # there is no syntax which both Python 2 and 3 parse, so we need a
+            # hack using exec() here
+            # Python 3:
+            if sys.version > '3':
+                raise self._exception[0](self._exception[1]).with_traceback(self._exception[2])
+            else:
+                exec('raise self._exception[0], self._exception[1], self._exception[2]')

=== added file 'apport/__init__.py'
--- apport/__init__.py	1970-01-01 00:00:00 +0000
+++ apport/__init__.py	2012-07-10 22:38:19 +0000
@@ -0,0 +1,42 @@
+from apport.report import Report
+
+from apport.packaging_impl import impl as packaging
+
+Report  # pyflakes
+packaging  # pyflakes
+
+import sys
+
+# fix gettext to output proper unicode strings
+import gettext
+
+
+def unicode_gettext(str):
+    trans = gettext.gettext(str)
+    if type(trans) == type(b''):
+        return trans.decode('UTF-8')
+    else:
+        return trans
+
+
+def fatal(msg, *args):
+    '''Print out an error message and exit the program.'''
+
+    error(msg, *args)
+    sys.exit(1)
+
+
+def error(msg, *args):
+    '''Print out an error message.'''
+
+    sys.stderr.write('ERROR: ')
+    sys.stderr.write(msg % args)
+    sys.stderr.write('\n')
+
+
+def warning(msg, *args):
+    '''Print out an warning message.'''
+
+    sys.stderr.write('WARNING: ')
+    sys.stderr.write(msg % args)
+    sys.stderr.write('\n')

=== added file 'apport/crashdb.py'
--- apport/crashdb.py	1970-01-01 00:00:00 +0000
+++ apport/crashdb.py	2012-07-10 22:38:19 +0000
@@ -0,0 +1,850 @@
+'''Abstract crash database interface.'''
+
+# Copyright (C) 2007 - 2009 Canonical Ltd.
+# Author: Martin Pitt <[e-mail]>
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 2 of the License, or (at your
+# option) any later version.  See http://www.gnu.org/copyleft/gpl.html for
+# the full text of the license.
+
+import os, os.path, sys, shutil
+
+try:
+    from exceptions import Exception
+    from urllib import quote_plus, urlopen
+    URLError = IOError
+    (quote_plus, urlopen)  # pyflakes
+except ImportError:
+    # python 3
+    from functools import cmp_to_key
+    from urllib.parse import quote_plus
+    from urllib.request import urlopen
+    from urllib.error import URLError
+
+import apport
+
+
+def _u(str):
+    '''Convert str to an unicode if it isn't already.'''
+
+    if type(str) == type(b''):
+        return str.decode('UTF-8', 'ignore')
+    return str
+
+
+class CrashDatabase:
+    def __init__(self, auth_file, options):
+        '''Initialize crash database connection.
+
+        You need to specify an implementation specific file with the
+        authentication credentials for retracing access for download() and
+        update(). For upload() and get_comment_url() you can use None.
+
+        options is a dictionary with additional settings from crashdb.conf; see
+        get_crashdb() for details.
+        '''
+        self.auth_file = auth_file
+        self.options = options
+        self.duplicate_db = None
+
+    def get_bugpattern_baseurl(self):
+        '''Return the base URL for bug patterns.
+
+        See apport.report.Report.search_bug_patterns() for details. If this
+        function returns None, bug patterns are disabled.
+        '''
+        return self.options.get('bug_pattern_url')
+
+    def accepts(self, report):
+        '''Check if this report can be uploaded to this database.
+
+        Crash databases might limit the types of reports they get with e. g.
+        the "problem_types" option.
+        '''
+        if 'problem_types' in self.options:
+            return report.get('ProblemType') in self.options['problem_types']
+
+        return True
+
+    #
+    # API for duplicate detection
+    #
+    # Tests are in apport/crashdb_impl/memory.py.
+
+    def init_duplicate_db(self, path):
+        '''Initialize duplicate database.
+
+        path specifies an SQLite database. It will be created if it does not
+        exist yet.
+        '''
+        import sqlite3 as dbapi2
+
+        assert dbapi2.paramstyle == 'qmark', \
+            'this module assumes qmark dbapi parameter style'
+
+        self.format_version = 3
+
+        init = not os.path.exists(path) or path == ':memory:' or \
+            os.path.getsize(path) == 0
+        self.duplicate_db = dbapi2.connect(path, timeout=7200)
+
+        if init:
+            cur = self.duplicate_db.cursor()
+            cur.execute('CREATE TABLE version (format INTEGER NOT NULL)')
+            cur.execute('INSERT INTO version VALUES (?)', [self.format_version])
+
+            cur.execute('''CREATE TABLE crashes (
+                signature VARCHAR(255) NOT NULL,
+                crash_id INTEGER NOT NULL,
+                fixed_version VARCHAR(50),
+                last_change TIMESTAMP,
+                CONSTRAINT crashes_pk PRIMARY KEY (crash_id))''')
+
+            cur.execute('''CREATE TABLE address_signatures (
+                signature VARCHAR(1000) NOT NULL,
+                crash_id INTEGER NOT NULL,
+                CONSTRAINT address_signatures_pk PRIMARY KEY (signature))''')
+
+            self.duplicate_db.commit()
+
+        # verify integrity
+        cur = self.duplicate_db.cursor()
+        cur.execute('PRAGMA integrity_check')
+        result = cur.fetchall()
+        if result != [('ok',)]:
+            raise SystemError('Corrupt duplicate db:' + str(result))
+
+        try:
+            cur.execute('SELECT format FROM version')
+            result = cur.fetchone()
+        except self.duplicate_db.OperationalError as e:
+            if 'no such table' in str(e):
+                # first db format did not have version table yet
+                result = [0]
+        if result[0] > self.format_version:
+            raise SystemError('duplicate DB has unknown format %i' % result[0])
+        if result[0] < self.format_version:
+            print('duplicate db has format %i, upgrading to %i' %
+                  (result[0], self.format_version))
+            self._duplicate_db_upgrade(result[0])
+
+    def check_duplicate(self, id, report=None):
+        '''Check whether a crash is already known.
+
+        If the crash is new, it will be added to the duplicate database and the
+        function returns None. If the crash is already known, the function
+        returns a pair (crash_id, fixed_version), where fixed_version might be
+        None if the crash is not fixed in the latest version yet. Depending on
+        whether the version in report is smaller than/equal to the fixed
+        version or larger, this calls close_duplicate() or mark_regression().
+
+        If the report does not have a valid crash signature, this function does
+        nothing and just returns None.
+
+        By default, the report gets download()ed, but for performance reasons
+        it can be explicitly passed to this function if it is already available.
+        '''
+        assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
+
+        if not report:
+            report = self.download(id)
+
+        self._mark_dup_checked(id, report)
+
+        if 'DuplicateSignature' in report:
+            sig = report['DuplicateSignature']
+        else:
+            sig = report.crash_signature()
+        existing = []
+        if sig:
+            # use real duplicate signature
+            existing = self._duplicate_search_signature(sig, id)
+
+            if existing:
+                # update status of existing master bugs
+                for (ex_id, _) in existing:
+                    self._duplicate_db_sync_status(ex_id)
+                existing = self._duplicate_search_signature(sig, id)
+
+        try:
+            report_package_version = report['Package'].split()[1]
+        except (KeyError, IndexError):
+            report_package_version = None
+
+        # check the existing IDs whether there is one that is unfixed or not
+        # older than the report's package version; if so, we have a duplicate.
+        master_id = None
+        master_ver = None
+        for (ex_id, ex_ver) in existing:
+            if not ex_ver or not report_package_version or apport.packaging.compare_versions(report_package_version, ex_ver) < 0:
+                master_id = ex_id
+                master_ver = ex_ver
+                break
+        else:
+            # if we did not find a new enough open master report,
+            # we have a regression of the latest fix. Mark it so, and create a
+            # new unfixed ID for it later on
+            if existing:
+                self.mark_regression(id, existing[-1][0])
+
+        # now query address signatures, they might turn up another duplicate
+        # (not necessarily the same, due to Stacktraces sometimes being
+        # slightly different)
+        addr_sig = report.crash_signature_addresses()
+        if addr_sig:
+            addr_match = self._duplicate_search_address_signature(addr_sig)
+            if addr_match and addr_match != master_id:
+                if master_id is None:
+                    # we have a duplicate only identified by address sig, close it
+                    master_id = addr_match
+                else:
+                    # our bug is a dupe of two different masters, one from
+                    # symbolic, the other from addr matching (see LP#943117);
+                    # make them all duplicates of each other, using the lower
+                    # number as master
+                    if master_id < addr_match:
+                        self.close_duplicate(report, addr_match, master_id)
+                        self._duplicate_db_merge_id(addr_match, master_id)
+                    else:
+                        self.close_duplicate(report, master_id, addr_match)
+                        self._duplicate_db_merge_id(master_id, addr_match)
+                        master_id = addr_match
+                        master_ver = None  # no version tracking for address signatures yet
+
+        if master_id is not None:
+            if addr_sig:
+                self._duplicate_db_add_address_signature(addr_sig, master_id)
+            self.close_duplicate(report, id, master_id)
+            return (master_id, master_ver)
+
+        # no duplicate detected; create a new record for the ID if we don't have one already
+        if sig:
+            cur = self.duplicate_db.cursor()
+            cur.execute('SELECT count(*) FROM crashes WHERE crash_id == ?', [id])
+            count_id = cur.fetchone()[0]
+            if count_id == 0:
+                cur.execute('INSERT INTO crashes VALUES (?, ?, ?, CURRENT_TIMESTAMP)', (_u(sig), id, None))
+                self.duplicate_db.commit()
+        if addr_sig:
+            self._duplicate_db_add_address_signature(addr_sig, id)
+
+        return None
+
+    def known(self, report):
+        '''Check if the crash db already knows about the crash signature.
+
+        Check if the report has a DuplicateSignature, crash_signature(), or
+        StacktraceAddressSignature, and ask the database whether the problem is
+        already known. If so, return an URL where the user can check the status
+        or subscribe (if available), or just return True if the report is known
+        but there is no public URL. In that case the report will not be
+        uploaded (i. e. upload() will not be called).
+
+        Return None if the report does not have any signature or the crash
+        database does not support checking for duplicates on the client side.
+
+        The default implementation uses a text file format generated by
+        duplicate_db_publish() at an URL specified by the "dupdb_url" option.
+        Subclasses are free to override this with a custom implementation, such
+        as a real database lookup.
+        '''
+        if not self.options.get('dupdb_url'):
+            return None
+
+        for kind in ('sig', 'address'):
+            # get signature
+            if kind == 'sig':
+                if 'DuplicateSignature' in report:
+                    sig = report['DuplicateSignature']
+                else:
+                    sig = report.crash_signature()
+            else:
+                sig = report.crash_signature_addresses()
+
+            if not sig:
+                continue
+
+            # build URL where the data should be
+            h = self.duplicate_sig_hash(sig)
+            if not h:
+                return None
+
+            # the hash is already quoted, but we really want to open the quoted
+            # file names; as urlopen() unquotes, we need to double-quote here
+            # again so that urlopen() sees the single-quoted file names
+            url = os.path.join(self.options['dupdb_url'], kind, quote_plus(h))
+
+            # read data file
+            try:
+                f = urlopen(url)
+                contents = f.read().decode('UTF-8')
+                f.close()
+                if '<title>404 Not Found' in contents:
+                    continue
+            except (IOError, URLError):
+                # does not exist, failed to load, etc.
+                continue
+
+            # now check if we find our signature
+            for line in contents.splitlines():
+                try:
+                    id, s = line.split(None, 1)
+                    id = int(id)
+                except ValueError:
+                    continue
+                if s == sig:
+                    result = self.get_id_url(report, id)
+                    if not result:
+                        # if we can't have an URL, just report as "known"
+                        result = '1'
+                    return result
+
+        return None
+
+    def duplicate_db_fixed(self, id, version):
+        '''Mark given crash ID as fixed in the duplicate database.
+
+        version specifies the package version the crash was fixed in (None for
+        'still unfixed').
+        '''
+        assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
+
+        cur = self.duplicate_db.cursor()
+        n = cur.execute('UPDATE crashes SET fixed_version = ?, last_change = CURRENT_TIMESTAMP WHERE crash_id = ?',
+                        (version, id))
+        assert n.rowcount == 1
+        self.duplicate_db.commit()
+
+    def duplicate_db_remove(self, id):
+        '''Remove crash from the duplicate database.
+
+        This happens when a report got rejected or manually duplicated.
+        '''
+        assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
+
+        cur = self.duplicate_db.cursor()
+        cur.execute('DELETE FROM crashes WHERE crash_id = ?', [id])
+        cur.execute('DELETE FROM address_signatures WHERE crash_id = ?', [id])
+        self.duplicate_db.commit()
+
+    def duplicate_db_change_master_id(self, old_id, new_id):
+        '''Change a crash ID.'''
+
+        assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
+
+        cur = self.duplicate_db.cursor()
+        cur.execute('UPDATE crashes SET crash_id = ?, last_change = CURRENT_TIMESTAMP WHERE crash_id = ?',
+                    [new_id, old_id])
+        cur.execute('UPDATE address_signatures SET crash_id = ? WHERE crash_id = ?',
+                    [new_id, old_id])
+        self.duplicate_db.commit()
+
+    def duplicate_db_publish(self, dir):
+        '''Create text files suitable for www publishing.
+
+        Create a number of text files in the given directory which Apport
+        clients can use to determine whether a problem is already reported to
+        the database, through the known() method. This directory is suitable
+        for publishing to the web.
+
+        The database is indexed by the first two fields of the duplicate or
+        crash signature, to avoid having to download the entire database every
+        time.
+
+        If the directory already exists, it will be updated. The new content is
+        built in a new directory which is the given one with ".new" appended,
+        then moved to the given name in an almost atomic way.
+        '''
+        assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
+
+        # first create the temporary new dir; if that fails, nothing has been
+        # changed and we fail early
+        out = dir + '.new'
+        os.mkdir(out)
+
+        # crash addresses
+        addr_base = os.path.join(out, 'address')
+        os.mkdir(addr_base)
+        cur_hash = None
+        cur_file = None
+
+        cur = self.duplicate_db.cursor()
+
+        cur.execute('SELECT * from address_signatures ORDER BY signature')
+        for (sig, id) in cur.fetchall():
+            h = self.duplicate_sig_hash(sig)
+            if h is None:
+                # some entries can't be represented in a single line
+                continue
+            if h != cur_hash:
+                cur_hash = h
+                if cur_file:
+                    cur_file.close()
+                cur_file = open(os.path.join(addr_base, cur_hash), 'w')
+
+            cur_file.write('%i %s\n' % (id, sig))
+
+        if cur_file:
+            cur_file.close()
+
+        # duplicate signatures
+        sig_base = os.path.join(out, 'sig')
+        os.mkdir(sig_base)
+        cur_hash = None
+        cur_file = None
+
+        cur.execute('SELECT signature, crash_id from crashes ORDER BY signature')
+        for (sig, id) in cur.fetchall():
+            h = self.duplicate_sig_hash(sig)
+            if h is None:
+                # some entries can't be represented in a single line
+                continue
+            if h != cur_hash:
+                cur_hash = h
+                if cur_file:
+                    cur_file.close()
+                cur_file = open(os.path.join(sig_base, cur_hash), 'wb')
+
+            cur_file.write(('%i %s\n' % (id, sig)).encode('UTF-8'))
+
+        if cur_file:
+            cur_file.close()
+
+        # switch over tree; this is as atomic as we can be with directories
+        if os.path.exists(dir):
+            os.rename(dir, dir + '.old')
+        os.rename(out, dir)
+        if os.path.exists(dir + '.old'):
+            shutil.rmtree(dir + '.old')
+
+    def _duplicate_db_upgrade(self, cur_format):
+        '''Upgrade database to current format'''
+
+        # Format 3 added a primary key which can't be done as an upgrade in
+        # SQLite
+        if cur_format < 3:
+            raise SystemError('Cannot upgrade database from format earlier than 3')
+
+        cur = self.duplicate_db.cursor()
+
+        cur.execute('UPDATE version SET format = ?', (cur_format,))
+        self.duplicate_db.commit()
+
+        assert cur_format == self.format_version
+
+    def _duplicate_search_signature(self, sig, id):
+        '''Look up signature in the duplicate db.
+
+        Return [(id, fixed_version)] tuple list.
+
+        There might be several matches if a crash has been reintroduced in a
+        later version. The results are sorted so that the highest fixed version
+        comes first, and "unfixed" being the last result.
+
+        id is the bug we are looking to find a duplicate for. The result will
+        never contain id, to avoid marking a bug as a duplicate of itself if a
+        bug is reprocessed more than once.
+        '''
+        cur = self.duplicate_db.cursor()
+        cur.execute('SELECT crash_id, fixed_version FROM crashes WHERE signature = ? AND crash_id <> ?', [_u(sig), id])
+        existing = cur.fetchall()
+
+        def cmp(x, y):
+            x = x[1]
+            y = y[1]
+            if x == y:
+                return 0
+            if x == '':
+                if y is None:
+                    return -1
+                else:
+                    return 1
+            if y == '':
+                if x is None:
+                    return 1
+                else:
+                    return -1
+            if x is None:
+                return 1
+            if y is None:
+                return -1
+            return apport.packaging.compare_versions(x, y)
+
+        if sys.version[0] >= '3':
+            existing.sort(key=cmp_to_key(cmp))
+        else:
+            existing.sort(cmp=cmp)
+
+        return existing
+
+    def _duplicate_search_address_signature(self, sig):
+        '''Return ID for crash address signature.
+
+        Return None if signature is unknown.
+        '''
+        if not sig:
+            return None
+
+        cur = self.duplicate_db.cursor()
+
+        cur.execute('SELECT crash_id FROM address_signatures WHERE signature == ?', [sig])
+        existing_ids = cur.fetchall()
+        assert len(existing_ids) <= 1
+        if existing_ids:
+            return existing_ids[0][0]
+        else:
+            return None
+
+    def _duplicate_db_dump(self, with_timestamps=False):
+        '''Return the entire duplicate database as a dictionary.
+
+        The returned dictionary maps "signature" to (crash_id, fixed_version)
+        pairs.
+
+        If with_timestamps is True, then the map will contain triples
+        (crash_id, fixed_version, last_change) instead.
+
+        This is mainly useful for debugging and test suites.
+        '''
+        assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
+
+        dump = {}
+        cur = self.duplicate_db.cursor()
+        cur.execute('SELECT * FROM crashes')
+        for (sig, id, ver, last_change) in cur:
+            if with_timestamps:
+                dump[sig] = (id, ver, last_change)
+            else:
+                dump[sig] = (id, ver)
+        return dump
+
+    def _duplicate_db_sync_status(self, id):
+        '''Update the duplicate db to the reality of the report in the crash db.
+
+        This uses get_fixed_version() to get the status of the given crash.
+        An invalid ID gets removed from the duplicate db, and a crash which got
+        fixed is marked as such in the database.
+        '''
+        assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
+
+        cur = self.duplicate_db.cursor()
+        cur.execute('SELECT fixed_version FROM crashes WHERE crash_id = ?', [id])
+        db_fixed_version = cur.fetchone()
+        if not db_fixed_version:
+            return
+        db_fixed_version = db_fixed_version[0]
+
+        real_fixed_version = self.get_fixed_version(id)
+
+        # crash got rejected
+        if real_fixed_version == 'invalid':
+            print('DEBUG: bug %i was invalidated, removing from database' % id)
+            self.duplicate_db_remove(id)
+            return
+
+        # crash got fixed
+        if not db_fixed_version and real_fixed_version:
+            print('DEBUG: bug %i got fixed in version %s, updating database' % (id, real_fixed_version))
+            self.duplicate_db_fixed(id, real_fixed_version)
+            return
+
+        # crash got reopened
+        if db_fixed_version and not real_fixed_version:
+            print('DEBUG: bug %i got reopened, dropping fixed version %s from database' % (id, db_fixed_version))
+            self.duplicate_db_fixed(id, real_fixed_version)
+            return
+
+    def _duplicate_db_add_address_signature(self, sig, id):
+        # sanity check
+        existing = self._duplicate_search_address_signature(sig)
+        if existing:
+            if existing != id:
+                raise SystemError('ID %i has signature %s, but database already has that signature for ID %i' % (
+                    id, sig, existing))
+        else:
+            cur = self.duplicate_db.cursor()
+            cur.execute('INSERT INTO address_signatures VALUES (?, ?)', (_u(sig), id))
+            self.duplicate_db.commit()
+
+    def _duplicate_db_merge_id(self, dup, master):
+        '''Merge two crash IDs.
+
+        This is necessary when having to mark a bug as a duplicate if it
+        already is in the duplicate DB.
+        '''
+        assert self.duplicate_db, 'init_duplicate_db() needs to be called before'
+
+        cur = self.duplicate_db.cursor()
+        cur.execute('DELETE FROM crashes WHERE crash_id = ?', [dup])
+        cur.execute('UPDATE address_signatures SET crash_id = ? WHERE crash_id = ?',
+                    [master, dup])
+        self.duplicate_db.commit()
+
+    @classmethod
+    def duplicate_sig_hash(klass, sig):
+        '''Create a www/URL proof hash for a duplicate signature'''
+
+        # cannot hash multi-line custom duplicate signatures
+        if '\n' in sig:
+            return None
+
+        # custom DuplicateSignatures have a free format, split off first word
+        i = sig.split(' ', 1)[0]
+        # standard crash/address signatures use ':' as field separator, usually
+        # for ExecutableName:Signal
+        i = '_'.join(i.split(':', 2)[:2])
+        # we manually quote '/' to make them nicer to read
+        i = i.replace('/', '_')
+        i = quote_plus(i.encode('UTF-8'))
+        # avoid too long file names
+        i = i[:200]
+        return i
+
+    #
+    # Abstract functions that need to be implemented by subclasses
+    #
+
+    def upload(self, report, progress_callback=None):
+        '''Upload given problem report return a handle for it.
+
+        This should happen noninteractively.
+
+        If the implementation supports it, and a function progress_callback is
+        passed, that is called repeatedly with two arguments: the number of
+        bytes already sent, and the total number of bytes to send. This can be
+        used to provide a proper upload progress indication on frontends.
+
+        Implementations ought to "assert self.accepts(report)". The UI logic
+        already prevents uploading a report to a database which does not accept
+        it, but for third-party users of the API this should still be checked.
+
+        This method can raise a NeedsCredentials exception in case of failure.
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def get_comment_url(self, report, handle):
+        '''Return an URL that should be opened after report has been uploaded
+        and upload() returned handle.
+
+        Should return None if no URL should be opened (anonymous filing without
+        user comments); in that case this function should do whichever
+        interactive steps it wants to perform.
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def get_id_url(self, report, id):
+        '''Return URL for a given report ID.
+
+        The report is passed in case building the URL needs additional
+        information from it, such as the SourcePackage name.
+
+        Return None if URL is not available or cannot be determined.
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def download(self, id):
+        '''Download the problem report from given ID and return a Report.'''
+
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def update(self, id, report, comment, change_description=False,
+               attachment_comment=None, key_filter=None):
+        '''Update the given report ID with all data from report.
+
+        This creates a text comment with the "short" data (see
+        ProblemReport.write_mime()), and creates attachments for all the
+        bulk/binary data.
+
+        If change_description is True, and the crash db implementation supports
+        it, the short data will be put into the description instead (like in a
+        new bug).
+
+        comment will be added to the "short" data. If attachment_comment is
+        given, it will be added to the attachment uploads.
+
+        If key_filter is a list or set, then only those keys will be added.
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def update_traces(self, id, report, comment=''):
+        '''Update the given report ID for retracing results.
+
+        This updates Stacktrace, ThreadStacktrace, StacktraceTop,
+        and StacktraceSource. You can also supply an additional comment.
+        '''
+        self.update(id, report, comment, key_filter=[
+            'Stacktrace', 'ThreadStacktrace', 'StacktraceSource', 'StacktraceTop'])
+
+    def set_credentials(self, username, password):
+        '''Set username and password.'''
+
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def get_distro_release(self, id):
+        '''Get 'DistroRelease: <release>' from the report ID.'''
+
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def get_unretraced(self):
+        '''Return set of crash IDs which have not been retraced yet.
+
+        This should only include crashes which match the current host
+        architecture.
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def get_dup_unchecked(self):
+        '''Return set of crash IDs which need duplicate checking.
+
+        This is mainly useful for crashes of scripting languages such as
+        Python, since they do not need to be retraced. It should not return
+        bugs that are covered by get_unretraced().
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def get_unfixed(self):
+        '''Return an ID set of all crashes which are not yet fixed.
+
+        The list must not contain bugs which were rejected or duplicate.
+
+        This function should make sure that the returned list is correct. If
+        there are any errors with connecting to the crash database, it should
+        raise an exception (preferably IOError).
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def get_fixed_version(self, id):
+        '''Return the package version that fixes a given crash.
+
+        Return None if the crash is not yet fixed, or an empty string if the
+        crash is fixed, but it cannot be determined by which version. Return
+        'invalid' if the crash report got invalidated, such as closed a
+        duplicate or rejected.
+
+        This function should make sure that the returned result is correct. If
+        there are any errors with connecting to the crash database, it should
+        raise an exception (preferably IOError).
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def get_affected_packages(self, id):
+        '''Return list of affected source packages for given ID.'''
+
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def is_reporter(self, id):
+        '''Check whether the user is the reporter of given ID.'''
+
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def can_update(self, id):
+        '''Check whether the user is eligible to update a report.
+
+        A user should add additional information to an existing ID if (s)he is
+        the reporter or subscribed, the bug is open, not a duplicate, etc. The
+        exact policy and checks should be done according to  the particular
+        implementation.
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def duplicate_of(self, id):
+        '''Return master ID for a duplicate bug.
+
+        If the bug is not a duplicate, return None.
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def close_duplicate(self, report, id, master):
+        '''Mark a crash id as duplicate of given master ID.
+
+        If master is None, id gets un-duplicated.
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def mark_regression(self, id, master):
+        '''Mark a crash id as reintroducing an earlier crash which is
+        already marked as fixed (having ID 'master').'''
+
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def mark_retraced(self, id):
+        '''Mark crash id as retraced.'''
+
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def mark_retrace_failed(self, id, invalid_msg=None):
+        '''Mark crash id as 'failed to retrace'.
+
+        If invalid_msg is given, the bug should be closed as invalid with given
+        message, otherwise just marked as a failed retrace.
+
+        This can be a no-op if you are not interested in this.
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+    def _mark_dup_checked(self, id, report):
+        '''Mark crash id as checked for being a duplicate
+
+        This is an internal method that should not be called from outside.
+        '''
+        raise NotImplementedError('this method must be implemented by a concrete subclass')
+
+#
+# factory
+#
+
+
+def get_crashdb(auth_file, name=None, conf=None):
+    '''Return a CrashDatabase object for the given crash db name.
+
+    This reads the configuration file 'conf'.
+
+    If name is None, it defaults to the 'default' value in conf.
+
+    If conf is None, it defaults to the environment variable
+    APPORT_CRASHDB_CONF; if that does not exist, the hardcoded default is
+    /etc/apport/crashdb.conf. This Python syntax file needs to specify:
+
+    - A string variable 'default', giving a default value for 'name' if that is
+      None.
+
+    - A dictionary 'databases' which maps names to crash db configuration
+      dictionaries. These need to have at least the key 'impl' (Python module
+      in apport.crashdb_impl which contains a concrete 'CrashDatabase' class
+      implementation for that crash db type). Other generally known options are
+      'bug_pattern_url', 'dupdb_url', and 'problem_types'.
+    '''
+    if not conf:
+        conf = os.environ.get('APPORT_CRASHDB_CONF', '/etc/apport/crashdb.conf')
+    settings = {}
+    with open(conf) as f:
+        exec(compile(f.read(), conf, 'exec'), settings)
+
+    # Load third parties crashdb.conf
+    confdDir = conf + '.d'
+    if os.path.isdir(confdDir):
+        for cf in os.listdir(confdDir):
+            cfpath = os.path.join(confdDir, cf)
+            if os.path.isfile(cfpath) and cf.endswith('.conf'):
+                try:
+                    with open(cfpath) as f:
+                        exec(compile(f.read(), cfpath, 'exec'), settings['databases'])
+                except Exception as e:
+                    # ignore broken files
+                    sys.stderr.write('Invalid file %s: %s\n' % (cfpath, str(e)))
+                    pass
+
+    if not name:
+        name = settings['default']
+
+    db = settings['databases'][name]
+
+    m = __import__('apport.crashdb_impl.' + db['impl'], globals(), locals(), ['CrashDatabase'])
+    return m.CrashDatabase(auth_file, db)
+
+
+class NeedsCredentials(Exception):
+    '''This may be raised when unable to log in to the crashdb.'''
+    pass

=== added directory 'apport/crashdb_impl'
=== added file 'apport/crashdb_impl/__init__.py'
=== added file 'apport/crashdb_impl/launchpad.py'
--- apport/crashdb_impl/launchpad.py	1970-01-01 00:00:00 +0000
+++ apport/crashdb_impl/launchpad.py	2012-07-10 22:38:19 +0000
@@ -0,0 +1,1931 @@
+# vim: set fileencoding=UTF-8 :
+'''Crash database implementation for Launchpad.'''
+
+# Copyright (C) 2007 - 2009 Canonical Ltd.
+# Authors: Martin Pitt <[e-mail]> and Markus Korn <[e-mail]>
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 2 of the License, or (at your
+# option) any later version.  See http://www.gnu.org/copyleft/gpl.html for
+# the full text of the license.
+
+import tempfile, os.path, re, gzip, sys, email, time
+
+from io import BytesIO
+
+if sys.version_info.major == 2:
+    from urllib2 import HTTPSHandler, Request, build_opener
+    from httplib import HTTPSConnection
+    from urllib import urlencode, urlopen
+    (HTTPSHandler, Request, build_opener, HTTPSConnection, urlencode, urlopen)  # pyflakes
+else:
+    from urllib.request import HTTPSHandler, Request, build_opener, urlopen
+    from urllib.parse import urlencode
+    from http.client import HTTPSConnection
+
+try:
+    from launchpadlib.errors import HTTPError
+    from launchpadlib.launchpad import Launchpad
+    Launchpad  # pyflakes
+except ImportError:
+    # if launchpadlib is not available, only client-side reporting will work
+    Launchpad = None
+
+import apport.crashdb
+import apport
+
+default_credentials_path = os.path.expanduser('~/.cache/apport/launchpad.credentials')
+
+
+def filter_filename(attachments):
+    for attachment in attachments:
+        try:
+            f = attachment.data.open()
+        except HTTPError:
+            apport.error('Broken attachment on bug, ignoring')
+            continue
+        name = f.filename
+        if name.endswith('.txt') or name.endswith('.gz'):
+            yield f
+
+
+def id_set(tasks):
+    # same as set(int(i.bug.id) for i in tasks) but faster
+    return set(int(i.self_link.split('/').pop()) for i in tasks)
+
+
+class CrashDatabase(apport.crashdb.CrashDatabase):
+    '''Launchpad implementation of crash database interface.'''
+
+    def __init__(self, auth, options):
+        '''Initialize Launchpad crash database.
+
+        You need to specify a launchpadlib-style credentials file to
+        access launchpad. If you supply None, it will use
+        default_credentials_path (~/.cache/apport/launchpad.credentials).
+
+        Recognized options are:
+        - distro: Name of the distribution in Launchpad
+        - project: Name of the project in Launchpad
+        (Note that exactly one of "distro" or "project" must be given.)
+        - launchpad_instance: If set, this uses the given launchpad instance
+          instead of production (optional). This can be overriden or set by
+          $APPORT_LAUNCHPAD_INSTANCE environment.
+        - cache_dir: Path to a permanent cache directory; by default it uses a
+          temporary one. (optional). This can be overridden or set by
+          $APPORT_LAUNCHPAD_CACHE environment.
+        - escalation_subscription: This subscribes the given person or team to
+          a bug once it gets the 10th duplicate.
+        - escalation_tag: This adds the given tag to a bug once it gets more
+          than 10 duplicates.
+        - initial_subscriber: The Launchpad user which gets subscribed to newly
+          filed bugs (default: "apport"). It should be a bot user which the
+          crash-digger instance runs as, as this will get to see all bug
+          details immediately.
+        - triaging_team: The Launchpad user/team which gets subscribed after
+          updating a crash report bug by the retracer (default:
+          "ubuntu-crashes-universe")
+        '''
+        if os.getenv('APPORT_LAUNCHPAD_INSTANCE'):
+            options['launchpad_instance'] = os.getenv(
+                'APPORT_LAUNCHPAD_INSTANCE')
+        if not auth:
+            lp_instance = options.get('launchpad_instance')
+            if lp_instance:
+                auth = default_credentials_path + '.' + lp_instance.split('://', 1)[-1]
+            else:
+                auth = default_credentials_path
+        apport.crashdb.CrashDatabase.__init__(self, auth, options)
+
+        self.distro = options.get('distro')
+        if self.distro:
+            assert 'project' not in options, 'Must not set both "project" and "distro" option'
+        else:
+            assert 'project' in options, 'Need to have either "project" or "distro" option'
+
+        self.arch_tag = 'need-%s-retrace' % apport.packaging.get_system_architecture()
+        self.options = options
+        self.auth = auth
+        assert self.auth
+
+        self.__launchpad = None
+        self.__lp_distro = None
+        self.__lpcache = os.getenv('APPORT_LAUNCHPAD_CACHE', options.get('cache_dir'))
+
+    @property
+    def launchpad(self):
+        '''Return Launchpad instance.'''
+
+        if self.__launchpad:
+            return self.__launchpad
+
+        if Launchpad is None:
+            sys.stderr.write('ERROR: The launchpadlib Python module is not installed. This functionality is not available.\n')
+            sys.exit(1)
+
+        if self.options.get('launchpad_instance'):
+            launchpad_instance = self.options.get('launchpad_instance')
+        else:
+            launchpad_instance = 'production'
+
+        auth_dir = os.path.dirname(self.auth)
+        if auth_dir and not os.path.isdir(auth_dir):
+            os.makedirs(auth_dir)
+
+        try:
+            self.__launchpad = Launchpad.login_with('apport-collect',
+                                                    launchpad_instance,
+                                                    launchpadlib_dir=self.__lpcache,
+                                                    allow_access_levels=['WRITE_PRIVATE'],
+                                                    credentials_file=self.auth,
+                                                    version='1.0')
+        except Exception as e:
+            if hasattr(e, 'content'):
+                msg = e.content
+            else:
+                msg = str(e)
+            apport.error('connecting to Launchpad failed: %s\nYou can reset the credentials by removing the file "%s"', msg, self.auth)
+            sys.exit(99)  # transient error
+
+        return self.__launchpad
+
+    def _get_distro_tasks(self, tasks):
+        if not self.distro:
+            raise StopIteration
+
+        for t in tasks:
+            if t.bug_target_name.lower() == self.distro or \
+                    re.match('^.+\(%s.*\)$' % self.distro, t.bug_target_name.lower()):
+                yield t
+
+    @property
+    def lp_distro(self):
+        if self.__lp_distro is None:
+            if self.distro:
+                self.__lp_distro = self.launchpad.distributions[self.distro]
+            elif 'project' in self.options:
+                self.__lp_distro = self.launchpad.projects[self.options['project']]
+            else:
+                raise SystemError('distro or project needs to be specified in crashdb options')
+
+        return self.__lp_distro
+
+    def upload(self, report, progress_callback=None):
+        '''Upload given problem report return a handle for it.
+
+        This should happen noninteractively.
+
+        If the implementation supports it, and a function progress_callback is
+        passed, that is called repeatedly with two arguments: the number of
+        bytes already sent, and the total number of bytes to send. This can be
+        used to provide a proper upload progress indication on frontends.
+        '''
+        assert self.accepts(report)
+
+        blob_file = self._generate_upload_blob(report)
+        ticket = upload_blob(blob_file, progress_callback, hostname=self.get_hostname())
+        blob_file.close()
+        assert ticket
+        return ticket
+
+    def get_hostname(self):
+        '''Return the hostname for the Launchpad instance.'''
+
+        launchpad_instance = self.options.get('launchpad_instance')
+        if launchpad_instance:
+            if launchpad_instance == 'staging':
+                hostname = 'staging.launchpad.net'
+            else:
+                hostname = 'launchpad.dev'
+        else:
+            hostname = 'launchpad.net'
+        return hostname
+
+    def get_comment_url(self, report, handle):
+        '''Return an URL that should be opened after report has been uploaded
+        and upload() returned handle.
+
+        Should return None if no URL should be opened (anonymous filing without
+        user comments); in that case this function should do whichever
+        interactive steps it wants to perform.'''
+
+        args = {}
+        title = report.get('Title', report.standard_title())
+        if title:
+            # always use UTF-8 encoding, urlencode() blows up otherwise in
+            # python 2.7
+            if type(title) != type(b''):
+                title = title.encode('UTF-8')
+            args['field.title'] = title
+
+        hostname = self.get_hostname()
+
+        project = self.options.get('project')
+
+        if not project:
+            if 'SourcePackage' in report:
+                return 'https://bugs.%s/%s/+source/%s/+filebug/%s?%s' % (
+                    hostname, self.distro, report['SourcePackage'], handle, urlencode(args))
+            else:
+                return 'https://bugs.%s/%s/+filebug/%s?%s' % (
+                    hostname, self.distro, handle, urlencode(args))
+        else:
+            return 'https://bugs.%s/%s/+filebug/%s?%s' % (
+                hostname, project, handle, urlencode(args))
+
+    def get_id_url(self, report, id):
+        '''Return URL for a given report ID.
+
+        The report is passed in case building the URL needs additional
+        information from it, such as the SourcePackage name.
+
+        Return None if URL is not available or cannot be determined.
+        '''
+        return 'https://bugs.launchpad.net/bugs/' + str(id)
+
+    def download(self, id):
+        '''Download the problem report from given ID and return a Report.'''
+
+        report = apport.Report()
+        b = self.launchpad.bugs[id]
+
+        # parse out fields from summary
+        m = re.search(r'(ProblemType:.*)$', b.description, re.S)
+        if not m:
+            m = re.search(r'^--- \r?$[\r\n]*(.*)', b.description, re.M | re.S)
+        assert m, 'bug description must contain standard apport format data'
+
+        description = m.group(1).encode('UTF-8').replace('\xc2\xa0', ' ').replace('\r\n', '\n')
+
+        if '\n\n' in description:
+            # this often happens, remove all empty lines between top and
+            # 'Uname'
+            if 'Uname:' in description:
+                # this will take care of bugs like LP #315728 where stuff
+                # is added after the apport data
+                (part1, part2) = description.split('Uname:', 1)
+                description = part1.replace('\n\n', '\n') + 'Uname:' \
+                    + part2.split('\n\n', 1)[0]
+            else:
+                # just parse out the Apport block; e. g. LP #269539
+                description = description.split('\n\n', 1)[0]
+
+        report.load(BytesIO(description))
+
+        if 'Date' not in report:
+            # We had not submitted this field for a while, claiming it
+            # redundant. But it is indeed required for up-to-the-minute
+            # comparison with log files, etc. For backwards compatibility with
+            # those reported bugs, read the creation date
+            try:
+                report['Date'] = b.date_created.ctime()
+            except AttributeError:
+                # support older wadllib API which returned strings
+                report['Date'] = b.date_created
+        if 'ProblemType' not in report:
+            if 'apport-bug' in b.tags:
+                report['ProblemType'] = 'Bug'
+            elif 'apport-crash' in b.tags:
+                report['ProblemType'] = 'Crash'
+            elif 'apport-kernelcrash' in b.tags:
+                report['ProblemType'] = 'KernelCrash'
+            elif 'apport-package' in b.tags:
+                report['ProblemType'] = 'Package'
+            else:
+                raise ValueError('cannot determine ProblemType from tags: ' + str(b.tags))
+
+        report['Tags'] = ' '.join(b.tags)
+
+        if 'Title' in report:
+            report['OriginalTitle'] = report['Title']
+
+        report['Title'] = b.title
+
+        for attachment in filter_filename(b.attachments):
+            key, ext = os.path.splitext(attachment.filename)
+            # ignore attachments with invalid keys
+            try:
+                report[key] = ''
+            except:
+                continue
+            if ext == '.txt':
+                report[key] = attachment.read()
+            elif ext == '.gz':
+                try:
+                    report[key] = gzip.GzipFile(fileobj=attachment).read()
+                except IOError as e:
+                    # some attachments are only called .gz, but are
+                    # uncompressed (LP #574360)
+                    if 'Not a gzip' not in str(e):
+                        raise
+                    attachment.seek(0)
+                    report[key] = attachment.read()
+            else:
+                raise Exception('Unknown attachment type: ' + attachment.filename)
+        return report
+
+    def update(self, id, report, comment, change_description=False,
+               attachment_comment=None, key_filter=None):
+        '''Update the given report ID with all data from report.
+
+        This creates a text comment with the "short" data (see
+        ProblemReport.write_mime()), and creates attachments for all the
+        bulk/binary data.
+
+        If change_description is True, and the crash db implementation supports
+        it, the short data will be put into the description instead (like in a
+        new bug).
+
+        comment will be added to the "short" data. If attachment_comment is
+        given, it will be added to the attachment uploads.
+
+        If key_filter is a list or set, then only those keys will be added.
+        '''
+        bug = self.launchpad.bugs[id]
+
+        if key_filter:
+            skip_keys = set(report.keys()) - set(key_filter)
+        else:
+            skip_keys = None
+
+        # we want to reuse the knowledge of write_mime() with all its different input
+        # types and output formatting; however, we have to dissect the mime ourselves,
+        # since we can't just upload it as a blob
+        mime = tempfile.TemporaryFile()
+        report.write_mime(mime, skip_keys=skip_keys)
+        mime.flush()
+        mime.seek(0)
+        msg = email.message_from_file(mime)
+        msg_iter = msg.walk()
+
+        # first part is the multipart container
+        part = msg_iter.next()
+        assert part.is_multipart()
+
+        # second part should be an inline text/plain attachments with all short
+        # fields
+        part = msg_iter.next()
+        assert not part.is_multipart()
+        assert part.get_content_type() == 'text/plain'
+
+        if not key_filter:
+            # when we update a complete report, we are updating an existing bug
+            # with apport-collect
+            x = bug.tags[:]  # LP#254901 workaround
+            x.append('apport-collected')
+            # add any tags (like the release) to the bug
+            if 'Tags' in report:
+                x += report['Tags'].lower().split()
+            bug.tags = x
+            bug.lp_save()
+            bug = self.launchpad.bugs[id]  # fresh bug object, LP#336866 workaround
+
+        # short text data
+        if change_description:
+            bug.description = bug.description + '\n--- \n' + part.get_payload(decode=True).decode('UTF-8', 'replace')
+            bug.lp_save()
+        else:
+            bug.newMessage(content=part.get_payload(decode=True), subject=comment)
+
+        # other parts are the attachments:
+        for part in msg_iter:
+            # print '   attachment: %s...' % part.get_filename()
+            bug.addAttachment(comment=attachment_comment or '',
+                              description=part.get_filename(),
+                              content_type=None,
+                              data=part.get_payload(decode=True),
+                              filename=part.get_filename(), is_patch=False)
+
+    def update_traces(self, id, report, comment=''):
+        '''Update the given report ID for retracing results.
+
+        This updates Stacktrace, ThreadStacktrace, StacktraceTop,
+        and StacktraceSource. You can also supply an additional comment.
+        '''
+        apport.crashdb.CrashDatabase.update_traces(self, id, report, comment)
+
+        bug = self.launchpad.bugs[id]
+        # ensure it's assigned to a package
+        if 'SourcePackage' in report:
+            for task in bug.bug_tasks:
+                if task.target.resource_type_link.endswith('#distribution'):
+                    task.target = self.lp_distro.getSourcePackage(name=report['SourcePackage'])
+                    task.lp_save()
+                    bug = self.launchpad.bugs[id]
+                    break
+
+        # remove core dump if stack trace is usable
+        if report.has_useful_stacktrace():
+            for a in bug.attachments:
+                if a.title == 'CoreDump.gz':
+                    try:
+                        a.removeFromBug()
+                    except HTTPError:
+                        pass  # LP#249950 workaround
+            try:
+                task = self._get_distro_tasks(bug.bug_tasks).next()
+                if task.importance == 'Undecided':
+                    task.importance = 'Medium'
+                    task.lp_save()
+            except StopIteration:
+                pass  # no distro tasks
+
+            # update bug title with retraced function name
+            fn = report.stacktrace_top_function()
+            if fn:
+                m = re.match('^(.*crashed with SIG.* in )([^( ]+)(\(\).*$)', bug.title)
+                if m and m.group(2) != fn:
+                    bug.title = m.group(1) + fn + m.group(3)
+                    try:
+                        bug.lp_save()
+                    except HTTPError:
+                        pass  # LP#336866 workaround
+                    bug = self.launchpad.bugs[id]
+
+        self._subscribe_triaging_team(bug, report)
+
+    def get_distro_release(self, id):
+        '''Get 'DistroRelease: <release>' from the given report ID and return
+        it.'''
+        bug = self.launchpad.bugs[id]
+        m = re.search('DistroRelease: ([-a-zA-Z0-9.+/ ]+)', bug.description)
+        if m:
+            return m.group(1)
+        raise ValueError('URL does not contain DistroRelease: field')
+
+    def get_affected_packages(self, id):
+        '''Return list of affected source packages for given ID.'''
+
+        bug_target_re = re.compile(
+            r'/%s/(?:(?P<suite>[^/]+)/)?\+source/(?P<source>[^/]+)$' % self.distro)
+
+        bug = self.launchpad.bugs[id]
+        result = []
+
+        for task in bug.bug_tasks:
+            match = bug_target_re.search(task.target.self_link)
+            if not match:
+                continue
+            if task.status in ('Invalid', "Won't Fix", 'Fix Released'):
+                continue
+            result.append(match.group('source'))
+        return result
+
+    def is_reporter(self, id):
+        '''Check whether the user is the reporter of given ID.'''
+
+        bug = self.launchpad.bugs[id]
+        return bug.owner.name == self.launchpad.me.name
+
+    def can_update(self, id):
+        '''Check whether the user is eligible to update a report.
+
+        A user should add additional information to an existing ID if (s)he is
+        the reporter or subscribed, the bug is open, not a duplicate, etc. The
+        exact policy and checks should be done according to  the particular
+        implementation.
+        '''
+        bug = self.launchpad.bugs[id]
+        if bug.duplicate_of:
+            return False
+
+        if bug.owner.name == self.launchpad.me.name:
+            return True
+
+        # check subscription
+        me = self.launchpad.me.self_link
+        for sub in bug.subscriptions.entries:
+            if sub['person_link'] == me:
+                return True
+
+        return False
+
+    def get_unretraced(self):
+        '''Return an ID set of all crashes which have not been retraced yet and
+        which happened on the current host architecture.'''
+        try:
+            bugs = self.lp_distro.searchTasks(tags=self.arch_tag, created_since='2011-08-01')
+            return id_set(bugs)
+        except Exception as e:
+            apport.error('connecting to Launchpad failed: %s', str(e))
+            sys.exit(99)  # transient error
+
+    def get_dup_unchecked(self):
+        '''Return an ID set of all crashes which have not been checked for
+        being a duplicate.
+
+        This is mainly useful for crashes of scripting languages such as
+        Python, since they do not need to be retraced. It should not return
+        bugs that are covered by get_unretraced().'''
+
+        try:
+            bugs = self.lp_distro.searchTasks(tags='need-duplicate-check', created_since='2011-08-01')
+            return id_set(bugs)
+        except Exception as e:
+            apport.error('connecting to Launchpad failed: %s', str(e))
+            sys.exit(99)  # transient error
+
+    def get_unfixed(self):
+        '''Return an ID set of all crashes which are not yet fixed.
+
+        The list must not contain bugs which were rejected or duplicate.
+
+        This function should make sure that the returned list is correct. If
+        there are any errors with connecting to the crash database, it should
+        raise an exception (preferably IOError).'''
+
+        bugs = self.lp_distro.searchTasks(tags='apport-crash')
+        return id_set(bugs)
+
+    def _get_source_version(self, package):
+        '''Return the version of given source package in the latest release of
+        given distribution.
+
+        If 'distro' is None, we will look for a launchpad project .
+        '''
+        sources = self.lp_distro.main_archive.getPublishedSources(
+            exact_match=True,
+            source_name=package,
+            distro_series=self.lp_distro.current_series
+        )
+        # first element is the latest one
+        return sources[0].source_package_version
+
+    def get_fixed_version(self, id):
+        '''Return the package version that fixes a given crash.
+
+        Return None if the crash is not yet fixed, or an empty string if the
+        crash is fixed, but it cannot be determined by which version. Return
+        'invalid' if the crash report got invalidated, such as closed a
+        duplicate or rejected.
+
+        This function should make sure that the returned result is correct. If
+        there are any errors with connecting to the crash database, it should
+        raise an exception (preferably IOError).
+        '''
+        # do not do version tracking yet; for that, we need to get the current
+        # distrorelease and the current package version in that distrorelease
+        # (or, of course, proper version tracking in Launchpad itself)
+
+        try:
+            b = self.launchpad.bugs[id]
+        except KeyError:
+            return 'invalid'
+
+        if b.duplicate_of:
+            return 'invalid'
+
+        tasks = list(b.bug_tasks)  # just fetch it once
+
+        if self.distro:
+            distro_identifier = '(%s)' % self.distro.lower()
+            fixed_tasks = filter(lambda task: task.status == 'Fix Released' and
+                                 distro_identifier in task.bug_target_display_name.lower(), tasks)
+
+            if not fixed_tasks:
+                fixed_distro = filter(lambda task: task.status == 'Fix Released' and
+                                      task.bug_target_name.lower() == self.distro.lower(), tasks)
+                if fixed_distro:
+                    # fixed in distro inself (without source package)
+                    return ''
+
+            if len(fixed_tasks) > 1:
+                apport.warning('There is more than one task fixed in %s %s, using first one to determine fixed version', self.distro, id)
+                return ''
+
+            if fixed_tasks:
+                task = fixed_tasks.pop()
+                try:
+                    return self._get_source_version(task.bug_target_display_name.split()[0])
+                except IndexError:
+                    # source does not exist any more
+                    return 'invalid'
+            else:
+                # check if there only invalid ones
+                invalid_tasks = filter(lambda task: task.status in ('Invalid', "Won't Fix", 'Expired') and
+                                       distro_identifier in task.bug_target_display_name.lower(), tasks)
+                if invalid_tasks:
+                    non_invalid_tasks = filter(
+                        lambda task: task.status not in ('Invalid', "Won't Fix", 'Expired') and
+                        distro_identifier in task.bug_target_display_name.lower(), tasks)
+                    if not non_invalid_tasks:
+                        return 'invalid'
+        else:
+            fixed_tasks = filter(lambda task: task.status == 'Fix Released', tasks)
+            if fixed_tasks:
+                # TODO: look for current series
+                return ''
+            # check if there any invalid ones
+            if filter(lambda task: task.status == 'Invalid', tasks):
+                return 'invalid'
+
+        return None
+
+    def duplicate_of(self, id):
+        '''Return master ID for a duplicate bug.
+
+        If the bug is not a duplicate, return None.
+        '''
+        b = self.launchpad.bugs[id].duplicate_of
+        if b:
+            return b.id
+        else:
+            return None
+
+    def close_duplicate(self, report, id, master_id):
+        '''Mark a crash id as duplicate of given master ID.
+
+        If master is None, id gets un-duplicated.
+        '''
+        bug = self.launchpad.bugs[id]
+
+        if master_id:
+            assert id != master_id, 'cannot mark bug %s as a duplicate of itself' % str(id)
+
+            # check whether the master itself is a dup
+            master = self.launchpad.bugs[master_id]
+            if master.duplicate_of:
+                master = master.duplicate_of
+                master_id = master.id
+                if master.id == id:
+                    # this happens if the bug was manually duped to a newer one
+                    apport.warning('Bug %i was manually marked as a dupe of newer bug %i, not closing as duplicate',
+                                   id, master_id)
+                    return
+
+            for a in bug.attachments:
+                if a.title in ('CoreDump.gz', 'Stacktrace.txt',
+                               'ThreadStacktrace.txt', 'ProcMaps.txt',
+                               'ProcStatus.txt', 'Registers.txt',
+                               'Disassembly.txt'):
+                    try:
+                        a.removeFromBug()
+                    except HTTPError:
+                        pass  # LP#249950 workaround
+
+            bug = self.launchpad.bugs[id]  # fresh bug object, LP#336866 workaround
+            bug.newMessage(content='Thank you for taking the time to report this crash and helping \
+to make this software better.  This particular crash has already been reported and \
+is a duplicate of bug #%i, so is being marked as such.  Please look at the \
+other bug report to see if there is any missing information that you can \
+provide, or to see if there is a workaround for the bug.  Additionally, any \
+further discussion regarding the bug should occur in the other report.  \
+Please continue to report any other bugs you may find.' % master_id,
+                           subject='This bug is a duplicate')
+
+            bug = self.launchpad.bugs[id]  # refresh, LP#336866 workaround
+            if bug.private:
+                bug.private = False
+
+            # set duplicate last, since we cannot modify already dup'ed bugs
+            if not bug.duplicate_of:
+                bug.duplicate_of = master
+
+            # cache tags of master bug report instead of performing multiple
+            # queries
+            master_tags = master.tags
+
+            if len(master.duplicates) == 10:
+                if 'escalation_tag' in self.options and self.options['escalation_tag'] not in master_tags and self.options.get('escalated_tag', ' invalid ') not in master_tags:
+                    master.tags = master_tags + [self.options['escalation_tag']]  # LP#254901 workaround
+                    master.lp_save()
+
+                if 'escalation_subscription' in self.options and self.options.get('escalated_tag', ' invalid ') not in master_tags:
+                    p = self.launchpad.people[self.options['escalation_subscription']]
+                    master.subscribe(person=p)
+
+            # requesting updated stack trace?
+            if report.has_useful_stacktrace() and ('apport-request-retrace' in master_tags
+                                                   or 'apport-failed-retrace' in master_tags):
+                self.update(master_id, report, 'Updated stack trace from duplicate bug %i' % id,
+                            key_filter=['Stacktrace', 'ThreadStacktrace',
+                                        'Package', 'Dependencies', 'ProcMaps', 'ProcCmdline'])
+
+                master = self.launchpad.bugs[master_id]
+                x = master.tags[:]  # LP#254901 workaround
+                try:
+                    x.remove('apport-failed-retrace')
+                except ValueError:
+                    pass
+                try:
+                    x.remove('apport-request-retrace')
+                except ValueError:
+                    pass
+                master.tags = x
+                try:
+                    master.lp_save()
+                except HTTPError:
+                    pass  # LP#336866 workaround
+
+            # white list of tags to copy from duplicates bugs to the master
+            tags_to_copy = ['bugpattern-needed', 'running-unity']
+            for series in self.lp_distro.series:
+                if series.status not in ['Active Development',
+                                         'Current Stable Release', 'Supported']:
+                    continue
+                tags_to_copy.append(series.name)
+            # copy tags over from the duplicate bug to the master bug
+            dupe_tags = set(bug.tags)
+            # reload master tags as they may have changed
+            master_tags = master.tags
+            missing_tags = dupe_tags.difference(master_tags)
+
+            for tag in missing_tags:
+                if tag in tags_to_copy:
+                    master_tags.append(tag)
+
+            master.tags = master_tags
+            master.lp_save()
+
+        else:
+            if bug.duplicate_of:
+                bug.duplicate_of = None
+
+        if bug._dirty_attributes:  # LP#336866 workaround
+            bug.lp_save()
+
+    def mark_regression(self, id, master):
+        '''Mark a crash id as reintroducing an earlier crash which is
+        already marked as fixed (having ID 'master').'''
+
+        bug = self.launchpad.bugs[id]
+        bug.newMessage(content='This crash has the same stack trace characteristics as bug #%i. \
+However, the latter was already fixed in an earlier package version than the \
+one in this report. This might be a regression or because the problem is \
+in a dependent package.' % master,
+                       subject='Possible regression detected')
+        bug = self.launchpad.bugs[id]  # fresh bug object, LP#336866 workaround
+        bug.tags = bug.tags + ['regression-retracer']  # LP#254901 workaround
+        bug.lp_save()
+
+    def mark_retraced(self, id):
+        '''Mark crash id as retraced.'''
+
+        bug = self.launchpad.bugs[id]
+        if self.arch_tag in bug.tags:
+            x = bug.tags[:]  # LP#254901 workaround
+            x.remove(self.arch_tag)
+            bug.tags = x
+            try:
+                bug.lp_save()
+            except HTTPError:
+                pass  # LP#336866 workaround
+
+    def mark_retrace_failed(self, id, invalid_msg=None):
+        '''Mark crash id as 'failed to retrace'.'''
+
+        bug = self.launchpad.bugs[id]
+        if invalid_msg:
+            try:
+                task = self._get_distro_tasks(bug.bug_tasks).next()
+            except StopIteration:
+                # no distro task, just use the first one
+                task = bug.bug_tasks[0]
+            task.status = 'Invalid'
+            task.lp_save()
+            bug.newMessage(content=invalid_msg,
+                           subject='Crash report cannot be processed')
+
+            for a in bug.attachments:
+                if a.title == 'CoreDump.gz':
+                    try:
+                        a.removeFromBug()
+                    except HTTPError:
+                        pass  # LP#249950 workaround
+        else:
+            if 'apport-failed-retrace' not in bug.tags:
+                bug.tags = bug.tags + ['apport-failed-retrace']  # LP#254901 workaround
+                bug.lp_save()
+
+    def _mark_dup_checked(self, id, report):
+        '''Mark crash id as checked for being a duplicate.'''
+
+        bug = self.launchpad.bugs[id]
+
+        # if we have a distro task without a package, fix it
+        if 'SourcePackage' in report:
+            for task in bug.bug_tasks:
+                if task.target.resource_type_link.endswith('#distribution'):
+                    task.target = self.lp_distro.getSourcePackage(
+                        name=report['SourcePackage'])
+                    task.lp_save()
+                    bug = self.launchpad.bugs[id]
+                    break
+
+        if 'need-duplicate-check' in bug.tags:
+            x = bug.tags[:]  # LP#254901 workaround
+            x.remove('need-duplicate-check')
+            bug.tags = x
+            bug.lp_save()
+            if 'Traceback' in report:
+                for task in bug.bug_tasks:
+                    if task.target.resource_type_link.endswith('#distribution'):
+                        if task.importance == 'Undecided':
+                            task.importance = 'Medium'
+                            task.lp_save()
+        self._subscribe_triaging_team(bug, report)
+
+    def known(self, report):
+        '''Check if the crash db already knows about the crash signature.
+
+        Check if the report has a DuplicateSignature, crash_signature(), or
+        StacktraceAddressSignature, and ask the database whether the problem is
+        already known. If so, return an URL where the user can check the status
+        or subscribe (if available), or just return True if the report is known
+        but there is no public URL. In that case the report will not be
+        uploaded (i. e. upload() will not be called).
+
+        Return None if the report does not have any signature or the crash
+        database does not support checking for duplicates on the client side.
+
+        The default implementation uses a text file format generated by
+        duplicate_db_publish() at an URL specified by the "dupdb_url" option.
+        Subclasses are free to override this with a custom implementation, such
+        as a real database lookup.
+        '''
+        # we override the method here to check if the user actually has access
+        # to the bug, and if the bug requests more retraces; in either case we
+        # should file it.
+        url = apport.crashdb.CrashDatabase.known(self, report)
+
+        if not url:
+            return url
+
+        # record the fact that it is a duplicate, for triagers
+        report['DuplicateOf'] = url
+
+        try:
+            f = urlopen(url + '/+text')
+        except IOError:
+            # if we are offline, or LP is down, upload will fail anyway, so we
+            # can just as well avoid the upload
+            return url
+
+        line = f.readline()
+        if not line.startswith(b'bug:'):
+            # presumably a 404 etc. page, which happens for private bugs
+            return True
+
+        # check tags
+        for line in f:
+            if line.startswith(b'tags:'):
+                if b'apport-failed-retrace' in line or b'apport-request-retrace' in line:
+                    return None
+                else:
+                    break
+
+            # stop at the first task, tags are in the first block
+            if not line.strip():
+                break
+
+        return url
+
+    def _subscribe_triaging_team(self, bug, report):
+        '''Subscribe the right triaging team to the bug.'''
+
+        #FIXME: this entire function is an ugly Ubuntu specific hack until LP
+        #gets a real crash db; see https://wiki.ubuntu.com/CrashReporting
+
+        if 'DistroRelease' in report and report['DistroRelease'].split()[0] != 'Ubuntu':
+            return  # only Ubuntu bugs are filed private
+
+        #use a url hack here, it is faster
+        person = '%s~%s' % (self.launchpad._root_uri,
+                            self.options.get('triaging_team', 'ubuntu-crashes-universe'))
+        bug.subscribe(person=person)
+
+    def _generate_upload_blob(self, report):
+        '''Generate a multipart/MIME temporary file for uploading.
+
+        You have to close the returned file object after you are done with it.
+        '''
+        # set reprocessing tags
+        hdr = {}
+        hdr['Tags'] = 'apport-%s' % report['ProblemType'].lower()
+        a = report.get('PackageArchitecture')
+        if not a or a == 'all':
+            a = report.get('Architecture')
+        if a:
+            hdr['Tags'] += ' ' + a
+        if 'Tags' in report:
+            hdr['Tags'] += ' ' + report['Tags'].lower()
+
+        # privacy/retracing for distro reports
+        # FIXME: ugly hack until LP has a real crash db
+        if 'DistroRelease' in report:
+            if a and ('VmCore' in report or 'CoreDump' in report):
+                hdr['Private'] = 'yes'
+                hdr['Subscribers'] = self.options.get('initial_subscriber', 'apport')
+                hdr['Tags'] += ' need-%s-retrace' % a
+            elif 'Traceback' in report:
+                hdr['Private'] = 'yes'
+                hdr['Subscribers'] = 'apport'
+                hdr['Tags'] += ' need-duplicate-check'
+        if 'DuplicateSignature' in report and 'need-duplicate-check' not in hdr['Tags']:
+                hdr['Tags'] += ' need-duplicate-check'
+
+        # if we have checkbox submission key, link it to the bug; keep text
+        # reference until the link is shown in Launchpad's UI
+        if 'CheckboxSubmission' in report:
+            hdr['HWDB-Submission'] = report['CheckboxSubmission']
+
+        # order in which keys should appear in the temporary file
+        order = ['ProblemType', 'DistroRelease', 'Package', 'Regression', 'Reproducible',
+                 'TestedUpstream', 'ProcVersionSignature', 'Uname', 'NonfreeKernelModules']
+
+        # write MIME/Multipart version into temporary file
+        mime = tempfile.TemporaryFile()
+        report.write_mime(mime, extra_headers=hdr, skip_keys=['Tags'], priority_fields=order)
+        mime.flush()
+        mime.seek(0)
+
+        return mime
+
+#
+# Launchpad storeblob API (should go into launchpadlib, see LP #315358)
+#
+
+_https_upload_callback = None
+
+
+#
+# This progress code is based on KodakLoader by Jason Hildebrand
+# <[e-mail]>. See http://www.opensky.ca/~jdhildeb/software/kodakloader/
+# for details.
+class HTTPSProgressConnection(HTTPSConnection):
+    '''Implement a HTTPSConnection with an optional callback function for
+    upload progress.'''
+
+    def send(self, data):
+        global _https_upload_callback
+
+        # if callback has not been set, call the old method
+        if not _https_upload_callback:
+            HTTPSConnection.send(self, data)
+            return
+
+        sent = 0
+        total = len(data)
+        chunksize = 1024
+        while sent < total:
+            _https_upload_callback(sent, total)
+            t1 = time.time()
+            HTTPSConnection.send(self, data[sent:(sent + chunksize)])
+            sent += chunksize
+            t2 = time.time()
+
+            # adjust chunksize so that it takes between .5 and 2
+            # seconds to send a chunk
+            if chunksize > 1024:
+                if t2 - t1 < .5:
+                    chunksize <<= 1
+                elif t2 - t1 > 2:
+                    chunksize >>= 1
+
+
+class HTTPSProgressHandler(HTTPSHandler):
+
+    def https_open(self, req):
+        return self.do_open(HTTPSProgressConnection, req)
+
+
+def upload_blob(blob, progress_callback=None, hostname='launchpad.net'):
+    '''Upload blob (file-like object) to Launchpad.
+
+    progress_callback can be set to a function(sent, total) which is regularly
+    called with the number of bytes already sent and total number of bytes to
+    send. It is called every 0.5 to 2 seconds (dynamically adapted to upload
+    bandwidth).
+
+    Return None on error, or the ticket number on success.
+
+    By default this uses the production Launchpad hostname. Set
+    hostname to 'launchpad.dev' or 'staging.launchpad.net' to use another
+    instance for testing.
+    '''
+    ticket = None
+    url = 'https://%s/+storeblob' % hostname
+
+    global _https_upload_callback
+    _https_upload_callback = progress_callback
+
+    # build the form-data multipart/MIME request
+    data = email.mime.multipart.MIMEMultipart()
+
+    submit = email.mime.text.MIMEText('1')
+    submit.add_header('Content-Disposition', 'form-data; name="FORM_SUBMIT"')
+    data.attach(submit)
+
+    form_blob = email.mime.base.MIMEBase('application', 'octet-stream')
+    form_blob.add_header('Content-Disposition', 'form-data; name="field.blob"; filename="x"')
+    form_blob.set_payload(blob.read().decode('ascii'))
+    data.attach(form_blob)
+
+    data_flat = BytesIO()
+    if sys.version_info.major == 2:
+        gen = email.generator.Generator(data_flat, mangle_from_=False)
+    else:
+        gen = email.generator.BytesGenerator(data_flat, mangle_from_=False)
+    gen.flatten(data)
+
+    # do the request; we need to explicitly set the content type here, as it
+    # defaults to x-www-form-urlencoded
+    req = Request(url, data_flat.getvalue())
+    req.add_header('Content-Type', 'multipart/form-data; boundary=' + data.get_boundary())
+    opener = build_opener(HTTPSProgressHandler)
+    result = opener.open(req)
+    ticket = result.info().get('X-Launchpad-Blob-Token')
+
+    assert ticket
+    return ticket
+
+#
+# Unit tests
+#
+
+if __name__ == '__main__':
+    import unittest, atexit, shutil, subprocess
+    import mock
+
+    crashdb = None
+    _segv_report = None
+    _python_report = None
+    _uncommon_description_report = None
+
+    class _T(unittest.TestCase):
+        # this assumes that a source package 'coreutils' exists and builds a
+        # binary package 'coreutils'
+        test_package = 'coreutils'
+        test_srcpackage = 'coreutils'
+
+        #
+        # Generic tests, should work for all CrashDB implementations
+        #
+
+        def setUp(self):
+            global crashdb
+            if not crashdb:
+                crashdb = self._get_instance()
+            self.crashdb = crashdb
+
+            # create a local reference report so that we can compare
+            # DistroRelease, Architecture, etc.
+            self.ref_report = apport.Report()
+            self.ref_report.add_os_info()
+            self.ref_report.add_user_info()
+            self.ref_report['SourcePackage'] = 'coreutils'
+
+            # Objects tests rely on.
+            self._create_project('langpack-o-matic')
+
+        def _create_project(self, name):
+            '''Create a project using launchpadlib to be used by tests.'''
+
+            project = self.crashdb.launchpad.projects[name]
+            if not project:
+                self.crashdb.launchpad.projects.new_project(
+                    description=name + 'description',
+                    display_name=name,
+                    name=name,
+                    summary=name + 'summary',
+                    title=name + 'title')
+
+        @property
+        def hostname(self):
+            '''Get the Launchpad hostname for the given crashdb.'''
+
+            return self.crashdb.get_hostname()
+
+        def get_segv_report(self, force_fresh=False):
+            '''Generate SEGV crash report.
+
+            This is only done once, subsequent calls will return the already
+            existing ID, unless force_fresh is True.
+
+            Return the ID.
+            '''
+            global _segv_report
+            if not force_fresh and _segv_report is not None:
+                return _segv_report
+
+            r = self._generate_sigsegv_report()
+            r.add_package_info(self.test_package)
+            r.add_os_info()
+            r.add_gdb_info()
+            r.add_user_info()
+            self.assertEqual(r.standard_title(), 'crash crashed with SIGSEGV in f()')
+
+            # add some binary gibberish which isn't UTF-8
+            r['ShortGibberish'] = ' "]\xb6"\n'
+            r['LongGibberish'] = 'a\nb\nc\nd\ne\n\xff\xff\xff\n\f'
+
+            # create a bug for the report
+            bug_target = self._get_bug_target(self.crashdb, r)
+            self.assertTrue(bug_target)
+
+            id = self._file_bug(bug_target, r)
+            self.assertTrue(id > 0)
+
+            sys.stderr.write('(Created SEGV report: https://%s/bugs/%i) ' % (self.hostname, id))
+            if not force_fresh:
+                _segv_report = id
+            return id
+
+        def get_python_report(self):
+            '''Generate Python crash report.
+
+            Return the ID.
+            '''
+            global _python_report
+            if _python_report is not None:
+                return _python_report
+
+            r = apport.Report('Crash')
+            r['ExecutablePath'] = '/bin/foo'
+            r['Traceback'] = '''Traceback (most recent call last):
+  File "/bin/foo", line 67, in fuzz
+    print(weird)
+NameError: global name 'weird' is not defined'''
+            r['Tags'] = 'boogus pybogus'
+            r.add_package_info(self.test_package)
+            r.add_os_info()
+            r.add_user_info()
+            self.assertEqual(r.standard_title(),
+                             "foo crashed with NameError in fuzz(): global name 'weird' is not defined")
+
+            bug_target = self._get_bug_target(self.crashdb, r)
+            self.assertTrue(bug_target)
+
+            id = self._file_bug(bug_target, r)
+            self.assertTrue(id > 0)
+            sys.stderr.write('(Created Python report: https://%s/bugs/%i) ' % (self.hostname, id))
+            _python_report = id
+            return id
+
+        def get_uncommon_description_report(self, force_fresh=False):
+            '''File a bug report with an uncommon description.
+
+            This is only done once, subsequent calls will return the already
+            existing ID, unless force_fresh is True.
+
+            Example taken from real LP bug 269539. It contains only
+            ProblemType/Architecture/DistroRelease in the description, and has
+            free-form description text after the Apport data.
+
+            Return the ID.
+            '''
+            global _uncommon_description_report
+            if not force_fresh and _uncommon_description_report is not None:
+                return _uncommon_description_report
+
+            desc = '''problem
+
+ProblemType: Package
+Architecture: amd64
+DistroRelease: Ubuntu 8.10
+
+more text
+
+and more
+'''
+            bug = self.crashdb.launchpad.bugs.createBug(
+                title=b'mixed description bug'.encode(),
+                description=desc,
+                target=self.crashdb.lp_distro)
+            sys.stderr.write('(Created uncommon description: https://%s/bugs/%i) ' % (self.hostname, bug.id))
+
+            if not force_fresh:
+                _uncommon_description_report = bug.id
+            return bug.id
+
+        def test_1_download(self):
+            '''download()'''
+
+            r = self.crashdb.download(self.get_segv_report())
+            self.assertEqual(r['ProblemType'], 'Crash')
+            self.assertEqual(r['Title'], 'crash crashed with SIGSEGV in f()')
+            self.assertEqual(r['DistroRelease'], self.ref_report['DistroRelease'])
+            self.assertEqual(r['Architecture'], self.ref_report['Architecture'])
+            self.assertEqual(r['Uname'], self.ref_report['Uname'])
+            self.assertEqual(r.get('NonfreeKernelModules'),
+                             self.ref_report.get('NonfreeKernelModules'))
+            self.assertEqual(r.get('UserGroups'), self.ref_report.get('UserGroups'))
+            tags = set(r['Tags'].split())
+            self.assertEqual(tags, set([self.crashdb.arch_tag, 'apport-crash',
+                                        apport.packaging.get_system_architecture()]))
+
+            self.assertEqual(r['Signal'], '11')
+            self.assertTrue(r['ExecutablePath'].endswith('/crash'))
+            self.assertEqual(r['SourcePackage'], self.test_srcpackage)
+            self.assertTrue(r['Package'].startswith(self.test_package + ' '))
+            self.assertTrue('f (x=42)' in r['Stacktrace'])
+            self.assertTrue('f (x=42)' in r['StacktraceTop'])
+            self.assertTrue('f (x=42)' in r['ThreadStacktrace'])
+            self.assertTrue(len(r['CoreDump']) > 1000)
+            self.assertTrue('Dependencies' in r)
+            self.assertTrue('Disassembly' in r)
+            self.assertTrue('Registers' in r)
+
+            # check tags
+            r = self.crashdb.download(self.get_python_report())
+            tags = set(r['Tags'].split())
+            self.assertEqual(tags, set(['apport-crash', 'boogus', 'pybogus',
+                                        'need-duplicate-check', apport.packaging.get_system_architecture()]))
+
+        def test_2_update_traces(self):
+            '''update_traces()'''
+
+            r = self.crashdb.download(self.get_segv_report())
+            self.assertTrue('CoreDump' in r)
+            self.assertTrue('Dependencies' in r)
+            self.assertTrue('Disassembly' in r)
+            self.assertTrue('Registers' in r)
+            self.assertTrue('Stacktrace' in r)
+            self.assertTrue('ThreadStacktrace' in r)
+            self.assertEqual(r['Title'], 'crash crashed with SIGSEGV in f()')
+
+            # updating with a useless stack trace retains core dump
+            r['StacktraceTop'] = '?? ()'
+            r['Stacktrace'] = 'long\ntrace'
+            r['ThreadStacktrace'] = 'thread\neven longer\ntrace'
+            r['FooBar'] = 'bogus'
+            self.crashdb.update_traces(self.get_segv_report(), r, 'I can has a better retrace?')
+            r = self.crashdb.download(self.get_segv_report())
+            self.assertTrue('CoreDump' in r)
+            self.assertTrue('Dependencies' in r)
+            self.assertTrue('Disassembly' in r)
+            self.assertTrue('Registers' in r)
+            self.assertTrue('Stacktrace' in r)  # TODO: ascertain that it's the updated one
+            self.assertTrue('ThreadStacktrace' in r)
+            self.assertFalse('FooBar' in r)
+            self.assertEqual(r['Title'], 'crash crashed with SIGSEGV in f()')
+
+            tags = self.crashdb.launchpad.bugs[self.get_segv_report()].tags
+            self.assertTrue('apport-crash' in tags)
+            self.assertFalse('apport-collected' in tags)
+
+            # updating with a useful stack trace removes core dump
+            r['StacktraceTop'] = 'read () from /lib/libc.6.so\nfoo (i=1) from /usr/lib/libfoo.so'
+            r['Stacktrace'] = 'long\ntrace'
+            r['ThreadStacktrace'] = 'thread\neven longer\ntrace'
+            self.crashdb.update_traces(self.get_segv_report(), r, 'good retrace!')
+            r = self.crashdb.download(self.get_segv_report())
+            self.assertFalse('CoreDump' in r)
+            self.assertTrue('Dependencies' in r)
+            self.assertTrue('Disassembly' in r)
+            self.assertTrue('Registers' in r)
+            self.assertTrue('Stacktrace' in r)
+            self.assertTrue('ThreadStacktrace' in r)
+            self.assertFalse('FooBar' in r)
+
+            # as previous title had standard form, the top function gets
+            # updated
+            self.assertEqual(r['Title'], 'crash crashed with SIGSEGV in read()')
+
+            # respects title amendments
+            bug = self.crashdb.launchpad.bugs[self.get_segv_report()]
+            bug.title = 'crash crashed with SIGSEGV in f() on exit'
+            try:
+                bug.lp_save()
+            except HTTPError:
+                pass  # LP#336866 workaround
+            r['StacktraceTop'] = 'read () from /lib/libc.6.so\nfoo (i=1) from /usr/lib/libfoo.so'
+            self.crashdb.update_traces(self.get_segv_report(), r, 'good retrace with title amendment')
+            r = self.crashdb.download(self.get_segv_report())
+            self.assertEqual(r['Title'], 'crash crashed with SIGSEGV in read() on exit')
+
+            # does not destroy custom titles
+            bug = self.crashdb.launchpad.bugs[self.get_segv_report()]
+            bug.title = 'crash is crashy'
+            try:
+                bug.lp_save()
+            except HTTPError:
+                pass  # LP#336866 workaround
+
+            r['StacktraceTop'] = 'read () from /lib/libc.6.so\nfoo (i=1) from /usr/lib/libfoo.so'
+            self.crashdb.update_traces(self.get_segv_report(), r, 'good retrace with custom title')
+            r = self.crashdb.download(self.get_segv_report())
+            self.assertEqual(r['Title'], 'crash is crashy')
+
+            # test various situations which caused crashes
+            r['Stacktrace'] = ''  # empty file
+            r['ThreadStacktrace'] = '"]\xb6"\n'  # not interpretable as UTF-8, LP #353805
+            r['StacktraceSource'] = 'a\nb\nc\nd\ne\n\xff\xff\xff\n\f'
+            self.crashdb.update_traces(self.get_segv_report(), r, 'tests')
+
+        def test_get_comment_url(self):
+            '''get_comment_url() for non-ASCII titles'''
+
+            # UTF-8 bytestring, works in both python 2.7 and 3
+            title = b'1\xc3\xa4\xe2\x99\xa52'
+
+            # distro, UTF-8 bytestring
+            r = apport.Report('Bug')
+            r['Title'] = title
+            url = self.crashdb.get_comment_url(r, 42)
+            self.assertTrue(url.endswith('/ubuntu/+filebug/42?field.title=1%C3%A4%E2%99%A52'))
+
+            # distro, unicode
+            r['Title'] = title.decode('UTF-8')
+            url = self.crashdb.get_comment_url(r, 42)
+            self.assertTrue(url.endswith('/ubuntu/+filebug/42?field.title=1%C3%A4%E2%99%A52'))
+
+            # package, unicode
+            r['SourcePackage'] = 'coreutils'
+            url = self.crashdb.get_comment_url(r, 42)
+            self.assertTrue(url.endswith('/ubuntu/+source/coreutils/+filebug/42?field.title=1%C3%A4%E2%99%A52'))
+
+        def test_update_description(self):
+            '''update() with changing description'''
+
+            bug_target = self.crashdb.lp_distro.getSourcePackage(name='bash')
+            bug = self.crashdb.launchpad.bugs.createBug(
+                description='test description for test bug.',
+                target=bug_target,
+                title='testbug')
+            id = bug.id
+            self.assertTrue(id > 0)
+            sys.stderr.write('(https://%s/bugs/%i) ' % (self.hostname, id))
+
+            r = apport.Report('Bug')
+
+            r['OneLiner'] = b'bogus\xe2\x86\x92'.decode('UTF-8')
+            r['StacktraceTop'] = 'f()\ng()\nh(1)'
+            r['ShortGoo'] = 'lineone\nlinetwo'
+            r['DpkgTerminalLog'] = 'one\ntwo\nthree\nfour\nfive\nsix'
+            r['VarLogDistupgradeBinGoo'] = b'\x01' * 1024
+
+            self.crashdb.update(id, r, 'NotMe', change_description=True)
+
+            r = self.crashdb.download(id)
+
+            self.assertEqual(r['OneLiner'], b'bogus\xe2\x86\x92'.decode('UTF-8'))
+            self.assertEqual(r['ShortGoo'], 'lineone\nlinetwo')
+            self.assertEqual(r['DpkgTerminalLog'], 'one\ntwo\nthree\nfour\nfive\nsix')
+            self.assertEqual(r['VarLogDistupgradeBinGoo'], b'\x01' * 1024)
+
+            self.assertEqual(self.crashdb.launchpad.bugs[id].tags,
+                             ['apport-collected'])
+
+        def test_update_comment(self):
+            '''update() with appending comment'''
+
+            bug_target = self.crashdb.lp_distro.getSourcePackage(name='bash')
+            # we need to fake an apport description separator here, since we
+            # want to be lazy and use download() for checking the result
+            bug = self.crashdb.launchpad.bugs.createBug(
+                description='Pr0blem\n\n--- \nProblemType: Bug',
+                target=bug_target,
+                title='testbug')
+            id = bug.id
+            self.assertTrue(id > 0)
+            sys.stderr.write('(https://%s/bugs/%i) ' % (self.hostname, id))
+
+            r = apport.Report('Bug')
+
+            r['OneLiner'] = 'bogus→'
+            r['StacktraceTop'] = 'f()\ng()\nh(1)'
+            r['ShortGoo'] = 'lineone\nlinetwo'
+            r['DpkgTerminalLog'] = 'one\ntwo\nthree\nfour\nfive\nsix'
+            r['VarLogDistupgradeBinGoo'] = '\x01' * 1024
+
+            self.crashdb.update(id, r, 'meow', change_description=False)
+
+            r = self.crashdb.download(id)
+
+            self.assertFalse('OneLiner' in r)
+            self.assertFalse('ShortGoo' in r)
+            self.assertEqual(r['ProblemType'], 'Bug')
+            self.assertEqual(r['DpkgTerminalLog'], 'one\ntwo\nthree\nfour\nfive\nsix')
+            self.assertEqual(r['VarLogDistupgradeBinGoo'], '\x01' * 1024)
+
+            self.assertEqual(self.crashdb.launchpad.bugs[id].tags,
+                             ['apport-collected'])
+
+        def test_update_filter(self):
+            '''update() with a key filter'''
+
+            bug_target = self.crashdb.lp_distro.getSourcePackage(name='bash')
+            bug = self.crashdb.launchpad.bugs.createBug(
+                description='test description for test bug',
+                target=bug_target,
+                title='testbug')
+            id = bug.id
+            self.assertTrue(id > 0)
+            sys.stderr.write('(https://%s/bugs/%i) ' % (self.hostname, id))
+
+            r = apport.Report('Bug')
+
+            r['OneLiner'] = 'bogus→'
+            r['StacktraceTop'] = 'f()\ng()\nh(1)'
+            r['ShortGoo'] = 'lineone\nlinetwo'
+            r['DpkgTerminalLog'] = 'one\ntwo\nthree\nfour\nfive\nsix'
+            r['VarLogDistupgradeBinGoo'] = '\x01' * 1024
+
+            self.crashdb.update(id, r, 'NotMe', change_description=True,
+                                key_filter=['ProblemType', 'ShortGoo', 'DpkgTerminalLog'])
+
+            r = self.crashdb.download(id)
+
+            self.assertFalse('OneLiner' in r)
+            self.assertEqual(r['ShortGoo'], 'lineone\nlinetwo')
+            self.assertEqual(r['ProblemType'], 'Bug')
+            self.assertEqual(r['DpkgTerminalLog'], 'one\ntwo\nthree\nfour\nfive\nsix')
+            self.assertFalse('VarLogDistupgradeBinGoo' in r)
+
+            self.assertEqual(self.crashdb.launchpad.bugs[id].tags, [])
+
+        def test_get_distro_release(self):
+            '''get_distro_release()'''
+
+            self.assertEqual(self.crashdb.get_distro_release(self.get_segv_report()),
+                             self.ref_report['DistroRelease'])
+
+        def test_get_affected_packages(self):
+            '''get_affected_packages()'''
+
+            self.assertEqual(self.crashdb.get_affected_packages(self.get_segv_report()),
+                             [self.ref_report['SourcePackage']])
+
+        def test_is_reporter(self):
+            '''is_reporter()'''
+
+            self.assertTrue(self.crashdb.is_reporter(self.get_segv_report()))
+            self.assertFalse(self.crashdb.is_reporter(1))
+
+        def test_can_update(self):
+            '''can_update()'''
+
+            self.assertTrue(self.crashdb.can_update(self.get_segv_report()))
+            self.assertFalse(self.crashdb.can_update(1))
+
+        def test_duplicates(self):
+            '''duplicate handling'''
+
+            # initially we have no dups
+            self.assertEqual(self.crashdb.duplicate_of(self.get_segv_report()), None)
+            self.assertEqual(self.crashdb.get_fixed_version(self.get_segv_report()), None)
+
+            segv_id = self.get_segv_report()
+            known_test_id = self.get_uncommon_description_report()
+            known_test_id2 = self.get_uncommon_description_report(force_fresh=True)
+
+            # dupe our segv_report and check that it worked; then undupe it
+            r = self.crashdb.download(segv_id)
+            self.crashdb.close_duplicate(r, segv_id, known_test_id)
+            self.assertEqual(self.crashdb.duplicate_of(segv_id), known_test_id)
+
+            # this should be a no-op
+            self.crashdb.close_duplicate(r, segv_id, known_test_id)
+            self.assertEqual(self.crashdb.duplicate_of(segv_id), known_test_id)
+
+            self.assertEqual(self.crashdb.get_fixed_version(segv_id), 'invalid')
+            self.crashdb.close_duplicate(r, segv_id, None)
+            self.assertEqual(self.crashdb.duplicate_of(segv_id), None)
+            self.assertEqual(self.crashdb.get_fixed_version(segv_id), None)
+
+            # this should have removed attachments; note that Stacktrace is
+            # short, and thus inline
+            r = self.crashdb.download(self.get_segv_report())
+            self.assertFalse('CoreDump' in r)
+            self.assertFalse('Disassembly' in r)
+            self.assertFalse('ProcMaps' in r)
+            self.assertFalse('ProcStatus' in r)
+            self.assertFalse('Registers' in r)
+            self.assertFalse('ThreadStacktrace' in r)
+
+            # now try duplicating to a duplicate bug; this should automatically
+            # transition to the master bug
+            self.crashdb.close_duplicate(apport.Report(), known_test_id,
+                                         known_test_id2)
+            self.crashdb.close_duplicate(r, segv_id, known_test_id)
+            self.assertEqual(self.crashdb.duplicate_of(segv_id),
+                             known_test_id2)
+
+            self.crashdb.close_duplicate(apport.Report(), known_test_id, None)
+            self.crashdb.close_duplicate(apport.Report(), known_test_id2, None)
+            self.crashdb.close_duplicate(r, segv_id, None)
+
+            # this should be a no-op
+            self.crashdb.close_duplicate(apport.Report(), known_test_id, None)
+            self.assertEqual(self.crashdb.duplicate_of(known_test_id), None)
+
+            self.crashdb.mark_regression(segv_id, known_test_id)
+            self._verify_marked_regression(segv_id)
+
+        def test_marking_segv(self):
+            '''processing status markings for signal crashes'''
+
+            # mark_retraced()
+            unretraced_before = self.crashdb.get_unretraced()
+            self.assertTrue(self.get_segv_report() in unretraced_before)
+            self.assertFalse(self.get_python_report() in unretraced_before)
+            self.crashdb.mark_retraced(self.get_segv_report())
+            unretraced_after = self.crashdb.get_unretraced()
+            self.assertFalse(self.get_segv_report() in unretraced_after)
+            self.assertEqual(unretraced_before,
+                             unretraced_after.union(set([self.get_segv_report()])))
+            self.assertEqual(self.crashdb.get_fixed_version(self.get_segv_report()), None)
+
+            # mark_retrace_failed()
+            self._mark_needs_retrace(self.get_segv_report())
+            self.crashdb.mark_retraced(self.get_segv_report())
+            self.crashdb.mark_retrace_failed(self.get_segv_report())
+            unretraced_after = self.crashdb.get_unretraced()
+            self.assertFalse(self.get_segv_report() in unretraced_after)
+            self.assertEqual(unretraced_before,
+                             unretraced_after.union(set([self.get_segv_report()])))
+            self.assertEqual(self.crashdb.get_fixed_version(self.get_segv_report()), None)
+
+            # mark_retrace_failed() of invalid bug
+            self._mark_needs_retrace(self.get_segv_report())
+            self.crashdb.mark_retraced(self.get_segv_report())
+            self.crashdb.mark_retrace_failed(self.get_segv_report(), "I don't like you")
+            unretraced_after = self.crashdb.get_unretraced()
+            self.assertFalse(self.get_segv_report() in unretraced_after)
+            self.assertEqual(unretraced_before,
+                             unretraced_after.union(set([self.get_segv_report()])))
+            self.assertEqual(self.crashdb.get_fixed_version(self.get_segv_report()),
+                             'invalid')
+
+        def test_marking_project(self):
+            '''processing status markings for a project CrashDB'''
+
+            # create a distro bug
+            distro_bug = self.crashdb.launchpad.bugs.createBug(
+                description='foo',
+                tags=self.crashdb.arch_tag,
+                target=self.crashdb.lp_distro,
+                title='ubuntu distro retrace bug')
+            #print('distro bug: https://staging.launchpad.net/bugs/%i' % distro_bug.id)
+
+            # create a project crash DB and a bug
+            launchpad_instance = os.environ.get('APPORT_LAUNCHPAD_INSTANCE') or 'staging'
+
+            project_db = CrashDatabase(
+                os.environ.get('LP_CREDENTIALS'),
+                {'project': 'langpack-o-matic', 'launchpad_instance': launchpad_instance})
+            project_bug = project_db.launchpad.bugs.createBug(
+                description='bar',
+                tags=project_db.arch_tag,
+                target=project_db.lp_distro,
+                title='project retrace bug')
+            #print('project bug: https://staging.launchpad.net/bugs/%i' % project_bug.id)
+
+            # on project_db, we recognize the project bug and can mark it
+            unretraced_before = project_db.get_unretraced()
+            self.assertTrue(project_bug.id in unretraced_before)
+            self.assertFalse(distro_bug.id in unretraced_before)
+            project_db.mark_retraced(project_bug.id)
+            unretraced_after = project_db.get_unretraced()
+            self.assertFalse(project_bug.id in unretraced_after)
+            self.assertEqual(unretraced_before,
+                             unretraced_after.union(set([project_bug.id])))
+            self.assertEqual(self.crashdb.get_fixed_version(project_bug.id), None)
+
+        def test_marking_python(self):
+            '''processing status markings for interpreter crashes'''
+
+            unchecked_before = self.crashdb.get_dup_unchecked()
+            self.assertTrue(self.get_python_report() in unchecked_before)
+            self.assertFalse(self.get_segv_report() in unchecked_before)
+            self.crashdb._mark_dup_checked(self.get_python_report(), self.ref_report)
+            unchecked_after = self.crashdb.get_dup_unchecked()
+            self.assertFalse(self.get_python_report() in unchecked_after)
+            self.assertEqual(unchecked_before,
+                             unchecked_after.union(set([self.get_python_report()])))
+            self.assertEqual(self.crashdb.get_fixed_version(self.get_python_report()), None)
+
+        def test_update_traces_invalid(self):
+            '''updating an invalid crash
+
+            This simulates a race condition where a crash being processed gets
+            invalidated by marking it as a duplicate.
+            '''
+            id = self.get_segv_report(force_fresh=True)
+
+            r = self.crashdb.download(id)
+
+            self.crashdb.close_duplicate(r, id, self.get_segv_report())
+
+            # updating with a useful stack trace removes core dump
+            r['StacktraceTop'] = 'read () from /lib/libc.6.so\nfoo (i=1) from /usr/lib/libfoo.so'
+            r['Stacktrace'] = 'long\ntrace'
+            r['ThreadStacktrace'] = 'thread\neven longer\ntrace'
+            self.crashdb.update_traces(id, r, 'good retrace!')
+
+            r = self.crashdb.download(id)
+            self.assertFalse('CoreDump' in r)
+
+        @mock.patch.object(CrashDatabase, '_get_source_version')
+        def test_get_fixed_version(self, *args):
+            '''get_fixed_version() for fixed bugs
+
+            Other cases are already checked in test_marking_segv() (invalid
+            bugs) and test_duplicates (duplicate bugs) for efficiency.
+            '''
+            # staging.launchpad.net often does not have Quantal, so mock-patch
+            # it to a known value
+            CrashDatabase._get_source_version.return_value = '3.14'
+            self._mark_report_fixed(self.get_segv_report())
+            fixed_ver = self.crashdb.get_fixed_version(self.get_segv_report())
+            self.assertEqual(fixed_ver, '3.14')
+            self._mark_report_new(self.get_segv_report())
+            self.assertEqual(self.crashdb.get_fixed_version(self.get_segv_report()), None)
+
+        #
+        # Launchpad specific implementation and tests
+        #
+
+        @classmethod
+        def _get_instance(klass):
+            '''Create a CrashDB instance'''
+
+            launchpad_instance = os.environ.get('APPORT_LAUNCHPAD_INSTANCE') or 'staging'
+
+            return CrashDatabase(os.environ.get('LP_CREDENTIALS'),
+                                 {'distro': 'ubuntu',
+                                  'launchpad_instance': launchpad_instance})
+
+        def _get_bug_target(self, db, report):
+            '''Return the bug_target for this report.'''
+
+            project = db.options.get('project')
+            if 'SourcePackage' in report:
+                return db.lp_distro.getSourcePackage(name=report['SourcePackage'])
+            elif project:
+                return db.launchpad.projects[project]
+            else:
+                return self.lp_distro
+
+        def _file_bug(self, bug_target, report, description=None):
+            '''File a bug report for a report.
+
+            Return the bug ID.
+            '''
+            # unfortunately staging's +storeblob API hardly ever works, so we
+            # must avoid using it. Fake it by manually doing the comments and
+            # attachments that +filebug would ordinarily do itself when given a
+            # blob handle.
+
+            if description is None:
+                description = 'some description'
+
+            mime = self.crashdb._generate_upload_blob(report)
+            msg = email.message_from_file(mime)
+            mime.close()
+            msg_iter = msg.walk()
+
+            # first one is the multipart container
+            header = msg_iter.next()
+            assert header.is_multipart()
+
+            # second part should be an inline text/plain attachments with all short
+            # fields
+            part = msg_iter.next()
+            assert not part.is_multipart()
+            assert part.get_content_type() == 'text/plain'
+            description += '\n\n' + part.get_payload(decode=True).decode('UTF-8', 'replace')
+
+            # create the bug from header and description data
+            bug = self.crashdb.launchpad.bugs.createBug(
+                description=description,
+                private=(header['Private'] == 'yes'),
+                tags=header['Tags'].split(),
+                target=bug_target,
+                title=report.get('Title', report.standard_title()))
+
+            # nwo add the attachments
+            for part in msg_iter:
+                assert not part.is_multipart()
+                bug.addAttachment(comment='',
+                                  description=part.get_filename(),
+                                  content_type=None,
+                                  data=part.get_payload(decode=True),
+                                  filename=part.get_filename(), is_patch=False)
+
+            for subscriber in header['Subscribers'].split():
+                sub = self.crashdb.launchpad.people[subscriber]
+                if sub:
+                    bug.subscribe(person=sub)
+
+            return bug.id
+
+        def _mark_needs_retrace(self, id):
+            '''Mark a report ID as needing retrace.'''
+
+            bug = self.crashdb.launchpad.bugs[id]
+            if self.crashdb.arch_tag not in bug.tags:
+                bug.tags = bug.tags + [self.crashdb.arch_tag]
+                bug.lp_save()
+
+        def _mark_needs_dupcheck(self, id):
+            '''Mark a report ID as needing duplicate check.'''
+
+            bug = self.crashdb.launchpad.bugs[id]
+            if 'need-duplicate-check' not in bug.tags:
+                bug.tags = bug.tags + ['need-duplicate-check']
+                bug.lp_save()
+
+        def _mark_report_fixed(self, id):
+            '''Close a report ID as "fixed".'''
+
+            bug = self.crashdb.launchpad.bugs[id]
+            tasks = list(bug.bug_tasks)
+            assert len(tasks) == 1
+            t = tasks[0]
+            t.status = 'Fix Released'
+            t.lp_save()
+
+        def _mark_report_new(self, id):
+            '''Reopen a report ID as "new".'''
+
+            bug = self.crashdb.launchpad.bugs[id]
+            tasks = list(bug.bug_tasks)
+            assert len(tasks) == 1
+            t = tasks[0]
+            t.status = 'New'
+            t.lp_save()
+
+        def _verify_marked_regression(self, id):
+            '''Verify that report ID is marked as regression.'''
+
+            bug = self.crashdb.launchpad.bugs[id]
+            self.assertTrue('regression-retracer' in bug.tags)
+
+        def test_project(self):
+            '''reporting crashes against a project instead of a distro'''
+
+            launchpad_instance = os.environ.get('APPORT_LAUNCHPAD_INSTANCE') or 'staging'
+            # crash database for langpack-o-matic project (this does not have
+            # packages in any distro)
+            crashdb = CrashDatabase(os.environ.get('LP_CREDENTIALS'),
+                                    {'project': 'langpack-o-matic',
+                                     'launchpad_instance': launchpad_instance})
+            self.assertEqual(crashdb.distro, None)
+
+            # create Python crash report
+            r = apport.Report('Crash')
+            r['ExecutablePath'] = '/bin/foo'
+            r['Traceback'] = '''Traceback (most recent call last):
+  File "/bin/foo", line 67, in fuzz
+    print(weird)
+NameError: global name 'weird' is not defined'''
+            r.add_os_info()
+            r.add_user_info()
+            self.assertEqual(r.standard_title(),
+                             "foo crashed with NameError in fuzz(): global name 'weird' is not defined")
+
+            # file it
+            bug_target = self._get_bug_target(crashdb, r)
+            self.assertEqual(bug_target.name, 'langpack-o-matic')
+
+            id = self._file_bug(bug_target, r)
+            self.assertTrue(id > 0)
+            sys.stderr.write('(https://%s/bugs/%i) ' % (self.hostname, id))
+
+            # update
+            r = crashdb.download(id)
+            r['StacktraceTop'] = 'read () from /lib/libc.6.so\nfoo (i=1) from /usr/lib/libfoo.so'
+            r['Stacktrace'] = 'long\ntrace'
+            r['ThreadStacktrace'] = 'thread\neven longer\ntrace'
+            crashdb.update_traces(id, r, 'good retrace!')
+            r = crashdb.download(id)
+
+            # test fixed version
+            self.assertEqual(crashdb.get_fixed_version(id), None)
+            crashdb.close_duplicate(r, id, self.get_uncommon_description_report())
+            self.assertEqual(crashdb.duplicate_of(id), self.get_uncommon_description_report())
+            self.assertEqual(crashdb.get_fixed_version(id), 'invalid')
+            crashdb.close_duplicate(r, id, None)
+            self.assertEqual(crashdb.duplicate_of(id), None)
+            self.assertEqual(crashdb.get_fixed_version(id), None)
+
+        def test_download_robustness(self):
+            '''download() of uncommon description formats'''
+
+            # only ProblemType/Architecture/DistroRelease in description
+            r = self.crashdb.download(self.get_uncommon_description_report())
+            self.assertEqual(r['ProblemType'], 'Package')
+            self.assertEqual(r['Architecture'], 'amd64')
+            self.assertTrue(r['DistroRelease'].startswith('Ubuntu '))
+
+        def test_escalation(self):
+            '''Escalating bugs with more than 10 duplicates'''
+
+            launchpad_instance = os.environ.get('APPORT_LAUNCHPAD_INSTANCE') or 'staging'
+            db = CrashDatabase(os.environ.get('LP_CREDENTIALS'),
+                               {'distro': 'ubuntu',
+                                'launchpad_instance': launchpad_instance,
+                                'escalation_tag': 'omgkittens',
+                                'escalation_subscription': 'apport-hackers'})
+
+            count = 0
+            p = db.launchpad.people[db.options['escalation_subscription']].self_link
+            first_dup = 59
+            try:
+                for b in range(first_dup, first_dup + 13):
+                    count += 1
+                    sys.stderr.write('%i ' % b)
+                    db.close_duplicate(apport.Report(), b, self.get_segv_report())
+                    b = db.launchpad.bugs[self.get_segv_report()]
+                    has_escalation_tag = db.options['escalation_tag'] in b.tags
+                    has_escalation_subscription = any([s.person_link == p for s in b.subscriptions])
+                    if count <= 10:
+                        self.assertFalse(has_escalation_tag)
+                        self.assertFalse(has_escalation_subscription)
+                    else:
+                        self.assertTrue(has_escalation_tag)
+                        self.assertTrue(has_escalation_subscription)
+            finally:
+                for b in range(first_dup, first_dup + count):
+                    sys.stderr.write('R%i ' % b)
+                    db.close_duplicate(apport.Report(), b, None)
+            sys.stderr.write('\n')
+
+        def test_marking_python_task_mangle(self):
+            '''source package task fixup for marking interpreter crashes'''
+
+            self._mark_needs_dupcheck(self.get_python_report())
+            unchecked_before = self.crashdb.get_dup_unchecked()
+            self.assertTrue(self.get_python_report() in unchecked_before)
+
+            # add an upstream task, and remove the package name from the
+            # package task; _mark_dup_checked is supposed to restore the
+            # package name
+            b = self.crashdb.launchpad.bugs[self.get_python_report()]
+            if b.private:
+                b.private = False
+                b.lp_save()
+            t = b.bug_tasks[0]
+            t.target = self.crashdb.launchpad.distributions['ubuntu']
+            t.lp_save()
+            b.addTask(target=self.crashdb.launchpad.projects['coreutils'])
+
+            self.crashdb._mark_dup_checked(self.get_python_report(), self.ref_report)
+
+            unchecked_after = self.crashdb.get_dup_unchecked()
+            self.assertFalse(self.get_python_report() in unchecked_after)
+            self.assertEqual(unchecked_before,
+                             unchecked_after.union(set([self.get_python_report()])))
+
+            # upstream task should be unmodified
+            b = self.crashdb.launchpad.bugs[self.get_python_report()]
+            self.assertEqual(b.bug_tasks[0].bug_target_name, 'coreutils')
+            self.assertEqual(b.bug_tasks[0].status, 'New')
+
+            # package-less distro task should have package name fixed
+            self.assertEqual(b.bug_tasks[1].bug_target_name, 'coreutils (Ubuntu)')
+            self.assertEqual(b.bug_tasks[1].status, 'New')
+
+            # should not confuse get_fixed_version()
+            self.assertEqual(self.crashdb.get_fixed_version(self.get_python_report()), None)
+
+        @classmethod
+        def _generate_sigsegv_report(klass, signal='11'):
+            '''Create a test executable which will die with a SIGSEGV, generate a
+            core dump for it, create a problem report with those two arguments
+            (ExecutablePath and CoreDump) and call add_gdb_info().
+
+            Return the apport.report.Report.
+            '''
+            workdir = None
+            orig_cwd = os.getcwd()
+            pr = apport.report.Report()
+            try:
+                workdir = tempfile.mkdtemp()
+                atexit.register(shutil.rmtree, workdir)
+                os.chdir(workdir)
+
+                # create a test executable
+                with open('crash.c', 'w') as fd:
+                    fd.write('''
+int f(x) {
+    int* p = 0; *p = x;
+    return x+1;
+}
+int main() { return f(42); }
+''')
+                assert subprocess.call(['gcc', '-g', 'crash.c', '-o', 'crash']) == 0
+                assert os.path.exists('crash')
+
+                # call it through gdb and dump core
+                subprocess.call(['gdb', '--batch', '--ex', 'run', '--ex',
+                                 'generate-core-file core', './crash'], stdout=subprocess.PIPE)
+                assert os.path.exists('core')
+                subprocess.check_call(['sync'])
+                assert subprocess.call(['readelf', '-n', 'core'],
+                                       stdout=subprocess.PIPE) == 0
+
+                pr['ExecutablePath'] = os.path.join(workdir, 'crash')
+                pr['CoreDump'] = (os.path.join(workdir, 'core'),)