Merge lp:~lifeless/python-oops-amqp/misc into lp:python-oops-amqp

[Robert Collins]

This updates the API to match current oops releases, and adds a useful trace helper.

[lp:~lifeless/python-oops-amqp/misc]

16. By Robert Collins on 2012-08-10

Release 0.0.7.

[James Westby (james-w)]

281 + description = "Load OOPS reports into oops-tools from AMQP."

Also, I think setup.py should change the requirement to oops>=0.0.11.

Otherwise I think this is ok.

Thanks,

James

[James Westby (james-w)]

review: Approve

[lp:~lifeless/python-oops-amqp/misc]

17. By Robert Collins on 2012-08-10

Fix bad trace description.

18. By Robert Collins on 2012-08-10

Encode the version constraint for oops in setup.py.

[William Grant (wgrant)]

review: Approve (code)

=== added file 'Makefile'
--- Makefile	1970-01-01 00:00:00 +0000
+++ Makefile	2012-08-10 03:02:19 +0000
@@ -0,0 +1,13 @@
+all:
+
+bin/buildout: buildout.cfg versions.cfg setup.py download-cache eggs
+	./bootstrap.py \
+		--setup-source=download-cache/ez_setup.py \
+		--download-base=download-cache/dist --eggs=eggs
+
+
+download-cache:
+	bzr checkout --lightweight lp:lp-source-dependencies download-cache
+
+eggs:
+	mkdir eggs
 
=== modified file 'NEWS'
--- NEWS	2012-02-10 14:47:11 +0000
+++ NEWS	2012-08-10 03:02:19 +0000
@@ -3,6 +3,18 @@
 
 Changes and improvements to oops-amqp, grouped by release.
 
+NEXT
+----
+
+0.0.7
+-----
+
+* New script 'oops-amqp-trace' which will trace oops reports received on an
+  AMQP exchange. (Robert Collins)
+
+* Updated to support the new publisher API in oops 0.0.11 and above. This is
+  incompatible with older versions of oops. (Robert Collins)
+
 0.0.6
 -----
 
 
=== modified file 'README'
--- README	2011-12-08 10:52:36 +0000
+++ README	2012-08-10 03:02:19 +0000
@@ -29,7 +29,7 @@
 
 * bson
 
-* oops (http://pypi.python.org/pypi/oops)
+* oops (http://pypi.python.org/pypi/oops) 0.0.11 or newer.
 
 * amqplib
 
@@ -63,7 +63,7 @@
 Provide the publisher to your OOPS config::
 
   >>> config = oops.Config()
-  >>> config.publishers.append(publisher)
+  >>> config.publisher = publisher
 
 Any oops published via that config will now be sent via amqp.
 
@@ -82,14 +82,14 @@
 the publication failed. To prevent losing the OOPS its a good idea to have a 
 fallback publisher - either another AMQP publisher (to a different server) or
 one that spools locally (where you can pick up the OOPSes via rsync or some
-other mechanism. Using the oops standard helper publish_new_only will let you
-wrap the fallback publisher so that it only gets invoked if the primary
+other mechanism. Using the oops standard helper publish_with_fallback will let
+you wrap the fallback publisher so that it only gets invoked if the primary
 method failed::
 
   >>> fallback_factory = partial(amqp.Connection, host="otherserver:5672",
   ...     userid="guest", password="guest", virtual_host="/", insist=False)
   >>> fallback_publisher = oops_amqp.Publisher(fallback_factory, "oopses", "")
-  >>> config.publishers.append(publish_new_only(fallback_publisher))
+  >>> config.publisher = publish_with_fallback(publisher, fallback_publisher)
 
 Receiving from AMQP
 +++++++++++++++++++
@@ -106,7 +106,7 @@
 
   >>> publisher = oops_datedir_repo.DateDirRepo('.', inherit_id=True)
   >>> config = oops.Config()
-  >>> config.publishers.append(publisher.publish)
+  >>> config.publisher = publisher.publish
   >>> receiver = oops_amqp.Receiver(config, factory, "my queue")
   >>> receiver.run_forever()
 
 
=== modified file 'oops_amqp/__init__.py'
--- oops_amqp/__init__.py	2012-02-13 00:03:12 +0000
+++ oops_amqp/__init__.py	2012-08-10 03:02:19 +0000
@@ -97,7 +97,7 @@
 # established at this point, and setup.py will use a version of next-$(revno).
 # If the releaselevel is 'final', then the tarball will be major.minor.micro.
 # Otherwise it is major.minor.micro~$(revno).
-__version__ = (0, 0, 6, 'beta', 0)
+__version__ = (0, 0, 7, 'final', 0)
 
 __all__ = [
     'Publisher',
 
=== modified file 'oops_amqp/publisher.py'
--- oops_amqp/publisher.py	2012-02-09 23:13:45 +0000
+++ oops_amqp/publisher.py	2012-08-10 03:02:19 +0000
@@ -88,7 +88,7 @@
         message.properties["delivery_mode"] = 2
         channel = self.get_channel()
         if channel is None:
-            return None
+            return []
         try:
             channel.basic_publish(
                 message, self.exchange_name, routing_key=self.routing_key)
@@ -96,7 +96,7 @@
             self.channels.channel = None
             if is_amqplib_connection_error(e):
                 # Could not connect / interrupted connection
-                return None
+                return []
             # Unknown error mode : don't hide it.
             raise
-        return report['id']
+        return [report['id']]
 
=== modified file 'oops_amqp/tests/test_publisher.py'
--- oops_amqp/tests/test_publisher.py	2012-02-09 23:13:45 +0000
+++ oops_amqp/tests/test_publisher.py	2012-08-10 03:02:19 +0000
@@ -40,9 +40,9 @@
         reference_oops = {'id': 'kept', 'akey': 'avalue'}
         oops = dict(reference_oops)
         expected_id = 'kept'
-        oops_id = publisher(oops)
+        oops_ids = publisher(oops)
         # Publication returns the oops ID allocated.
-        self.assertEqual(expected_id, oops_id)
+        self.assertEqual([expected_id], oops_ids)
         # The oops should not be altered by publication.
         self.assertEqual(reference_oops, oops)
         # The received OOPS should have the ID embedded and be a bson dict.
@@ -66,14 +66,14 @@
         oops = dict(reference_oops)
         id_bson = md5(bson.dumps(oops)).hexdigest()
         expected_id = "OOPS-%s" % id_bson
-        oops_id = publisher(oops)
+        oops_ids = publisher(oops)
         # Publication returns the oops ID allocated.
-        self.assertEqual(expected_id, oops_id)
+        self.assertEqual([expected_id], oops_ids)
         # The oops should not be altered by publication.
         self.assertEqual(reference_oops, oops)
         # The received OOPS should have the ID embedded and be a bson dict.
         expected_oops = dict(reference_oops)
-        expected_oops['id'] = oops_id
+        expected_oops['id'] = oops_ids[0]
         def check_oops(msg):
             self.assertEqual(expected_oops, bson.loads(msg.body))
             channel.basic_ack(msg.delivery_tag)
@@ -98,11 +98,11 @@
             publisher = Publisher(
                 self.connection_factory, queue.exchange_name, "")
             oops = {'akey': 42}
-            self.assertEqual(None, publisher(oops))
+            self.assertEqual([], publisher(oops))
         finally:
             self.rabbit.runner._start()
             queue.channel = self.connection_factory().channel()
-        self.assertNotEqual(None, publisher(oops))
+        self.assertNotEqual([], publisher(oops))
 
     def test_publish_amqp_down_after_use(self):
         # If amqp goes down after its been successfully used, None is returned
@@ -119,9 +119,9 @@
         # release.
         self.rabbit.runner._stop()
         try:
-            self.assertEqual(None, publisher(oops))
+            self.assertEqual([], publisher(oops))
         finally:
             self.rabbit.runner._start()
             queue.channel = self.connection_factory().channel()
-        self.assertNotEqual(None, publisher(oops))
+        self.assertNotEqual([], publisher(oops))
 
 
=== modified file 'oops_amqp/tests/test_receiver.py'
--- oops_amqp/tests/test_receiver.py	2012-02-09 23:13:45 +0000
+++ oops_amqp/tests/test_receiver.py	2012-08-10 03:02:19 +0000
@@ -39,7 +39,7 @@
         reports = []
         def capture(report):
             reports.append(report)
-            return report['id']
+            return [report['id']]
         expected_report = {'id': 'foo', 'otherkey': 42}
         message = amqp.Message(bson.dumps(expected_report))
         channel = self.useFixture(
@@ -51,7 +51,7 @@
         channel.basic_publish(
             amqp.Message(sentinel), queue.exchange_name, routing_key="")
         config = Config()
-        config.publishers.append(capture)
+        config.publisher = capture
         receiver = Receiver(config, self.connection_factory, queue.queue_name)
         receiver.sentinel = sentinel
         receiver.run_forever()
@@ -62,7 +62,7 @@
         reports = []
         def capture(report):
             reports.append(report)
-            return report['id']
+            return [report['id']]
         expected_report = {'id': 'foo', 'otherkey': 42}
         message = amqp.Message(bson.dumps(expected_report))
         channel = self.useFixture(
@@ -71,7 +71,7 @@
         channel.basic_publish(
             message, queue.exchange_name, routing_key="")
         config = Config()
-        config.publishers.append(capture)
+        config.publisher = capture
         # We don't want to loop forever: patch the channel so that after one
         # call to wait (which will get our injected message) the loop will shut
         # down.
@@ -134,7 +134,7 @@
         reports = []
         def capture(report):
             reports.append(report)
-            return report['id']
+            return [report['id']]
         expected_report = {'id': 'foo', 'otherkey': 42}
         message = amqp.Message(bson.dumps(expected_report))
         channel = self.useFixture(
@@ -142,7 +142,7 @@
         queue = self.useFixture(QueueFixture(channel, self.getUniqueString))
         channel.basic_publish(message, queue.exchange_name, routing_key="")
         config = Config()
-        config.publishers.append(capture)
+        config.publisher = capture
         state = {}
         def error_once(func):
             def wrapped(*args, **kwargs):
 
=== added file 'oops_amqp/trace.py'
--- oops_amqp/trace.py	1970-01-01 00:00:00 +0000
+++ oops_amqp/trace.py	2012-08-10 03:02:19 +0000
@@ -0,0 +1,74 @@
+# Copyright (c) 2012, Canonical Ltd
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License as published by
+# the Free Software Foundation, version 3 only.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+# GNU Lesser General Public License version 3 (see the file LICENSE).
+
+"""Trace OOPS reports coming from an AMQP queue."""
+
+from functools import partial
+import sys
+import optparse
+from textwrap import dedent
+
+import amqplib.client_0_8 as amqp
+import oops
+import oops_amqp
+
+import anybson as bson
+
+
+def main(argv=None):
+    if argv is None:
+        argv=sys.argv
+    usage = dedent("""\
+        %prog [options]
+
+        The following options must be supplied:
+         --host
+
+        e.g.
+        oops-amqp-trace --host "localhost:3472"
+
+        If you do not have a persistent queue, you should run this script
+        before generating oopses, as AMQP will discard messages with no
+        consumers.
+        """)
+    description = "Trace OOPS reports coming from an AMQP queue."
+    parser = optparse.OptionParser(
+        description=description, usage=usage)
+    parser.add_option('--host', help="AQMP host / host:port.")
+    parser.add_option('--username', help="AQMP username.", default="guest")
+    parser.add_option('--password', help="AQMP password.", default="guest")
+    parser.add_option('--vhost', help="AMQP vhost.", default="/")
+    parser.add_option('--exchange', help="AMQP exchange name.", default="oopses")
+    options, args = parser.parse_args(argv[1:])
+    def needed(optname):
+        if getattr(options, optname, None) is None:
+            raise ValueError('option "%s" must be supplied' % optname)
+    needed('host')
+    factory = partial(
+        amqp.Connection, host=options.host, userid=options.username,
+        password=options.password, virtual_host=options.vhost)
+    connection = factory()
+    channel = connection.channel()
+    channel.exchange_declare(options.exchange, type="fanout", durable=False,
+        auto_delete=True)
+    queue = channel.queue_declare(durable=False, auto_delete=True)[0]
+    channel.queue_bind(queue, options.exchange)
+    config = oops.Config()
+    config.publisher = oops.pprint_to_stream(sys.stdout)
+    receiver = oops_amqp.Receiver(config, factory, queue)
+    try:
+        receiver.run_forever()
+    except KeyboardInterrupt:
+        pass
 
=== modified file 'setup.py'
--- setup.py	2012-02-10 14:47:11 +0000
+++ setup.py	2012-08-10 03:02:19 +0000
@@ -23,7 +23,7 @@
         os.path.join(os.path.dirname(__file__), 'README'), 'rb').read()
 
 setup(name="oops_amqp",
-      version="0.0.6",
+      version="0.0.7",
       description=\
               "OOPS AMQP transport.",
       long_description=description,
@@ -41,7 +41,7 @@
           ],
       install_requires = [
           'pymongo',
-          'oops',
+          'oops>=0.0.11',
           'amqplib',
           ],
       extras_require = dict(
@@ -51,4 +51,8 @@
               'testtools',
               ]
           ),
+        entry_points=dict(
+            console_scripts=[ # `console_scripts` is a magic name to setuptools
+                'oops-amqp-trace = oops_amqp.trace:main',
+                ]),
       )
 
=== modified file 'versions.cfg'
--- versions.cfg	2011-10-10 21:49:50 +0000
+++ versions.cfg	2012-08-10 03:02:19 +0000
@@ -6,7 +6,8 @@
 bson = 0.3.2
 fixtures = 0.3.6
 iso8601 = 0.1.4
-oops = 0.0.6
+oops = 0.0.13
+pymongo = 2.1.1
 pytz = 2010o
 rabbitfixture = 0.3.2
 setuptools = 0.6c11