CaptureMock

Merge lp:~msabramo/capturemock/tests into lp:capturemock

  1. tests
  2. Merge into trunk
Proposed by Marc Abramowitz
Status: Needs review
Proposed branch: lp:~msabramo/capturemock/tests
Merge into: lp:capturemock
Diff against target: 322 lines (+248/-0)
14 files modified
.bzrignore (+1/-0)
setup.cfg (+21/-0)
tests/capturemock/replay_thing.mock (+19/-0)
tests/capturemock/requests_get.mock (+9/-0)
tests/capturemock/sqlalchemy.mock (+13/-0)
tests/capturemock/urllib2.mock (+13/-0)
tests/cars.py (+17/-0)
tests/test_cars.py (+16/-0)
tests/test_requests.py (+24/-0)
tests/test_sqlalchemy.py (+18/-0)
tests/test_thing.py (+62/-0)
tests/test_urllib2.py (+19/-0)
tests/thing.py (+4/-0)
tox.ini (+12/-0)
To merge this branch: bzr merge lp:~msabramo/capturemock/tests
Reviewer Review Type Date Requested Status
Geoff Bache Pending
Review via email: mp+197297@code.launchpad.net

Description of the change

This branch adds some simple tests that work with py.test or nosetests.

It looks like you have been testing CaptureMock with your own tool, TextTest. I'm not very familiar with TextTest and had some trouble setting it up because of the Gtk dependency, so it seemed nice to have tests that work other test runners.

```
marca@marca-mac2:/tmp$ bzr branch lp:~msabramo/capturemock/tests capturemock
...
marca@marca-mac2:/tmp$ cd capturemock/
marca@marca-mac2:/tmp/capturemock$ virtualenv venv
...
marca@marca-mac2:/tmp/capturemock$ source venv/bin/activate
(venv)marca@marca-mac2:/tmp/capturemock$ pip install -e . pytest nose
...
(venv)marca@marca-mac2:/tmp/capturemock$ py.test -v --assert=reinterp tests/
============================================================================= test session starts ==============================================================================
platform darwin -- Python 2.7.2 -- pytest-2.4.2 -- /private/tmp/capturemock/venv/bin/python
collected 3 items

tests/test_cars.py <- capturemock/__init__.py:204: test_cars PASSED
tests/test_thing.py <- capturemock/__init__.py:204: test_replay_thing PASSED
tests/test_thing.py:34: test_thing_recorded PASSED

=========================================================================== 3 passed in 0.34 seconds ===========================================================================
(venv)marca@marca-mac2:/tmp/capturemock$ nosetests -s -v tests/
test_cars.test_cars ... wheel.id = 1
wheel.id = 1
wheel.id = 1
wheel.id = 1
ok
test_thing.test_thing_recorded ... ok
test_thing.test_replay_thing ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.256s

OK

To post a comment you must log in.
Revision history for this message
Marc Abramowitz (msabramo) wrote :

Note that with py.test, the option `--assert=reinterp` is needed; otherwise it seems that some metaclass hackery that py.test assertion rewriting does conflicts with the metaclass hackery that CaptureMock does?

Reply
Revision history for this message
Marc Abramowitz (msabramo) wrote :

Note that the fact that the output of the nosetests has `wheel.id = 1` in it 4 times, might be a bug in CaptureMock? If I comment out the `@capturemock` line, then I get 4 different wheel ids. @capturemock seems to be changing the semantics of the code so that it returns one object instead of 4 unique objects.

(with @capturemock commented out):

```
(venv)marca@marca-mac2:/tmp/capturemock$ nosetests -s -v tests/test_cars.py
test_cars.test_cars ... wheel.id = 1
wheel.id = 2
wheel.id = 3
wheel.id = 4
ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK
```

Reply
lp:~msabramo/capturemock/tests updated
4635. By Marc Abramowitz <email address hidden>

Add support for tox

4636. By Marc Abramowitz <email address hidden>

Add a test that demonstrates replaying stuff from urllib2, urllib, and httplib

4637. By Marc Abramowitz <email address hidden>

Enhance tests/test_requests.py

4638. By Marc Abramowitz <email address hidden>

Add tests/capturemock/requests_get.mock

4639. By Marc Abramowitz <email address hidden>

Add setup.cfg so that you don't need to remember to run py.test with --assert=reinterp

Revision history for this message
Marc Abramowitz (msabramo) wrote :

I added a `setup.cfg` file that automatically passes the appropriate options for py.test, so now you can simply do:

```
(capturemock)marca@marca-mac2:~/dev/bzr-repos/capturemock$ py.test
============================================================================= test session starts ==============================================================================
platform darwin -- Python 2.7.2 -- pytest-2.4.2 -- /Users/marca/.virtualenvs/capturemock/bin/python
collected 5 items

tests/test_cars.py <- capturemock/__init__.py:204: test_cars wheel.id = 1
wheel.id = 1
wheel.id = 1
wheel.id = 1
PASSED
tests/test_requests.py <- capturemock/__init__.py:204: test_requests_get PASSED
tests/test_thing.py <- capturemock/__init__.py:204: test_replay_thing PASSED
tests/test_thing.py:34: test_thing_recorded PASSED
tests/test_urllib2.py <- capturemock/__init__.py:204: test_urllib2 PASSED
```

Reply
Revision history for this message
Marc Abramowitz (msabramo) wrote :

I also added a tox.ini so you if you have tox installed, you can run tox to test in multiple Python environments:

```
marca@marca-mac2:~/dev/bzr-repos/capturemock$ tox
...
  py26: commands succeeded
  py27: commands succeeded
  congratulations :)
```

Reply
Revision history for this message
Marc Abramowitz (msabramo) wrote :

If this were on GitHub, then I would add a .travis.yml for Travis CI so there is automatic CI. But it seems that you have your own CI system anyway, as you mention "night jobs" on the TextTest page so Travis CI might not be necessary.

Reply
lp:~msabramo/capturemock/tests updated
4640. By Marc Abramowitz <email address hidden>

Add test of mocking sqlalchemy and pymssql

Revision history for this message
Marc Abramowitz (msabramo) wrote :

I just added a test of mocking sqlalchemy. I think I'm probably done for now and will probably wait for you to review what I've done.

Thanks for sharing CaptureMock!

Reply
Revision history for this message
Geoff Bache (geoff.bache) wrote :

Hi Marc,

Thanks for all this, I'll try and review it in more detail later this week.
Seems like you might have found a couple of bugs here as well, which is very useful!

Fundamentally, there are already 138 fairly comprehensive acceptance tests for CaptureMock using TextTest (around 97% coverage), so simply adding some new ones that run via nose or py.test won't really help us very much. To actually be useful, we would need to translate the entire test suite really, which is probably only worth the effort if you (or some other non-TextTest user) are planning to do serious development and has insurmountable problems installing TextTest. In addition, CaptureMock is not just Python interception, it's also command-line and network traffic interception, and these parts particularly seem to fit very naturally to TextTest's model of running external processes and comparing files produced, and less so to nose and py.test which are more designed for arranging and asserting within a Python process.

I'd be happy to help troubleshoot why you couldn't install TextTest if you'd like, or take these tests and convert them to TextTest myself. If you have a Mac, the TextTest GUI is likely to be problematic as GTK isn't maintained there. But you can still run TextTest tests via the console interface, without needing GTK. On Linux or Windows the GUI should be no problem. On a Mac you can always create a Linux VM and run the GUI there.

Regards,
Geoff Bache

Reply
Revision history for this message
Marc Abramowitz (msabramo) wrote :

Understandable that you want to use TextTest.

Take a look at
https://code.launchpad.net/~msabramo/capturemock/tests_add_misc_tests/+merge/197564when
you get a chance. I tried to convert some of my py.test tests into
equivalent TextTest tests.

I'm not sure how to do the test that exercises recording while using the
requests module -- that adds a dependency on having the requests module
installed and you may not want to require that (or maybe you do, I don't
know). I could vendor "requests" into the CaptureMock tests but then that
would make the tests repo bigger, so you might not want that. Let me know
if you have a preferred approach for dealing with testing things that
require external modules...

Marc

On Mon, Dec 2, 2013 at 12:48 PM, Geoff Bache <email address hidden> wrote:

> Hi Marc,
>
> Thanks for all this, I'll try and review it in more detail later this week.
> Seems like you might have found a couple of bugs here as well, which is
> very useful!
>
> Fundamentally, there are already 138 fairly comprehensive acceptance tests
> for CaptureMock using TextTest (around 97% coverage), so simply adding some
> new ones that run via nose or py.test won't really help us very much. To
> actually be useful, we would need to translate the entire test suite
> really, which is probably only worth the effort if you (or some other
> non-TextTest user) are planning to do serious development and has
> insurmountable problems installing TextTest. In addition, CaptureMock is
> not just Python interception, it's also command-line and network traffic
> interception, and these parts particularly seem to fit very naturally to
> TextTest's model of running external processes and comparing files
> produced, and less so to nose and py.test which are more designed for
> arranging and asserting within a Python process.
>
> I'd be happy to help troubleshoot why you couldn't install TextTest if
> you'd like, or take these tests and convert them to TextTest myself. If you
> have a Mac, the TextTest GUI is likely to be problematic as GTK isn't
> maintained there. But you can still run TextTest tests via the console
> interface, without needing GTK. On Linux or Windows the GUI should be no
> problem. On a Mac you can always create a Linux VM and run the GUI there.
>
> Regards,
> Geoff Bache
> --
> https://code.launchpad.net/~msabramo/capturemock/tests/+merge/197297
> You are the owner of lp:~msabramo/capturemock/tests.
>

Reply

Unmerged revisions

4640. By Marc Abramowitz <email address hidden>

Add test of mocking sqlalchemy and pymssql

4639. By Marc Abramowitz <email address hidden>

Add setup.cfg so that you don't need to remember to run py.test with --assert=reinterp

4638. By Marc Abramowitz <email address hidden>

Add tests/capturemock/requests_get.mock

4637. By Marc Abramowitz <email address hidden>

Enhance tests/test_requests.py

4636. By Marc Abramowitz <email address hidden>

Add a test that demonstrates replaying stuff from urllib2, urllib, and httplib

4635. By Marc Abramowitz <email address hidden>

Add support for tox

4634. By Marc Abramowitz <email address hidden>

Add tests that work with nosetests/py.test

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file '.bzrignore'
--- .bzrignore 2011-03-24 19:54:45 +0000
+++ .bzrignore 2013-12-02 14:51:57 +0000
@@ -4,3 +4,4 @@
4dist4dist
5capturemock.exe5capturemock.exe
6capturemock.py6capturemock.py
7.tox
78
=== added file 'setup.cfg'
--- setup.cfg 1970-01-01 00:00:00 +0000
+++ setup.cfg 2013-12-02 14:51:57 +0000
@@ -0,0 +1,21 @@
1[pytest]
2norecursedirs = build docs/_build *.egg .tox *.venv
3addopts =
4 --verbose
5 --tb short
6 --capture no
7 # show extra test summary info as specified by chars (f)ailed, (E)error, (s)skipped, (x)failed, (X)passed.
8 -rfEsxX
9 --assert=reinterp
10 # --junitxml junit.xml
11 # --cov capturemock --cov-report xml --cov-report term-missing
12
13[nosetests]
14match=^test
15verbosity=2
16# nocapture=1
17# with-xunit=1
18# xunit-file=junit.xml
19# cover-package=capturemock
20# with-xcoverage=1
21# cover-erase=1
022
=== added directory 'tests'
=== added directory 'tests/capturemock'
=== added file 'tests/capturemock/replay_thing.mock'
--- tests/capturemock/replay_thing.mock 1970-01-01 00:00:00 +0000
+++ tests/capturemock/replay_thing.mock 2013-12-02 14:51:57 +0000
@@ -0,0 +1,19 @@
1<-PYT:import thing
2<-PYT:thing.__path__
3->RET:raise exceptions.AttributeError("'module' object has no attribute '__path__'")
4<-PYT:thing.Thing(id=1)
5->RET:Instance('Thing(object)', 'thing1')
6<-PYT:thing1.id
7->RET:1
8<-PYT:thing.Thing(id=2)
9->RET:Instance('Thing', 'thing2')
10<-PYT:thing2.id
11->RET:2
12<-PYT:thing.Thing(id=3)
13->RET:Instance('Thing', 'thing3')
14<-PYT:thing3.id
15->RET:3
16<-PYT:thing.Thing(id=4)
17->RET:Instance('Thing', 'thing4')
18<-PYT:thing4.id
19->RET:4
020
=== added file 'tests/capturemock/requests_get.mock'
--- tests/capturemock/requests_get.mock 1970-01-01 00:00:00 +0000
+++ tests/capturemock/requests_get.mock 2013-12-02 14:51:57 +0000
@@ -0,0 +1,9 @@
1<-PYT:import requests
2<-PYT:requests.get('http://www.google.com/')
3->RET:Instance('Response(object)', 'response1')
4<-PYT:response1.url
5->RET:'http://www.google.com/when-pigs-fly'
6<-PYT:response1.status_code
7->RET:702
8<-PYT:response1.text
9->RET:u'''This is a fake response'''
010
=== added file 'tests/capturemock/sqlalchemy.mock'
--- tests/capturemock/sqlalchemy.mock 1970-01-01 00:00:00 +0000
+++ tests/capturemock/sqlalchemy.mock 2013-12-02 14:51:57 +0000
@@ -0,0 +1,13 @@
1<-PYT:import sqlalchemy
2<-PYT:sqlalchemy.create_engine('mssql+pymssql://user:password@localhost/database')
3->RET:Instance('Engine(Connectable, Identified, object)', 'engine1')
4<-PYT:engine1.connect()
5->RET:Instance('Connection(Connectable, object)', 'connection1')
6<-PYT:connection1.execute('SELECT @@VERSION')
7->RET:Instance('ResultProxy(object)', 'resultproxy_select___version')
8<-PYT:resultproxy_select___version.scalar()
9->RET:u'''Microsoft SQL Server 2012 - 11.0.2100.60 (X64)
10 Feb 10 2012 19:39:15
11 Copyright (c) Microsoft Corporation
12 Developer Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1)
13'''
014
=== added file 'tests/capturemock/urllib2.mock'
--- tests/capturemock/urllib2.mock 1970-01-01 00:00:00 +0000
+++ tests/capturemock/urllib2.mock 2013-12-02 14:51:57 +0000
@@ -0,0 +1,13 @@
1<-PYT:import urllib2
2<-PYT:urllib2.urlopen('http://www.google.com/')
3->RET:Instance('addinfourl(addbase)', 'addinfourl1')
4<-PYT:addinfourl1.code
5->RET:702
6<-PYT:addinfourl1.url
7->RET:'http://www.google.com/when-pigs-fly'
8<-PYT:addinfourl1.headers
9->RET:Instance('HTTPMessage(mimetools.Message)', 'httpmessage1')
10<-PYT:httpmessage1.__getitem__('X-Fake-Message')
11->RET:'This is a fake response header'
12<-PYT:addinfourl1.read()
13->RET:'''Simulating a fake error'''
014
=== added file 'tests/cars.py'
--- tests/cars.py 1970-01-01 00:00:00 +0000
+++ tests/cars.py 2013-12-02 14:51:57 +0000
@@ -0,0 +1,17 @@
1class Wheel(object):
2 size = 15
3
4 def __init__(self, id):
5 self.id = id
6
7class Car(object):
8 wheels = [
9 Wheel(id=1),
10 Wheel(id=2),
11 Wheel(id=3),
12 Wheel(id=4),
13 ]
14
15class CarFactory(object):
16 def make_car(self):
17 return Car()
018
=== added file 'tests/test_cars.py'
--- tests/test_cars.py 1970-01-01 00:00:00 +0000
+++ tests/test_cars.py 2013-12-02 14:51:57 +0000
@@ -0,0 +1,16 @@
1from capturemock import capturemock, RECORD, REPLAY
2
3@capturemock('cars', mode=RECORD)
4def test_cars():
5 from cars import CarFactory, Car, Wheel
6
7 car_factory = CarFactory()
8 assert isinstance(car_factory, CarFactory)
9
10 car = car_factory.make_car()
11 assert isinstance(car, Car)
12
13 for wheel in car.wheels:
14 assert isinstance(wheel, Wheel)
15 assert wheel.size == 15
16 print('wheel.id = %r' % wheel.id)
017
=== added file 'tests/test_requests.py'
--- tests/test_requests.py 1970-01-01 00:00:00 +0000
+++ tests/test_requests.py 2013-12-02 14:51:57 +0000
@@ -0,0 +1,24 @@
1import pytest
2
3from capturemock import capturemock, RECORD, REPLAY
4
5
6# @pytest.mark.xfail
7# # This test fails because capturemock chokes when trying to get response
8# # headers
9# @capturemock("requests", mode=RECORD)
10# def test_requests_headers():
11# import requests
12#
13# resp = requests.get("http://www.google.com/")
14# resp.headers
15
16
17@capturemock("requests", mode=REPLAY)
18def test_requests_get():
19 import requests
20
21 resp = requests.get("http://www.google.com/")
22 assert resp.url == 'http://www.google.com/when-pigs-fly'
23 assert resp.status_code == 702
24 assert resp.text == 'This is a fake response'
025
=== added file 'tests/test_sqlalchemy.py'
--- tests/test_sqlalchemy.py 1970-01-01 00:00:00 +0000
+++ tests/test_sqlalchemy.py 2013-12-02 14:51:57 +0000
@@ -0,0 +1,18 @@
1from capturemock import capturemock, RECORD, REPLAY
2
3
4@capturemock('sqlalchemy', mode=REPLAY)
5def test_sqlalchemy():
6 import sqlalchemy
7
8 db_uri = 'mssql+pymssql://user:password@localhost/database'
9 engine = sqlalchemy.create_engine(db_uri)
10 conn = engine.connect()
11 result = conn.execute('SELECT @@VERSION').scalar()
12 expected_output = \
13 'Microsoft SQL Server 2012 - 11.0.2100.60 (X64) \n' \
14 + '\tFeb 10 2012 19:39:15 \n' \
15 + '\tCopyright (c) Microsoft Corporation\n' \
16 + '\tDeveloper Edition (64-bit) on Windows NT 6.1 <X64> ' \
17 + '(Build 7601: Service Pack 1)\n'
18 assert result == expected_output
019
=== added file 'tests/test_thing.py'
--- tests/test_thing.py 1970-01-01 00:00:00 +0000
+++ tests/test_thing.py 2013-12-02 14:51:57 +0000
@@ -0,0 +1,62 @@
1import contextlib
2import os
3import sys
4from StringIO import StringIO
5
6import pytest
7
8from capturemock import capturemock, RECORD, REPLAY
9
10
11@contextlib.contextmanager
12def capture_stdout():
13 sys.stdout = StringIO()
14 yield sys.stdout
15 sys.stdout = sys.__stdout__
16
17
18@capturemock('thing', mode=RECORD)
19def record_thing():
20 from thing import Thing
21
22 with capture_stdout() as s:
23 for id in range(1, 5):
24 thing = Thing(id=id)
25 assert thing.id == id
26
27 out = s.getvalue()
28 assert out == 'Creating Thing with id=1\n' \
29 'Creating Thing with id=2\n' \
30 'Creating Thing with id=3\n' \
31 'Creating Thing with id=4\n'
32
33
34def test_thing_recorded():
35 record_thing()
36
37 record_file = os.path.join(
38 os.path.dirname(__file__),
39 'capturemock', 'record_thing.mock')
40
41 with open(record_file) as f:
42 content = f.read()
43 assert "<-PYT:import thing" in content
44
45 for id in range(1, 5):
46 assert "<-PYT:thing.Thing(id=%d)" % id in content
47 assert "'thing%d')" % id in content
48 assert "<-PYT:thing%d.id" % id in content
49 assert "->RET:%d" % id in content
50
51
52@capturemock('thing', mode=REPLAY)
53def test_replay_thing():
54 from thing import Thing
55
56 with capture_stdout() as s:
57 for id in range(1, 5):
58 thing = Thing(id=id)
59 assert thing.id == id
60
61 out = s.getvalue()
62 assert out == ''
063
=== added file 'tests/test_urllib2.py'
--- tests/test_urllib2.py 1970-01-01 00:00:00 +0000
+++ tests/test_urllib2.py 2013-12-02 14:51:57 +0000
@@ -0,0 +1,19 @@
1from capturemock import capturemock, REPLAY
2
3@capturemock(["urllib2", "urllib", "httplib"], mode=REPLAY)
4def test_urllib2():
5 # Test capturemock replay by replaying from a file that simulates Google
6 # responding with a fake error.
7 #
8 # Note that we have to capture urllib2 as well as urllib and httplib.
9 # This is because the response from urllib2.urlopen is in the urllib module
10 # and the `headers` attribute of the response is of a class defined in the
11 # httplib module.
12
13 import urllib2
14
15 resp = urllib2.urlopen("http://www.google.com/")
16 assert resp.code == 702
17 assert resp.url == "http://www.google.com/when-pigs-fly"
18 assert resp.headers['X-Fake-Message'] == 'This is a fake response header'
19 assert resp.read() == 'Simulating a fake error'
020
=== added file 'tests/thing.py'
--- tests/thing.py 1970-01-01 00:00:00 +0000
+++ tests/thing.py 2013-12-02 14:51:57 +0000
@@ -0,0 +1,4 @@
1class Thing(object):
2 def __init__(self, id):
3 print("Creating Thing with id=%d" % id)
4 self.id = id
05
=== added file 'tox.ini'
--- tox.ini 1970-01-01 00:00:00 +0000
+++ tox.ini 2013-12-02 14:51:57 +0000
@@ -0,0 +1,12 @@
1# Tox (http://tox.testrun.org/) is a tool for running tests
2# in multiple virtualenvs. This configuration file will run the
3# test suite on all supported python versions. To use it, "pip install tox"
4# and then run "tox" from this directory.
5
6[tox]
7envlist = py26, py27
8
9[testenv]
10commands = py.test -v --assert=reinterp tests/
11deps =
12 pytest

Subscribers

People subscribed via source and target branches