Merge lp:~allenap/maas/database-locks-rerevisited into lp:~maas-committers/maas/trunk

Proposed by Gavin Panella
Status: Merged
Approved by: Gavin Panella
Approved revision: no longer in the source branch.
Merged at revision: 4196
Proposed branch: lp:~allenap/maas/database-locks-rerevisited
Merge into: lp:~maas-committers/maas/trunk
Diff against target: 570 lines (+294/-39)
9 files modified
src/maasserver/locks.py (+6/-3)
src/maasserver/models/signals/tests/test_power.py (+5/-2)
src/maasserver/rpc/regionservice.py (+6/-2)
src/maasserver/start_up.py (+5/-2)
src/maasserver/utils/dblocks.py (+16/-5)
src/maasserver/utils/orm.py (+53/-6)
src/maasserver/utils/tests/test_dblocks.py (+71/-3)
src/maasserver/utils/tests/test_orm.py (+86/-16)
src/maastesting/doubles.py (+46/-0)
To merge this branch: bzr merge lp:~allenap/maas/database-locks-rerevisited
Reviewer Review Type Date Requested Status
Blake Rouse (community) Approve
Review via email: mp+266446@code.launchpad.net

Commit message

Use non-transactional advisory locks when starting the region and setting up the event-loop.

Previously locks were obtained after the transaction had begun. This meant that, once the lock was acquired, the transaction may be working with stale data, resulting in serialization failures at best.

To post a comment you must log in.
Revision history for this message
Blake Rouse (blake-rouse) wrote :

This looks good. This seems like it relates to a bug. Do you know of a specific bug?

Maybe this bug: https://bugs.launchpad.net/maas/+bug/1458895

review: Approve
Revision history for this message
Gavin Panella (allenap) wrote :

> This looks good. This seems like it relates to a bug. Do you know of a
> specific bug?
>
> Maybe this bug: https://bugs.launchpad.net/maas/+bug/1458895

That bug looks like a likely candidate; I probably read it and later figured out the fix without remembering the original. Thanks for digging that out, and for the review.

Revision history for this message
Gavin Panella (allenap) wrote :

FTR, I QA'ed this at home first, and it seems to work as intended.

Revision history for this message
MAAS Lander (maas-lander) wrote :
Download full text (79.6 KiB)

The attempt to merge lp:~allenap/maas/database-locks-rerevisited into lp:maas failed. Below is the output from the failed tests.

Ign http://security.ubuntu.com trusty-security InRelease
Get:1 http://security.ubuntu.com trusty-security Release.gpg [933 B]
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Get:2 http://security.ubuntu.com trusty-security Release [63.5 kB]
Ign http://nova.clouds.archive.ubuntu.com trusty-updates InRelease
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Get:3 http://nova.clouds.archive.ubuntu.com trusty-updates Release.gpg [933 B]
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Get:4 http://nova.clouds.archive.ubuntu.com trusty-updates Release [63.5 kB]
Get:5 http://security.ubuntu.com trusty-security/main Sources [90.7 kB]
Get:6 http://security.ubuntu.com trusty-security/universe Sources [28.5 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Get:7 http://security.ubuntu.com trusty-security/main amd64 Packages [326 kB]
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Get:8 http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources [227 kB]
Get:9 http://security.ubuntu.com trusty-security/universe amd64 Packages [112 kB]
Get:10 http://security.ubuntu.com trusty-security/main Translation-en [178 kB]
Get:11 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources [130 kB]
Get:12 http://security.ubuntu.com trusty-security/universe Translation-en [65.6 kB]
Get:13 http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages [598 kB]
Get:14 http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages [301 kB]
Get:15 http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en [288 kB]
Get:16 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en [160 kB]
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Fetched 2,635 kB in 3s (740 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
     --no-install-recommends install apache2 authbind bind9 bind9utils build-essential bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm pep8 phantomjs postgresql pyflakes python-apt python-bson python-bzrlib python-convoy python-coverage python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-extras python-fixtures python-flake8 python-formencode python-hivex python-httplib2 python-jinja2 python-jsonschema python-lockfile python-lxml pytho...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'src/maasserver/locks.py'
--- src/maasserver/locks.py 2015-08-06 13:04:24 +0000
+++ src/maasserver/locks.py 2015-08-14 16:45:29 +0000
@@ -18,17 +18,20 @@
18 "startup",18 "startup",
19]19]
2020
21from maasserver.utils.dblocks import DatabaseXactLock21from maasserver.utils.dblocks import (
22 DatabaseLock,
23 DatabaseXactLock,
24)
2225
23# Lock around starting-up a MAAS region.26# Lock around starting-up a MAAS region.
24startup = DatabaseXactLock(1)27startup = DatabaseLock(1)
2528
26# Lock around performing critical security-related operations, like29# Lock around performing critical security-related operations, like
27# generating or signing certificates.30# generating or signing certificates.
28security = DatabaseXactLock(2)31security = DatabaseXactLock(2)
2932
30# Lock used when starting up the event-loop.33# Lock used when starting up the event-loop.
31eventloop = DatabaseXactLock(3)34eventloop = DatabaseLock(3)
3235
33# Lock used to only allow one instance of importing boot images to occur36# Lock used to only allow one instance of importing boot images to occur
34# at a time.37# at a time.
3538
=== modified file 'src/maasserver/models/signals/tests/test_power.py'
--- src/maasserver/models/signals/tests/test_power.py 2015-06-10 11:24:44 +0000
+++ src/maasserver/models/signals/tests/test_power.py 2015-08-14 16:45:29 +0000
@@ -32,7 +32,10 @@
32 MAASServerTestCase,32 MAASServerTestCase,
33 MAASTransactionServerTestCase,33 MAASTransactionServerTestCase,
34)34)
35from maasserver.utils.orm import post_commit_hooks35from maasserver.utils.orm import (
36 post_commit_hooks,
37 transactional,
38)
36from maastesting.matchers import (39from maastesting.matchers import (
37 MockCalledOnceWith,40 MockCalledOnceWith,
38 MockNotCalled,41 MockNotCalled,
@@ -139,7 +142,7 @@
139142
140 def delete_node_then_get_client(uuid):143 def delete_node_then_get_client(uuid):
141 from maasserver.rpc import getClientFor144 from maasserver.rpc import getClientFor
142 d = deferToThread(node.delete) # Auto-commit outside txn.145 d = deferToThread(transactional(node.delete))
143 d.addCallback(lambda _: getClientFor(uuid))146 d.addCallback(lambda _: getClientFor(uuid))
144 return d147 return d
145148
146149
=== modified file 'src/maasserver/rpc/regionservice.py'
--- src/maasserver/rpc/regionservice.py 2015-06-16 21:10:26 +0000
+++ src/maasserver/rpc/regionservice.py 2015-08-14 16:45:29 +0000
@@ -55,7 +55,10 @@
55 make_validation_error_message,55 make_validation_error_message,
56 synchronised,56 synchronised,
57)57)
58from maasserver.utils.orm import transactional58from maasserver.utils.orm import (
59 transactional,
60 with_connection,
61)
59from netaddr import IPAddress62from netaddr import IPAddress
60from provisioningserver.rpc import (63from provisioningserver.rpc import (
61 cluster,64 cluster,
@@ -757,8 +760,9 @@
757760
758 @synchronous761 @synchronous
759 @synchronised(lock)762 @synchronised(lock)
760 @transactional763 @with_connection # Needed by the following lock.
761 @synchronised(locks.eventloop)764 @synchronised(locks.eventloop)
765 @transactional
762 def prepare(self):766 def prepare(self):
763 """Ensure that the ``eventloops`` table exists.767 """Ensure that the ``eventloops`` table exists.
764768
765769
=== modified file 'src/maasserver/start_up.py'
--- src/maasserver/start_up.py 2015-07-30 23:44:59 +0000
+++ src/maasserver/start_up.py 2015-08-14 16:45:29 +0000
@@ -45,6 +45,7 @@
45 get_psycopg2_exception,45 get_psycopg2_exception,
46 post_commit_do,46 post_commit_do,
47 transactional,47 transactional,
48 with_connection,
48)49)
49from provisioningserver.logger import get_maas_logger50from provisioningserver.logger import get_maas_logger
50from provisioningserver.upgrade_cluster import create_gnupg_home51from provisioningserver.upgrade_cluster import create_gnupg_home
@@ -114,8 +115,9 @@
114 break115 break
115116
116117
118@with_connection # Needed by the following lock.
119@synchronised(locks.startup)
117@transactional120@transactional
118@synchronised(locks.startup)
119def start_import_on_upgrade():121def start_import_on_upgrade():
120 """Starts importing `BootResource`s on upgrade from MAAS where the boot122 """Starts importing `BootResource`s on upgrade from MAAS where the boot
121 images where only stored on the clusters."""123 images where only stored on the clusters."""
@@ -168,8 +170,9 @@
168 import_resources()170 import_resources()
169171
170172
173@with_connection # Needed by the following lock.
174@synchronised(locks.startup)
171@transactional175@transactional
172@synchronised(locks.startup)
173def inner_start_up():176def inner_start_up():
174 """Startup jobs that must run serialized w.r.t. other starting servers."""177 """Startup jobs that must run serialized w.r.t. other starting servers."""
175 # Register our MAC data type with psycopg.178 # Register our MAC data type with psycopg.
176179
=== modified file 'src/maasserver/utils/dblocks.py'
--- src/maasserver/utils/dblocks.py 2014-10-15 10:56:09 +0000
+++ src/maasserver/utils/dblocks.py 2015-08-14 16:45:29 +0000
@@ -15,7 +15,9 @@
15__all__ = [15__all__ = [
16 "DatabaseLock",16 "DatabaseLock",
17 "DatabaseXactLock",17 "DatabaseXactLock",
18
18 "DatabaseLockAttemptOutsideTransaction",19 "DatabaseLockAttemptOutsideTransaction",
20 "DatabaseLockAttemptWithoutConnection",
19 "DatabaseLockNotHeld",21 "DatabaseLockNotHeld",
20]22]
2123
@@ -29,12 +31,21 @@
29classid = 2012011631classid = 20120116
3032
3133
34class DatabaseLockAttemptWithoutConnection(Exception):
35 """A locking attempt was made without a preexisting connection.
36
37 :class:`DatabaseLock` should only be used with a preexisting connection.
38 While this restriction is not absolutely necessary, it's here to ensure
39 that users of :class:`DatabaseLock` take care with the lifecycle of their
40 database connection: a connection that is inadvertently closed (by Django,
41 by MAAS, by anything) will release all locks too.
42 """
43
44
32class DatabaseLockAttemptOutsideTransaction(Exception):45class DatabaseLockAttemptOutsideTransaction(Exception):
33 """A locking attempt was made outside of a transaction.46 """A locking attempt was made outside of a transaction.
3447
35 :class:`DatabaseLock` should only be used within a transaction.48 :class:`DatabaseXactLock` should only be used within a transaction.
36 Django agressively closes connections outside of atomic blocks to
37 the extent that session-level locks are rendered unreliable at best.
38 """49 """
3950
4051
@@ -125,8 +136,8 @@
125 __slots__ = ()136 __slots__ = ()
126137
127 def __enter__(self):138 def __enter__(self):
128 if not in_transaction():139 if connection.connection is None:
129 raise DatabaseLockAttemptOutsideTransaction(self)140 raise DatabaseLockAttemptWithoutConnection(self)
130 with closing(connection.cursor()) as cursor:141 with closing(connection.cursor()) as cursor:
131 cursor.execute("SELECT pg_advisory_lock(%s, %s)", self)142 cursor.execute("SELECT pg_advisory_lock(%s, %s)", self)
132143
133144
=== modified file 'src/maasserver/utils/orm.py'
--- src/maasserver/utils/orm.py 2015-08-11 16:49:56 +0000
+++ src/maasserver/utils/orm.py 2015-08-14 16:45:29 +0000
@@ -30,6 +30,7 @@
30 'savepoint',30 'savepoint',
31 'transactional',31 'transactional',
32 'validate_in_transaction',32 'validate_in_transaction',
33 'with_connection',
33 ]34 ]
3435
35from contextlib import contextmanager36from contextlib import contextmanager
@@ -44,7 +45,6 @@
4445
45from django.core.exceptions import MultipleObjectsReturned46from django.core.exceptions import MultipleObjectsReturned
46from django.db import (47from django.db import (
47 close_old_connections,
48 connection,48 connection,
49 transaction,49 transaction,
50)50)
@@ -398,6 +398,44 @@
398 raise AssertionError("Not callable: %r" % (func,))398 raise AssertionError("Not callable: %r" % (func,))
399399
400400
401@contextmanager
402def connected():
403 """Context manager that ensures we're connected to the database.
404
405 If there is not yet a connection to the database, this will connect on
406 entry and disconnect on exit. Preexisting connections will be left alone.
407 """
408 if connection.connection is None:
409 connection.ensure_connection()
410 try:
411 yield
412 finally:
413 connection.close()
414 else:
415 yield
416
417
418def with_connection(func):
419 """Ensure that we're connected to the database before calling `func`.
420
421 If there is not yet a connection to the database, this will connect before
422 calling the decorated function, and then it will disconnect when done.
423 Preexisting connections will be left alone.
424
425 This can be important when using non-transactional advisory locks.
426 """
427 @wraps(func)
428 def call_with_connection(*args, **kwargs):
429 with connected():
430 return func(*args, **kwargs)
431
432 # For convenience, when introspecting for example, expose the original
433 # function on the function we're returning.
434 call_with_connection.func = func
435
436 return call_with_connection
437
438
401def transactional(func):439def transactional(func):
402 """Decorator that wraps calls to `func` in a Django-managed transaction.440 """Decorator that wraps calls to `func` in a Django-managed transaction.
403441
@@ -420,11 +458,20 @@
420 return func_within_txn(*args, **kwargs)458 return func_within_txn(*args, **kwargs)
421 else:459 else:
422 # Use the retry-capable function, firing post-transaction hooks.460 # Use the retry-capable function, firing post-transaction hooks.
423 try:461 #
424 with post_commit_hooks:462 # If there is not yet a connection to the database, connect before
425 return func_outside_txn(*args, **kwargs)463 # calling the decorated function, then disconnect when done. This
426 finally:464 # can be important when using non-transactional advisory locks
427 close_old_connections()465 # that may be held before, during, and/or after this transactional
466 # block.
467 #
468 # Previously, close_old_connections() was used here, which would
469 # close connections without realising that they were still in use
470 # for non-transactional advisory locking. This had the effect of
471 # releasing all locks prematurely: not good.
472 #
473 with connected(), post_commit_hooks:
474 return func_outside_txn(*args, **kwargs)
428475
429 # For convenience, when introspecting for example, expose the original476 # For convenience, when introspecting for example, expose the original
430 # function on the function we're returning.477 # function on the function we're returning.
431478
=== modified file 'src/maasserver/utils/tests/test_dblocks.py'
--- src/maasserver/utils/tests/test_dblocks.py 2015-05-07 18:14:38 +0000
+++ src/maasserver/utils/tests/test_dblocks.py 2015-08-14 16:45:29 +0000
@@ -15,6 +15,7 @@
15__all__ = []15__all__ = []
1616
17from contextlib import closing17from contextlib import closing
18import sys
1819
19from django.db import (20from django.db import (
20 connection,21 connection,
@@ -32,8 +33,18 @@
32 return {result[0] for result in cursor.fetchall()}33 return {result[0] for result in cursor.fetchall()}
3334
3435
36@transaction.atomic
37def divide_by_zero():
38 0 / 0 # In a transaction.
39
40
35class TestDatabaseLock(MAASTestCase):41class TestDatabaseLock(MAASTestCase):
3642
43 def tearDown(self):
44 super(TestDatabaseLock, self).tearDown()
45 with closing(connection.cursor()) as cursor:
46 cursor.execute("SELECT pg_advisory_unlock_all()")
47
37 def test_create_lock(self):48 def test_create_lock(self):
38 objid = self.getUniqueInteger()49 objid = self.getUniqueInteger()
39 lock = dblocks.DatabaseLock(objid)50 lock = dblocks.DatabaseLock(objid)
@@ -69,12 +80,69 @@
69 self.assertTrue(lock.is_locked())80 self.assertTrue(lock.is_locked())
70 self.assertFalse(lock.is_locked())81 self.assertFalse(lock.is_locked())
7182
72 def test_obtaining_lock_fails_when_outside_of_transaction(self):83 def test_lock_remains_held_when_committing_transaction(self):
84 objid = self.getUniqueInteger()
85 lock = dblocks.DatabaseLock(objid)
86 txn = transaction.atomic()
87
88 self.assertFalse(lock.is_locked())
89 txn.__enter__()
90 self.assertFalse(lock.is_locked())
91 lock.__enter__()
92 self.assertTrue(lock.is_locked())
93 txn.__exit__(None, None, None)
94 self.assertTrue(lock.is_locked())
95 lock.__exit__(None, None, None)
96 self.assertFalse(lock.is_locked())
97
98 def test_lock_remains_held_when_aborting_transaction(self):
99 objid = self.getUniqueInteger()
100 lock = dblocks.DatabaseLock(objid)
101 txn = transaction.atomic()
102
103 self.assertFalse(lock.is_locked())
104 txn.__enter__()
105 self.assertFalse(lock.is_locked())
106 lock.__enter__()
107 self.assertTrue(lock.is_locked())
108
109 self.assertRaises(ZeroDivisionError, divide_by_zero)
110 exc_info = sys.exc_info()
111
112 txn.__exit__(*exc_info)
113 self.assertTrue(lock.is_locked())
114 lock.__exit__(None, None, None)
115 self.assertFalse(lock.is_locked())
116
117 def test_lock_is_held_around_transaction(self):
118 objid = self.getUniqueInteger()
119 lock = dblocks.DatabaseLock(objid)
120
121 self.assertFalse(lock.is_locked())
122 with lock:
123 self.assertTrue(lock.is_locked())
124 with transaction.atomic():
125 self.assertTrue(lock.is_locked())
126 self.assertTrue(lock.is_locked())
127 self.assertFalse(lock.is_locked())
128
129 def test_lock_is_held_around_breaking_transaction(self):
130 objid = self.getUniqueInteger()
131 lock = dblocks.DatabaseLock(objid)
132
133 self.assertFalse(lock.is_locked())
134 with lock:
135 self.assertTrue(lock.is_locked())
136 self.assertRaises(ZeroDivisionError, divide_by_zero)
137 self.assertTrue(lock.is_locked())
138 self.assertFalse(lock.is_locked())
139
140 def test_lock_requires_preexisting_connection(self):
141 connection.close()
73 objid = self.getUniqueInteger()142 objid = self.getUniqueInteger()
74 lock = dblocks.DatabaseLock(objid)143 lock = dblocks.DatabaseLock(objid)
75 self.assertRaises(144 self.assertRaises(
76 dblocks.DatabaseLockAttemptOutsideTransaction,145 dblocks.DatabaseLockAttemptWithoutConnection, lock.__enter__)
77 lock.__enter__)
78146
79 def test_releasing_lock_fails_when_lock_not_held(self):147 def test_releasing_lock_fails_when_lock_not_held(self):
80 objid = self.getUniqueInteger()148 objid = self.getUniqueInteger()
81149
=== modified file 'src/maasserver/utils/tests/test_orm.py'
--- src/maasserver/utils/tests/test_orm.py 2015-08-11 17:03:04 +0000
+++ src/maasserver/utils/tests/test_orm.py 2015-08-14 16:45:29 +0000
@@ -52,6 +52,7 @@
52 validate_in_transaction,52 validate_in_transaction,
53)53)
54from maastesting.djangotestcase import DjangoTransactionTestCase54from maastesting.djangotestcase import DjangoTransactionTestCase
55from maastesting.doubles import StubContext
55from maastesting.factory import factory56from maastesting.factory import factory
56from maastesting.matchers import (57from maastesting.matchers import (
57 HasLength,58 HasLength,
@@ -561,6 +562,55 @@
561 self.assertRaises(AssertionError, post_commit_do, sentinel.hook)562 self.assertRaises(AssertionError, post_commit_do, sentinel.hook)
562563
563564
565class TestConnected(DjangoTransactionTestCase):
566 """Tests for the `orm.connected` context manager."""
567
568 def test__ensures_connection(self):
569 with orm.connected():
570 self.assertThat(connection.connection, Not(Is(None)))
571
572 def test__opens_and_closes_connection_when_no_preexisting_connection(self):
573 connection.close()
574
575 self.assertThat(connection.connection, Is(None))
576 with orm.connected():
577 self.assertThat(connection.connection, Not(Is(None)))
578 self.assertThat(connection.connection, Is(None))
579
580 def test__leaves_preexisting_connections_alone(self):
581 connection.ensure_connection()
582 preexisting_connection = connection.connection
583
584 self.assertThat(connection.connection, Not(Is(None)))
585 with orm.connected():
586 self.assertThat(connection.connection, Is(preexisting_connection))
587 self.assertThat(connection.connection, Is(preexisting_connection))
588
589
590class TestWithConnection(DjangoTransactionTestCase):
591 """Tests for the `orm.with_connection` decorator."""
592
593 def test__exposes_original_function(self):
594 function = Mock(__name__=self.getUniqueString())
595 self.assertThat(orm.with_connection(function).func, Is(function))
596
597 def test__ensures_function_is_called_within_connected_context(self):
598 context = self.patch(orm, "connected").return_value = StubContext()
599
600 @orm.with_connection
601 def function(arg, kwarg):
602 self.assertThat(arg, Is(sentinel.arg))
603 self.assertThat(kwarg, Is(sentinel.kwarg))
604 self.assertTrue(context.active)
605 return sentinel.result
606
607 self.assertFalse(context.active)
608 self.assertThat(
609 function(sentinel.arg, kwarg=sentinel.kwarg),
610 Is(sentinel.result))
611 self.assertFalse(context.active)
612
613
564class TestTransactional(DjangoTransactionTestCase):614class TestTransactional(DjangoTransactionTestCase):
565615
566 def test__exposes_original_function(self):616 def test__exposes_original_function(self):
@@ -568,18 +618,19 @@
568 self.assertThat(orm.transactional(function).func, Is(function))618 self.assertThat(orm.transactional(function).func, Is(function))
569619
570 def test__calls_function_within_transaction_then_closes_connections(self):620 def test__calls_function_within_transaction_then_closes_connections(self):
571 close_old_connections = self.patch(orm, "close_old_connections")621 # Close the database connection to begin with.
622 connection.close()
572623
573 # No transaction has been entered (what Django calls an atomic624 # No transaction has been entered (what Django calls an atomic block),
574 # block), and old connections have not been closed.625 # and the connection has not yet been established.
575 self.assertFalse(connection.in_atomic_block)626 self.assertFalse(connection.in_atomic_block)
576 self.assertThat(close_old_connections, MockNotCalled())627 self.expectThat(connection.connection, Is(None))
577628
578 def check_inner(*args, **kwargs):629 def check_inner(*args, **kwargs):
579 # In here, the transaction (`atomic`) has been started but630 # In here, the transaction (`atomic`) has been started but is not
580 # is not over, and old connections have not yet been closed.631 # over, and the connection to the database is open.
581 self.assertTrue(connection.in_atomic_block)632 self.assertTrue(connection.in_atomic_block)
582 self.assertThat(close_old_connections, MockNotCalled())633 self.expectThat(connection.connection, Not(Is(None)))
583634
584 function = Mock()635 function = Mock()
585 function.__name__ = self.getUniqueString()636 function.__name__ = self.getUniqueString()
@@ -595,27 +646,49 @@
595 sentinel.arg, kwarg=sentinel.kwarg))646 sentinel.arg, kwarg=sentinel.kwarg))
596647
597 # After the decorated function has returned the transaction has648 # After the decorated function has returned the transaction has
598 # been exited, and old connections have been closed.649 # been exited, and the connection has been closed.
599 self.assertFalse(connection.in_atomic_block)650 self.assertFalse(connection.in_atomic_block)
600 self.assertThat(close_old_connections, MockCalledOnceWith())651 self.expectThat(connection.connection, Is(None))
652
653 def test__leaves_preexisting_connections_open(self):
654 # Ensure there's a database connection to begin with.
655 connection.ensure_connection()
656
657 # No transaction has been entered (what Django calls an atomic block),
658 # but the connection has been established.
659 self.assertFalse(connection.in_atomic_block)
660 self.expectThat(connection.connection, Not(Is(None)))
661
662 # Call a function via the `transactional` decorator.
663 decorated_function = orm.transactional(lambda: None)
664 decorated_function()
665
666 # After the decorated function has returned the transaction has
667 # been exited, but the preexisting connection remains open.
668 self.assertFalse(connection.in_atomic_block)
669 self.expectThat(connection.connection, Not(Is(None)))
601670
602 def test__closes_connections_only_when_leaving_atomic_block(self):671 def test__closes_connections_only_when_leaving_atomic_block(self):
603 close_old_connections = self.patch(orm, "close_old_connections")672 # Close the database connection to begin with.
673 connection.close()
674 self.expectThat(connection.connection, Is(None))
604675
605 @orm.transactional676 @orm.transactional
606 def inner():677 def inner():
607 # We're inside a `transactional` context here.678 # We're inside a `transactional` context here.
679 self.expectThat(connection.connection, Not(Is(None)))
608 return "inner"680 return "inner"
609681
610 @orm.transactional682 @orm.transactional
611 def outer():683 def outer():
612 # We're inside a `transactional` context here too.684 # We're inside a `transactional` context here too.
685 self.expectThat(connection.connection, Not(Is(None)))
613 # Call `inner`, thus nesting `transactional` contexts.686 # Call `inner`, thus nesting `transactional` contexts.
614 return "outer > " + inner()687 return "outer > " + inner()
615688
616 self.assertEqual("outer > inner", outer())689 self.assertEqual("outer > inner", outer())
617 # Old connections have been closed only once.690 # The connection has been closed.
618 self.assertThat(close_old_connections, MockCalledOnceWith())691 self.expectThat(connection.connection, Is(None))
619692
620 def test__fires_post_commit_hooks_when_done(self):693 def test__fires_post_commit_hooks_when_done(self):
621 fire = self.patch(orm.post_commit_hooks, "fire")694 fire = self.patch(orm.post_commit_hooks, "fire")
@@ -652,9 +725,6 @@
652class TestTransactionalRetries(SerializationFailureTestCase):725class TestTransactionalRetries(SerializationFailureTestCase):
653726
654 def test__retries_upon_serialization_failures(self):727 def test__retries_upon_serialization_failures(self):
655 # No-op close_old_connections().
656 self.patch(orm, "close_old_connections")
657
658 function = Mock()728 function = Mock()
659 function.__name__ = self.getUniqueString()729 function.__name__ = self.getUniqueString()
660 function.side_effect = self.cause_serialization_failure730 function.side_effect = self.cause_serialization_failure
661731
=== added file 'src/maastesting/doubles.py'
--- src/maastesting/doubles.py 1970-01-01 00:00:00 +0000
+++ src/maastesting/doubles.py 2015-08-14 16:45:29 +0000
@@ -0,0 +1,46 @@
1# Copyright 2012-2015 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Miscellaneous test doubles.
5
6See http://www.martinfowler.com/bliki/TestDouble.html for the nomenclature
7used.
8"""
9
10from __future__ import (
11 absolute_import,
12 print_function,
13 unicode_literals,
14 )
15
16str = None
17
18__metaclass__ = type
19__all__ = [
20 "StubContext",
21]
22
23
24class StubContext:
25 """A stub context manager.
26
27 :ivar entered: A boolean indicating if the context has been entered.
28 :ivar exited: A boolean indicating if the context has been exited.
29 :ivar active: A boolean indicating if the context is currently active
30 (i.e. it has been entered but not exited).
31 :ivar exc_info: The ``exc_info`` tuple passed into ``__exit__``.
32 """
33
34 entered = False
35 exited = False
36
37 @property
38 def active(self):
39 return self.entered and not self.exited
40
41 def __enter__(self):
42 self.entered = True
43
44 def __exit__(self, *exc_info):
45 self.exc_info = exc_info
46 self.exited = True