Merge lp:~abompard/mailman/sqlalchemy into lp:~raj-abhilash1/mailman/sqlalchemy

Proposed by Aurélien Bompard on 2014-10-29
Status: Needs review
Proposed branch: lp:~abompard/mailman/sqlalchemy
Merge into: lp:~raj-abhilash1/mailman/sqlalchemy
Diff against target: 1466 lines (+339/-301)
28 files modified
MANIFEST.in (+1/-4)
src/mailman/app/docs/moderator.rst (+29/-40)
src/mailman/commands/docs/conf.rst (+1/-2)
src/mailman/commands/docs/withlist.rst (+2/-2)
src/mailman/config/alembic.cfg (+20/-0)
src/mailman/config/config.py (+0/-2)
src/mailman/config/schema.cfg (+3/-9)
src/mailman/core/docs/runner.rst (+2/-0)
src/mailman/core/logging.py (+28/-21)
src/mailman/database/alembic/__init__.py (+4/-4)
src/mailman/database/alembic/env.py (+0/-5)
src/mailman/database/alembic/versions/51b7f92bd06c_initial.py (+46/-14)
src/mailman/database/base.py (+5/-1)
src/mailman/database/factory.py (+43/-48)
src/mailman/database/tests/test_factory.py (+80/-82)
src/mailman/handlers/docs/owner-recips.rst (+2/-2)
src/mailman/model/docs/autorespond.rst (+11/-11)
src/mailman/model/docs/mailinglist.rst (+6/-6)
src/mailman/model/docs/requests.rst (+10/-10)
src/mailman/model/domain.py (+4/-3)
src/mailman/model/listmanager.py (+2/-1)
src/mailman/model/mailinglist.py (+7/-6)
src/mailman/model/user.py (+2/-2)
src/mailman/rest/docs/moderation.rst (+15/-17)
src/mailman/testing/layers.py (+4/-8)
src/mailman/testing/testing.cfg (+1/-1)
src/mailman/utilities/importer.py (+1/-0)
src/mailman/utilities/tests/test_import.py (+10/-0)
To merge this branch: bzr merge lp:~abompard/mailman/sqlalchemy
Reviewer Review Type Date Requested Status
Abhilash Raj 2014-10-29 Pending
Review via email: mp+239952@code.launchpad.net

Description of the change

A small import fix

To post a comment you must log in.
lp:~abompard/mailman/sqlalchemy updated on 2014-10-31
7287. By Aurélien Bompard on 2014-10-30

Fix doctests with PostgreSQL

- drop the tables after the whole testsuite has run
- don't expose DB ids in the doctests
- sort lists by list-id in the ListManager

------------- This line and the following will be ignored --------------

modified:
  src/mailman/app/docs/moderator.rst
  src/mailman/commands/docs/withlist.rst
  src/mailman/database/base.py
  src/mailman/model/docs/mailinglist.rst
  src/mailman/model/domain.py
  src/mailman/model/listmanager.py
  src/mailman/rest/docs/moderation.rst
  src/mailman/testing/layers.py

7288. By Aurélien Bompard on 2014-10-31

Add a cleanup instruction in a doctest

7289. By Aurélien Bompard on 2014-10-31

One more sorting issue

Unmerged revisions

7289. By Aurélien Bompard on 2014-10-31

One more sorting issue

7288. By Aurélien Bompard on 2014-10-31

Add a cleanup instruction in a doctest

7287. By Aurélien Bompard on 2014-10-30

Fix doctests with PostgreSQL

- drop the tables after the whole testsuite has run
- don't expose DB ids in the doctests
- sort lists by list-id in the ListManager

------------- This line and the following will be ignored --------------

modified:
  src/mailman/app/docs/moderator.rst
  src/mailman/commands/docs/withlist.rst
  src/mailman/database/base.py
  src/mailman/model/docs/mailinglist.rst
  src/mailman/model/domain.py
  src/mailman/model/listmanager.py
  src/mailman/rest/docs/moderation.rst
  src/mailman/testing/layers.py

7286. By Aurélien Bompard on 2014-10-29

Importer: encode_ascii_prefixes need to be converted to bool

7285. By Barry Warsaw on 2014-10-14

Use print() to smooth over the SA return of Python longs in PostgreSQL.

7284. By Barry Warsaw on 2014-10-13

Remove some unnecessary code.

7283. By Barry Warsaw on 2014-10-13

Move alembic settings to a separate alembic.cfg.

7282. By Barry Warsaw on 2014-10-13

Merge Aurélien Bompard's latest merge branch, with some cleaning up by Barry.

7281. By Barry Warsaw on 2014-10-12

Add the [logging.database] section and use it to configure the SQLAlchemy and
Alembic loggers.

7280. By Barry Warsaw on 2014-10-12

Remove some unused stuff.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'MANIFEST.in'
2--- MANIFEST.in 2014-10-07 10:06:44 +0000
3+++ MANIFEST.in 2014-10-31 13:23:25 +0000
4@@ -1,4 +1,4 @@
5-include *.py *.rc
6+include *.py *.rc *.mako
7 include COPYING
8 recursive-include .buildout *
9 recursive-include contrib *
10@@ -6,12 +6,9 @@
11 recursive-include data *
12 global-include *.txt *.rst *.po *.mo *.cfg *.sql *.zcml *.html
13 global-exclude *.egg-info
14-exclude MANIFEST.in
15 prune src/attic
16 prune src/web
17 prune eggs
18 prune parts
19 include MANIFEST.in
20 include src/mailman/testing/config.pck
21-include src/mailman/database/alembic/script.py.mako
22-include src/mailman/database/alembic/versions/*.py
23
24=== modified file 'src/mailman/app/docs/moderator.rst'
25--- src/mailman/app/docs/moderator.rst 2014-04-28 15:23:35 +0000
26+++ src/mailman/app/docs/moderator.rst 2014-10-31 13:23:25 +0000
27@@ -211,9 +211,8 @@
28 ...
29 ... Here's something important about our mailing list.
30 ... """)
31- >>> hold_message(mlist, msg, {}, 'Needs approval')
32- 2
33- >>> handle_message(mlist, 2, Action.discard, forward=['zack@example.com'])
34+ >>> req_id = hold_message(mlist, msg, {}, 'Needs approval')
35+ >>> handle_message(mlist, req_id, Action.discard, forward=['zack@example.com'])
36
37 The forwarded message is in the virgin queue, destined for the moderator.
38 ::
39@@ -243,10 +242,9 @@
40
41 >>> from mailman.app.moderator import hold_subscription
42 >>> from mailman.interfaces.member import DeliveryMode
43- >>> hold_subscription(mlist,
44+ >>> req_id = hold_subscription(mlist,
45 ... 'fred@example.org', 'Fred Person',
46 ... '{NONE}abcxyz', DeliveryMode.regular, 'en')
47- 2
48
49
50 Disposing of membership change requests
51@@ -257,26 +255,26 @@
52 simply defer a decision for now.
53
54 >>> from mailman.app.moderator import handle_subscription
55- >>> handle_subscription(mlist, 2, Action.defer)
56- >>> requests.get_request(2) is not None
57+ >>> handle_subscription(mlist, req_id, Action.defer)
58+ >>> requests.get_request(req_id) is not None
59 True
60
61 The held subscription can also be discarded.
62
63- >>> handle_subscription(mlist, 2, Action.discard)
64- >>> print(requests.get_request(2))
65+ >>> handle_subscription(mlist, req_id, Action.discard)
66+ >>> print(requests.get_request(req_id))
67 None
68
69 Gwen tries to subscribe to the mailing list, but...
70
71- >>> hold_subscription(mlist,
72+ >>> req_id = hold_subscription(mlist,
73 ... 'gwen@example.org', 'Gwen Person',
74 ... '{NONE}zyxcba', DeliveryMode.regular, 'en')
75- 2
76+
77
78 ...her request is rejected...
79
80- >>> handle_subscription(mlist, 2, Action.reject, 'This is a closed list')
81+ >>> handle_subscription(mlist, req_id, Action.reject, 'This is a closed list')
82 >>> messages = get_queue_messages('virgin')
83 >>> len(messages)
84 1
85@@ -304,14 +302,13 @@
86 mailing list.
87
88 >>> mlist.send_welcome_message = False
89- >>> hold_subscription(mlist,
90+ >>> req_id = hold_subscription(mlist,
91 ... 'herb@example.org', 'Herb Person',
92 ... 'abcxyz', DeliveryMode.regular, 'en')
93- 2
94
95 The moderators accept the subscription request.
96
97- >>> handle_subscription(mlist, 2, Action.accept)
98+ >>> handle_subscription(mlist, req_id, Action.accept)
99
100 And now Herb is a member of the mailing list.
101
102@@ -328,29 +325,27 @@
103 Herb now wants to leave the mailing list, but his request must be approved.
104
105 >>> from mailman.app.moderator import hold_unsubscription
106- >>> hold_unsubscription(mlist, 'herb@example.org')
107- 2
108+ >>> req_id = hold_unsubscription(mlist, 'herb@example.org')
109
110 As with subscription requests, the unsubscription request can be deferred.
111
112 >>> from mailman.app.moderator import handle_unsubscription
113- >>> handle_unsubscription(mlist, 2, Action.defer)
114+ >>> handle_unsubscription(mlist, req_id, Action.defer)
115 >>> print(mlist.members.get_member('herb@example.org').address)
116 Herb Person <herb@example.org>
117
118 The held unsubscription can also be discarded, and the member will remain
119 subscribed.
120
121- >>> handle_unsubscription(mlist, 2, Action.discard)
122+ >>> handle_unsubscription(mlist, req_id, Action.discard)
123 >>> print(mlist.members.get_member('herb@example.org').address)
124 Herb Person <herb@example.org>
125
126 The request can be rejected, in which case a message is sent to the member,
127 and the person remains a member of the mailing list.
128
129- >>> hold_unsubscription(mlist, 'herb@example.org')
130- 2
131- >>> handle_unsubscription(mlist, 2, Action.reject, 'No can do')
132+ >>> req_id = hold_unsubscription(mlist, 'herb@example.org')
133+ >>> handle_unsubscription(mlist, req_id, Action.reject, 'No can do')
134 >>> print(mlist.members.get_member('herb@example.org').address)
135 Herb Person <herb@example.org>
136
137@@ -381,10 +376,9 @@
138 The unsubscription request can also be accepted. This removes the member from
139 the mailing list.
140
141- >>> hold_unsubscription(mlist, 'herb@example.org')
142- 2
143+ >>> req_id = hold_unsubscription(mlist, 'herb@example.org')
144 >>> mlist.send_goodbye_message = False
145- >>> handle_unsubscription(mlist, 2, Action.accept)
146+ >>> handle_unsubscription(mlist, req_id, Action.accept)
147 >>> print(mlist.members.get_member('herb@example.org'))
148 None
149
150@@ -403,9 +397,8 @@
151
152 Iris tries to subscribe to the mailing list.
153
154- >>> hold_subscription(mlist, 'iris@example.org', 'Iris Person',
155+ >>> req_id = hold_subscription(mlist, 'iris@example.org', 'Iris Person',
156 ... 'password', DeliveryMode.regular, 'en')
157- 2
158
159 There's now a message in the virgin queue, destined for the list owner.
160
161@@ -429,8 +422,7 @@
162 Similarly, the administrator gets notifications on unsubscription requests.
163 Jeff is a member of the mailing list, and chooses to unsubscribe.
164
165- >>> hold_unsubscription(mlist, 'jeff@example.org')
166- 3
167+ >>> unsub_req_id = hold_unsubscription(mlist, 'jeff@example.org')
168 >>> messages = get_queue_messages('virgin')
169 >>> len(messages)
170 1
171@@ -457,7 +449,7 @@
172
173 >>> mlist.admin_notify_mchanges = True
174 >>> mlist.admin_immed_notify = False
175- >>> handle_subscription(mlist, 2, Action.accept)
176+ >>> handle_subscription(mlist, req_id, Action.accept)
177 >>> messages = get_queue_messages('virgin')
178 >>> len(messages)
179 1
180@@ -474,9 +466,8 @@
181 Similarly when an unsubscription request is accepted, the administrators can
182 get a notification.
183
184- >>> hold_unsubscription(mlist, 'iris@example.org')
185- 4
186- >>> handle_unsubscription(mlist, 4, Action.accept)
187+ >>> req_id = hold_unsubscription(mlist, 'iris@example.org')
188+ >>> handle_unsubscription(mlist, req_id, Action.accept)
189 >>> messages = get_queue_messages('virgin')
190 >>> len(messages)
191 1
192@@ -498,10 +489,9 @@
193
194 >>> mlist.admin_notify_mchanges = False
195 >>> mlist.send_welcome_message = True
196- >>> hold_subscription(mlist, 'kate@example.org', 'Kate Person',
197- ... 'password', DeliveryMode.regular, 'en')
198- 4
199- >>> handle_subscription(mlist, 4, Action.accept)
200+ >>> req_id = hold_subscription(mlist, 'kate@example.org', 'Kate Person',
201+ ... 'password', DeliveryMode.regular, 'en')
202+ >>> handle_subscription(mlist, req_id, Action.accept)
203 >>> messages = get_queue_messages('virgin')
204 >>> len(messages)
205 1
206@@ -523,9 +513,8 @@
207 goodbye message.
208
209 >>> mlist.send_goodbye_message = True
210- >>> hold_unsubscription(mlist, 'kate@example.org')
211- 4
212- >>> handle_unsubscription(mlist, 4, Action.accept)
213+ >>> req_id = hold_unsubscription(mlist, 'kate@example.org')
214+ >>> handle_unsubscription(mlist, req_id, Action.accept)
215 >>> messages = get_queue_messages('virgin')
216 >>> len(messages)
217 1
218
219=== modified file 'src/mailman/commands/docs/conf.rst'
220--- src/mailman/commands/docs/conf.rst 2014-10-10 04:59:43 +0000
221+++ src/mailman/commands/docs/conf.rst 2014-10-31 13:23:25 +0000
222@@ -22,7 +22,7 @@
223 command without any options.
224
225 >>> command.process(FakeArgs)
226- [alembic] script_location: mailman.database:alembic
227+ [logging.archiver] path: mailman.log
228 ...
229 [passwords] password_length: 8
230 ...
231@@ -43,7 +43,6 @@
232 >>> FakeArgs.section = None
233 >>> FakeArgs.key = 'path'
234 >>> command.process(FakeArgs)
235- [logging.dbmigration] path: mailman.log
236 [logging.archiver] path: mailman.log
237 [logging.locks] path: mailman.log
238 [logging.mischief] path: mailman.log
239
240=== modified file 'src/mailman/commands/docs/withlist.rst'
241--- src/mailman/commands/docs/withlist.rst 2014-04-28 15:23:35 +0000
242+++ src/mailman/commands/docs/withlist.rst 2014-10-31 13:23:25 +0000
243@@ -90,13 +90,13 @@
244 >>> args.listname = '^.*example.com'
245 >>> command.process(args)
246 The list's display name is Aardvark
247+ The list's display name is Badboys
248 The list's display name is Badger
249- The list's display name is Badboys
250
251 >>> args.listname = '^bad.*'
252 >>> command.process(args)
253+ The list's display name is Badboys
254 The list's display name is Badger
255- The list's display name is Badboys
256
257 >>> args.listname = '^foo'
258 >>> command.process(args)
259
260=== added file 'src/mailman/config/alembic.cfg'
261--- src/mailman/config/alembic.cfg 1970-01-01 00:00:00 +0000
262+++ src/mailman/config/alembic.cfg 2014-10-31 13:23:25 +0000
263@@ -0,0 +1,20 @@
264+# Copyright (C) 2014 by the Free Software Foundation, Inc.
265+#
266+# This file is part of GNU Mailman.
267+#
268+# GNU Mailman is free software: you can redistribute it and/or modify it under
269+# the terms of the GNU General Public License as published by the Free
270+# Software Foundation, either version 3 of the License, or (at your option)
271+# any later version.
272+#
273+# GNU Mailman is distributed in the hope that it will be useful, but WITHOUT
274+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
275+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
276+# more details.
277+#
278+# You should have received a copy of the GNU General Public License along with
279+# GNU Mailman. If not, see <http://www.gnu.org/licenses/>.
280+
281+[alembic]
282+# Path to Alembic migration scripts.
283+script_location: mailman.database:alembic
284
285=== modified file 'src/mailman/config/config.py'
286--- src/mailman/config/config.py 2014-10-02 14:46:00 +0000
287+++ src/mailman/config/config.py 2014-10-31 13:23:25 +0000
288@@ -87,7 +87,6 @@
289 self.pipelines = {}
290 self.commands = {}
291 self.password_context = None
292- self.initialized = False
293
294 def _clear(self):
295 """Clear the cached configuration variables."""
296@@ -137,7 +136,6 @@
297 # Expand and set up all directories.
298 self._expand_paths()
299 self.ensure_directories_exist()
300- self.initialized = True
301 notify(ConfigurationUpdatedEvent(self))
302
303 def _expand_paths(self):
304
305=== modified file 'src/mailman/config/schema.cfg'
306--- src/mailman/config/schema.cfg 2014-10-10 04:59:43 +0000
307+++ src/mailman/config/schema.cfg 2014-10-31 13:23:25 +0000
308@@ -226,6 +226,7 @@
309 # - archiver -- All archiver output
310 # - bounce -- All bounce processing logs go here
311 # - config -- Configuration issues
312+# - database -- Database logging (SQLAlchemy and Alembic)
313 # - debug -- Only used for development
314 # - error -- All exceptions go to this log
315 # - fromusenet -- Information related to the Usenet to Mailman gateway
316@@ -237,8 +238,6 @@
317 # - smtp-failure -- Unsuccessful SMTP activity
318 # - subscribe -- Information about leaves/joins
319 # - vette -- Message vetting information
320-# - database -- Database activity
321-# - dbmigration -- Database migrations
322 format: %(asctime)s (%(process)d) %(message)s
323 datefmt: %b %d %H:%M:%S %Y
324 propagate: no
325@@ -254,6 +253,8 @@
326
327 [logging.config]
328
329+[logging.database]
330+
331 [logging.debug]
332 path: debug.log
333 level: info
334@@ -306,9 +307,6 @@
335 [logging.database]
336 level: warn
337
338-[logging.dbmigration]
339-level: warn
340-
341
342 [webservice]
343 # The hostname at which admin web service resources are exposed.
344@@ -645,7 +643,3 @@
345 CC X-Original-CC
346 Content-Transfer-Encoding X-Original-Content-Transfer-Encoding
347 MIME-Version X-MIME-Version
348-
349-[alembic]
350-# path to migration scripts
351-script_location = mailman.database:alembic
352
353=== modified file 'src/mailman/core/docs/runner.rst'
354--- src/mailman/core/docs/runner.rst 2014-04-28 15:23:35 +0000
355+++ src/mailman/core/docs/runner.rst 2014-10-31 13:23:25 +0000
356@@ -73,3 +73,5 @@
357 version : 3
358
359 XXX More of the Runner API should be tested.
360+
361+ >>> config.pop('test-runner')
362
363=== modified file 'src/mailman/core/logging.py'
364--- src/mailman/core/logging.py 2014-10-07 09:19:20 +0000
365+++ src/mailman/core/logging.py 2014-10-31 13:23:25 +0000
366@@ -104,6 +104,27 @@
367
368
369
370
371+def _init_logger(propagate, sub_name, log, logger_config):
372+ # Get settings from log configuration file (or defaults).
373+ log_format = logger_config.format
374+ log_datefmt = logger_config.datefmt
375+ # Propagation to the root logger is how we handle logging to stderr
376+ # when the runners are not run as a subprocess of 'bin/mailman start'.
377+ log.propagate = (as_boolean(logger_config.propagate)
378+ if propagate is None else propagate)
379+ # Set the logger's level.
380+ log.setLevel(as_log_level(logger_config.level))
381+ # Create a formatter for this logger, then a handler, and link the
382+ # formatter to the handler.
383+ formatter = logging.Formatter(fmt=log_format, datefmt=log_datefmt)
384+ path_str = logger_config.path
385+ path_abs = os.path.normpath(os.path.join(config.LOG_DIR, path_str))
386+ handler = ReopenableFileHandler(sub_name, path_abs)
387+ _handlers[sub_name] = handler
388+ handler.setFormatter(formatter)
389+ log.addHandler(handler)
390+
391+
392 def initialize(propagate=None):
393 """Initialize all logs.
394
395@@ -126,32 +147,18 @@
396 continue
397 if sub_name == 'locks':
398 log = logging.getLogger('flufl.lock')
399- elif sub_name == 'database':
400+ if sub_name == 'database':
401+ # Set both the SQLAlchemy and Alembic logs to the mailman.database
402+ # log configuration, essentially ignoring the alembic.cfg
403+ # settings. Do the SQLAlchemy one first, then let the Alembic one
404+ # fall through to the common code path.
405 log = logging.getLogger('sqlalchemy')
406- elif sub_name == 'dbmigration':
407+ _init_logger(propagate, sub_name, log, logger_config)
408 log = logging.getLogger('alembic')
409 else:
410 logger_name = 'mailman.' + sub_name
411 log = logging.getLogger(logger_name)
412- # Get settings from log configuration file (or defaults).
413- log_format = logger_config.format
414- log_datefmt = logger_config.datefmt
415- # Propagation to the root logger is how we handle logging to stderr
416- # when the runners are not run as a subprocess of 'bin/mailman start'.
417- log.propagate = (as_boolean(logger_config.propagate)
418- if propagate is None else propagate)
419- # Set the logger's level.
420- log.setLevel(as_log_level(logger_config.level))
421- # Create a formatter for this logger, then a handler, and link the
422- # formatter to the handler.
423- formatter = logging.Formatter(fmt=log_format, datefmt=log_datefmt)
424- path_str = logger_config.path
425- path_abs = os.path.normpath(os.path.join(config.LOG_DIR, path_str))
426- handler = ReopenableFileHandler(sub_name, path_abs)
427- _handlers[sub_name] = handler
428- handler.setFormatter(formatter)
429- log.addHandler(handler)
430-
431+ _init_logger(propagate, sub_name, log, logger_config)
432
433
434
435 def reopen():
436
437=== modified file 'src/mailman/database/alembic/__init__.py'
438--- src/mailman/database/alembic/__init__.py 2014-10-10 04:59:43 +0000
439+++ src/mailman/database/alembic/__init__.py 2014-10-31 13:23:25 +0000
440@@ -15,18 +15,18 @@
441 # You should have received a copy of the GNU General Public License along with
442 # GNU Mailman. If not, see <http://www.gnu.org/licenses/>.
443
444-"Alembic config init."
445+"""Alembic configuration initization."""
446
447 from __future__ import absolute_import, print_function, unicode_literals
448
449 __metaclass__ = type
450 __all__ = [
451- 'alembic_cfg'
452-]
453+ 'alembic_cfg',
454+ ]
455
456
457 from alembic.config import Config
458 from mailman.utilities.modules import expand_path
459
460
461-alembic_cfg=Config(expand_path("python:mailman.config.schema"))
462+alembic_cfg = Config(expand_path('python:mailman.config.alembic'))
463
464=== modified file 'src/mailman/database/alembic/env.py'
465--- src/mailman/database/alembic/env.py 2014-10-10 04:59:43 +0000
466+++ src/mailman/database/alembic/env.py 2014-10-31 13:23:25 +0000
467@@ -30,15 +30,10 @@
468 from contextlib import closing
469 from sqlalchemy import create_engine
470
471-from mailman.core import initialize
472 from mailman.config import config
473-from mailman.database.alembic import alembic_cfg
474 from mailman.database.model import Model
475 from mailman.utilities.string import expand
476
477-if not config.initialized:
478- initialize.initialize_1(context.config.config_file_name)
479-
480
481
482
483 def run_migrations_offline():
484
485=== modified file 'src/mailman/database/alembic/versions/51b7f92bd06c_initial.py'
486--- src/mailman/database/alembic/versions/51b7f92bd06c_initial.py 2014-10-10 16:58:48 +0000
487+++ src/mailman/database/alembic/versions/51b7f92bd06c_initial.py 2014-10-31 13:23:25 +0000
488@@ -1,34 +1,66 @@
489-"""initial
490+# Copyright (C) 2014 by the Free Software Foundation, Inc.
491+#
492+# This file is part of GNU Mailman.
493+#
494+# GNU Mailman is free software: you can redistribute it and/or modify it under
495+# the terms of the GNU General Public License as published by the Free
496+# Software Foundation, either version 3 of the License, or (at your option)
497+# any later version.
498+#
499+# GNU Mailman is distributed in the hope that it will be useful, but WITHOUT
500+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
501+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
502+# more details.
503+#
504+# You should have received a copy of the GNU General Public License along with
505+# GNU Mailman. If not, see <http://www.gnu.org/licenses/>.
506+
507+"""Initial migration.
508+
509+This empty migration file makes sure there is always an alembic_version
510+in the database. As a consequence, if the database version is reported
511+as None, it means the database needs to be created from scratch with
512+SQLAlchemy itself.
513+
514+It also removes schema items left over from Storm.
515
516 Revision ID: 51b7f92bd06c
517 Revises: None
518 Create Date: 2014-10-10 09:53:35.624472
519-
520 """
521
522-# revision identifiers, used by Alembic.
523+from __future__ import absolute_import, print_function, unicode_literals
524+
525+__metaclass__ = type
526+__all__ = [
527+ 'downgrade',
528+ 'upgrade',
529+ ]
530+
531+
532+from alembic import op
533+import sqlalchemy as sa
534+
535+
536+# Revision identifiers, used by Alembic.
537 revision = '51b7f92bd06c'
538 down_revision = None
539
540-from alembic import op
541-import sqlalchemy as sa
542-
543
544 def upgrade():
545- ### commands auto generated by Alembic - please adjust! ###
546 op.drop_table('version')
547- if op.get_bind().dialect.name != "sqlite":
548- # SQLite does not support dropping columns
549+ if op.get_bind().dialect.name != 'sqlite':
550+ # SQLite does not support dropping columns.
551 op.drop_column('mailinglist', 'acceptable_aliases_id')
552- op.create_index(op.f('ix_user__user_id'), 'user', ['_user_id'], unique=False)
553+ op.create_index(op.f('ix_user__user_id'), 'user',
554+ ['_user_id'], unique=False)
555 op.drop_index('ix_user_user_id', table_name='user')
556- ### end Alembic commands ###
557
558
559 def downgrade():
560- ### commands auto generated by Alembic - please adjust! ###
561 op.create_table('version')
562 op.create_index('ix_user_user_id', 'user', ['_user_id'], unique=False)
563 op.drop_index(op.f('ix_user__user_id'), table_name='user')
564- op.add_column('mailinglist', sa.Column('acceptable_aliases_id', sa.INTEGER(), nullable=True))
565- ### end Alembic commands ###
566+ op.add_column(
567+ 'mailinglist',
568+ sa.Column('acceptable_aliases_id', sa.INTEGER(), nullable=True))
569
570=== modified file 'src/mailman/database/base.py'
571--- src/mailman/database/base.py 2014-10-06 13:58:58 +0000
572+++ src/mailman/database/base.py 2014-10-31 13:23:25 +0000
573@@ -25,7 +25,6 @@
574
575 import logging
576
577-from alembic import command
578 from sqlalchemy import create_engine
579 from sqlalchemy.orm import sessionmaker
580 from zope.interface import implementer
581@@ -115,3 +114,8 @@
582 session = sessionmaker(bind=self.engine)
583 self.store = session()
584 self.store.commit()
585+
586+ def destroy(self):
587+ """Drop all database tables"""
588+ from mailman.database.model import Model
589+ Model.metadata.drop_all(self.engine)
590
591=== modified file 'src/mailman/database/factory.py'
592--- src/mailman/database/factory.py 2014-10-10 16:44:01 +0000
593+++ src/mailman/database/factory.py 2014-10-31 13:23:25 +0000
594@@ -28,8 +28,8 @@
595
596 import os
597 import types
598+import alembic.command
599
600-from alembic import command
601 from alembic.migration import MigrationContext
602 from alembic.script import ScriptDirectory
603 from flufl.lock import Lock
604@@ -38,10 +38,14 @@
605 from zope.interface.verify import verifyObject
606
607 from mailman.config import config
608+from mailman.database.alembic import alembic_cfg
609 from mailman.database.model import Model
610-from mailman.database.alembic import alembic_cfg
611-from mailman.interfaces.database import IDatabase, IDatabaseFactory
612-from mailman.utilities.modules import call_name, expand_path
613+from mailman.interfaces.database import (
614+ DatabaseError, IDatabase, IDatabaseFactory)
615+from mailman.utilities.modules import call_name
616+
617+
618+LAST_STORM_SCHEMA_VERSION = '20130406000000'
619
620
621
622
623@@ -57,67 +61,58 @@
624 database = call_name(database_class)
625 verifyObject(IDatabase, database)
626 database.initialize()
627- schema_mgr = SchemaManager(database)
628- schema_mgr.setup_db()
629+ SchemaManager(database).setup_database()
630 database.commit()
631 return database
632
633
634
635
636 class SchemaManager:
637-
638- LAST_STORM_SCHEMA_VERSION = '20130406000000'
639+ "Manage schema migrations."""
640
641 def __init__(self, database):
642- self.database = database
643- self.script = ScriptDirectory.from_config(alembic_cfg)
644+ self._database = database
645+ self._script = ScriptDirectory.from_config(alembic_cfg)
646
647- def get_storm_schema_version(self):
648- md = MetaData()
649- md.reflect(bind=self.database.engine)
650- if "version" not in md.tables:
651+ def _get_storm_schema_version(self):
652+ metadata = MetaData()
653+ metadata.reflect(bind=self._database.engine)
654+ if 'version' not in metadata.tables:
655+ # There are no Storm artifacts left.
656 return None
657- Version = md.tables["version"]
658- last_version = self.database.store.query(Version.c.version).filter(
659- Version.c.component == "schema"
660- ).order_by(Version.c.version.desc()).first()
661- # Don't leave open transactions or they will block any schema change
662- self.database.commit()
663+ Version = metadata.tables['version']
664+ last_version = self._database.store.query(Version.c.version).filter(
665+ Version.c.component == 'schema'
666+ ).order_by(Version.c.version.desc()).first()
667+ # Don't leave open transactions or they will block any schema change.
668+ self._database.commit()
669 return last_version
670
671- def _create(self):
672- # initial DB creation
673- Model.metadata.create_all(self.database.engine)
674- self.database.commit()
675- command.stamp(alembic_cfg, "head")
676-
677- def _upgrade(self):
678- command.upgrade(alembic_cfg, "head")
679-
680- def setup_db(self):
681- context = MigrationContext.configure(self.database.store.connection())
682+ def setup_database(self):
683+ context = MigrationContext.configure(self._database.store.connection())
684 current_rev = context.get_current_revision()
685- head_rev = self.script.get_current_head()
686+ head_rev = self._script.get_current_head()
687 if current_rev == head_rev:
688- return head_rev # already at the latest revision, nothing to do
689- if current_rev == None:
690- # no alembic information
691- storm_version = self.get_storm_schema_version()
692+ # We're already at the latest revision so there's nothing to do.
693+ return head_rev
694+ if current_rev is None:
695+ # No Alembic information is available.
696+ storm_version = self._get_storm_schema_version()
697 if storm_version is None:
698- # initial DB creation
699- self._create()
700+ # Initial database creation.
701+ Model.metadata.create_all(self._database.engine)
702+ self._database.commit()
703+ alembic.command.stamp(alembic_cfg, 'head')
704 else:
705- # DB from a previous version managed by Storm
706- if storm_version.version < self.LAST_STORM_SCHEMA_VERSION:
707- raise RuntimeError(
708- "Upgrading while skipping beta version is "
709- "unsupported, please install the previous "
710- "Mailman beta release")
711- # Run migrations to remove the Storm-specific table and
712- # upgrade to SQLAlchemy & Alembic
713- self._upgrade()
714+ # The database was previously managed by Storm.
715+ if storm_version.version < LAST_STORM_SCHEMA_VERSION:
716+ raise DatabaseError(
717+ 'Upgrades skipping beta versions is not supported.')
718+ # Run migrations to remove the Storm-specific table and upgrade
719+ # to SQLAlchemy and Alembic.
720+ alembic.command.upgrade(alembic_cfg, 'head')
721 elif current_rev != head_rev:
722- self._upgrade()
723+ alembic.command.upgrade(alembic_cfg, 'head')
724 return head_rev
725
726
727
728=== modified file 'src/mailman/database/tests/test_factory.py'
729--- src/mailman/database/tests/test_factory.py 2014-10-10 16:58:48 +0000
730+++ src/mailman/database/tests/test_factory.py 2014-10-31 13:23:25 +0000
731@@ -21,24 +21,24 @@
732
733 __metaclass__ = type
734 __all__ = [
735+ 'TestSchemaManager',
736 ]
737
738
739 import unittest
740-import types
741-
742 import alembic.command
743-from mock import Mock
744+
745+from mock import patch
746 from sqlalchemy import MetaData, Table, Column, Integer, Unicode
747+from sqlalchemy.exc import ProgrammingError, OperationalError
748 from sqlalchemy.schema import Index
749-from sqlalchemy.exc import ProgrammingError, OperationalError
750
751 from mailman.config import config
752-from mailman.testing.layers import ConfigLayer
753-from mailman.database.factory import SchemaManager, _reset
754-from mailman.database.sqlite import SQLiteDatabase
755 from mailman.database.alembic import alembic_cfg
756+from mailman.database.factory import LAST_STORM_SCHEMA_VERSION, SchemaManager
757 from mailman.database.model import Model
758+from mailman.interfaces.database import DatabaseError
759+from mailman.testing.layers import ConfigLayer
760
761
762
763
764@@ -47,116 +47,114 @@
765 layer = ConfigLayer
766
767 def setUp(self):
768- # Drop the existing database
769+ # Drop the existing database.
770 Model.metadata.drop_all(config.db.engine)
771 md = MetaData()
772 md.reflect(bind=config.db.engine)
773- for tablename in ("alembic_version", "version"):
774+ for tablename in ('alembic_version', 'version'):
775 if tablename in md.tables:
776 md.tables[tablename].drop(config.db.engine)
777 self.schema_mgr = SchemaManager(config.db)
778
779 def tearDown(self):
780 self._drop_storm_database()
781- # Restore a virgin DB
782+ # Restore a virgin database.
783 Model.metadata.create_all(config.db.engine)
784
785-
786 def _table_exists(self, tablename):
787 md = MetaData()
788 md.reflect(bind=config.db.engine)
789 return tablename in md.tables
790
791 def _create_storm_database(self, revision):
792- version_table = Table("version", Model.metadata,
793- Column("id", Integer, primary_key=True),
794- Column("component", Unicode),
795- Column("version", Unicode),
796- )
797+ version_table = Table(
798+ 'version', Model.metadata,
799+ Column('id', Integer, primary_key=True),
800+ Column('component', Unicode),
801+ Column('version', Unicode),
802+ )
803 version_table.create(config.db.engine)
804 config.db.store.execute(version_table.insert().values(
805- component='schema', version=revision))
806+ component='schema', version=revision))
807 config.db.commit()
808 # Other Storm specific changes, those SQL statements hopefully work on
809 # all DB engines...
810 config.db.engine.execute(
811- "ALTER TABLE mailinglist ADD COLUMN acceptable_aliases_id INT")
812- Index("ix_user__user_id").drop(bind=config.db.engine)
813- # Don't pollute our main metadata object, create a new one
814+ 'ALTER TABLE mailinglist ADD COLUMN acceptable_aliases_id INT')
815+ Index('ix_user__user_id').drop(bind=config.db.engine)
816+ # Don't pollute our main metadata object, create a new one.
817 md = MetaData()
818- user_table = Model.metadata.tables["user"].tometadata(md)
819- Index("ix_user_user_id", user_table.c._user_id
820- ).create(bind=config.db.engine)
821+ user_table = Model.metadata.tables['user'].tometadata(md)
822+ Index('ix_user_user_id', user_table.c._user_id).create(
823+ bind=config.db.engine)
824 config.db.commit()
825
826 def _drop_storm_database(self):
827- """
828- Remove the leftovers from a Storm DB.
829- (you must issue a drop_all() afterwards)
830- """
831- if "version" in Model.metadata.tables:
832- version = Model.metadata.tables["version"]
833+ """Remove the leftovers from a Storm DB.
834+
835+ A drop_all() must be issued afterwards.
836+ """
837+ if 'version' in Model.metadata.tables:
838+ version = Model.metadata.tables['version']
839 version.drop(config.db.engine, checkfirst=True)
840 Model.metadata.remove(version)
841 try:
842- Index("ix_user_user_id").drop(bind=config.db.engine)
843- except (ProgrammingError, OperationalError) as e:
844- # non-existant (PGSQL raises a ProgrammingError, while SQLite
845- # raises an OperationalError)
846+ Index('ix_user_user_id').drop(bind=config.db.engine)
847+ except (ProgrammingError, OperationalError):
848+ # Nonexistent. PostgreSQL raises a ProgrammingError, while SQLite
849+ # raises an OperationalError.
850 pass
851 config.db.commit()
852
853-
854- def test_current_db(self):
855- """The database is already at the latest version"""
856- alembic.command.stamp(alembic_cfg, "head")
857- self.schema_mgr._create = Mock()
858- self.schema_mgr._upgrade = Mock()
859- self.schema_mgr.setup_db()
860- self.assertFalse(self.schema_mgr._create.called)
861- self.assertFalse(self.schema_mgr._upgrade.called)
862-
863- def test_initial(self):
864- """No existing database"""
865- self.assertFalse(self._table_exists("mailinglist"))
866- self.assertFalse(self._table_exists("alembic_version"))
867- self.schema_mgr._upgrade = Mock()
868- self.schema_mgr.setup_db()
869- self.assertFalse(self.schema_mgr._upgrade.called)
870- self.assertTrue(self._table_exists("mailinglist"))
871- self.assertTrue(self._table_exists("alembic_version"))
872-
873- def test_storm(self):
874- """Existing Storm database"""
875- Model.metadata.create_all(config.db.engine)
876- self._create_storm_database(
877- self.schema_mgr.LAST_STORM_SCHEMA_VERSION)
878- self.schema_mgr._create = Mock()
879- self.schema_mgr.setup_db()
880- self.assertFalse(self.schema_mgr._create.called)
881- self.assertTrue(self._table_exists("mailinglist")
882- and self._table_exists("alembic_version")
883- and not self._table_exists("version"))
884-
885- def test_old_storm(self):
886- """Existing Storm database in an old version"""
887- Model.metadata.create_all(config.db.engine)
888- self._create_storm_database("001")
889- self.schema_mgr._create = Mock()
890- self.assertRaises(RuntimeError, self.schema_mgr.setup_db)
891- self.assertFalse(self.schema_mgr._create.called)
892+ def test_current_database(self):
893+ # The database is already at the latest version.
894+ alembic.command.stamp(alembic_cfg, 'head')
895+ with patch('alembic.command') as alembic_command:
896+ self.schema_mgr.setup_database()
897+ self.assertFalse(alembic_command.stamp.called)
898+ self.assertFalse(alembic_command.upgrade.called)
899+
900+ @patch('alembic.command')
901+ def test_initial(self, alembic_command):
902+ # No existing database.
903+ self.assertFalse(self._table_exists('mailinglist'))
904+ self.assertFalse(self._table_exists('alembic_version'))
905+ self.schema_mgr.setup_database()
906+ self.assertFalse(alembic_command.upgrade.called)
907+ self.assertTrue(self._table_exists('mailinglist'))
908+ self.assertTrue(self._table_exists('alembic_version'))
909+
910+ @patch('alembic.command.stamp')
911+ def test_storm(self, alembic_command_stamp):
912+ # Existing Storm database.
913+ Model.metadata.create_all(config.db.engine)
914+ self._create_storm_database(LAST_STORM_SCHEMA_VERSION)
915+ self.schema_mgr.setup_database()
916+ self.assertFalse(alembic_command_stamp.called)
917+ self.assertTrue(
918+ self._table_exists('mailinglist')
919+ and self._table_exists('alembic_version')
920+ and not self._table_exists('version'))
921+
922+ @patch('alembic.command')
923+ def test_old_storm(self, alembic_command):
924+ # Existing Storm database in an old version.
925+ Model.metadata.create_all(config.db.engine)
926+ self._create_storm_database('001')
927+ self.assertRaises(DatabaseError, self.schema_mgr.setup_database)
928+ self.assertFalse(alembic_command.stamp.called)
929+ self.assertFalse(alembic_command.upgrade.called)
930
931 def test_old_db(self):
932- """The database is in an old revision, must upgrade"""
933- alembic.command.stamp(alembic_cfg, "head")
934+ # The database is in an old revision, must upgrade.
935+ alembic.command.stamp(alembic_cfg, 'head')
936 md = MetaData()
937 md.reflect(bind=config.db.engine)
938- config.db.store.execute(md.tables["alembic_version"].delete())
939- config.db.store.execute(md.tables["alembic_version"].insert().values(
940- version_num="dummyrevision"))
941+ config.db.store.execute(md.tables['alembic_version'].delete())
942+ config.db.store.execute(md.tables['alembic_version'].insert().values(
943+ version_num='dummyrevision'))
944 config.db.commit()
945- self.schema_mgr._create = Mock()
946- self.schema_mgr._upgrade = Mock()
947- self.schema_mgr.setup_db()
948- self.assertFalse(self.schema_mgr._create.called)
949- self.assertTrue(self.schema_mgr._upgrade.called)
950+ with patch('alembic.command') as alembic_command:
951+ self.schema_mgr.setup_database()
952+ self.assertFalse(alembic_command.stamp.called)
953+ self.assertTrue(alembic_command.upgrade.called)
954
955=== modified file 'src/mailman/handlers/docs/owner-recips.rst'
956--- src/mailman/handlers/docs/owner-recips.rst 2012-03-23 20:34:54 +0000
957+++ src/mailman/handlers/docs/owner-recips.rst 2014-10-31 13:23:25 +0000
958@@ -41,7 +41,7 @@
959 >>> handler.process(mlist_1, msg, msgdata)
960 >>> dump_list(msgdata['recipients'])
961 bart@example.com
962-
963+
964 If Bart also disables his owner delivery, then no one could contact the list's
965 owners. Since this is unacceptable, the site owner is used as a fallback.
966
967@@ -55,7 +55,7 @@
968 a fallback.
969
970 >>> mlist_2 = create_list('beta@example.com')
971- >>> mlist_2.administrators.member_count
972+ >>> print(mlist_2.administrators.member_count)
973 0
974 >>> msgdata = {}
975 >>> handler.process(mlist_2, msg, msgdata)
976
977=== modified file 'src/mailman/model/docs/autorespond.rst'
978--- src/mailman/model/docs/autorespond.rst 2014-04-28 15:23:35 +0000
979+++ src/mailman/model/docs/autorespond.rst 2014-10-31 13:23:25 +0000
980@@ -37,34 +37,34 @@
981 ... 'aperson@example.com')
982
983 >>> from mailman.interfaces.autorespond import Response
984- >>> response_set.todays_count(address, Response.hold)
985+ >>> print(response_set.todays_count(address, Response.hold))
986 0
987- >>> response_set.todays_count(address, Response.command)
988+ >>> print(response_set.todays_count(address, Response.command))
989 0
990
991 Using the response set, we can record that a hold response is sent to the
992 address.
993
994 >>> response_set.response_sent(address, Response.hold)
995- >>> response_set.todays_count(address, Response.hold)
996+ >>> print(response_set.todays_count(address, Response.hold))
997 1
998- >>> response_set.todays_count(address, Response.command)
999+ >>> print(response_set.todays_count(address, Response.command))
1000 0
1001
1002 We can also record that a command response was sent.
1003
1004 >>> response_set.response_sent(address, Response.command)
1005- >>> response_set.todays_count(address, Response.hold)
1006+ >>> print(response_set.todays_count(address, Response.hold))
1007 1
1008- >>> response_set.todays_count(address, Response.command)
1009+ >>> print(response_set.todays_count(address, Response.command))
1010 1
1011
1012 Let's send one more.
1013
1014 >>> response_set.response_sent(address, Response.command)
1015- >>> response_set.todays_count(address, Response.hold)
1016+ >>> print(response_set.todays_count(address, Response.hold))
1017 1
1018- >>> response_set.todays_count(address, Response.command)
1019+ >>> print(response_set.todays_count(address, Response.command))
1020 2
1021
1022 Now the day flips over and all the counts reset.
1023@@ -73,9 +73,9 @@
1024 >>> from mailman.utilities.datetime import factory
1025 >>> factory.fast_forward()
1026
1027- >>> response_set.todays_count(address, Response.hold)
1028+ >>> print(response_set.todays_count(address, Response.hold))
1029 0
1030- >>> response_set.todays_count(address, Response.command)
1031+ >>> print(response_set.todays_count(address, Response.command))
1032 0
1033
1034
1035@@ -110,7 +110,7 @@
1036
1037 >>> address = getUtility(IUserManager).create_address(
1038 ... 'bperson@example.com')
1039- >>> response_set.todays_count(address, Response.command)
1040+ >>> print(response_set.todays_count(address, Response.command))
1041 0
1042 >>> print(response_set.last_response(address, Response.command))
1043 None
1044
1045=== modified file 'src/mailman/model/docs/mailinglist.rst'
1046--- src/mailman/model/docs/mailinglist.rst 2014-04-28 15:23:35 +0000
1047+++ src/mailman/model/docs/mailinglist.rst 2014-10-31 13:23:25 +0000
1048@@ -50,7 +50,7 @@
1049
1050 Both addresses appear on the roster of members.
1051
1052- >>> for member in mlist.members.members:
1053+ >>> for member in sorted(mlist.members.members, key=lambda m: m.address.email):
1054 ... print(member)
1055 <Member: aperson@example.com on aardvark@example.com as MemberRole.member>
1056 <Member: bperson@example.com on aardvark@example.com as MemberRole.member>
1057@@ -72,7 +72,7 @@
1058 an owner and a moderator.
1059 ::
1060
1061- >>> for member in mlist.owners.members:
1062+ >>> for member in sorted(mlist.owners.members, key=lambda m: m.address.email):
1063 ... print(member)
1064 <Member: aperson@example.com on aardvark@example.com as MemberRole.owner>
1065 <Member: cperson@example.com on aardvark@example.com as MemberRole.owner>
1066@@ -87,13 +87,13 @@
1067 ::
1068
1069 >>> roster = mlist.get_roster(MemberRole.member)
1070- >>> for member in roster.members:
1071+ >>> for member in sorted(roster.members, key=lambda m: m.address.email):
1072 ... print(member)
1073 <Member: aperson@example.com on aardvark@example.com as MemberRole.member>
1074 <Member: bperson@example.com on aardvark@example.com as MemberRole.member>
1075
1076 >>> roster = mlist.get_roster(MemberRole.owner)
1077- >>> for member in roster.members:
1078+ >>> for member in sorted(roster.members, key=lambda m: m.address.email):
1079 ... print(member)
1080 <Member: aperson@example.com on aardvark@example.com as MemberRole.owner>
1081 <Member: cperson@example.com on aardvark@example.com as MemberRole.owner>
1082@@ -122,7 +122,7 @@
1083 >>> mlist.subscribe(user)
1084 <Member: Dave Person <dperson@example.com> on aardvark@example.com
1085 as MemberRole.member>
1086- >>> for member in mlist.members.members:
1087+ >>> for member in sorted(mlist.members.members, key=lambda m: m.address.email):
1088 ... print(member)
1089 <Member: aperson@example.com on aardvark@example.com as MemberRole.member>
1090 <Member: bperson@example.com on aardvark@example.com as MemberRole.member>
1091@@ -133,7 +133,7 @@
1092 >>> new_address.verified_on = now()
1093 >>> user.preferred_address = new_address
1094
1095- >>> for member in mlist.members.members:
1096+ >>> for member in sorted(mlist.members.members, key=lambda m: m.address.email):
1097 ... print(member)
1098 <Member: aperson@example.com on aardvark@example.com as MemberRole.member>
1099 <Member: bperson@example.com on aardvark@example.com as MemberRole.member>
1100
1101=== modified file 'src/mailman/model/docs/requests.rst'
1102--- src/mailman/model/docs/requests.rst 2014-04-28 15:23:35 +0000
1103+++ src/mailman/model/docs/requests.rst 2014-10-31 13:23:25 +0000
1104@@ -35,7 +35,7 @@
1105
1106 The list's requests database starts out empty.
1107
1108- >>> requests.count
1109+ >>> print(requests.count)
1110 0
1111 >>> dump_list(requests.held_requests)
1112 *Empty*
1113@@ -68,21 +68,21 @@
1114
1115 We can see the total number of requests being held.
1116
1117- >>> requests.count
1118+ >>> print(requests.count)
1119 3
1120
1121 We can also see the number of requests being held by request type.
1122
1123- >>> requests.count_of(RequestType.subscription)
1124+ >>> print(requests.count_of(RequestType.subscription))
1125 1
1126- >>> requests.count_of(RequestType.unsubscription)
1127+ >>> print(requests.count_of(RequestType.unsubscription))
1128 1
1129
1130 We can also see when there are multiple held requests of a particular type.
1131
1132- >>> requests.hold_request(RequestType.held_message, 'hold_4')
1133+ >>> print(requests.hold_request(RequestType.held_message, 'hold_4'))
1134 4
1135- >>> requests.count_of(RequestType.held_message)
1136+ >>> print(requests.count_of(RequestType.held_message))
1137 2
1138
1139 We can ask the requests database for a specific request, by providing the id
1140@@ -132,7 +132,7 @@
1141 To make it easier to find specific requests, the list requests can be iterated
1142 over by type.
1143
1144- >>> requests.count_of(RequestType.held_message)
1145+ >>> print(requests.count_of(RequestType.held_message))
1146 3
1147 >>> for request in requests.of_type(RequestType.held_message):
1148 ... key, data = requests.get_request(request.id)
1149@@ -154,10 +154,10 @@
1150 Once a specific request has been handled, it can be deleted from the requests
1151 database.
1152
1153- >>> requests.count
1154+ >>> print(requests.count)
1155 5
1156 >>> requests.delete_request(2)
1157- >>> requests.count
1158+ >>> print(requests.count)
1159 4
1160
1161 Request 2 is no longer in the database.
1162@@ -167,5 +167,5 @@
1163
1164 >>> for request in requests.held_requests:
1165 ... requests.delete_request(request.id)
1166- >>> requests.count
1167+ >>> print(requests.count)
1168 0
1169
1170=== modified file 'src/mailman/model/domain.py'
1171--- src/mailman/model/domain.py 2014-09-22 18:47:02 +0000
1172+++ src/mailman/model/domain.py 2014-10-31 13:23:25 +0000
1173@@ -48,7 +48,7 @@
1174
1175 id = Column(Integer, primary_key=True)
1176
1177- mail_host = Column(Unicode)
1178+ mail_host = Column(Unicode) # TODO: add index?
1179 base_url = Column(Unicode)
1180 description = Column(Unicode)
1181 contact_address = Column(Unicode)
1182@@ -95,7 +95,8 @@
1183 def mailing_lists(self, store):
1184 """See `IDomain`."""
1185 mailing_lists = store.query(MailingList).filter(
1186- MailingList.mail_host == self.mail_host)
1187+ MailingList.mail_host == self.mail_host
1188+ ).order_by(MailingList._list_id)
1189 for mlist in mailing_lists:
1190 yield mlist
1191
1192@@ -170,7 +171,7 @@
1193 @dbconnection
1194 def __iter__(self, store):
1195 """See `IDomainManager`."""
1196- for domain in store.query(Domain).all():
1197+ for domain in store.query(Domain).order_by(Domain.mail_host).all():
1198 yield domain
1199
1200 @dbconnection
1201
1202=== modified file 'src/mailman/model/listmanager.py'
1203--- src/mailman/model/listmanager.py 2014-09-21 21:06:40 +0000
1204+++ src/mailman/model/listmanager.py 2014-10-31 13:23:25 +0000
1205@@ -86,7 +86,8 @@
1206 @dbconnection
1207 def mailing_lists(self, store):
1208 """See `IListManager`."""
1209- for mlist in store.query(MailingList).all():
1210+ for mlist in store.query(MailingList).order_by(
1211+ MailingList._list_id).all():
1212 yield mlist
1213
1214 @dbconnection
1215
1216=== modified file 'src/mailman/model/mailinglist.py'
1217--- src/mailman/model/mailinglist.py 2014-10-10 04:59:43 +0000
1218+++ src/mailman/model/mailinglist.py 2014-10-31 13:23:25 +0000
1219@@ -504,10 +504,9 @@
1220
1221 id = Column(Integer, primary_key=True)
1222
1223- mailing_list_id = Column(Integer,
1224- ForeignKey('mailinglist.id'),
1225- index=True,
1226- nullable=False)
1227+ mailing_list_id = Column(
1228+ Integer, ForeignKey('mailinglist.id'),
1229+ index=True, nullable=False)
1230 mailing_list = relationship('MailingList', backref='acceptable_alias')
1231 alias = Column(Unicode, index=True, nullable=False)
1232
1233@@ -561,9 +560,11 @@
1234
1235 id = Column(Integer, primary_key=True)
1236
1237- mailing_list_id = Column(Integer, ForeignKey('mailinglist.id'),
1238- index=True, nullable=False)
1239+ mailing_list_id = Column(
1240+ Integer, ForeignKey('mailinglist.id'),
1241+ index=True, nullable=False)
1242 mailing_list = relationship('MailingList')
1243+
1244 name = Column(Unicode, nullable=False)
1245 _is_enabled = Column(Boolean)
1246
1247
1248=== modified file 'src/mailman/model/user.py'
1249--- src/mailman/model/user.py 2014-10-10 04:59:43 +0000
1250+++ src/mailman/model/user.py 2014-10-31 13:23:25 +0000
1251@@ -56,7 +56,7 @@
1252
1253 id = Column(Integer, primary_key=True)
1254 display_name = Column(Unicode)
1255- _password = Column('password', LargeBinary) # TODO : was RawStr()
1256+ _password = Column('password', LargeBinary)
1257 _user_id = Column(UUID, index=True)
1258 _created_on = Column(DateTime)
1259
1260@@ -68,7 +68,7 @@
1261 Integer,
1262 ForeignKey('address.id', use_alter=True,
1263 name='_preferred_address',
1264- ondelete="SET NULL"))
1265+ ondelete='SET NULL'))
1266
1267 _preferred_address = relationship(
1268 'Address', primaryjoin=(_preferred_address_id==Address.id),
1269
1270=== modified file 'src/mailman/rest/docs/moderation.rst'
1271--- src/mailman/rest/docs/moderation.rst 2014-04-28 15:23:35 +0000
1272+++ src/mailman/rest/docs/moderation.rst 2014-10-31 13:23:25 +0000
1273@@ -226,10 +226,9 @@
1274
1275 >>> from mailman.app.moderator import hold_subscription
1276 >>> from mailman.interfaces.member import DeliveryMode
1277- >>> hold_subscription(
1278+ >>> sub_req_id = hold_subscription(
1279 ... ant, 'anne@example.com', 'Anne Person',
1280 ... 'password', DeliveryMode.regular, 'en')
1281- 1
1282 >>> transaction.commit()
1283
1284 The subscription request is available from the mailing list.
1285@@ -242,7 +241,7 @@
1286 http_etag: "..."
1287 language: en
1288 password: password
1289- request_id: 1
1290+ request_id: ...
1291 type: subscription
1292 when: 2005-08-01T07:49:23
1293 http_etag: "..."
1294@@ -259,8 +258,7 @@
1295 >>> from mailman.app.moderator import hold_unsubscription
1296 >>> bart = add_member(ant, 'bart@example.com', 'Bart Person',
1297 ... 'password', DeliveryMode.regular, 'en')
1298- >>> hold_unsubscription(ant, 'bart@example.com')
1299- 2
1300+ >>> unsub_req_id = hold_unsubscription(ant, 'bart@example.com')
1301 >>> transaction.commit()
1302
1303 The unsubscription request is also available from the mailing list.
1304@@ -273,13 +271,13 @@
1305 http_etag: "..."
1306 language: en
1307 password: password
1308- request_id: 1
1309+ request_id: ...
1310 type: subscription
1311 when: 2005-08-01T07:49:23
1312 entry 1:
1313 address: bart@example.com
1314 http_etag: "..."
1315- request_id: 2
1316+ request_id: ...
1317 type: unsubscription
1318 http_etag: "..."
1319 start: 0
1320@@ -292,23 +290,25 @@
1321 You can view an individual membership change request by providing the
1322 request id. Anne's subscription request looks like this.
1323
1324- >>> dump_json('http://localhost:9001/3.0/lists/ant@example.com/requests/1')
1325+ >>> dump_json('http://localhost:9001/3.0/lists/ant@example.com/'
1326+ ... 'requests/{}'.format(sub_req_id))
1327 address: anne@example.com
1328 delivery_mode: regular
1329 display_name: Anne Person
1330 http_etag: "..."
1331 language: en
1332 password: password
1333- request_id: 1
1334+ request_id: ...
1335 type: subscription
1336 when: 2005-08-01T07:49:23
1337
1338 Bart's unsubscription request looks like this.
1339
1340- >>> dump_json('http://localhost:9001/3.0/lists/ant@example.com/requests/2')
1341+ >>> dump_json('http://localhost:9001/3.0/lists/ant@example.com/'
1342+ ... 'requests/{}'.format(unsub_req_id))
1343 address: bart@example.com
1344 http_etag: "..."
1345- request_id: 2
1346+ request_id: ...
1347 type: unsubscription
1348
1349
1350@@ -328,9 +328,8 @@
1351 Anne's subscription request is accepted.
1352
1353 >>> dump_json('http://localhost:9001/3.0/lists/'
1354- ... 'ant@example.com/requests/1', {
1355- ... 'action': 'accept',
1356- ... })
1357+ ... 'ant@example.com/requests/{}'.format(sub_req_id),
1358+ ... {'action': 'accept'})
1359 content-length: 0
1360 date: ...
1361 server: ...
1362@@ -347,9 +346,8 @@
1363 Bart's unsubscription request is discarded.
1364
1365 >>> dump_json('http://localhost:9001/3.0/lists/'
1366- ... 'ant@example.com/requests/2', {
1367- ... 'action': 'discard',
1368- ... })
1369+ ... 'ant@example.com/requests/{}'.format(unsub_req_id),
1370+ ... {'action': 'discard'})
1371 content-length: 0
1372 date: ...
1373 server: ...
1374
1375=== modified file 'src/mailman/testing/layers.py'
1376--- src/mailman/testing/layers.py 2014-10-06 17:17:50 +0000
1377+++ src/mailman/testing/layers.py 2014-10-31 13:23:25 +0000
1378@@ -47,16 +47,13 @@
1379
1380 from lazr.config import as_boolean
1381 from pkg_resources import resource_string
1382-from sqlalchemy import MetaData
1383 from textwrap import dedent
1384-from zope import event
1385 from zope.component import getUtility
1386
1387 from mailman.config import config
1388 from mailman.core import initialize
1389 from mailman.core.initialize import INHIBIT_CONFIG_FILE
1390 from mailman.core.logging import get_handler
1391-from mailman.database.model import Model
1392 from mailman.database.transaction import transaction
1393 from mailman.interfaces.domain import IDomainManager
1394 from mailman.testing.helpers import (
1395@@ -101,9 +98,7 @@
1396 # Set up the basic configuration stuff. Turn off path creation until
1397 # we've pushed the testing config.
1398 config.create_paths = False
1399- if not event.subscribers:
1400- # only if not yet initialized by another layer
1401- initialize.initialize_1(INHIBIT_CONFIG_FILE)
1402+ initialize.initialize_1(INHIBIT_CONFIG_FILE)
1403 assert cls.var_dir is None, 'Layer already set up'
1404 # Calculate a temporary VAR_DIR directory so that run-time artifacts
1405 # of the tests won't tread on the installation's data. This also
1406@@ -195,10 +190,11 @@
1407 @classmethod
1408 def tearDown(cls):
1409 assert cls.var_dir is not None, 'Layer not set up'
1410- # Reset the test database after the tests are done so that there is no
1411+ reset_the_world()
1412+ # Destroy the test database after the tests are done so that there is no
1413 # data in case the tests are rerun with a database layer like mysql or
1414 # postgresql which are not deleted in teardown.
1415- reset_the_world()
1416+ config.db.destroy()
1417 config.pop('test config')
1418 shutil.rmtree(cls.var_dir)
1419 cls.var_dir = None
1420
1421=== modified file 'src/mailman/testing/testing.cfg'
1422--- src/mailman/testing/testing.cfg 2014-10-10 04:59:43 +0000
1423+++ src/mailman/testing/testing.cfg 2014-10-31 13:23:25 +0000
1424@@ -20,7 +20,7 @@
1425 # For testing against PostgreSQL.
1426 # [database]
1427 # class: mailman.database.postgresql.PostgreSQLDatabase
1428-# url: postgresql://maxking:maxking@localhost/mailman_test
1429+# url: postgresql://$USER:$USER@localhost/mailman_test
1430
1431 [mailman]
1432 site_owner: noreply@example.com
1433
1434=== modified file 'src/mailman/utilities/importer.py'
1435--- src/mailman/utilities/importer.py 2014-09-28 00:17:05 +0000
1436+++ src/mailman/utilities/importer.py 2014-10-31 13:23:25 +0000
1437@@ -175,6 +175,7 @@
1438 allow_list_posts=bool,
1439 include_rfc2369_headers=bool,
1440 nntp_prefix_subject_too=bool,
1441+ encode_ascii_prefixes=bool,
1442 )
1443
1444
1445
1446=== modified file 'src/mailman/utilities/tests/test_import.py'
1447--- src/mailman/utilities/tests/test_import.py 2014-04-28 15:23:35 +0000
1448+++ src/mailman/utilities/tests/test_import.py 2014-10-31 13:23:25 +0000
1449@@ -34,6 +34,7 @@
1450 from datetime import timedelta, datetime
1451 from enum import Enum
1452 from pkg_resources import resource_filename
1453+from sqlalchemy.exc import IntegrityError
1454 from zope.component import getUtility
1455
1456 from mailman.app.lifecycle import create_list
1457@@ -291,6 +292,15 @@
1458 else:
1459 self.fail('Import21Error was not raised')
1460
1461+ def test_encode_ascii_prefixes(self):
1462+ self._pckdict['encode_ascii_prefixes'] = 2
1463+ self.assertEqual(self._mlist.encode_ascii_prefixes, False)
1464+ try:
1465+ self._import()
1466+ except IntegrityError as e:
1467+ self.fail(e)
1468+ self.assertEqual(self._mlist.encode_ascii_prefixes, True)
1469+
1470
1471
1472
1473 class TestArchiveImport(unittest.TestCase):

Subscribers

People subscribed via source and target branches