Merge lp:~cjwatson/launchpad/upload-mail into lp:launchpad

Proposed by Colin Watson
Status: Merged
Merged at revision: 17698
Proposed branch: lp:~cjwatson/launchpad/upload-mail
Merge into: lp:launchpad
Diff against target: 3299 lines (+1182/-1013)
22 files modified
lib/lp/archiveuploader/tests/nascentupload-announcements.txt (+54/-53)
lib/lp/archiveuploader/tests/test_ppauploadprocessor.py (+62/-72)
lib/lp/archiveuploader/tests/test_sync_notification.py (+5/-5)
lib/lp/archiveuploader/tests/test_uploadprocessor.py (+111/-98)
lib/lp/services/mail/basemailer.py (+6/-1)
lib/lp/services/mail/notificationrecipientset.py (+29/-5)
lib/lp/soyuz/doc/distroseriesqueue-notify.txt (+99/-56)
lib/lp/soyuz/doc/soyuz-set-of-uploads.txt (+15/-14)
lib/lp/soyuz/emailtemplates/ppa-upload-accepted.txt (+0/-4)
lib/lp/soyuz/emailtemplates/ppa-upload-rejection.txt (+0/-4)
lib/lp/soyuz/emailtemplates/upload-accepted.txt (+0/-4)
lib/lp/soyuz/emailtemplates/upload-new.txt (+0/-4)
lib/lp/soyuz/emailtemplates/upload-rejection.txt (+0/-4)
lib/lp/soyuz/mail/packageupload.py (+533/-535)
lib/lp/soyuz/mail/tests/test_packageupload.py (+205/-100)
lib/lp/soyuz/model/queue.py (+9/-6)
lib/lp/soyuz/scripts/packagecopier.py (+13/-10)
lib/lp/soyuz/scripts/tests/test_copypackage.py (+9/-17)
lib/lp/soyuz/stories/soyuz/xx-queue-pages.txt (+10/-6)
lib/lp/soyuz/tests/test_distroseriesqueue_debian_installer.py (+9/-3)
lib/lp/soyuz/tests/test_packagecopyjob.py (+7/-5)
lib/lp/soyuz/tests/test_packageupload.py (+6/-7)
To merge this branch: bzr merge lp:~cjwatson/launchpad/upload-mail
Reviewer Review Type Date Requested Status
William Grant (community) code Approve
Review via email: mp+269066@code.launchpad.net

Commit message

Convert package upload notifications to BaseMailer.

Description of the change

Convert package upload notifications to BaseMailer.

The main effect of this is that each recipient now receives a separate mail. This allows us to set X-Launchpad-Message-Rationale headers and use a much less vague footer.

Other minor effects:
 * There's now an "X-Launchpad-Notification-Type: package-upload" header.
 * Rejection mails without any spr, bprs, or customfiles (e.g. rejections from archiveuploader) are a little more uniform: as well as the above header changes, the last word changes from "rejected" to "(Rejected)".
 * PPA upload notifications have a proper footer separator ("-- " rather than "--").
 * PPA upload notifications are no longer unnecessarily multipart.

The changes list required some special handling, because it doesn't correspond to a Person. I added a StubPerson to the notification recipient set edifice to cope with this.

X-LP-M-R: Requester and "... because you made this upload" are intentionally a little vague, because at that point it could be either the signer of a direct upload or the requester of a sync.

To post a comment you must log in.
Revision history for this message
William Grant (wgrant) :
review: Approve (code)
Revision history for this message
William Grant (wgrant) wrote :

As an alternative to commenting the StubPerson bits, perhaps it would be safer and also more obvious to have an AnnouncementStubPerson as a marker.

Revision history for this message
Colin Watson (cjwatson) :

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/lp/archiveuploader/tests/nascentupload-announcements.txt'
2--- lib/lp/archiveuploader/tests/nascentupload-announcements.txt 2015-07-08 16:52:34 +0000
3+++ lib/lp/archiveuploader/tests/nascentupload-announcements.txt 2015-08-26 13:41:53 +0000
4@@ -101,9 +101,8 @@
5 DEBUG above if files already exist in other distroseries.
6 ...
7 DEBUG --
8- DEBUG You are receiving this email because you are the uploader,
9- maintainer or
10- DEBUG signer of the above package.
11+ DEBUG You are receiving this email because you are the most recent person
12+ DEBUG listed in this package's changelog.
13
14 There is only one email generated:
15
16@@ -143,7 +142,7 @@
17 DEBUG bar diff from 1.0-1 to 1.0-1 requested
18 DEBUG Setting it to ACCEPTED
19 ...
20- DEBUG Sending rejection email.
21+ DEBUG Sent a mail:
22 ...
23 DEBUG Rejected:
24 DEBUG The source bar - 1.0-1 is already accepted in ubuntu/hoary
25@@ -206,8 +205,8 @@
26 <BLANKLINE>
27 -- =
28 <BLANKLINE>
29- You are receiving this email because you are the uploader, maintainer or
30- signer of the above package.
31+ You are receiving this email because you are the most recent person
32+ listed in this package's changelog.
33 <BLANKLINE>
34
35 In order to facilitate automated processing of announcement emails, the
36@@ -215,10 +214,10 @@
37
38 >>> attachment = notification.get_payload()[1]
39 >>> print attachment.as_string() # doctest: -NORMALIZE_WHITESPACE
40+ Content-Disposition: attachment; filename="changesfile"
41+ MIME-Version: 1.0
42 Content-Type: text/plain; charset="utf-8"
43- MIME-Version: 1.0
44 Content-Transfer-Encoding: quoted-printable
45- Content-Disposition: attachment; filename="changesfile"
46 <BLANKLINE>
47 -----BEGIN PGP SIGNED MESSAGE-----
48 Hash: SHA1
49@@ -296,10 +295,8 @@
50 However, PPA upload notifications do not contain an attachment with the
51 original changesfile.
52
53- >>> attachment = notification.get_payload()[1]
54- Traceback (most recent call last):
55- ...
56- IndexError: list index out of range
57+ >>> notification.is_multipart()
58+ False
59
60 See further tests upon PPA upload notifications on
61 archiveuploader/tests/test_ppauploadprocessor.
62@@ -348,8 +345,7 @@
63 >>> lang_pack.logger = FakeLogger()
64 >>> result = lang_pack.do_accept()
65 DEBUG Creating queue entry
66- DEBUG Skipping acceptance and announcement, it is a language-package
67- upload.
68+ DEBUG Skipping acceptance and announcement for language packs.
69
70 >>> lang_pack.queue_root.status.name
71 'NEW'
72@@ -381,8 +377,7 @@
73 >>> result = lang_pack.do_accept()
74 DEBUG Creating queue entry
75 ...
76- DEBUG Skipping acceptance and announcement, it is a language-package
77- upload.
78+ DEBUG Skipping acceptance and announcement for language packs.
79
80 >>> lang_pack.queue_root.status.name
81 'DONE'
82@@ -412,22 +407,21 @@
83 DEBUG Creating queue entry
84 DEBUG language-pack-pt diff from 1.0-2 to 1.0-3 requested
85 DEBUG Setting it to UNAPPROVED
86- DEBUG Skipping acceptance and announcement, it is a language-package
87- upload.
88+ DEBUG Skipping acceptance and announcement for language packs.
89
90 >>> lang_pack.queue_root.status.name
91 'UNAPPROVED'
92
93 UNAPPROVED message was also skipped for an upload targeted to
94-'translation' section:
95-
96+'translation' section:
97 >>> transaction.commit()
98 >>> len(stub.test_emails)
99 0
100
101
102-An UNAPPROVED binary upload via insecure will send one email saying that
103-the upload is waiting for approval:
104+An UNAPPROVED binary upload via insecure will send emails (in this case, one
105+to the signer and one to the changer) saying that the upload is waiting for
106+approval:
107
108 >>> bar_src = NascentUpload.from_changesfile_path(
109 ... datadir('suite/bar_1.0-2/bar_1.0-2_source.changes'),
110@@ -439,16 +433,21 @@
111 DEBUG Creating queue entry
112 ...
113
114- >>> [notification] = pop_notifications()
115-
116- >>> notification['X-Katie']
117- 'Launchpad actually'
118-
119- >>> print_addrlist(notification['To'])
120+ >>> changer_notification, signer_notification = pop_notifications()
121+
122+ >>> changer_notification['X-Katie']
123+ 'Launchpad actually'
124+ >>> signer_notification['X-Katie']
125+ 'Launchpad actually'
126+
127+ >>> print_addrlist(changer_notification['To'])
128 Daniel Silverstone <daniel.silverstone@canonical.com>
129+ >>> print_addrlist(signer_notification['To'])
130 Foo Bar <foo.bar@canonical.com>
131
132- >>> notification['Subject']
133+ >>> changer_notification['Subject']
134+ '[ubuntu/hoary-updates] bar 1.0-2 (Waiting for approval)'
135+ >>> signer_notification['Subject']
136 '[ubuntu/hoary-updates] bar 1.0-2 (Waiting for approval)'
137
138 And clean up.
139@@ -457,7 +456,7 @@
140 >>> upload_data = datadir('suite/bar_1.0-2')
141 >>> os.remove(os.path.join(upload_data, 'bar_1.0.orig.tar.gz'))
142
143-UNAPPROVED upload to BACKPORTS via insecure policy will send a notification
144+UNAPPROVED upload to BACKPORTS via insecure policy will send notifications
145 saying they are waiting for approval:
146
147 >>> unapproved_backports_policy = getPolicy(
148@@ -475,16 +474,21 @@
149 DEBUG Setting it to UNAPPROVED
150 ...
151
152- >>> [notification] = pop_notifications()
153-
154- >>> notification['X-Katie']
155- 'Launchpad actually'
156-
157- >>> print_addrlist(notification['To'])
158+ >>> changer_notification, signer_notification = pop_notifications()
159+
160+ >>> changer_notification['X-Katie']
161+ 'Launchpad actually'
162+ >>> signer_notification['X-Katie']
163+ 'Launchpad actually'
164+
165+ >>> print_addrlist(changer_notification['To'])
166 Daniel Silverstone <daniel.silverstone@canonical.com>
167+ >>> print_addrlist(signer_notification['To'])
168 Foo Bar <foo.bar@canonical.com>
169
170- >>> notification['Subject']
171+ >>> changer_notification['Subject']
172+ '[ubuntu/hoary-backports] bar 1.0-3 (Waiting for approval)'
173+ >>> signer_notification['Subject']
174 '[ubuntu/hoary-backports] bar 1.0-3 (Waiting for approval)'
175
176 AUTO-APPROVED upload to BACKPORTS pocket via 'sync' policy:
177@@ -528,9 +532,8 @@
178 DEBUG Thank you for your contribution to Ubuntu.
179 DEBUG
180 DEBUG --
181- DEBUG You are receiving this email because you are the uploader,
182- maintainer or
183- DEBUG signer of the above package.
184+ DEBUG You are receiving this email because you are the most recent person
185+ DEBUG listed in this package's changelog.
186
187 There is one email generated:
188
189@@ -662,9 +665,8 @@
190 ...
191 DEBUG Announcing to hoary-announce@lists.ubuntu.com
192 ...
193- DEBUG You are receiving this email because you are the uploader, maintainer
194- or
195- DEBUG signer of the above package.
196+ DEBUG You are receiving this email because you are the most recent person
197+ DEBUG listed in this package's changelog.
198 DEBUG Would have sent a mail:
199 DEBUG Subject: [ubuntu/hoary] bar 1.0-6 (Accepted)
200 DEBUG Sender: Celso Providelo <cprov@ubuntu.com>
201@@ -698,10 +700,11 @@
202
203 >>> msgs = pop_notifications(sort_key=operator.itemgetter('To'))
204 >>> len(msgs)
205- 2
206+ 3
207
208 >>> [message['From'].replace('\n ', ' ') for message in msgs]
209- ['Root <root@localhost>', '=?utf-8?q?Non-ascii_changed-by_=C4=8Ciha=C5=99?=
210+ ['Root <root@localhost>', 'Root <root@localhost>',
211+ '=?utf-8?q?Non-ascii_changed-by_=C4=8Ciha=C5=99?=
212 <daniel.silverstone@canonical.com>']
213
214 UTF-8 text in the changes file that is sent on the email is preserved
215@@ -746,8 +749,8 @@
216 <BLANKLINE>
217 -- =
218 <BLANKLINE>
219- You are receiving this email because you are the uploader, maintainer or
220- signer of the above package.
221+ You are receiving this email because you are the most recent person
222+ listed in this package's changelog.
223 <BLANKLINE>
224
225 In order to facilitate scripts that parse announcement emails, the changes
226@@ -763,10 +766,10 @@
227 And what follows is the content of the attachment.
228
229 >>> print attachment.as_string() # doctest: -NORMALIZE_WHITESPACE
230+ Content-Disposition: attachment; filename="changesfile"
231+ MIME-Version: 1.0
232 Content-Type: text/plain; charset="utf-8"
233- MIME-Version: 1.0
234 Content-Transfer-Encoding: quoted-printable
235- Content-Disposition: attachment; filename="changesfile"
236 <BLANKLINE>
237 -----BEGIN PGP SIGNED MESSAGE-----
238 Hash: SHA1
239@@ -832,7 +835,6 @@
240 >>> result = bar_src.do_accept()
241 DEBUG Building recipients list.
242 ...
243- DEBUG Sending rejection email.
244 DEBUG Sent a mail:
245 ...
246 DEBUG Rejected:
247@@ -844,9 +846,8 @@
248 DEBUG http://answers.launchpad.net/soyuz
249 DEBUG
250 DEBUG --
251- DEBUG You are receiving this email because you are the uploader,
252- maintainer or
253- DEBUG signer of the above package.
254+ DEBUG You are receiving this email because you are the most recent person
255+ DEBUG listed in this package's changelog.
256
257 >>> [notification] = pop_notifications()
258
259
260=== modified file 'lib/lp/archiveuploader/tests/test_ppauploadprocessor.py'
261--- lib/lp/archiveuploader/tests/test_ppauploadprocessor.py 2015-04-20 15:59:52 +0000
262+++ lib/lp/archiveuploader/tests/test_ppauploadprocessor.py 2015-08-26 13:41:53 +0000
263@@ -2,7 +2,7 @@
264 # NOTE: The first line above must stay first; do not move the copyright
265 # notice to the top. See http://www.python.org/dev/peps/pep-0263/.
266 #
267-# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
268+# Copyright 2009-2015 Canonical Ltd. This software is licensed under the
269 # GNU Affero General Public License version 3 (see the file LICENSE).
270
271 """Functional tests for uploadprocessor.py."""
272@@ -73,56 +73,48 @@
273 def makeArchive(self, owner):
274 return self.factory.makeArchive(owner=owner, name='ppa')
275
276- def assertEmail(self, contents=None, recipients=None, ppa_header='name16'):
277- """Check email last upload notification attributes.
278+ def assertEmails(self, expected):
279+ """Check recent email upload notification attributes.
280
281- :param: contents: can be a list of one or more lines, if passed
282- they will be checked against the lines in Subject + Body.
283- :param: recipients: can be a list of recipients lines, it defaults
284- to 'Foo Bar <foo.bar@canonical.com>' (name16 account) and
285- should match the email To: header content.
286- :param: ppa_header: is the content of the 'X-Launchpad-PPA' header,
287- it defaults to 'name16' and should be explicitly set to None for
288- non-PPA or rejection notifications.
289+ :param expected: A list of dicts, each of which represents an
290+ expected email and may have "contents", "recipient", and
291+ "ppa_header" keys. All the items in expected must match in the
292+ correct order, with none left over. "contents" is a list of
293+ lines; assert that each is in Subject + Body. "recipient" is
294+ the To address the email must have, defaulting to
295+ "foo.bar@canonical.com" which is the signer on most of the test
296+ data uploads. "ppa_header" is the content of the
297+ "X-Launchpad-PPA" header; it defaults to "name16" and should be
298+ explicitly set to None for non-PPA or rejection notifications.
299 """
300- if not recipients:
301- recipients = [self.name16_recipient]
302-
303- if not contents:
304- contents = []
305-
306- queue_size = len(stub.test_emails)
307- messages = "\n".join(m for f, t, m in stub.test_emails)
308- self.assertEqual(
309- queue_size, 1, 'Unexpected number of emails sent: %s\n%s'
310- % (queue_size, messages))
311-
312- from_addr, to_addrs, raw_msg = stub.test_emails.pop()
313- msg = message_from_string(raw_msg)
314-
315- # This is now a MIMEMultipart message.
316- body = msg.get_payload(0)
317- body = body.get_payload(decode=True)
318-
319- clean_recipients = [r.strip() for r in to_addrs]
320- for recipient in list(recipients):
321- self.assertTrue(
322- recipient in clean_recipients,
323- "%s not in %s" % (recipient, clean_recipients))
324- self.assertEqual(
325- len(recipients), len(clean_recipients),
326- "Email recipients do not match exactly. Expected %s, got %s" %
327- (recipients, clean_recipients))
328-
329- subject = "Subject: %s\n" % msg['Subject']
330- body = subject + body
331-
332- for content in list(contents):
333- self.assertIn(content, body)
334-
335- if ppa_header is not None:
336- self.assertIn('X-Launchpad-PPA', msg.keys())
337- self.assertEqual(msg['X-Launchpad-PPA'], ppa_header)
338+ for item in expected:
339+ recipient = item.get("recipient", self.name16_recipient)
340+ contents = item.get("contents", [])
341+ ppa_header = item.get("ppa_header", "name16")
342+
343+ from_addr, to_addrs, raw_msg = stub.test_emails.pop()
344+ msg = message_from_string(raw_msg)
345+
346+ # This is now a non-multipart message.
347+ self.assertFalse(msg.is_multipart())
348+ body = msg.get_payload(decode=True)
349+
350+ clean_recipients = [r.strip() for r in to_addrs]
351+ self.assertContentEqual([recipient], clean_recipients)
352+
353+ subject = "Subject: %s\n" % msg['Subject']
354+ body = subject + body
355+
356+ for content in list(contents):
357+ self.assertIn(content, body)
358+
359+ if ppa_header is not None:
360+ self.assertIn('X-Launchpad-PPA', msg.keys())
361+ self.assertEqual(msg['X-Launchpad-PPA'], ppa_header)
362+
363+ self.assertEqual(
364+ [], stub.test_emails,
365+ "%d emails left over" % len(stub.test_emails))
366
367 def checkFilesRestrictedInLibrarian(self, queue_item, condition):
368 """Check the libraryfilealias restricted flag.
369@@ -255,7 +247,7 @@
370 # it's the default PPA.
371 contents = [
372 "Subject: [~name16/ubuntu/ppa/breezy] bar 1.0-1 (Accepted)"]
373- self.assertEmail(contents, ppa_header='name16')
374+ self.assertEmails([{"contents": contents, "ppa_header": "name16"}])
375
376 def testNamedPPAUploadNonDefault(self):
377 """Test PPA uploads to a named PPA."""
378@@ -273,7 +265,8 @@
379 # Subject and PPA email-header are specific for this named-ppa.
380 contents = [
381 "Subject: [~name16/ubuntu/testing/breezy] bar 1.0-1 (Accepted)"]
382- self.assertEmail(contents, ppa_header='name16-testing')
383+ self.assertEmails(
384+ [{"contents": contents, "ppa_header": "name16-testing"}])
385
386 def testNamedPPAUploadWithSeries(self):
387 """Test PPA uploads to a named PPA location and with a distroseries.
388@@ -447,7 +440,7 @@
389 # name16 is Foo Bar, who signed the upload. The package that was
390 # uploaded also contains two other valid (in sampledata) email
391 # addresses for maintainer and changed-by which must be ignored.
392- self.assertEmail()
393+ self.assertEmails([{}])
394
395 def testUploadSendsEmailToPeopleInArchivePermissions(self):
396 """PPA uploads result in notifications to ArchivePermission uploaders.
397@@ -474,17 +467,17 @@
398 upload_dir = self.queueUpload("bar_1.0-1", "~cprov/ppa/ubuntu")
399 self.processUpload(self.uploadprocessor, upload_dir)
400
401- name12_email = "%s <%s>" % (
402- name12.displayname, name12.preferredemail.email)
403- team_email = "%s <%s>" % (team.displayname, team.preferredemail.email)
404-
405 # We expect the recipients to be:
406- # - the package signer (name15),
407+ # - the package signer (name16),
408 # - the team in the extra permissions,
409 # - name12 who is in the extra permissions.
410 expected_recipients = (
411- self.name16_recipient, name12_email, team_email)
412- self.assertEmail(ppa_header="cprov", recipients=expected_recipients)
413+ self.name16_recipient,
414+ team.preferredemail.email,
415+ name12.preferredemail.email)
416+ self.assertEmails([
417+ {"ppa_header": "cprov", "recipient": expected_recipient}
418+ for expected_recipient in reversed(sorted(expected_recipients))])
419
420 def testPPADistroSeriesOverrides(self):
421 """It's possible to override target distroseries of PPA uploads.
422@@ -805,17 +798,15 @@
423 'previous error.'], rejection_message.splitlines())
424
425 contents = [
426- "Subject: [~cprov/ubuntu/ppa] bar_1.0-1_source.changes rejected",
427+ "Subject: [~cprov/ubuntu/ppa] bar_1.0-1_source.changes (Rejected)",
428 "Could not find person or team named 'boing'",
429 "https://help.launchpad.net/Packaging/PPA/Uploading",
430 "If you don't understand why your files were rejected please "
431 "send an email",
432 ("to %s for help (requires membership)."
433 % config.launchpad.users_address),
434- "You are receiving this email because you are the uploader "
435- "of the above",
436- "PPA package."]
437- self.assertEmail(contents, ppa_header=None)
438+ "You are receiving this email because you made this upload."]
439+ self.assertEmails([{"contents": contents, "ppa_header": None}])
440
441
442 class TestPPAUploadProcessorFileLookups(TestPPAUploadProcessorBase):
443@@ -1003,8 +994,7 @@
444 # Also, the email generated should be sane.
445 from_addr, to_addrs, raw_msg = stub.test_emails.pop()
446 msg = message_from_string(raw_msg)
447- body = msg.get_payload(0)
448- body = body.get_payload(decode=True)
449+ body = msg.get_payload(decode=True)
450
451 self.assertTrue(
452 "File bar_1.0.orig.tar.gz already exists in unicode PPA name: "
453@@ -1027,8 +1017,7 @@
454 # The email generated should be sane.
455 from_addr, to_addrs, raw_msg = stub.test_emails.pop()
456 msg = message_from_string(raw_msg)
457- body = msg.get_payload(0)
458- body = body.get_payload(decode=True)
459+ body = msg.get_payload(decode=True)
460
461 self.assertTrue(
462 "Rejected:\n"
463@@ -1229,12 +1218,13 @@
464 # An email communicating the rejection and the reason why it was
465 # rejected is sent to the uploaders.
466 contents = [
467- "Subject: [~name16/ubuntu/ppa] bar_1.0-1_source.changes rejected",
468+ "Subject: [~name16/ubuntu/ppa] bar_1.0-1_source.changes "
469+ "(Rejected)",
470 "Rejected:",
471 "PPA exceeded its size limit (2048.00 of 2048.00 MiB). "
472 "Ask a question in https://answers.launchpad.net/soyuz/ "
473 "if you need more space."]
474- self.assertEmail(contents)
475+ self.assertEmails([{"contents": contents}])
476
477 def testPPASizeNoQuota(self):
478 self.name16.archive.authorized_size = None
479@@ -1242,7 +1232,7 @@
480 self.processUpload(self.uploadprocessor, upload_dir)
481 contents = [
482 "Subject: [~name16/ubuntu/ppa/breezy] bar 1.0-1 (Accepted)"]
483- self.assertEmail(contents)
484+ self.assertEmails([{"contents": contents}])
485 self.assertEqual(
486 self.uploadprocessor.last_processed_upload.queue_root.status,
487 PackageUploadStatus.DONE)
488@@ -1266,7 +1256,7 @@
489 "PPA exceeded 95 % of its size limit (2000.00 of 2048.00 MiB). "
490 "Ask a question in https://answers.launchpad.net/soyuz/ "
491 "if you need more space."]
492- self.assertEmail(contents)
493+ self.assertEmails([{"contents": contents}])
494
495 # User was warned about quota limits but the source was accepted
496 # as informed in the upload notification.
497
498=== modified file 'lib/lp/archiveuploader/tests/test_sync_notification.py'
499--- lib/lp/archiveuploader/tests/test_sync_notification.py 2012-11-15 01:41:14 +0000
500+++ lib/lp/archiveuploader/tests/test_sync_notification.py 2015-08-26 13:41:53 +0000
501@@ -1,4 +1,4 @@
502-# Copyright 2012 Canonical Ltd. This software is licensed under the
503+# Copyright 2012-2015 Canonical Ltd. This software is licensed under the
504 # GNU Affero General Public License version 3 (see the file LICENSE).
505
506 """Test notification behaviour for cross-distro package syncs."""
507@@ -149,10 +149,10 @@
508 In a situation like that, we should not bother those people with the
509 failure. We notify the person who requested the sync instead.
510
511- (The logic in lp.soyuz.adapters.notification may still notify the
512- author of the last change, if that person is also an uploader for the
513- archive that the failure happened in. For this particular situation
514- we consider that not so much an intended behaviour, as an emergent one
515+ (The logic in lp.soyuz.mail.packageupload may still notify the author
516+ of the last change, if that person is also an uploader for the archive
517+ that the failure happened in. For this particular situation we
518+ consider that not so much an intended behaviour, as an emergent one
519 that does not seem inappropriate. It'd be hard to change if we wanted
520 to.)
521
522
523=== modified file 'lib/lp/archiveuploader/tests/test_uploadprocessor.py'
524--- lib/lp/archiveuploader/tests/test_uploadprocessor.py 2015-02-17 11:20:15 +0000
525+++ lib/lp/archiveuploader/tests/test_uploadprocessor.py 2015-08-26 13:41:53 +0000
526@@ -159,10 +159,8 @@
527 self.options.nomails = False
528 self.options.context = 'insecure'
529
530- # common recipients
531- self.kinnison_recipient = (
532- "Daniel Silverstone <daniel.silverstone@canonical.com>")
533- self.name16_recipient = "Foo Bar <foo.bar@canonical.com>"
534+ # common recipient
535+ self.name16_recipient = "foo.bar@canonical.com"
536
537 self.log = BufferLogger()
538
539@@ -344,50 +342,49 @@
540 self.switchToUploader()
541 return upload_processor
542
543- def assertEmail(self, contents=None, recipients=None):
544- """Check last email content and recipients.
545+ def assertEmails(self, expected, allow_leftover=False):
546+ """Check recent email content and recipients.
547
548- :param contents: A list of lines; assert that each is in the email.
549- :param recipients: A list of recipients that must be on the email.
550- Supply an empty list if you don't want them
551- checked. Default action is to check that the
552- recipient is foo.bar@canonical.com, which is the
553- signer on most of the test data uploads.
554+ :param expected: A list of dicts, each of which represents an
555+ expected email and may have "contents" and "recipient" keys.
556+ All the items in expected must match in the correct order, with
557+ none left over. "contents" is a list of lines; assert that each
558+ is in Subject + Body. "recipient" is the To address the email
559+ must have, defaulting to "foo.bar@canonical.com" which is the
560+ signer on most of the test data uploads; supply None if you
561+ don't want it checked.
562+ :param allow_leftover: If True, allow additional emails to be left
563+ over after checking the ones in expected.
564 """
565- if recipients is None:
566- recipients = [self.name16_recipient]
567- if contents is None:
568- contents = []
569-
570- self.assertEqual(
571- len(stub.test_emails), 1,
572- 'Unexpected number of emails sent: %s' % len(stub.test_emails))
573-
574- from_addr, to_addrs, raw_msg = stub.test_emails.pop()
575- msg = message_from_string(raw_msg)
576- # This is now a MIMEMultipart message.
577- body = msg.get_payload(0)
578- body = body.get_payload(decode=True)
579-
580- # Only check recipients if callsite didn't provide an empty list.
581- if recipients != []:
582- clean_recipients = [r.strip() for r in to_addrs]
583- for recipient in list(recipients):
584+
585+ for item in expected:
586+ recipient = item.get("recipient", self.name16_recipient)
587+ contents = item.get("contents", [])
588+
589+ from_addr, to_addrs, raw_msg = stub.test_emails.pop()
590+ msg = message_from_string(raw_msg)
591+ # This is now a MIMEMultipart message.
592+ body = msg.get_payload(0)
593+ body = body.get_payload(decode=True)
594+
595+ # Only check the recipient if the caller didn't explicitly pass
596+ # "recipient": None.
597+ if recipient is not None:
598+ clean_recipients = [r.strip() for r in to_addrs]
599+ self.assertContentEqual([recipient], clean_recipients)
600+
601+ subject = "Subject: %s\n" % msg['Subject']
602+ body = subject + body
603+
604+ for content in list(contents):
605 self.assertTrue(
606- recipient in clean_recipients,
607- "%s not found in %s" % (recipients, clean_recipients))
608+ content in body,
609+ "Expect: '%s'\nGot:\n%s" % (content, body))
610+
611+ if not allow_leftover:
612 self.assertEqual(
613- len(recipients), len(clean_recipients),
614- "Email recipients do not match exactly. Expected %s, got %s" %
615- (recipients, clean_recipients))
616-
617- subject = "Subject: %s\n" % msg['Subject']
618- body = subject + body
619-
620- for content in list(contents):
621- self.assertTrue(
622- content in body,
623- "Expect: '%s'\nGot:\n%s" % (content, body))
624+ [], stub.test_emails,
625+ "%d emails left over" % len(stub.test_emails))
626
627 def PGPSignatureNotPreserved(self, archive=None):
628 """PGP signatures should be removed from .changes files.
629@@ -427,7 +424,7 @@
630 def _checkPartnerUploadEmailSuccess(self):
631 """Ensure partner uploads generate the right email."""
632 from_addr, to_addrs, raw_msg = stub.test_emails.pop()
633- foo_bar = "Foo Bar <foo.bar@canonical.com>"
634+ foo_bar = "foo.bar@canonical.com"
635 self.assertEqual([e.strip() for e in to_addrs], [foo_bar])
636 self.assertTrue(
637 "rejected" not in raw_msg,
638@@ -572,8 +569,7 @@
639 body = msg.get_payload(0)
640 body = body.get_payload(decode=True)
641
642- daniel = "Daniel Silverstone <daniel.silverstone@canonical.com>"
643- self.assertEqual(to_addrs, [daniel])
644+ self.assertEqual(["daniel.silverstone@canonical.com"], to_addrs)
645 self.assertTrue("Unhandled exception processing upload: Exception "
646 "raised by BrokenUploadPolicy for testing."
647 in body)
648@@ -603,14 +599,15 @@
649 self.processUpload(uploadprocessor, upload_dir)
650
651 # Check it went ok to the NEW queue and all is going well so far.
652- from_addr, to_addrs, raw_msg = stub.test_emails.pop()
653- to_addrs = [e.strip() for e in to_addrs]
654- foo_bar = "Foo Bar <foo.bar@canonical.com>"
655- daniel = "Daniel Silverstone <daniel.silverstone@canonical.com>"
656- self.assertContentEqual(to_addrs, [foo_bar, daniel])
657- self.assertTrue(
658- "NEW" in raw_msg, "Expected email containing 'NEW', got:\n%s"
659- % raw_msg)
660+ foo_bar = "foo.bar@canonical.com"
661+ daniel = "daniel.silverstone@canonical.com"
662+ for expected_to_addr in foo_bar, daniel:
663+ from_addr, to_addrs, raw_msg = stub.test_emails.pop()
664+ to_addrs = [e.strip() for e in to_addrs]
665+ self.assertContentEqual(to_addrs, [expected_to_addr])
666+ self.assertTrue(
667+ "NEW" in raw_msg, "Expected email containing 'NEW', got:\n%s"
668+ % raw_msg)
669
670 # Accept and publish the upload.
671 # This is required so that the next upload of a later version of
672@@ -638,14 +635,13 @@
673 self.processUpload(uploadprocessor, upload_dir)
674
675 # Verify we get an email talking about awaiting approval.
676- from_addr, to_addrs, raw_msg = stub.test_emails.pop()
677- to_addrs = [e.strip() for e in to_addrs]
678- daniel = "Daniel Silverstone <daniel.silverstone@canonical.com>"
679- foo_bar = "Foo Bar <foo.bar@canonical.com>"
680- self.assertContentEqual(to_addrs, [foo_bar, daniel])
681- self.assertTrue("Waiting for approval" in raw_msg,
682- "Expected an 'upload awaits approval' email.\n"
683- "Got:\n%s" % raw_msg)
684+ for expected_to_addr in foo_bar, daniel:
685+ from_addr, to_addrs, raw_msg = stub.test_emails.pop()
686+ to_addrs = [e.strip() for e in to_addrs]
687+ self.assertContentEqual(to_addrs, [expected_to_addr])
688+ self.assertTrue("Waiting for approval" in raw_msg,
689+ "Expected an 'upload awaits approval' email.\n"
690+ "Got:\n%s" % raw_msg)
691
692 # And verify that the queue item is in the unapproved state.
693 queue_items = self.breezy.getPackageUploads(
694@@ -942,7 +938,7 @@
695
696 # Check that it was rejected.
697 from_addr, to_addrs, raw_msg = stub.test_emails.pop()
698- foo_bar = "Foo Bar <foo.bar@canonical.com>"
699+ foo_bar = "foo.bar@canonical.com"
700 self.assertEqual([e.strip() for e in to_addrs], [foo_bar])
701 self.assertTrue(
702 "Cannot mix partner files with non-partner." in raw_msg,
703@@ -1049,10 +1045,10 @@
704 build_uploadprocessor, upload_dir, build=foocomm_build)
705
706 contents = [
707- "Subject: [ubuntu/partner] foocomm_1.0-1_i386.changes rejected",
708+ "Subject: [ubuntu/partner] foocomm_1.0-1_i386.changes (Rejected)",
709 "Attempt to upload binaries specifying build 31, "
710 "where they don't fit."]
711- self.assertEmail(contents)
712+ self.assertEmails([{"contents": contents}])
713
714 # Reset upload queue directory for a new upload.
715 shutil.rmtree(upload_dir)
716@@ -1292,7 +1288,8 @@
717 # Check that the sourceful upload to the copy archive is rejected.
718 contents = [
719 "Invalid upload path (1/ubuntu) for this policy (insecure)"]
720- self.assertEmail(contents=contents, recipients=[])
721+ self.assertEmails(
722+ [{"contents": contents, "recipient": None}], allow_leftover=True)
723
724 # Uploads that are new should have the component overridden
725 # such that:
726@@ -1689,7 +1686,7 @@
727 from_addr, to_addrs, raw_msg = stub.test_emails.pop()
728 msg = message_from_string(raw_msg)
729 self.assertEqual(
730- msg['Subject'], '[ubuntu] bar_1.0-2_source.changes rejected')
731+ msg['Subject'], '[ubuntu] bar_1.0-2_source.changes (Rejected)')
732
733 # Grant the permissions in the proper series.
734 self.switchToAdmin()
735@@ -1716,7 +1713,7 @@
736 def testUploadPathErrorIntendedForHumans(self):
737 # Distribution upload path errors are augmented with a hint
738 # to fix the current dput/dupload configuration.
739- # This information gets included in the rejection email along
740+ # This information gets included in the rejection emails along
741 # with pointer to the Soyuz questions in Launchpad and the
742 # reason why the message was sent to the current recipients.
743 self.setupBreezy()
744@@ -1741,20 +1738,26 @@
745 ],
746 rejection_message.splitlines())
747
748- contents = [
749- "Subject: [ubuntu] bar_1.0-1_source.changes rejected",
750+ base_contents = [
751+ "Subject: [ubuntu] bar_1.0-1_source.changes (Rejected)",
752 "Could not find distribution 'boing'",
753 "If you don't understand why your files were rejected",
754 "http://answers.launchpad.net/soyuz",
755- "You are receiving this email because you are the "
756- "uploader, maintainer or",
757- "signer of the above package.",
758- ]
759- recipients = [
760- 'Foo Bar <foo.bar@canonical.com>',
761- 'Daniel Silverstone <daniel.silverstone@canonical.com>',
762- ]
763- self.assertEmail(contents, recipients=recipients)
764+ ]
765+ expected = []
766+ expected.append({
767+ "contents": base_contents + [
768+ "You are receiving this email because you made this upload."],
769+ "recipient": "foo.bar@canonical.com",
770+ })
771+ expected.append({
772+ "contents": base_contents + [
773+ "You are receiving this email because you are the most "
774+ "recent person",
775+ "listed in this package's changelog."],
776+ "recipient": "daniel.silverstone@canonical.com",
777+ })
778+ self.assertEmails(expected)
779
780 def test30QuiltUploadToUnsupportingSeriesIsRejected(self):
781 """Ensure that uploads to series without format support are rejected.
782@@ -1909,21 +1912,27 @@
783 "the 'CURRENT' state.",
784 rejection_message)
785
786- contents = [
787- "Subject: [ubuntu] bar_1.0-1_source.changes rejected",
788+ base_contents = [
789+ "Subject: [ubuntu] bar_1.0-1_source.changes (Rejected)",
790 "Not permitted to upload to the RELEASE pocket in a series "
791 "in the 'CURRENT' state.",
792 "If you don't understand why your files were rejected",
793 "http://answers.launchpad.net/soyuz",
794- "You are receiving this email because you are the "
795- "uploader, maintainer or",
796- "signer of the above package.",
797- ]
798- recipients = [
799- 'Foo Bar <foo.bar@canonical.com>',
800- 'Daniel Silverstone <daniel.silverstone@canonical.com>',
801- ]
802- self.assertEmail(contents, recipients=recipients)
803+ ]
804+ expected = []
805+ expected.append({
806+ "contents": base_contents + [
807+ "You are receiving this email because you made this upload."],
808+ "recipient": "foo.bar@canonical.com",
809+ })
810+ expected.append({
811+ "contents": base_contents + [
812+ "You are receiving this email because you are the most "
813+ "recent person",
814+ "listed in this package's changelog."],
815+ "recipient": "daniel.silverstone@canonical.com",
816+ })
817+ self.assertEmails(expected)
818
819 def testPGPSignatureNotPreserved(self):
820 """PGP signatures should be removed from .changes files.
821@@ -1998,9 +2007,11 @@
822 self.switchToUploader()
823 upload_dir = self.queueUpload("bar_1.0-1")
824 self.processUpload(uploadprocessor, upload_dir)
825- self.assertEmail(
826- contents=["Redirecting ubuntu breezy to ubuntu breezy-proposed."],
827- recipients=[])
828+ self.assertEmails([{
829+ "contents":
830+ ["Redirecting ubuntu breezy to ubuntu breezy-proposed."],
831+ "recipient": None,
832+ }], allow_leftover=True)
833 [queue_item] = self.breezy.getPackageUploads(
834 status=PackageUploadStatus.NEW, name=u"bar",
835 version=u"1.0-1", exact_match=True)
836@@ -2010,9 +2021,11 @@
837 pop_notifications()
838 upload_dir = self.queueUpload("bar_1.0-2")
839 self.processUpload(uploadprocessor, upload_dir)
840- self.assertEmail(
841- contents=["Redirecting ubuntu breezy to ubuntu breezy-proposed."],
842- recipients=[])
843+ self.assertEmails([{
844+ "contents":
845+ ["Redirecting ubuntu breezy to ubuntu breezy-proposed."],
846+ "recipient": None,
847+ }], allow_leftover=True)
848 [queue_item] = self.breezy.getPackageUploads(
849 status=PackageUploadStatus.DONE, name=u"bar",
850 version=u"1.0-2", exact_match=True)
851@@ -2271,7 +2284,7 @@
852 status=PackageUploadStatus.ACCEPTED,
853 version=u"1.0-1", name=u"bar")
854 queue_item.setDone()
855- stub.test_emails.pop()
856+ stub.test_emails = []
857
858 build.buildqueue_record.markAsBuilding(self.factory.makeBuilder())
859 build.updateStatus(status)
860
861=== modified file 'lib/lp/services/mail/basemailer.py'
862--- lib/lp/services/mail/basemailer.py 2015-08-23 22:53:55 +0000
863+++ lib/lp/services/mail/basemailer.py 2015-08-26 13:41:53 +0000
864@@ -135,6 +135,10 @@
865 """
866 pass
867
868+ def _getTemplateName(self, email, recipient):
869+ """Return the name of the template to use for this email body."""
870+ return self._template_name
871+
872 def _getTemplateParams(self, email, recipient):
873 """Return a dict of values to use in the body and subject."""
874 reason, rationale = self._recipients.getReason(email)
875@@ -150,7 +154,8 @@
876
877 def _getBody(self, email, recipient):
878 """Return the complete body to use for this email."""
879- template = get_email_template(self._template_name, app=self.app)
880+ template = get_email_template(
881+ self._getTemplateName(email, recipient), app=self.app)
882 params = self._getTemplateParams(email, recipient)
883 body = template % params
884 footer = self._getFooter(email, recipient, params)
885
886=== modified file 'lib/lp/services/mail/notificationrecipientset.py'
887--- lib/lp/services/mail/notificationrecipientset.py 2015-07-08 16:05:11 +0000
888+++ lib/lp/services/mail/notificationrecipientset.py 2015-08-26 13:41:53 +0000
889@@ -1,4 +1,4 @@
890-# Copyright 2009-2011 Canonical Ltd. This software is licensed under the
891+# Copyright 2009-2015 Canonical Ltd. This software is licensed under the
892 # GNU Affero General Public License version 3 (see the file LICENSE).
893
894 """The implementation of the Notification Rec."""
895@@ -6,6 +6,7 @@
896 __metaclass__ = type
897 __all__ = [
898 'NotificationRecipientSet',
899+ 'StubPerson',
900 ]
901
902
903@@ -21,6 +22,22 @@
904 )
905
906
907+class StubPerson:
908+ """A stub recipient person.
909+
910+ This can be used when sending to special email addresses that do not
911+ correspond to a real Person.
912+ """
913+
914+ displayname = None
915+ is_team = False
916+ expanded_notification_footers = False
917+
918+ def __init__(self, email):
919+ self.preferredemail = type(
920+ "StubEmailAddress", (object,), {"email": email})
921+
922+
923 @implementer(INotificationRecipientSet)
924 class NotificationRecipientSet:
925 """Set of recipients along the rationale for being in the set."""
926@@ -87,17 +104,24 @@
927 """See `INotificationRecipientSet`."""
928 from zope.security.proxy import removeSecurityProxy
929 from lp.registry.model.person import get_recipients
930- if IPerson.providedBy(persons):
931+ if (IPerson.providedBy(persons) or
932+ zope_isinstance(persons, StubPerson)):
933 persons = [persons]
934
935 for person in persons:
936- assert IPerson.providedBy(person), (
937- 'You can only add() an IPerson: %r' % person)
938+ assert (
939+ IPerson.providedBy(person) or
940+ zope_isinstance(person, StubPerson)), (
941+ 'You can only add() an IPerson or a StubPerson: %r' % person)
942 # If the person already has a rationale, keep the first one.
943 if person in self._personToRationale:
944 continue
945 self._personToRationale[person] = reason, header
946- for receiving_person in get_recipients(person):
947+ if IPerson.providedBy(person):
948+ recipients = get_recipients(person)
949+ else:
950+ recipients = [person]
951+ for receiving_person in recipients:
952 # Bypass zope's security because IEmailAddress.email is not
953 # public.
954 preferred_email = removeSecurityProxy(
955
956=== modified file 'lib/lp/soyuz/doc/distroseriesqueue-notify.txt'
957--- lib/lp/soyuz/doc/distroseriesqueue-notify.txt 2013-07-25 11:58:55 +0000
958+++ lib/lp/soyuz/doc/distroseriesqueue-notify.txt 2015-08-26 13:41:53 +0000
959@@ -59,7 +59,8 @@
960 ...
961 DEBUG above if files already exist in other distroseries.
962 ...
963- DEBUG signer of the above package.
964+ DEBUG You are receiving this email because you are the most recent person
965+ DEBUG listed in this package's changelog.
966
967 Helper functions to examine emails that were sent:
968
969@@ -99,8 +100,8 @@
970 <BLANKLINE>
971 -- =
972 <BLANKLINE>
973- You are receiving this email because you are the uploader, maintainer or
974- signer of the above package.
975+ You are receiving this email because you are the most recent person
976+ listed in this package's changelog.
977 <BLANKLINE>
978
979 Now we will process a signed package. Signed packages will potentially
980@@ -133,15 +134,17 @@
981 DEBUG Sent a mail:
982 ...
983
984-There are two emails, the upload acknowledgement and the announcement,
985-because this upload is already accepted.
986+There are three emails, the upload acknowledgement to the changer, the
987+upload acknowledgement to the signer, and the announcement, because this
988+upload is already accepted.
989
990 >>> msgs = pop_notifications()
991 >>> len(msgs)
992- 2
993+ 3
994
995-The mail 'To:' addresses contain the signer and the changer's email.
996-The announcement email contains the serieses changeslist.
997+The two upload acknowledgements contain the changer's email and the signer's
998+email in their respective 'To:' headers.
999+The announcement email contains the series's changeslist.
1000
1001 >>> def to_lower(address):
1002 ... """Return lower-case version of email address."""
1003@@ -153,12 +156,10 @@
1004 ... [addr.strip() for addr in header_field.split(',')],
1005 ... key=to_lower)
1006
1007- >>> for addr in extract_addresses(msgs[0]['To']):
1008- ... print addr
1009+ >>> for msg in msgs:
1010+ ... print msg['To']
1011 Daniel Silverstone <daniel.silverstone@canonical.com>
1012 Foo Bar <foo.bar@canonical.com>
1013-
1014- >>> print msgs[1]['To']
1015 autotest_changes@ubuntu.com
1016
1017 The mail 'Bcc:' address is the uploader. The announcement has the
1018@@ -167,35 +168,36 @@
1019 >>> for msg in msgs:
1020 ... print extract_addresses(msg['Bcc'])
1021 ['Root <root@localhost>']
1022+ ['Root <root@localhost>']
1023 ['netapplet_derivatives@packages.qa.debian.org', 'Root <root@localhost>']
1024
1025-The mail 'From:' addresses are the uploader and the changer.
1026+The mail 'From:' addresses are the uploader (for acknowledgements sent to
1027+the uploader and the changer) and the changer.
1028
1029 >>> for msg in msgs:
1030 ... print msg['From']
1031 Root <root@localhost>
1032+ Root <root@localhost>
1033 Daniel Silverstone <daniel.silverstone@canonical.com>
1034
1035- >>> print notification['Subject']
1036- [ubuntu/breezy-autotest] netapplet 0.99.6-1 (New)
1037+ >>> print msgs[0]['Subject']
1038+ [ubuntu/breezy-autotest] netapplet 0.99.6-1 (Accepted)
1039
1040 The mail body contains the same list of files again:
1041
1042- >>> print notification.get_payload(0) # doctest: -NORMALIZE_WHITESPACE
1043+ >>> print msgs[0].get_payload(0) # doctest: -NORMALIZE_WHITESPACE
1044 From nobody ...
1045 ...
1046- NEW: netapplet_1.0-1.dsc
1047- NEW: netapplet_1.0.orig.tar.gz
1048- NEW: netapplet_1.0-1.diff.gz
1049+ OK: netapplet_1.0-1.dsc
1050+ -> Component: main Section: web
1051+ OK: netapplet_1.0.orig.tar.gz
1052+ OK: netapplet_1.0-1.diff.gz
1053 <BLANKLINE>
1054 ...
1055- You may have gotten the distroseries wrong. If so, you may get warnings
1056- above if files already exist in other distroseries.
1057- <BLANKLINE>
1058 -- =
1059 <BLANKLINE>
1060- You are receiving this email because you are the uploader, maintainer or
1061- signer of the above package.
1062+ You are receiving this email because you are the most recent person
1063+ listed in this package's changelog.
1064 <BLANKLINE>
1065
1066 notify() will also work without passing the changes_file_object
1067@@ -216,46 +218,75 @@
1068 ...
1069 DEBUG Sent a mail:
1070 ...
1071- DEBUG Recipients: ... Silverstone ...
1072- ...
1073- DEBUG above if files already exist in other distroseries.
1074- ...
1075- DEBUG signer of the above package.
1076-
1077-Only one email is generated:
1078-
1079- >>> [notification] = pop_notifications()
1080+ DEBUG Recipients: ... Silverstone ...
1081+ ...
1082+ DEBUG above if files already exist in other distroseries.
1083+ ...
1084+ DEBUG You are receiving this email because you are the most recent person
1085+ DEBUG listed in this package's changelog.
1086+ DEBUG Sent a mail:
1087+ ...
1088+ DEBUG Recipients: ... Bar ...
1089+ ...
1090+ DEBUG above if files already exist in other distroseries.
1091+ ...
1092+ DEBUG You are receiving this email because you made this upload.
1093+
1094+Two emails are generated, one to the changer and one to the signer:
1095+
1096+ >>> [changer_notification, signer_notification] = pop_notifications()
1097
1098 The mail headers are the same as before:
1099
1100- >>> for addr in extract_addresses(notification['To']):
1101- ... print addr
1102+ >>> print changer_notification['To']
1103 Daniel Silverstone <daniel.silverstone@canonical.com>
1104+ >>> print signer_notification['To']
1105 Foo Bar <foo.bar@canonical.com>
1106
1107- >>> print notification['Bcc']
1108+ >>> print changer_notification['Bcc']
1109+ Root <root@localhost>
1110+ >>> print signer_notification['Bcc']
1111 Root <root@localhost>
1112
1113- >>> print notification['Subject']
1114+ >>> print changer_notification['Subject']
1115+ [ubuntu/breezy-autotest] netapplet 0.99.6-1 (New)
1116+ >>> print signer_notification['Subject']
1117 [ubuntu/breezy-autotest] netapplet 0.99.6-1 (New)
1118
1119 The mail body contains the same list of files again:
1120
1121- >>> print notification.get_payload(0) # doctest: -NORMALIZE_WHITESPACE
1122- From nobody ...
1123- ...
1124- NEW: netapplet_1.0-1.dsc
1125- NEW: netapplet_1.0.orig.tar.gz
1126- NEW: netapplet_1.0-1.diff.gz
1127- <BLANKLINE>
1128- ...
1129- You may have gotten the distroseries wrong. If so, you may get warnings
1130- above if files already exist in other distroseries.
1131- <BLANKLINE>
1132- -- =
1133- <BLANKLINE>
1134- You are receiving this email because you are the uploader, maintainer or
1135- signer of the above package.
1136+ >>> print changer_notification.get_payload(0)
1137+ ... # doctest: -NORMALIZE_WHITESPACE
1138+ From nobody ...
1139+ ...
1140+ NEW: netapplet_1.0-1.dsc
1141+ NEW: netapplet_1.0.orig.tar.gz
1142+ NEW: netapplet_1.0-1.diff.gz
1143+ <BLANKLINE>
1144+ ...
1145+ You may have gotten the distroseries wrong. If so, you may get warnings
1146+ above if files already exist in other distroseries.
1147+ <BLANKLINE>
1148+ -- =
1149+ <BLANKLINE>
1150+ You are receiving this email because you are the most recent person
1151+ listed in this package's changelog.
1152+ <BLANKLINE>
1153+ >>> print signer_notification.get_payload(0)
1154+ ... # doctest: -NORMALIZE_WHITESPACE
1155+ From nobody ...
1156+ ...
1157+ NEW: netapplet_1.0-1.dsc
1158+ NEW: netapplet_1.0.orig.tar.gz
1159+ NEW: netapplet_1.0-1.diff.gz
1160+ <BLANKLINE>
1161+ ...
1162+ You may have gotten the distroseries wrong. If so, you may get warnings
1163+ above if files already exist in other distroseries.
1164+ <BLANKLINE>
1165+ -- =
1166+ <BLANKLINE>
1167+ You are receiving this email because you made this upload.
1168 <BLANKLINE>
1169
1170 notify() will also generate rejection notices if the upload failed. The
1171@@ -268,7 +299,19 @@
1172 ... summary_text="Testing rejection message", logger=FakeLogger())
1173 DEBUG Building recipients list.
1174 ...
1175- DEBUG Sending rejection email.
1176+ DEBUG Sent a mail:
1177+ DEBUG Subject: [ubuntu/breezy-autotest] netapplet 0.99.6-1 (Rejected)
1178+ DEBUG Sender: Root <root@localhost>
1179+ DEBUG Recipients: ... Silverstone ...
1180+ DEBUG Bcc: Root <root@localhost>
1181+ DEBUG Body:
1182+ DEBUG Rejected:
1183+ DEBUG Testing rejection message
1184+ ...
1185+ DEBUG If you don't understand why your files were rejected, or if the
1186+ ...
1187+ DEBUG You are receiving this email because you are the most recent person
1188+ DEBUG listed in this package's changelog.
1189 ...
1190 DEBUG Subject: [ubuntu/breezy-autotest] netapplet 0.99.6-1 (Rejected)
1191 DEBUG Sender: Root <root@localhost>
1192@@ -280,13 +323,13 @@
1193 ...
1194 DEBUG If you don't understand why your files were rejected, or if the
1195 ...
1196- DEBUG signer of the above package.
1197+ DEBUG You are receiving this email because you made this upload.
1198
1199-Only one email is generated:
1200+Two emails are generated:
1201
1202 >>> transaction.commit()
1203 >>> len(stub.test_emails)
1204- 1
1205+ 2
1206
1207 Clean up, otherwise stuff is left lying around in /var/tmp.
1208
1209
1210=== modified file 'lib/lp/soyuz/doc/soyuz-set-of-uploads.txt'
1211--- lib/lp/soyuz/doc/soyuz-set-of-uploads.txt 2015-07-29 05:56:50 +0000
1212+++ lib/lp/soyuz/doc/soyuz-set-of-uploads.txt 2015-08-26 13:41:53 +0000
1213@@ -304,10 +304,11 @@
1214 Rejected uploads: ['bar_1.0-3']
1215
1216 >>> read_email()
1217- To:
1218- Daniel Silverstone <daniel.silverstone@canonical.com>,
1219- Foo Bar <foo.bar@canonical.com>
1220- Subject: [ubuntutest] bar_1.0-3_source.changes rejected
1221+ To: Daniel Silverstone <daniel.silverstone@canonical.com>
1222+ Subject: [ubuntutest] bar_1.0-3_source.changes (Rejected)
1223+ ...
1224+ To: Foo Bar <foo.bar@canonical.com>
1225+ Subject: [ubuntutest] bar_1.0-3_source.changes (Rejected)
1226 ...
1227
1228 Force weird behaviour with rfc2047 sentences containing '.' on
1229@@ -321,10 +322,8 @@
1230 '.', must be rfc2047 compliant:
1231
1232 >>> simulate_upload('bar_1.0-4')
1233- >>> uninteresting_email = stub.test_emails.pop()
1234 >>> read_email()
1235- To: "Foo B. Bar" <foo.bar@canonical.com>,
1236- Celso Providelo <celso.providelo@canonical.com>
1237+ To: "Foo B. Bar" <foo.bar@canonical.com>
1238 Subject: [ubuntutest/breezy] bar 1.0-4 (Accepted)
1239 Content-Type: text/plain; charset="utf-8"
1240 MIME-Version: 1.0
1241@@ -353,10 +352,13 @@
1242 <BLANKLINE>
1243 -- =
1244 <BLANKLINE>
1245- You are receiving this email because you are the uploader, maintainer or
1246- signer of the above package.
1247- <BLANKLINE>
1248- <BLANKLINE>
1249+ You are receiving this email because you made this upload.
1250+ <BLANKLINE>
1251+ <BLANKLINE>
1252+ To: Celso Providelo <celso.providelo@canonical.com>
1253+ ...
1254+ To: breezy-changes@ubuntu.com
1255+ ...
1256
1257 Revert changes:
1258
1259@@ -468,9 +470,8 @@
1260 DEBUG above if files already exist in other distroseries.
1261 DEBUG
1262 DEBUG --
1263- DEBUG You are receiving this email because you are the uploader,
1264- maintainer or
1265- DEBUG signer of the above package.
1266+ DEBUG You are receiving this email because you are the most recent person
1267+ DEBUG listed in this package's changelog.
1268 INFO Committing the transaction and any mails associated with this
1269 upload.
1270 ...
1271
1272=== modified file 'lib/lp/soyuz/emailtemplates/ppa-upload-accepted.txt'
1273--- lib/lp/soyuz/emailtemplates/ppa-upload-accepted.txt 2012-02-10 09:31:39 +0000
1274+++ lib/lp/soyuz/emailtemplates/ppa-upload-accepted.txt 2015-08-26 13:41:53 +0000
1275@@ -2,7 +2,3 @@
1276 %(SUMMARY)s
1277
1278 %(CHANGESFILE)s
1279-
1280---%(ARCHIVE_URL)s
1281-You are receiving this email because you are the uploader of the above
1282-PPA package.
1283
1284=== modified file 'lib/lp/soyuz/emailtemplates/ppa-upload-rejection.txt'
1285--- lib/lp/soyuz/emailtemplates/ppa-upload-rejection.txt 2012-02-10 09:31:39 +0000
1286+++ lib/lp/soyuz/emailtemplates/ppa-upload-rejection.txt 2015-08-26 13:41:53 +0000
1287@@ -7,7 +7,3 @@
1288
1289 If you don't understand why your files were rejected please send an email
1290 to %(USERS_ADDRESS)s for help (requires membership).
1291-
1292---%(ARCHIVE_URL)s
1293-You are receiving this email because you are the uploader of the above
1294-PPA package.
1295
1296=== modified file 'lib/lp/soyuz/emailtemplates/upload-accepted.txt'
1297--- lib/lp/soyuz/emailtemplates/upload-accepted.txt 2012-10-24 09:43:58 +0000
1298+++ lib/lp/soyuz/emailtemplates/upload-accepted.txt 2015-08-26 13:41:53 +0000
1299@@ -10,7 +10,3 @@
1300 %(ANNOUNCE)s
1301
1302 Thank you for your contribution to %(DISTRO)s.
1303-
1304---
1305-You are receiving this email because you are the uploader, maintainer or
1306-signer of the above package.
1307
1308=== modified file 'lib/lp/soyuz/emailtemplates/upload-new.txt'
1309--- lib/lp/soyuz/emailtemplates/upload-new.txt 2011-12-18 23:30:56 +0000
1310+++ lib/lp/soyuz/emailtemplates/upload-new.txt 2015-08-26 13:41:53 +0000
1311@@ -9,7 +9,3 @@
1312
1313 You may have gotten the distroseries wrong. If so, you may get warnings
1314 above if files already exist in other distroseries.
1315-
1316---
1317-You are receiving this email because you are the uploader, maintainer or
1318-signer of the above package.
1319
1320=== modified file 'lib/lp/soyuz/emailtemplates/upload-rejection.txt'
1321--- lib/lp/soyuz/emailtemplates/upload-rejection.txt 2011-12-18 23:30:56 +0000
1322+++ lib/lp/soyuz/emailtemplates/upload-rejection.txt 2015-08-26 13:41:53 +0000
1323@@ -10,7 +10,3 @@
1324 If you don't understand why your files were rejected, or if the
1325 override file requires editing, please go to:
1326 http://answers.launchpad.net/soyuz
1327-
1328---
1329-You are receiving this email because you are the uploader, maintainer or
1330-signer of the above package.
1331
1332=== renamed file 'lib/lp/soyuz/adapters/notification.py' => 'lib/lp/soyuz/mail/packageupload.py'
1333--- lib/lp/soyuz/adapters/notification.py 2015-07-29 06:58:37 +0000
1334+++ lib/lp/soyuz/mail/packageupload.py 2015-08-26 13:41:53 +0000
1335@@ -1,20 +1,16 @@
1336 # Copyright 2011-2015 Canonical Ltd. This software is licensed under the
1337 # GNU Affero General Public License version 3 (see the file LICENSE).
1338
1339-"""Notification for uploads and copies."""
1340-
1341 __metaclass__ = type
1342-
1343 __all__ = [
1344- 'notify',
1345+ 'PackageUploadMailer',
1346 ]
1347
1348-
1349-from email.mime.multipart import MIMEMultipart
1350-from email.mime.text import MIMEText
1351-import os
1352+from collections import OrderedDict
1353+import os.path
1354
1355 from zope.component import getUtility
1356+from zope.security.proxy import isinstance as zope_isinstance
1357
1358 from lp.app.interfaces.launchpad import ILaunchpadCelebrities
1359 from lp.archivepublisher.utils import get_ppa_reference
1360@@ -28,67 +24,188 @@
1361 from lp.registry.interfaces.pocket import PackagePublishingPocket
1362 from lp.services.config import config
1363 from lp.services.encoding import guess as guess_encoding
1364-from lp.services.mail.helpers import get_email_template
1365+from lp.services.mail.basemailer import (
1366+ BaseMailer,
1367+ RecipientReason,
1368+ )
1369+from lp.services.mail.mailwrapper import MailWrapper
1370+from lp.services.mail.notificationrecipientset import StubPerson
1371 from lp.services.mail.sendmail import (
1372 format_address,
1373 format_address_for_person,
1374- sendmail,
1375 )
1376 from lp.services.webapp import canonical_url
1377 from lp.soyuz.interfaces.archivepermission import IArchivePermissionSet
1378
1379
1380-def reject_changes_file(blamer, changes_file_path, changes, archive,
1381- distroseries, reason, logger=None):
1382- """Notify about a rejection where all of the details are not known.
1383-
1384- :param blamer: The `IPerson` that is to blame for this notification.
1385- :param changes_file_path: The path to the changes file.
1386- :param changes: A dictionary of the parsed changes file.
1387- :param archive: The `IArchive` the notification is regarding.
1388- :param distroseries: The `IDistroSeries` the notification is regarding.
1389- :param reason: The reason for the rejection.
1390+class AnnouncementStubPerson(StubPerson):
1391+ """Marker class for sending announcements to distroseries changes lists."""
1392+
1393+
1394+class PackageUploadRecipientReason(RecipientReason):
1395+
1396+ @classmethod
1397+ def forRequester(cls, requester, recipient):
1398+ header = cls.makeRationale("Requester", requester)
1399+ # This is a little vague - copies may end up here too - but it's
1400+ # close enough.
1401+ reason = "You are receiving this email because you made this upload."
1402+ return cls(requester, recipient, header, reason)
1403+
1404+ @classmethod
1405+ def forMaintainer(cls, maintainer, recipient):
1406+ header = cls.makeRationale("Maintainer", maintainer)
1407+ reason = (
1408+ "You are receiving this email because %(lc_entity_is)s listed as "
1409+ "this package's maintainer.")
1410+ return cls(maintainer, recipient, header, reason)
1411+
1412+ @classmethod
1413+ def forChangedBy(cls, changed_by, recipient):
1414+ header = cls.makeRationale("Changed-By", changed_by)
1415+ reason = (
1416+ "You are receiving this email because %(lc_entity_is)s the most "
1417+ "recent person listed in this package's changelog.")
1418+ return cls(changed_by, recipient, header, reason)
1419+
1420+ @classmethod
1421+ def forPPAUploader(cls, uploader, recipient):
1422+ header = cls.makeRationale("PPA Uploader", uploader)
1423+ reason = (
1424+ "You are receiving this email because %(lc_entity_has)s upload "
1425+ "permissions to this PPA.")
1426+ return cls(uploader, recipient, header, reason)
1427+
1428+ @classmethod
1429+ def forAnnouncement(cls, recipient):
1430+ return cls(recipient, recipient, "Announcement", "")
1431+
1432+ def _getTemplateValues(self):
1433+ template_values = super(
1434+ PackageUploadRecipientReason, self)._getTemplateValues()
1435+ template_values["lc_entity_has"] = "you have"
1436+ if self.recipient != self.subscriber or self.subscriber.is_team:
1437+ template_values["lc_entity_has"] = (
1438+ "your team %s has" % self.subscriber.displayname)
1439+ return template_values
1440+
1441+ def getReason(self):
1442+ """See `RecipientReason`."""
1443+ return MailWrapper(width=72).format(
1444+ super(PackageUploadRecipientReason, self).getReason())
1445+
1446+
1447+def debug(logger, msg, *args, **kwargs):
1448+ """Shorthand debug notation."""
1449+ if logger is not None:
1450+ logger.debug(msg, *args, **kwargs)
1451+
1452+
1453+def sanitize_string(s):
1454+ """Make sure string does not trigger 'ascii' codec errors.
1455+
1456+ Convert string to unicode if needed so that characters outside
1457+ the (7-bit) ASCII range do not cause errors like these:
1458+
1459+ 'ascii' codec can't decode byte 0xc4 in position 21: ordinal
1460+ not in range(128)
1461 """
1462- ignored, filename = os.path.split(changes_file_path)
1463- information = {
1464- 'SUMMARY': reason,
1465- 'CHANGESFILE': '',
1466- 'DATE': '',
1467- 'CHANGEDBY': '',
1468- 'MAINTAINER': '',
1469- 'SIGNER': '',
1470- 'ORIGIN': '',
1471- 'ARCHIVE_URL': '',
1472- 'USERS_ADDRESS': config.launchpad.users_address,
1473- }
1474- subject = '%s rejected' % filename
1475- if archive:
1476- subject = '[%s] %s' % (archive.reference, subject)
1477- information['ARCHIVE_URL'] = '\n%s' % canonical_url(archive)
1478- template = get_template(archive, 'rejected')
1479- body = template % information
1480- to_addrs = get_upload_notification_recipients(
1481- blamer, archive, distroseries, logger, changes=changes)
1482- debug(logger, "Sending rejection email.")
1483- if not to_addrs:
1484- debug(logger, "No recipients have a preferred email.")
1485+ if isinstance(s, unicode):
1486+ return s
1487+ else:
1488+ return guess_encoding(s)
1489+
1490+
1491+def add_recipient(recipients, person, reason_factory, logger=None):
1492+ # Circular import.
1493+ from lp.registry.model.person import get_recipients
1494+
1495+ if person is None:
1496 return
1497- send_mail(None, archive, to_addrs, subject, body, False, logger=logger)
1498-
1499-
1500-def get_template(archive, action):
1501- """Return the appropriate email template."""
1502- template_name = 'upload-'
1503- if action in ('new', 'accepted', 'announcement'):
1504- template_name += action
1505- elif action == 'unapproved':
1506- template_name += 'accepted'
1507- elif action == 'rejected':
1508- template_name += 'rejection'
1509- if archive.is_ppa:
1510- template_name = 'ppa-%s' % template_name
1511- template_name += '.txt'
1512- return get_email_template(template_name, app='soyuz')
1513+ for recipient in get_recipients(person):
1514+ if recipient not in recipients:
1515+ debug(
1516+ logger, "Adding recipient: '%s'" % format_address_for_person(
1517+ recipient))
1518+ reason = reason_factory(person, recipient)
1519+ recipients[recipient] = reason
1520+
1521+
1522+def fetch_information(spr, bprs, changes, previous_version=None):
1523+ changelog = date = changedby = maintainer = None
1524+
1525+ if changes:
1526+ changelog = ChangesFile.formatChangesComment(
1527+ sanitize_string(changes.get('Changes')))
1528+ date = changes.get('Date')
1529+ try:
1530+ changedby = parse_maintainer_bytes(
1531+ changes.get('Changed-By'), 'Changed-By')
1532+ except ParseMaintError:
1533+ pass
1534+ try:
1535+ maintainer = parse_maintainer_bytes(
1536+ changes.get('Maintainer'), 'Maintainer')
1537+ except ParseMaintError:
1538+ pass
1539+ elif spr or bprs:
1540+ if not spr and bprs:
1541+ spr = bprs[0].build.source_package_release
1542+ changelog = spr.aggregate_changelog(previous_version)
1543+ date = spr.dateuploaded
1544+ if spr.creator and spr.creator.preferredemail:
1545+ changedby = (
1546+ spr.creator.displayname, spr.creator.preferredemail.email)
1547+ if spr.maintainer and spr.maintainer.preferredemail:
1548+ maintainer = (
1549+ spr.maintainer.displayname,
1550+ spr.maintainer.preferredemail.email)
1551+
1552+ return {
1553+ 'changelog': changelog,
1554+ 'date': date,
1555+ 'changedby': changedby,
1556+ 'maintainer': maintainer,
1557+ }
1558+
1559+
1560+def addr_to_person(addr):
1561+ """Return an `IPerson` given a name and email address.
1562+
1563+ :param addr: (name, email) tuple. The name is ignored.
1564+ :return: `IPerson` with the given email address. None if there
1565+ isn't one, or if `addr` is None.
1566+ """
1567+ if addr is None:
1568+ return None
1569+ return getUtility(IPersonSet).getByEmail(addr[1])
1570+
1571+
1572+def is_valid_uploader(person, distribution):
1573+ """Is `person` an uploader for `distribution`?
1574+
1575+ A `None` person is not an uploader.
1576+ """
1577+ if person is None:
1578+ return None
1579+ else:
1580+ return not getUtility(IArchivePermissionSet).componentsForUploader(
1581+ distribution.main_archive, person).is_empty()
1582+
1583+
1584+def is_auto_sync_upload(spr, bprs, pocket, changed_by):
1585+ """Return True if this is a (Debian) auto sync upload.
1586+
1587+ Sync uploads are source-only, unsigned and not targeted to
1588+ the security pocket. The Changed-By field is also the Katie
1589+ user (archive@ubuntu.com).
1590+ """
1591+ changed_by = addr_to_person(changed_by)
1592+ return (
1593+ spr and
1594+ not bprs and
1595+ changed_by == getUtility(ILaunchpadCelebrities).katie and
1596+ pocket != PackagePublishingPocket.SECURITY)
1597
1598
1599 ACTION_DESCRIPTIONS = {
1600@@ -101,9 +218,8 @@
1601
1602
1603 def calculate_subject(spr, bprs, customfiles, archive, distroseries,
1604- pocket, action):
1605+ pocket, action, changesfile_object=None):
1606 """Return the email subject for the notification."""
1607- suite = distroseries.getSuite(pocket)
1608 names = set()
1609 version = '-'
1610 if spr:
1611@@ -114,431 +230,46 @@
1612 version = bprs[0].build.source_package_release.version
1613 for custom in customfiles:
1614 names.add(custom.libraryfilealias.filename)
1615- name_str = ', '.join(names)
1616- subject = '[%s/%s] %s %s (%s)' % (
1617- archive.reference, suite, name_str, version,
1618- ACTION_DESCRIPTIONS[action])
1619+ if names:
1620+ archive_and_suite = '%s/%s' % (
1621+ archive.reference, distroseries.getSuite(pocket))
1622+ name_and_version = '%s %s' % (', '.join(names), version)
1623+ else:
1624+ if changesfile_object is None:
1625+ return None
1626+ # The suite may not be meaningful if we have no
1627+ # spr/bprs/customfiles, since this must be a very early rejection.
1628+ # Don't introduce confusion by including it.
1629+ archive_and_suite = archive.reference
1630+ name_and_version = os.path.basename(changesfile_object.name)
1631+ subject = '[%s] %s (%s)' % (
1632+ archive_and_suite, name_and_version, ACTION_DESCRIPTIONS[action])
1633 return subject
1634
1635
1636-def notify(blamer, spr, bprs, customfiles, archive, distroseries, pocket,
1637- summary_text=None, changes=None, changesfile_content=None,
1638- changesfile_object=None, action=None, dry_run=False,
1639- logger=None, announce_from_person=None, previous_version=None):
1640- """Notify about an upload or package copy.
1641-
1642- :param blamer: The `IPerson` who is to blame for this notification.
1643- :param spr: The `ISourcePackageRelease` that was created.
1644- :param bprs: A list of `IBinaryPackageRelease` that were created.
1645- :param customfiles: An `ILibraryFileAlias` that was created.
1646- :param archive: The target `IArchive`.
1647- :param distroseries: The target `IDistroSeries`.
1648- :param pocket: The target `PackagePublishingPocket`.
1649- :param summary_text: The summary of the notification.
1650- :param changes: A dictionary of the parsed changes file.
1651- :param changesfile_content: The raw content of the changes file, so it
1652- can be attached to the mail if desired.
1653- :param changesfile_object: The raw object of the changes file. Only used
1654- to work out the filename for `reject_changes_file`.
1655- :param action: A string of what action to notify for, such as 'new',
1656- 'accepted'.
1657- :param dry_run: If True, only log the mail.
1658- :param announce_from_person: If passed, use this `IPerson` as the From: in
1659- announcement emails. If the person has no preferred email address,
1660- the person is ignored and the default From: is used instead.
1661- :param previous_version: If specified, the change log on the email will
1662- include all of the source package's change logs after that version
1663- up to and including the passed spr's version.
1664- """
1665- # If this is a binary or mixed upload, we don't send *any* emails
1666- # provided it's not a rejection or a security upload:
1667- if (
1668- bprs and action != 'rejected' and
1669- pocket != PackagePublishingPocket.SECURITY):
1670- debug(logger, "Not sending email; upload is from a build.")
1671- return
1672-
1673- if spr and spr.source_package_recipe_build and action == 'accepted':
1674- debug(logger, "Not sending email; upload is from a recipe.")
1675- return
1676-
1677- if spr is None and not bprs and not customfiles:
1678- # We do not have enough context to do a normal notification, so
1679- # reject what we do have.
1680- if changesfile_object is None:
1681- return
1682- reject_changes_file(
1683- blamer, changesfile_object.name, changes, archive, distroseries,
1684- summary_text, logger=logger)
1685- return
1686-
1687- # "files" will contain a list of tuples of filename,component,section.
1688- # If files is empty, we don't need to send an email if this is not
1689- # a rejection.
1690- try:
1691- files = build_uploaded_files_list(spr, bprs, customfiles, logger)
1692- except LanguagePackEncountered:
1693- # Don't send emails for language packs.
1694- return
1695-
1696- if not files and action != 'rejected':
1697- return
1698-
1699- recipients = get_upload_notification_recipients(
1700- blamer, archive, distroseries, logger, changes=changes, spr=spr,
1701- bprs=bprs)
1702-
1703- # There can be no recipients if none of the emails are registered
1704- # in LP.
1705- if not recipients:
1706- debug(logger, "No recipients on email, not sending.")
1707- return
1708-
1709- if action == 'rejected':
1710- default_recipient = "%s <%s>" % (
1711- config.uploader.default_recipient_name,
1712- config.uploader.default_recipient_address)
1713- if not recipients:
1714- recipients = [default_recipient]
1715- debug(logger, "Sending rejection email.")
1716- summarystring = summary_text
1717- else:
1718- summary = build_summary(spr, files, action)
1719- if summary_text:
1720- summary.append(summary_text)
1721- summarystring = "\n".join(summary)
1722-
1723- attach_changes = not archive.is_ppa
1724-
1725- def build_and_send_mail(action, recipients, from_addr=None, bcc=None,
1726- previous_version=None):
1727- subject = calculate_subject(
1728- spr, bprs, customfiles, archive, distroseries, pocket, action)
1729- body = assemble_body(
1730- blamer, spr, bprs, archive, distroseries, summarystring, changes,
1731- action, previous_version=previous_version)
1732- body = body.encode("utf8")
1733- send_mail(
1734- spr, archive, recipients, subject, body, dry_run,
1735- changesfile_content=changesfile_content,
1736- attach_changes=attach_changes, from_addr=from_addr, bcc=bcc,
1737- logger=logger)
1738-
1739- build_and_send_mail(
1740- action, recipients, previous_version=previous_version)
1741-
1742- info = fetch_information(spr, bprs, changes)
1743- from_addr = info['changedby']
1744- if (announce_from_person is not None
1745- and announce_from_person.preferredemail is not None):
1746- from_addr = (
1747- announce_from_person.displayname,
1748- announce_from_person.preferredemail.email)
1749-
1750- # If we're sending an acceptance notification for a non-PPA upload,
1751- # announce if possible. Avoid announcing backports, binary-only
1752- # security uploads, or autosync uploads.
1753- if (action == 'accepted' and distroseries.changeslist
1754- and not archive.is_ppa
1755- and pocket != PackagePublishingPocket.BACKPORTS
1756- and not (pocket == PackagePublishingPocket.SECURITY and spr is None)
1757- and not is_auto_sync_upload(spr, bprs, pocket, from_addr)):
1758- name = None
1759- bcc_addr = None
1760- if spr:
1761- name = spr.name
1762- elif bprs:
1763- name = bprs[0].build.source_package_release.name
1764- if name:
1765- email_base = distroseries.distribution.package_derivatives_email
1766- if email_base:
1767- bcc_addr = email_base.format(package_name=name)
1768-
1769- build_and_send_mail(
1770- 'announcement', [str(distroseries.changeslist)],
1771- format_address(*from_addr) if from_addr else None, bcc_addr,
1772- previous_version=previous_version)
1773-
1774-
1775-def assemble_body(blamer, spr, bprs, archive, distroseries, summary, changes,
1776- action, previous_version=None):
1777- """Assemble the email notification body."""
1778- if changes is None:
1779- changes = {}
1780- info = fetch_information(
1781- spr, bprs, changes, previous_version=previous_version)
1782- information = {
1783- 'STATUS': ACTION_DESCRIPTIONS[action],
1784- 'SUMMARY': summary,
1785- 'DATE': 'Date: %s' % info['date'],
1786- 'CHANGESFILE': info['changelog'],
1787- 'DISTRO': distroseries.distribution.title,
1788- 'ANNOUNCE': 'No announcement sent',
1789- 'CHANGEDBY': '',
1790- 'MAINTAINER': '',
1791- 'ORIGIN': '',
1792- 'SIGNER': '',
1793- 'SPR_URL': '',
1794- 'ARCHIVE_URL': '\n%s' % canonical_url(archive),
1795- 'USERS_ADDRESS': config.launchpad.users_address,
1796- }
1797- if spr:
1798- information['SPR_URL'] = canonical_url(
1799- distroseries.distribution.getSourcePackageRelease(spr))
1800-
1801- # Some syncs (e.g. from Debian) will involve packages whose
1802- # changed-by person was auto-created in LP and hence does not have a
1803- # preferred email address set. We'll get a None here.
1804- changedby_person = addr_to_person(info['changedby'])
1805- if info['changedby']:
1806- information['CHANGEDBY'] = (
1807- '\nChanged-By: %s' % rfc822_encode_address(*info['changedby']))
1808- if (blamer is not None and blamer != changedby_person
1809- and blamer.preferredemail):
1810- information['SIGNER'] = '\nSigned-By: %s' % rfc822_encode_address(
1811- blamer.displayname, blamer.preferredemail.email)
1812- if info['maintainer'] and info['maintainer'] != info['changedby']:
1813- information['MAINTAINER'] = (
1814- '\nMaintainer: %s' % rfc822_encode_address(*info['maintainer']))
1815-
1816- origin = changes.get('Origin')
1817- if origin:
1818- information['ORIGIN'] = '\nOrigin: %s' % origin
1819- if action == 'unapproved':
1820- information['SUMMARY'] += (
1821- "\nThis upload awaits approval by a distro manager\n")
1822- if distroseries.changeslist:
1823- information['ANNOUNCE'] = "Announcing to %s" % (
1824- distroseries.changeslist)
1825-
1826- return get_template(archive, action) % information
1827-
1828-
1829-def send_mail(
1830- spr, archive, to_addrs, subject, mail_text, dry_run, from_addr=None,
1831- bcc=None, changesfile_content=None, attach_changes=False, logger=None):
1832- """Send an email to to_addrs with the given text and subject.
1833-
1834- :param spr: The `ISourcePackageRelease` to be notified about.
1835- :param archive: The target `IArchive`.
1836- :param to_addrs: A list of email addresses to be used as recipients.
1837- Each email must be a valid ASCII str instance or a unicode one.
1838- :param subject: The email's subject.
1839- :param mail_text: The text body of the email. Unicode is preserved in the
1840- email.
1841- :param dry_run: Whether or not an email should actually be sent. But
1842- please note that this flag is (largely) ignored.
1843- :param from_addr: The email address to be used as the sender. Must be a
1844- valid ASCII str instance or a unicode one. Defaults to the email
1845- for config.uploader.
1846- :param bcc: Optional email Blind Carbon Copy address(es).
1847- :param param changesfile_content: The content of the actual changesfile.
1848- :param attach_changes: A flag governing whether the original changesfile
1849- content shall be attached to the email.
1850- """
1851- extra_headers = {
1852- 'X-Katie': 'Launchpad actually',
1853- 'X-Launchpad-Archive': archive.reference,
1854- }
1855-
1856- # The deprecated PPA reference header is included for Ubuntu PPAs to
1857- # avoid breaking existing consumers.
1858- if archive.is_ppa and archive.distribution.name == u'ubuntu':
1859- extra_headers['X-Launchpad-PPA'] = get_ppa_reference(archive)
1860-
1861- # Include a 'X-Launchpad-Component' header with the component and
1862- # the section of the source package uploaded in order to facilitate
1863- # filtering on the part of the email recipients.
1864- if spr:
1865- xlp_component_header = 'component=%s, section=%s' % (
1866- spr.component.name, spr.section.name)
1867- extra_headers['X-Launchpad-Component'] = xlp_component_header
1868-
1869- if from_addr is None:
1870- from_addr = format_address(
1871- config.uploader.default_sender_name,
1872- config.uploader.default_sender_address)
1873-
1874- # All emails from here have a Bcc to the default recipient.
1875- bcc_text = format_address(
1876- config.uploader.default_recipient_name,
1877- config.uploader.default_recipient_address)
1878- if bcc:
1879- bcc_text = "%s, %s" % (bcc_text, bcc)
1880- extra_headers['Bcc'] = bcc_text
1881-
1882- recipients = ", ".join(to_addrs)
1883-
1884- if dry_run and logger is not None:
1885- debug(logger, "Would have sent a mail:")
1886- else:
1887- debug(logger, "Sent a mail:")
1888- debug(logger, " Subject: %s" % subject)
1889- debug(logger, " Sender: %s" % from_addr)
1890- debug(logger, " Recipients: %s" % recipients)
1891- if 'Bcc' in extra_headers:
1892- debug(logger, " Bcc: %s" % extra_headers['Bcc'])
1893- debug(logger, " Body:")
1894- for line in mail_text.splitlines():
1895- if isinstance(line, str):
1896- line = line.decode('utf-8', 'replace')
1897- debug(logger, line)
1898-
1899- if not dry_run:
1900- # Since we need to send the original changesfile as an
1901- # attachment the sendmail() method will be used as opposed to
1902- # simple_sendmail().
1903- message = MIMEMultipart()
1904- message['from'] = from_addr
1905- message['subject'] = subject
1906- message['to'] = recipients
1907-
1908- # Set the extra headers if any are present.
1909- for key, value in extra_headers.iteritems():
1910- message.add_header(key, value)
1911-
1912- # Add the email body.
1913- message.attach(
1914- MIMEText(sanitize_string(mail_text).encode('utf-8'),
1915- 'plain', 'utf-8'))
1916-
1917- if attach_changes:
1918- # Add the original changesfile as an attachment.
1919- if changesfile_content is not None:
1920- changesfile_text = sanitize_string(changesfile_content)
1921- else:
1922- changesfile_text = ("Sorry, changesfile not available.")
1923-
1924- attachment = MIMEText(
1925- changesfile_text.encode('utf-8'), 'plain', 'utf-8')
1926- attachment.add_header(
1927- 'Content-Disposition',
1928- 'attachment; filename="changesfile"')
1929- message.attach(attachment)
1930-
1931- # And finally send the message.
1932- sendmail(message)
1933-
1934-
1935-def sanitize_string(s):
1936- """Make sure string does not trigger 'ascii' codec errors.
1937-
1938- Convert string to unicode if needed so that characters outside
1939- the (7-bit) ASCII range do not cause errors like these:
1940-
1941- 'ascii' codec can't decode byte 0xc4 in position 21: ordinal
1942- not in range(128)
1943- """
1944- if isinstance(s, unicode):
1945- return s
1946- else:
1947- return guess_encoding(s)
1948-
1949-
1950-def debug(logger, msg, *args, **kwargs):
1951- """Shorthand debug notation for publish() methods."""
1952- if logger is not None:
1953- logger.debug(msg, *args, **kwargs)
1954-
1955-
1956-def is_valid_uploader(person, distribution):
1957- """Is `person` an uploader for `distribution`?
1958-
1959- A `None` person is not an uploader.
1960- """
1961- if person is None:
1962- return None
1963- else:
1964- return not getUtility(IArchivePermissionSet).componentsForUploader(
1965- distribution.main_archive, person).is_empty()
1966-
1967-
1968-def get_upload_notification_recipients(blamer, archive, distroseries,
1969- logger=None, changes=None, spr=None,
1970- bprs=None):
1971- """Return a list of recipients for notification emails."""
1972- debug(logger, "Building recipients list.")
1973- candidate_recipients = [blamer]
1974- info = fetch_information(spr, bprs, changes)
1975-
1976- changer = addr_to_person(info['changedby'])
1977- maintainer = addr_to_person(info['maintainer'])
1978-
1979- if blamer is None and not archive.is_copy:
1980- debug(logger, "Changes file is unsigned; adding changer as recipient.")
1981- candidate_recipients.append(changer)
1982-
1983- if archive.is_ppa:
1984- # For PPAs, any person or team mentioned explicitly in the
1985- # ArchivePermissions as uploaders for the archive will also
1986- # get emailed.
1987- candidate_recipients.extend([
1988- permission.person
1989- for permission in archive.getUploadersForComponent()])
1990- elif archive.is_copy:
1991- # For copy archives, notifying anyone else will probably only
1992- # confuse them.
1993- pass
1994- else:
1995- # If this is not a PPA, we also consider maintainer and changed-by.
1996- if blamer is not None:
1997- if is_valid_uploader(maintainer, distroseries.distribution):
1998- debug(logger, "Adding maintainer to recipients")
1999- candidate_recipients.append(maintainer)
2000-
2001- if is_valid_uploader(changer, distroseries.distribution):
2002- debug(logger, "Adding changed-by to recipients")
2003- candidate_recipients.append(changer)
2004-
2005- # Collect email addresses to notify. Skip persons who do not have a
2006- # preferredemail set, such as people who have not activated their
2007- # Launchpad accounts (and are therefore not expecting this email).
2008- recipients = [
2009- format_address_for_person(person)
2010- for person in filter(None, set(candidate_recipients))
2011- if person.preferredemail is not None]
2012-
2013- for recipient in recipients:
2014- debug(logger, "Adding recipient: '%s'", recipient)
2015-
2016- return recipients
2017-
2018-
2019 def build_uploaded_files_list(spr, builds, customfiles, logger):
2020- """Return a list of tuples of (filename, component, section).
2021+ """Generate a list of tuples of (filename, component, section).
2022
2023 Component and section are only set where the file is a source upload.
2024 If an empty list is returned, it means there are no files.
2025- Raises LanguagePackRejection if a language pack is detected.
2026- No emails should be sent for language packs.
2027 """
2028- files = []
2029- # Bail out early if this is an upload for the translations
2030- # section.
2031 if spr:
2032- if spr.section.name == 'translations':
2033- debug(logger,
2034- "Skipping acceptance and announcement, it is a "
2035- "language-package upload.")
2036- raise LanguagePackEncountered
2037 for sprfile in spr.files:
2038- files.append(
2039- (sprfile.libraryfile.filename, spr.component.name,
2040- spr.section.name))
2041+ yield (
2042+ sprfile.libraryfile.filename, spr.component.name,
2043+ spr.section.name)
2044
2045 # Component and section don't get set for builds and custom, since
2046 # this information is only used in the summary string for source
2047 # uploads.
2048 for build in builds:
2049 for bpr in build.build.binarypackages:
2050- files.extend([
2051- (bpf.libraryfile.filename, '', '') for bpf in bpr.files])
2052+ for bpf in bpr.files:
2053+ yield bpf.libraryfile.filename, '', ''
2054
2055 if customfiles:
2056- files.extend(
2057- [(file.libraryfilealias.filename, '', '') for file in customfiles])
2058-
2059- return files
2060+ for customfile in customfiles:
2061+ yield customfile.libraryfilealias.filename, '', ''
2062
2063
2064 def build_summary(spr, files, action):
2065@@ -555,70 +286,337 @@
2066 return summary
2067
2068
2069-def addr_to_person(addr):
2070- """Return an `IPerson` given a name and email address.
2071-
2072- :param addr: (name, email) tuple. The name is ignored.
2073- :return: `IPerson` with the given email address. None if there
2074- isn't one, or if `addr` is None.
2075- """
2076- if addr is None:
2077- return None
2078- return getUtility(IPersonSet).getByEmail(addr[1])
2079-
2080-
2081-def is_auto_sync_upload(spr, bprs, pocket, changed_by):
2082- """Return True if this is a (Debian) auto sync upload.
2083-
2084- Sync uploads are source-only, unsigned and not targeted to
2085- the security pocket. The Changed-By field is also the Katie
2086- user (archive@ubuntu.com).
2087- """
2088- changed_by = addr_to_person(changed_by)
2089- return (
2090- spr and
2091- not bprs and
2092- changed_by == getUtility(ILaunchpadCelebrities).katie and
2093- pocket != PackagePublishingPocket.SECURITY)
2094-
2095-
2096-def fetch_information(spr, bprs, changes, previous_version=None):
2097- changelog = date = changedby = maintainer = None
2098-
2099- if changes:
2100- changelog = ChangesFile.formatChangesComment(
2101- sanitize_string(changes.get('Changes')))
2102- date = changes.get('Date')
2103- try:
2104- changedby = parse_maintainer_bytes(
2105- changes.get('Changed-By'), 'Changed-By')
2106- except ParseMaintError:
2107- pass
2108- try:
2109- maintainer = parse_maintainer_bytes(
2110- changes.get('Maintainer'), 'Maintainer')
2111- except ParseMaintError:
2112- pass
2113- elif spr or bprs:
2114- if not spr and bprs:
2115- spr = bprs[0].build.source_package_release
2116- changelog = spr.aggregate_changelog(previous_version)
2117- date = spr.dateuploaded
2118- if spr.creator and spr.creator.preferredemail:
2119- changedby = (
2120- spr.creator.displayname, spr.creator.preferredemail.email)
2121- if spr.maintainer and spr.maintainer.preferredemail:
2122- maintainer = (
2123- spr.maintainer.displayname,
2124- spr.maintainer.preferredemail.email)
2125-
2126- return {
2127- 'changelog': changelog,
2128- 'date': date,
2129- 'changedby': changedby,
2130- 'maintainer': maintainer,
2131- }
2132-
2133-
2134-class LanguagePackEncountered(Exception):
2135- """Thrown when not wanting to email notifications for language packs."""
2136+class PackageUploadMailer(BaseMailer):
2137+
2138+ app = 'soyuz'
2139+
2140+ @classmethod
2141+ def getRecipientsForAction(cls, action, info, blamee, spr, bprs, archive,
2142+ distroseries, pocket, announce_from_person=None,
2143+ logger=None):
2144+ # If this is a binary or mixed upload, we don't send *any* emails
2145+ # provided it's not a rejection or a security upload:
2146+ if (
2147+ bprs and action != 'rejected' and
2148+ pocket != PackagePublishingPocket.SECURITY):
2149+ debug(logger, "Not sending email; upload is from a build.")
2150+ return {}, ''
2151+
2152+ if spr and spr.source_package_recipe_build and action == 'accepted':
2153+ debug(logger, "Not sending email; upload is from a recipe.")
2154+ return {}, ''
2155+
2156+ if spr and spr.section.name == 'translations':
2157+ debug(
2158+ logger,
2159+ "Skipping acceptance and announcement for language packs.")
2160+ return {}, ''
2161+
2162+ debug(logger, "Building recipients list.")
2163+ recipients = OrderedDict()
2164+
2165+ add_recipient(
2166+ recipients, blamee, PackageUploadRecipientReason.forRequester,
2167+ logger=logger)
2168+
2169+ changer = addr_to_person(info['changedby'])
2170+ maintainer = addr_to_person(info['maintainer'])
2171+
2172+ if blamee is None and not archive.is_copy:
2173+ debug(
2174+ logger,
2175+ "Changes file is unsigned; adding changer as recipient.")
2176+ add_recipient(
2177+ recipients, changer, PackageUploadRecipientReason.forChangedBy,
2178+ logger=logger)
2179+
2180+ if archive.is_ppa:
2181+ # For PPAs, any person or team mentioned explicitly in the
2182+ # ArchivePermissions as uploaders for the archive will also get
2183+ # emailed.
2184+ for permission in archive.getUploadersForComponent():
2185+ add_recipient(
2186+ recipients, permission.person,
2187+ PackageUploadRecipientReason.forPPAUploader, logger=logger)
2188+ elif archive.is_copy:
2189+ # For copy archives, notifying anyone else will probably only
2190+ # confuse them.
2191+ pass
2192+ else:
2193+ # If this is not a PPA, we also consider maintainer and changed-by.
2194+ if blamee is not None:
2195+ if is_valid_uploader(maintainer, distroseries.distribution):
2196+ debug(logger, "Adding maintainer to recipients")
2197+ add_recipient(
2198+ recipients, maintainer,
2199+ PackageUploadRecipientReason.forMaintainer,
2200+ logger=logger)
2201+
2202+ if is_valid_uploader(changer, distroseries.distribution):
2203+ debug(logger, "Adding changed-by to recipients")
2204+ add_recipient(
2205+ recipients, changer,
2206+ PackageUploadRecipientReason.forChangedBy,
2207+ logger=logger)
2208+
2209+ if (announce_from_person is not None and
2210+ announce_from_person.preferredemail is not None):
2211+ announce_from_addr = (
2212+ announce_from_person.displayname,
2213+ announce_from_person.preferredemail.email)
2214+ else:
2215+ announce_from_addr = info['changedby']
2216+
2217+ # If we're sending an acceptance notification for a non-PPA upload,
2218+ # announce if possible. Avoid announcing backports, binary-only
2219+ # security uploads, or autosync uploads.
2220+ if (action == 'accepted' and distroseries.changeslist
2221+ and not archive.is_ppa
2222+ and pocket != PackagePublishingPocket.BACKPORTS
2223+ and not (
2224+ pocket == PackagePublishingPocket.SECURITY and spr is None)
2225+ and not is_auto_sync_upload(
2226+ spr, bprs, pocket, announce_from_addr)):
2227+ recipient = AnnouncementStubPerson(distroseries.changeslist)
2228+ recipients[recipient] = (
2229+ PackageUploadRecipientReason.forAnnouncement(recipient))
2230+
2231+ if announce_from_addr is not None:
2232+ announce_from_address = format_address(*announce_from_addr)
2233+ else:
2234+ announce_from_address = None
2235+ return recipients, announce_from_address
2236+
2237+ @classmethod
2238+ def forAction(cls, action, blamee, spr, bprs, customfiles, archive,
2239+ distroseries, pocket, changes=None, changesfile_object=None,
2240+ announce_from_person=None, previous_version=None,
2241+ logger=None, **kwargs):
2242+ info = fetch_information(
2243+ spr, bprs, changes, previous_version=previous_version)
2244+ recipients, announce_from_address = cls.getRecipientsForAction(
2245+ action, info, blamee, spr, bprs, archive, distroseries, pocket,
2246+ announce_from_person=announce_from_person, logger=logger)
2247+ subject = calculate_subject(
2248+ spr, bprs, customfiles, archive, distroseries, pocket, action,
2249+ changesfile_object=changesfile_object)
2250+ if subject is None:
2251+ # We don't even have enough information to build a minimal
2252+ # subject, so do nothing.
2253+ recipients = {}
2254+ template_name = "upload-"
2255+ if action in ("new", "accepted", "announcement"):
2256+ template_name += action
2257+ elif action == "unapproved":
2258+ template_name += "accepted"
2259+ elif action == "rejected":
2260+ template_name += "rejection"
2261+ if archive.is_ppa:
2262+ template_name = "ppa-%s" % template_name
2263+ template_name += ".txt"
2264+ from_address = format_address(
2265+ config.uploader.default_sender_name,
2266+ config.uploader.default_sender_address)
2267+ return cls(
2268+ subject, template_name, recipients, from_address, action, info,
2269+ blamee, spr, bprs, customfiles, archive, distroseries, pocket,
2270+ changes=changes, announce_from_address=announce_from_address,
2271+ logger=logger, **kwargs)
2272+
2273+ def __init__(self, subject, template_name, recipients, from_address,
2274+ action, info, blamee, spr, bprs, customfiles, archive,
2275+ distroseries, pocket, summary_text=None, changes=None,
2276+ changesfile_content=None, dry_run=False,
2277+ announce_from_address=None, previous_version=None,
2278+ logger=None):
2279+ super(PackageUploadMailer, self).__init__(
2280+ subject, template_name, recipients, from_address,
2281+ notification_type="package-upload")
2282+ self.action = action
2283+ self.info = info
2284+ self.blamee = blamee
2285+ self.spr = spr
2286+ self.bprs = bprs
2287+ self.customfiles = customfiles
2288+ self.archive = archive
2289+ self.distroseries = distroseries
2290+ self.pocket = pocket
2291+ self.changes = changes
2292+ self.changesfile_content = changesfile_content
2293+ self.dry_run = dry_run
2294+ self.logger = logger
2295+ self.announce_from_address = announce_from_address
2296+ self.previous_version = previous_version
2297+
2298+ if action == 'rejected':
2299+ self.summarystring = summary_text
2300+ else:
2301+ files = build_uploaded_files_list(spr, bprs, customfiles, logger)
2302+ summary = build_summary(spr, files, action)
2303+ if summary_text:
2304+ summary.append(summary_text)
2305+ self.summarystring = "\n".join(summary)
2306+
2307+ def _getFromAddress(self, email, recipient):
2308+ """See `BaseMailer`."""
2309+ if (zope_isinstance(recipient, AnnouncementStubPerson) and
2310+ self.announce_from_address is not None):
2311+ return self.announce_from_address
2312+ else:
2313+ return super(PackageUploadMailer, self)._getFromAddress(
2314+ email, recipient)
2315+
2316+ def _getHeaders(self, email, recipient):
2317+ """See `BaseMailer`."""
2318+ headers = super(PackageUploadMailer, self)._getHeaders(
2319+ email, recipient)
2320+ headers['X-Katie'] = 'Launchpad actually'
2321+ headers['X-Launchpad-Archive'] = self.archive.reference
2322+
2323+ # The deprecated PPA reference header is included for Ubuntu PPAs to
2324+ # avoid breaking existing consumers.
2325+ if self.archive.is_ppa and self.archive.distribution.name == u'ubuntu':
2326+ headers['X-Launchpad-PPA'] = get_ppa_reference(self.archive)
2327+
2328+ # Include a 'X-Launchpad-Component' header with the component and
2329+ # the section of the source package uploaded in order to facilitate
2330+ # filtering on the part of the email recipients.
2331+ if self.spr:
2332+ headers['X-Launchpad-Component'] = 'component=%s, section=%s' % (
2333+ self.spr.component.name, self.spr.section.name)
2334+
2335+ # All emails from here have a Bcc to the default recipient.
2336+ bcc_text = format_address(
2337+ config.uploader.default_recipient_name,
2338+ config.uploader.default_recipient_address)
2339+ if zope_isinstance(recipient, AnnouncementStubPerson):
2340+ name = None
2341+ if self.spr:
2342+ name = self.spr.name
2343+ elif self.bprs:
2344+ name = self.bprs[0].build.source_package_release.name
2345+ if name:
2346+ distribution = self.distroseries.distribution
2347+ email_base = distribution.package_derivatives_email
2348+ if email_base:
2349+ bcc_text += ", " + email_base.format(package_name=name)
2350+ headers['Bcc'] = bcc_text
2351+
2352+ return headers
2353+
2354+ def _addAttachments(self, ctrl, email):
2355+ """See `BaseMailer`."""
2356+ if not self.archive.is_ppa:
2357+ if self.changesfile_content is not None:
2358+ changesfile_text = sanitize_string(self.changesfile_content)
2359+ else:
2360+ changesfile_text = "Sorry, changesfile not available."
2361+ ctrl.addAttachment(
2362+ changesfile_text, content_type='text/plain',
2363+ filename='changesfile', charset='utf-8')
2364+
2365+ def _getTemplateName(self, email, recipient):
2366+ """See `BaseMailer`."""
2367+ if zope_isinstance(recipient, AnnouncementStubPerson):
2368+ return "upload-announcement.txt"
2369+ else:
2370+ return self._template_name
2371+
2372+ def _getTemplateParams(self, email, recipient):
2373+ """See `BaseMailer`."""
2374+ params = super(PackageUploadMailer, self)._getTemplateParams(
2375+ email, recipient)
2376+ params.update({
2377+ 'STATUS': ACTION_DESCRIPTIONS[self.action],
2378+ 'SUMMARY': self.summarystring,
2379+ 'DATE': '',
2380+ 'CHANGESFILE': '',
2381+ 'DISTRO': self.distroseries.distribution.title,
2382+ 'ANNOUNCE': 'No announcement sent',
2383+ 'CHANGEDBY': '',
2384+ 'MAINTAINER': '',
2385+ 'ORIGIN': '',
2386+ 'SIGNER': '',
2387+ 'SPR_URL': '',
2388+ 'ARCHIVE_URL': canonical_url(self.archive),
2389+ 'USERS_ADDRESS': config.launchpad.users_address,
2390+ })
2391+ changes = self.changes
2392+ if changes is None:
2393+ changes = {}
2394+
2395+ if self.info['date'] is not None:
2396+ params['DATE'] = 'Date: %s' % self.info['date']
2397+ if self.info['changelog'] is not None:
2398+ params['CHANGESFILE'] = self.info['changelog']
2399+ if self.spr:
2400+ params['SPR_URL'] = canonical_url(
2401+ self.distroseries.distribution.getSourcePackageRelease(
2402+ self.spr))
2403+
2404+ # Some syncs (e.g. from Debian) will involve packages whose
2405+ # changed-by person was auto-created in LP and hence does not have a
2406+ # preferred email address set. We'll get a None here.
2407+ changedby_person = addr_to_person(self.info['changedby'])
2408+ if self.info['changedby']:
2409+ params['CHANGEDBY'] = '\nChanged-By: %s' % rfc822_encode_address(
2410+ *self.info['changedby'])
2411+ if (self.blamee is not None and self.blamee != changedby_person
2412+ and self.blamee.preferredemail):
2413+ params['SIGNER'] = '\nSigned-By: %s' % rfc822_encode_address(
2414+ self.blamee.displayname, self.blamee.preferredemail.email)
2415+ if (self.info['maintainer']
2416+ and self.info['maintainer'] != self.info['changedby']):
2417+ params['MAINTAINER'] = '\nMaintainer: %s' % rfc822_encode_address(
2418+ *self.info['maintainer'])
2419+
2420+ origin = changes.get('Origin')
2421+ if origin:
2422+ params['ORIGIN'] = '\nOrigin: %s' % origin
2423+ if self.action == 'unapproved':
2424+ params['SUMMARY'] += (
2425+ "\nThis upload awaits approval by a distro manager\n")
2426+ if self.distroseries.changeslist:
2427+ params['ANNOUNCE'] = "Announcing to %s" % (
2428+ self.distroseries.changeslist)
2429+
2430+ return params
2431+
2432+ def _getFooter(self, email, recipient, params):
2433+ """See `BaseMailer`."""
2434+ if zope_isinstance(recipient, AnnouncementStubPerson):
2435+ return None
2436+ else:
2437+ footer_lines = []
2438+ if self.archive.is_ppa:
2439+ footer_lines.append("%(ARCHIVE_URL)s\n")
2440+ footer_lines.append("%(reason)s\n")
2441+ return "".join(footer_lines) % params
2442+
2443+ def generateEmail(self, email, recipient, force_no_attachments=False):
2444+ """See `BaseMailer`."""
2445+ ctrl = super(PackageUploadMailer, self).generateEmail(
2446+ email, recipient, force_no_attachments=force_no_attachments)
2447+ if self.dry_run:
2448+ debug(self.logger, "Would have sent a mail:")
2449+ else:
2450+ debug(self.logger, "Sent a mail:")
2451+ debug(self.logger, " Subject: %s" % ctrl.subject)
2452+ debug(self.logger, " Sender: %s" % ctrl.from_addr)
2453+ debug(self.logger, " Recipients: %s" % ", ".join(ctrl.to_addrs))
2454+ if 'Bcc' in ctrl.headers:
2455+ debug(self.logger, " Bcc: %s" % ctrl.headers['Bcc'])
2456+ debug(self.logger, " Body:")
2457+ for line in ctrl.body.splitlines():
2458+ if isinstance(line, bytes):
2459+ line = line.decode('utf-8', 'replace')
2460+ debug(self.logger, line)
2461+ return ctrl
2462+
2463+ def sendOne(self, email, recipient):
2464+ """See `BaseMailer`."""
2465+ if self.dry_run:
2466+ # Just generate the email for the sake of debugging output.
2467+ self.generateEmail(email, recipient)
2468+ else:
2469+ super(PackageUploadMailer, self).sendOne(email, recipient)
2470
2471=== added directory 'lib/lp/soyuz/mail/tests'
2472=== added file 'lib/lp/soyuz/mail/tests/__init__.py'
2473=== renamed file 'lib/lp/soyuz/adapters/tests/test_notification.py' => 'lib/lp/soyuz/mail/tests/test_packageupload.py'
2474--- lib/lp/soyuz/adapters/tests/test_notification.py 2015-07-29 07:01:04 +0000
2475+++ lib/lp/soyuz/mail/tests/test_packageupload.py 2015-08-26 13:41:53 +0000
2476@@ -2,35 +2,35 @@
2477 # NOTE: The first line above must stay first; do not move the copyright
2478 # notice to the top. See http://www.python.org/dev/peps/pep-0263/.
2479 #
2480-# Copyright 2011-2014 Canonical Ltd. This software is licensed under the
2481+# Copyright 2011-2015 Canonical Ltd. This software is licensed under the
2482 # GNU Affero General Public License version 3 (see the file LICENSE).
2483
2484 from textwrap import dedent
2485
2486-from storm.store import Store
2487+from testtools.matchers import (
2488+ Contains,
2489+ ContainsDict,
2490+ Equals,
2491+ KeysEqual,
2492+ )
2493 from zope.component import getUtility
2494 from zope.security.proxy import removeSecurityProxy
2495
2496+from lp.archivepublisher.utils import get_ppa_reference
2497 from lp.registry.interfaces.pocket import PackagePublishingPocket
2498-from lp.services.log.logger import BufferLogger
2499-from lp.services.mail.sendmail import format_address_for_person
2500 from lp.services.propertycache import get_property_cache
2501 from lp.services.webapp.publisher import canonical_url
2502-from lp.soyuz.adapters.notification import (
2503- assemble_body,
2504- calculate_subject,
2505- fetch_information,
2506- get_upload_notification_recipients,
2507- is_auto_sync_upload,
2508- notify,
2509- reject_changes_file,
2510- )
2511 from lp.soyuz.enums import (
2512 ArchivePurpose,
2513 PackageUploadCustomFormat,
2514 )
2515 from lp.soyuz.interfaces.component import IComponentSet
2516-from lp.soyuz.model.component import ComponentSelection
2517+from lp.soyuz.mail.packageupload import (
2518+ calculate_subject,
2519+ fetch_information,
2520+ is_auto_sync_upload,
2521+ PackageUploadMailer,
2522+ )
2523 from lp.soyuz.model.distributionsourcepackagerelease import (
2524 DistributionSourcePackageRelease,
2525 )
2526@@ -49,7 +49,7 @@
2527
2528 layer = LaunchpadZopelessLayer
2529
2530- def test_notify_from_unicode_names(self):
2531+ def test_mail_from_unicode_names(self):
2532 # People with unicode in their names should appear correctly in the
2533 # email and not get smashed to ASCII or otherwise transliterated.
2534 creator = self.factory.makePerson(displayname=u"Loïc")
2535@@ -60,9 +60,9 @@
2536 distroseries = self.factory.makeDistroSeries()
2537 distroseries.changeslist = "blah@example.com"
2538 blamer = self.factory.makePerson(displayname=u"Stéphane")
2539- notify(
2540- blamer, spr, [], [], archive, distroseries, pocket,
2541- action='accepted')
2542+ mailer = PackageUploadMailer.forAction(
2543+ "accepted", blamer, spr, [], [], archive, distroseries, pocket)
2544+ mailer.sendAll()
2545 notifications = pop_notifications()
2546 self.assertEqual(2, len(notifications))
2547 msg = notifications[1].get_payload(0)
2548@@ -98,13 +98,15 @@
2549 blamer = self.factory.makePerson()
2550 if from_person is None:
2551 from_person = self.factory.makePerson()
2552- notify(
2553- blamer, spr, [], [], archive, distroseries, pocket,
2554- action='accepted', announce_from_person=from_person)
2555+ mailer = PackageUploadMailer.forAction(
2556+ "accepted", blamer, spr, [], [], archive, distroseries, pocket,
2557+ announce_from_person=from_person)
2558+ mailer.sendAll()
2559
2560- def test_notify_from_person_override(self):
2561- # notify() takes an optional from_person to override the calculated
2562- # From: address in announcement emails.
2563+ def test_forAction_announce_from_person_override(self):
2564+ # PackageUploadMailer.forAction() takes an optional
2565+ # announce_from_person to override the calculated From: address in
2566+ # announcement emails.
2567 spr = self.factory.makeSourcePackageRelease()
2568 self.factory.makeSourcePackageReleaseFile(sourcepackagerelease=spr)
2569 archive = self.factory.makeArchive(purpose=ArchivePurpose.PRIMARY)
2570@@ -114,21 +116,28 @@
2571 blamer = self.factory.makePerson()
2572 from_person = self.factory.makePerson(
2573 email="lemmy@example.com", displayname="Lemmy Kilmister")
2574- notify(
2575- blamer, spr, [], [], archive, distroseries, pocket,
2576- action='accepted', announce_from_person=from_person)
2577+ mailer = PackageUploadMailer.forAction(
2578+ "accepted", blamer, spr, [], [], archive, distroseries, pocket,
2579+ announce_from_person=from_person)
2580+ mailer.sendAll()
2581 notifications = pop_notifications()
2582 self.assertEqual(2, len(notifications))
2583 # The first notification is to the blamer, the second notification is
2584 # to the announce list, which is the one that gets the overridden
2585 # From:
2586- self.assertEqual(
2587- "Lemmy Kilmister <lemmy@example.com>", notifications[1]["From"])
2588+ self.assertThat(
2589+ dict(notifications[1]),
2590+ ContainsDict({
2591+ "From": Equals("Lemmy Kilmister <lemmy@example.com>"),
2592+ "X-Launchpad-Message-Rationale": Equals("Announcement"),
2593+ "X-Launchpad-Notification-Type": Equals("package-upload"),
2594+ }))
2595
2596- def test_notify_from_person_override_with_unicode_names(self):
2597- # notify() takes an optional from_person to override the calculated
2598- # From: address in announcement emails. Non-ASCII real names should be
2599- # correctly encoded in the From heade.
2600+ def test_forAction_announce_from_person_override_with_unicode_names(self):
2601+ # PackageUploadMailer.forAction() takes an optional
2602+ # announce_from_person to override the calculated From: address in
2603+ # announcement emails. Non-ASCII real names should be correctly
2604+ # encoded in the From header.
2605 spr = self.factory.makeSourcePackageRelease()
2606 self.factory.makeSourcePackageReleaseFile(sourcepackagerelease=spr)
2607 archive = self.factory.makeArchive(purpose=ArchivePurpose.PRIMARY)
2608@@ -138,21 +147,28 @@
2609 blamer = self.factory.makePerson()
2610 from_person = self.factory.makePerson(
2611 email="loic@example.com", displayname=u"Loïc Motörhead")
2612- notify(
2613- blamer, spr, [], [], archive, distroseries, pocket,
2614- action='accepted', announce_from_person=from_person)
2615+ mailer = PackageUploadMailer.forAction(
2616+ "accepted", blamer, spr, [], [], archive, distroseries, pocket,
2617+ announce_from_person=from_person)
2618+ mailer.sendAll()
2619 notifications = pop_notifications()
2620 self.assertEqual(2, len(notifications))
2621 # The first notification is to the blamer, the second notification is
2622 # to the announce list, which is the one that gets the overridden
2623 # From:
2624- self.assertEqual(
2625- "=?utf-8?q?Lo=C3=AFc_Mot=C3=B6rhead?= <loic@example.com>",
2626- notifications[1]["From"])
2627+ self.assertThat(
2628+ dict(notifications[1]),
2629+ ContainsDict({
2630+ "From": Equals(
2631+ "=?utf-8?q?Lo=C3=AFc_Mot=C3=B6rhead?= <loic@example.com>"),
2632+ "X-Launchpad-Message-Rationale": Equals("Announcement"),
2633+ "X-Launchpad-Notification-Type": Equals("package-upload"),
2634+ }))
2635
2636- def test_notify_bcc_to_derivatives_list(self):
2637- # notify() will BCC the announcement email to the address defined in
2638- # Distribution.package_derivatives_email if it's defined.
2639+ def test_forAction_bcc_to_derivatives_list(self):
2640+ # PackageUploadMailer.forAction() will BCC the announcement email to
2641+ # the address defined in Distribution.package_derivatives_email if
2642+ # it's defined.
2643 email = "{package_name}_thing@foo.com"
2644 distroseries = self.factory.makeDistroSeries()
2645 with person_logged_in(distroseries.distribution.owner):
2646@@ -162,9 +178,14 @@
2647
2648 notifications = pop_notifications()
2649 self.assertEqual(2, len(notifications))
2650- bcc_address = notifications[1]["Bcc"]
2651 expected_email = email.format(package_name=spr.sourcepackagename.name)
2652- self.assertIn(expected_email, bcc_address)
2653+ self.assertThat(
2654+ dict(notifications[1]),
2655+ ContainsDict({
2656+ "Bcc": Contains(expected_email),
2657+ "X-Launchpad-Message-Rationale": Equals("Announcement"),
2658+ "X-Launchpad-Notification-Type": Equals("package-upload"),
2659+ }))
2660
2661 def test_fetch_information_spr_multiple_changelogs(self):
2662 # If previous_version is passed the "changelog" entry in the
2663@@ -182,9 +203,9 @@
2664 self.assertIn("foo (1.1)", info['changelog'])
2665 self.assertIn("foo (1.2)", info['changelog'])
2666
2667- def test_notify_bpr_rejected(self):
2668- # If we notify about a rejected bpr with no source, a notification is
2669- # sent.
2670+ def test_forAction_bpr_rejected(self):
2671+ # If we try to send mail about a rejected bpr with no source, a
2672+ # notification is sent.
2673 bpr = self.factory.makeBinaryPackageRelease()
2674 changelog = self.factory.makeChangelog(spn="foo", versions=["1.1"])
2675 removeSecurityProxy(
2676@@ -196,31 +217,30 @@
2677 distroseries = self.factory.makeDistroSeries()
2678 person = self.factory.makePerson(
2679 displayname=u'Blamer', email='blamer@example.com')
2680- notify(
2681- person, None, [bpr], [], archive, distroseries, pocket,
2682- summary_text="Rejected by archive administrator.",
2683- action='rejected')
2684+ mailer = PackageUploadMailer.forAction(
2685+ "rejected", person, None, [bpr], [], archive, distroseries, pocket,
2686+ summary_text="Rejected by archive administrator.")
2687+ mailer.sendAll()
2688 [notification] = pop_notifications()
2689- body = notification.get_payload()[0].get_payload()
2690+ body = notification.get_payload(decode=True)
2691 self.assertEqual('Blamer <blamer@example.com>', notification['To'])
2692 expected_body = dedent("""\
2693 Rejected:
2694 Rejected by archive administrator.
2695
2696- foo (1.1) unstable; urgency=3Dlow
2697+ foo (1.1) unstable; urgency=low
2698
2699 * 1.1.
2700
2701- =3D=3D=3D
2702+ ===
2703
2704 If you don't understand why your files were rejected please send an email
2705 to launchpad-users@lists.launchpad.net for help (requires membership).
2706
2707- --
2708+ %s
2709 http://launchpad.dev/~archiver/+archive/ubuntu/ppa
2710- You are receiving this email because you are the uploader of the above
2711- PPA package.
2712- """)
2713+ You are receiving this email because you made this upload.
2714+ """ % "-- ")
2715 self.assertEqual(expected_body, body)
2716
2717
2718@@ -295,42 +315,41 @@
2719 None, [bpr], [], archive, distroseries, pocket, 'accepted')
2720 self.assertEqual(expected_subject, subject)
2721
2722- def test_notify_bpr(self):
2723- # If we notify about an accepted bpr with no source, it is from a
2724- # build, and no notification is sent.
2725+ def test_forAction_bpr(self):
2726+ # If we try to send mail about an accepted bpr with no source, it is
2727+ # from a build, and no notification is sent.
2728 bpr = self.factory.makeBinaryPackageRelease()
2729 archive = self.factory.makeArchive()
2730 pocket = self.factory.getAnyPocket()
2731 distroseries = self.factory.makeDistroSeries()
2732 person = self.factory.makePerson()
2733- notify(
2734- person, None, [bpr], [], archive, distroseries, pocket,
2735- action='accepted')
2736+ mailer = PackageUploadMailer.forAction(
2737+ "accepted", person, None, [bpr], [], archive, distroseries, pocket)
2738+ mailer.sendAll()
2739 notifications = pop_notifications()
2740 self.assertEqual(0, len(notifications))
2741
2742 def test_reject_changes_file_no_email(self):
2743- # If we are rejecting a mail, and the person to notify has no
2744+ # If we are rejecting an upload, and the person to notify has no
2745 # preferred email, we should return early.
2746 archive = self.factory.makeArchive()
2747 distroseries = self.factory.makeDistroSeries()
2748 uploader = self.factory.makePerson()
2749 get_property_cache(uploader).preferredemail = None
2750- email = '%s <foo@example.com>' % uploader.displayname
2751- changes = {'Changed-By': email, 'Maintainer': email}
2752- logger = BufferLogger()
2753- reject_changes_file(
2754- uploader, '/tmp/changes', changes, archive, distroseries, '',
2755- logger=logger)
2756- self.assertIn(
2757- 'No recipients have a preferred email.', logger.getLogBuffer())
2758+ info = fetch_information(None, None, None)
2759+ recipients, _ = PackageUploadMailer.getRecipientsForAction(
2760+ 'rejected', info, uploader, None, [], archive, distroseries,
2761+ PackagePublishingPocket.RELEASE)
2762+ self.assertEqual({}, recipients)
2763
2764 def test_reject_with_no_changes(self):
2765 # If we don't have any files and no changes content, nothing happens.
2766 archive = self.factory.makeArchive()
2767 distroseries = self.factory.makeDistroSeries()
2768 pocket = self.factory.getAnyPocket()
2769- notify(None, None, (), (), archive, distroseries, pocket)
2770+ mailer = PackageUploadMailer.forAction(
2771+ "rejected", None, None, (), (), archive, distroseries, pocket)
2772+ mailer.sendAll()
2773 notifications = pop_notifications()
2774 self.assertEqual(0, len(notifications))
2775
2776@@ -351,20 +370,18 @@
2777 # Now set the uploaders.
2778 component = getUtility(IComponentSet).ensure('main')
2779 if component not in distroseries.components:
2780- store = Store.of(distroseries)
2781- store.add(
2782- ComponentSelection(
2783- distroseries=distroseries, component=component))
2784+ self.factory.makeComponentSelection(
2785+ distroseries=distroseries, component=component)
2786 distribution.main_archive.newComponentUploader(maintainer, component)
2787 distribution.main_archive.newComponentUploader(changer, component)
2788- observed = get_upload_notification_recipients(
2789- blamer, archive, distroseries, logger=None, changes=changes)
2790- self.assertContentEqual(
2791- [format_address_for_person(person) for person in expected],
2792- observed)
2793+ info = fetch_information(None, None, changes)
2794+ observed, _ = PackageUploadMailer.getRecipientsForAction(
2795+ 'accepted', info, blamer, None, [], archive, distroseries,
2796+ PackagePublishingPocket.RELEASE)
2797+ self.assertThat(observed, KeysEqual(*expected))
2798
2799- def test_get_upload_notification_recipients_good_emails(self):
2800- # Test get_upload_notification_recipients with good email addresses..
2801+ def test_getRecipientsForAction_good_emails(self):
2802+ # Test getRecipientsForAction with good email addresses..
2803 blamer, maintainer, changer = self._setup_recipients()
2804 changes = {
2805 'Date': '2001-01-01',
2806@@ -376,7 +393,7 @@
2807 [blamer, maintainer, changer],
2808 changes, blamer, maintainer, changer)
2809
2810- def test_get_upload_notification_recipients_bad_maintainer_email(self):
2811+ def test_getRecipientsForAction_bad_maintainer_email(self):
2812 blamer, maintainer, changer = self._setup_recipients()
2813 changes = {
2814 'Date': '2001-01-01',
2815@@ -387,9 +404,8 @@
2816 self.assertRecipientsEqual(
2817 [blamer, changer], changes, blamer, maintainer, changer)
2818
2819- def test_get_upload_notification_recipients_bad_changedby_email(self):
2820- # Test get_upload_notification_recipients with invalid changedby
2821- # email address.
2822+ def test_getRecipientsForAction_bad_changedby_email(self):
2823+ # Test getRecipientsForAction with invalid changedby email address.
2824 blamer, maintainer, changer = self._setup_recipients()
2825 changes = {
2826 'Date': '2001-01-01',
2827@@ -400,7 +416,7 @@
2828 self.assertRecipientsEqual(
2829 [blamer, maintainer], changes, blamer, maintainer, changer)
2830
2831- def test_get_upload_notification_recipients_unsigned_copy_archive(self):
2832+ def test_getRecipientsForAction_unsigned_copy_archive(self):
2833 # Notifications for unsigned build uploads to copy archives only go
2834 # to the archive owner.
2835 _, maintainer, changer = self._setup_recipients()
2836@@ -414,9 +430,92 @@
2837 [], changes, None, maintainer, changer,
2838 purpose=ArchivePurpose.COPY)
2839
2840- def test_assemble_body_handles_no_preferred_email_for_changer(self):
2841+ def test__getHeaders_primary(self):
2842+ # _getHeaders returns useful values for headers used for filtering.
2843+ # For a primary archive, this includes the maintainer and changer.
2844+ blamer, maintainer, changer = self._setup_recipients()
2845+ distroseries = self.factory.makeDistroSeries()
2846+ archive = distroseries.distribution.main_archive
2847+ component = getUtility(IComponentSet).ensure("main")
2848+ if component not in distroseries.components:
2849+ self.factory.makeComponentSelection(
2850+ distroseries=distroseries, component=component)
2851+ archive.newComponentUploader(maintainer, component)
2852+ archive.newComponentUploader(changer, component)
2853+ spr = self.factory.makeSourcePackageRelease(
2854+ component=component, section_name="libs")
2855+ changes = {
2856+ 'Date': '2001-01-01',
2857+ 'Changed-By': 'Changer <changer@example.com>',
2858+ 'Maintainer': 'Maintainer <maintainer@example.com>',
2859+ 'Changes': ' * Foo!',
2860+ }
2861+ mailer = PackageUploadMailer.forAction(
2862+ "accepted", blamer, spr, [], [], archive, distroseries,
2863+ PackagePublishingPocket.RELEASE, changes=changes)
2864+ recipients = dict(mailer._recipients.getRecipientPersons())
2865+ for email, rationale in (
2866+ (blamer.preferredemail.email, "Requester"),
2867+ ("maintainer@example.com", "Maintainer"),
2868+ ("changer@example.com", "Changed-By")):
2869+ headers = mailer._getHeaders(email, recipients[email])
2870+ self.assertThat(
2871+ headers,
2872+ ContainsDict({
2873+ "X-Launchpad-Message-Rationale": Equals(rationale),
2874+ "X-Launchpad-Notification-Type": Equals("package-upload"),
2875+ "X-Katie": Equals("Launchpad actually"),
2876+ "X-Launchpad-Archive": Equals(archive.reference),
2877+ "X-Launchpad-Component": Equals(
2878+ "component=main, section=libs"),
2879+ }))
2880+ self.assertNotIn("X-Launchpad-PPA", headers)
2881+
2882+ def test__getHeaders_ppa(self):
2883+ # _getHeaders returns useful values for headers used for filtering.
2884+ # For a PPA, this includes other people with component upload
2885+ # permissions.
2886+ blamer = self.factory.makePerson()
2887+ uploader = self.factory.makePerson()
2888+ distroseries = self.factory.makeUbuntuDistroSeries()
2889+ archive = self.factory.makeArchive(
2890+ distribution=distroseries.distribution, purpose=ArchivePurpose.PPA)
2891+ component = getUtility(IComponentSet).ensure("main")
2892+ if component not in distroseries.components:
2893+ self.factory.makeComponentSelection(
2894+ distroseries=distroseries, component=component)
2895+ archive.newComponentUploader(uploader, component)
2896+ spr = self.factory.makeSourcePackageRelease(
2897+ component=component, section_name="libs")
2898+ changes = {
2899+ 'Date': '2001-01-01',
2900+ 'Changed-By': 'Changer <changer@example.com>',
2901+ 'Maintainer': 'Maintainer <maintainer@example.com>',
2902+ 'Changes': ' * Foo!',
2903+ }
2904+ mailer = PackageUploadMailer.forAction(
2905+ "accepted", blamer, spr, [], [], archive, distroseries,
2906+ PackagePublishingPocket.RELEASE, changes=changes)
2907+ recipients = dict(mailer._recipients.getRecipientPersons())
2908+ for email, rationale in (
2909+ (blamer.preferredemail.email, "Requester"),
2910+ (uploader.preferredemail.email, "PPA Uploader")):
2911+ headers = mailer._getHeaders(email, recipients[email])
2912+ self.assertThat(
2913+ headers,
2914+ ContainsDict({
2915+ "X-Launchpad-Message-Rationale": Equals(rationale),
2916+ "X-Launchpad-Notification-Type": Equals("package-upload"),
2917+ "X-Katie": Equals("Launchpad actually"),
2918+ "X-Launchpad-Archive": Equals(archive.reference),
2919+ "X-Launchpad-PPA": Equals(get_ppa_reference(archive)),
2920+ "X-Launchpad-Component": Equals(
2921+ "component=main, section=libs"),
2922+ }))
2923+
2924+ def test__getTemplateParams_handles_no_preferred_email_for_changer(self):
2925 # If changer has no preferred email address,
2926- # assemble_body should still work.
2927+ # _getTemplateParams should still work.
2928 spr = self.factory.makeSourcePackageRelease()
2929 blamer = self.factory.makePerson()
2930 archive = self.factory.makeArchive()
2931@@ -424,11 +523,14 @@
2932
2933 spr.creator.setPreferredEmail(None)
2934
2935- body = assemble_body(blamer, spr, [], archive, series, "",
2936- None, "unapproved")
2937- self.assertIn("Waiting for approval", body)
2938+ mailer = PackageUploadMailer.forAction(
2939+ "unapproved", blamer, spr, [], [], archive, series,
2940+ PackagePublishingPocket.RELEASE)
2941+ email, recipient = list(mailer._recipients.getRecipientPersons())[0]
2942+ params = mailer._getTemplateParams(email, recipient)
2943+ self.assertEqual("Waiting for approval", params["STATUS"])
2944
2945- def test_assemble_body_inserts_package_url_for_distro_upload(self):
2946+ def test__getTemplateParams_inserts_package_url_for_distro_upload(self):
2947 # The email body should contain the canonical url to the package
2948 # page in the target distroseries.
2949 spr = self.factory.makeSourcePackageRelease()
2950@@ -436,13 +538,16 @@
2951 archive = self.factory.makeArchive(purpose=ArchivePurpose.PRIMARY)
2952 series = self.factory.makeDistroSeries()
2953
2954- body = assemble_body(blamer, spr, [], archive, series, "",
2955- None, "unapproved")
2956+ mailer = PackageUploadMailer.forAction(
2957+ "unapproved", blamer, spr, [], [], archive, series,
2958+ PackagePublishingPocket.RELEASE)
2959+ email, recipient = list(mailer._recipients.getRecipientPersons())[0]
2960+ params = mailer._getTemplateParams(email, recipient)
2961 dsspr = DistributionSourcePackageRelease(series.distribution, spr)
2962 url = canonical_url(dsspr)
2963- self.assertIn(url, body)
2964+ self.assertEqual(url, params["SPR_URL"])
2965
2966- def test__is_auto_sync_upload__no_preferred_email_for_changer(self):
2967+ def test_is_auto_sync_upload__no_preferred_email_for_changer(self):
2968 # If changer has no preferred email address,
2969 # is_auto_sync_upload should still work.
2970 result = is_auto_sync_upload(
2971
2972=== modified file 'lib/lp/soyuz/model/queue.py'
2973--- lib/lp/soyuz/model/queue.py 2015-07-08 16:05:11 +0000
2974+++ lib/lp/soyuz/model/queue.py 2015-08-26 13:41:53 +0000
2975@@ -82,7 +82,6 @@
2976 cachedproperty,
2977 get_property_cache,
2978 )
2979-from lp.soyuz.adapters.notification import notify
2980 from lp.soyuz.enums import (
2981 PackageUploadCustomFormat,
2982 PackageUploadStatus,
2983@@ -115,6 +114,7 @@
2984 QueueStateWriteProtectedError,
2985 )
2986 from lp.soyuz.interfaces.section import ISectionSet
2987+from lp.soyuz.mail.packageupload import PackageUploadMailer
2988 from lp.soyuz.model.binarypackagename import BinaryPackageName
2989 from lp.soyuz.model.binarypackagerelease import BinaryPackageRelease
2990 from lp.soyuz.model.component import Component
2991@@ -915,11 +915,14 @@
2992 else:
2993 changesfile_content = 'No changes file content available.'
2994 blamee = self.findPersonToNotify()
2995- notify(
2996- blamee, self.sourcepackagerelease, self.builds, self.customfiles,
2997- self.archive, self.distroseries, self.pocket, summary_text,
2998- changes, changesfile_content, changes_file_object,
2999- status_action[self.status], dry_run=dry_run, logger=logger)
3000+ mailer = PackageUploadMailer.forAction(
3001+ status_action[self.status], blamee, self.sourcepackagerelease,
3002+ self.builds, self.customfiles, self.archive, self.distroseries,
3003+ self.pocket, summary_text=summary_text, changes=changes,
3004+ changesfile_content=changesfile_content,
3005+ changesfile_object=changes_file_object, dry_run=dry_run,
3006+ logger=logger)
3007+ mailer.sendAll()
3008
3009 @property
3010 def components(self):
3011
3012=== modified file 'lib/lp/soyuz/scripts/packagecopier.py'
3013--- lib/lp/soyuz/scripts/packagecopier.py 2015-07-09 20:06:17 +0000
3014+++ lib/lp/soyuz/scripts/packagecopier.py 2015-08-26 13:41:53 +0000
3015@@ -1,4 +1,4 @@
3016-# Copyright 2009-2013 Canonical Ltd. This software is licensed under the
3017+# Copyright 2009-2015 Canonical Ltd. This software is licensed under the
3018 # GNU Affero General Public License version 3 (see the file LICENSE).
3019
3020 """Package copying utilities."""
3021@@ -19,7 +19,6 @@
3022 from zope.security.proxy import removeSecurityProxy
3023
3024 from lp.services.database.bulk import load_related
3025-from lp.soyuz.adapters.notification import notify
3026 from lp.soyuz.adapters.overrides import SourceOverride
3027 from lp.soyuz.enums import SourcePackageFormat
3028 from lp.soyuz.interfaces.archive import CannotCopy
3029@@ -31,6 +30,7 @@
3030 ISourcePackagePublishingHistory,
3031 )
3032 from lp.soyuz.interfaces.queue import IPackageUploadCustom
3033+from lp.soyuz.mail.packageupload import PackageUploadMailer
3034 from lp.soyuz.model.processacceptedbugsjob import (
3035 close_bugs_for_sourcepublication,
3036 )
3037@@ -572,9 +572,11 @@
3038 if series is None:
3039 series = source.distroseries
3040 # In zopeless mode this email will be sent immediately.
3041- notify(
3042- person, source.sourcepackagerelease, [], [], archive,
3043- series, pocket, summary_text=error_text, action='rejected')
3044+ mailer = PackageUploadMailer.forAction(
3045+ 'rejected', person, source.sourcepackagerelease, [], [],
3046+ archive, series, pocket, summary_text=error_text,
3047+ logger=logger)
3048+ mailer.sendAll()
3049 raise CannotCopy(error_text)
3050
3051 overrides_index = 0
3052@@ -610,14 +612,15 @@
3053 sponsor=sponsor, packageupload=packageupload,
3054 phased_update_percentage=phased_update_percentage, logger=logger)
3055 if send_email:
3056- notify(
3057- person, source.sourcepackagerelease, [], [], archive,
3058- destination_series, pocket, action='accepted',
3059+ mailer = PackageUploadMailer.forAction(
3060+ 'accepted', person, source.sourcepackagerelease, [], [],
3061+ archive, destination_series, pocket,
3062 announce_from_person=announce_from_person,
3063- previous_version=old_version)
3064+ previous_version=old_version, logger=logger)
3065+ mailer.sendAll()
3066 if not archive.private and has_restricted_files(source):
3067 # Fix copies by unrestricting files with privacy mismatch.
3068- # We must do this *after* calling notify (which only
3069+ # We must do this *after* calling mailer.sendAll (which only
3070 # actually sends mail on commit), because otherwise the new
3071 # changelog LFA won't be visible without a commit, which may
3072 # not be safe here.
3073
3074=== modified file 'lib/lp/soyuz/scripts/tests/test_copypackage.py'
3075--- lib/lp/soyuz/scripts/tests/test_copypackage.py 2015-05-19 02:24:48 +0000
3076+++ lib/lp/soyuz/scripts/tests/test_copypackage.py 2015-08-26 13:41:53 +0000
3077@@ -1,13 +1,10 @@
3078-# Copyright 2009-2014 Canonical Ltd. This software is licensed under the
3079+# Copyright 2009-2015 Canonical Ltd. This software is licensed under the
3080 # GNU Affero General Public License version 3 (see the file LICENSE).
3081
3082 __metaclass__ = type
3083
3084 import datetime
3085-from textwrap import (
3086- dedent,
3087- fill,
3088- )
3089+from textwrap import dedent
3090
3091 import pytz
3092 from testtools.content import text_content
3093@@ -1426,24 +1423,20 @@
3094 [notification] = pop_notifications()
3095 self.assertEqual(
3096 target_archive.reference, notification['X-Launchpad-Archive'])
3097- body = notification.get_payload()[0].get_payload()
3098- expected = (dedent("""\
3099+ body = notification.get_payload(decode=True)
3100+ expected = dedent("""\
3101 Accepted:
3102 OK: foo_1.0-2.dsc
3103 -> Component: main Section: base
3104
3105- foo (1.0-2) unstable; urgency=3Dlow
3106+ foo (1.0-2) unstable; urgency=low
3107
3108 * 1.0-2.
3109
3110- --
3111+ %s
3112 http://launchpad.dev/~archiver/+archive/ubuntutest/ppa
3113- """) +
3114- # Slight contortion to avoid a long line.
3115- fill(dedent("""\
3116- You are receiving this email because you are the uploader of the
3117- above PPA package.
3118- """), 72) + "\n")
3119+ You are receiving this email because you made this upload.
3120+ """ % "-- ")
3121 self.assertEqual(expected, body)
3122
3123 def test_copy_generates_notification(self):
3124@@ -1481,8 +1474,7 @@
3125 # Spurious newlines are a pain and don't really affect the end
3126 # results so stripping is the easiest route here.
3127 expected_text.strip()
3128- body = mail.get_payload()[0].get_payload()
3129- self.assertEqual(expected_text, body)
3130+ body = announcement.get_payload()[0].get_payload()
3131 self.assertEqual(expected_text, body)
3132
3133 def test_sponsored_copy_notification(self):
3134
3135=== modified file 'lib/lp/soyuz/stories/soyuz/xx-queue-pages.txt'
3136--- lib/lp/soyuz/stories/soyuz/xx-queue-pages.txt 2013-09-27 04:13:23 +0000
3137+++ lib/lp/soyuz/stories/soyuz/xx-queue-pages.txt 2015-08-26 13:41:53 +0000
3138@@ -305,7 +305,7 @@
3139 Swallow any email generated at the upload:
3140
3141 >>> from lp.services.mail import stub
3142- >>> from lp.testing.mail_helpers import pop_notifications, sort_addresses
3143+ >>> from lp.testing.mail_helpers import pop_notifications
3144 >>> swallow = pop_notifications()
3145
3146 Set up a second browser on the same page to simulate accidentally posting to
3147@@ -347,9 +347,11 @@
3148 if it is someone other than the uploader) and (usually) an email to the
3149 distroseries' announcement list (see nascentupload-announcements.txt).
3150
3151- >>> [notification, announcement] = pop_notifications()
3152- >>> print sort_addresses(notification['To'])
3153- Daniel Silverstone <daniel.silverstone@canonical.com>,
3154+ >>> [changer_notification, signer_notification,
3155+ ... announcement] = pop_notifications()
3156+ >>> print changer_notification['To']
3157+ Daniel Silverstone <daniel.silverstone@canonical.com>
3158+ >>> print signer_notification['To']
3159 Foo Bar <foo.bar@canonical.com>
3160 >>> print announcement['To']
3161 autotest_changes@ubuntu.com
3162@@ -511,6 +513,8 @@
3163
3164 Rejecting 'alsa-utils' source:
3165
3166+ >>> stub.test_emails = []
3167+
3168 >>> upload_manager_browser.getControl(name="QUEUE_ID").value = ['4']
3169 >>> upload_manager_browser.getControl(name="Reject").disabled
3170 False
3171@@ -543,8 +547,8 @@
3172 Rejected:
3173 Rejected by Sample Person: Foo
3174 ...
3175- You are receiving this email because you are the uploader, maintainer or
3176- signer of the above package.
3177+ You are receiving this email because you are the most recent person
3178+ listed in this package's changelog.
3179 <BLANKLINE>
3180
3181 The override controls are now available for rejected packages.
3182
3183=== modified file 'lib/lp/soyuz/tests/test_distroseriesqueue_debian_installer.py'
3184--- lib/lp/soyuz/tests/test_distroseriesqueue_debian_installer.py 2015-07-21 09:04:01 +0000
3185+++ lib/lp/soyuz/tests/test_distroseriesqueue_debian_installer.py 2015-08-26 13:41:53 +0000
3186@@ -1,4 +1,4 @@
3187-# Copyright 2012 Canonical Ltd. This software is licensed under the
3188+# Copyright 2012-2015 Canonical Ltd. This software is licensed under the
3189 # GNU Affero General Public License version 3 (see the file LICENSE).
3190
3191 """Test upload and queue manipulation of debian-installer custom uploads.
3192@@ -7,6 +7,7 @@
3193 of debian-installer custom upload extraction.
3194 """
3195
3196+from itertools import chain
3197 import os
3198
3199 import transaction
3200@@ -52,11 +53,16 @@
3201 self.assertEqual(1, len(upload.queue_root.customfiles))
3202
3203 def test_generates_mail(self):
3204- # Two email messages were generated (acceptance and announcement).
3205+ # Three email messages were generated (acceptance to signer,
3206+ # acceptance to changer, and announcement).
3207 self.anything_policy.setDistroSeriesAndPocket("hoary-test")
3208 self.anything_policy.distroseries.changeslist = "announce@example.com"
3209 self.uploadTestData()
3210- self.assertEqual(2, len(stub.test_emails))
3211+ self.assertContentEqual(
3212+ ["announce@example.com", "celso.providelo@canonical.com",
3213+ "foo.bar@canonical.com"],
3214+ list(chain.from_iterable(
3215+ [to_addrs for _, to_addrs, _ in stub.test_emails])))
3216
3217 def test_bad_upload_remains_in_accepted(self):
3218 # Bad debian-installer uploads remain in accepted. Simulate an
3219
3220=== modified file 'lib/lp/soyuz/tests/test_packagecopyjob.py'
3221--- lib/lp/soyuz/tests/test_packagecopyjob.py 2015-04-09 05:16:37 +0000
3222+++ lib/lp/soyuz/tests/test_packagecopyjob.py 2015-08-26 13:41:53 +0000
3223@@ -1230,12 +1230,14 @@
3224 # do it here.
3225 emails = pop_notifications(sort_key=operator.itemgetter('To'))
3226
3227- # We expect an uploader email and an announcement to the changeslist.
3228- self.assertEqual(2, len(emails))
3229- self.assertIn("requester@example.com", emails[0]['To'])
3230- self.assertIn("changes@example.com", emails[1]['To'])
3231+ # We expect an email to the signer, an email to the uploader, and an
3232+ # announcement to the changeslist.
3233+ self.assertEqual(3, len(emails))
3234+ self.assertIn("foo.bar@canonical.com", emails[0]['To'])
3235+ self.assertIn("requester@example.com", emails[1]['To'])
3236+ self.assertIn("changes@example.com", emails[2]['To'])
3237 self.assertEqual(
3238- "Nancy Requester <requester@example.com>", emails[1]['From'])
3239+ "Nancy Requester <requester@example.com>", emails[2]['From'])
3240
3241 def test_silent(self):
3242 # Copies into a non-PPA archive normally send emails. They can
3243
3244=== modified file 'lib/lp/soyuz/tests/test_packageupload.py'
3245--- lib/lp/soyuz/tests/test_packageupload.py 2015-08-03 12:59:18 +0000
3246+++ lib/lp/soyuz/tests/test_packageupload.py 2015-08-26 13:41:53 +0000
3247@@ -1,4 +1,4 @@
3248-# Copyright 2009-2013 Canonical Ltd. This software is licensed under the
3249+# Copyright 2009-2015 Canonical Ltd. This software is licensed under the
3250 # GNU Affero General Public License version 3 (see the file LICENSE).
3251
3252 """Test Build features."""
3253@@ -29,7 +29,6 @@
3254 from lp.services.job.interfaces.job import JobStatus
3255 from lp.services.librarian.browser import ProxiedLibraryFileAlias
3256 from lp.services.mail import stub
3257-from lp.services.mail.sendmail import format_address_for_person
3258 from lp.soyuz.adapters.overrides import SourceOverride
3259 from lp.soyuz.enums import (
3260 PackagePublishingStatus,
3261@@ -195,9 +194,9 @@
3262 upload, uploader = self.makeSourcePackageUpload()
3263 upload.acceptFromQueue()
3264 self.assertEqual(2, len(stub.test_emails))
3265- # Emails sent are the announcement and the uploader's notification:
3266+ # Emails sent are the uploader's notification and the announcement:
3267+ self.assertEmail([uploader.preferredemail.email])
3268 self.assertEmail(["autotest_changes@ubuntu.com"])
3269- self.assertEmail([format_address_for_person(uploader)])
3270
3271 def test_acceptFromQueue_source_backports_sends_no_announcement(self):
3272 # Accepting a source package into BACKPORTS does not send an
3273@@ -211,7 +210,7 @@
3274 self.assertEqual(1, len(stub.test_emails))
3275 # Only one email is sent, to the person in the changed-by field. No
3276 # announcement email is sent.
3277- self.assertEmail([format_address_for_person(uploader)])
3278+ self.assertEmail([uploader.preferredemail.email])
3279
3280 def test_acceptFromQueue_source_translations_sends_no_email(self):
3281 # Accepting source packages in the "translations" section (i.e.
3282@@ -316,7 +315,7 @@
3283 upload, uploader = self.makeSourcePackageUpload()
3284 upload.rejectFromQueue(self.factory.makePerson())
3285 self.assertEqual(1, len(stub.test_emails))
3286- self.assertEmail([format_address_for_person(uploader)])
3287+ self.assertEmail([uploader.preferredemail.email])
3288
3289 def test_rejectFromQueue_binary_sends_email(self):
3290 # Rejecting a binary package sends an email to the uploader.
3291@@ -324,7 +323,7 @@
3292 upload, uploader = self.makeBuildPackageUpload()
3293 upload.rejectFromQueue(self.factory.makePerson())
3294 self.assertEqual(1, len(stub.test_emails))
3295- self.assertEmail([format_address_for_person(uploader)])
3296+ self.assertEmail([uploader.preferredemail.email])
3297
3298 def test_rejectFromQueue_source_translations_sends_no_email(self):
3299 # Rejecting a language pack sends no email.