Merge lp:~danilo/launchpad/bug-410579 into lp:launchpad

Proposed by Данило Шеган
Status: Merged
Approved by: Barry Warsaw
Approved revision: no longer in the source branch.
Merged at revision: not available
Proposed branch: lp:~danilo/launchpad/bug-410579
Merge into: lp:launchpad
Diff against target: None lines
To merge this branch: bzr merge lp:~danilo/launchpad/bug-410579
Reviewer Review Type Date Requested Status
Barry Warsaw (community) Approve
Review via email: mp+10179@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Данило Шеган (danilo) wrote :
Download full text (5.0 KiB)

= Summary =

This branch fixes #92751 (#410579 is a duplicate which I originally
started on), doing a bunch of drive-by cleanups along the way.

Our current download email for translations is very terse and not very
informative. It goes something like the following:

Subject: Translation download request: danilo
> Hello Данило Шеган,
>
> The translation files you requested from Launchpad are ready for
> download from the following location:
>
> http://launchpadlibrarian.net/30048422/po_kio4-es.po

== Proposed fix ==

We should include more information, like what the request is for, and
how download link will expire in 1 week. We should also include
information on how you can re-request the download if you need to.

Subject should not mention your username, but instead a short
description of what the download is about.

== Implementation details ==

The branch has quickly gotten very big (~900 lines), but it's mostly
because of textual changes to email templates. I hope it's not a problem.

 * scripts/po_export_queue.py:
   This is where the core change is: everything related to emailing is moved into
   ExportResult implementation, which generates appropriate emails.

   Two methods are of notice:

     * _getExportRequestOrigin(): figures out where export request was most
       likely made;
       TranslationImportQueue keeps only a list of POTemplate and POFile objects
       and depending this method figures out the place where the request was
       "most likely" made (i.e. context where you could have requested these
       files, but then trying for the most relevant context as well).

       For instance, if we've got two pofiles on a single template, request was
       likely made on a potemplate object, so we return that (for nicer URLs
       and emails subjects though, if there's only one template in
       a productseries/sourcepackage, we return them instead).

     * _getShortRequestName(request) takes the context returned by previous method
       and returns a nice short description in the form of 'Evolution trunk' or
       'Ubuntu Karmic gimp-2.0 - Spanish translation' suitable for use as email
       subjects

   These are not unit tested, though they could be, simply because they are
   sufficiently well exercised by existing doctests, and because if these start
   failing, it'll be easy to notice (so, I'd rather not increase CPU cycles
   we spend on tests). They are mostly for 'nicer UI output', so their correctness
   is not critical to the operation.

   Other changes in this file are mostly for using emailtemplates instead of hardcoding text.

 * emailtemplates/poexport-*.txt:
   All the email templates have been moved out of the code and into separate
   template files. Some 75 lines of diff are basically

 * doc/poexport-request.txt,
   doc/poexport-request-productseries.txt,
   doc/poexport-queue.txt:
   There are a lot of changes to tests that test how emails are generated:
   basically, half of the branch is there (440 lines). The changes there
   include introducing a shared helper print_mail_subject_and_body method,
   print subject along with body of the email, and changes to test new
   e...

Read more...

Revision history for this message
Barry Warsaw (barry) wrote :
Download full text (19.0 KiB)

On Aug 14, 2009, at 06:35 PM, Данило Шеган wrote:

> Данило Шеган has proposed merging lp:~danilo/launchpad/
> bug-410579 into lp:launchpad/devel.
>
> We should include more information, like what the request is for, and
> how download link will expire in 1 week. We should also include
> information on how you can re-request the download if you need to.

Thanks for working on this Danilo. It looks like a very useful change
that
will help our users!

> Subject should not mention your username, but instead a short
> description of what the download is about.
>
> == Implementation details ==
>
> The branch has quickly gotten very big (~900 lines), but it's mostly
> because of textual changes to email templates. I hope it's not a
> problem.
>
> * scripts/po_export_queue.py:
> This is where the core change is: everything related to emailing is
> moved into
> ExportResult implementation, which generates appropriate emails.
>
> Two methods are of notice:
>
> * _getExportRequestOrigin(): figures out where export request was
> most
> likely made;
> TranslationImportQueue keeps only a list of POTemplate and
> POFile objects
> and depending this method figures out the place where the
> request was
> "most likely" made (i.e. context where you could have requested
> these
> files, but then trying for the most relevant context as well).
>
> For instance, if we've got two pofiles on a single template,
> request was
> likely made on a potemplate object, so we return that (for
> nicer URLs
> and emails subjects though, if there's only one template in
> a productseries/sourcepackage, we return them instead).
>
> * _getShortRequestName(request) takes the context returned by
> previous method
> and returns a nice short description in the form of 'Evolution
> trunk' or
> 'Ubuntu Karmic gimp-2.0 - Spanish translation' suitable for use
> as email
> subjects
>
> These are not unit tested, though they could be, simply because
> they are
> sufficiently well exercised by existing doctests, and because if
> these start
> failing, it'll be easy to notice (so, I'd rather not increase CPU
> cycles
> we spend on tests). They are mostly for 'nicer UI output', so
> their correctness
> is not critical to the operation.

Do you really think it would take up that much more cycles to unit
test these
two methods? I'm generally not in favor of these kinds of indirect
test when
they can be reasonably avoided because that makes it much more
difficult to
change the methods later. You never really quite know what the effect
will
be, so you end up having to run the full test suite "just in case".
With a
unit test, you at least know you need to run that test if you change the
method, and can presume that if those unit tests pass, everything else
implicitly testing them will too.

>
> Other changes in this file are mostly for using emailtemplates
> instead of hardcoding text.
>
> * emailtemplates/poexport-*.txt:
> All the email templates have been moved out of the code and into
> separate
> template files. Some 75 lines of diff are basically
>
> * doc/poexpor...

Revision history for this message
Barry Warsaw (barry) wrote :

merge-conditional

review: Approve
Revision history for this message
Данило Шеган (danilo) wrote :
Download full text (11.5 KiB)

Barry, thanks for the review.

I've applied most of your recommendations. The ones I didn't were about
'positive tests' (i.e. is 'is not None' really a negative test?) and I
haven't provided unit tests. Still, I've modified doctests to use
mail_helpers, so that resulted in a nice big incremental diff.

У пет, 14. 08 2009. у 16:05 -0400, Barry Warsaw пише:
> > * doc/poexport-request.txt,
> > doc/poexport-request-productseries.txt,
> > doc/poexport-queue.txt:
> > There are a lot of changes to tests that test how emails are generated:
> > basically, half of the branch is there (440 lines). The changes there
> > include introducing a shared helper print_mail_subject_and_body method,
> > print subject along with body of the email, and changes to test new
> > email contents and subjects. Some minor changes to how ExportResult is
> > constructed were necessary as well to conform with the new API (i.e. pass
> > person, requested exports and logger).
>
> Do you know about lp.testing.mail_helpers.print_emails()? I generally use
> that plus ellipses to cut down on the output. Is there a reason why you'd
> want print_mail_subject_and_body instead?

Yes, I've looked through existing helpers. I've already struggled to
keep the diff size reasonable, and this would have made it only worse.
Though, I am happy to migrate existing tests to the use them, and I did
so. Note that only this change means 640 lines of diff in the
incremental diff ;)

> > To not increase size of the diff, some bits of the test are left
> > indented with 2 spaces. I'd be happy to fix before submitting.
>
> Sure, thanks! rs=me on those additional cleanups.

Thanks.

> For the most part, the branch looks fine. I do have a few questions and
> suggestions, but I'll mark this as merge-conditional on the hope that those
> are easy to address or answer. I'm happy to answer any questions or chat
> about my suggestions with you on Monday.

Great, thanks.

> === added file 'lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt'
> --- lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt 1970-01-01 00:00:00 +0000
> +++ lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt 2009-08-14 16:33:57 +0000
> > @@ -0,0 +1,16 @@
> > +Hello admins,
>
> Maybe "Hello Launchpad administrators," ?

Sure, fixed. Note that these emails only ended up on our internal
error-reports list, but no reason not to make them even nicer :)

> > +--
>
> Note that the signature separator is actually "-- " yep with a trailing space!
> I know that kind of screws the no-trailing-whitespace thing, but there you
> have it. ;)

Ah, interesting. Fixed all over the place.

> > +Automatic message from Launchpad.net — https://launchpad.net/
>
> Wow, this is like a failure failure. :)

My mail client linkifies the second, but not the first. I still prefer
the former in an email message, so I made it

  "Automatic message from Launchpad.net."

in all the email templates.

> === modified file 'lib/lp/translations/scripts/po_export_queue.py'
> --- lib/lp/translations/scripts/po_export_queue.py 2009-07-17 00:26:05 +0000
> +++ lib/lp/...

=== modified file 'lib/lp/testing/mail_helpers.py'
--- lib/lp/testing/mail_helpers.py 2009-07-17 00:26:05 +0000
+++ lib/lp/testing/mail_helpers.py 2009-08-17 12:22:39 +0000
@@ -22,7 +22,7 @@ def pop_notifications(sort_key=None, com
22 :param sort_key: define sorting function. sort_key specifies a22 :param sort_key: define sorting function. sort_key specifies a
23 function of one argument that is used to extract a comparison key from23 function of one argument that is used to extract a comparison key from
24 each list element. (See the sorted() Python built-in.)24 each list element. (See the sorted() Python built-in.)
25 :param commit: whether to commit before reading email (defauls to False).25 :param commit: whether to commit before reading email (defaults to True).
26 """26 """
27 if commit:27 if commit:
28 transaction.commit()28 transaction.commit()
2929
=== modified file 'lib/lp/translations/doc/poexport-queue.txt'
--- lib/lp/translations/doc/poexport-queue.txt 2009-08-14 17:46:32 +0000
+++ lib/lp/translations/doc/poexport-queue.txt 2009-08-17 13:15:52 +0000
@@ -12,9 +12,8 @@ and the ones that failed with the error
12 >>> import transaction12 >>> import transaction
13 >>> from zope.component import getUtility13 >>> from zope.component import getUtility
14 >>> from canonical.launchpad.interfaces import IPersonSet14 >>> from canonical.launchpad.interfaces import IPersonSet
15 >>> from lp.services.mail.stub import test_emails15 >>> from lp.testing.mail_helpers import pop_notifications, print_emails
16 >>> from lp.translations.scripts.po_export_queue import ExportResult16 >>> from lp.translations.scripts.po_export_queue import ExportResult
17 >>> from lp.translations.tests.helpers import print_mail_subject_and_body
18 >>> import logging17 >>> import logging
19 >>> logger = logging.getLogger()18 >>> logger = logging.getLogger()
2019
@@ -57,76 +56,73 @@ In this case, it should be None, so the
57 >>> result.url = None56 >>> result.url = None
58 >>> result.notify()57 >>> result.notify()
5958
60 # We need to commit the transaction to get the emails in the queue.
61 >>> transaction.commit()
62
63As usual, when there is an error, two emails should be sent:59As usual, when there is an error, two emails should be sent:
6460
61 >>> test_emails = pop_notifications()
65 >>> len(test_emails)62 >>> len(test_emails)
66 263 2
6764
68 >>> while len(test_emails) > 0:65 >>> for email in test_emails:
69 ... from_addrs, to_addrs, body = test_emails.pop()66 ... if 'carlos@canonical.com' in email['to']:
70 ... if 'carlos@canonical.com' in to_addrs:67 ... carlos_email = email
71 ... carlos_addrs = to_addrs
72 ... carlos_body = body
73 ... else:68 ... else:
74 ... admins_addrs = to_addrs69 ... admin_email = email
75 ... admins_body = body
7670
77One is for the user with the error notification.71One is for the user with the error notification.
7872
79 >>> print carlos_addrs73 >>> print_emails(notifications=[carlos_email])
80 ['carlos@canonical.com']74 From: ...
81 >>> print_mail_subject_and_body(carlos_body)75 To: carlos@canonical.com
82 Subject: Launchpad translation download: Evolution trunk -76 Subject: Launchpad translation download: Evolution trunk -
83 evolution-2.2 template77 evolution-2.2 template
84 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,78 Hello Carlos Perell=C3=B3 Mar=C3=ADn,
85 >79 <BLANKLINE>
86 > Launchpad encountered problems exporting the files you requested.80 Launchpad encountered problems exporting the files you requested.
87 > The Launchpad Translations team has been notified of this problem.81 The Launchpad Translations team has been notified of this problem.
88 > Please reply to this email for further assistance.82 Please reply to this email for further assistance.
89 >83 <BLANKLINE>
90 > If you want to retry your request, you can do so at84 If you want to retry your request, you can do so at
91 >85 <BLANKLINE>
92 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=86 http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
93 > port.87 port.
94 >88 <BLANKLINE>
95 > --89 -- =
96 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/90 Automatic message from Launchpad.net.
97 >91 <BLANKLINE>
92 ----------------------------------------
9893
99And the other to the admins. This one lists the files that were being94And the other to the admins. This one lists the files that were being
100exported as context to help tracking down any bugs.95exported as context to help tracking down any bugs.
10196
102 >>> print admins_addrs97 >>> print_emails(notifications=[admin_email])
103 ['launchpad-error-reports@lists.canonical.com']98 From: ...
104 >>> print_mail_subject_and_body(admins_body)99 To: launchpad-error-reports@lists.canonical.com
105 Subject: Launchpad translation download errors: Evolution trunk -100 Subject: Launchpad translation download errors: Evolution trunk -
106 evolution-2.2 template101 evolution-2.2 template
107 > Hello admins,102 Hello Launchpad administrators,
108 >103 <BLANKLINE>
109 > Launchpad encountered problems exporting translation files104 Launchpad encountered problems exporting translation files
110 > requested by Carlos Perell=C3=B3 Mar=C3=ADn (carlos) at105 requested by Carlos Perell=C3=B3 Mar=C3=ADn (carlos) at
111 >106 <BLANKLINE>
112 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=107 http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
113 > port108 port
114 >109 <BLANKLINE>
115 > This means we have a bug in Launchpad that needs to be fixed110 This means we have a bug in Launchpad that needs to be fixed
116 > before this export can proceed. Here is the error we got:111 before this export can proceed. Here is the error we got:
117 >112 <BLANKLINE>
118 > Traceback (most recent call last):113 Traceback (most recent call last):
119 ...114 ...
120 > AssertionError: It's just an error for testing purposes115 AssertionError: It's just an error for testing purposes
121 >116 <BLANKLINE>
122 >117 <BLANKLINE>
123 > Failed export request included:118 Failed export request included:
124 > * evolution-2.2 in Evolution trunk119 * evolution-2.2 in Evolution trunk
125 > * Spanish (es) translation of evolution-2.2 in Evolution trunk120 * Spanish (es) translation of evolution-2.2 in Evolution trunk
126 >121 <BLANKLINE>
127 > --122 -- =
128 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/123 Automatic message from Launchpad.net.
129 >124 <BLANKLINE>
125 ----------------------------------------
130126
131As a special case, some error messages are poisoned with non-ASCII characters127As a special case, some error messages are poisoned with non-ASCII characters
132and can't be reported without triggering an error themselves. Those are128and can't be reported without triggering an error themselves. Those are
@@ -137,61 +133,67 @@ specially handled and reported.
137 ... except AssertionError:133 ... except AssertionError:
138 ... result.addFailure()134 ... result.addFailure()
139 >>> result.notify()135 >>> result.notify()
140 >>> transaction.commit()136
137 >>> test_emails = pop_notifications()
141 >>> len(test_emails)138 >>> len(test_emails)
142 2139 2
143140
144 >>> carlos_body = None141 >>> carlos_email = None
145 >>> admins_body = None142 >>> admins_email = None
146 >>> while len(test_emails) > 0:143 >>> for email in test_emails:
147 ... from_addrs, to_addrs, body = test_emails.pop()144 ... if 'carlos@canonical.com' in email['to']:
148 ... if 'carlos@canonical.com' in to_addrs:145 ... carlos_email = email
149 ... carlos_body = body
150 ... else:146 ... else:
151 ... admins_body = body147 ... admin_email = email
152148
153The user's notification looks no different from that for an ordinary error.149The user's notification looks no different from that for an ordinary error.
154150
155 >>> print_mail_subject_and_body(carlos_body)151 >>> print_emails(notifications=[carlos_email])
152 From: ...
153 To: carlos@canonical.com
156 Subject: Launchpad translation download: Evolution trunk -154 Subject: Launchpad translation download: Evolution trunk -
157 evolution-2.2 template155 evolution-2.2 template
158 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,156 Hello Carlos Perell=C3=B3 Mar=C3=ADn,
159 >157 <BLANKLINE>
160 > Launchpad encountered problems exporting the files you requested.158 Launchpad encountered problems exporting the files you requested.
161 > The Launchpad Translations team has been notified of this problem.159 The Launchpad Translations team has been notified of this problem.
162 > Please reply to this email for further assistance.160 Please reply to this email for further assistance.
163 >161 <BLANKLINE>
164 > If you want to retry your request, you can do so at162 If you want to retry your request, you can do so at
165 >163 <BLANKLINE>
166 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=164 http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
167 > port.165 port.
168 >166 <BLANKLINE>
169 > --167 -- =
170 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/168 Automatic message from Launchpad.net.
171 >169 <BLANKLINE>
170 ----------------------------------------
172171
173The one for the administrators, however, does not include the unprintable172The one for the administrators, however, does not include the unprintable
174exception text.173exception text.
175174
176 >>> print_mail_subject_and_body(admins_body)175 >>> print_emails(notifications=[admin_email])
176 From: ...
177 To: launchpad-error-reports@lists.canonical.com
177 Subject: Launchpad translation download errors: Evolution trunk -178 Subject: Launchpad translation download errors: Evolution trunk -
178 evolution-2.2 template179 evolution-2.2 template
179 > Hello admins,180 Hello Launchpad administrators,
180 >181 <BLANKLINE>
181 > A UnicodeDecodeError occurred while trying to notify you of a182 A UnicodeDecodeError occurred while trying to notify you of a
182 > failure during a translation export requested by Carlos ...183 failure during a translation export requested by Carlos ...
183 > (carlos) at184 (carlos) at
184 >185 <BLANKLINE>
185 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=186 http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
186 > port187 port
187 >188 <BLANKLINE>
188 > Failed export request included:189 Failed export request included:
189 > * evolution-2.2 in Evolution trunk190 * evolution-2.2 in Evolution trunk
190 > * Spanish (es) translation of evolution-2.2 in Evolution trunk191 * Spanish (es) translation of evolution-2.2 in Evolution trunk
191 >192 <BLANKLINE>
192 > --193 -- =
193 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/194 Automatic message from Launchpad.net.
194 >195 <BLANKLINE>
196 ----------------------------------------
195197
196198
197Finally, there is the case when there are no errors at all. This is the usual199Finally, there is the case when there are no errors at all. This is the usual
@@ -207,46 +209,40 @@ download the requested files. If we don'
207 ...209 ...
208 AssertionError: On success, an exported URL is expected.210 AssertionError: On success, an exported URL is expected.
209211
210So let's add it:212So let's add it and notify the user:
211213
212 >>> result.url = 'http://someplace.com/somefile.tar.gz'214 >>> result.url = 'http://someplace.com/somefile.tar.gz'
213
214And notify the user.
215
216 >>> result.notify()215 >>> result.notify()
217216
218 # We need to commit the transaction to get the email in the queue.
219 >>> transaction.commit()
220
221In this case, there are no errors, so we should get just a single email217In this case, there are no errors, so we should get just a single email
222218
219 >>> test_emails = pop_notifications()
223 >>> len(test_emails)220 >>> len(test_emails)
224 1221 1
225222
226 >>> from_addrs, to_addrs, body = test_emails.pop()223 >>> print_emails(notifications=test_emails)
227224 From: ...
228 >>> print to_addrs225 To: carlos@canonical.com
229 ['carlos@canonical.com']
230 >>> print_mail_subject_and_body(body)
231 Subject: Launchpad translation download: Evolution trunk -226 Subject: Launchpad translation download: Evolution trunk -
232 evolution-2.2 template227 evolution-2.2 template
233 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,228 Hello Carlos Perell=C3=B3 Mar=C3=ADn,
234 >229 <BLANKLINE>
235 > The translation files you requested from Launchpad are ready for230 The translation files you requested from Launchpad are ready for
236 > download from the following location:231 download from the following location:
237 >232 <BLANKLINE>
238 > http://someplace.com/somefile.tar.gz233 http://someplace.com/somefile.tar.gz
239 >234 <BLANKLINE>
240 > Note: this link will expire in about 1 week. If you want to235 Note: this link will expire in about 1 week. If you want to
241 > download these translations again, you will have to request236 download these translations again, you will have to request
242 > them again at237 them again at
243 >238 <BLANKLINE>
244 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=239 http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
245 > port240 port
246 >241 <BLANKLINE>
247 > --242 -- =
248 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/243 Automatic message from Launchpad.net.
249 >244 <BLANKLINE>
245 ----------------------------------------
250246
251247
252248
@@ -287,34 +283,35 @@ Once the queue is processed, the queue i
287283
288And a confirmation email was sent to carlos, the importer.284And a confirmation email was sent to carlos, the importer.
289285
286 >>> test_emails = pop_notifications()
290 >>> len(test_emails)287 >>> len(test_emails)
291 1288 1
292289
293The confirmation email shows no errors at all.290The confirmation email shows no errors at all.
294291
295 >>> from_addrs, to_addrs, body = test_emails.pop()292 >>> print_emails(notifications=test_emails)
296 >>> print to_addrs293 From: ...
297 ['carlos@canonical.com']294 To: carlos@canonical.com
298 >>> print_mail_subject_and_body(body)
299 Subject: Launchpad translation download: Evolution trunk -295 Subject: Launchpad translation download: Evolution trunk -
300 evolution-2.2 template296 evolution-2.2 template
301 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,297 Hello Carlos Perell=C3=B3 Mar=C3=ADn,
302 >298 <BLANKLINE>
303 > The translation files you requested from Launchpad are ready for299 The translation files you requested from Launchpad are ready for
304 > download from the following location:300 download from the following location:
305 >301 <BLANKLINE>
306 > http://localhost:58000/.../po_evolution-2.2.pot302 http://localhost:58000/.../po_evolution-2.2.pot
307 >303 <BLANKLINE>
308 > Note: this link will expire in about 1 week. If you want to304 Note: this link will expire in about 1 week. If you want to
309 > download these translations again, you will have to request305 download these translations again, you will have to request
310 > them again at306 them again at
311 >307 <BLANKLINE>
312 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=308 http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
313 > port309 port
314 >310 <BLANKLINE>
315 > --311 -- =
316 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/312 Automatic message from Launchpad.net.
317 >313 <BLANKLINE>
314 ----------------------------------------
318315
319Let's have a closer look at what is being exported. Usually all messages are316Let's have a closer look at what is being exported. Usually all messages are
320exported but not all messages are equal. Some messages have been imported317exported but not all messages are equal. Some messages have been imported
@@ -372,12 +369,11 @@ after having been imported.
372369
373Two more email notifications were sent, we'd better get rid of them.370Two more email notifications were sent, we'd better get rid of them.
374371
375 >>> email = test_emails.pop()372 >>> discard = pop_notifications()
376 >>> email = test_emails.pop()
377373
378Finally, if we try to do an export with an empty queue, we don't do374Finally, if we try to do an export with an empty queue, we don't do
379anything:375anything:
380376
381 >>> process_queue(transaction, logging.getLogger())377 >>> process_queue(transaction, logging.getLogger())
382 >>> len(test_emails)378 >>> len(pop_notifications())
383 0379 0
384380
=== modified file 'lib/lp/translations/doc/poexport-request-productseries.txt'
--- lib/lp/translations/doc/poexport-request-productseries.txt 2009-08-14 17:46:32 +0000
+++ lib/lp/translations/doc/poexport-request-productseries.txt 2009-08-17 13:39:30 +0000
@@ -49,35 +49,31 @@ Now we request that the queue be process
49 Evolution trunk49 Evolution trunk
50 log> Stored file at http://.../launchpad-export.tar.gz50 log> Stored file at http://.../launchpad-export.tar.gz
5151
52We need to make sure the Librarian data has settled into place.
53
54 >>> import transaction
55 >>> transaction.commit()
56
57The user receives a confirmation email.52The user receives a confirmation email.
5853
59 >>> from lp.translations.tests.helpers import print_mail_subject_and_body54 >>> from lp.testing.mail_helpers import pop_notifications, print_emails
60 >>> from lp.services.mail.stub import test_emails55 >>> test_emails = pop_notifications()
61 >>> len(test_emails)56 >>> len(test_emails)
62 157 1
63 >>> from_addr, to_addrs, body = test_emails.pop()58 >>> print_emails(notifications=test_emails)
64 >>> print_mail_subject_and_body(body)59 From: ...
65 Subject: Launchpad translation download: Evolution trunk60 Subject: Launchpad translation download: Evolution trunk
66 > Hello ...,61 Hello ...,
67 >62 <BLANKLINE>
68 > The translation files you requested from Launchpad are ready for63 The translation files you requested from Launchpad are ready for
69 > download from the following location:64 download from the following location:
70 >65 <BLANKLINE>
71 > http://localhost:58000/.../launchpad-export.tar.gz66 http://localhost:58000/.../launchpad-export.tar.gz
72 >67 <BLANKLINE>
73 > Note: this link will expire in about 1 week. If you want to68 Note: this link will expire in about 1 week. If you want to
74 > download these translations again, you will have to request69 download these translations again, you will have to request
75 > them again at70 them again at
76 >71 <BLANKLINE>
77 > http://translations.launchpad.dev/evolution/trunk/+export72 http://translations.launchpad.dev/evolution/trunk/+export
78 >73 <BLANKLINE>
79 > --74 -- =
80 > Automatic message from Launchpad.net...75 Automatic message from Launchpad.net.
76 ----------------------------------------
8177
82The email contains a URL linking to where the exported file can be downloaded.78The email contains a URL linking to where the exported file can be downloaded.
8379
@@ -87,6 +83,7 @@ The email contains a URL linking to wher
87 ... urls = re.compile(r'^ *(http://.*)$', re.M).findall(text)83 ... urls = re.compile(r'^ *(http://.*)$', re.M).findall(text)
88 ... return urls[0]84 ... return urls[0]
8985
86 >>> body = test_emails[0].get_payload()
90 >>> url = extract_url(body)87 >>> url = extract_url(body)
9188
92Let's download it and make sure the contents look ok.89Let's download it and make sure the contents look ok.
9390
=== modified file 'lib/lp/translations/doc/poexport-request.txt'
--- lib/lp/translations/doc/poexport-request.txt 2009-08-14 17:46:32 +0000
+++ lib/lp/translations/doc/poexport-request.txt 2009-08-17 13:13:13 +0000
@@ -8,6 +8,7 @@ serve those requests properly.
8 ... IPOExportRequestSet)8 ... IPOExportRequestSet)
9 >>> from lp.translations.utilities.tests.helpers import (9 >>> from lp.translations.utilities.tests.helpers import (
10 ... is_valid_mofile)10 ... is_valid_mofile)
11 >>> from lp.testing.mail_helpers import pop_notifications, print_emails
1112
12This is a dummy logger class to capture the export's log messages.13This is a dummy logger class to capture the export's log messages.
1314
@@ -33,9 +34,10 @@ This is a dummy logger class to capture
3334
34Here's somebody to make a request.35Here's somebody to make a request.
3536
36 >>> from lp.registry.model.person import Person37 >>> person = factory.makePerson(
37 >>> person = Person.get(1)38 ... email='downloader@example.com',
3839 ... name='downloader',
40 ... displayname='Happy Downloader')
3941
40== Requesting PO files ==42== Requesting PO files ==
4143
@@ -57,42 +59,36 @@ Now we request that the queue be process
5759
58 >>> from lp.translations.scripts.po_export_queue import process_queue60 >>> from lp.translations.scripts.po_export_queue import process_queue
59 >>> process_queue(MockTransactionManager(), MockLogger())61 >>> process_queue(MockTransactionManager(), MockLogger())
60 log> Exporting objects for Mark Shuttleworth, related to template pmount62 log> Exporting objects for Happy Downloader, related to template pmount
61 in Ubuntu Hoary package "pmount"63 in Ubuntu Hoary package "pmount"
62 log> Stored file at http://.../launchpad-export.tar.gz64 log> Stored file at http://.../launchpad-export.tar.gz
6365
64We need to make sure the Librarian data has settled into place.
65
66 >>> import transaction
67 >>> transaction.commit()
68
69The user receives a confirmation email.66The user receives a confirmation email.
7067
71 >>> from lp.translations.tests.helpers import print_mail_subject_and_body68 >>> emails = pop_notifications()
72 >>> from lp.services.mail.stub import test_emails69 >>> len(emails)
73 >>> len(test_emails)
74 170 1
75 >>> from_addr, to_addrs, body = test_emails.pop()71 >>> print_emails(notifications=emails)
76 >>> to_addrs72 From: ...
77 ['mark@hbd.com']73 To: downloader@example.com
78 >>> print_mail_subject_and_body(body)
79 Subject: Launchpad translation download: Ubuntu Hoary pmount -74 Subject: Launchpad translation download: Ubuntu Hoary pmount -
80 pmount template75 pmount template
81 > Hello Mark Shuttleworth,76 Hello Happy Downloader,
82 >77 <BLANKLINE>
83 > The translation files you requested from Launchpad are ready for78 The translation files you requested from Launchpad are ready for
84 > download from the following location:79 download from the following location:
85 >80 <BLANKLINE>
86 > http://localhost:58000/.../launchpad-export.tar.gz81 http://localhost:58000/.../launchpad-export.tar.gz
87 >82 <BLANKLINE>
88 > Note: this link will expire in about 1 week. If you want to83 Note: this link will expire in about 1 week. If you want to
89 > download these translations again, you will have to request84 download these translations again, you will have to request
90 > them again at85 them again at
91 >86 <BLANKLINE>
92 > http://translations.launchpad.../hoary/+source/pmount/+pots/p...87 http://translations.launchpad.../hoary/+source/pmount/+pots/p...
93 >88 <BLANKLINE>
94 > --89 -- =
95 > Automatic message from Launchpad.net...90 Automatic message from Launchpad.net.
91 ----------------------------------------
9692
97The email contains a URL linking to where the exported file can be downloaded.93The email contains a URL linking to where the exported file can be downloaded.
9894
@@ -102,6 +98,7 @@ The email contains a URL linking to wher
102 ... urls = re.compile(r'^ *(http://.*)$', re.M).findall(text)98 ... urls = re.compile(r'^ *(http://.*)$', re.M).findall(text)
103 ... return urls[0]99 ... return urls[0]
104100
101 >>> body = emails.pop().get_payload()
105 >>> url = extract_url(body)102 >>> url = extract_url(body)
106103
107Let's download it and make sure the contents look ok.104Let's download it and make sure the contents look ok.
@@ -192,41 +189,40 @@ Let's try requesting an export in MO for
192 ... TranslationFileFormat)189 ... TranslationFileFormat)
193 >>> request_set.addRequest(person, None, [cs], TranslationFileFormat.MO)190 >>> request_set.addRequest(person, None, [cs], TranslationFileFormat.MO)
194 >>> process_queue(MockTransactionManager(), MockLogger())191 >>> process_queue(MockTransactionManager(), MockLogger())
195 log> Exporting objects for Mark Shuttleworth, related to template pmount192 log> Exporting objects for Happy Downloader, related to template pmount
196 in Ubuntu Hoary package "pmount"193 in Ubuntu Hoary package "pmount"
197 log> Stored file at http://.../cs_LC_MESSAGES_pmount.mo194 log> Stored file at http://.../cs_LC_MESSAGES_pmount.mo
198195
199 >>> transaction.commit()196 >>> emails = pop_notifications()
200197 >>> len(emails)
201 >>> len(test_emails)
202 1198 1
203 >>> from_addr, to_addrs, body = test_emails.pop()199 >>> print_emails(notifications=emails)
204 >>> to_addrs200 From: ...
205 ['mark@hbd.com']201 To: downloader@example.com
206 >>> print_mail_subject_and_body(body)
207 Subject: Launchpad translation download: Ubuntu Hoary pmount - Czech202 Subject: Launchpad translation download: Ubuntu Hoary pmount - Czech
208 translation of pmount203 translation of pmount
209 > Hello Mark Shuttleworth,204 Hello Happy Downloader,
210 >205 <BLANKLINE>
211 > The translation files you requested from Launchpad are ready for206 The translation files you requested from Launchpad are ready for
212 > download from the following location:207 download from the following location:
213 >208 <BLANKLINE>
214 > http://localhost:58000/.../cs_LC_MESSAGES_pmount.mo209 http://localhost:58000/.../cs_LC_MESSAGES_pmount.mo
215 >210 <BLANKLINE>
216 > Note: this link will expire in about 1 week. If you want to211 Note: this link will expire in about 1 week. If you want to
217 > download these translations again, you will have to request212 download these translations again, you will have to request
218 > them again at213 them again at
219 >214 <BLANKLINE>
220 > http://translations.launchpad.../pmount/+pots/pmoun=215 http://translations.launchpad.../pmount/+pots/pmoun=
221 > t/cs/+export216 t/cs/+export
222 >217 <BLANKLINE>
223 > --218 -- =
224 > Automatic message from Launchpad.net...219 Automatic message from Launchpad.net.
225220 ----------------------------------------
226 >>> url = extract_url(body)
227221
228Check whether we generated a good .mo file.222Check whether we generated a good .mo file.
229223
224 >>> body = emails.pop().get_payload()
225 >>> url = extract_url(body)
230 >>> is_valid_mofile(urllib2.urlopen(url).read())226 >>> is_valid_mofile(urllib2.urlopen(url).read())
231 True227 True
232228
@@ -269,13 +265,13 @@ The script is run.
269 >>> (output, empty) = process.communicate()265 >>> (output, empty) = process.communicate()
270 >>> print output266 >>> print output
271 INFO creating lockfile267 INFO creating lockfile
272 DEBUG Exporting objects for Mark Shuttleworth, related to template268 DEBUG Exporting objects for Happy Downloader, related to template
273 evolution-2.2 in Evolution trunk269 evolution-2.2 in Evolution trunk
274 DEBUG Exporting objects for Mark Shuttleworth, related to template270 DEBUG Exporting objects for Happy Downloader, related to template
275 pmount in Ubuntu Hoary package "pmount"271 pmount in Ubuntu Hoary package "pmount"
276 DEBUG Exporting objects for Mark Shuttleworth, related to template272 DEBUG Exporting objects for Happy Downloader, related to template
277 netapplet in NetApplet trunk273 netapplet in NetApplet trunk
278 DEBUG Exporting objects for Mark Shuttleworth, related to template274 DEBUG Exporting objects for Happy Downloader, related to template
279 alsa-utils in alsa-utils trunk275 alsa-utils in alsa-utils trunk
280 INFO Stored file at http://.../launchpad-export.tar.gz276 INFO Stored file at http://.../launchpad-export.tar.gz
281 DEBUG Removing lock file: /var/lock/launchpad-rosetta-export-queue.lock277 DEBUG Removing lock file: /var/lock/launchpad-rosetta-export-queue.lock
282278
=== modified file 'lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt'
--- lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt 2009-08-14 16:33:57 +0000
+++ lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt 2009-08-17 13:16:32 +0000
@@ -1,4 +1,4 @@
1Hello admins,1Hello Launchpad administrators,
22
3Launchpad encountered problems exporting translation files3Launchpad encountered problems exporting translation files
4requested by %(person)s (%(person_id)s) at4requested by %(person)s (%(person_id)s) at
@@ -12,5 +12,5 @@ before this export can proceed. Here is
1212
13%(failed_requests)s13%(failed_requests)s
1414
15--15--
16Automatic message from Launchpad.net — https://launchpad.net/16Automatic message from Launchpad.net.
1717
=== modified file 'lib/lp/translations/emailtemplates/poexport-failure-unicodedecodeerror.txt'
--- lib/lp/translations/emailtemplates/poexport-failure-unicodedecodeerror.txt 2009-08-14 16:33:57 +0000
+++ lib/lp/translations/emailtemplates/poexport-failure-unicodedecodeerror.txt 2009-08-17 13:16:39 +0000
@@ -1,4 +1,4 @@
1Hello admins,1Hello Launchpad administrators,
22
3A UnicodeDecodeError occurred while trying to notify you of a3A UnicodeDecodeError occurred while trying to notify you of a
4failure during a translation export requested by %(person)s4failure during a translation export requested by %(person)s
@@ -8,5 +8,5 @@ failure during a translation export requ
88
9%(failed_requests)s9%(failed_requests)s
1010
11--11--
12Automatic message from Launchpad.net — https://launchpad.net/12Automatic message from Launchpad.net.
1313
=== modified file 'lib/lp/translations/emailtemplates/poexport-failure.txt'
--- lib/lp/translations/emailtemplates/poexport-failure.txt 2009-08-14 16:33:57 +0000
+++ lib/lp/translations/emailtemplates/poexport-failure.txt 2009-08-17 12:51:36 +0000
@@ -8,5 +8,5 @@ If you want to retry your request, you c
88
9 %(request_url)s.9 %(request_url)s.
1010
11--11--
12Automatic message from Launchpad.net — https://launchpad.net/12Automatic message from Launchpad.net.
1313
=== modified file 'lib/lp/translations/emailtemplates/poexport-success.txt'
--- lib/lp/translations/emailtemplates/poexport-success.txt 2009-08-14 16:33:57 +0000
+++ lib/lp/translations/emailtemplates/poexport-success.txt 2009-08-17 12:51:44 +0000
@@ -11,5 +11,5 @@ them again at
1111
12 %(request_url)s12 %(request_url)s
1313
14--14--
15Automatic message from Launchpad.net — https://launchpad.net/15Automatic message from Launchpad.net.
1616
=== modified file 'lib/lp/translations/scripts/po_export_queue.py'
--- lib/lp/translations/scripts/po_export_queue.py 2009-08-14 18:02:09 +0000
+++ lib/lp/translations/scripts/po_export_queue.py 2009-08-17 13:36:26 +0000
@@ -38,7 +38,7 @@ class ExportResult:
3838
39 This class has three main attributes:39 This class has three main attributes:
4040
41 - person: A person requesting this export.41 - person: The person requesting this export.
42 - url: The Librarian URL for any successfully exported files.42 - url: The Librarian URL for any successfully exported files.
43 - failure: Failure gotten while exporting.43 - failure: Failure gotten while exporting.
44 """44 """
@@ -138,7 +138,9 @@ class ExportResult:
138 potemplate.sourcepackagename)138 potemplate.sourcepackagename)
139 sourcepackages.add(sourcepackage)139 sourcepackages.add(sourcepackage)
140 else:140 else:
141 pass141 raise AssertionError(
142 "Requesting a translation export which belongs to "
143 "neither a ProductSeries nor a SourcePackage.")
142144
143 if len(pofiles) == 1 and len(direct_potemplates) == 0:145 if len(pofiles) == 1 and len(direct_potemplates) == 0:
144 # One POFile was requested.146 # One POFile was requested.
@@ -162,18 +164,21 @@ class ExportResult:
162164
163 if IPOTemplate.providedBy(export_requested_at):165 if IPOTemplate.providedBy(export_requested_at):
164 if len(sourcepackages) > 0:166 if len(sourcepackages) > 0:
165 sp = sourcepackages.pop()167 container = sourcepackages.pop()
166 if sp.getCurrentTranslationTemplates().count() == 1:
167 export_requested_at = sp
168 elif len(productseries) > 0:168 elif len(productseries) > 0:
169 ps = productseries.pop()169 container = productseries.pop()
170 if ps.getCurrentTranslationTemplates().count() == 1:170 else:
171 export_requested_at = ps171 raise AssertionError(
172 "Requesting a translation export which belongs to "
173 "neither a ProductSeries nor a SourcePackage.")
174 if container.getCurrentTranslationTemplates().count() == 1:
175 export_requested_at = container
172176
173 return export_requested_at177 return export_requested_at
174178
175179
176 def _getRequestedExportsNames(self):180 def _getRequestedExportsNames(self):
181 """Return a list of display names for requested exports."""
177 requested_names = []182 requested_names = []
178 for translation_object in self.requested_exports:183 for translation_object in self.requested_exports:
179 if IPOTemplate.providedBy(translation_object):184 if IPOTemplate.providedBy(translation_object):
@@ -187,7 +192,7 @@ class ExportResult:
187 def _getFailureEmailBody(self):192 def _getFailureEmailBody(self):
188 """Send an email notification about the export failing."""193 """Send an email notification about the export failing."""
189 template = helpers.get_email_template(194 template = helpers.get_email_template(
190 'poexport-failure.txt', 'translations').decode('utf-8')195 'poexport-failure.txt', 'translations')
191 return template % {196 return template % {
192 'person' : self.person.displayname,197 'person' : self.person.displayname,
193 'request_url' : self.request_url,198 'request_url' : self.request_url,
@@ -199,7 +204,7 @@ class ExportResult:
199 if len(failed_requests) > 0:204 if len(failed_requests) > 0:
200 failed_requests_text = 'Failed export request included:\n'205 failed_requests_text = 'Failed export request included:\n'
201 failed_requests_text += '\n'.join(206 failed_requests_text += '\n'.join(
202 [' * ' + request for request in failed_requests])207 ' * ' + request for request in failed_requests)
203 else:208 else:
204 failed_requests_text = 'There were no export requests.'209 failed_requests_text = 'There were no export requests.'
205 return failed_requests_text210 return failed_requests_text
@@ -208,7 +213,7 @@ class ExportResult:
208 """Send an email notification about failed export to admins."""213 """Send an email notification about failed export to admins."""
209 template = helpers.get_email_template(214 template = helpers.get_email_template(
210 'poexport-failure-admin-notification.txt',215 'poexport-failure-admin-notification.txt',
211 'translations').decode('utf-8')216 'translations')
212 failed_requests = self._getFailedRequestsDescription()217 failed_requests = self._getFailedRequestsDescription()
213 return template % {218 return template % {
214 'person' : self.person.displayname,219 'person' : self.person.displayname,
@@ -222,7 +227,7 @@ class ExportResult:
222 """Send an email notification to admins about UnicodeDecodeError."""227 """Send an email notification to admins about UnicodeDecodeError."""
223 template = helpers.get_email_template(228 template = helpers.get_email_template(
224 'poexport-failure-unicodedecodeerror.txt',229 'poexport-failure-unicodedecodeerror.txt',
225 'translations').decode('utf-8')230 'translations')
226 failed_requests = self._getFailedRequestsDescription()231 failed_requests = self._getFailedRequestsDescription()
227 return template % {232 return template % {
228 'person' : self.person.displayname,233 'person' : self.person.displayname,
@@ -234,7 +239,7 @@ class ExportResult:
234 def _getSuccessEmailBody(self):239 def _getSuccessEmailBody(self):
235 """Send an email notification about the export working."""240 """Send an email notification about the export working."""
236 template = helpers.get_email_template(241 template = helpers.get_email_template(
237 'poexport-success.txt', 'translations').decode('utf-8')242 'poexport-success.txt', 'translations')
238 return template % {243 return template % {
239 'person' : self.person.displayname,244 'person' : self.person.displayname,
240 'download_url' : self.url,245 'download_url' : self.url,
241246
=== removed file 'lib/lp/translations/tests/helpers.py'
--- lib/lp/translations/tests/helpers.py 2009-08-14 17:46:32 +0000
+++ lib/lp/translations/tests/helpers.py 1970-01-01 00:00:00 +0000
@@ -1,19 +0,0 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Helper module reused in different tests."""
5
6__metaclass__ = type
7
8__all__ = [
9 'print_mail_subject_and_body'
10 ]
11
12import email
13
14def print_mail_subject_and_body(contents):
15 msg = email.message_from_string(contents)
16 body = msg.get_payload()
17 print 'Subject: %s' % (msg['subject'])
18 for line in body.split('\n'):
19 print ">", line
200
Revision history for this message
Barry Warsaw (barry) wrote :
Download full text (8.5 KiB)

On Aug 17, 2009, at 01:48 PM, Данило Шеган wrote:

>I've applied most of your recommendations. The ones I didn't were about
>'positive tests' (i.e. is 'is not None' really a negative test?) and I
>haven't provided unit tests. Still, I've modified doctests to use
>mail_helpers, so that resulted in a nice big incremental diff.

Indeed, that looks fantastic. Thanks!

>У пет, 14. 08 2009. у 16:05 -0400, Barry Warsaw пише:
>> > * doc/poexport-request.txt,
>> > doc/poexport-request-productseries.txt,
>> > doc/poexport-queue.txt:
>> > There are a lot of changes to tests that test how emails are generated:
>> > basically, half of the branch is there (440 lines). The changes there
>> > include introducing a shared helper print_mail_subject_and_body method,
>> > print subject along with body of the email, and changes to test new
>> > email contents and subjects. Some minor changes to how ExportResult is
>> > constructed were necessary as well to conform with the new API (i.e. pass
>> > person, requested exports and logger).
>>
>> Do you know about lp.testing.mail_helpers.print_emails()? I generally use
>> that plus ellipses to cut down on the output. Is there a reason why you'd
>> want print_mail_subject_and_body instead?
>
>Yes, I've looked through existing helpers. I've already struggled to
>keep the diff size reasonable, and this would have made it only worse.
>Though, I am happy to migrate existing tests to the use them, and I did
>so. Note that only this change means 640 lines of diff in the
>incremental diff ;)

That's the way to do it! :) Really, it looks great now.

>> === added file 'lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt'
>> --- lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt 1970-01-01 00:00:00 +0000
>> +++ lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt 2009-08-14 16:33:57 +0000
>> > @@ -0,0 +1,16 @@
>> > +Hello admins,
>>
>> Maybe "Hello Launchpad administrators," ?
>
>Sure, fixed. Note that these emails only ended up on our internal
>error-reports list, but no reason not to make them even nicer :)

If we can't be nice to ourselves, we can't be nice to our users :)

>> > +Automatic message from Launchpad.net — https://launchpad.net/
>>
>> Wow, this is like a failure failure. :)
>
>My mail client linkifies the second, but not the first. I still prefer
>the former in an email message, so I made it
>
> "Automatic message from Launchpad.net."
>
>in all the email templates.

Y'know, I didn't really notice that that was an em-dash. I thought it was a
plain old '-' but I guess that was just my editor's font playing tricks on
me. In any case, I think keeping it pure ASCII is a good thing (as you saw by
removing all those utf-8 conversions).

But I wasn't actually complaining about it, I was just observing that the
message appeared to be sent when a failure message failed. Still, I'm glad my
Friday-addled brain was able to reach through the fog and offer a not-horrible
suggestion. ;)

>> Is it really more convenient to access the logger through the instance rather
>> than use the module global? I can almost promise you...

Read more...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'lib/lp/translations/doc/poexport-queue.txt'
--- lib/lp/translations/doc/poexport-queue.txt 2009-07-02 17:16:50 +0000
+++ lib/lp/translations/doc/poexport-queue.txt 2009-08-14 17:46:32 +0000
@@ -9,17 +9,14 @@
9ExportResult class is used to control the list of exported files that succeed9ExportResult class is used to control the list of exported files that succeed
10and the ones that failed with the error associated.10and the ones that failed with the error associated.
1111
12 >>> import transaction12 >>> import transaction
13 >>> from zope.component import getUtility13 >>> from zope.component import getUtility
14 >>> from canonical.launchpad.interfaces import IPersonSet14 >>> from canonical.launchpad.interfaces import IPersonSet
15 >>> from lp.services.mail.stub import test_emails15 >>> from lp.services.mail.stub import test_emails
16 >>> from lp.translations.scripts.po_export_queue import ExportResult16 >>> from lp.translations.scripts.po_export_queue import ExportResult
1717 >>> from lp.translations.tests.helpers import print_mail_subject_and_body
18 # Function to print mail content without headers.18 >>> import logging
19 >>> def print_mail_body(body):19 >>> logger = logging.getLogger()
20 ... lines = body[body.index('\n\n')+2:].split('\n')
21 ... for line in lines:
22 ... print ">", line
2320
24When there is an error, the system will notify it.21When there is an error, the system will notify it.
2522
@@ -29,10 +26,10 @@
29 >>> from lp.translations.model.potemplate import POTemplate26 >>> from lp.translations.model.potemplate import POTemplate
30 >>> potemplate = POTemplate.get(1)27 >>> potemplate = POTemplate.get(1)
31 >>> pofile = potemplate.getPOFileByLang('es')28 >>> pofile = potemplate.getPOFileByLang('es')
29 >>> personset = getUtility(IPersonSet)
30 >>> carlos = personset.getByName('carlos')
3231
33 >>> result = ExportResult('testing export')32 >>> result = ExportResult(carlos, [potemplate, pofile], logger)
34 >>> result.object_names.append(potemplate.displayname)
35 >>> result.object_names.append(pofile.title)
3633
37Record the error.34Record the error.
3835
@@ -50,9 +47,7 @@
50given that we set a URL, the notification will detect the programming error.47given that we set a URL, the notification will detect the programming error.
51In this example, 'carlos' will be the one that did the request.48In this example, 'carlos' will be the one that did the request.
5249
53 >>> personset = getUtility(IPersonSet)50 >>> result.notify()
54 >>> carlos = personset.getByName('carlos')
55 >>> result.notify(carlos)
56 Traceback (most recent call last):51 Traceback (most recent call last):
57 ...52 ...
58 AssertionError: We cannot have a URL for the export and a failure.53 AssertionError: We cannot have a URL for the export and a failure.
@@ -60,15 +55,15 @@
60In this case, it should be None, so the notify works.55In this case, it should be None, so the notify works.
6156
62 >>> result.url = None57 >>> result.url = None
63 >>> result.notify(carlos)58 >>> result.notify()
6459
65 # We need to commit the transaction to get the emails in the queue.60 # We need to commit the transaction to get the emails in the queue.
66 >>> transaction.commit()61 >>> transaction.commit()
6762
68As usual, when there is an error, two emails should be sent:63As usual, when there is an error, two emails should be sent:
6964
70 >>> len(test_emails) == 265 >>> len(test_emails)
71 True66 2
7267
73 >>> while len(test_emails) > 0:68 >>> while len(test_emails) > 0:
74 ... from_addrs, to_addrs, body = test_emails.pop()69 ... from_addrs, to_addrs, body = test_emails.pop()
@@ -83,38 +78,55 @@
8378
84 >>> print carlos_addrs79 >>> print carlos_addrs
85 ['carlos@canonical.com']80 ['carlos@canonical.com']
86 >>> print_mail_body(carlos_body)81 >>> print_mail_subject_and_body(carlos_body)
87 > 82 Subject: Launchpad translation download: Evolution trunk -
83 evolution-2.2 template
88 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,84 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,
89 > 85 >
90 > Launchpad encountered problems exporting the files you requested.86 > Launchpad encountered problems exporting the files you requested.
91 > The Launchpad Translations team has been notified of this problem.87 > The Launchpad Translations team has been notified of this problem.
92 > Please reply to this email for further assistance.88 > Please reply to this email for further assistance.
93 >89 >
90 > If you want to retry your request, you can do so at
91 >
92 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
93 > port.
94 >
95 > --
96 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/
97 >
9498
95And the other to the admins. This one lists the files that were being99And the other to the admins. This one lists the files that were being
96exported as context to help tracking down any bugs.100exported as context to help tracking down any bugs.
97101
98 >>> print admins_addrs102 >>> print admins_addrs
99 ['launchpad-error-reports@lists.canonical.com']103 ['launchpad-error-reports@lists.canonical.com']
100 >>> print_mail_body(admins_body)104 >>> print_mail_subject_and_body(admins_body)
101 > 105 Subject: Launchpad translation download errors: Evolution trunk -
106 evolution-2.2 template
102 > Hello admins,107 > Hello admins,
103 > 108 >
104 > Launchpad encountered problems exporting translation files109 > Launchpad encountered problems exporting translation files
105 > requested by Carlos Perell=C3=B3 Mar=C3=ADn.110 > requested by Carlos Perell=C3=B3 Mar=C3=ADn (carlos) at
111 >
112 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
113 > port
106 >114 >
107 > This means we have a bug in Launchpad that needs to be fixed115 > This means we have a bug in Launchpad that needs to be fixed
108 > before this export can proceed. Here is the list of failed116 > before this export can proceed. Here is the error we got:
109 > files and the error we got:117 >
110 >
111 > Traceback (most recent call last):118 > Traceback (most recent call last):
112 ...119 ...
113 > AssertionError: It's just an error for testing purposes120 > AssertionError: It's just an error for testing purposes
114 >121 >
115 > The failed request involved these objects:122 >
116 > evolution-2.2 in Evolution trunk123 > Failed export request included:
117 > Spanish (es) translation of evolution-2.2 in Evolution trunk124 > * evolution-2.2 in Evolution trunk
125 > * Spanish (es) translation of evolution-2.2 in Evolution trunk
126 >
127 > --
128 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/
129 >
118130
119As a special case, some error messages are poisoned with non-ASCII characters131As a special case, some error messages are poisoned with non-ASCII characters
120and can't be reported without triggering an error themselves. Those are132and can't be reported without triggering an error themselves. Those are
@@ -124,10 +136,10 @@
124 ... raise AssertionError, "Really nasty \xc3 non-ASCII error!"136 ... raise AssertionError, "Really nasty \xc3 non-ASCII error!"
125 ... except AssertionError:137 ... except AssertionError:
126 ... result.addFailure()138 ... result.addFailure()
127 >>> result.notify(carlos)139 >>> result.notify()
128 >>> transaction.commit()140 >>> transaction.commit()
129 >>> len(test_emails) == 2141 >>> len(test_emails)
130 True142 2
131143
132 >>> carlos_body = None144 >>> carlos_body = None
133 >>> admins_body = None145 >>> admins_body = None
@@ -140,39 +152,57 @@
140152
141The user's notification looks no different from that for an ordinary error.153The user's notification looks no different from that for an ordinary error.
142154
143 >>> print_mail_body(carlos_body)155 >>> print_mail_subject_and_body(carlos_body)
144 > 156 Subject: Launchpad translation download: Evolution trunk -
157 evolution-2.2 template
145 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,158 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,
146 > 159 >
147 > Launchpad encountered problems exporting the files you requested.160 > Launchpad encountered problems exporting the files you requested.
148 > The Launchpad Translations team has been notified of this problem.161 > The Launchpad Translations team has been notified of this problem.
149 > Please reply to this email for further assistance.162 > Please reply to this email for further assistance.
150 >163 >
164 > If you want to retry your request, you can do so at
165 >
166 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
167 > port.
168 >
169 > --
170 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/
171 >
151172
152The one for the administrators, however, does not include the unprintable173The one for the administrators, however, does not include the unprintable
153exception text.174exception text.
154175
155 >>> print_mail_body(admins_body)176 >>> print_mail_subject_and_body(admins_body)
156 > 177 Subject: Launchpad translation download errors: Evolution trunk -
178 evolution-2.2 template
157 > Hello admins,179 > Hello admins,
158 > 180 >
159 > A UnicodeDecodeError occurred while trying to notify you of a181 > A UnicodeDecodeError occurred while trying to notify you of a
160 > failure during a translation export requested by Carlos ...182 > failure during a translation export requested by Carlos ...
161 >183 > (carlos) at
162 > The failed request involved these objects:184 >
163 > evolution-2.2 in Evolution trunk185 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
164 > Spanish (es) translation of evolution-2.2 in Evolution trunk186 > port
187 >
188 > Failed export request included:
189 > * evolution-2.2 in Evolution trunk
190 > * Spanish (es) translation of evolution-2.2 in Evolution trunk
191 >
192 > --
193 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/
194 >
165195
166196
167Finally, there is the case when there are no errors at all. This is the usual197Finally, there is the case when there are no errors at all. This is the usual
168case.198case.
169199
170 >>> result = ExportResult('testing export')200 >>> result = ExportResult(carlos, [potemplate, pofile], logger)
171201
172As noted before, result.url should be set to the URL where the user can202As noted before, result.url should be set to the URL where the user can
173download the requested files. If we don't set it, the export will fail:203download the requested files. If we don't set it, the export will fail:
174204
175 >>> result.notify(carlos)205 >>> result.notify()
176 Traceback (most recent call last):206 Traceback (most recent call last):
177 ...207 ...
178 AssertionError: On success, an exported URL is expected.208 AssertionError: On success, an exported URL is expected.
@@ -183,28 +213,41 @@
183213
184And notify the user.214And notify the user.
185215
186 >>> result.notify(carlos)216 >>> result.notify()
187217
188 # We need to commit the transaction to get the email in the queue.218 # We need to commit the transaction to get the email in the queue.
189 >>> transaction.commit()219 >>> transaction.commit()
190220
191In this case, there are no errors, so we should get just a single email221In this case, there are no errors, so we should get just a single email
192222
193 >>> len(test_emails) == 1223 >>> len(test_emails)
194 True224 1
195225
196 >>> from_addrs, to_addrs, body = test_emails.pop()226 >>> from_addrs, to_addrs, body = test_emails.pop()
197227
198 >>> print to_addrs228 >>> print to_addrs
199 ['carlos@canonical.com']229 ['carlos@canonical.com']
200 >>> print_mail_body(body)230 >>> print_mail_subject_and_body(body)
201 > 231 Subject: Launchpad translation download: Evolution trunk -
232 evolution-2.2 template
202 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,233 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,
203 > 234 >
204 > The translation files you requested from Launchpad are ready for235 > The translation files you requested from Launchpad are ready for
205 > download from the following location:236 > download from the following location:
206 > 237 >
207 > http://someplace.com/somefile.tar.gz238 > http://someplace.com/somefile.tar.gz
239 >
240 > Note: this link will expire in about 1 week. If you want to
241 > download these translations again, you will have to request
242 > them again at
243 >
244 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
245 > port
246 >
247 > --
248 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/
249 >
250
208251
209252
210== process_queue() ==253== process_queue() ==
@@ -236,7 +279,6 @@
236279
237Once the queue is processed, the queue is empty again.280Once the queue is processed, the queue is empty again.
238281
239 >>> import logging
240 >>> process_queue(transaction, logging.getLogger())282 >>> process_queue(transaction, logging.getLogger())
241 INFO:...Stored file at http://.../po_evolution-2.2.pot283 INFO:...Stored file at http://.../po_evolution-2.2.pot
242284
@@ -253,14 +295,26 @@
253 >>> from_addrs, to_addrs, body = test_emails.pop()295 >>> from_addrs, to_addrs, body = test_emails.pop()
254 >>> print to_addrs296 >>> print to_addrs
255 ['carlos@canonical.com']297 ['carlos@canonical.com']
256 >>> print_mail_body(body)298 >>> print_mail_subject_and_body(body)
257 > 299 Subject: Launchpad translation download: Evolution trunk -
300 evolution-2.2 template
258 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,301 > Hello Carlos Perell=C3=B3 Mar=C3=ADn,
259 > 302 >
260 > The translation files you requested from Launchpad are ready for303 > The translation files you requested from Launchpad are ready for
261 > download from the following location:304 > download from the following location:
262 > 305 >
263 > http://localhost:58000/.../po_evolution-2.2.pot306 > http://localhost:58000/.../po_evolution-2.2.pot
307 >
308 > Note: this link will expire in about 1 week. If you want to
309 > download these translations again, you will have to request
310 > them again at
311 >
312 > http://translations.launchpad.../trunk/+pots/evolution-2.2/+ex=
313 > port
314 >
315 > --
316 > Automatic message from Launchpad.net =E2=80=94 https://launchpad.net/
317 >
264318
265Let's have a closer look at what is being exported. Usually all messages are319Let's have a closer look at what is being exported. Usually all messages are
266exported but not all messages are equal. Some messages have been imported320exported but not all messages are equal. Some messages have been imported
267321
=== modified file 'lib/lp/translations/doc/poexport-request-productseries.txt'
--- lib/lp/translations/doc/poexport-request-productseries.txt 2009-07-23 17:49:31 +0000
+++ lib/lp/translations/doc/poexport-request-productseries.txt 2009-08-14 17:46:32 +0000
@@ -56,23 +56,28 @@
5656
57The user receives a confirmation email.57The user receives a confirmation email.
5858
59 >>> def print_mail_body(body):59 >>> from lp.translations.tests.helpers import print_mail_subject_and_body
60 ... lines = body[body.index('\n\n')+2:].split('\n')
61 ... for line in lines:
62 ... print ">", line
63
64 >>> from lp.services.mail.stub import test_emails60 >>> from lp.services.mail.stub import test_emails
65 >>> len(test_emails)61 >>> len(test_emails)
66 162 1
67 >>> from_addr, to_addrs, body = test_emails.pop()63 >>> from_addr, to_addrs, body = test_emails.pop()
68 >>> print_mail_body(body)64 >>> print_mail_subject_and_body(body)
69 >65 Subject: Launchpad translation download: Evolution trunk
70 > Hello ...,66 > Hello ...,
71 >67 >
72 > The translation files you requested from Launchpad are ready for68 > The translation files you requested from Launchpad are ready for
73 > download from the following location:69 > download from the following location:
74 >70 >
75 > http://localhost:58000/.../launchpad-export.tar.gz71 > http://localhost:58000/.../launchpad-export.tar.gz
72 >
73 > Note: this link will expire in about 1 week. If you want to
74 > download these translations again, you will have to request
75 > them again at
76 >
77 > http://translations.launchpad.dev/evolution/trunk/+export
78 >
79 > --
80 > Automatic message from Launchpad.net...
7681
77The email contains a URL linking to where the exported file can be downloaded.82The email contains a URL linking to where the exported file can be downloaded.
7883
7984
=== modified file 'lib/lp/translations/doc/poexport-request.txt'
--- lib/lp/translations/doc/poexport-request.txt 2009-07-23 17:49:31 +0000
+++ lib/lp/translations/doc/poexport-request.txt 2009-08-14 17:46:32 +0000
@@ -68,25 +68,31 @@
6868
69The user receives a confirmation email.69The user receives a confirmation email.
7070
71 >>> def print_mail_body(body):71 >>> from lp.translations.tests.helpers import print_mail_subject_and_body
72 ... lines = body[body.index('\n\n')+2:].split('\n')
73 ... for line in lines:
74 ... print ">", line
75
76 >>> from lp.services.mail.stub import test_emails72 >>> from lp.services.mail.stub import test_emails
77 >>> len(test_emails)73 >>> len(test_emails)
78 174 1
79 >>> from_addr, to_addrs, body = test_emails.pop()75 >>> from_addr, to_addrs, body = test_emails.pop()
80 >>> to_addrs76 >>> to_addrs
81 ['mark@hbd.com']77 ['mark@hbd.com']
82 >>> print_mail_body(body)78 >>> print_mail_subject_and_body(body)
83 >79 Subject: Launchpad translation download: Ubuntu Hoary pmount -
80 pmount template
84 > Hello Mark Shuttleworth,81 > Hello Mark Shuttleworth,
85 >82 >
86 > The translation files you requested from Launchpad are ready for83 > The translation files you requested from Launchpad are ready for
87 > download from the following location:84 > download from the following location:
88 >85 >
89 > http://localhost:58000/.../launchpad-export.tar.gz86 > http://localhost:58000/.../launchpad-export.tar.gz
87 >
88 > Note: this link will expire in about 1 week. If you want to
89 > download these translations again, you will have to request
90 > them again at
91 >
92 > http://translations.launchpad.../hoary/+source/pmount/+pots/p...
93 >
94 > --
95 > Automatic message from Launchpad.net...
9096
91The email contains a URL linking to where the exported file can be downloaded.97The email contains a URL linking to where the exported file can be downloaded.
9298
@@ -197,14 +203,26 @@
197 >>> from_addr, to_addrs, body = test_emails.pop()203 >>> from_addr, to_addrs, body = test_emails.pop()
198 >>> to_addrs204 >>> to_addrs
199 ['mark@hbd.com']205 ['mark@hbd.com']
200 >>> print_mail_body(body)206 >>> print_mail_subject_and_body(body)
201 > 207 Subject: Launchpad translation download: Ubuntu Hoary pmount - Czech
208 translation of pmount
202 > Hello Mark Shuttleworth,209 > Hello Mark Shuttleworth,
203 > 210 >
204 > The translation files you requested from Launchpad are ready for211 > The translation files you requested from Launchpad are ready for
205 > download from the following location:212 > download from the following location:
206 > 213 >
207 > http://localhost:58000/.../cs_LC_MESSAGES_pmount.mo214 > http://localhost:58000/.../cs_LC_MESSAGES_pmount.mo
215 >
216 > Note: this link will expire in about 1 week. If you want to
217 > download these translations again, you will have to request
218 > them again at
219 >
220 > http://translations.launchpad.../pmount/+pots/pmoun=
221 > t/cs/+export
222 >
223 > --
224 > Automatic message from Launchpad.net...
225
208 >>> url = extract_url(body)226 >>> url = extract_url(body)
209227
210Check whether we generated a good .mo file.228Check whether we generated a good .mo file.
211229
=== added file 'lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt'
--- lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt 1970-01-01 00:00:00 +0000
+++ lib/lp/translations/emailtemplates/poexport-failure-admin-notification.txt 2009-08-14 16:33:57 +0000
@@ -0,0 +1,16 @@
1Hello admins,
2
3Launchpad encountered problems exporting translation files
4requested by %(person)s (%(person_id)s) at
5
6 %(request_url)s
7
8This means we have a bug in Launchpad that needs to be fixed
9before this export can proceed. Here is the error we got:
10
11%(failure_message)s
12
13%(failed_requests)s
14
15--
16Automatic message from Launchpad.net — https://launchpad.net/
017
=== added file 'lib/lp/translations/emailtemplates/poexport-failure-unicodedecodeerror.txt'
--- lib/lp/translations/emailtemplates/poexport-failure-unicodedecodeerror.txt 1970-01-01 00:00:00 +0000
+++ lib/lp/translations/emailtemplates/poexport-failure-unicodedecodeerror.txt 2009-08-14 16:33:57 +0000
@@ -0,0 +1,12 @@
1Hello admins,
2
3A UnicodeDecodeError occurred while trying to notify you of a
4failure during a translation export requested by %(person)s
5(%(person_id)s) at
6
7 %(request_url)s
8
9%(failed_requests)s
10
11--
12Automatic message from Launchpad.net — https://launchpad.net/
013
=== added file 'lib/lp/translations/emailtemplates/poexport-failure.txt'
--- lib/lp/translations/emailtemplates/poexport-failure.txt 1970-01-01 00:00:00 +0000
+++ lib/lp/translations/emailtemplates/poexport-failure.txt 2009-08-14 16:33:57 +0000
@@ -0,0 +1,12 @@
1Hello %(person)s,
2
3Launchpad encountered problems exporting the files you requested.
4The Launchpad Translations team has been notified of this problem.
5Please reply to this email for further assistance.
6
7If you want to retry your request, you can do so at
8
9 %(request_url)s.
10
11--
12Automatic message from Launchpad.net — https://launchpad.net/
013
=== added file 'lib/lp/translations/emailtemplates/poexport-success.txt'
--- lib/lp/translations/emailtemplates/poexport-success.txt 1970-01-01 00:00:00 +0000
+++ lib/lp/translations/emailtemplates/poexport-success.txt 2009-08-14 16:33:57 +0000
@@ -0,0 +1,15 @@
1Hello %(person)s,
2
3The translation files you requested from Launchpad are ready for
4download from the following location:
5
6 %(download_url)s
7
8Note: this link will expire in about 1 week. If you want to
9download these translations again, you will have to request
10them again at
11
12 %(request_url)s
13
14--
15Automatic message from Launchpad.net — https://launchpad.net/
016
=== modified file 'lib/lp/translations/interfaces/translationimportqueue.py'
--- lib/lp/translations/interfaces/translationimportqueue.py 2009-07-17 00:26:05 +0000
+++ lib/lp/translations/interfaces/translationimportqueue.py 2009-08-14 16:35:06 +0000
@@ -457,7 +457,7 @@
457 file_type = Choice(457 file_type = Choice(
458 title=_("File Type"),458 title=_("File Type"),
459 description=_(459 description=_(
460 "The type of the file being imported imported."),460 "The type of the file being imported."),
461 required=True,461 required=True,
462 vocabulary = TranslationFileType)462 vocabulary = TranslationFileType)
463463
464464
=== modified file 'lib/lp/translations/scripts/po_export_queue.py'
--- lib/lp/translations/scripts/po_export_queue.py 2009-07-17 00:26:05 +0000
+++ lib/lp/translations/scripts/po_export_queue.py 2009-08-14 18:02:09 +0000
@@ -10,7 +10,6 @@
1010
11import os11import os
12import psycopg212import psycopg2
13import textwrap
14import traceback13import traceback
15from StringIO import StringIO14from StringIO import StringIO
16from zope.component import getAdapter, getUtility15from zope.component import getAdapter, getUtility
@@ -18,9 +17,13 @@
18from canonical.config import config17from canonical.config import config
19from canonical.launchpad import helpers18from canonical.launchpad import helpers
20from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet19from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet
20from canonical.launchpad.webapp import canonical_url
21from lp.registry.interfaces.productseries import IProductSeries
22from lp.registry.interfaces.sourcepackage import ISourcePackage
21from lp.translations.interfaces.poexportrequest import (23from lp.translations.interfaces.poexportrequest import (
22 IPOExportRequestSet)24 IPOExportRequestSet)
23from lp.translations.interfaces.potemplate import IPOTemplate25from lp.translations.interfaces.potemplate import IPOTemplate
26from lp.translations.interfaces.pofile import IPOFile
24from lp.translations.interfaces.translationcommonformat import (27from lp.translations.interfaces.translationcommonformat import (
25 ITranslationFileData)28 ITranslationFileData)
26from lp.translations.interfaces.translationexporter import (29from lp.translations.interfaces.translationexporter import (
@@ -35,39 +38,210 @@
3538
36 This class has three main attributes:39 This class has three main attributes:
3740
38 - name: A short identifying string for this export.41 - person: A person requesting this export.
39 - url: The Librarian URL for any successfully exported files.42 - url: The Librarian URL for any successfully exported files.
40 - failure: Failure got while exporting.43 - failure: Failure gotten while exporting.
41 """44 """
4245
43 def __init__(self, name):46 def __init__(self, person, requested_exports, logger):
44 self.name = name47 self.person = person
45 self.url = None48 self.url = None
46 self.failure = None49 self.failure = None
47 self.object_names = []50 self.logger = logger
4851
49 def _getFailureEmailBody(self, person):52 self.requested_exports = list(requested_exports)
53 export_requested_at = self._getExportRequestOrigin()
54 self.name = self._getShortRequestName(export_requested_at)
55
56 self.request_url = canonical_url(
57 export_requested_at,
58 rootsite='translations') + '/+export'
59
60 def _getShortRequestName(self, request):
61 """Return a short request name for use in email subjects."""
62 if IPOFile.providedBy(request):
63 title = '%s translation of %s' % (
64 request.language.englishname,
65 request.potemplate.name)
66 productseries = request.potemplate.productseries
67 distroseries = request.potemplate.distroseries
68 sourcepackagename = request.potemplate.sourcepackagename
69 elif IPOTemplate.providedBy(request):
70 title = '%s template' % (request.name)
71 productseries = request.productseries
72 distroseries = request.distroseries
73 sourcepackagename = request.sourcepackagename
74 elif IProductSeries.providedBy(request):
75 title = None
76 productseries = request
77 distroseries = None
78 sourcepackagename = None
79 elif ISourcePackage.providedBy(request):
80 title = None
81 productseries = None
82 distroseries = request.distroseries
83 sourcepackagename = request.sourcepackagename
84 else:
85 raise AssertionError(
86 "We can not figure out short name for this translation "
87 "export origin.")
88
89 if productseries is not None:
90 root = '%s %s' % (
91 productseries.product.displayname,
92 productseries.name)
93 else:
94 root = '%s %s %s' % (
95 distroseries.distribution.displayname,
96 distroseries.displayname,
97 sourcepackagename.name)
98 if title is not None:
99 return '%s - %s' % (root, title)
100 else:
101 return root
102
103 def _getExportRequestOrigin(self):
104 """Figure out where an export request was made."""
105 # Determine all objects that export request could have
106 # originated on.
107 export_requested_at = None
108 pofiles = set()
109 implicit_potemplates = set()
110 direct_potemplates = set()
111 productseries = set()
112 sourcepackages = set()
113
114 last_template_name = None
115 for request in self.requested_exports:
116 if IPOTemplate.providedBy(request):
117 # If we are exporting a template, add it to
118 # the list of directly requested potemplates.
119 potemplate = request
120 direct_potemplates.add(potemplate)
121 else:
122 # Otherwise, we are exporting a POFile.
123 potemplate = request.potemplate
124 implicit_potemplates.add(potemplate)
125 pofiles.add(request)
126 if potemplate.displayname != last_template_name:
127 self.logger.debug(
128 'Exporting objects for %s, related to template %s'
129 % (self.person.displayname, potemplate.displayname))
130 last_template_name = potemplate.displayname
131
132 # Determine productseries or sourcepackage for any
133 # productseries/sourcepackage an export was requested at.
134 if potemplate.productseries is not None:
135 productseries.add(potemplate.productseries)
136 elif potemplate.sourcepackagename is not None:
137 sourcepackage = potemplate.distroseries.getSourcePackage(
138 potemplate.sourcepackagename)
139 sourcepackages.add(sourcepackage)
140 else:
141 pass
142
143 if len(pofiles) == 1 and len(direct_potemplates) == 0:
144 # One POFile was requested.
145 export_requested_at = pofiles.pop()
146 elif len(pofiles) == 0 and len(direct_potemplates) == 1:
147 # A POTemplate was requested.
148 export_requested_at = direct_potemplates.pop()
149 elif len(pofiles) + len(direct_potemplates) >= 2:
150 # More than one file was requested.
151 all_potemplates = implicit_potemplates.union(direct_potemplates)
152 if len(all_potemplates) == 1:
153 # It's all part of a single POTemplate.
154 export_requested_at = all_potemplates.pop()
155 else:
156 # More than one POTemplate: request was made on
157 # either ProductSeries or SourcePackage.
158 if len(sourcepackages) > 0:
159 export_requested_at = sourcepackages.pop()
160 elif len(productseries) > 0:
161 export_requested_at = productseries.pop()
162
163 if IPOTemplate.providedBy(export_requested_at):
164 if len(sourcepackages) > 0:
165 sp = sourcepackages.pop()
166 if sp.getCurrentTranslationTemplates().count() == 1:
167 export_requested_at = sp
168 elif len(productseries) > 0:
169 ps = productseries.pop()
170 if ps.getCurrentTranslationTemplates().count() == 1:
171 export_requested_at = ps
172
173 return export_requested_at
174
175
176 def _getRequestedExportsNames(self):
177 requested_names = []
178 for translation_object in self.requested_exports:
179 if IPOTemplate.providedBy(translation_object):
180 request_name = translation_object.displayname
181 else:
182 request_name = translation_object.title
183 requested_names.append(request_name)
184
185 return requested_names
186
187 def _getFailureEmailBody(self):
50 """Send an email notification about the export failing."""188 """Send an email notification about the export failing."""
51 return textwrap.dedent('''189 template = helpers.get_email_template(
52 Hello %s,190 'poexport-failure.txt', 'translations').decode('utf-8')
53191 return template % {
54 Launchpad encountered problems exporting the files you requested.192 'person' : self.person.displayname,
55 The Launchpad Translations team has been notified of this problem.193 'request_url' : self.request_url,
56 Please reply to this email for further assistance.194 }
57 ''' % person.displayname)195
58196 def _getFailedRequestsDescription(self):
59 def _getSuccessEmailBody(self, person):197 """Return a printable description of failed export requests."""
198 failed_requests = self._getRequestedExportsNames()
199 if len(failed_requests) > 0:
200 failed_requests_text = 'Failed export request included:\n'
201 failed_requests_text += '\n'.join(
202 [' * ' + request for request in failed_requests])
203 else:
204 failed_requests_text = 'There were no export requests.'
205 return failed_requests_text
206
207 def _getAdminFailureNotificationEmailBody(self):
208 """Send an email notification about failed export to admins."""
209 template = helpers.get_email_template(
210 'poexport-failure-admin-notification.txt',
211 'translations').decode('utf-8')
212 failed_requests = self._getFailedRequestsDescription()
213 return template % {
214 'person' : self.person.displayname,
215 'person_id' : self.person.name,
216 'request_url' : self.request_url,
217 'failure_message' : self.failure,
218 'failed_requests' : failed_requests,
219 }
220
221 def _getUnicodeDecodeErrorEmailBody(self):
222 """Send an email notification to admins about UnicodeDecodeError."""
223 template = helpers.get_email_template(
224 'poexport-failure-unicodedecodeerror.txt',
225 'translations').decode('utf-8')
226 failed_requests = self._getFailedRequestsDescription()
227 return template % {
228 'person' : self.person.displayname,
229 'person_id' : self.person.name,
230 'request_url' : self.request_url,
231 'failed_requests' : failed_requests,
232 }
233
234 def _getSuccessEmailBody(self):
60 """Send an email notification about the export working."""235 """Send an email notification about the export working."""
61 return textwrap.dedent('''236 template = helpers.get_email_template(
62 Hello %s,237 'poexport-success.txt', 'translations').decode('utf-8')
63238 return template % {
64 The translation files you requested from Launchpad are ready for239 'person' : self.person.displayname,
65 download from the following location:240 'download_url' : self.url,
66241 'request_url' : self.request_url,
67 \t%s''' % (person.displayname, self.url)242 }
68 )243
69244 def notify(self):
70 def notify(self, person):
71 """Send a notification email to the given person about the export.245 """Send a notification email to the given person about the export.
72246
73 If there is a failure, a copy of the email is also sent to the247 If there is a failure, a copy of the email is also sent to the
@@ -76,22 +250,22 @@
76 if self.failure is None and self.url is not None:250 if self.failure is None and self.url is not None:
77 # There is no failure, so we have a full export without251 # There is no failure, so we have a full export without
78 # problems.252 # problems.
79 body = self._getSuccessEmailBody(person)253 body = self._getSuccessEmailBody()
80 elif self.failure is not None and self.url is None:254 elif self.failure is not None and self.url is None:
81 body = self._getFailureEmailBody(person)255 body = self._getFailureEmailBody()
82 elif self.failure is not None and self.url is not None:256 elif self.failure is not None and self.url is not None:
83 raise AssertionError(257 raise AssertionError(
84 'We cannot have a URL for the export and a failure.')258 'We cannot have a URL for the export and a failure.')
85 else:259 else:
86 raise AssertionError('On success, an exported URL is expected.')260 raise AssertionError('On success, an exported URL is expected.')
87261
88 recipients = list(helpers.get_contact_email_addresses(person))262 recipients = list(helpers.get_contact_email_addresses(self.person))
89263
90 for recipient in [str(recipient) for recipient in recipients]:264 for recipient in [str(recipient) for recipient in recipients]:
91 simple_sendmail(265 simple_sendmail(
92 from_addr=config.rosetta.admin_email,266 from_addr=config.rosetta.admin_email,
93 to_addrs=[recipient],267 to_addrs=[recipient],
94 subject='Translation download request: %s' % self.name,268 subject='Launchpad translation download: %s' % self.name,
95 body=body)269 body=body)
96270
97 if self.failure is None:271 if self.failure is None:
@@ -99,42 +273,20 @@
99 return273 return
100274
101 # The export process had errors that we should notify admins about.275 # The export process had errors that we should notify admins about.
102 if self.object_names:
103 names = '\n'.join(self.object_names)
104 template_sentence = "\n" + textwrap.dedent(
105 "The failed request involved these objects:\n%s" % names)
106 else:
107 template_sentence = ""
108
109 try:276 try:
110 admins_email_body = textwrap.dedent('''277 admins_email_body = self._getAdminFailureNotificationEmailBody()
111 Hello admins,
112
113 Launchpad encountered problems exporting translation files
114 requested by %s.
115
116 This means we have a bug in Launchpad that needs to be fixed
117 before this export can proceed. Here is the list of failed
118 files and the error we got:
119
120 %s%s''') % (
121 person.displayname, self.failure, template_sentence)
122 except UnicodeDecodeError:278 except UnicodeDecodeError:
123 # Unfortunately this happens sometimes: invalidly-encoded data279 # Unfortunately this happens sometimes: invalidly-encoded data
124 # makes it into the exception description, possibly from error280 # makes it into the exception description, possibly from error
125 # messages printed by msgfmt. Before we can fix that, we need to281 # messages printed by msgfmt. Before we can fix that, we need to
126 # know what exports suffer from this problem.282 # know what exports suffer from this problem.
127 admins_email_body = textwrap.dedent('''283 admins_email_body = self._getUnicodeDecodeErrorEmailBody()
128 Hello admins,
129
130 A UnicodeDecodeError occurred while trying to notify you of a
131 failure during a translation export requested by %s.
132 %s''') % (person.displayname, template_sentence)
133284
134 simple_sendmail(285 simple_sendmail(
135 from_addr=config.rosetta.admin_email,286 from_addr=config.rosetta.admin_email,
136 to_addrs=[config.launchpad.errors_address],287 to_addrs=[config.launchpad.errors_address],
137 subject='Translation download errors: %s' % self.name,288 subject=(
289 'Launchpad translation download errors: %s' % self.name),
138 body=admins_email_body)290 body=admins_email_body)
139291
140 def addFailure(self):292 def addFailure(self):
@@ -174,26 +326,11 @@
174 translation_format_exporter = (326 translation_format_exporter = (
175 translation_exporter.getExporterProducingTargetFileFormat(format))327 translation_exporter.getExporterProducingTargetFileFormat(format))
176328
177 result = ExportResult(person.name)329 result = ExportResult(person, objects, logger)
178 translation_file_list = list(objects)
179 last_template_name = None
180 for obj in translation_file_list:
181 if IPOTemplate.providedBy(obj):
182 template_name = obj.displayname
183 object_name = template_name
184 else:
185 template_name = obj.potemplate.displayname
186 object_name = obj.title
187 result.object_names.append(object_name)
188 if template_name != last_template_name:
189 logger.debug(
190 'Exporting objects for %s, related to template %s'
191 % (person.displayname, template_name))
192 last_template_name = template_name
193330
194 try:331 try:
195 exported_file = translation_format_exporter.exportTranslationFiles(332 exported_file = translation_format_exporter.exportTranslationFiles(
196 generate_translationfiledata(translation_file_list, format))333 generate_translationfiledata(list(objects), format))
197 except (KeyboardInterrupt, SystemExit):334 except (KeyboardInterrupt, SystemExit):
198 # We should never catch KeyboardInterrupt or SystemExit.335 # We should never catch KeyboardInterrupt or SystemExit.
199 raise336 raise
@@ -227,7 +364,7 @@
227 result.url = alias.http_url364 result.url = alias.http_url
228 logger.info("Stored file at %s" % result.url)365 logger.info("Stored file at %s" % result.url)
229366
230 result.notify(person)367 result.notify()
231368
232369
233def process_queue(transaction_manager, logger):370def process_queue(transaction_manager, logger):
234371
=== added file 'lib/lp/translations/tests/helpers.py'
--- lib/lp/translations/tests/helpers.py 1970-01-01 00:00:00 +0000
+++ lib/lp/translations/tests/helpers.py 2009-08-14 17:46:32 +0000
@@ -0,0 +1,19 @@
1# Copyright 2009 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).
3
4"""Helper module reused in different tests."""
5
6__metaclass__ = type
7
8__all__ = [
9 'print_mail_subject_and_body'
10 ]
11
12import email
13
14def print_mail_subject_and_body(contents):
15 msg = email.message_from_string(contents)
16 body = msg.get_payload()
17 print 'Subject: %s' % (msg['subject'])
18 for line in body.split('\n'):
19 print ">", line
020