Merge lp:~adiroiban/launchpad/bug-525371 into lp:launchpad
| Status: | Merged | ||||
|---|---|---|---|---|---|
| Merged at revision: | 10953 | ||||
| Proposed branch: | lp:~adiroiban/launchpad/bug-525371 | ||||
| Merge into: | lp:launchpad | ||||
| Diff against target: |
1034 lines (+338/-153) 18 files modified
lib/canonical/launchpad/interfaces/_schema_circular_imports.py (+18/-4) lib/canonical/launchpad/security.py (+11/-2) lib/lp/registry/configure.zcml (+0/-6) lib/lp/registry/interfaces/distroseries.py (+5/-3) lib/lp/registry/interfaces/productseries.py (+3/-1) lib/lp/registry/interfaces/sourcepackage.py (+8/-6) lib/lp/registry/model/distroseries.py (+5/-8) lib/lp/registry/model/productseries.py (+2/-4) lib/lp/registry/model/sourcepackage.py (+2/-4) lib/lp/registry/stories/webservice/xx-distroseries.txt (+2/-1) lib/lp/registry/stories/webservice/xx-project-registry.txt (+10/-5) lib/lp/translations/browser/potemplate.py (+11/-12) lib/lp/translations/interfaces/pofile.py (+12/-5) lib/lp/translations/interfaces/potemplate.py (+107/-82) lib/lp/translations/interfaces/webservice.py (+15/-0) lib/lp/translations/model/distroseries_translations_copy.py (+1/-1) lib/lp/translations/model/potemplate.py (+5/-9) lib/lp/translations/stories/webservice/xx-potemplate.txt (+121/-0) |
||||
| To merge this branch: | bzr merge lp:~adiroiban/launchpad/bug-525371 | ||||
| Related bugs: |
|
| Reviewer | Review Type | Date Requested | Status |
|---|---|---|---|
| Данило Шеган (community) | Needs Fixing on 2010-05-26 | ||
| Jeroen T. Vermeulen | 2010-05-17 | Pending | |
|
Review via email:
|
|||
Commit Message
Export POTemplate attributes in the API.
Description of the Change
= Bug 525371 =
To help creating various translations statistics (ie for Ubuntu translations) it would be nice to have an LP API that could export the following information about a POTemplate:
* template name in Launchpad
* priority
* length
* date last updated
* domain name
* iscurrent (active or not)
* source package
* is in lang pack
* in how many languages it is translates (nice to have)
The API should return a list of all templates for a series, but also the info for a single template
The requirements are tracked on this wiki page:
https:/
== Proposed fix ==
This is the first step in implementing an API for POTemplates.
We will start with exposing the POTemplates attributes.
Here are the exported attributes:
active: True
all_
date_
description: None
exported_
format: u'PO format'
id: 2
language_count: 8
message_count: 63
name: u'pmount'
owner_link: u'http://
path: u'po/template.pot'
priority: 0
resource_
self_link: u'http://
translation
== Pre-implementation notes ==
During the last UDS I have talked with Henning and we had a pre-implementation talk.
He agreed with the current exported attributes. He said that he can not comment on the technical implementation details for the API.
If you think it is needed you can ask someone experienced with the LP API to review this branch.
== Implementation details ==
POFile API export is just a stub to illustrate how pofiles are linked to the corresponding po templates.
I don't know how to fix the /usr/share lint errors. Any hints are much appreciated.
== Tests ==
lib/lp/
lib/lp/
lib/lp/
lib/lp/
== Demo and Q/A ==
To get the potemplates for a sourcepackage
curl -ks https:/
To get the potemplates for a productseries
curl -ks https:/
To get the potemplates for a distroseries
curl -ks https:/
"all_translatio
curl -ks https:/
curl -ks https:/
curl -ks https:/
= Launchpad lint =
Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.
Linting changed files:
lib/lp/
== Pyflakes notices ==
lib/lp/
8: 'IHasTranslatio
8: 'ITranslationIm
8: 'ITranslationIm
13: 'IPOTemplate' imported but unused
15: 'IPOFile' imported but unused
warning: Not importing directory '/usr/share/
warning: Not importing directory '/usr/share/
warning: Not importing directory '/usr/share/
| Jeroen T. Vermeulen (jtv) wrote : | # |
| Adi Roiban (adiroiban) wrote : | # |
Hi,
I have improved the tests to check for templates count from the db.
Here is the latest diff:
=== modified file 'lib/lp/
--- lib/lp/
+++ lib/lp/
@@ -107,9 +107,19 @@
All templates associated to a distribution series are available at the
'all_translati
+ >>> from zope.component import getUtility
+ >>> from canonical.
+ ... ILaunchpadCeleb
+ >>> from lp.translations
+ >>> login('<email address hidden>')
+ >>> hoary = getUtility(
+ >>> templates = getUtility(
+ >>> db_count = len(list(
+ >>> logout()
>>> all_translation
... '/ubuntu/
- >>> print all_translation
- 6
+ >>> api_count = all_translation
+ >>> api_count == db_count
+ True
>>> print(all_
http://
=== modified file 'lib/lp/
--- lib/lp/
+++ lib/lp/
@@ -897,11 +897,20 @@
"all_translati
product series.
+
+ >>> from zope.component import getUtility
+ >>> from lp.translations
+ >>> login('<email address hidden>')
+ >>> templates = getUtility(
+ ... IPOTemplateSet)
+ >>> db_count = len(list(
+ >>> logout()
>>> all_translation
... babadoo_foobadoo[
... 'all_translatio
- >>> print all_translation
- 1
+ >>> api_count = all_translation
+ >>> api_count == db_count
+ True
>>> print(all_
http://
=== modified file 'lib/lp/
--- lib/lp/
+++ lib/lp/
@@ -130,10 +130,27 @@
All translation templates for a source package are available at the
'all_translati
+
+ >>> from zope.component import getUtility
+ >>> from canonical.
+ ... ILaunchpadCeleb
+ >>> from lp.registry.
+ ... ISourcePackageN
+ >>> from lp.translations
+ >...
| Данило Шеган (danilo) wrote : | # |
Hi Adi,
Thanks a lot for the work on this. Overall, it's pretty good.
У пон, 17. 05 2010. у 11:07 +0000, Adi Roiban пише:
> == Tests ==
>
> lib/lp/
> lib/lp/
> lib/lp/
> lib/lp/
It usually helps if you provide a bin/test command which runs them for
lazy reviewers to copy-paste them :)
> == Demo and Q/A ==
>
> To get the potemplates for a sourcepackage
> curl -ks
https:/
>
> To get the potemplates for a productseries
> curl -ks
https:/
>
> To get the potemplates for a distroseries
> curl -ks
https:/
>
> "all_translatio
distroseries and productseries
> curl -ks
https:/
> curl -ks https:/
> curl -ks https:/
>
> = Launchpad lint =
>
> Checking for conflicts. and issues in doctests and templates.
> Running jslint, xmllint, pyflakes, and pylint.
> Using normal rules.
>
> Linting changed files:
> lib/lp/
There are many more files with many more lint issues: you should try to
run it after you commit your final change, not before (if you've got
uncommitted changes, it lints just the uncommitted files). Please fix
all the issues.
> == Pyflakes notices ==
>
> lib/lp/
> 8: 'IHasTranslatio
> 8: 'ITranslationIm
> 8: 'ITranslationIm
> 13: 'IPOTemplate' imported but unused
> 15: 'IPOFile' imported but unused
You should add the following to webservice.py if that's the intention:
__all__ = [
'IHasTransl
'IPOFile',
'IPOTemplate',
'ITranslati
'ITranslati
]
But, why is this necessary?
> warning: Not importing directory '/usr/share/
__init__.py
> warning: Not importing directory '/usr/share/
__init__.py
> warning: Not importing directory '/usr/share/
__init__.py
This might be a bug in lazr that needs fixing. Can you please check and
file appropriate bug if it is?
> === modified file 'lib/canonical/
> --- lib/canonical/
+0000
> +++ lib/canonical/
+0000
> @@ -61,8 +61,7 @@ from lp.hardwaredb.
> from lp.services.
ILanguageSet
> from lp.translations
> from canonical.
> - IBazaarApplication, IHasBug, IHasDrivers, ILaunchpadCeleb
> - IPersonRoles)
> + IHas...
| Adi Roiban (adiroiban) wrote : | # |
On Wed, 2010-05-19 at 14:43 +0000, Данило Шеган wrote:
> Hi Adi,
>
> Thanks a lot for the work on this. Overall, it's pretty good.
>
> У пон, 17. 05 2010. у 11:07 +0000, Adi Roiban пише:
>
> > == Tests ==
> >
> > lib/lp/
> > lib/lp/
> > lib/lp/
> > lib/lp/
>
> It usually helps if you provide a bin/test command which runs them for
> lazy reviewers to copy-paste them :)
Sorry. I will add it with the next MP.
./bin/test -t xx-potemplate.txt -t xx-distroseries.txt \
-t xx-project-
:)
[snip]
> > = Launchpad lint =
> >
> > Checking for conflicts. and issues in doctests and templates.
> > Running jslint, xmllint, pyflakes, and pylint.
> > Using normal rules.
> >
> > Linting changed files:
> > lib/lp/
>
> There are many more files with many more lint issues: you should try to
> run it after you commit your final change, not before (if you've got
> uncommitted changes, it lints just the uncommitted files). Please fix
> all the issues.
I have fixed some of the other warning.
There are still some warnings for code like:
@operation_
and it looks fine to me, but pylint complains about:
lib/lp/
191: [C0322, ISourcePackage.
space
required=True,
^
vocabulary=
> > == Pyflakes notices ==
> >
> > lib/lp/
> > 8: 'IHasTranslatio
> > 8: 'ITranslationIm
> > 8: 'ITranslationIm
> > 13: 'IPOTemplate' imported but unused
> > 15: 'IPOFile' imported but unused
>
> You should add the following to webservice.py if that's the intention:
>
> __all__ = [
> 'IHasTranslatio
> 'IPOFile',
> 'IPOTemplate',
> 'ITranslationIm
> 'ITranslationIm
> ]
>
> But, why is this necessary?
Only interfaces listed in webservices.py will be exported by
lazr.restful.
Adding __all__ will still generate Pyflakes notices as they are still
unused in that file.
> > warning: Not importing directory '/usr/share/
> __init__.py
> > warning: Not importing directory '/usr/share/
> __init__.py
> > warning: Not importing directory '/usr/share/
> __init__.py
>
> This might be a bug in lazr that needs fixing. Can you please check and
> file appropriate bug if it is?
Is is not a problem in lazr. I have the same problem with:
'/usr/share/
Those file were installed as dependencies for ubuntuone-
python-protobuf and python-lazr-uri, python-
maintained by main Ubuntu developers... not in the LP PPA.
> > === modified file 'lib/...
| Данило Шеган (danilo) wrote : | # |
Hi Adi,
Thanks for the updates. There's still just a little bit more to do :)
У сре, 19. 05 2010. у 20:13 +0000, Adi Roiban пише:
> There are still some warnings for code like:
>
> @operation_
> pocket=Choice(
> title=_("Pocket"), required=True,
> vocabulary=
>
> and it looks fine to me, but pylint complains about:
>
> lib/lp/
> 191: [C0322, ISourcePackage.
> space
> required=True,
> ^
> vocabulary=
>
>
> > > == Pyflakes notices ==
> > >
> > > lib/lp/
> > > 8: 'IHasTranslatio
> > > 8: 'ITranslationIm
> > > 8: 'ITranslationIm
> > > 13: 'IPOTemplate' imported but unused
> > > 15: 'IPOFile' imported but unused
> >
> > You should add the following to webservice.py if that's the intention:
> >
> > __all__ = [
> > 'IHasTranslatio
> > 'IPOFile',
> > 'IPOTemplate',
> > 'ITranslationIm
> > 'ITranslationIm
> > ]
> >
> > But, why is this necessary?
>
> Only interfaces listed in webservices.py will be exported by
> lazr.restful.
>
> Adding __all__ will still generate Pyflakes notices as they are still
> unused in that file.
It actually fixed it for me. I'm attaching a patch which fixes all the
lint issues I've seen. It would be nicer if you did that instead :)
Also, "operator not preceded by space" lint issues in this particular
context seem to be opposed to our style guide. It would be best to
raise this on launchpad-dev mailing list to come to an agreement and to
see if we can fix either pylint or style guide.
> > > warning: Not importing directory '/usr/share/
> > __init__.py
> > > warning: Not importing directory '/usr/share/
> > __init__.py
> > > warning: Not importing directory '/usr/share/
> > __init__.py
> >
> > This might be a bug in lazr that needs fixing. Can you please check and
> > file appropriate bug if it is?
>
> Is is not a problem in lazr. I have the same problem with:
> '/usr/share/
>
> Those file were installed as dependencies for ubuntuone-
> python-protobuf and python-lazr-uri, python-
> maintained by main Ubuntu developers... not in the LP PPA.
Well, LAZR is a library shared between Launchpad and UbuntuOne. It's
still a bug that would need to be filed. I've worked-around it locally
by doing "sudo touch /usr/share/
probably no reason why the package shouldn't do the same for everybody.
> Should POTemplate and POFile API be available only to authenticated
> users?
Not, it's fine if they are anonymously available.
> > === modified file
> > 'lib/lp/
> > === modified file
> > 'lib/lp/
> > === modified file
> > 'lib/lp/
| Adi Roiban (adiroiban) wrote : | # |
On Thu, 2010-05-20 at 16:33 +0000, Данило Шеган wrote:
> Hi Adi,
>
> Thanks for the updates. There's still just a little bit more to do :)
>
> У сре, 19. 05 2010. у 20:13 +0000, Adi Roiban пише:
>
> > There are still some warnings for code like:
> >
> > @operation_
> > pocket=Choice(
> > title=_("Pocket"), required=True,
> > vocabulary=
> >
> > and it looks fine to me, but pylint complains about:
> >
> > lib/lp/
> > 191: [C0322, ISourcePackage.
> > space
> > required=True,
> > ^
> > vocabulary=
> >
> >
> > > > == Pyflakes notices ==
> > > >
> > > > lib/lp/
> > > > 8: 'IHasTranslatio
> > > > 8: 'ITranslationIm
> > > > 8: 'ITranslationIm
> > > > 13: 'IPOTemplate' imported but unused
> > > > 15: 'IPOFile' imported but unused
> > >
> > > You should add the following to webservice.py if that's the intention:
> > >
> > > __all__ = [
> > > 'IHasTranslatio
> > > 'IPOFile',
> > > 'IPOTemplate',
> > > 'ITranslationIm
> > > 'ITranslationIm
> > > ]
> > >
> > > But, why is this necessary?
> >
> > Only interfaces listed in webservices.py will be exported by
> > lazr.restful.
> >
> > Adding __all__ will still generate Pyflakes notices as they are still
> > unused in that file.
>
> It actually fixed it for me. I'm attaching a patch which fixes all the
> lint issues I've seen. It would be nicer if you did that instead :)
I have no idea why this is not fixing for me.
Here is the output for `make lint` after applying your patch:
= Launchpad lint =
Checking for conflicts. and issues in doctests and templates.
Running jslint, xmllint, pyflakes, and pylint.
Using normal rules.
Linting changed files:
lib/lp/
lib/lp/
lib/lp/
lib/lp/
== Pyflakes notices ==
lib/lp/
16: 'IHasTranslatio
16: 'ITranslationIm
16: 'ITranslationIm
21: 'IPOTemplate' imported but unused
23: 'IPOFile' imported but unused
> Also, "operator not preceded by space" lint issues in this particular
> context seem to be opposed to our style guide. It would be best to
> raise this on launchpad-dev mailing list to come to an agreement and to
> see if we can fix either pylint or style guide.
I have sent and email to lp-dev ML.
> > > > warning: Not importing directory '/usr/share/
> > > __init__.py
> > > > warning: Not importing directory '/usr/share/
> > > __init__.py
> > > > warning: Not importing directory '/usr/share/
> > > __init__.py
> > >
> > > This might be a bug in lazr that needs fixing. Can you please check and
> > > file app...
| Данило Шеган (danilo) wrote : | # |
Hey Adi, we are getting very close. Just a few more small issues to
fix.
У чет, 20. 05 2010. у 18:00 +0000, Adi Roiban пише:
> > It actually fixed it for me. I'm attaching a patch which fixes all the
> > lint issues I've seen. It would be nicer if you did that instead :)
>
> I have no idea why this is not fixing for me.
>
> Here is the output for `make lint` after applying your patch:
>
> = Launchpad lint =
>
> Checking for conflicts. and issues in doctests and templates.
> Running jslint, xmllint, pyflakes, and pylint.
> Using normal rules.
>
> Linting changed files:
> lib/lp/
> lib/lp/
> lib/lp/
> lib/lp/
>
> == Pyflakes notices ==
>
> lib/lp/
> 16: 'IHasTranslatio
> 16: 'ITranslationIm
> 16: 'ITranslationIm
> 21: 'IPOTemplate' imported but unused
> 23: 'IPOFile' imported but unused
Hum, very interesting. Perhaps it's a problem with pyflakes
incompatibilities? (FWIW, it seems your branch now specifies __all__
twice)
FWIW, defining __all__ fixes pyflakes warnings for me, but not the
pylint warnings.
> > Also, "operator not preceded by space" lint issues in this particular
> > context seem to be opposed to our style guide. It would be best to
> > raise this on launchpad-dev mailing list to come to an agreement and to
> > see if we can fix either pylint or style guide.
>
> I have sent and email to lp-dev ML.
Cool. It seems everybody agrees that we should not change the
styleguide, but instead not use pylint as is. You can ignore those
warnings and revert that bit of my patch. It'd still be very useful to
resolve the pyflakes warnings that you are getting, though.
> I have filled a bug report for that:
> https:/
Excellent, thanks a lot.
> [snip]
> >
> > > > > === modified file 'lib/lp/
> > > > ...
> > > > > source_file = Object(
> > > > > title=_('Source file for this translation template'),
> > > > > readonly=True, schema=
> > > >
> > > > I don't remember why we decided not to export this one? It could be
> > > > useful for things like xpipo conversion (with XPI-based translations,
> > > > it should contain a reference to imported en-US.xpi).
> > > >
> > > > Though, we should probably just export it as the public URL, so not a
> > > > big deal (let's not block this branch on this).
> > >
> > > I will leave it for another branch, but for me, it would make sense if
> > > this attribute would be used for all translation templates, not only
> > > XPI.
> >
> > Well, we can reconstruct the gettext POT file completely from what we
> > have in the DB. Thus, it's not needed.
>
> My note was about creating a more consistent API so that in the API
> source_file (or whatever name we choose for it) would return the
> template file for all formats, not only XPI.
Sure...
> Instead of...
| Adi Roiban (adiroiban) wrote : | # |
On Thu, 2010-05-20 at 18:48 +0000, Данило Шеган wrote:
> Hey Adi, we are getting very close. Just a few more small issues to
> fix.
>
> У чет, 20. 05 2010. у 18:00 +0000, Adi Roiban пише:
>
> > > It actually fixed it for me. I'm attaching a patch which fixes all the
> > > lint issues I've seen. It would be nicer if you did that instead :)
> >
> > I have no idea why this is not fixing for me.
> >
> > Here is the output for `make lint` after applying your patch:
> >
> > = Launchpad lint =
> >
> > Checking for conflicts. and issues in doctests and templates.
> > Running jslint, xmllint, pyflakes, and pylint.
> > Using normal rules.
> >
> > Linting changed files:
> > lib/lp/
> > lib/lp/
> > lib/lp/
> > lib/lp/
> >
> > == Pyflakes notices ==
> >
> > lib/lp/
> > 16: 'IHasTranslatio
> > 16: 'ITranslationIm
> > 16: 'ITranslationIm
> > 21: 'IPOTemplate' imported but unused
> > 23: 'IPOFile' imported but unused
>
> Hum, very interesting. Perhaps it's a problem with pyflakes
> incompatibilities? (FWIW, it seems your branch now specifies __all__
> twice)
>
> FWIW, defining __all__ fixes pyflakes warnings for me, but not the
> pylint warnings.
I have removed the duplicate __all__ and only leave it before the
imports.
[snip]
> > > Add __all__: have you tried it at all? It totally works for me, and I'd
> > > be very surprised if it doesn't work for you. Also, __all__ is required
> > > in all our modules as well (importfascist used to check that before,
> > > maybe it doesn't anymore)
>
> I'd really like to get this resolved. It seems the line length is also
> slightly differently defined for you:
>
> === modified file 'lib/lp/
> --- lib/lp/
> +++ lib/lp/
> @@ -127,9 +127,9 @@
> '''),
> required=False)
>
> - translation_
> - All `ITranslationMe
> - '''))
> + translation_
> + "All `ITranslationMe
> + "translation file."))
>
> plural_forms = Int(
> title=_('Number of plural forms for the language of this PO file.'),
>
>
> This is not needed for me, because just putting it as
>
> translation_
> 'All `ITranslationMe
> ))
>
> works fine. Not that there's anything wrong with your approach, I am just noting this.
This will break the styleguide as there is a space before the ')'
> > > > === modified file 'lib/lp/
> > > > --- lib/lp/
> > > > +++ lib/lp/
| Данило Шеган (danilo) wrote : | # |
Hi Adi,
У чет, 20. 05 2010. у 19:06 +0000, Adi Roiban пише:
> > This is not needed for me, because just putting it as
> >
> > translation_
> > 'All `ITranslationMe
> > ))
> >
> > works fine. Not that there's anything wrong with your approach, I am just noting this.
>
> This will break the styleguide as there is a space before the ')'
Heh, this is not really a space: it's a multiline wrapping of braces,
and we do exactly the same thing with tuples (we'd do it with imports as
well if we didn't have too many of those), as mentioned on [1]:
something = (
FirstVery
SecondVer
)
[1] https:/
Anyway, as I said, your style is just as fine as well.
review approve
merge approve
I'll get this into ec2 land soon.
Cheers,
Danilo
| Данило Шеган (danilo) wrote : | # |
Hi Adi,
Ok, I realized there are more things to fix first. We've discussed them
over on IRC, but I will repeat myself a bit for reference.
review needs-fixing
Attribute names should include as little gettextisms as possible:
"all_pofiles" and "all_potemplates" should be "translation_files" and
"translation_
branch: I am not sure about the "all").
And further on that topic, we should only show most relevant translation
templates: these are usually only the current, non-obsolete ones.
Obsolete templates are useful only for management purposes, and if we
want, we can export them either separately, or export all of
getCurrentTra
getObsoleteTr
getTranslatio
There is no need to wrap either of the methods in a property in order to
export them. What's else, these methods expose some implementation
details (like "just_ids" parameter, needed for performance reasons
elsewhere) and we should not get them exported as such.
Since I want to see this branch move in soon, I don't want to dwell on
cleaning up these right now.
Now, performance reasons are very important to consider here. We may
have to investigate whether `doNotSnapshot` is needed, and how batching
works over Collection results ([1] suggests it splits results into
batches of 75, but we'd have to check that). If API framework splits
the results into batches and only serializes a single batch (vs
serializing a whole result set), it'd be ok even if we don't use
doNotSnapshot (unless it turns out to be too slow once it hits
edge/staging, when we'll need to fix it).
Also, at the moment, we should export only the most useful list of
templates (because most of this code needs slight refactoring, we should
not export all methods and then stop exporting them soon after). At the
moment, for the "reporting" use case, I believe only "current" templates
make sense to be exported. Now, exporting it with just_ids parameter is
fine as well, and we can fix it with the refactoring in the future.
Next, we did mention how it should already allow read-write access if
you've got sufficient privileges. In my tests that has turned out to be
false. This is definitely not a blocker (it's ok if we start with
read-only API), but we at least need to understand why this is so and
how is write access created (so we don't do it by accident).
Example script I tried out (after doing 'make run' in this branch, run
this in a python console and log in with sufficient privileges, eg.
<email address hidden>:test):
cachedir = "/home/
from launchpadlib.
launchpad = Launchpad.
ubuntu = launchpad.
hoary = ubuntu.series[2]
pots = hoary.all_
pot = pots[0]
pot.translati
pot.lp_save()
So, suggested course of action to get this landed:
0. fix "all_pofile" attribute name
1. Check if we get batching of results on API method calls/attributes
for free (without needing to do anything else)
2. If 1 is true, check if batching snapshots the en...
| Adi Roiban (adiroiban) wrote : | # |
On Wed, 2010-05-26 at 16:12 +0000, Данило Шеган wrote:
> Review: Needs Fixing
> Hi Adi,
>
> Ok, I realized there are more things to fix first. We've discussed them
> over on IRC, but I will repeat myself a bit for reference.
>
> review needs-fixing
>
> Attribute names should include as little gettextisms as possible:
> "all_pofiles" and "all_potemplates" should be "translation_files" and
> "translation_
> branch: I am not sure about the "all").
renamed all_pofiles to translation_files
removed all_translation
a method.
I have renamed the name used for exporting pofile and potemplate entries
and collections.
> And further on that topic, we should only show most relevant translation
> templates: these are usually only the current, non-obsolete ones.
> Obsolete templates are useful only for management purposes, and if we
> want, we can export them either separately, or export all of
>
> getCurrentTrans
> getObsoleteTran
> getTranslationT
>
> There is no need to wrap either of the methods in a property in order to
> export them. What's else, these methods expose some implementation
> details (like "just_ids" parameter, needed for performance reasons
> elsewhere) and we should not get them exported as such.
>
> Since I want to see this branch move in soon, I don't want to dwell on
> cleaning up these right now.
We can hide the "just_ids" in the exported API call.
We can export getCurrentTrans
hidden and always set to False.
I still think that exporting getTranslationT
simpler (rather than exporting all 3 methods) and the client side check
for active/inactive templates is not that complicated.
> Now, performance reasons are very important to consider here. We may
> have to investigate whether `doNotSnapshot` is needed, and how batching
> works over Collection results ([1] suggests it splits results into
> batches of 75, but we'd have to check that). If API framework splits
> the results into batches and only serializes a single batch (vs
> serializing a whole result set), it'd be ok even if we don't use
> doNotSnapshot (unless it turns out to be too slow once it hits
> edge/staging, when we'll need to fix it).
I will raise this discussion over the lp-dev ML. I still have to read
the code and see exactly why, where and how those snosphots are
needed/used in order to ask a valid question.
I think that we should land this branch and the start of next cycle.
The first attempt should be without doNotSnapshot. If something goes
wrong we can reconsider our options.
> Also, at the moment, we should export only the most useful list of
> templates (because most of this code needs slight refactoring, we should
> not export all methods and then stop exporting them soon after). At the
> moment, for the "reporting" use case, I believe only "current" templates
> make sense to be exported. Now, exporting it with just_ids parameter is
> fine as well, and we can fix it with the refactoring in the future.
See above comments.
> Next, we...
| Jeroen T. Vermeulen (jtv) wrote : | # |
We've been discussing this in IRC. It turns out that IDistroSeries.
The docstring deliberately makes no promise that getTranslationT
| Jeroen T. Vermeulen (jtv) wrote : | # |
Revision 10369 looks good to me (except the part where you update the getTranslationT
You can also make the check easier to read by eliminating individual, simple cases:
if official_rosetta and potemplate.
# This template is available for translation.
return potemplate
elif check_permissio
# User has special privileges for this template.
return potemplate
else:
raise NotFoundError(name)
Also, a bit higher up, I don't think the assertion for "unknown context" carries its weight. We have database constraints enforcing it. So might as well do something simpler such as:
if potemplate.
product_
else:
product_
official_rosetta = product_
| Jeroen T. Vermeulen (jtv) wrote : | # |
For the record I should add that we first discussed the bigger issues (performance, security, various batching hazards) as per Danilo's proposed course of action above. All those points have been satisfied, so the nitpicking about small bits of code means that the branch is in good shape. There is no problematic snapshotting, performance is good, and with the removal of that shortlist we no longer fetch all templates in a distroseries at once.
| Adi Roiban (adiroiban) wrote : | # |
Hi,
Here is the diff with the changes discussed in the latest comment.
If everything is ok, can you please sent this branch to ec2-test?
=== modified file 'lib/lp/
--- lib/lp/
+++ lib/lp/
@@ -1886,7 +1886,7 @@
"""See `IHasTranslatio
result = POTemplate.
- return shortlist(result, 2000)
+ return result
def getCurrentTrans
"""See `IHasTranslatio
=== modified file 'lib/lp/
--- lib/lp/
+++ lib/lp/
@@ -427,7 +427,7 @@
"""See `IHasTranslatio
result = POTemplate.
- return shortlist(result, 300)
+ return result
def getCurrentTrans
"""See `IHasTranslatio
=== modified file 'lib/lp/
--- lib/lp/
+++ lib/lp/
@@ -602,7 +602,7 @@
result = POTemplate.
- return shortlist(
+ return result.
def getCurrentTrans
"""See `IHasTranslatio
=== modified file 'lib/lp/
--- lib/lp/
+++ lib/lp/
@@ -774,18 +774,17 @@
# Get whether the target for the requested template is officially
# using Launchpad Translations.
- if potemplate.
- official_rosetta = potemplate.
- elif potemplate.product is not None:
- official_rosetta = potemplate.
+ if potemplate.
+ product_or_distro = potemplate.
else:
- raise AssertionError(
+ product_or_distro = potemplate.
+ official_rosetta = product_
- if ((official_rosetta and potemplate.
- check_permissio
- # The target is using officially Launchpad Translations and the
- # template is available to be translated, or the user is a is a
- # Launchpad administrator in which case we show everything.
+ if official_rosetta and potemplate.
+ # This template is av...

Hi Adi,
It's really great that you're doing this. I'm reading it through and it looks good to me so far. One note though about the testing in xx-distroseries .txt.
You've got the problem here that you have to rely on existing test data, which can be a bit arbitrary. For example, the fact that there are 6 templates in the distroseries is pretty meaningless in the context of the test. But it's a lot of trouble to set up a fresh distroseries with fresh translation templates, especially if the existing tests don't do it either.
So here's an idea: instead of printing the exact number of templates, why not show that int(all_ translation_ templates[ 'total_ size']) is identical to len(list( getUtility( ILaunchpadCeleb rities) .ubuntu. getSeries( 'hoary' ))).
Jeroen