Merge lp:~danilo/launchpad/recife-translation-credits into lp:~launchpad/launchpad/recife

[Данило Шеган]

= Translation credits handling =

== Background ==

Update translation credits handling to use new methods (we are deprecating updateTranslation call), and adjust it to cope with the model changes.

The features that we have for translation credits handling is automatic generation on display/export to include translators working through Launchpad, and automatic creation of a stub generated message so we don't have to special case it in an otherwise slow statistics counting code (i.e. we make a translated message so you don't end up with a complete translation except the credits message, which you can't translate anyway).

== Implementation ==

The new call that should be used instead of IPOTMsgSet.updateTranslation is IPOTMsgSet.setCurrentTranslation. Replacement for the read-method getCurrent/ImportedTranslationMessage is getCurrentTranslation (the concept of two "sides" is moved to the side parameter).

Model changes we are doing in the integration branch (the one this is proposed to be merged into) basically change the semantics of "current" and "imported" to mean "current [translation] in Ubuntu" and "current upstream".

Along the way, I've moved unit-tests for prepareTranslationCredits masquerading as doctests from doc/pofile.txt to tests/test_pofile.py (and removed sampledata dependence).

I didn't even consider fixing entire doc/pofile.txt so some lint issues remain (i.e. comparisons between very long strings: they are not a single line over 78 characters, but are multi-line strings instead).

= Tests =

bin/test -cvvt setTranslationCreditsToTranslated -t prepareTranslationCredits

= Launchpad lint =

Checking for conflicts and issues in changed files.

Linting changed files:
  lib/lp/translations/doc/pofile.txt
  lib/lp/translations/doc/poimport.txt
  lib/lp/translations/interfaces/translationmessage.py
  lib/lp/translations/model/pofile.py
  lib/lp/translations/model/potmsgset.py
  lib/lp/translations/tests/test_pofile.py
  lib/lp/translations/tests/test_potmsgset.py
  lib/lp/translations/utilities/translation_import.py

./lib/lp/translations/doc/pofile.txt
     267: source exceeds 78 characters.
     458: want exceeds 78 characters.
     479: want exceeds 78 characters.
     985: want exceeds 78 characters.
     987: want exceeds 78 characters.

[Gavin Panella (allenap)]

review: Approve

=== modified file 'lib/lp/translations/doc/pofile.txt'
--- lib/lp/translations/doc/pofile.txt	2010-11-15 16:25:05 +0000
+++ lib/lp/translations/doc/pofile.txt	2010-11-18 14:17:59 +0000
@@ -1048,76 +1048,6 @@
     >>> print current.translations
     [u'Happy translator']
 
-In automatically generated credits, alsa_translation.owner is now
-credited as well.
-
-    >>> potmsgset = alsa_template.getPOTMsgSetByMsgIDText(
-    ...     u'translation-credits')
-    >>> print alsa_translation.prepareTranslationCredits(
-    ...     potmsgset)
-    Happy translator
-    Launchpad Contributions:
-    Carlos ...
-
-Emails of translators just has the dummy translation.
-
-    >>> emails_of_translators = alsa_template.getPOTMsgSetByMsgIDText(
-    ...     u"_: EMAIL OF TRANSLATORS\nYour emails")
-    >>> current = emails_of_translators.getCurrentTranslationMessage(
-    ...     alsa_template, alsa_translation.language)
-    >>> print current.translations
-    [u'This is a dummy translation so that
-    the credits are counted as translated.']
-
-But now credits Carlos when prepared automatically.  Two empty emails
-are added to make formatting nicer (first one is used as a separation
-between package and Launchpad contributors, and second is empty email
-for 'Launchpad Contributions' title; see bug #133817 for details).
-
-    >>> print alsa_translation.prepareTranslationCredits(
-    ...     emails_of_translators)
-    ,,[e-mail]
-
-Names of translators just has the dummy translation.
-
-    >>> names_of_translators = alsa_template.getPOTMsgSetByMsgIDText(
-    ...     u"_: NAME OF TRANSLATORS\nYour names")
-    >>> current = names_of_translators.getCurrentTranslationMessage(
-    ...     alsa_template, alsa_translation.language)
-    >>> print current.translations
-    [u'This is a dummy translation so that
-    the credits are counted as translated.']
-
-But now names credit Carlos when prepared automatically.  As above, an
-empty name is added first to serve as a separator, and "Launchpad
-Contributions:" is used as a header for the added credits.
-
-    >>> print alsa_translation.prepareTranslationCredits(
-    ...     names_of_translators)
-    ,Launchpad Contributions:,Carlos P... M...
-
-Lets add a newly supported KDE context message with credit strings split
-into two: context and message, instead of a single message with embedded
-context.
-
-    >>> new_kde_name_credits = alsa_template.createMessageSetFromText(
-    ...     singular_text=u"Your names", plural_text=None,
-    ...     context=u"NAME OF TRANSLATORS")
-
-prepareTranslationCredits works on such messages as well:
-
-    >>> print alsa_translation.prepareTranslationCredits(new_kde_name_credits)
-    ,Launchpad Contributions:,Carlos P... M...
-
-Similar happens for email credits as with name credits.
-
-    >>> new_kde_email_credits = alsa_template.createMessageSetFromText(
-    ...     singular_text=u"Your emails", plural_text=None,
-    ...     context=u"EMAIL OF TRANSLATORS")
-    >>> print alsa_translation.prepareTranslationCredits(
-    ...     new_kde_email_credits)
-    ,,[e-mail]
-
 
 POFileToTranslationFileDataAdapter
 ----------------------------------

=== modified file 'lib/lp/translations/doc/poimport.txt'
--- lib/lp/translations/doc/poimport.txt	2010-11-09 09:46:20 +0000
+++ lib/lp/translations/doc/poimport.txt	2010-11-18 14:17:59 +0000
@@ -9,7 +9,6 @@
 
 Here are some imports we need to get this test running.
 
-    >>> from canonical.launchpad.ftests import syncUpdate
     >>> from canonical.launchpad.interfaces.launchpad import (
     ...     ILaunchpadCelebrities)
     >>> from lp.registry.interfaces.person import IPersonSet
@@ -224,7 +223,7 @@
 
 And the statistics reflect it.
 
-    >>> pofile.rosettaCount()
+    >>> pofile.currentCount()
     1
 
 

=== modified file 'lib/lp/translations/interfaces/translationmessage.py'
--- lib/lp/translations/interfaces/translationmessage.py	2010-10-18 16:36:46 +0000
+++ lib/lp/translations/interfaces/translationmessage.py	2010-11-18 14:17:59 +0000
@@ -68,6 +68,13 @@
         the community web site.
         """)
 
+    LAUNCHPAD_GENERATED = DBItem(3, """
+        Generated by Launchpad itself.
+
+        This is used for messages like generated translation
+        credits messages.
+        """)
+
 
 class TranslationValidationStatus(DBEnumeratedType):
     """Translation Validation Status

=== modified file 'lib/lp/translations/model/pofile.py'
--- lib/lp/translations/model/pofile.py	2010-11-15 16:25:05 +0000
+++ lib/lp/translations/model/pofile.py	2010-11-18 14:17:59 +0000
@@ -77,6 +77,7 @@
     BrokenTextError,
     TranslationCreditsType,
     )
+from lp.translations.interfaces.side import TranslationSide
 from lp.translations.interfaces.translationcommonformat import (
     ITranslationFileData,
     )
@@ -92,11 +93,12 @@
     TranslationFormatSyntaxError,
     )
 from lp.translations.interfaces.translationmessage import (
+    RosettaTranslationOrigin,
     TranslationValidationStatus,
     )
 from lp.translations.interfaces.translations import TranslationConstants
 from lp.translations.model.pomsgid import POMsgID
-from lp.translations.model.potmsgset import POTMsgSet
+from lp.translations.model.potmsgset import POTMsgSet, credits_message_str
 from lp.translations.model.translatablemessage import TranslatableMessage
 from lp.translations.model.translationimportqueue import collect_import_info
 from lp.translations.model.translationmessage import (
@@ -460,12 +462,15 @@
         assert credits_type != TranslationCreditsType.NOT_CREDITS, (
             "Calling prepareTranslationCredits on a message with "
             "msgid '%s'." % potmsgset.singular_text)
-        imported = potmsgset.getImportedTranslationMessage(
-            self.potemplate, self.language)
-        if imported is None:
+        upstream = potmsgset.getCurrentTranslation(
+            None, self.language, TranslationSide.UPSTREAM)
+        if (upstream is None or
+            upstream.origin == RosettaTranslationOrigin.LAUNCHPAD_GENERATED or
+            upstream.translations[0] == credits_message_str):
             text = None
         else:
-            text = imported.translations[0]
+            text = upstream.translations[0]
+
         if credits_type == TranslationCreditsType.KDE_EMAILS:
             emails = []
             if text is not None:

=== modified file 'lib/lp/translations/model/potmsgset.py'
--- lib/lp/translations/model/potmsgset.py	2010-11-18 05:28:57 +0000
+++ lib/lp/translations/model/potmsgset.py	2010-11-18 14:17:59 +0000
@@ -5,15 +5,14 @@
 
 __metaclass__ = type
 __all__ = [
+    'credits_message_str',
     'POTMsgSet',
     ]
 
 
-import datetime
 import logging
 import re
 
-import pytz
 from sqlobject import (
     ForeignKey,
     IntCol,
@@ -56,7 +55,10 @@
     POTMsgSetInIncompatibleTemplatesError,
     TranslationCreditsType,
     )
-from lp.translations.interfaces.side import ITranslationSideTraitsSet
+from lp.translations.interfaces.side import (
+    ITranslationSideTraitsSet,
+    TranslationSide,
+    )
 from lp.translations.interfaces.translationfileformat import (
     TranslationFileFormat,
     )
@@ -1596,17 +1598,20 @@
         if not self.is_translation_credit:
             return
 
-        if self.getSharedTranslationMessage(pofile.language) is not None:
+        shared_upstream_translation = self.getCurrentTranslation(
+            None, pofile.language, TranslationSide.UPSTREAM)
+
+        if shared_upstream_translation is not None:
             return
 
         # The credits message has a fixed "translator."
         translator = getUtility(ILaunchpadCelebrities).rosetta_experts
 
-        self.updateTranslation(
-            pofile, translator, [credits_message_str],
-            is_current_upstream=False, allow_credits=True,
-            force_shared=True, force_edition_rights=True,
-            lock_timestamp=datetime.datetime.now(pytz.UTC))
+        generated_translation = self.setCurrentTranslation(
+            pofile, translator, {0: credits_message_str},
+            RosettaTranslationOrigin.LAUNCHPAD_GENERATED,
+            share_with_other_side=True)
+        generated_translation.shareIfPossible()
 
     def setSequence(self, potemplate, sequence):
         """See `IPOTMsgSet`."""

=== modified file 'lib/lp/translations/tests/test_pofile.py'
--- lib/lp/translations/tests/test_pofile.py	2010-11-16 14:25:31 +0000
+++ lib/lp/translations/tests/test_pofile.py	2010-11-18 14:17:59 +0000
@@ -35,6 +35,7 @@
     )
 from lp.translations.interfaces.translationgroup import TranslationPermission
 from lp.translations.interfaces.translationmessage import (
+    RosettaTranslationOrigin,
     TranslationValidationStatus,
     )
 from lp.translations.interfaces.translationsperson import ITranslationsPerson
@@ -1088,7 +1089,32 @@
                                         canonical_url(person))
                              for person in self.pofile.contributors]))
 
-    def test_prepareTranslationCredits_extending(self):
+    def test_prepareTranslationCredits_noop(self):
+        # With no contributions, translator credits message is not None,
+        # yet it's ignored in prepareTranslationCredits.
+        credits = self.credits_potmsgset.getCurrentTranslation(
+            self.potemplate, self.pofile.language,
+            self.potemplate.translation_side)
+        self.assertIsNot(None, credits)
+        self.assertIs(
+            None,
+            self.pofile.prepareTranslationCredits(self.credits_potmsgset))
+
+    def test_prepareTranslationCredits_gnome(self):
+        # Preparing translation credits for GNOME-like credits message.
+        translator = self.factory.makePerson(
+            name=u'the-translator',
+            displayname=u'Launchpad Translator')
+        upstream_credits = self.credits_potmsgset.setCurrentTranslation(
+            self.pofile, translator, {0: 'upstream credits'},
+            RosettaTranslationOrigin.SCM, share_with_other_side=True)
+        self.assertEquals(
+            u'upstream credits\n\n'
+            'Launchpad Contributions:\n'
+            '  Launchpad Translator http://launchpad.dev/~the-translator',
+            self.pofile.prepareTranslationCredits(self.credits_potmsgset))
+
+    def test_prepareTranslationCredits_gnome_extending(self):
         # This test ensures that continuous updates to the translation credits
         # don't result in duplicate entries.
         # Only the 'translator-credits' message is covered right now.
@@ -1129,6 +1155,74 @@
             self.compose_launchpad_credits_text(imported_credits_text),
             credits_text)
 
+    def test_prepareTranslationCredits_old_kde_names(self):
+        # Preparing translation credits for old (pre-KDE4) KDE-like
+        # credits message for contributor names.
+        translator = self.factory.makePerson(
+            displayname=u'Launchpad Translator')
+        kde_names_potmsgset = self.factory.makePOTMsgSet(
+            potemplate=self.potemplate,
+            singular=u'_: NAME OF TRANSLATORS\nYour names')
+        upstream_credits = kde_names_potmsgset.setCurrentTranslation(
+            self.pofile, translator,
+            {0: 'Upstream credits'},
+            RosettaTranslationOrigin.SCM, share_with_other_side=True)
+        self.assertEquals(
+            u'Upstream credits, ,Launchpad Contributions:,'
+            'Launchpad Translator',
+            self.pofile.prepareTranslationCredits(kde_names_potmsgset))
+
+    def test_prepareTranslationCredits_old_kde_emails(self):
+        # Preparing translation credits for old (pre-KDE4) KDE-like
+        # credits message for contributor emails.
+        translator = self.factory.makePerson(
+            email=u'translator@launchpad')
+        kde_emails_potmsgset = self.factory.makePOTMsgSet(
+            potemplate=self.potemplate,
+            singular=u'_: EMAIL OF TRANSLATORS\nYour emails')
+        upstream_credits = kde_emails_potmsgset.setCurrentTranslation(
+            self.pofile, translator,
+            {0: 'translator@upstream'},
+            RosettaTranslationOrigin.SCM, share_with_other_side=True)
+        self.assertEquals(
+            u'translator@upstream,,,translator@launchpad',
+            self.pofile.prepareTranslationCredits(kde_emails_potmsgset))
+
+    def test_prepareTranslationCredits_kde_names(self):
+        # Preparing translation credits for new (KDE4 and later)
+        # KDE-like credits message for contributor names.
+        translator = self.factory.makePerson(
+            displayname=u'Launchpad Translator')
+        kde_names_potmsgset = self.factory.makePOTMsgSet(
+            potemplate=self.potemplate,
+            context=u'NAME OF TRANSLATORS',
+            singular=u'Your names')
+        upstream_credits = kde_names_potmsgset.setCurrentTranslation(
+            self.pofile, translator,
+            {0: 'Upstream credits'},
+            RosettaTranslationOrigin.SCM, share_with_other_side=True)
+        self.assertEquals(
+            u'Upstream credits, ,Launchpad Contributions:,'
+            'Launchpad Translator',
+            self.pofile.prepareTranslationCredits(kde_names_potmsgset))
+
+    def test_prepareTranslationCredits_kde_emails(self):
+        # Preparing translation credits for new (KDE4 and later)
+        # KDE-like credits message for contributor emails.
+        translator = self.factory.makePerson(
+            email=u'translator@launchpad')
+        kde_emails_potmsgset = self.factory.makePOTMsgSet(
+            potemplate=self.potemplate,
+            context=u'EMAIL OF TRANSLATORS',
+            singular=u'Your emails')
+        upstream_credits = kde_emails_potmsgset.setCurrentTranslation(
+            self.pofile, translator,
+            {0: 'translator@upstream'},
+            RosettaTranslationOrigin.SCM, share_with_other_side=True)
+        self.assertEquals(
+            u'translator@upstream,,,translator@launchpad',
+            self.pofile.prepareTranslationCredits(kde_emails_potmsgset))
+
 
 class TestTranslationPOFilePOTMsgSetOrdering(TestCaseWithFactory):
     """Test ordering of POTMsgSets as returned by PO file methods."""

=== modified file 'lib/lp/translations/tests/test_potmsgset.py'
--- lib/lp/translations/tests/test_potmsgset.py	2010-11-17 12:30:06 +0000
+++ lib/lp/translations/tests/test_potmsgset.py	2010-11-18 14:17:59 +0000
@@ -22,7 +22,6 @@
     )
 from lp.app.enums import ServiceUsage
 from lp.registry.interfaces.person import IPersonSet
-from lp.registry.interfaces.product import IProductSet
 from lp.testing import TestCaseWithFactory
 from lp.translations.interfaces.potemplate import IPOTemplateSet
 from lp.translations.interfaces.potmsgset import (
@@ -678,36 +677,63 @@
         credits_potmsgset = self.factory.makePOTMsgSet(
             self.devel_potemplate, singular=u'translator-credits')
         credits_potmsgset.setTranslationCreditsToTranslated(sr_pofile)
-        current = credits_potmsgset.getCurrentTranslationMessage(
-            self.devel_potemplate, sr_pofile.language)
+        current = credits_potmsgset.getCurrentTranslation(
+            self.devel_potemplate, sr_pofile.language,
+            TranslationSide.UPSTREAM)
         self.assertNotEqual(None, current)
+        self.assertEquals(
+            RosettaTranslationOrigin.LAUNCHPAD_GENERATED, current.origin)
+
+    def test_setTranslationCreditsToTranslated_noop_when_translated(self):
+        """Test that translation credits don't change."""
+        sr_pofile = self.factory.makePOFile('sr', self.devel_potemplate)
+        credits_potmsgset = self.factory.makePOTMsgSet(
+            self.devel_potemplate, singular=u'translator-credits')
+        old_credits = credits_potmsgset.setCurrentTranslation(
+            sr_pofile, sr_pofile.potemplate.owner, {0: 'credits'},
+            RosettaTranslationOrigin.SCM, share_with_other_side=True)
+        credits_potmsgset.setTranslationCreditsToTranslated(sr_pofile)
+        current = credits_potmsgset.getCurrentTranslation(
+            self.devel_potemplate, sr_pofile.language,
+            TranslationSide.UPSTREAM)
+        self.assertEquals(old_credits, current)
+
+    def test_setTranslationCreditsToTranslated_noop_when_not_credits(self):
+        """Test that translation doesn't change on a non-credits message."""
+        sr_pofile = self.factory.makePOFile('sr', self.devel_potemplate)
+        not_credits_potmsgset = self.factory.makePOTMsgSet(
+            self.devel_potemplate, singular=u'non-credit message')
+        not_credits_potmsgset.setTranslationCreditsToTranslated(sr_pofile)
+        current = not_credits_potmsgset.getCurrentTranslation(
+            self.devel_potemplate, sr_pofile.language,
+            TranslationSide.UPSTREAM)
+        self.assertIs(None, current)
 
     def test_setTranslationCreditsToTranslated_diverged(self):
         # Even if there's a diverged translation credits translation,
         # we should provide an automatic shared translation instead.
-        alsa_utils = getUtility(IProductSet).getByName('alsa-utils')
-        trunk = alsa_utils.getSeries('trunk')
-        potemplate = trunk.getPOTemplate('alsa-utils')
-        es_pofile = potemplate.getPOFileByLang('es')
-        credits_potmsgset = potemplate.getPOTMsgSetByMsgIDText(
-            u'_: EMAIL OF TRANSLATORS\nYour emails')
-
-        es_current = credits_potmsgset.getCurrentTranslationMessage(
-            potemplate, es_pofile.language)
-        # Let's make sure this message is also marked as imported
-        # and diverged.
-        es_current.makeCurrentUpstream(True)
-        removeSecurityProxy(es_current).potemplate = potemplate
-
-        self.assertTrue(es_current.is_current_ubuntu)
-        self.assertNotEqual(None, es_current.potemplate)
-
-        # Setting credits as translated will give us a shared translation.
-        credits_potmsgset.setTranslationCreditsToTranslated(es_pofile)
-        current_shared = credits_potmsgset.getSharedTranslationMessage(
-            es_pofile.language)
-        self.assertNotEqual(None, current_shared)
-        self.assertEqual(None, current_shared.potemplate)
+        sr_pofile = self.factory.makePOFile('sr', self.devel_potemplate)
+        credits_potmsgset = self.factory.makePOTMsgSet(
+            self.devel_potemplate, singular=u'translator-credits')
+        diverged_credits = credits_potmsgset.setCurrentTranslation(
+            sr_pofile, sr_pofile.potemplate.owner, {0: 'credits'},
+            RosettaTranslationOrigin.SCM, share_with_other_side=True)
+        # Since translation credits are special, we can't easily create
+        # a diverged translation on it, though it may already exist in
+        # the DB.
+        removeSecurityProxy(diverged_credits).potemplate = (
+            sr_pofile.potemplate)
+        # Make sure that worked (not a real test).
+        self.assertTrue(diverged_credits.is_current_upstream)
+        self.assertEquals(sr_pofile.potemplate, diverged_credits.potemplate)
+
+        credits_potmsgset.setTranslationCreditsToTranslated(sr_pofile)
+
+        # Shared translation is generated.
+        shared = credits_potmsgset.getCurrentTranslation(
+            None, sr_pofile.language, TranslationSide.UPSTREAM)
+        self.assertNotEquals(diverged_credits, shared)
+        self.assertIsNot(None, shared)
 
     def test_setTranslationCreditsToTranslated_submitter(self):
         # Submitter on the automated translation message is always
@@ -718,8 +744,9 @@
         sr_pofile.owner = translator
         credits_potmsgset = self.factory.makePOTMsgSet(
             self.devel_potemplate, singular=u'translator-credits')
-        current = credits_potmsgset.getCurrentTranslationMessage(
-            self.devel_potemplate, sr_pofile.language)
+        current = credits_potmsgset.getCurrentTranslation(
+            self.devel_potemplate, sr_pofile.language,
+            TranslationSide.UPSTREAM)
 
         rosetta_experts = getUtility(ILaunchpadCelebrities).rosetta_experts
         self.assertEqual(rosetta_experts, current.submitter)

=== modified file 'lib/lp/translations/utilities/translation_import.py'
--- lib/lp/translations/utilities/translation_import.py	2010-11-10 11:01:28 +0000
+++ lib/lp/translations/utilities/translation_import.py	2010-11-18 14:17:59 +0000
@@ -482,7 +482,7 @@
             return None
         return potmsgset.setCurrentTranslation(
             self.pofile, self.last_translator, credits,
-            RosettaTranslationOrigin.SCM, self.share_with_other_side)
+            RosettaTranslationOrigin.SCM, share_with_other_side=True)
 
     def _validateMessage(self, potmsgset, message,
                          translations, message_data):