Merge lp:~mterry/quickly/arb into lp:quickly

[Michael Terry]

This branch lets us drop all the craziness I put into debian/rules for --extras packages. We put those there because for the SRU, we wanted to reduce the surface of the changes.

But now we can put them back into the user bin wrapper and setup.py like they should have been all along.

Note that this branch relies on the overwrite branch, so only a little bit of it (the gettext->locale stuff) needs to have explicit upgrade.py support.

[lp:~mterry/quickly/arb]

691. By Michael Terry on 2012-08-07

make sure to not lose the bug fix for supporting added script arguments in desktop files

692. By Michael Terry on 2012-08-08

provide correct default for XDG_DATA_DIRS

[Didier Roche-Tolomelli (didrocks)]

\o/

The changes are looking good to me (and way better than the packaging insanity).

Some comments:
299 + print ("ERROR: Can't find", filepath)
you missed a %s or at least a space after "find" ;)

370 +print (project_version, template_version)
Is that a leftover?

379 + print("running", "find %s -name '*.py' -exec %s {} \;" % (python_name, sedline))
Hum, same here about print(a, b), this return a tuple, like print("bar", "foo")
('bar', 'foo')
not a friendly one line message, isn't it?

230 -import gettext
231 -from gettext import gettext as _
232 -gettext.textdomain('project_name')
233 +from locale import gettext as _
-> is it better? I have both in OneConf because of some UTF-8 errors and a fallback to the other one. If you have some info on that just for the info, I'll take them gladely :)

Thanks for adding the Exec=foo bar baz test case btw!

Approving (but I would appreciate some answers on my minor quesions before) ;) However, look at the pre-requisite branch which needs a little bit more work IMHO.

review: Approve

[lp:~mterry/quickly/arb]

693. By Michael Terry on 2012-08-17

merge latest overwrite branch

694. By Michael Terry on 2012-08-17

minor cleanup

[Michael Terry (mterry)]

Thanks, dropped the accidentally included print message.

The locale version of gettext is apparently better because it supports when you've changed the localedir as we do for extras. The preference for locale was originally from a branch by dpm. I haven't seen bad side effects like utf8 errors.

Branch updated and merged my latest overwrite changes.

=== modified file 'data/templates/ubuntu-application/help/tutorial.xml'
--- data/templates/ubuntu-application/help/tutorial.xml	2012-08-02 14:29:46 +0000
+++ data/templates/ubuntu-application/help/tutorial.xml	2012-08-17 21:00:25 +0000
@@ -3,7 +3,7 @@
 "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd" [
 
  <!ENTITY appname "<application>Quickly</application>" >
- <!ENTITY appversion "12.08" >
+ <!ENTITY appversion "12.08.1" >
 ] >
 
 <book lang="en" id="index">
 
=== modified file 'data/templates/ubuntu-application/internal/packaging.py'
--- data/templates/ubuntu-application/internal/packaging.py	2012-07-09 12:16:02 +0000
+++ data/templates/ubuntu-application/internal/packaging.py	2012-08-17 21:00:25 +0000
@@ -159,99 +159,7 @@
         return(_continue_if_errors(err_output, warn_output, proc.returncode,
                                      ask_on_warn_or_error))
 
-
-# TODO: Vastly simplify this function.  This was implemented post-12.04 to
-# make it easier to get an SRU for these fixes.  But the only things here
-# that can't be done in-code (either wrapper code or setup.py) is compiling
-# the schemas and moving the desktop file.
 def update_rules():
-    project_name = configurationhandler.project_config['project']
-
-    install_rules = """
-override_dh_install:
-	dh_install"""
-
-    opt_root = "/opt/extras.ubuntu.com/" + project_name
-
-    # Move script to bin/ folder.
-    # There are some complications here.  As of this writing, current versions
-    # of python-mkdebian do not correctly install our executable script in a
-    # bin/ subdirectory.  Instead, they either install it in the opt-project
-    # root or in the opt-project python library folder (depending on whether
-    # the project name is the same as its python name).  So if we find that to
-    # be the case, we move the script accordingly.
-    bin_path = "%(opt_root)s/bin/%(project_name)s" % {
-        'opt_root': opt_root, 'project_name': project_name}
-    bad_bin_debpath = "debian/%(project_name)s%(opt_root)s/%(project_name)s" % {
-        'opt_root': opt_root, 'project_name': project_name}
-    python_name = templatetools.python_name(project_name)
-    if project_name == python_name:
-        bad_bin_debpath += "/" + project_name
-    install_rules += """
-	mkdir -p debian/%(project_name)s%(opt_root)s/bin
-	if [ -x %(bad_bin_debpath)s ]; then mv %(bad_bin_debpath)s debian/%(project_name)s%(opt_root)s/bin; fi""" % {
-        'project_name': project_name, 'opt_root': opt_root, 'bad_bin_debpath': bad_bin_debpath}
-
-    # Move desktop file and update it to point to our /opt locations.
-    # The file starts, as expected, under /opt.  But the ARB wants and allows
-    # us to install it in /usr, so we do.
-    old_desktop_debdir = "debian/%(project_name)s%(opt_root)s/share/applications" % {
-        'project_name': project_name, 'opt_root': opt_root}
-    new_desktop_debdir = "debian/%(project_name)s/usr/share/applications" % {'project_name': project_name}
-    new_desktop_debpath = new_desktop_debdir + "/extras-" + project_name + ".desktop"
-    install_rules += """
-	if [ -f %(old_desktop_debdir)s/%(project_name)s.desktop ]; then \\
-		mkdir -p %(new_desktop_debdir)s; \\
-		mv %(old_desktop_debdir)s/%(project_name)s.desktop %(new_desktop_debpath)s; \\
-		rmdir --ignore-fail-on-non-empty %(old_desktop_debdir)s; \\
-		sed -i 's|Exec=[^ ]*|Exec=%(bin_path)s|' %(new_desktop_debpath)s; \\
-		sed -i 's|Icon=/usr/|Icon=%(opt_root)s/|' %(new_desktop_debpath)s; \\
-	fi""" % {
-        'bin_path': bin_path, 'old_desktop_debdir': old_desktop_debdir,
-        'new_desktop_debdir': new_desktop_debdir, 'project_name': project_name,
-        'opt_root': opt_root, 'new_desktop_debpath': new_desktop_debpath}
-
-    # Set gettext's bindtextdomain to point to /opt and use the locale
-    # module (gettext's C API) instead of the gettext module (gettext's Python
-    # API), so that translations are loaded from /opt
-    localedir = os.path.join(opt_root, 'share/locale') 
-    install_rules += """
-	grep -RlZ 'import gettext' debian/%(project_name)s/* | xargs -0 -r sed -i 's|\(import\) gettext$$|\\1 locale|'
-	grep -RlZ 'from gettext import gettext as _' debian/%(project_name)s/* | xargs -0 -r sed -i 's|from gettext \(import gettext as _\)|from locale \\1|'
-	grep -RlZ "gettext.textdomain('%(project_name)s')" debian/%(project_name)s/* | xargs -0 -r sed -i "s|gettext\(\.textdomain('%(project_name)s')\)|locale\.bindtextdomain('%(project_name)s', '%(localedir)s')\\nlocale\\1|" """ % {
-        'project_name': project_name, 'localedir': localedir}
-
-    # We install a python_nameconfig.py file that contains a pointer to the
-    # data directory.  But that will be determined by setup.py, so it will be
-    # wrong (python-mkdebian's --prefix command only affects where it moves
-    # files during build, but not what it passes to setup.py)
-    config_debpath = "debian/%(project_name)s%(opt_root)s/%(python_name)s*/%(python_name)sconfig.py" % {
-        'project_name': project_name, 'opt_root': opt_root, 'python_name': python_name}
-    install_rules += """
-	sed -i "s|__%(python_name)s_data_directory__ =.*|__%(python_name)s_data_directory__ = '%(opt_root)s/share/%(project_name)s/'|" %(config_debpath)s""" % {
-        'opt_root': opt_root, 'project_name': project_name,
-        'python_name': python_name, 'config_debpath': config_debpath}
-
-    # Adjust XDG_DATA_DIRS so we can find schemas and help files
-    install_rules += """
-	sed -i 's|        sys.path.insert(0, opt_path)|\\0\\n    os.putenv("XDG_DATA_DIRS", "%%s:%%s" %% ("%(opt_root)s/share/", os.getenv("XDG_DATA_DIRS", "")))|' debian/%(project_name)s%(bin_path)s""" % {
-        'opt_root': opt_root, 'project_name': project_name, 'bin_path': bin_path}
-
-    # Compile the glib schema, since it is in a weird place that normal glib
-    # triggers won't catch during package install.
-    schema_debdir = "debian/%(project_name)s%(opt_root)s/share/glib-2.0/schemas" % {
-        'opt_root': opt_root, 'project_name': project_name}
-    install_rules += """
-	if [ -d %(schema_debdir)s ]; then glib-compile-schemas %(schema_debdir)s; fi""" % {
-        'schema_debdir': schema_debdir}
-
-    # Set rules back to include our changes
-    rules = ''
-    with open('debian/rules', 'r') as f:
-        rules = f.read()
-    rules += install_rules
-    templatetools.set_file_contents('debian/rules', rules)
-
     # Also update debian/control to add a new Build-Depends needed for
     # glib-compile-schemas
     templatetools.update_file_content(
 
=== modified file 'data/templates/ubuntu-application/project_root/bin/project_name'
--- data/templates/ubuntu-application/project_root/bin/project_name	2012-08-17 21:00:25 +0000
+++ data/templates/ubuntu-application/project_root/bin/project_name	2012-08-17 21:00:25 +0000
@@ -9,9 +9,8 @@
 import sys
 import os
 
-import gettext
-from gettext import gettext as _
-gettext.textdomain('project_name')
+import locale
+locale.textdomain('project_name')
 
 # Add project root directory (enable symlink and trunk execution)
 PROJECT_ROOT_DIRECTORY = os.path.abspath(
@@ -19,11 +18,13 @@
 
 python_path = []
 if os.path.abspath(__file__).startswith('/opt'):
+    locale.bindtextdomain('project_name', '/opt/extras.ubuntu.com/project_name/share/locale')
     syspath = sys.path[:] # copy to avoid infinite loop in pending objects
     for path in syspath:
         opt_path = path.replace('/usr', '/opt/extras.ubuntu.com/project_name')
         python_path.insert(0, opt_path)
         sys.path.insert(0, opt_path)
+    os.putenv("XDG_DATA_DIRS", "%s:%s" % ("/opt/extras.ubuntu.com/project_name/share/", os.getenv("XDG_DATA_DIRS", "/usr/local/share/:/usr/share/")))
 if (os.path.exists(os.path.join(PROJECT_ROOT_DIRECTORY, 'python_name'))
     and PROJECT_ROOT_DIRECTORY not in sys.path):
     python_path.insert(0, PROJECT_ROOT_DIRECTORY)
 
=== modified file 'data/templates/ubuntu-application/project_root/python/Aboutcamel_case_nameDialog.py'
--- data/templates/ubuntu-application/project_root/python/Aboutcamel_case_nameDialog.py	2011-04-04 14:28:36 +0000
+++ data/templates/ubuntu-application/project_root/python/Aboutcamel_case_nameDialog.py	2012-08-17 21:00:25 +0000
@@ -3,9 +3,7 @@
 # This file is in the public domain
 ### END LICENSE
 
-import gettext
-from gettext import gettext as _
-gettext.textdomain('project_name')
+from locale import gettext as _
 
 import logging
 logger = logging.getLogger('python_name')
 
=== modified file 'data/templates/ubuntu-application/project_root/python/Preferencescamel_case_nameDialog.py'
--- data/templates/ubuntu-application/project_root/python/Preferencescamel_case_nameDialog.py	2011-11-08 02:27:24 +0000
+++ data/templates/ubuntu-application/project_root/python/Preferencescamel_case_nameDialog.py	2012-08-17 21:00:25 +0000
@@ -11,9 +11,7 @@
 
 from gi.repository import Gio # pylint: disable=E0611
 
-import gettext
-from gettext import gettext as _
-gettext.textdomain('project_name')
+from locale import gettext as _
 
 import logging
 logger = logging.getLogger('python_name')
 
=== modified file 'data/templates/ubuntu-application/project_root/python/__init__.py'
--- data/templates/ubuntu-application/project_root/python/__init__.py	2011-11-07 21:55:54 +0000
+++ data/templates/ubuntu-application/project_root/python/__init__.py	2012-08-17 21:00:25 +0000
@@ -5,9 +5,7 @@
 
 import optparse
 
-import gettext
-from gettext import gettext as _
-gettext.textdomain('project_name')
+from locale import gettext as _
 
 from gi.repository import Gtk # pylint: disable=E0611
 
 
=== modified file 'data/templates/ubuntu-application/project_root/python/camel_case_nameWindow.py'
--- data/templates/ubuntu-application/project_root/python/camel_case_nameWindow.py	2011-06-08 19:20:41 +0000
+++ data/templates/ubuntu-application/project_root/python/camel_case_nameWindow.py	2012-08-17 21:00:25 +0000
@@ -3,9 +3,7 @@
 # This file is in the public domain
 ### END LICENSE
 
-import gettext
-from gettext import gettext as _
-gettext.textdomain('project_name')
+from locale import gettext as _
 
 from gi.repository import Gtk # pylint: disable=E0611
 import logging
 
=== modified file 'data/templates/ubuntu-application/project_root/python_lib/helpers.py'
--- data/templates/ubuntu-application/project_root/python_lib/helpers.py	2012-08-17 21:00:25 +0000
+++ data/templates/ubuntu-application/project_root/python_lib/helpers.py	2012-08-17 21:00:25 +0000
@@ -12,9 +12,7 @@
 from . python_nameconfig import get_data_file
 from . Builder import Builder
 
-import gettext
-from gettext import gettext as _
-gettext.textdomain('project_name')
+from locale import gettext as _
 
 def get_builder(builder_file_name):
     """Return a fully-instantiated Gtk.Builder instance from specified ui 
 
=== modified file 'data/templates/ubuntu-application/project_root/python_lib/python_nameconfig.py'
--- data/templates/ubuntu-application/project_root/python_lib/python_nameconfig.py	2012-08-17 21:00:25 +0000
+++ data/templates/ubuntu-application/project_root/python_lib/python_nameconfig.py	2012-08-17 21:00:25 +0000
@@ -19,9 +19,7 @@
 
 import os
 
-import gettext
-from gettext import gettext as _
-gettext.textdomain('project_name')
+from locale import gettext as _
 
 class project_path_not_found(Exception):
     """Raised when we can't find the project directory."""
 
=== modified file 'data/templates/ubuntu-application/project_root/setup.py'
--- data/templates/ubuntu-application/project_root/setup.py	2012-08-17 21:00:25 +0000
+++ data/templates/ubuntu-application/project_root/setup.py	2012-08-17 21:00:25 +0000
@@ -41,19 +41,49 @@
     return oldvalues
 
 
-def update_desktop_file(installdatadir, prefix):
-
-    filename = os.path.join(installdatadir, 'share', 'applications',
-                            'project_name.desktop')
+def move_desktop_file(root, target_data, prefix):
+    # The desktop file is rightly installed into install_data.  But it should
+    # always really be installed into prefix, because while we can install
+    # normal data files anywhere we want, the desktop file needs to exist in
+    # the main system to be found.  Only actually useful for /opt installs.
+
+    old_desktop_path = os.path.normpath(root + target_data +
+                                        '/share/applications')
+    old_desktop_file = old_desktop_path + '/project_name.desktop'
+    desktop_path = os.path.normpath(root + prefix + '/share/applications')
+    desktop_file = desktop_path + '/project_name.desktop'
+
+    if not os.path.exists(old_desktop_file):
+        print ("ERROR: Can't find", old_desktop_file)
+        sys.exit(1)
+    elif target_data != prefix + '/':
+        # This is an /opt install, so rename desktop file to use extras-
+        desktop_file = desktop_path + '/extras-project_name.desktop'
+        try:
+            os.makedirs(desktop_path)
+            os.rename(old_desktop_file, desktop_file)
+            os.rmdir(old_desktop_path)
+        except OSError as e:
+            print ("ERROR: Can't rename", old_desktop_file, ":", e)
+            sys.exit(1)
+
+    return desktop_file
+
+def update_desktop_file(filename, target_pkgdata, target_scripts):
+
     try:
         fin = file(filename, 'r')
         fout = file(filename + '.new', 'w')
 
-        for line in fin:            
+        for line in fin:
             if 'Icon=' in line:
-                line = "Icon=%s\n" % os.path.join(prefix, 'share',
-                                                  'project_name', 'media',
-                                                  'project_name.svg')
+                line = "Icon=%s\n" % (target_pkgdata + 'media/project_name.svg')
+            elif 'Exec=' in line:
+                cmd = line.split("=")[1].split(None, 1)
+                line = "Exec=%s" % (target_scripts + 'project_name')
+                if len(cmd) > 1:
+                    line += " %s" % cmd[1].strip()  # Add script arguments back
+                line += "\n"
             fout.write(line)
         fout.flush()
         fout.close()
@@ -63,16 +93,30 @@
         print ("ERROR: Can't find %s" % filename)
         sys.exit(1)
 
+def compile_schemas(root, target_data):
+    if target_data == '/usr/':
+        return  # /usr paths don't need this, they will be handled by dpkg
+    schemadir = os.path.normpath(root + target_data + 'share/glib-2.0/schemas')
+    if (os.path.isdir(schemadir) and
+            os.path.isfile('/usr/bin/glib-compile-schemas')):
+        os.system('/usr/bin/glib-compile-schemas "%s"' % schemadir)
+
 
 class InstallAndUpdateDataDirectory(DistUtilsExtra.auto.install_auto):
     def run(self):
         DistUtilsExtra.auto.install_auto.run(self)
 
-        values = {'__python_name_data_directory__': "'%s'" % (self.prefix + '/share/project_name/'),
+        target_data = '/' + os.path.relpath(self.install_data, self.root) + '/'
+        target_pkgdata = target_data + 'share/project_name/'
+        target_scripts = '/' + os.path.relpath(self.install_scripts, self.root) + '/'
+
+        values = {'__python_name_data_directory__': "'%s'" % (target_pkgdata),
                   '__version__': "'%s'" % self.distribution.get_version()}
         update_config(self.install_lib, values)
-        update_desktop_file(self.install_data, self.prefix)
 
+        desktop_file = move_desktop_file(self.root, target_data, self.prefix)
+        update_desktop_file(desktop_file, target_pkgdata, target_scripts)
+        compile_schemas(self.root, target_data)
 
         
 ##################################################################################
 
=== modified file 'data/templates/ubuntu-application/test/extras.sh'
--- data/templates/ubuntu-application/test/extras.sh	2012-05-25 17:48:13 +0000
+++ data/templates/ubuntu-application/test/extras.sh	2012-08-17 21:00:25 +0000
@@ -15,6 +15,8 @@
 
 quickly license GPL-3
 
+sed -i 's/Exec=.*/Exec=test-project arg1 arg2/' test-project.desktop.in
+
 quickly package --extras | sed 's/^\.\+//'
 # Ubuntu packaging created in debian/
 # Ubuntu package has been successfully created in ../test-project_0.1_all.deb
@@ -82,7 +84,7 @@
 # Name=Test Project
 # Comment=TestProject application
 # Categories=GNOME;Utility;
-# Exec=/opt/extras.ubuntu.com/test-project/bin/test-project
+# Exec=/opt/extras.ubuntu.com/test-project/bin/test-project arg1 arg2
 # Icon=/opt/extras.ubuntu.com/test-project/share/test-project/media/test-project.svg
 # Terminal=false
 # Type=Application
@@ -91,9 +93,9 @@
 # __test_project_data_directory__ = '/opt/extras.ubuntu.com/test-project/share/test-project/'
 
 grep -Rh "locale.bindtextdomain" ./opt/extras.ubuntu.com/test-project/bin/test-project
-# locale.bindtextdomain('test-project', '/opt/extras.ubuntu.com/test-project/share/locale')
+#     locale.bindtextdomain('test-project', '/opt/extras.ubuntu.com/test-project/share/locale')
 
 grep -Rh "^import gettext" ./opt/extras.ubuntu.com/test-project
 
 grep -Rh "XDG_DATA_DIRS" ./opt/extras.ubuntu.com/test-project/bin/test-project
-#     os.putenv("XDG_DATA_DIRS", "%s:%s" % ("/opt/extras.ubuntu.com/test-project/share/", os.getenv("XDG_DATA_DIRS", "")))
+#     os.putenv("XDG_DATA_DIRS", "%s:%s" % ("/opt/extras.ubuntu.com/test-project/share/", os.getenv("XDG_DATA_DIRS", "/usr/local/share/:/usr/share/")))
 
=== modified file 'data/templates/ubuntu-application/upgrade.py'
--- data/templates/ubuntu-application/upgrade.py	2012-08-17 21:00:25 +0000
+++ data/templates/ubuntu-application/upgrade.py	2012-08-17 21:00:25 +0000
@@ -199,6 +199,16 @@
 'quickly upgrade' to get rid of this message.""")
     sys.exit(0)
 
+if project_version < '12.08.1':
+    sedline = "sed -i '" + \
+              "s|^import gettext$|import locale|g;" + \
+              "s|^from gettext import |from locale import |g;" + \
+              "s|^gettext.textdomain|locale.textdomain|g;" + \
+              "'"
+    # This find only hits the main module, and that is fine, other files will
+    # be updated by normal overwriting mechanism below.
+    os.system("find %s -name '*.py' -exec %s {} \;" % (python_name, sedline))
+
 # Overwrite quickly-owned files as necessary
 if project_version < template_version:
     print _(
 
=== modified file 'data/templates/ubuntu-cli/project_root/python/__init__.py'
--- data/templates/ubuntu-cli/project_root/python/__init__.py	2011-07-26 07:30:55 +0000
+++ data/templates/ubuntu-cli/project_root/python/__init__.py	2012-08-17 21:00:25 +0000
@@ -1,9 +1,7 @@
 import logging
 import optparse
 
-import gettext
-from gettext import gettext as _
-gettext.textdomain('project_name')
+from locale import gettext as _
 
 from python_name import python_nameconfig
 
 
=== modified file 'data/templates/ubuntu-cli/project_root/python/python_nameconfig.py'
--- data/templates/ubuntu-cli/project_root/python/python_nameconfig.py	2011-03-30 17:57:06 +0000
+++ data/templates/ubuntu-cli/project_root/python/python_nameconfig.py	2012-08-17 21:00:25 +0000
@@ -22,9 +22,7 @@
 
 import os
 
-import gettext
-from gettext import gettext as _
-gettext.textdomain('project_name')
+from locale import gettext as _
 
 class project_path_not_found(Exception):
     """Raised when we can't find the project directory."""
 
=== modified file 'quickly/quicklyconfig.py'
--- quickly/quicklyconfig.py	2012-08-02 14:29:46 +0000
+++ quickly/quicklyconfig.py	2012-08-17 21:00:25 +0000
@@ -20,7 +20,7 @@
 # you're warned :)
 
 # quickly version used for project format compatibility
-__version__ = '12.08'
+__version__ = '12.08.1'
 
 # where quickly will head for quickly data (for instance, templates)
 # by default, this is ../data, relative to trunk layout
 
=== modified file 'setup.py'
--- setup.py	2012-08-02 14:29:46 +0000
+++ setup.py	2012-08-17 21:00:25 +0000
@@ -18,7 +18,7 @@
 
 
 # UPDATE VERSION WHEN NEEDED (it updates all versions needed to be updated)
-VERSION = '12.08'
+VERSION = '12.08.1'
 
 import glob
 import os