Merge lp:~mvo/snappy/snapy-out-own-xgettext into lp:~snappy-dev/snappy/snappy-moved-to-github

[Michael Vogt]

Use a native (go) xgettext to get translation support for go string of the `` type.

This is a bit of a RFC branch.

It implements a native xgettext to allow us to extract strings that use the `` style. I looked into what it would take to add that to the gettext upstream xgettext.c but that looks like it will be a lot more work. The powerful go ast package makes this pretty straightforward to implement. I send a wishlist bugreport to https://github.com/gosexy/gettext/issues/3 and intend to eventually move it upstream (if upstream is open for this).

[Leo Arias (elopio)]

I think this works for our purposes. We lose some niceties that gettext has, like optional location, sorting the messages and choosing the translators comment tag; but maybe we don't need those. Or are you planning to add more features?

There's one thing that gettext does nicely that I would miss, that's not duplicating the messages with the same string: http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-tools/tests/xgettext-7

I've a comment on the diff.

[Michael Vogt (mvo)]

Thanks for your excellent comments Leo! I think most of this is addressed now (tests are not quite where I want them to be though :/), I wonder if our version should also implement the normalization for the lines,
i.e.
 "very-long-line"
is split by gettext into
 "very"
 "long"
 "line"

[John Lenton (chipaca)]

Looks good to go; needs fixing a conflict and then set the commit message (i'll top approve now).

review: Approve

[Snappy Tarmac (snappydevtarmac)]

Attempt to merge into lp:snappy failed due to conflicts:

text conflict in po/snappy.pot

[Michael Vogt (mvo)]

review: Approve

[Snappy Tarmac (snappydevtarmac)]

The attempt to merge lp:~mvo/snappy/snapy-out-own-xgettext into lp:snappy failed. Below is the output from the failed tests.

Checking docs
Checking formatting
Installing godeps
Install golint
Obtaining dependencies
update github.com/gosexy/gettext failed; trying to fetch newer version
update github.com/mvo5/goconfigparser failed; trying to fetch newer version
github.com/gosexy/gettext now at 98b7b91596d20b96909e6b60d57411547dd9959c
update github.com/blakesmith/ar failed; trying to fetch newer version
github.com/mvo5/goconfigparser now at 26426272dda20cc76aa1fa44286dc743d2972fe8
update github.com/cheggaaa/pb failed; trying to fetch newer version
github.com/blakesmith/ar now at c9a977dd0cc1392b023382c7bfa5a22af8d3b730
update github.com/jessevdk/go-flags failed; trying to fetch newer version
github.com/cheggaaa/pb now at e8c7cc515bfde3e267957a3b110080ceed51354e
update github.com/mvo5/uboot-go failed; trying to fetch newer version
github.com/jessevdk/go-flags now at 15347ef417a300349807983f15af9e65cd2e1b3a
update golang.org/x/crypto failed; trying to fetch newer version
github.com/mvo5/uboot-go now at 361f6ebcbb54f389d15dc9faefa000e996ba3e37
update gopkg.in/check.v1 failed; trying to fetch newer version
golang.org/x/crypto now at 60052bd85f2d91293457e8811b0cf26b773de469
update gopkg.in/yaml.v2 failed; trying to fetch newer version
gopkg.in/check.v1 now at 64131543e7896d5bcc6bd5a76287eb75ea96c673
gopkg.in/yaml.v2 now at 49c95bdc21843256fb6c4e0d370a05f24a0bf213
Building
Running tests from /tmp/tmp.jixSzlUJXd/src/launchpad.net/snappy
=== RUN Test
OK: 16 passed
--- PASS: Test (0.35 seconds)
PASS
coverage: 74.8% of statements
ok launchpad.net/snappy/clickdeb 0.353s coverage: 74.8% of statements
=== RUN Test
2015/07/24 12:02:08.786595 common.go:40: PANIC can not set option description for "package name"
OK: 9 passed
--- PASS: Test (0.02 seconds)
PASS
coverage: 20.0% of statements
ok launchpad.net/snappy/cmd/snappy 0.031s coverage: 20.0% of statements
=== RUN Test
OK: 24 passed
--- PASS: Test (0.06 seconds)
PASS
coverage: 100.0% of statements
ok launchpad.net/snappy/coreconfig 0.060s coverage: 100.0% of statements
=== RUN Test
OK: 55 passed
--- PASS: Test (0.30 seconds)
PASS
coverage: 81.4% of statements
ok launchpad.net/snappy/helpers 0.307s coverage: 81.4% of statements
=== RUN Test
OK: 0 passed, 2 skipped
--- PASS: Test (0.01 seconds)
PASS
coverage: 33.3% of statements
ok launchpad.net/snappy/i18n 0.013s coverage: 33.3% of statements
=== RUN Test
OK: 11 passed
--- PASS: Test (0.00 seconds)
PASS
coverage: 93.4% of statements
ok launchpad.net/snappy/i18n/xgettext-go 0.015s coverage: 93.4% of statements
=== RUN Test
OK: 6 passed
--- PASS: Test (0.00 seconds)
PASS
coverage: 93.5% of statements
ok launchpad.net/snappy/logger 0.015s coverage: 93.5% of statements
=== RUN Test
OK: 4 passed
--- PASS: Test (0.00 seconds)
PASS
coverage: 100.0% of statements
ok launchpad.net/snappy/oauth 0.005s coverage: 100.0% of statements
=== RUN Test
OK: 39 passed
--- PASS: Test (0.35 seconds)
PASS
coverage: 79.1% of statements
ok launchpad.net/snappy/partition 0.355s coverage: 79.1% of statements
=== RUN Test
OK: 3 passed
--- PASS: Test (0.00 seconds)
PAS...

[Michael Vogt (mvo)]

review: Approve

[Snappy Tarmac (snappydevtarmac)]

There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions.

=== added directory 'i18n/xgettext-go'
=== added file 'i18n/xgettext-go/main.go'
--- i18n/xgettext-go/main.go	1970-01-01 00:00:00 +0000
+++ i18n/xgettext-go/main.go	2015-07-24 12:08:04 +0000
@@ -0,0 +1,269 @@
+package main
+
+import (
+	"fmt"
+	"go/ast"
+	"go/parser"
+	"go/token"
+	"io"
+	"io/ioutil"
+	"log"
+	"os"
+	"sort"
+	"strings"
+	"time"
+
+	"github.com/jessevdk/go-flags"
+)
+
+type msgID struct {
+	msgidPlural string
+	comment     string
+	fname       string
+	line        int
+	formatHint  string
+}
+
+var msgIDs map[string][]msgID
+
+func formatComment(com string) string {
+	out := ""
+	for _, rawline := range strings.Split(com, "\n") {
+		line := rawline
+		line = strings.TrimPrefix(line, "//")
+		line = strings.TrimPrefix(line, "/*")
+		line = strings.TrimSuffix(line, "*/")
+		line = strings.TrimSpace(line)
+		if line != "" {
+			out += fmt.Sprintf("#. %s\n", line)
+		}
+	}
+
+	return out
+}
+
+func findCommentsForTranslation(fset *token.FileSet, f *ast.File, posCall token.Position) string {
+	com := ""
+	for _, cg := range f.Comments {
+		// search for all comments in the previous line
+		for i := len(cg.List) - 1; i >= 0; i-- {
+			c := cg.List[i]
+
+			posComment := fset.Position(c.End())
+			//println(posCall.Line, posComment.Line, c.Text)
+			if posCall.Line == posComment.Line+1 {
+				posCall = posComment
+				com = fmt.Sprintf("%s\n%s", c.Text, com)
+			}
+		}
+	}
+
+	// only return if we have a matching prefix
+	formatedComment := formatComment(com)
+	needle := fmt.Sprintf("#. %s", opts.AddCommentsTag)
+	if !strings.HasPrefix(formatedComment, needle) {
+		formatedComment = ""
+	}
+
+	return formatedComment
+}
+
+func inspectNodeForTranslations(fset *token.FileSet, f *ast.File, n ast.Node) bool {
+	// FIXME: this assume we always have a "gettext.Gettext" style keyword
+	l := strings.Split(opts.Keyword, ".")
+	gettextSelector := l[0]
+	gettextFuncName := l[1]
+
+	l = strings.Split(opts.KeywordPlural, ".")
+	gettextSelectorPlural := l[0]
+	gettextFuncNamePlural := l[1]
+
+	switch x := n.(type) {
+	case *ast.CallExpr:
+		if sel, ok := x.Fun.(*ast.SelectorExpr); ok {
+			i18nStr := ""
+			i18nStrPlural := ""
+			if sel.Sel.Name == gettextFuncNamePlural && sel.X.(*ast.Ident).Name == gettextSelectorPlural {
+				i18nStr = x.Args[0].(*ast.BasicLit).Value
+				i18nStrPlural = x.Args[1].(*ast.BasicLit).Value
+			}
+
+			if sel.Sel.Name == gettextFuncName && sel.X.(*ast.Ident).Name == gettextSelector {
+				i18nStr = x.Args[0].(*ast.BasicLit).Value
+			}
+
+			formatI18nStr := func(s string) string {
+				if s == "" {
+					return ""
+				}
+				// strip " (or `)
+				s = s[1 : len(s)-1]
+				return strings.Replace(s, "\n", "\\n", -1)
+			}
+
+			// FIXME: too simplistic(?), no %% is considered
+			formatHint := ""
+			if strings.Contains(i18nStr, "%") || strings.Contains(i18nStrPlural, "%") {
+				// well, not quite correct but close enough
+				formatHint = "c-format"
+			}
+
+			if i18nStr != "" {
+				msgidStr := formatI18nStr(i18nStr)
+				posCall := fset.Position(n.Pos())
+				msgIDs[msgidStr] = append(msgIDs[msgidStr], msgID{
+					formatHint:  formatHint,
+					msgidPlural: formatI18nStr(i18nStrPlural),
+					fname:       posCall.Filename,
+					line:        posCall.Line,
+					comment:     findCommentsForTranslation(fset, f, posCall),
+				})
+			}
+		}
+	}
+
+	return true
+}
+
+func processFiles(args []string) error {
+	// go over the input files
+	msgIDs = make(map[string][]msgID)
+
+	fset := token.NewFileSet()
+	for _, fname := range args {
+		if err := processSingleGoSource(fset, fname); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func processSingleGoSource(fset *token.FileSet, fname string) error {
+	fnameContent, err := ioutil.ReadFile(fname)
+	if err != nil {
+		panic(err)
+	}
+
+	// Create the AST by parsing src.
+	f, err := parser.ParseFile(fset, fname, fnameContent, parser.ParseComments)
+	if err != nil {
+		panic(err)
+	}
+
+	ast.Inspect(f, func(n ast.Node) bool {
+		return inspectNodeForTranslations(fset, f, n)
+	})
+
+	return nil
+}
+
+var formatTime = func() string {
+	return time.Now().Format("2006-01-02 15:04-0700")
+}
+
+func writePotFile(out io.Writer) {
+
+	header := fmt.Sprintf(`# SOME DESCRIPTIVE TITLE.
+# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
+# This file is distributed under the same license as the PACKAGE package.
+# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
+#
+#, fuzzy
+msgid   ""
+msgstr  "Project-Id-Version: %s\n"
+        "Report-Msgid-Bugs-To: %s\n"
+        "POT-Creation-Date: %s\n"
+        "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
+        "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+        "Language-Team: LANGUAGE <[e-mail]>\n"
+        "Language: \n"
+        "MIME-Version: 1.0\n"
+        "Content-Type: text/plain; charset=CHARSET\n"
+        "Content-Transfer-Encoding: 8bit\n"
+
+`, opts.PackageName, opts.MsgIDBugsAddress, formatTime())
+	fmt.Fprintf(out, "%s", header)
+
+	// yes, this is the way to do it in go
+	sortedKeys := []string{}
+	for k := range msgIDs {
+		sortedKeys = append(sortedKeys, k)
+	}
+	if opts.SortOutput {
+		sort.Strings(sortedKeys)
+	}
+
+	// FIXME: use template here?
+	for _, k := range sortedKeys {
+		msgidList := msgIDs[k]
+		for _, msgid := range msgidList {
+			if opts.AddComments || opts.AddCommentsTag != "" {
+				fmt.Fprintf(out, "%s", msgid.comment)
+			}
+		}
+		if !opts.NoLocation {
+			fmt.Fprintf(out, "#:")
+			for _, msgid := range msgidList {
+				fmt.Fprintf(out, " %s:%d", msgid.fname, msgid.line)
+			}
+			fmt.Fprintf(out, "\n")
+		}
+		msgid := msgidList[0]
+		if msgid.formatHint != "" {
+			fmt.Fprintf(out, "#, %s\n", msgid.formatHint)
+		}
+		fmt.Fprintf(out, "msgid   \"%v\"\n", k)
+		if msgid.msgidPlural != "" {
+			fmt.Fprintf(out, "msgid_plural   \"%v\"\n", msgid.msgidPlural)
+			fmt.Fprintf(out, "msgstr[0]  \"\"\n")
+			fmt.Fprintf(out, "msgstr[1]  \"\"\n")
+		} else {
+			fmt.Fprintf(out, "msgstr  \"\"\n")
+		}
+		fmt.Fprintf(out, "\n")
+	}
+
+}
+
+// FIXME: this must be setable via go-flags
+var opts struct {
+	Output string `short:"o" long:"output" description:"output to specified file"`
+
+	AddComments bool `short:"c" long:"add-comments" description:"place all comment blocks preceding keyword lines in output file"`
+
+	AddCommentsTag string `long:"add-comments-tag" description:"place comment blocks starting with TAG and prceding keyword lines in output file"`
+
+	SortOutput bool `short:"s" long:"sort-output" description:"generate sorted output"`
+
+	NoLocation bool `long:"no-location" description:"do not write '#: filename:line' lines"`
+
+	MsgIDBugsAddress string `long:"msgid-bugs-address" default:"EMAIL" description:"set report address for msgid bugs"`
+
+	PackageName string `long:"package-name" description:"set package name in output"`
+
+	Keyword       string `short:"k" long:"keyword" default:"gettext.Gettext" description:"look for WORD as the keyword for singular strings"`
+	KeywordPlural string `long:"keyword-plural" default:"gettext.NGettext" description:"look for WORD as the keyword for plural strings"`
+}
+
+func main() {
+	// parse args
+	args, err := flags.ParseArgs(&opts, os.Args)
+	if err != nil {
+		log.Fatalf("ParseArgs failed %s", err)
+	}
+
+	if err := processFiles(args[1:]); err != nil {
+		log.Fatalf("processFiles failed with: %s", err)
+	}
+
+	out := os.Stdout
+	if opts.Output != "" {
+		var err error
+		out, err = os.Create(opts.Output)
+		if err != nil {
+			log.Fatalf("failed to create %s: %s", opts.Output, err)
+		}
+	}
+	writePotFile(out)
+}

=== added file 'i18n/xgettext-go/main_test.go'
--- i18n/xgettext-go/main_test.go	1970-01-01 00:00:00 +0000
+++ i18n/xgettext-go/main_test.go	2015-07-24 12:08:04 +0000
@@ -0,0 +1,392 @@
+// -*- Mode: Go; indent-tabs-mode: t -*-
+
+/*
+ * Copyright (C) 2014-2015 Canonical Ltd
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 3 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+package main
+
+import (
+	"bytes"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"testing"
+
+	. "gopkg.in/check.v1"
+)
+
+// Hook up check.v1 into the "go test" runner
+func Test(t *testing.T) { TestingT(t) }
+
+type xgettextTestSuite struct {
+}
+
+var _ = Suite(&xgettextTestSuite{})
+
+// test helper
+func makeGoSourceFile(c *C, content []byte) string {
+	fname := filepath.Join(c.MkDir(), "foo.go")
+	err := ioutil.WriteFile(fname, []byte(content), 0644)
+	c.Assert(err, IsNil)
+
+	return fname
+}
+
+func (s *xgettextTestSuite) SetUpTest(c *C) {
+	// our test defaults
+	opts.NoLocation = false
+	opts.AddCommentsTag = "TRANSLATORS:"
+	opts.Keyword = "i18n.G"
+	opts.KeywordPlural = "i18n.NG"
+	opts.SortOutput = true
+	opts.PackageName = "snappy"
+	opts.MsgIDBugsAddress = "[e-mail]"
+
+	// mock time
+	formatTime = func() string {
+		return "2015-06-30 14:48+0200"
+	}
+}
+
+func (s *xgettextTestSuite) TestFormatComment(c *C) {
+	var tests = []struct {
+		in  string
+		out string
+	}{
+		{in: "// foo ", out: "#. foo\n"},
+		{in: "/* foo */", out: "#. foo\n"},
+		{in: "/* foo\n */", out: "#. foo\n"},
+		{in: "/* foo\nbar   */", out: "#. foo\n#. bar\n"},
+	}
+
+	for _, test := range tests {
+		c.Assert(formatComment(test.in), Equals, test.out)
+	}
+}
+
+func (s *xgettextTestSuite) TestProcessFilesSimple(c *C) {
+	fname := makeGoSourceFile(c, []byte(`package main
+
+func main() {
+    // TRANSLATORS: foo comment
+    i18n.G("foo")
+}
+`))
+	err := processFiles([]string{fname})
+	c.Assert(err, IsNil)
+
+	c.Assert(msgIDs, DeepEquals, map[string][]msgID{
+		"foo": []msgID{
+			{
+				comment: "#. TRANSLATORS: foo comment\n",
+				fname:   fname,
+				line:    5,
+			},
+		},
+	})
+}
+
+func (s *xgettextTestSuite) TestProcessFilesMultiple(c *C) {
+	fname := makeGoSourceFile(c, []byte(`package main
+
+func main() {
+    // TRANSLATORS: foo comment
+    i18n.G("foo")
+
+    // TRANSLATORS: bar comment
+    i18n.G("foo")
+}
+`))
+	err := processFiles([]string{fname})
+	c.Assert(err, IsNil)
+
+	c.Assert(msgIDs, DeepEquals, map[string][]msgID{
+		"foo": []msgID{
+			{
+				comment: "#. TRANSLATORS: foo comment\n",
+				fname:   fname,
+				line:    5,
+			},
+			{
+				comment: "#. TRANSLATORS: bar comment\n",
+				fname:   fname,
+				line:    8,
+			},
+		},
+	})
+}
+
+const header = `# SOME DESCRIPTIVE TITLE.
+# Copyright (C) YEAR THE PACKAGE'S COPYRIGHT HOLDER
+# This file is distributed under the same license as the PACKAGE package.
+# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.
+#
+#, fuzzy
+msgid   ""
+msgstr  "Project-Id-Version: snappy\n"
+        "Report-Msgid-Bugs-To: [e-mail]\n"
+        "POT-Creation-Date: 2015-06-30 14:48+0200\n"
+        "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
+        "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+        "Language-Team: LANGUAGE <[e-mail]>\n"
+        "Language: \n"
+        "MIME-Version: 1.0\n"
+        "Content-Type: text/plain; charset=CHARSET\n"
+        "Content-Transfer-Encoding: 8bit\n"
+`
+
+func (s *xgettextTestSuite) TestWriteOutputSimple(c *C) {
+	msgIDs = map[string][]msgID{
+		"foo": []msgID{
+			{
+				fname:   "fname",
+				line:    2,
+				comment: "#. foo\n",
+			},
+		},
+	}
+	out := bytes.NewBuffer([]byte(""))
+	writePotFile(out)
+
+	expected := fmt.Sprintf(`%s
+#. foo
+#: fname:2
+msgid   "foo"
+msgstr  ""
+
+`, header)
+	c.Assert(out.String(), Equals, expected)
+}
+
+func (s *xgettextTestSuite) TestWriteOutputMultiple(c *C) {
+	msgIDs = map[string][]msgID{
+		"foo": []msgID{
+			{
+				fname:   "fname",
+				line:    2,
+				comment: "#. comment1\n",
+			},
+			{
+				fname:   "fname",
+				line:    4,
+				comment: "#. comment2\n",
+			},
+		},
+	}
+	out := bytes.NewBuffer([]byte(""))
+	writePotFile(out)
+
+	expected := fmt.Sprintf(`%s
+#. comment1
+#. comment2
+#: fname:2 fname:4
+msgid   "foo"
+msgstr  ""
+
+`, header)
+	c.Assert(out.String(), Equals, expected)
+}
+
+func (s *xgettextTestSuite) TestWriteOutputNoComment(c *C) {
+	msgIDs = map[string][]msgID{
+		"foo": []msgID{
+			{
+				fname: "fname",
+				line:  2,
+			},
+		},
+	}
+	out := bytes.NewBuffer([]byte(""))
+	writePotFile(out)
+
+	expected := fmt.Sprintf(`%s
+#: fname:2
+msgid   "foo"
+msgstr  ""
+
+`, header)
+	c.Assert(out.String(), Equals, expected)
+}
+
+func (s *xgettextTestSuite) TestWriteOutputNoLocation(c *C) {
+	msgIDs = map[string][]msgID{
+		"foo": []msgID{
+			{
+				fname: "fname",
+				line:  2,
+			},
+		},
+	}
+
+	opts.NoLocation = true
+	out := bytes.NewBuffer([]byte(""))
+	writePotFile(out)
+
+	expected := fmt.Sprintf(`%s
+msgid   "foo"
+msgstr  ""
+
+`, header)
+	c.Assert(out.String(), Equals, expected)
+}
+
+func (s *xgettextTestSuite) TestWriteOutputFormatHint(c *C) {
+	msgIDs = map[string][]msgID{
+		"foo": []msgID{
+			{
+				fname:      "fname",
+				line:       2,
+				formatHint: "c-format",
+			},
+		},
+	}
+
+	out := bytes.NewBuffer([]byte(""))
+	writePotFile(out)
+
+	expected := fmt.Sprintf(`%s
+#: fname:2
+#, c-format
+msgid   "foo"
+msgstr  ""
+
+`, header)
+	c.Assert(out.String(), Equals, expected)
+}
+
+func (s *xgettextTestSuite) TestWriteOutputPlural(c *C) {
+	msgIDs = map[string][]msgID{
+		"foo": []msgID{
+			{
+				msgidPlural: "plural",
+				fname:       "fname",
+				line:        2,
+			},
+		},
+	}
+
+	out := bytes.NewBuffer([]byte(""))
+	writePotFile(out)
+
+	expected := fmt.Sprintf(`%s
+#: fname:2
+msgid   "foo"
+msgid_plural   "plural"
+msgstr[0]  ""
+msgstr[1]  ""
+
+`, header)
+	c.Assert(out.String(), Equals, expected)
+}
+
+func (s *xgettextTestSuite) TestWriteOutputSorted(c *C) {
+	msgIDs = map[string][]msgID{
+		"aaa": []msgID{
+			{
+				fname: "fname",
+				line:  2,
+			},
+		},
+		"zzz": []msgID{
+			{
+				fname: "fname",
+				line:  2,
+			},
+		},
+	}
+
+	opts.SortOutput = true
+	// we need to run this a bunch of times as the ordering might
+	// be right by pure chance
+	for i := 0; i < 10; i++ {
+		out := bytes.NewBuffer([]byte(""))
+		writePotFile(out)
+
+		expected := fmt.Sprintf(`%s
+#: fname:2
+msgid   "aaa"
+msgstr  ""
+
+#: fname:2
+msgid   "zzz"
+msgstr  ""
+
+`, header)
+		c.Assert(out.String(), Equals, expected)
+	}
+}
+
+func (s *xgettextTestSuite) TestIntegration(c *C) {
+	fname := makeGoSourceFile(c, []byte(`package main
+
+func main() {
+    // TRANSLATORS: foo comment
+    //              with multiple lines
+    i18n.G("foo")
+
+    // this comment has no translators tag
+    i18n.G("abc")
+
+    // TRANSLATORS: plural
+    i18n.NG("singular", "plural", 99)
+
+    i18n.G("zz %s")
+}
+`))
+
+	// a real integration test :)
+	outName := filepath.Join(c.MkDir(), "snappy.pot")
+	os.Args = []string{"test-binary",
+		"--output", outName,
+		"--keyword", "i18n.G",
+		"--keyword-plural", "i18n.NG",
+		"--msgid-bugs-address", "[e-mail]",
+		"--package-name", "snappy",
+		fname,
+	}
+	main()
+
+	// verify its what we expect
+	got, err := ioutil.ReadFile(outName)
+	c.Assert(err, IsNil)
+	expected := fmt.Sprintf(`%s
+#: %[2]s:9
+msgid   "abc"
+msgstr  ""
+
+#. TRANSLATORS: foo comment
+#. with multiple lines
+#: %[2]s:6
+msgid   "foo"
+msgstr  ""
+
+#. TRANSLATORS: plural
+#: %[2]s:12
+msgid   "singular"
+msgid_plural   "plural"
+msgstr[0]  ""
+msgstr[1]  ""
+
+#: %[2]s:14
+#, c-format
+msgid   "zz %%s"
+msgstr  ""
+
+`, header, fname)
+	c.Assert(string(got), Equals, expected)
+}

=== modified file 'po/snappy.pot'
--- po/snappy.pot	2015-07-08 08:00:32 +0000
+++ po/snappy.pot	2015-07-24 12:08:04 +0000
@@ -7,7 +7,7 @@
 msgid   ""
 msgstr  "Project-Id-Version: snappy\n"
         "Report-Msgid-Bugs-To: [e-mail]\n"
-        "POT-Creation-Date: 2015-07-08 10:00+0200\n"
+        "POT-Creation-Date: 2015-07-24 14:07+0200\n"
         "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
         "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
         "Language-Team: LANGUAGE <[e-mail]>\n"
@@ -16,16 +16,16 @@
         "Content-Type: text/plain; charset=CHARSET\n"
         "Content-Transfer-Encoding: 8bit\n"
 
+#. TRANSLATORS: the %s is a pkgname, the second a comma separated list of paths
+#, c-format
+msgid   "%s: %s\n"
+msgstr  ""
+
 #. TRANSLATORS: the %s stand for "name", "version", "description"
 #, c-format
 msgid   "%s\t%s\t%s (forks not shown: %d)\t"
 msgstr  ""
 
-#. TRANSLATORS: the %s is a pkgname, the second a comma separated list of paths
-#, c-format
-msgid   "%s: %s\n"
-msgstr  ""
-
 #. TRANSLATORS: the first %s is a pkgname, the second %s is a path
 #, c-format
 msgid   "'%s' is no longer allowed to access '%s'\n"
@@ -52,10 +52,10 @@
 msgid   "2fa code: "
 msgstr  ""
 
-msgid   "Allows rollback of a snap to a previous installed version. Without "
-        "any arguments, the previous installed version is selected. It is "
-        "also possible to specify the version to rollback to as a additional "
-        "argument.\n"
+msgid   "A concise summary of key attributes of the snappy system, such as the release and channel.\n\nThe verbose output includes the specific version information for the factory image, the running image and the image that will be run on reboot, together with a list of the available channels for this image.\n\nProviding a package name will display information about a specific installed package.\n\nThe verbose version of the info command for a package will also tell you the available channels for that package, when it was installed for the first time, disk space utilization, and in the case of frameworks, which apps are able to use the framework."
+msgstr  ""
+
+msgid   "Allows rollback of a snap to a previous installed version. Without any arguments, the previous installed version is selected. It is also possible to specify the version to rollback to as a additional argument.\n"
 msgstr  ""
 
 msgid   "Assign a hardware device to a package"
@@ -68,13 +68,11 @@
 msgstr  ""
 
 #. TRANSLATORS: the first %q is the file that can not be read and %v is the error message
+#, c-format
 msgid   "Can't read hook file %q: %v"
 msgstr  ""
 
-msgid   "Configures a package. The configuration is a YAML file, provided in "
-        "the specified file which can be \"-\" for stdin. Output of the "
-        "command is the current configuration, so running this command with "
-        "no input file provides a snapshot of the app's current config."
+msgid   "Configures a package. The configuration is a YAML file, provided in the specified file which can be \"-\" for stdin. Output of the command is the current configuration, so running this command with no input file provides a snapshot of the app's current config."
 msgstr  ""
 
 msgid   "Creates a snap package and if available, runs the review scripts."
@@ -143,6 +141,9 @@
 msgid   "Provide information about a specific installed package"
 msgstr  ""
 
+msgid   "Provides a list of all active components installed on a snappy system.\n\nIf requested, the command will find out if there are updates for any of the components and indicate that by appending a * to the date. This will be slower as it requires a round trip to the app store on the network.\n\nThe developer information refers to non-mainline versions of a package (much like PPAs in deb-based Ubuntu). If the package is the primary version of that package in Ubuntu then the developer info is not shown. This allows one to identify packages which have custom, non-standard versions installed. As a special case, the “sideload” developer refers to packages installed manually on the system.\n\nWhen a verbose listing is requested, information about the channel used is displayed; which is one of alpha, beta, rc or stable, and all fields are fully expanded too. In some cases, older (inactive) versions of snappy packages will be installed, these will be shown in the verbose output and the active version indicated with a * appended to the name of the component."
+msgstr  ""
+
 msgid   "Provides more detailed information"
 msgstr  ""
 
@@ -182,6 +183,9 @@
 msgid   "Remove all the data from the listed packages"
 msgstr  ""
 
+msgid   "Remove all the data from the listed packages. Normally this is used for packages that have been removed and attempting to purge data for an installed package will result in an error. The --installed option  overrides that and enables the administrator to purge all data for an installed package (effectively resetting the package completely)."
+msgstr  ""
+
 msgid   "Remove hardware from a specific installed package"
 msgstr  ""
 
@@ -205,6 +209,9 @@
 msgid   "Set properties of system or package"
 msgstr  ""
 
+msgid   "Set properties of system or package\n\nSupported properties are:\n  active=VERSION\n\nExample:\n  set hello-world active=1.0\n"
+msgstr  ""
+
 #. TRANSLATORS: the first %s is a pkgname, the second %s is the new version
 #, c-format
 msgid   "Setting %s to version %s\n"
@@ -222,6 +229,9 @@
 msgid   "Specify an alternate output directory for the resulting package"
 msgstr  ""
 
+msgid   "The "versions" command is no longer available.\n\nPlease use the "list" command instead to see what is installed.\nThe "list -u" (or "list --updates") will show you the available updates\nand "list -v" (or "list --verbose") will show all installed versions.\n"
+msgstr  ""
+
 msgid   "The Package to install (name or path)"
 msgstr  ""
 
@@ -240,8 +250,7 @@
 msgid   "The version to rollback to"
 msgstr  ""
 
-msgid   "This command adds access to a specific hardware device (e.g. /dev/"
-        "ttyUSB0) for an installed package."
+msgid   "This command adds access to a specific hardware device (e.g. /dev/ttyUSB0) for an installed package."
 msgstr  ""
 
 msgid   "This command is no longer available, please use the \"list\" command"
@@ -253,8 +262,7 @@
 msgid   "This command logs the given username into the store"
 msgstr  ""
 
-msgid   "This command removes access of a specific hardware device (e.g. /dev/"
-        "ttyUSB0) for an installed package."
+msgid   "This command removes access of a specific hardware device (e.g. /dev/ttyUSB0) for an installed package."
 msgstr  ""
 
 msgid   "Unassign a hardware device to a package"
@@ -281,6 +289,7 @@
 msgstr  ""
 
 #. TRANSLATORS: the %s is a size
+#, c-format
 msgid   "binary-size: %v\n"
 msgstr  ""
 
@@ -312,9 +321,7 @@
 msgid   "release: %s\n"
 msgstr  ""
 
-msgid   "snappy autopilot triggered a reboot to boot into an up to date "
-        "system -- temprorarily disable the reboot by running 'sudo shutdown -"
-        "c'"
+msgid   "snappy autopilot triggered a reboot to boot into an up to date system -- temprorarily disable the reboot by running 'sudo shutdown -c'"
 msgstr  ""
 
 #. TRANSLATORS: the %s is a date
@@ -327,5 +334,3 @@
 msgid   "version: %s\n"
 msgstr  ""
 
-msgid   "versions"
-msgstr  ""

=== modified file 'update-pot'
--- update-pot	2015-07-02 17:16:11 +0000
+++ update-pot	2015-07-24 12:08:04 +0000
@@ -10,12 +10,26 @@
 	OUTPUT="$1"
 fi
 
-xgettext -d snappy -o "$OUTPUT" --c++ --from-code=UTF-8 \
-	--indent --add-comments=TRANSLATORS: --no-location --sort-output \
-	--package-name=snappy \
-	--msgid-bugs-address=[e-mail] \
-	--keyword=NG:1,2 --keyword=G \
-	$HERE/*/*.go $HERE/cmd/*/*.go
+# ensure we have it
+go install launchpad.net/snappy/i18n/xgettext-go
+
+$GOPATH/bin/xgettext-go \
+    -o "$OUTPUT" \
+    --add-comments-tag=TRANSLATORS: \
+    --no-location \
+    --sort-output \
+    --package-name=snappy\
+    --msgid-bugs-address=[e-mail] \
+    --keyword=i18n.G \
+    --keyword-plural=i18n.DG \
+    $HERE/*/*.go $HERE/cmd/*/*.go
+
+#xgettext -d snappy -o "$OUTPUT" --c++ --from-code=UTF-8 \
+#	--indent --add-comments=TRANSLATORS: --no-location --sort-output \
+#	--package-name=snappy \
+#	--msgid-bugs-address=[e-mail] \
+#	--keyword=NG:1,2 --keyword=G \
+#	$HERE/*/*.go $HERE/cmd/*/*.go
 
 # language packs
 for p in ${HERE}/po/*.po; do