Merge lp:~widelands-dev/widelands/translation_stats into lp:widelands

[GunChleoc]

Show translation stats next to the language selection menu and invite translators if a translation is incomplete, with the help of the Translate Toolkit.

- Added a new utils script "update_translation_stats.py" that will write translation statistics to data/i18n/translation_stats.conf.
  This script is now called on every translation pull.
- Added translation stats + invitation to translators to Options.

Some translations have very little work accomplished, so I thought we'd better tell the user so. This also gives us an opportunity to invite translators.

[bunnybot (widelandsofficial)]

Continuous integration builds have changed state:

Travis build 2702. State: failed. Details: https://travis-ci.org/widelands/widelands/builds/285758712.
Appveyor build 2517. State: success. Details: https://ci.appveyor.com/project/widelands-dev/widelands/build/_widelands_dev_widelands_translation_stats-2517.

[kaputtnik (franku)]

This is good approach to get more translators, i think.

Translation stats: I guess the count of 'total' is ever the same, because English is the base language. Is it needed to store this value for each language?

The text for RTL languages are messed up somehow. I guess this will be fixed if the string is translated:

"is 0% complete. If you whish to help us translate, please visit [language name] The translation intowidelands.org/wiki/TranlatingWidelands"

But if some one is willing to help translate, he should know English. So i would suggest to keep the English sentence here. Anyway adding "https://" to the underlined words would be helpful to show that this is an internet address.

[SirVer (sirver)]

code lgtm, a few nits.

not tested.

review: Approve

[GunChleoc (gunchleoc)]

Answers to SirVer's comments in-line.

> Translation stats: I guess the count of 'total' is ever the same, because English is the base language. Is it needed to store this value for each language?

You are right, I should refactor that.

> The text for RTL languages are messed up somehow. I guess this will be fixed if the string is translated: <snip>

This is an issue with the BiDi support in the font renderer, so I'd rather not change this just to hide the bug. And there are actually translators who don't speak English that well - I was recently at a localizer's meeting and having a language other than English in the translation tool's UI was a desired feature for some of them.

[SirVer (sirver)]

more comments inline.

[GunChleoc (gunchleoc)]

I just pushed another update, where I write the total only once as suggested by Kaputtnik. I still need to clean up the Python script incl. the regex.

[GunChleoc (gunchleoc)]

Thanks for the comment, I mixed something up there. Will replace the vector with a map and get rid of the std::sort call.

[GunChleoc (gunchleoc)]

This should be ready now.

I left the data container in Python as it is with the total for each locale, because the entires come in per directory rather than per locale. So, I sacrificed some memory in favour of code readability + performance.

review: Needs Resubmitting

[kaputtnik (franku)]

I have run the update_translation_stats.py:

Looks like this could be run only with python2. Maybe add a comment?

I got a list out of range error:

$ > python2 utils/update_translation_stats.py
Fetching translation stats .......................

Locale Total Translated
------ ----- ----------
Something went wrong:
Traceback (most recent call last):
  File "utils/update_translation_stats.py", line 118, in main
    result = generate_translation_stats(po_dir, output_file)
  File "utils/update_translation_stats.py", line 97, in generate_translation_stats
    result = result + 'total=' + str(locale_stats[locale_stats.keys()[0]].total) + '\n\n'
IndexError: list index out of range

Do i miss something?

[bunnybot (widelandsofficial)]

Continuous integration builds have changed state:

Travis build 2722. State: failed. Details: https://travis-ci.org/widelands/widelands/builds/295046876.
Appveyor build 2534. State: failed. Details: https://ci.appveyor.com/project/widelands-dev/widelands/build/_widelands_dev_widelands_translation_stats-2534.

[GunChleoc (gunchleoc)]

Thanks for pointing that out - there are multiple things in that script that won't work with Python 3 and a fix isn't trivial, so I've changed the Python header.

[kaputtnik (franku)]

The regex doesn't match here and the match is always 'None', resulting in the List out of range error.

Don't know how to fix the regex... but maybe adding an else clause? See diff comment.

review: Needs Fixing

[GunChleoc (gunchleoc)]

It fails for me with Python3 too, but not with Python2. Good idea about the error handling though. I've added some :)

[kaputtnik (franku)]

Just want to mention that i did run the file with python2 :-) This is what i get now:

$:> python2 utils/update_translation_stats.py
Fetching translation stats .
ERROR: Invalid line in pocount output:
/home/kaputtnik/Quellcode/widelands-repo/translation_stats/po/map_the_green_plateau.wmf/la.po source words: total: 438 | 438t 0f 0u | 100.0%t 0.0%f 0.0%u

[bunnybot (widelandsofficial)]

Continuous integration builds have changed state:

Travis build 2729. State: passed. Details: https://travis-ci.org/widelands/widelands/builds/295908441.
Appveyor build 2541. State: failed. Details: https://ci.appveyor.com/project/widelands-dev/widelands/build/_widelands_dev_widelands_translation_stats-2541.

[GunChleoc (gunchleoc)]

Looks like your version of pocount produces slightly different output. Can you please try again and double-check if the resulting conf looks OK?

[kaputtnik (franku)]

My version of translate-toolkit is 2.2.5-1.

Pulled your changes but no change:

$:> python2 utils/update_translation_stats.py
Fetching translation stats .
ERROR: Invalid line in pocount output:
/home/kaputtnik/Quellcode/widelands-repo/translation_stats/po/map_the_green_plateau.wmf/la.po source words: total: 438 | 438t 0f 0u | 100.0%t 0.0%f 0.0%u

I have tried to print the line showing escape sequences by adjusting line 106 like "...line.encode('string_escape'))" and the result is:

$:> python2 utils/update_translation_stats.py
Fetching translation stats .
ERROR: Invalid line in pocount output:
\x1b[95m/home/kaputtnik/Quellcode/widelands-repo/translation_stats/po/map_the_green_plateau.wmf/la.po\x1b[0m source words: total: 438\t| \x1b[92m438\x1b[0mt\t\x1b[93m0\x1b[0mf\t\x1b[91m0\x1b[0mu\t| \x1b[92m100.0%\x1b[0mt\t\x1b[93m0.0%\x1b[0mf\t\x1b[91m0.0%\x1b[0mu

So tabs are there plus escape sequences showing colored output.

[GunChleoc (gunchleoc)]

I am now testing for non-Word characters rather than tabs or spaces. Does that work for you?

[bunnybot (widelandsofficial)]

Continuous integration builds have changed state:

Travis build 2743. State: errored. Details: https://travis-ci.org/widelands/widelands/builds/296669731.
Appveyor build 2555. State: failed. Details: https://ci.appveyor.com/project/widelands-dev/widelands/build/_widelands_dev_widelands_translation_stats-2555.

[kaputtnik (franku)]

No, it doesn't work :(

Whereas the version of pocount used by me has an option '--no-color', it is maybe better to use another pocount output format like --csv in combination with python csv module?

https://docs.python.org/2/library/csv.html?highlight=csv#module-csv

I guess the csv format provided by pocount will not change when switching the version of translate-toolkit. So the script might get more stable without using a fragile regexp.

For comparison her is the output with option --csv on my machine:

$:> pocount --csv po/map_the_green_plateau.wmf/la.po
Filename, Translated Messages, Translated Source Words, Translated Target Words, Fuzzy Messages, Fuzzy Source Words, Untranslated Messages, Untranslated Source Words, Total Message, Total Source Words, Review Messages, Review Source Words
po/map_the_green_plateau.wmf/la.po, 25, 438, 283, 0, 0, 0, 0, 25, 438

[GunChleoc (gunchleoc)]

Much better! No idea why I didn't use the csv format in the first place.

[kaputtnik (franku)]

Now its working :-)

Could you please add 'https://' to the underlined link for "...please visit widelands.org/wiki/TranslatinWidelands"

[GunChleoc (gunchleoc)]

I actually removed that https on purpose, it adds visual clutter and makes the string too long to fir.

[bunnybot (widelandsofficial)]

Continuous integration builds have changed state:

Travis build 2746. State: errored. Details: https://travis-ci.org/widelands/widelands/builds/296921592.
Appveyor build 2558. State: success. Details: https://ci.appveyor.com/project/widelands-dev/widelands/build/_widelands_dev_widelands_translation_stats-2558.

[GunChleoc (gunchleoc)]

I gave it another try, adding https:// results in a string truncation, so I'm leaving it out.

I found another bug where I was parsing the wrong column, so I've added some error handling and am parsing the header info.

Thanks for all your help in debugging this thing!

@bunnybot merge

[bunnybot (widelandsofficial)]

Continuous integration builds have changed state:

Travis build 2754. State: passed. Details: https://travis-ci.org/widelands/widelands/builds/297152789.
Appveyor build 2566. State: success. Details: https://ci.appveyor.com/project/widelands-dev/widelands/build/_widelands_dev_widelands_translation_stats-2566.

=== added file 'data/i18n/translation_stats.conf'
--- data/i18n/translation_stats.conf	1970-01-01 00:00:00 +0000
+++ data/i18n/translation_stats.conf	2017-11-04 09:41:54 +0000
@@ -0,0 +1,182 @@
+[global]
+total=44699
+
+[ar]
+translated=8657
+
+[ast]
+translated=469
+
+[bg]
+translated=44699
+
+[br]
+translated=422
+
+[ca]
+translated=44689
+
+[cs]
+translated=43115
+
+[da]
+translated=42285
+
+[de]
+translated=44689
+
+[el]
+translated=491
+
+[en_CA]
+translated=546
+
+[en_GB]
+translated=41937
+
+[en_US]
+translated=455
+
+[eo]
+translated=4713
+
+[es]
+translated=44288
+
+[et]
+translated=575
+
+[eu]
+translated=1338
+
+[fa]
+translated=115
+
+[fi]
+translated=44689
+
+[fr]
+translated=44699
+
+[ga]
+translated=0
+
+[gd]
+translated=42984
+
+[gl]
+translated=10382
+
+[he]
+translated=420
+
+[hi]
+translated=26
+
+[hr]
+translated=3172
+
+[hu]
+translated=33179
+
+[ia]
+translated=126
+
+[id]
+translated=31
+
+[it]
+translated=36868
+
+[ja]
+translated=6944
+
+[jv]
+translated=22
+
+[ka]
+translated=4
+
+[ko]
+translated=24920
+
+[krl]
+translated=103
+
+[la]
+translated=3994
+
+[lt]
+translated=202
+
+[mr]
+translated=7
+
+[ms]
+translated=10192
+
+[my]
+translated=30
+
+[nb]
+translated=9055
+
+[nds]
+translated=874
+
+[nl]
+translated=33794
+
+[nn]
+translated=2845
+
+[oc]
+translated=193
+
+[pl]
+translated=41731
+
+[pt]
+translated=32884
+
+[pt_BR]
+translated=14496
+
+[ro]
+translated=1105
+
+[ru]
+translated=43920
+
+[rw]
+translated=48
+
+[si]
+translated=283
+
+[sk]
+translated=39387
+
+[sl]
+translated=1010
+
+[sr]
+translated=177
+
+[sv]
+translated=41420
+
+[tr]
+translated=2030
+
+[uk]
+translated=4161
+
+[vi]
+translated=1197
+
+[zh_CN]
+translated=2202
+
+[zh_TW]
+translated=641

=== modified file 'src/ui_fsmenu/CMakeLists.txt'
--- src/ui_fsmenu/CMakeLists.txt	2017-05-14 20:06:48 +0000
+++ src/ui_fsmenu/CMakeLists.txt	2017-11-04 09:41:54 +0000
@@ -10,6 +10,7 @@
     graphic_fonthandler
     graphic_text
     graphic_text_constants
+    graphic_text_layout
     helper
     io_filesystem
     logic_constants

=== modified file 'src/ui_fsmenu/options.cc'
--- src/ui_fsmenu/options.cc	2017-09-11 16:59:41 +0000
+++ src/ui_fsmenu/options.cc	2017-11-04 09:41:54 +0000
@@ -36,6 +36,7 @@
 #include "graphic/text/bidi.h"
 #include "graphic/text/font_set.h"
 #include "graphic/text_constants.h"
+#include "graphic/text_layout.h"
 #include "helper.h"
 #include "io/filesystem/layered_filesystem.h"
 #include "logic/constants.h"
@@ -47,23 +48,6 @@
 
 namespace {
 
-// Data model for the entries in the language selection list.
-struct LanguageEntry {
-	LanguageEntry(const std::string& init_localename,
-	              const std::string& init_descname,
-	              const std::string& init_sortname)
-	   : localename(init_localename), descname(init_descname), sortname(init_sortname) {
-	}
-
-	bool operator<(const LanguageEntry& other) const {
-		return sortname < other.sortname;
-	}
-
-	std::string localename;  // ISO code for the locale
-	std::string descname;    // Native language name
-	std::string sortname;    // ASCII Language name used for sorting
-};
-
 // Locale identifiers can look like this: [e-mail]-8
 // The contents of 'selected_locale' will be changed to match the 'current_locale'
 void find_selected_locale(std::string* selected_locale, const std::string& current_locale) {
@@ -120,21 +104,22 @@
      // Tabs
      tabs_(this, g_gr->images().get("images/ui_basic/but1.png"), UI::TabPanel::Type::kBorder),
 
-     box_interface_(&tabs_, 0, 0, UI::Box::Vertical, 0, 0, padding_),
+     box_interface_(&tabs_, 0, 0, UI::Box::Horizontal, 0, 0, padding_),
+     box_interface_left_(&box_interface_, 0, 0, UI::Box::Vertical, 0, 0, padding_),
      box_windows_(&tabs_, 0, 0, UI::Box::Vertical, 0, 0, padding_),
      box_sound_(&tabs_, 0, 0, UI::Box::Vertical, 0, 0, padding_),
      box_saving_(&tabs_, 0, 0, UI::Box::Vertical, 0, 0, padding_),
      box_game_(&tabs_, 0, 0, UI::Box::Vertical, 0, 0, padding_),
 
      // Interface options
-     language_dropdown_(&box_interface_,
+     language_dropdown_(&box_interface_left_,
                         0,
                         0,
                         100,  // 100 is arbitrary, will be resized in layout().
                         100,  // 100 is arbitrary, will be resized in layout().
                         24,
                         _("Language")),
-     resolution_dropdown_(&box_interface_,
+     resolution_dropdown_(&box_interface_left_,
                           0,
                           0,
                           100,  // 100 is arbitrary, will be resized in layout().
@@ -142,10 +127,10 @@
                           24,
                           _("In-game resolution")),
 
-     fullscreen_(&box_interface_, Vector2i::zero(), _("Fullscreen"), "", 0),
-     inputgrab_(&box_interface_, Vector2i::zero(), _("Grab Input"), "", 0),
-
-     sb_maxfps_(&box_interface_, 0, 0, 0, 0, opt.maxfps, 0, 99, _("Maximum FPS:")),
+     fullscreen_(&box_interface_left_, Vector2i::zero(), _("Fullscreen"), "", 0),
+     inputgrab_(&box_interface_left_, Vector2i::zero(), _("Grab Input"), "", 0),
+     sb_maxfps_(&box_interface_left_, 0, 0, 0, 0, opt.maxfps, 0, 99, _("Maximum FPS:")),
+     translation_info_(&box_interface_, 0, 0, 100, 100),
 
      // Windows options
      snap_win_overlap_only_(
@@ -232,6 +217,7 @@
      os_(opt) {
 	// Set up UI Elements
 	title_.set_fontsize(UI_FONT_SIZE_BIG);
+	translation_info_.force_new_renderer();
 
 	// Buttons
 	button_box_.add(UI::g_fh1->fontset()->is_rtl() ? &ok_ : &cancel_);
@@ -252,18 +238,14 @@
 		tabs_.activate(os_.active_tab);
 	}
 
-	box_interface_.set_size(tabs_.get_inner_w(), tabs_.get_inner_h());
-	box_windows_.set_size(tabs_.get_inner_w(), tabs_.get_inner_h());
-	box_sound_.set_size(tabs_.get_inner_w(), tabs_.get_inner_h());
-	box_saving_.set_size(tabs_.get_inner_w(), tabs_.get_inner_h());
-	box_game_.set_size(tabs_.get_inner_w(), tabs_.get_inner_h());
-
 	// Interface
-	box_interface_.add(&language_dropdown_);
-	box_interface_.add(&resolution_dropdown_);
-	box_interface_.add(&fullscreen_);
-	box_interface_.add(&inputgrab_);
-	box_interface_.add(&sb_maxfps_);
+	box_interface_.add(&box_interface_left_);
+	box_interface_.add(&translation_info_, UI::Box::Resizing::kExpandBoth);
+	box_interface_left_.add(&language_dropdown_);
+	box_interface_left_.add(&resolution_dropdown_);
+	box_interface_left_.add(&fullscreen_);
+	box_interface_left_.add(&inputgrab_);
+	box_interface_left_.add(&sb_maxfps_);
 
 	// Windows
 	box_windows_.add(&snap_win_overlap_only_);
@@ -290,6 +272,8 @@
 	box_game_.add(&single_watchwin_);
 
 	// Bind actions
+	language_dropdown_.selected.connect(
+	   boost::bind(&FullscreenMenuOptions::update_language_stats, this, false));
 	cancel_.sigclicked.connect(boost::bind(&FullscreenMenuOptions::clicked_back, this));
 	apply_.sigclicked.connect(boost::bind(&FullscreenMenuOptions::clicked_apply, this));
 	ok_.sigclicked.connect(boost::bind(&FullscreenMenuOptions::clicked_ok, this));
@@ -360,6 +344,7 @@
 
 	// Language options
 	add_languages_to_list(opt.language);
+	update_language_stats(true);
 	layout();
 }
 
@@ -369,8 +354,7 @@
 	butw_ = get_w() / 5;
 	buth_ = get_h() * 9 / 200;
 	hmargin_ = get_w() * 19 / 200;
-	tab_panel_width_ = get_inner_w() - 2 * hmargin_;
-	column_width_ = tab_panel_width_ - padding_;
+	int tab_panel_width = get_inner_w() - 2 * hmargin_;
 	tab_panel_y_ = get_h() * 14 / 100;
 
 	// Title
@@ -382,48 +366,53 @@
 	apply_.set_desired_size(butw_, buth_);
 	ok_.set_desired_size(butw_, buth_);
 	button_box_.set_pos(Vector2i(hmargin_ + butw_ / 3, get_inner_h() - hmargin_));
-	button_box_.set_size(tab_panel_width_ - 2 * butw_ / 3, buth_);
+	button_box_.set_size(tab_panel_width - 2 * butw_ / 3, buth_);
 
 	// Tabs
 	tabs_.set_pos(Vector2i(hmargin_, tab_panel_y_));
-	tabs_.set_size(tab_panel_width_, get_inner_h() - tab_panel_y_ - buth_ - hmargin_);
+	tabs_.set_size(tab_panel_width, get_inner_h() - tab_panel_y_ - buth_ - hmargin_);
+
+	tab_panel_width -= padding_;
+	const int column_width = tab_panel_width / 2;
 
 	// Interface
-	language_dropdown_.set_desired_size(column_width_ / 2, language_dropdown_.get_h());
+	box_interface_left_.set_desired_size(column_width + padding_, tabs_.get_inner_h());
+	box_interface_.set_size(tabs_.get_inner_w(), tabs_.get_inner_h());
+	language_dropdown_.set_desired_size(column_width, language_dropdown_.get_h());
 	language_dropdown_.set_height(tabs_.get_h() - language_dropdown_.get_y() - buth_ - 3 * padding_);
-	resolution_dropdown_.set_desired_size(column_width_ / 2, resolution_dropdown_.get_h());
+	resolution_dropdown_.set_desired_size(column_width, resolution_dropdown_.get_h());
 	resolution_dropdown_.set_height(tabs_.get_h() - resolution_dropdown_.get_y() - buth_ -
 	                                3 * padding_);
 
-	fullscreen_.set_desired_size(column_width_, fullscreen_.get_h());
-	inputgrab_.set_desired_size(column_width_, inputgrab_.get_h());
-	sb_maxfps_.set_unit_width(column_width_ / 4);
-	sb_maxfps_.set_desired_size(column_width_ / 2, sb_maxfps_.get_h());
+	fullscreen_.set_desired_size(column_width, fullscreen_.get_h());
+	inputgrab_.set_desired_size(column_width, inputgrab_.get_h());
+	sb_maxfps_.set_unit_width(column_width / 2);
+	sb_maxfps_.set_desired_size(column_width, sb_maxfps_.get_h());
 
 	// Windows options
-	snap_win_overlap_only_.set_desired_size(column_width_, snap_win_overlap_only_.get_h());
-	dock_windows_to_edges_.set_desired_size(column_width_, dock_windows_to_edges_.get_h());
-	animate_map_panning_.set_desired_size(column_width_, animate_map_panning_.get_h());
+	snap_win_overlap_only_.set_desired_size(tab_panel_width, snap_win_overlap_only_.get_h());
+	dock_windows_to_edges_.set_desired_size(tab_panel_width, dock_windows_to_edges_.get_h());
+	animate_map_panning_.set_desired_size(tab_panel_width, animate_map_panning_.get_h());
 	sb_dis_panel_.set_unit_width(200);
-	sb_dis_panel_.set_desired_size(column_width_, sb_dis_panel_.get_h());
+	sb_dis_panel_.set_desired_size(tab_panel_width, sb_dis_panel_.get_h());
 	sb_dis_border_.set_unit_width(200);
-	sb_dis_border_.set_desired_size(column_width_, sb_dis_border_.get_h());
+	sb_dis_border_.set_desired_size(tab_panel_width, sb_dis_border_.get_h());
 
 	// Sound options
-	music_.set_desired_size(column_width_, music_.get_h());
-	fx_.set_desired_size(column_width_, fx_.get_h());
-	message_sound_.set_desired_size(column_width_, message_sound_.get_h());
+	music_.set_desired_size(tab_panel_width, music_.get_h());
+	fx_.set_desired_size(tab_panel_width, fx_.get_h());
+	message_sound_.set_desired_size(tab_panel_width, message_sound_.get_h());
 
 	// Saving options
 	sb_autosave_.set_unit_width(250);
-	sb_autosave_.set_desired_size(column_width_, sb_autosave_.get_h());
+	sb_autosave_.set_desired_size(tab_panel_width, sb_autosave_.get_h());
 	sb_rolling_autosave_.set_unit_width(250);
-	sb_rolling_autosave_.set_desired_size(column_width_, sb_rolling_autosave_.get_h());
-	zip_.set_desired_size(column_width_, zip_.get_h());
-	write_syncstreams_.set_desired_size(column_width_, write_syncstreams_.get_h());
+	sb_rolling_autosave_.set_desired_size(tab_panel_width, sb_rolling_autosave_.get_h());
+	zip_.set_desired_size(tab_panel_width, zip_.get_h());
+	write_syncstreams_.set_desired_size(tab_panel_width, write_syncstreams_.get_h());
 
 	// Game options
-	transparent_chat_.set_desired_size(column_width_, transparent_chat_.get_h());
+	transparent_chat_.set_desired_size(tab_panel_width, transparent_chat_.get_h());
 }
 
 void FullscreenMenuOptions::add_languages_to_list(const std::string& current_locale) {
@@ -432,8 +421,8 @@
 	language_dropdown_.add(_("Try system language"), "", nullptr, current_locale == "");
 	language_dropdown_.add("English", "en", nullptr, current_locale == "en");
 
-	// Add translation directories to the list
-	std::vector<LanguageEntry> entries;
+	// Add translation directories to the list. Using the LanguageEntries' sortnames as a key for getting a sorted result.
+	std::map<std::string, LanguageEntry> entries;
 	std::string selected_locale;
 
 	try {  // Begin read locales table
@@ -460,7 +449,9 @@
 
 				std::string name = i18n::make_ligatures(table->get_string("name").c_str());
 				const std::string sortname = table->get_string("sort_name");
-				entries.push_back(LanguageEntry(localename, name, sortname));
+				LanguageEntry* entry = new LanguageEntry(localename, name);
+				entries.insert(std::make_pair(sortname, *entry));
+				language_entries_.insert(std::make_pair(localename, *entry));
 
 				if (localename == current_locale) {
 					selected_locale = current_locale;
@@ -468,7 +459,7 @@
 
 			} catch (const WException&) {
 				log("Could not read locale for: %s\n", localename.c_str());
-				entries.push_back(LanguageEntry(localename, localename, localename));
+				entries.insert(std::make_pair(localename, LanguageEntry(localename, localename)));
 			}  // End read locale from table
 		}     // End scan locales directory
 	} catch (const LuaError& err) {
@@ -477,11 +468,76 @@
 	}           // End read locales table
 
 	find_selected_locale(&selected_locale, current_locale);
-	std::sort(entries.begin(), entries.end());
-	for (const LanguageEntry& entry : entries) {
-		language_dropdown_.add(entry.descname.c_str(), entry.localename, nullptr,
-		                       entry.localename == selected_locale, "");
-	}
+	for (const auto& entry : entries) {
+		const LanguageEntry& language_entry = entry.second;
+		language_dropdown_.add(language_entry.descname.c_str(), language_entry.localename, nullptr,
+		                       language_entry.localename == selected_locale, "");
+	}
+}
+
+/**
+ * Updates the language statistics message according to the currently selected locale.
+ * @param include_system_lang We only want to include the system lang if it matches the Widelands
+ * locale.
+ */
+void FullscreenMenuOptions::update_language_stats(bool include_system_lang) {
+	int percent = 100;
+	std::string message = "";
+	if (language_dropdown_.has_selection()) {
+		std::string locale = language_dropdown_.get_selected();
+		// Empty locale means try system locale
+		if (locale.empty() && include_system_lang) {
+			std::vector<std::string> parts;
+			boost::split(parts, i18n::get_locale(), boost::is_any_of("."));
+			if (language_entries_.count(parts[0]) == 1) {
+				locale = parts[0];
+			} else {
+				boost::split(parts, parts[0], boost::is_any_of("@"));
+				if (language_entries_.count(parts[0]) == 1) {
+					locale = parts[0];
+				} else {
+					boost::split(parts, parts[0], boost::is_any_of("_"));
+					if (language_entries_.count(parts[0]) == 1) {
+						locale = parts[0];
+					}
+				}
+			}
+		}
+
+		// If we have the locale, grab the stats and set the message
+		if (language_entries_.count(locale) == 1) {
+			try {
+				const LanguageEntry& entry = language_entries_[locale];
+				Profile prof("i18n/translation_stats.conf");
+				Section& s = prof.get_safe_section("global");
+				const int total = s.get_int("total");
+				s = prof.get_safe_section(locale);
+				percent = floor(100.f * s.get_int("translated") / total);
+				if (percent == 100) {
+					message = (boost::format(_("The translation into %s is complete.")) %
+								  entry.descname)
+									 .str();
+				} else {
+					message = (boost::format(_("The translation into %s is %d%% complete.")) %
+								  entry.descname % percent)
+									 .str();
+				}
+			} catch (...) {
+			}
+		}
+	}
+
+	// We will want some help with incomplete translations. We set this lower than 100%,
+	// because some translators let things drop a bit sometimes because they're busy and
+	// will catch up with the work later.
+	if (percent <= 90) {
+		message = message + " " +
+		          (boost::format(_("If you wish to help us translate, please visit %s")) %
+		           "<font underline=1>widelands.org/wiki/TranslatingWidelands</font>")
+		             .str();
+	}
+	// Make font a bit smaller so the link will fit at 800x600 resolution.
+	translation_info_.set_text(as_uifont(message, 12));
 }
 
 void FullscreenMenuOptions::clicked_apply() {

=== modified file 'src/ui_fsmenu/options.h'
--- src/ui_fsmenu/options.h	2017-09-11 08:09:07 +0000
+++ src/ui_fsmenu/options.h	2017-11-04 09:41:54 +0000
@@ -104,6 +104,7 @@
 
 	// Fills the language selection list
 	void add_languages_to_list(const std::string& current_locale);
+	void update_language_stats(bool include_system_lang);
 
 	// Saves the options and reloads the active tab
 	void clicked_apply();
@@ -112,8 +113,6 @@
 	uint32_t butw_;
 	uint32_t buth_;
 	uint32_t hmargin_;
-	uint32_t tab_panel_width_;
-	uint32_t column_width_;
 	uint32_t tab_panel_y_;
 
 	UI::Textarea title_;
@@ -123,6 +122,7 @@
 	// UI elements
 	UI::TabPanel tabs_;
 	UI::Box box_interface_;
+	UI::Box box_interface_left_;
 	UI::Box box_windows_;
 	UI::Box box_sound_;
 	UI::Box box_saving_;
@@ -134,6 +134,7 @@
 	UI::Checkbox fullscreen_;
 	UI::Checkbox inputgrab_;
 	UI::SpinBox sb_maxfps_;
+	UI::MultilineTextarea translation_info_;
 
 	// Windows options
 	UI::Checkbox snap_win_overlap_only_;
@@ -170,6 +171,19 @@
 
 	/// All supported screen resolutions.
 	std::vector<ScreenResolution> resolutions_;
+
+	// Data model for the entries in the language selection list.
+	struct LanguageEntry {
+		LanguageEntry(const std::string& init_localename,
+		              const std::string& init_descname)
+		   : localename(init_localename), descname(init_descname) {
+		}
+		LanguageEntry() : LanguageEntry("", "") {
+		}
+		std::string localename;  // ISO code for the locale
+		std::string descname;    // Native language name
+	};
+	std::map<std::string, LanguageEntry> language_entries_;
 };
 
 #endif  // end of include guard: WL_UI_FSMENU_OPTIONS_H

=== modified file 'utils/merge_and_push_translations.sh'
--- utils/merge_and_push_translations.sh	2017-05-19 07:41:26 +0000
+++ utils/merge_and_push_translations.sh	2017-11-04 09:41:54 +0000
@@ -67,6 +67,11 @@
 
 # Update catalogues.
 utils/buildcat.py
+
+# Update statistics.
+utils/update_translation_stats.py
+
+# Commit and push.
 bzr commit -m "Fetched translations and updated catalogues."
 bzr push lp:widelands
 

=== added file 'utils/update_translation_stats.py'
--- utils/update_translation_stats.py	1970-01-01 00:00:00 +0000
+++ utils/update_translation_stats.py	2017-11-04 09:41:54 +0000
@@ -0,0 +1,147 @@
+#!/usr/bin/python2
+# encoding: utf-8
+
+
+"""Uses pocount from the Translate Toolkit to write translation statistics to
+data/i18n/translation_stats.conf.
+
+You will need to have the Translate Toolkit installed:
+http://toolkit.translatehouse.org/
+
+For Debian-based Linux: sudo apt-get install translate-toolkit
+
+"""
+
+from collections import defaultdict
+from subprocess import call, check_output, CalledProcessError
+import os.path
+import re
+import subprocess
+import sys
+import traceback
+
+#############################################################################
+# Data Containers                                                           #
+#############################################################################
+
+
+class TranslationStats:
+    """Total source words and translated source words."""
+
+    def __init__(self):
+        # We need the total only once, but since the entries come in per
+        # directory rather than per locale, we just store it here to keep the
+        # algorithm simpler
+        self.total = 0
+        self.translated = 0
+
+
+#############################################################################
+# Main Loop                                                                 #
+#############################################################################
+
+def generate_translation_stats(po_dir, output_file):
+    locale_stats = defaultdict(TranslationStats)
+
+    sys.stdout.write('Fetching translation stats ')
+
+    # Regex to extract the locale from the po filenames.
+    regex_po = re.compile(r"/\S+/(\w+)\.po")
+
+    # We get errors for non-po files in the base po dir, so we have to walk
+    # the subdirs.
+    for subdir in sorted(os.listdir(po_dir), key=str.lower):
+        subdir = os.path.join(po_dir, subdir)
+        if not os.path.isdir(subdir):
+            continue
+
+        sys.stdout.write('.')
+        sys.stdout.flush()
+
+        try:
+            # We need shell=True, otherwise we get "No such file or directory".
+            stats_output = check_output(
+                ['pocount ' + subdir + ' --csv'], stderr=subprocess.STDOUT, shell=True)
+            if 'ERROR' in stats_output:
+                print('\nError running pocount:\n' + stats_output.split('\n', 0)
+                      [0]) + '\nAborted creating translation statistics.'
+                return False
+
+        except CalledProcessError:
+            print('Failed to run pocount:\n  FILE: ' + po_dir +
+                  '\n  ' + stats_output.split('\n', 1)[1])
+            return False
+
+        result = stats_output.split('\n')
+
+        # pocount distributes its header over multiple lines, so we have to do some
+        # collecting here before we parse it
+        header = ""
+        for line in result:
+            # Non-header rows will have a file path in them
+            if "/" in line or "\\" in line:
+                break
+            header = header + " " + line
+        header_entries = header.split(",")
+        column_counter = 0
+        total_column = 0
+        translated_column = 0
+        while column_counter < len(header_entries):
+            if header_entries[column_counter].strip() == "Total Source Words":
+                total_column = column_counter
+            elif header_entries[column_counter].strip() == "Translated Source Words":
+                translated_column = column_counter
+            column_counter = column_counter + 1
+
+        # Now do the actual counting for the current textdomain
+        for line in result:
+            cells = line.split(",")
+            po_filename = cells[0]
+            if po_filename.endswith(".po"):
+                entry = TranslationStats()
+                locale = regex_po.match(po_filename).group(1)
+                if locale in locale_stats:
+                    entry = locale_stats[locale]
+                entry.total = entry.total + int(cells[total_column])
+                entry.translated = entry.translated + int(cells[translated_column])
+                if entry.translated > entry.total:
+                    print("Error! Translated " + str(entry.translated) + " (+"+(cells[translated_column])+") is bigger than the total of " + str(entry.total) + "("+(cells[total_column])+")\n" + line)
+                    sys.exit(1)
+                locale_stats[locale] = entry
+
+    print('\n\nLocale\tTotal\tTranslated')
+    print('------\t-----\t----------')
+
+    # The total goes in a [global] section and is identical for all locales
+    result = '[global]\n'
+    result = result + 'total=' + str(locale_stats[locale_stats.keys()[0]].total) + '\n\n'
+
+    # Write translation stats for all locales
+    for locale in sorted(locale_stats.keys(), key=str.lower):
+        entry = locale_stats[locale]
+        print(locale + '\t' + str(entry.total) + '\t' + str(entry.translated))
+        result = result + '[' + locale + ']\n'
+        result = result + 'translated=' + str(entry.translated) + '\n\n'
+
+    with open(output_file, 'w+') as destination:
+        destination.write(result[:-1])  # Strip the final \n
+    print('\nResult written to ' + output_file)
+    return True
+
+
+def main():
+    try:
+        po_dir = os.path.abspath(os.path.join(
+            os.path.dirname(__file__), '../po'))
+        output_file = os.path.abspath(os.path.join(
+            os.path.dirname(__file__), '../data/i18n/translation_stats.conf'))
+        result = generate_translation_stats(po_dir, output_file)
+        return result
+
+    except Exception:
+        print('Something went wrong:')
+        traceback.print_exc()
+        return 1
+
+if __name__ == '__main__':
+    sys.exit(main())