Merge lp:~mterry/ubiquity/translated-timezones into lp:ubiquity

Proposed by Michael Terry
Status: Merged
Merged at revision: not available
Proposed branch: lp:~mterry/ubiquity/translated-timezones
Merge into: lp:ubiquity
Diff against target: None lines
To merge this branch: bzr merge lp:~mterry/ubiquity/translated-timezones
Reviewer Review Type Date Requested Status
Ubuntu Installer Team Pending
Review via email: mp+8698@code.launchpad.net
To post a comment you must log in.
Revision history for this message
Michael Terry (mterry) wrote :
Download full text (4.2 KiB)

OK, this is a branch that supports translated timezones. It does so in an upstream-friendly manner by taking advantage of localechooser's list of translated countries/regions and tzsetup's list of translated timezones.

It replaces the timezone page's dropdowns from Region/City to Country/Timezone (though the labels stay the same, more on that in 'open issues' below).

The country dropdown entries comes from localechooser's template. It is translated for the current language. There are two sections to the dropbox. At the top is a list of 'suggested countries' or countries where localechooser thinks you're likely to be based on your language. Then a separator, then a list of all countries (including duplicates of the 'suggested' list).

The timezone dropdown entries come from tzsetup's template. At the top is a list of 'suggested timezones'. This comes from 'tzsetup/country/%CC%' if available, else it is a list of locations mapped to that country in the tz.Database compiled from zone.tab (which is often just one city). Then a separator, then a list of all timezones in human readable format (which again comes from zone.tab). Any data that comes from zone.tab is untranslated, so the only translation that actually happens here is in the 'suggested' list. 'Human readable format' here means that 'Foo/Bar/Some_City' becomes 'Some City'.

The 'suggestions' in each dropdown are called 'shortlists.'

When you change the country dropdown, the timezone dropdown's shortlist changes to be appropriate for that country. The debconf default for the shortlist is selected, or if there isn't one, the first entry in the shortlist.

When you select a timezone not in the current country, the country dropdown is changed to match.

Some countries (notably US, Mexico, and Canada) have non-one-to-one mappings for timezones in tzsetup. That is, they use 'area-zones' not 'city-zones' like 'US/Eastern' instead of 'America/New_York.' Presumably that is because that is how the citizens think of timezones (which I can confirm as an American anyway). Under the covers, they map to the identical city-zone (matched by md5sum of the locale file). If you click on the map, you will always get a city-zone. If you select the area-zone in the dropdown, the map will show the same location as the under-the-covers city-zone, but the dropdown will continue to show the translated name.

Now that we are showing long lists of translated names, sorting them becomes a problem. Python provides no native unicode collation facilities. GLib does, but pygobject does not seem to wrap them. So... I added optional support for PyICU. It's a python wrapper around the ICU IBM library, which provides lots of unicode helper stuff, including a nice collator algorithm. If the python-pyicu package is not installed, it just falls back to unicode-code-point comparison, which is terrible for any thing non-ASCII. python-pyicu is in universe. If we merge this, I would really like to see it move into main and have us depend upon it. Or find some other collation solution. Thoughts?

PyICU seems to be locale-dependent (and takes a locale object and all that), but I'm not seeing much evidence...

Read more...

3320. By Michael Terry

merge with trunk

3321. By Michael Terry

use translations from PyICU

3322. By Michael Terry

merge from trunk

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'debian/control'
2--- debian/control 2009-06-26 04:58:02 +0000
3+++ debian/control 2009-07-13 15:02:43 +0000
4@@ -12,6 +12,7 @@
5 Architecture: any
6 Depends: ${shlibs:Depends}, ${misc:Depends}, ${python:Depends}, debconf (>= 1.4.72ubuntu5), ubiquity-frontend-${mangled-version}, ubiquity-artwork-${mangled-version}, laptop-detect, lsb-release, ubiquity-casper, python-apt (>= 0.6.16.2ubuntu4), ${console-setup-depends}, iso-codes, passwd, adduser, os-prober, rdate, ${partman-depends}, ecryptfs-utils
7 Recommends: ${bootloader-recommends}
8+Suggests: python-pyicu
9 Conflicts: ubuntu-express, espresso, espresso-utils, espresso-locale, espresso-keyboard-setup, espresso-kbd-chooser, espresso-timezone, user-setup (<< 0.05ubuntu6), partman, espresso-grub, espresso-yaboot
10 Replaces: ubuntu-express, espresso, espresso-utils, espresso-locale, espresso-keyboard-setup, espresso-kbd-chooser, espresso-timezone, user-setup (<< 0.05ubuntu6), partman, espresso-grub, espresso-yaboot, ubiquity-frontend-gtk (<< 0.99.82)
11 XB-Python-Version: ${python:Versions}
12
13=== modified file 'ubiquity/components/timezone.py'
14--- ubiquity/components/timezone.py 2009-04-15 15:08:14 +0000
15+++ ubiquity/components/timezone.py 2009-07-13 15:02:43 +0000
16@@ -26,10 +26,24 @@
17 from ubiquity import i18n
18 import ubiquity.tz
19
20+try:
21+ import PyICU
22+except:
23+ PyICU = None
24+
25 class Timezone(FilteredCommand):
26 def prepare(self):
27+ self.regions = []
28+ self.timezones = []
29 self.tzdb = ubiquity.tz.Database()
30 self.multiple = False
31+ try:
32+ # Strip .UTF-8 from locale, PyICU doesn't parse it
33+ locale = self.frontend.locale and self.frontend.locale.rsplit('.', 1)[0]
34+ self.collator = locale and PyICU and \
35+ PyICU.Collator.createInstance(PyICU.Locale(locale))
36+ except:
37+ self.collator = None
38 if not 'UBIQUITY_AUTOMATIC' in os.environ:
39 self.db.fset('time/zone', 'seen', 'false')
40 cc = self.db.get('debian-installer/country')
41@@ -58,15 +72,71 @@
42 choices_c = self.choices_untranslated(question)
43 if choices_c:
44 zone = choices_c[0]
45- # special cases where default is not in zone.tab
46- if zone == 'Canada/Eastern':
47- zone = 'America/Toronto'
48- elif zone == 'US/Eastern':
49- zone = 'America/New_York'
50 self.frontend.set_timezone(zone)
51
52 return FilteredCommand.run(self, priority, question)
53
54+ def get_default_for_region(self, region):
55+ try:
56+ return self.db.get('tzsetup/country/%s' % region)
57+ except debconf.DebconfError:
58+ return None
59+
60+ def collation_key(self, s):
61+ if self.collator:
62+ try:
63+ return self.collator.getCollationKey(s[0]).getByteArray()
64+ except:
65+ pass
66+ return s[0]
67+
68+ # Returns [('translated country name', 'country iso3166 code')...] list
69+ def build_region_pairs(self):
70+ if self.regions: return self.regions
71+ continents = self.choices_untranslated('localechooser/continentlist')
72+ for continent in continents:
73+ question = 'localechooser/countrylist/%s' % continent.replace(' ', '_')
74+ self.regions.extend(self.choices_display_map(question).items())
75+ self.regions.sort(key=self.collation_key)
76+ return self.regions
77+
78+ # Returns [('human timezone name', 'timezone')...] list
79+ def build_timezone_pairs(self):
80+ if self.timezones: return self.timezones
81+ for location in self.tzdb.locations:
82+ self.timezones.append((location.human_zone, location.zone))
83+ self.timezones.sort(key=self.collation_key)
84+ return self.timezones
85+
86+ # Returns [('translated short list of countries', 'timezone')...] list
87+ def build_shortlist_region_pairs(self, language_code):
88+ try:
89+ shortlist = self.choices_display_map('localechooser/shortlist/%s' % language_code)
90+ # Remove any 'other' entry
91+ for pair in shortlist.items():
92+ if pair[1] == 'other':
93+ del shortlist[pair[0]]
94+ break
95+ shortlist = shortlist.items()
96+ shortlist.sort(key=self.collation_key)
97+ return shortlist
98+ except debconf.DebconfError:
99+ return None
100+
101+ # Returns [('translated short list of timezones', 'timezone')...] list
102+ def build_shortlist_timezone_pairs(self, country_code):
103+ try:
104+ shortlist = self.choices_display_map('tzsetup/country/%s' % country_code)
105+ for pair in shortlist.items():
106+ # Remove any 'other' entry, we don't need it
107+ if pair[1] == 'other':
108+ del shortlist[pair[0]]
109+ shortlist = shortlist.items()
110+ shortlist.sort(key=self.collation_key)
111+ return shortlist
112+ except debconf.DebconfError:
113+ return None
114+
115 def ok_handler(self):
116 zone = self.frontend.get_timezone()
117 if zone is None:
118
119=== modified file 'ubiquity/frontend/gtk_ui.py'
120--- ubiquity/frontend/gtk_ui.py 2009-06-30 23:33:13 +0000
121+++ ubiquity/frontend/gtk_ui.py 2009-07-13 15:35:37 +0000
122@@ -176,6 +176,7 @@
123 self.format_warnings = {}
124 self.format_warning = None
125 self.format_warning_align = None
126+ self.timezone_city_combo_has_shortlist = False
127
128 self.laptop = execute("laptop-detect")
129
130@@ -549,72 +550,108 @@
131 self.allow_go_backward(False)
132
133 def setup_timezone_page(self):
134+ def is_separator(m, i):
135+ return m[i][0] is None
136
137 renderer = gtk.CellRendererText()
138 self.timezone_zone_combo.pack_start(renderer, True)
139 self.timezone_zone_combo.add_attribute(renderer, 'text', 0)
140- list_store = gtk.ListStore(gobject.TYPE_STRING)
141- self.timezone_zone_combo.set_model(list_store)
142+ zone_store = gtk.ListStore(gobject.TYPE_STRING, gobject.TYPE_STRING)
143+ self.timezone_zone_combo.set_model(zone_store)
144+ self.timezone_zone_combo.set_row_separator_func(is_separator)
145 self.timezone_zone_combo.connect('changed', self.zone_combo_selection_changed)
146- self.timezone_city_combo.connect('changed', self.city_combo_selection_changed)
147
148 renderer = gtk.CellRendererText()
149 self.timezone_city_combo.pack_start(renderer, True)
150 self.timezone_city_combo.add_attribute(renderer, 'text', 0)
151- city_store = gtk.ListStore(gobject.TYPE_STRING)
152+ city_store = gtk.ListStore(gobject.TYPE_STRING, gobject.TYPE_STRING)
153 self.timezone_city_combo.set_model(city_store)
154-
155- self.regions = {}
156- for location in self.tzdb.locations:
157- region, city = location.zone.replace('_', ' ').split('/', 1)
158- if region in self.regions:
159- self.regions[region].append(city)
160- else:
161- self.regions[region] = [city]
162-
163- r = self.regions.keys()
164- r.sort()
165- for region in r:
166- list_store.append([region])
167+ self.timezone_city_combo.set_row_separator_func(is_separator)
168+ self.timezone_city_combo.connect('changed', self.city_combo_selection_changed)
169
170 def zone_combo_selection_changed(self, widget):
171+ if not isinstance(self.dbfilter, timezone.Timezone):
172+ return
173+
174 i = self.timezone_zone_combo.get_active()
175 m = self.timezone_zone_combo.get_model()
176- region = m[i][0]
177-
178+ if not i:
179+ return
180+ region = m[i][1]
181+
182 m = self.timezone_city_combo.get_model()
183- m.clear()
184- for city in self.regions[region]:
185- m.append([city])
186+
187+ # Remove any existing shortlist
188+ if self.timezone_city_combo_has_shortlist:
189+ iterator = m.get_iter_first()
190+ while iterator:
191+ is_sep = m[iterator][0] is None
192+ if not m.remove(iterator) or is_sep:
193+ break
194+ self.timezone_city_combo_has_shortlist = False
195+
196+ pairs = self.dbfilter.build_shortlist_timezone_pairs(region)
197+ if not pairs:
198+ # Build our own shortlist
199+ pairs = []
200+ locs = self.tzdb.cc_to_locs[region]
201+ for loc in locs:
202+ pairs.append((loc.human_zone, loc.zone))
203+
204+ sep = m.prepend([None, None])
205+ for pair in pairs:
206+ m.insert_before(sep, pair)
207+ self.timezone_city_combo_has_shortlist = True
208+
209+ default = self.dbfilter.get_default_for_region(region)
210+ if default:
211+ self.select_city(None, default)
212+ else:
213+ iterator = m.get_iter_first()
214+ self.timezone_city_combo.set_active_iter(iterator)
215
216 def city_combo_selection_changed(self, widget):
217- i = self.timezone_zone_combo.get_active()
218- m = self.timezone_zone_combo.get_model()
219- region = m[i][0]
220-
221 i = self.timezone_city_combo.get_active()
222 if i < 0:
223 # There's no selection yet.
224 return
225- m = self.timezone_city_combo.get_model()
226- city = m[i][0].replace(' ', '_')
227- city = region + '/' + city
228- self.tzmap.select_city(city)
229+
230+ zone = self.get_timezone()
231+ self.tzmap.select_city(zone)
232+
233+ # have to update region as well, in case user picked a city outside
234+ # current region
235+ loc = self.tzdb.get_loc(zone)
236+ if not loc:
237+ return
238+ m = self.timezone_zone_combo.get_model()
239+ iterator = m.get_iter_first()
240+ while iterator:
241+ if m[iterator][1] == loc.country:
242+ self.timezone_zone_combo.handler_block_by_func(self.zone_combo_selection_changed)
243+ self.timezone_zone_combo.set_active_iter(iterator)
244+ self.timezone_zone_combo.handler_unblock_by_func(self.zone_combo_selection_changed)
245+ break
246+ iterator = m.iter_next(iterator)
247
248 def select_city(self, widget, city):
249- region, city = city.replace('_', ' ').split('/', 1)
250+ loc = self.tzdb.get_loc(city)
251+ if not loc:
252+ return
253+ region = loc.country
254+
255 m = self.timezone_zone_combo.get_model()
256 iterator = m.get_iter_first()
257 while iterator:
258- if m[iterator][0] == region:
259+ if m[iterator][1] == region:
260 self.timezone_zone_combo.set_active_iter(iterator)
261 break
262 iterator = m.iter_next(iterator)
263-
264+
265 m = self.timezone_city_combo.get_model()
266 iterator = m.get_iter_first()
267 while iterator:
268- if m[iterator][0] == city:
269+ if m[iterator][1] == city:
270 self.timezone_city_combo.set_active_iter(iterator)
271 break
272 iterator = m.iter_next(iterator)
273@@ -1095,6 +1132,31 @@
274 if layout is not None and variant is not None:
275 self.dbfilter.apply_keyboard(layout, variant)
276
277+ def fill_timezone_boxes(self):
278+ m = self.timezone_zone_combo.get_model()
279+ if m.get_iter_first():
280+ return
281+ if not isinstance(self.dbfilter, timezone.Timezone):
282+ return
283+ tz = self.dbfilter
284+
285+ # Regions are a translated shortlist of regions, followed by full list
286+ m.clear()
287+ region_pairs = tz.build_shortlist_region_pairs(self.get_language())
288+ if region_pairs:
289+ for pair in region_pairs:
290+ m.append(pair)
291+ m.append([None, None])
292+ region_pairs = tz.build_region_pairs()
293+ for pair in region_pairs:
294+ m.append(pair)
295+
296+ m = self.timezone_city_combo.get_model()
297+ if not m.get_iter_first():
298+ pairs = self.dbfilter.build_timezone_pairs()
299+ for pair in pairs:
300+ m.append(pair)
301+
302 def prepare_page(self):
303 """Set up the frontend in preparation for running a step."""
304
305@@ -1238,6 +1300,8 @@
306 lang = lang.split('.')[0].lower()
307 for widget in self.language_questions:
308 self.translate_widget(getattr(self, widget), lang)
309+ # Clear zone combo, it will need to be regenerated
310+ self.timezone_zone_combo.get_model().clear()
311
312 def on_steps_switch_page (self, foo, bar, current):
313 self.current_page = current
314@@ -1455,18 +1519,13 @@
315
316
317 def set_timezone (self, timezone):
318+ self.fill_timezone_boxes()
319 self.select_city(None, timezone)
320
321 def get_timezone (self):
322- i = self.timezone_zone_combo.get_active()
323- m = self.timezone_zone_combo.get_model()
324- region = m[i][0]
325-
326 i = self.timezone_city_combo.get_active()
327 m = self.timezone_city_combo.get_model()
328- city = m[i][0].replace(' ', '_')
329- city = region + '/' + city
330- return city
331+ return m[i][1]
332
333 def set_keyboard_choices(self, choices):
334 layouts = gtk.ListStore(gobject.TYPE_STRING)
335
336=== modified file 'ubiquity/timezone_map.py'
337--- ubiquity/timezone_map.py 2009-04-14 11:00:35 +0000
338+++ ubiquity/timezone_map.py 2009-07-10 19:04:39 +0000
339@@ -225,9 +225,8 @@
340 width = self.background.get_width()
341
342 only_draw_selected = True
343- for loc in self.tzdb.locations:
344- if not (self.selected and loc.zone == self.selected):
345- continue
346+ loc = self.selected and self.tzdb.get_loc(self.selected)
347+ if loc:
348 pointx = convert_longitude_to_x(loc.longitude, width)
349 pointy = convert_latitude_to_y(loc.latitude, height)
350
351@@ -275,11 +274,11 @@
352
353 def select_city(self, city):
354 self.selected = city
355- for loc in self.tzdb.locations:
356- if loc.zone == city:
357- offset = (loc.raw_utc_offset.days * 24) + \
358- (loc.raw_utc_offset.seconds / 60.0 / 60.0)
359- self.selected_offset = str(offset)
360+ loc = self.tzdb.get_loc(city)
361+ if loc:
362+ offset = (loc.raw_utc_offset.days * 24) + \
363+ (loc.raw_utc_offset.seconds / 60.0 / 60.0)
364+ self.selected_offset = str(offset)
365 self.queue_draw()
366
367 def button_press(self, widget, event):
368
369=== modified file 'ubiquity/tz.py'
370--- ubiquity/tz.py 2009-03-10 18:00:30 +0000
371+++ ubiquity/tz.py 2009-07-10 19:02:56 +0000
372@@ -21,6 +21,8 @@
373 import datetime
374 import time
375 import xml.dom.minidom
376+import md5 # should be hashlib once we depend on >=2.5
377+import sys
378
379
380 TZ_DATA_FILE = '/usr/share/zoneinfo/zone.tab'
381@@ -166,6 +168,7 @@
382 else:
383 self.human_country = self.country
384 self.zone = bits[2]
385+ self.human_zone = self.zone.replace('_', ' ').split('/')[-1]
386 if len(bits) > 3:
387 self.comment = bits[3]
388 else:
389@@ -173,6 +176,14 @@
390 self.latitude = _parse_position(latitude, 2)
391 self.longitude = _parse_position(longitude, 3)
392
393+ # Grab md5sum of the timezone file for later comparison
394+ try:
395+ tz_file = file(os.path.join('/usr/share/zoneinfo', self.zone) ,'rb')
396+ self.md5sum = md5.md5(tz_file.read()).digest()
397+ tz_file.close()
398+ except IOError, e:
399+ self.md5sum = None
400+
401 try:
402 today = datetime.datetime.today()
403 except ValueError:
404@@ -197,7 +208,42 @@
405 continue
406 self.locations.append(Location(line, iso3166))
407 tzdata.close()
408- self.locations.sort(cmp, lambda location: location.zone)
409+
410+ # Build mappings from timezone->location and country->locations
411+ self.cc_to_locs = {}
412+ self.tz_to_loc = {}
413+ for loc in self.locations:
414+ self.tz_to_loc[loc.zone] = loc
415+ if loc.country in self.cc_to_locs:
416+ self.cc_to_locs[loc.country] += [loc]
417+ else:
418+ self.cc_to_locs[loc.country] = [loc]
419+
420+ def get_loc(self, tz):
421+ # Sometimes we'll encounter timezones that aren't really
422+ # city-zones, like "US/Eastern" or "Mexico/General". So first,
423+ # we check if the timezone is known. If it isn't, we search for
424+ # one with the same md5sum and make a reference to it
425+ try:
426+ return self.tz_to_loc[tz]
427+ except:
428+ try:
429+ tz_file = file(os.path.join('/usr/share/zoneinfo', tz) ,'rb')
430+ md5sum = md5.md5(tz_file.read()).digest()
431+ tz_file.close()
432+
433+ for loc in self.locations:
434+ if md5sum == loc.md5sum:
435+ self.tz_to_loc[tz] = loc
436+ return loc
437+ except IOError, e:
438+ pass
439+
440+ # If not found, oh well, just warn and move on.
441+ print >> sys.stderr, 'Could not understand timezone', tz
442+ self.tz_to_loc[tz] = None # save it for the future
443+ return None
444+
445
446 _database = None
447

Subscribers

People subscribed via source and target branches

to status/vote changes: