Merge lp:~parnold-x/slingshot/conversion-plugin into lp:~elementary-pantheon/slingshot/trunk

Proposed by Djax on 2014-08-03
Status: Work in progress
Proposed branch: lp:~parnold-x/slingshot/conversion-plugin
Merge into: lp:~elementary-pantheon/slingshot/trunk
Diff against target: 422 lines (+362/-3)
4 files modified
lib/synapse-plugins/CMakeLists.txt (+6/-2)
lib/synapse-plugins/conversion-plugin.vala (+349/-0)
lib/synapse-plugins/vapi/monetary.vapi (+6/-0)
src/Backend/SynapseSearch.vala (+1/-1)
To merge this branch: bzr merge lp:~parnold-x/slingshot/conversion-plugin
Reviewer Review Type Date Requested Status
Daniel Fore 2014-08-03 Needs Fixing on 2015-09-15
Review via email: mp+229368@code.launchpad.net

Commit message

Add a unit conversion-plugin

Description of the change

Add a unit conversion-plugin in synapse. It uses GNU units (http://www.gnu.org/software/units/) so basically it can convert everything in everything.
The "conversion word" is set to "to" and should be localised. To test type for example "100$ to €" or "123l to gallons".
To support temperature conversion some additional code is added.
Most code in the plugin is needed to update the currency conversion rates. GNU units has a python script for this purpose but it needs root rights and additional python modules that are not included in elementary os so I implemented the update mechanism in vala.
The plugin checks for a update every 6 hours by checking if the currency rate file modification date is from today. If not it get the rates from timegenie.com, parses it and write it to the file.
At the first run it also copies the definition file from /usr/share/units/ to ~/.local/share/units and creates the directories if needed.

TODO: Add good icon. Till now it takes the calculator icon.

To post a comment you must log in.
Daniel Fore (danrabbit) wrote :

hmm this doesn't seem to do anything for me. I don't have a /usr/share/units or ~/.local/share/units. Is there a dependency that wasn't added to cmake?

review: Needs Fixing
Djax (parnold-x) wrote :

It depends on GNU units. So you need to install the "units" package. Same as the calculator depends on the "bc" package that i installed by default.

Daniel Fore (danrabbit) wrote :

It would probably be good to show the final units in the result like 100 meters to miles = "0.06 miles"

Also, rounding off after the second decimal might be good :)

Can we also add "as" and "in"? so we could have "100 liters in gallons" etc

433. By Djax on 2014-08-05

conversion word = to,as,in and formating the output

434. By Djax on 2014-08-05

reformating the output

Djax (parnold-x) wrote :

Added "as" and "in". Should the "conversion words" stay the same in all languages or should they be localised?
Also rounding the output and add the unit to the result. When having very big or small numbers it prints e+14 or e-9. I think this is better than 100000000000000. OK?

Is there a way to ammend to a commit like in git in bazaar?

Daniel Fore (danrabbit) wrote :

I imagine it should probably be localized. I'm not sure how many words make sense in all languages so I wonder if there's a way to define an array that localizations can fill with however many words make sense for them?

If possible, I would prefer something more human readable like "100 Million Miles" until we get over billion (I think anything more than that is probably not common language) and then e+14 probably makes sense.

You can do 'uncommit' and 'commit -m "whatever"' and then 'bzr push --overwrite'

435. By Djax on 2014-08-05

localisation and formating output

Djax (parnold-x) wrote :

Good to know. Thanks for the bzr commands.
Yep, no problem. I added a comment for the localisation guys. But if they localise it wrong the regex match will fail.

/* @ localization if you want use mutiple words, split them with a "|" between the words.
   If it's just one word use "to". Don't use whitspaces before or after the words!
*/
private const string CONVERSION_WORD = _("to|as|in");
private const string MILLION = _("million");
private const string BILLION = _("billion");

I also added formating the result to million and billion. And when over 999 billion it uses the e+xx format. Don't know if you also want thousand? Till now it prints 120000 as number or do you want 1 tousand till 999 thousand also reformated?

Also happy to do anything else :) I am just couple days offline starting tomorrow.

Daniel Fore (danrabbit) wrote :

Hey I hate to mention this after the fact xD But I was just thinking "There must be a library for this" and it turns out there is a library that will format strings based on locale for decimal places and separators and all that. So I guess we should probably be using that :p

http://www.gnu.org/software/libc/manual/html_node/Formatting-Numbers.html

436. By Djax on 2014-08-09

using libc strfmon function

Djax (parnold-x) wrote :

Hey, I am back online.
I tried to write a vapi file that use the libc function and integrated it into cmake. I hope this is the right way since this is my first .vapi file. It seems to work well.

437. By Djax on 2014-08-15

replacement after split

438. By Djax on 2014-08-15

improved regex and codestyle thanks to jacobparker1992; print small and very big numbers scientifically

Djax (parnold-x) wrote :

Added improved regex match for the temperatures. Thanks to Jakob Parker.

For the record:

I wondered if we now exclude other units and the symbols C and F are assign to different units.
http://en.wikipedia.org/wiki/Coulomb
http://en.wikipedia.org/wiki/Farad

celsius is assigned to °C
fahrenheit to °F
and rankin to °R
only kelvin is assigned to K without the degree symbol.

But google does it also without a degree symbol so I think temperature conversion is used way more often and converting in Farad/Coulomb is still possible with the full names.
https://www.google.de/search?q=100C+to+F&oq=100C+to+F

Jacob Parker (jacobparker1992) wrote :

Don't kill me, but I changed the code style once again to match the other files in the plugins directory (which is apparently different to the elementary code style :P).

How are digits handled? We pick up '1.3 meters to yards', but I don't think we handle the funny countries that would write '1,3' instead. I'm not sure why we pick up '1.3' though---is that somehow part of the digit regex (JS considers \d to only be 0-9)?

- Jacob

439. By Djax on 2014-08-16

allow , as digit seperator

Djax (parnold-x) wrote :

The other plugins are just imported from synapse, so no need to adapt to their code style.
Ah yeah, my "funny" country uses 1,3 but it is easy to fix.
"1.3 meters to yards". The match_regex splits just at " to " with no digit before the whitespace. So no problem with what digit is used.
In the temperature case there was a problem to match the digit in that case but I fixed that also.

Jacob Parker (jacobparker1992) wrote :

Any chance this could get another review?

Djax (parnold-x) wrote :

Note: Some superscripts look slightly missplaced. This is caused by the font size of the search result. With 12pt or greater everything looks ok.

Daniel Fore (danrabbit) wrote :

Bumping. Needs to be merged into trunk

review: Needs Fixing

Unmerged revisions

439. By Djax on 2014-08-16

allow , as digit seperator

438. By Djax on 2014-08-15

improved regex and codestyle thanks to jacobparker1992; print small and very big numbers scientifically

437. By Djax on 2014-08-15

replacement after split

436. By Djax on 2014-08-09

using libc strfmon function

435. By Djax on 2014-08-05

localisation and formating output

434. By Djax on 2014-08-05

reformating the output

433. By Djax on 2014-08-05

conversion word = to,as,in and formating the output

432. By Djax on 2014-08-03

add unit conversion-plugin

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'lib/synapse-plugins/CMakeLists.txt'
2--- lib/synapse-plugins/CMakeLists.txt 2014-06-29 14:50:36 +0000
3+++ lib/synapse-plugins/CMakeLists.txt 2014-08-16 22:26:39 +0000
4@@ -6,6 +6,8 @@
5 gio-unix-2.0
6 gee-0.8
7 gtk+-3.0
8+ libsoup-2.4
9+ libxml-2.0
10 )
11
12 pkg_check_modules(PLUGINS_DEPS REQUIRED ${PLUGINS_PKG})
13@@ -13,6 +15,7 @@
14 set(PLUGINS_SOURCE
15 calculator-plugin.vala
16 command-plugin.vala
17+ conversion-plugin.vala
18 desktop-file-plugin.vala
19 )
20
21@@ -23,6 +26,8 @@
22 PACKAGES
23 ${PLUGINS_PKG}
24 synapse-core
25+CUSTOM_VAPIS
26+ vapi/monetary.vapi
27 OPTIONS
28 --vapidir=${CMAKE_BINARY_DIR}/lib/synapse-core
29 GENERATE_VAPI
30@@ -43,5 +48,4 @@
31 SOVERSION ${PLUGINS_LIB_SOVERSION}
32 )
33
34-target_link_libraries (${PLUGINS_LIBRARY_NAME} ${PLUGINS_DEPS_LIBRARIES} synapse-core)
35-
36+target_link_libraries (${PLUGINS_LIBRARY_NAME} ${PLUGINS_DEPS_LIBRARIES} synapse-core)
37\ No newline at end of file
38
39=== added file 'lib/synapse-plugins/conversion-plugin.vala'
40--- lib/synapse-plugins/conversion-plugin.vala 1970-01-01 00:00:00 +0000
41+++ lib/synapse-plugins/conversion-plugin.vala 2014-08-16 22:26:39 +0000
42@@ -0,0 +1,349 @@
43+/*
44+ * Copyright (C) 2014 Peter Arnold <parnold1@gmail.com>
45+ *
46+ * This program is free software; you can redistribute it and/or modify
47+ * it under the terms of the GNU General Public License as published by
48+ * the Free Software Foundation; either version 2 of the License, or
49+ * (at your option) any later version.
50+ *
51+ * This program is distributed in the hope that it will be useful,
52+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
53+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
54+ * GNU General Public License for more details.
55+ *
56+ * You should have received a copy of the GNU General Public License
57+ * along with this program; if not, write to the Free Software
58+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA.
59+ *
60+ * Authored by Peter Arnold <parnold1@gmail.com>
61+ *
62+ */
63+
64+namespace Synapse
65+{
66+ errordomain IOError {
67+ FILE_NOT_FOUND
68+ }
69+
70+ public class ConversionPlugin: Object, Activatable, ItemProvider {
71+ public bool enabled { get; set; default = true; }
72+
73+ public void activate () {}
74+
75+ public void deactivate () {}
76+
77+ private class Result: Object, Match {
78+ // from Match interface
79+ public string title { get; construct set; }
80+ public string description { get; set; }
81+ public string icon_name { get; construct set; }
82+ public bool has_thumbnail { get; construct set; }
83+ public string thumbnail_path { get; construct set; }
84+ public MatchType match_type { get; construct set; }
85+
86+ public int default_relevancy { get; set; default = 0; }
87+
88+ public Result (string result) {
89+ Object (match_type: MatchType.TEXT,
90+ title: result,
91+ description: result,
92+ has_thumbnail: false,
93+ icon_name: "accessories-calculator");
94+ }
95+ }
96+
97+ static void register_plugin () {
98+ DataSink.PluginRegistry.get_default ().register_plugin (
99+ typeof (ConversionPlugin),
100+ _ ("Conversion"),
101+ _ ("Conversion of units."),
102+ "accessories-calculator",
103+ register_plugin,
104+ Environment.find_program_in_path ("units") != null,
105+ _ ("units is not installed")
106+ );
107+ }
108+
109+ static construct {
110+ register_plugin ();
111+ }
112+
113+ /* @ localization if you want use mutiple words, split them with a "|" between the words.
114+ If it's just one word use "to". Don't use whitspaces before or after the words!
115+ */
116+ private const string CONVERSION_WORD = _("to|as|in");
117+ private Regex match_regex;
118+ private Regex temp_regex;
119+ private Regex digit_regex;
120+
121+ construct {
122+ try {
123+ /* regex to match the conversion words (to, as, in) with no digit before the whitespace in cases the conversion word
124+ has the same name as a unit for example in and in (inch)
125+ */
126+ match_regex = new Regex (@"(?<=\\D)\\s*\\b(?:$CONVERSION_WORD)\\b\\s*",
127+ RegexCompileFlags.OPTIMIZE);
128+ // regex to match the temperature units
129+ temp_regex = new Regex ("(?i)(?<=[\\d\\s])[FCKR]\\b|fahrenheit|celsius|kelvin|rankine",
130+ RegexCompileFlags.OPTIMIZE);
131+ digit_regex = new Regex ("(\\d*\\.?\\d*)",
132+ RegexCompileFlags.OPTIMIZE);
133+ } catch (Error e) {
134+ Utils.Logger.error (this, "Error creating regexp.");
135+ }
136+
137+ try {
138+ // set a timer that wake ups every 6hours = 21600s to check if the currency rates should be updated
139+ check_for_update.begin ();
140+ Timeout.add_seconds_full (Priority.DEFAULT, 21600, () => {
141+ check_for_update.begin ();
142+ return true;
143+ });
144+ } catch (Error e) {
145+ Utils.Logger.error (this, "Error updating the currency rates");
146+ }
147+
148+ }
149+
150+ public bool handles_query (Query query) {
151+ return (QueryFlags.ACTIONS in query.query_type);
152+ }
153+
154+ static bool regex_function (MatchInfo info, StringBuilder resource) {
155+ string match = info.fetch (0).slice (0, 1).up ();
156+ resource.append (@"temp$match ");
157+ return false;
158+ }
159+
160+ public async ResultSet? search (Query query) throws SearchError {
161+ string input = query.query_string;
162+ bool matched = match_regex.match (input);
163+
164+ if (!matched) {
165+ query.check_cancellable ();
166+ return null;
167+ }
168+
169+ try {
170+ input = input.replace ("²", "^2").replace ("³", "^3").replace ("°", "").replace(",",".");
171+
172+ // to support the conversion of temperatures see http://www.gnu.org/software/units/manual/units.html#Temperature-Conversions
173+ bool temparaturematch = temp_regex.match (input);
174+
175+ if (temparaturematch)
176+ input = temp_regex.replace_eval (input, input.length, 0, RegexMatchFlags.NOTBOL, (RegexEvalCallback) regex_function);
177+
178+ string[] inputsplit = match_regex.split (input);
179+
180+ if (temparaturematch) {
181+ MatchInfo info;
182+ digit_regex.match (inputsplit[0], 0, out info);
183+ string digit = info.fetch (0);
184+ inputsplit[0] = inputsplit[0].replace (digit, "") + @"($digit)";
185+ }
186+
187+ Pid pid;
188+ int read_fd, write_fd;
189+ string[] argv = {"units", "-1", "-f", Environment.get_home_dir () + "/.local/share/units/definitions.units", inputsplit[0], inputsplit[1]};
190+ string? solution = null;
191+
192+ Process.spawn_async_with_pipes (null, argv, null,
193+ SpawnFlags.SEARCH_PATH,
194+ null, out pid, out write_fd, out read_fd);
195+
196+ UnixInputStream read_stream = new UnixInputStream (read_fd, true);
197+ DataInputStream bc_output = new DataInputStream (read_stream);
198+ solution = yield bc_output.read_line_async (Priority.DEFAULT_IDLE, query.cancellable);
199+
200+ if (solution != null && (solution.contains ("*") || temparaturematch)) {
201+ query.check_cancellable ();
202+ solution = solution.replace ("*", "");
203+ solution = solution.chug ();
204+ bool res = double.try_parse (solution);
205+ if (!res)
206+ return null;
207+
208+ double d = double.parse (solution);
209+ string digit;
210+ // the smallest output of the strfmon function is 0.01 and > 10^11 is not really readable in float format
211+ // so print it scientifically in those cases
212+ if (d >= 0.01 && d < 999999999999.9) {
213+ char buffer[100];
214+ Monetary.strfmon(buffer, "%!n", d);
215+ digit = (string) buffer;
216+ } else
217+ digit = print_scientifically (d);
218+
219+ string unit = " " + inputsplit[1];
220+ unit = unit.replace (" tempK", " K").replace (" temp", " °");
221+
222+ Result result = new Result (digit + unit);
223+ ResultSet results = new ResultSet ();
224+ results.add (result, Match.Score.AVERAGE);
225+
226+ return results;
227+ }
228+ } catch (Error err) {
229+ if (!query.is_cancelled ())
230+ warning ("%s", err.message);
231+ }
232+
233+ return null;
234+ }
235+
236+ // prints the %g output of printf in "x 10⁷⁷" format
237+ private string print_scientifically (double d) {
238+ string digit = "%g".printf(d);
239+ if (!digit.contains ("e"))
240+ return digit;
241+ string exponent = digit.substring (digit.last_index_of ("e")+2);
242+ if (exponent.has_prefix ("0"))
243+ exponent = exponent.replace ("0","");
244+ exponent = exponent.replace ("0","⁰").replace ("1","¹").replace ("2","²").replace ("3","³").replace ("4","⁴").replace ("5","⁵")
245+ .replace ("6","⁶").replace ("7","⁷").replace ("8","⁸").replace ("9","⁹");
246+ if (digit.substring (digit.last_index_of ("e")+1,1) == "-")
247+ exponent = "⁻" + exponent;
248+ return digit.substring (0,digit.last_index_of ("e")) + " x 10" + exponent;
249+ }
250+
251+ private async void check_for_update () {
252+ var file = File.new_for_path (Environment.get_home_dir () + "/.local/share/units/currency.units");
253+
254+ if (!file.query_exists ()) {
255+ try {
256+ //copy data to user directory to be able to modifiy it without root rights
257+ var dir = File.new_for_path (Environment.get_home_dir () + "/.local/share/units/");
258+ if (!dir.query_exists ())
259+ dir.make_directory ();
260+ var definitions_file = File.new_for_path ("/usr/share/units/definitions.units");
261+ var target_definitions = File.new_for_path (Environment.get_home_dir()+"/.local/share/units/definitions.units");
262+ if (!target_definitions.query_exists ())
263+ definitions_file.copy (target_definitions, FileCopyFlags.NONE);
264+ var currency_file = File.new_for_path ("/usr/share/units/currency.units");
265+ currency_file.copy (file, FileCopyFlags.NONE);
266+ } catch (Error e) {
267+ Utils.Logger.error (this, "Failed to copy /usr/share/units/definitions.units");
268+ }
269+ }
270+ if (file.query_exists ()) {
271+ try {
272+ var file_info = file.query_info ("*", FileQueryInfoFlags.NONE);
273+ var modification_date = new DateTime.from_timeval_local (file_info.get_modification_time ());
274+ var now = new DateTime.now_local ();
275+ // timegenie update their rates once a day
276+ if (now.get_day_of_year () != modification_date.get_day_of_year ())
277+ update (file, now);
278+ } catch (Error e) {
279+ Utils.Logger.error (this, "Failed to update '%s'".printf (file.get_path ()));
280+ }
281+ }
282+ }
283+
284+ private void update (File file, DateTime now) throws Error {
285+ file.delete ();
286+ var dos = new DataOutputStream (file.create (FileCreateFlags.REPLACE_DESTINATION));
287+ Array<string> codes;
288+ Array<string> names;
289+ Array<string> rates;
290+ // load and write data from timegenie
291+ load_currency_rates_data (out codes, out names, out rates);
292+ write_currency_data (dos, codes, names, rates, now);
293+ }
294+
295+ private void write_currency_data (DataOutputStream dos, Array<string> codes, Array<string> names, Array<string> values, DateTime now) throws Error {
296+ var builder = new StringBuilder ("# ISO Currency Codes\n\n");
297+
298+ for (int i = 0; i < codes.length; i++)
299+ builder.append ("%s%s%s\n".printf (codes.index (i), string.nfill (15, ' '), names.index (i)));
300+
301+ builder.append ("\n# Currency exchange rates from Time Genie (www.timegenie.com) from %s\n\n".printf (now.format ("%F")));
302+
303+ for (int i = 0; i < codes.length; i++)
304+ builder.append ("%s%s%s\n".printf (names.index (i), string.nfill (30-names.index (i).length, ' '), values.index (i)));
305+
306+ uint8[] data = builder.str.data;
307+ long written = 0;
308+
309+ while (written < data.length)
310+ written += dos.write (data[written:data.length]);
311+ }
312+
313+ private void load_currency_rates_data (out Array<string> codes, out Array<string> names, out Array<string> values) throws IOError {
314+ codes = new Array<string> ();
315+ names = new Array<string> ();
316+ values = new Array<string> ();
317+ var tmp_values = new Array<string> ();
318+ string url = "http://rss.timegenie.com/forex.xml";
319+
320+ var session = new Soup.Session ();
321+ var msg = new Soup.Message ("GET", url);
322+ session.send_message (msg);
323+ Xml.Doc* doc = Xml.Parser.parse_memory (
324+ (string) (msg.response_body.data),
325+ (int) (msg.response_body.length));
326+ if (doc == null)
327+ throw new IOError.FILE_NOT_FOUND("Failed to parse http://rss.timegenie.com/forex.xml");
328+ Xml.Node* root = doc->get_root_element ();
329+ if (root == null) {
330+ delete doc;
331+ throw new IOError.FILE_NOT_FOUND ("The xml file is empty");
332+ }
333+ int index_usd = -1;
334+ int index_euro = -1;
335+ int index = 0;
336+ for (Xml.Node* iter = root->children; iter != null; iter = iter->next) {
337+ if (iter->type == Xml.ElementType.ELEMENT_NODE) {
338+ if (iter->name == "data") {
339+ for (Xml.Node* iter2 = iter->children; iter2 != null; iter2 = iter2->next) {
340+ if (iter2->type == Xml.ElementType.ELEMENT_NODE) {
341+ switch (iter2->name) {
342+ case "code":
343+ var content = get_node_content (iter2);
344+ if (content != "") {
345+ if (content == "USD")
346+ index_usd = index;
347+ else if (content == "EUR")
348+ index_euro = index;
349+ codes.append_val (content);
350+ }
351+ break;
352+ case "description":
353+ var content = get_node_content (iter2);
354+ if (content != "") {
355+ if (content == "Anguilla (ECD)") content = "eastcaribbeandollar";
356+ content = content.replace (" ", "").down ();
357+ names.append_val (content);
358+ }
359+ break;
360+ case "rate":
361+ var content = get_node_content (iter2);
362+ if (content != "")
363+ tmp_values.append_val (content);
364+ break;
365+ default:
366+ // do nothing
367+ break;
368+ }
369+ }
370+ }
371+ index++;
372+ }
373+ }
374+ }
375+ delete doc;
376+ for (int i = 0; i < tmp_values.length; i++) {
377+ if (i == index_euro) values.append_val (tmp_values.index (index_usd) + " US$");
378+ else if (i == index_usd) values.append_val ("1 US$");
379+ else values.append_val ("1|" + tmp_values.index (i) + " euro");
380+ }
381+ }
382+
383+ private string get_node_content (Xml.Node* node) {
384+ for (Xml.Node* iter = node->children; iter != null; iter = iter->next) {
385+ if (iter->type == Xml.ElementType.TEXT_NODE)
386+ return iter->get_content ();
387+ }
388+ return "";
389+ }
390+ }
391+}
392\ No newline at end of file
393
394=== added directory 'lib/synapse-plugins/vapi'
395=== added file 'lib/synapse-plugins/vapi/monetary.vapi'
396--- lib/synapse-plugins/vapi/monetary.vapi 1970-01-01 00:00:00 +0000
397+++ lib/synapse-plugins/vapi/monetary.vapi 2014-08-16 22:26:39 +0000
398@@ -0,0 +1,6 @@
399+[CCode (cheader_filename = "monetary.h")]
400+public class Monetary {
401+ [CCode(cname = "strfmon")]
402+ public static ssize_t strfmon(char[] s, string format, double data);
403+
404+}
405\ No newline at end of file
406
407=== modified file 'src/Backend/SynapseSearch.vala'
408--- src/Backend/SynapseSearch.vala 2014-06-29 14:50:36 +0000
409+++ src/Backend/SynapseSearch.vala 2014-08-16 22:26:39 +0000
410@@ -23,6 +23,7 @@
411 private static Type[] plugins = {
412 typeof (Synapse.CalculatorPlugin),
413 typeof (Synapse.CommandPlugin),
414+ typeof (Synapse.ConversionPlugin),
415 typeof (Synapse.DesktopFilePlugin)
416 };
417
418@@ -161,4 +162,3 @@
419 }
420 }
421 }
422-

Subscribers

People subscribed via source and target branches