Merge lp:~parnold-x/slingshot/conversion-plugin into lp:~elementary-pantheon/slingshot/trunk

[Djax]

Add a unit conversion-plugin

Add a unit conversion-plugin in synapse. It uses GNU units (http://www.gnu.org/software/units/) so basically it can convert everything in everything.
The "conversion word" is set to "to" and should be localised. To test type for example "100$ to €" or "123l to gallons".
To support temperature conversion some additional code is added.
Most code in the plugin is needed to update the currency conversion rates. GNU units has a python script for this purpose but it needs root rights and additional python modules that are not included in elementary os so I implemented the update mechanism in vala.
The plugin checks for a update every 6 hours by checking if the currency rate file modification date is from today. If not it get the rates from timegenie.com, parses it and write it to the file.
At the first run it also copies the definition file from /usr/share/units/ to ~/.local/share/units and creates the directories if needed.

TODO: Add good icon. Till now it takes the calculator icon.

[Danielle Foré (danrabbit)]

hmm this doesn't seem to do anything for me. I don't have a /usr/share/units or ~/.local/share/units. Is there a dependency that wasn't added to cmake?

review: Needs Fixing

[Djax (parnold-x)]

It depends on GNU units. So you need to install the "units" package. Same as the calculator depends on the "bc" package that i installed by default.

[Danielle Foré (danrabbit)]

It would probably be good to show the final units in the result like 100 meters to miles = "0.06 miles"

Also, rounding off after the second decimal might be good :)

Can we also add "as" and "in"? so we could have "100 liters in gallons" etc

[lp:~parnold-x/slingshot/conversion-plugin]

433. By Djax on 2014-08-05

conversion word = to,as,in and formating the output

434. By Djax on 2014-08-05

reformating the output

[Djax (parnold-x)]

Added "as" and "in". Should the "conversion words" stay the same in all languages or should they be localised?
Also rounding the output and add the unit to the result. When having very big or small numbers it prints e+14 or e-9. I think this is better than 100000000000000. OK?

Is there a way to ammend to a commit like in git in bazaar?

[Danielle Foré (danrabbit)]

I imagine it should probably be localized. I'm not sure how many words make sense in all languages so I wonder if there's a way to define an array that localizations can fill with however many words make sense for them?

If possible, I would prefer something more human readable like "100 Million Miles" until we get over billion (I think anything more than that is probably not common language) and then e+14 probably makes sense.

You can do 'uncommit' and 'commit -m "whatever"' and then 'bzr push --overwrite'

[lp:~parnold-x/slingshot/conversion-plugin]

435. By Djax on 2014-08-05

localisation and formating output

[Djax (parnold-x)]

Good to know. Thanks for the bzr commands.
Yep, no problem. I added a comment for the localisation guys. But if they localise it wrong the regex match will fail.

/* @ localization if you want use mutiple words, split them with a "|" between the words.
   If it's just one word use "to". Don't use whitspaces before or after the words!
*/
private const string CONVERSION_WORD = _("to|as|in");
private const string MILLION = _("million");
private const string BILLION = _("billion");

I also added formating the result to million and billion. And when over 999 billion it uses the e+xx format. Don't know if you also want thousand? Till now it prints 120000 as number or do you want 1 tousand till 999 thousand also reformated?

Also happy to do anything else :) I am just couple days offline starting tomorrow.

[Danielle Foré (danrabbit)]

Hey I hate to mention this after the fact xD But I was just thinking "There must be a library for this" and it turns out there is a library that will format strings based on locale for decimal places and separators and all that. So I guess we should probably be using that :p

http://www.gnu.org/software/libc/manual/html_node/Formatting-Numbers.html

[lp:~parnold-x/slingshot/conversion-plugin]

436. By Djax on 2014-08-09

using libc strfmon function

[Djax (parnold-x)]

Hey, I am back online.
I tried to write a vapi file that use the libc function and integrated it into cmake. I hope this is the right way since this is my first .vapi file. It seems to work well.

[lp:~parnold-x/slingshot/conversion-plugin]

437. By Djax on 2014-08-15

replacement after split

438. By Djax on 2014-08-15

improved regex and codestyle thanks to jacobparker1992; print small and very big numbers scientifically

[Djax (parnold-x)]

Added improved regex match for the temperatures. Thanks to Jakob Parker.

For the record:

I wondered if we now exclude other units and the symbols C and F are assign to different units.
http://en.wikipedia.org/wiki/Coulomb
http://en.wikipedia.org/wiki/Farad

celsius is assigned to °C
fahrenheit to °F
and rankin to °R
only kelvin is assigned to K without the degree symbol.

But google does it also without a degree symbol so I think temperature conversion is used way more often and converting in Farad/Coulomb is still possible with the full names.
https://www.google.de/search?q=100C+to+F&oq=100C+to+F

[Jacob Parker (jacobparker1992)]

Don't kill me, but I changed the code style once again to match the other files in the plugins directory (which is apparently different to the elementary code style :P).

How are digits handled? We pick up '1.3 meters to yards', but I don't think we handle the funny countries that would write '1,3' instead. I'm not sure why we pick up '1.3' though---is that somehow part of the digit regex (JS considers \d to only be 0-9)?

- Jacob

[lp:~parnold-x/slingshot/conversion-plugin]

439. By Djax on 2014-08-16

allow , as digit seperator

[Djax (parnold-x)]

The other plugins are just imported from synapse, so no need to adapt to their code style.
Ah yeah, my "funny" country uses 1,3 but it is easy to fix.
"1.3 meters to yards". The match_regex splits just at " to " with no digit before the whitespace. So no problem with what digit is used.
In the temperature case there was a problem to match the digit in that case but I fixed that also.

[Jacob Parker (jacobparker1992)]

Any chance this could get another review?

[Djax (parnold-x)]

Note: Some superscripts look slightly missplaced. This is caused by the font size of the search result. With 12pt or greater everything looks ok.

[Danielle Foré (danrabbit)]

Bumping. Needs to be merged into trunk

review: Needs Fixing

Unmerged revisions

439. By Djax on 2014-08-16

allow , as digit seperator

438. By Djax on 2014-08-15

improved regex and codestyle thanks to jacobparker1992; print small and very big numbers scientifically

437. By Djax on 2014-08-15

replacement after split

436. By Djax on 2014-08-09

using libc strfmon function

435. By Djax on 2014-08-05

localisation and formating output

434. By Djax on 2014-08-05

reformating the output

433. By Djax on 2014-08-05

conversion word = to,as,in and formating the output

432. By Djax on 2014-08-03

add unit conversion-plugin

=== modified file 'lib/synapse-plugins/CMakeLists.txt'
--- lib/synapse-plugins/CMakeLists.txt	2014-06-29 14:50:36 +0000
+++ lib/synapse-plugins/CMakeLists.txt	2014-08-16 22:26:39 +0000
@@ -6,6 +6,8 @@
     gio-unix-2.0
     gee-0.8
     gtk+-3.0
+    libsoup-2.4
+    libxml-2.0
 )
 
 pkg_check_modules(PLUGINS_DEPS REQUIRED ${PLUGINS_PKG})
@@ -13,6 +15,7 @@
 set(PLUGINS_SOURCE
     calculator-plugin.vala
     command-plugin.vala
+    conversion-plugin.vala
     desktop-file-plugin.vala
 )
 
@@ -23,6 +26,8 @@
 PACKAGES
     ${PLUGINS_PKG}
     synapse-core
+CUSTOM_VAPIS
+    vapi/monetary.vapi
 OPTIONS
     --vapidir=${CMAKE_BINARY_DIR}/lib/synapse-core
 GENERATE_VAPI
@@ -43,5 +48,4 @@
     SOVERSION ${PLUGINS_LIB_SOVERSION}
 )
 
-target_link_libraries (${PLUGINS_LIBRARY_NAME} ${PLUGINS_DEPS_LIBRARIES} synapse-core)
-
+target_link_libraries (${PLUGINS_LIBRARY_NAME} ${PLUGINS_DEPS_LIBRARIES} synapse-core)
 \ No newline at end of file
 
=== added file 'lib/synapse-plugins/conversion-plugin.vala'
--- lib/synapse-plugins/conversion-plugin.vala	1970-01-01 00:00:00 +0000
+++ lib/synapse-plugins/conversion-plugin.vala	2014-08-16 22:26:39 +0000
@@ -0,0 +1,349 @@
+/*
+ * Copyright (C) 2014 Peter Arnold <[e-mail]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301  USA.
+ *
+ * Authored by Peter Arnold <[e-mail]>
+ *
+ */
+
+namespace Synapse
+{
+    errordomain IOError {
+        FILE_NOT_FOUND
+    }
+    
+    public class ConversionPlugin: Object, Activatable, ItemProvider {
+        public bool enabled { get; set; default = true; }
+        
+        public void activate () {}
+        
+        public void deactivate () {}
+        
+        private class Result: Object, Match {
+            // from Match interface
+            public string title { get; construct set; }
+            public string description { get; set; }
+            public string icon_name { get; construct set; }
+            public bool has_thumbnail { get; construct set; }
+            public string thumbnail_path { get; construct set; }
+            public MatchType match_type { get; construct set; }
+            
+            public int default_relevancy { get; set; default = 0; }
+            
+            public Result (string result) {
+                Object (match_type: MatchType.TEXT,
+                    title: result,
+                    description: result,
+                    has_thumbnail: false,
+                    icon_name: "accessories-calculator");
+            }
+        }
+        
+        static void register_plugin () {
+            DataSink.PluginRegistry.get_default ().register_plugin (
+                typeof (ConversionPlugin),
+                _ ("Conversion"),
+                _ ("Conversion of units."),
+                "accessories-calculator",
+                register_plugin,
+                Environment.find_program_in_path ("units") != null,
+                _ ("units is not installed")
+            );
+        }
+        
+        static construct {
+            register_plugin ();
+        }
+        
+        /* @ localization if you want use mutiple words, split them with a "|" between the words.
+        If it's just one word use "to". Don't use whitspaces before or after the words!
+        */
+        private const string CONVERSION_WORD = _("to|as|in");
+        private Regex match_regex;
+        private Regex temp_regex;
+        private Regex digit_regex;
+        
+        construct {
+            try {
+                /* regex to match the conversion words (to, as, in) with no digit before the whitespace in cases the conversion word
+                has the same name as a unit for example in and in (inch)
+                */
+                match_regex = new Regex (@"(?<=\\D)\\s*\\b(?:$CONVERSION_WORD)\\b\\s*",
+                    RegexCompileFlags.OPTIMIZE);
+                // regex to match the temperature units
+                temp_regex = new Regex ("(?i)(?<=[\\d\\s])[FCKR]\\b|fahrenheit|celsius|kelvin|rankine",
+                    RegexCompileFlags.OPTIMIZE);
+                digit_regex = new Regex ("(\\d*\\.?\\d*)",
+                    RegexCompileFlags.OPTIMIZE);
+            } catch (Error e) {
+                Utils.Logger.error (this, "Error creating regexp.");
+            }
+            
+            try {
+                // set a timer that wake ups every 6hours = 21600s to check if the currency rates should be updated
+                check_for_update.begin ();
+                Timeout.add_seconds_full (Priority.DEFAULT, 21600, () => {
+                    check_for_update.begin ();
+                    return true;
+                });
+            } catch (Error e) {
+                Utils.Logger.error (this, "Error updating the currency rates");
+            }
+            
+        }
+      
+        public bool handles_query (Query query) {
+            return (QueryFlags.ACTIONS in query.query_type);
+        }
+        
+        static bool regex_function (MatchInfo info, StringBuilder resource) {
+            string match = info.fetch (0).slice (0, 1).up ();
+            resource.append (@"temp$match ");            
+            return false;
+        }
+
+        public async ResultSet? search (Query query) throws SearchError {
+            string input = query.query_string;
+            bool matched = match_regex.match (input);
+            
+            if (!matched) {
+                query.check_cancellable ();
+                return null;
+            }
+
+            try {
+                input = input.replace ("²", "^2").replace ("³", "^3").replace ("°", "").replace(",",".");
+                
+                // to support the conversion of temperatures see http://www.gnu.org/software/units/manual/units.html#Temperature-Conversions
+                bool temparaturematch = temp_regex.match (input);
+                
+                if (temparaturematch)
+                    input = temp_regex.replace_eval (input, input.length, 0, RegexMatchFlags.NOTBOL, (RegexEvalCallback) regex_function);
+                
+                string[] inputsplit = match_regex.split (input);
+                
+                if (temparaturematch) {
+                    MatchInfo info;
+                    digit_regex.match (inputsplit[0], 0, out info);
+                    string digit = info.fetch (0);
+                    inputsplit[0] = inputsplit[0].replace (digit, "") + @"($digit)";
+                }
+                
+                Pid pid;
+                int read_fd, write_fd;
+                string[] argv = {"units", "-1", "-f", Environment.get_home_dir () + "/.local/share/units/definitions.units", inputsplit[0], inputsplit[1]};
+                string? solution = null;
+                
+                Process.spawn_async_with_pipes (null, argv, null,
+                    SpawnFlags.SEARCH_PATH,
+                    null, out pid, out write_fd, out read_fd);
+                
+                UnixInputStream read_stream = new UnixInputStream (read_fd, true);
+                DataInputStream bc_output = new DataInputStream (read_stream);
+                solution = yield bc_output.read_line_async (Priority.DEFAULT_IDLE, query.cancellable);
+    
+                if (solution != null && (solution.contains ("*") || temparaturematch)) {
+                    query.check_cancellable ();
+                    solution = solution.replace ("*", "");
+                    solution = solution.chug ();
+                    bool res = double.try_parse (solution);                    
+                    if (!res)
+                        return null;
+                    
+                    double d = double.parse (solution);
+                    string digit;                    
+                    // the smallest output of the strfmon function is 0.01 and > 10^11 is not really readable in float format
+                    // so print it scientifically in those cases
+                    if (d >= 0.01 && d < 999999999999.9) {
+                        char buffer[100];
+                        Monetary.strfmon(buffer, "%!n", d);
+                        digit = (string) buffer;
+                    } else 
+                        digit = print_scientifically (d);
+                    
+                    string unit = " " + inputsplit[1];
+                    unit = unit.replace (" tempK", " K").replace (" temp", " °");
+                    
+                    Result result = new Result (digit + unit);                    
+                    ResultSet results = new ResultSet ();
+                    results.add (result, Match.Score.AVERAGE);
+                    
+                    return results;
+                }
+            } catch (Error err) {
+                if (!query.is_cancelled ())
+                    warning ("%s", err.message);
+            }
+                 
+            return null;
+        }
+        
+        // prints the %g output of printf in "x 10⁷⁷" format
+        private string print_scientifically (double d) {
+            string digit = "%g".printf(d);
+            if (!digit.contains ("e"))
+                return digit;
+            string exponent = digit.substring (digit.last_index_of ("e")+2);
+            if (exponent.has_prefix ("0"))
+                exponent = exponent.replace ("0","");
+            exponent = exponent.replace ("0","⁰").replace ("1","¹").replace ("2","²").replace ("3","³").replace ("4","⁴").replace ("5","⁵")
+                          .replace ("6","⁶").replace ("7","⁷").replace ("8","⁸").replace ("9","⁹");
+            if (digit.substring (digit.last_index_of ("e")+1,1) == "-")
+                exponent = "⁻" + exponent;
+            return digit.substring (0,digit.last_index_of ("e")) + " x 10" + exponent;
+        }
+        
+        private async void check_for_update () {
+            var file = File.new_for_path (Environment.get_home_dir () + "/.local/share/units/currency.units");
+            
+            if (!file.query_exists ()) {
+                try {
+                    //copy data to user directory to be able to modifiy it without root rights
+                    var dir = File.new_for_path (Environment.get_home_dir () + "/.local/share/units/");
+                    if (!dir.query_exists ())
+                    dir.make_directory ();
+                    var definitions_file = File.new_for_path ("/usr/share/units/definitions.units");
+                    var target_definitions = File.new_for_path (Environment.get_home_dir()+"/.local/share/units/definitions.units");
+                    if (!target_definitions.query_exists ())
+                    definitions_file.copy (target_definitions, FileCopyFlags.NONE);
+                    var currency_file = File.new_for_path ("/usr/share/units/currency.units");
+                    currency_file.copy (file, FileCopyFlags.NONE);
+                } catch (Error e) {
+                    Utils.Logger.error (this, "Failed to copy /usr/share/units/definitions.units");
+                }
+            }
+            if (file.query_exists ()) {
+                try {
+                    var file_info = file.query_info ("*", FileQueryInfoFlags.NONE);
+                    var modification_date = new DateTime.from_timeval_local (file_info.get_modification_time ());
+                    var now = new DateTime.now_local ();
+                    // timegenie update their rates once a day
+                    if (now.get_day_of_year () != modification_date.get_day_of_year ())
+                        update (file, now);
+                } catch (Error e) {
+                    Utils.Logger.error (this, "Failed to update '%s'".printf (file.get_path ()));
+                }
+            }
+        }
+        
+        private void update (File file, DateTime now) throws Error {
+            file.delete ();
+            var dos = new DataOutputStream (file.create (FileCreateFlags.REPLACE_DESTINATION));
+            Array<string> codes;
+            Array<string> names;
+            Array<string> rates;
+            // load and write data from timegenie
+            load_currency_rates_data (out codes, out names, out rates);
+            write_currency_data (dos, codes, names, rates, now);
+        }
+        
+        private void write_currency_data (DataOutputStream dos, Array<string> codes, Array<string> names, Array<string> values, DateTime now) throws Error {
+            var builder = new StringBuilder ("# ISO Currency Codes\n\n");
+            
+            for (int i = 0; i < codes.length; i++)
+                builder.append ("%s%s%s\n".printf (codes.index (i), string.nfill (15, ' '), names.index (i)));
+            
+            builder.append ("\n# Currency exchange rates from Time Genie (www.timegenie.com) from %s\n\n".printf (now.format ("%F")));
+            
+            for (int i = 0; i < codes.length; i++)
+                builder.append ("%s%s%s\n".printf (names.index (i), string.nfill (30-names.index (i).length, ' '), values.index (i)));
+                
+            uint8[] data = builder.str.data;
+            long written = 0;
+            
+            while (written < data.length)
+                written += dos.write (data[written:data.length]);
+        }
+        
+        private void load_currency_rates_data (out Array<string> codes, out Array<string> names, out Array<string> values) throws IOError {
+            codes = new Array<string> ();
+            names = new Array<string> ();
+            values = new Array<string> ();
+            var tmp_values = new Array<string> ();
+            string url = "http://rss.timegenie.com/forex.xml";
+            
+            var session = new Soup.Session ();
+            var msg = new Soup.Message ("GET", url);
+            session.send_message (msg);
+            Xml.Doc* doc = Xml.Parser.parse_memory (
+            (string) (msg.response_body.data),
+            (int) (msg.response_body.length));
+            if (doc == null)
+                throw new IOError.FILE_NOT_FOUND("Failed to parse http://rss.timegenie.com/forex.xml");
+            Xml.Node* root = doc->get_root_element ();
+            if (root == null) {
+                delete doc;
+                throw new IOError.FILE_NOT_FOUND ("The xml file is empty");
+            }
+            int index_usd = -1;
+            int index_euro = -1;
+            int index = 0;
+            for (Xml.Node* iter = root->children; iter != null; iter = iter->next) {
+                if (iter->type == Xml.ElementType.ELEMENT_NODE) {
+                    if (iter->name == "data") {
+                        for (Xml.Node* iter2 = iter->children; iter2 != null; iter2 = iter2->next) {
+                            if (iter2->type == Xml.ElementType.ELEMENT_NODE) {
+                                switch (iter2->name) {
+                                    case "code":
+                                        var content = get_node_content (iter2);
+                                        if (content != "") {
+                                            if (content == "USD")
+                                                index_usd = index;
+                                            else if (content == "EUR")
+                                                index_euro = index;
+                                            codes.append_val (content);
+                                        }
+                                        break;
+                                    case "description":
+                                        var content = get_node_content (iter2);
+                                        if (content != "") {
+                                            if (content == "Anguilla (ECD)") content = "eastcaribbeandollar";
+                                            content = content.replace (" ", "").down ();
+                                            names.append_val (content);
+                                        }
+                                        break;
+                                    case "rate":
+                                        var content = get_node_content (iter2);
+                                        if (content != "")
+                                            tmp_values.append_val (content);
+                                        break;
+                                    default:
+                                        // do nothing
+                                        break;
+                                }
+                            }
+                        }
+                        index++;
+                    }
+                }
+            }
+            delete doc;
+            for (int i = 0; i < tmp_values.length; i++) {
+                if (i == index_euro) values.append_val (tmp_values.index (index_usd) + " US$");
+                else if (i == index_usd) values.append_val ("1 US$");
+                else values.append_val ("1|" + tmp_values.index (i) + " euro");
+            }
+        }
+        
+        private string get_node_content (Xml.Node* node) {
+            for (Xml.Node* iter = node->children; iter != null; iter = iter->next) {
+                if (iter->type == Xml.ElementType.TEXT_NODE)
+                    return iter->get_content ();
+            }
+            return "";
+        }
+    }
+}
 \ No newline at end of file
 
=== added directory 'lib/synapse-plugins/vapi'
=== added file 'lib/synapse-plugins/vapi/monetary.vapi'
--- lib/synapse-plugins/vapi/monetary.vapi	1970-01-01 00:00:00 +0000
+++ lib/synapse-plugins/vapi/monetary.vapi	2014-08-16 22:26:39 +0000
@@ -0,0 +1,6 @@
+[CCode (cheader_filename = "monetary.h")]
+public class Monetary {
+ [CCode(cname = "strfmon")]
+ public static ssize_t strfmon(char[] s, string format, double data);
+
+}
 \ No newline at end of file
 
=== modified file 'src/Backend/SynapseSearch.vala'
--- src/Backend/SynapseSearch.vala	2014-06-29 14:50:36 +0000
+++ src/Backend/SynapseSearch.vala	2014-08-16 22:26:39 +0000
@@ -23,6 +23,7 @@
         private static Type[] plugins = {
             typeof (Synapse.CalculatorPlugin),
             typeof (Synapse.CommandPlugin),
+            typeof (Synapse.ConversionPlugin),
             typeof (Synapse.DesktopFilePlugin)
         };
 
@@ -161,4 +162,3 @@
         }
     }
 }
-