GTG

Merge lp:~izidor/gtg/performance into lp:~gtg/gtg/old-trunk

Proposed by Izidor Matušov
Status: Merged
Merged at revision: 1177
Proposed branch: lp:~izidor/gtg/performance
Merge into: lp:~gtg/gtg/old-trunk
Diff against target: 319 lines (+59/-62)
10 files modified
CHANGELOG (+1/-0)
GTG/core/datastore.py (+2/-15)
GTG/core/plugins/api.py (+6/-7)
GTG/gtk/browser/browser.py (+5/-1)
GTG/gtk/browser/treeview_factory.py (+11/-3)
GTG/plugins/urgency_color/urgency_color.py (+1/-2)
GTG/tools/import_liblarch.py (+11/-5)
GTG/tools/synchronized.py (+0/-14)
gtg (+4/-1)
scripts/debug.sh (+18/-14)
To merge this branch: bzr merge lp:~izidor/gtg/performance
Reviewer Review Type Date Requested Status
Bertrand Rousseau (community) Approve
Review via email: mp+104525@code.launchpad.net

Description of the change

I finally made my proposed performance optimization and hopefully, solve all new bugs I introduced. I did have to make changes to liblarch and there is a merge request as well: https://github.com/liblarch/liblarch/pull/2

How to test it? Run following commands:

mkdir fast-gtg && cd fast-gtg/
bzr branch lp:~izidor/gtg/performance gtg
git clone --branch=performance https://github.com/liblarch/liblarch liblarch
cd gtg/
./scripts/import_my_tasks_into_debug_tasks.sh
./scripts/debug.sh -l

Changes I made:
- added a new parameter -l which makes GTG to prefer a local installation of liblarch in ../liblarch rather than the system installation

- parameter -b works again! Run "time ./scripts/debug.sh -l -b" to test how fast your GTG boot up

- background colors are cached instead of recomputing every time. It makes GTG run smoother.

- quiet boot: GTG window is shown only when all tasks are loaded. By the size of your tasks, GTG would start in few seconds:

150 tasks: 4.6 seconds
My 450+ tasks: 10 seconds
Legendary Bryce set: 21 seconds

(The quiet boot should be okay because applications like Firefox takes some time to start too.)

- several improvement on side of liblarch

After merging this patch, GTG won't be perfectly fast. There is still room for improvement but it would make GTG more useable.

To post a comment you must log in.
Revision history for this message
Bertrand Rousseau (bertrand-rousseau) wrote :

Wow, the performance boost is amazing! It's great to have GTG feeling snappy! Congrats Izidor, you've done a great work there!

The quiet boot option is ok it's WAY better than having the tasks and tags appearing progressively, and still not be able to interact with GTG. I guess we could expect not to have more bryce's task count (I expect most people to have less than 100 tasks - btw, we could make a survey about this sometimes), so it should be allright. If it bothers people, there are ways to help with this (e.g. popup with loading bar, show GTG brower but make the lists inactive, etc.)

For me, your GTG patch is good to go!

review: Approve
Revision history for this message
Xuan (Sean) Hu (huxuan) wrote :

Really really fantastic!

I do see huge preformance improvement on my laptop with bryce's dataset.
Cause I even can not open bryce before (My laptop is a little old too) but now it's able to see all tasks even with tag sidebar and modification. It's sure to be more useable now.

A little problem is that there will be "* active tasks -" in the window title before "Getting Things GNOME!", but it's missing in this branch.

I looked into the code and found the window_title is updated by GTG.gtk.browser.browser._update_windows_title
And it's only called by callback of 'node-added-inview' and 'node-deleted-inview', maybe there need one more callback something like 'node-loaded-inview'

lp:~izidor/gtg/performance updated
1182. By Izidor Matušov

Refresh title on start

Revision history for this message
Izidor Matušov (izidor) wrote :

Xuan> thanks for the notice about the title! I updated code and the title is manually refreshed on startup - no regression anymore. Would anybody find other glitches?

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'CHANGELOG'
--- CHANGELOG 2012-04-22 15:16:24 +0000
+++ CHANGELOG 2012-05-04 06:22:17 +0000
@@ -18,6 +18,7 @@
18 * Reimplement the tag context menu as a widget, in order to - hopefully - be able to implement an enhanced context menu with, for instance, a color picker.18 * Reimplement the tag context menu as a widget, in order to - hopefully - be able to implement an enhanced context menu with, for instance, a color picker.
19 * Background Colors option was moved from View menu into preferences dialog19 * Background Colors option was moved from View menu into preferences dialog
20 * Reorganised notification area menu (New task, Show browser, <tasks>, Quit)20 * Reorganised notification area menu (New task, Show browser, <tasks>, Quit)
21 * Added an option to import local (development) version of liblarch instead of system installed one
2122
222012-02-13 Getting Things GNOME! 0.2.9232012-02-13 Getting Things GNOME! 0.2.9
23 * Big refractorization of code, now using liblarch24 * Big refractorization of code, now using liblarch
2425
=== modified file 'GTG/core/datastore.py'
--- GTG/core/datastore.py 2012-03-22 14:19:18 +0000
+++ GTG/core/datastore.py 2012-05-04 06:22:17 +0000
@@ -39,7 +39,6 @@
39from GTG.backends.genericbackend import GenericBackend39from GTG.backends.genericbackend import GenericBackend
40from GTG.tools import cleanxml40from GTG.tools import cleanxml
41from GTG.backends.backendsignals import BackendSignals41from GTG.backends.backendsignals import BackendSignals
42from GTG.tools.synchronized import synchronized
43from GTG.tools.borg import Borg42from GTG.tools.borg import Borg
44from GTG.core.search import parse_search_query, search_filter, InvalidQuery43from GTG.core.search import parse_search_query, search_filter, InvalidQuery
4544
@@ -334,7 +333,6 @@
334 self.__tasks.add_node(task)333 self.__tasks.add_node(task)
335 return task334 return task
336335
337 @synchronized
338 def push_task(self, task):336 def push_task(self, task):
339 '''337 '''
340 Adds the given task object to the task tree. In other words, registers338 Adds the given task object to the task tree. In other words, registers
@@ -647,19 +645,8 @@
647 self.to_set_timer = None645 self.to_set_timer = None
648646
649 def start_get_tasks(self):647 def start_get_tasks(self):
650 ''''648 """ Loads all task from the backend and connects its signals
651 Maps the TaskSource to the backend and starts threading.649 afterwards. """
652 '''
653 self.start_get_tasks_thread = \
654 threading.Thread(target=self.__start_get_tasks)
655 self.start_get_tasks_thread.setDaemon(True)
656 self.start_get_tasks_thread.start()
657
658 def __start_get_tasks(self):
659 '''
660 Loads all task from the backend and connects its signals afterwards.
661 Launched as a thread by start_get_tasks
662 '''
663 self.backend.start_get_tasks()650 self.backend.start_get_tasks()
664 self._connect_signals()651 self._connect_signals()
665 if self.backend.is_default():652 if self.backend.is_default():
666653
=== modified file 'GTG/core/plugins/api.py'
--- GTG/core/plugins/api.py 2012-04-22 12:29:43 +0000
+++ GTG/core/plugins/api.py 2012-05-04 06:22:17 +0000
@@ -177,20 +177,19 @@
177 "%s" % e)177 "%s" % e)
178178
179 def set_bgcolor_func(self, func=None):179 def set_bgcolor_func(self, func=None):
180 """ Set a function which defines a background color for each task """180 """ Set a function which defines a background color for each task
181 # NOTE: this function is strongly dependend of browser structure181
182 # after refaractoring browser, this might need to review182 NOTE: This function stronglye depend on browser and could be easily
183 broken by changes in browser code
184 """
183 browser = self.__ui185 browser = self.__ui
184186
185 # set default bgcolor?187 # set default bgcolor?
186 if func is None:188 if func is None:
187 func = browser.tv_factory.task_bg_color189 func = browser.tv_factory.task_bg_color
188 info_col = 'tags'
189 else:
190 info_col = 'task_id'
191190
192 for pane in browser.vtree_panes.itervalues():191 for pane in browser.vtree_panes.itervalues():
193 pane.set_bg_color(func, info_col)192 pane.set_bg_color(func, 'bg_color')
194 pane.basetree.get_basetree().refresh_all()193 pane.basetree.get_basetree().refresh_all()
195194
196#=== file saving/loading ======================================================195#=== file saving/loading ======================================================
197196
=== modified file 'GTG/gtk/browser/browser.py'
--- GTG/gtk/browser/browser.py 2012-04-23 21:50:54 +0000
+++ GTG/gtk/browser/browser.py 2012-05-04 06:22:17 +0000
@@ -93,7 +93,7 @@
9393
94 # Set up models94 # Set up models
95 # Active Tasks95 # Active Tasks
96 self.activetree.apply_filter('active',refresh=False)96 self.activetree.apply_filter('active')
97 # Tags97 # Tags
98 self.tagtree = None98 self.tagtree = None
99 self.tagtreeview = None99 self.tagtreeview = None
@@ -131,6 +131,7 @@
131 #Update the title when a task change131 #Update the title when a task change
132 self.activetree.register_cllbck('node-added-inview', self._update_window_title)132 self.activetree.register_cllbck('node-added-inview', self._update_window_title)
133 self.activetree.register_cllbck('node-deleted-inview', self._update_window_title)133 self.activetree.register_cllbck('node-deleted-inview', self._update_window_title)
134 self._update_window_title()
134135
135### INIT HELPER FUNCTIONS #####################################################136### INIT HELPER FUNCTIONS #####################################################
136#137#
@@ -209,6 +210,9 @@
209 self.on_tag_treeview_key_press_event)210 self.on_tag_treeview_key_press_event)
210 self.sidebar_container.add(self.tagtreeview)211 self.sidebar_container.add(self.tagtreeview)
211212
213 # Refresh tree
214 self.tagtree.reset_filters(transparent_only=True)
215
212 # expanding search tag does not work automatically, request it216 # expanding search tag does not work automatically, request it
213 self.expand_search_tag()217 self.expand_search_tag()
214218
215219
=== modified file 'GTG/gtk/browser/treeview_factory.py'
--- GTG/gtk/browser/treeview_factory.py 2012-04-22 14:06:51 +0000
+++ GTG/gtk/browser/treeview_factory.py 2012-05-04 06:22:17 +0000
@@ -85,9 +85,9 @@
85 real_count = real_count + 185 real_count = real_count + 1
86 return display_count < real_count86 return display_count < real_count
87 87
88 def task_bg_color(self,tags,bg):88 def task_bg_color(self, node, default_color):
89 if self.config.get('bg_color_enable'):89 if self.config.get('bg_color_enable'):
90 return colors.background_color(tags,bg)90 return colors.background_color(node.get_tags(), default_color)
91 else:91 else:
92 return None92 return None
93 93
@@ -424,6 +424,13 @@
424 col['order'] = 0424 col['order'] = 0
425 desc[col_name] = col425 desc[col_name] = col
426426
427 #invisible 'bg_color' column
428 col_name = 'bg_color'
429 col = {}
430 col['value'] = [str, lambda node: None]
431 col['visible'] = False
432 desc[col_name] = col
433
427 #invisible 'title' column434 #invisible 'title' column
428 col_name = 'title'435 col_name = 'title'
429 col = {}436 col = {}
@@ -463,6 +470,7 @@
463 desc[col_name] = col470 desc[col_name] = col
464 return desc471 return desc
465 472
473
466 def build_task_treeview(self,tree,desc):474 def build_task_treeview(self,tree,desc):
467 treeview = AutoExpandTreeView(tree,desc)475 treeview = AutoExpandTreeView(tree,desc)
468 #Now that the treeview is done, we can polish476 #Now that the treeview is done, we can polish
@@ -470,7 +478,7 @@
470 treeview.set_expander_column('label')478 treeview.set_expander_column('label')
471 treeview.set_dnd_name('gtg/task-iter-str')479 treeview.set_dnd_name('gtg/task-iter-str')
472 #Background colors480 #Background colors
473 treeview.set_bg_color(self.task_bg_color,'tags')481 treeview.set_bg_color(self.task_bg_color, 'bg_color')
474 # Global treeview properties482 # Global treeview properties
475 treeview.set_property("enable-tree-lines", False)483 treeview.set_property("enable-tree-lines", False)
476 treeview.set_rules_hint(False)484 treeview.set_rules_hint(False)
477485
=== modified file 'GTG/plugins/urgency_color/urgency_color.py'
--- GTG/plugins/urgency_color/urgency_color.py 2012-03-17 02:20:46 +0000
+++ GTG/plugins/urgency_color/urgency_color.py 2012-05-04 06:22:17 +0000
@@ -53,8 +53,7 @@
53 else:53 else:
54 return None54 return None
5555
56 def bgcolor(self, node_id, standard_color):56 def bgcolor(self, node, standard_color):
57 node = self.req.get_task(node_id)
58 sdate = node.get_start_date()57 sdate = node.get_start_date()
59 ddate = node.get_due_date()58 ddate = node.get_due_date()
60 if (sdate != Date.no_date() != ddate):59 if (sdate != Date.no_date() != ddate):
6160
=== modified file 'GTG/tools/import_liblarch.py'
--- GTG/tools/import_liblarch.py 2012-04-23 04:16:49 +0000
+++ GTG/tools/import_liblarch.py 2012-05-04 06:22:17 +0000
@@ -25,11 +25,11 @@
25REQUIRED_LIBLARCH_API = "1.0"25REQUIRED_LIBLARCH_API = "1.0"
26GIT_CMD = "git clone https://github.com/liblarch/liblarch ../liblarch"26GIT_CMD = "git clone https://github.com/liblarch/liblarch ../liblarch"
2727
2828def import_liblarch(use_local=False):
29def import_liblarch():
30 """ Check if liblarch is installed and is compatible29 """ Check if liblarch is installed and is compatible
3130
32 If not, provide information how to obtain the newest version """31 If not, provide information how to obtain the newest version.
32 If use_local, prioritize local (development) liblarch in ../liblarch"""
3333
34 def check_liblarch():34 def check_liblarch():
35 """ Import liblarch and find out which one is missing """35 """ Import liblarch and find out which one is missing """
@@ -49,8 +49,12 @@
4949
50 return has_libraries, " and ".join(missing)50 return has_libraries, " and ".join(missing)
5151
52 if use_local:
53 sys.path.insert(0, "../liblarch")
54
52 has_libraries, missing = check_liblarch()55 has_libraries, missing = check_liblarch()
53 if not has_libraries:56
57 if not use_local and not has_libraries:
54 sys.path.append("../liblarch/")58 sys.path.append("../liblarch/")
55 has_libraries, missing = check_liblarch()59 has_libraries, missing = check_liblarch()
5660
@@ -83,7 +87,9 @@
83 return True87 return True
8488
85if __name__ == "__main__":89if __name__ == "__main__":
86 if import_liblarch():90 use_local = "-l" in sys.argv[1:] or "--local-liblarch" in sys.argv[1:]
91
92 if import_liblarch(use_local):
87 sys.exit(0)93 sys.exit(0)
88 else:94 else:
89 sys.exit(1)95 sys.exit(1)
9096
=== removed file 'GTG/tools/synchronized.py'
--- GTG/tools/synchronized.py 2012-03-05 15:23:05 +0000
+++ GTG/tools/synchronized.py 1970-01-01 00:00:00 +0000
@@ -1,14 +0,0 @@
1from __future__ import with_statement
2from threading import Lock
3
4def synchronized(fun):
5 the_lock = Lock()
6
7 def fwrap(function):
8 def newFunction(*args, **kw):
9 with the_lock:
10 return function(*args, **kw)
11
12 return newFunction
13
14 return fwrap(fun)
150
=== modified file 'gtg'
--- gtg 2012-04-30 05:56:11 +0000
+++ gtg 2012-05-04 06:22:17 +0000
@@ -50,6 +50,9 @@
50 default=False)50 default=False)
51 parser.add_option('-d', '--debug', action='store_true', dest='debug',51 parser.add_option('-d', '--debug', action='store_true', dest='debug',
52 help="Enable debug output", default=False)52 help="Enable debug output", default=False)
53 parser.add_option('-l', '--local-liblarch', action='store_true',
54 dest='local_liblarch', default=False,
55 help="Use local liblarch located in ../liblarch if it is posible")
53 parser.add_option('-v', '--version', action='store_true',56 parser.add_option('-v', '--version', action='store_true',
54 dest='print_version', help="Print GTG's version number", default=False)57 dest='print_version', help="Print GTG's version number", default=False)
55 (options, args) = parser.parse_args()58 (options, args) = parser.parse_args()
@@ -64,7 +67,7 @@
64 print "Could not open X display"67 print "Could not open X display"
65 sys.exit(1)68 sys.exit(1)
6669
67 elif import_liblarch():70 elif import_liblarch(options.local_liblarch):
68 from GTG import gtg71 from GTG import gtg
69 sys.exit(gtg.main(options, args))72 sys.exit(gtg.main(options, args))
7073
7174
=== modified file 'scripts/debug.sh'
--- scripts/debug.sh 2012-03-26 17:43:15 +0000
+++ scripts/debug.sh 2012-05-04 06:22:17 +0000
@@ -15,14 +15,18 @@
15mkdir -p tmp15mkdir -p tmp
1616
17# Interpret arguments17# Interpret arguments
18while getopts bdnps: o18while getopts bdlnps: o
19do case "$o" in19do case "$o" in
20 b) args="$args --boot-test";;20 b) args="$args --boot-test";;
21 d) args="$args -d";;21 d) args="$args -d";;
22 # Request usage local liblarch if it is possible
23 l) args="$args -l"
24 liblarchArgs="$liblarchArgs -l"
25 ;;
22 n) norun=1;;26 n) norun=1;;
23 p) profile=1;;27 p) profile=1;;
24 s) set="$OPTARG";;28 s) set="$OPTARG";;
25 [?]) echo >&2 "Usage: $0 [-s dataset] [-b] [-d] [-n] [-p]"29 [?]) echo >&2 "Usage: $0 [-s dataset] [-b] [-d] [-l] [-n] [-p]"
26 exit 1;;30 exit 1;;
27 esac31 esac
28done32done
@@ -46,19 +50,19 @@
46 export XDG_CONFIG_HOME="./tmp/default/xdg/config"50 export XDG_CONFIG_HOME="./tmp/default/xdg/config"
47fi51fi
4852
49# Check for liblarch
50if ! ./GTG/tools/import_liblarch.py ; then
51 echo
52 echo -n "Download latest liblarch? [y/N] "
53 read answer
54 if [ "$answer" = "y" -o "$answer" = "Y" -o "$answer" = "yes" ]; then
55 git clone https://github.com/liblarch/liblarch ../liblarch
56 else
57 exit 1
58 fi
59fi
60
61if [ $norun -eq 0 ]; then53if [ $norun -eq 0 ]; then
54 # Check for liblarch
55 if ! ./GTG/tools/import_liblarch.py $liblarchArgs; then
56 echo
57 echo -n "Download latest liblarch? [y/N] "
58 read answer
59 if [ "$answer" = "y" -o "$answer" = "Y" -o "$answer" = "yes" ]; then
60 git clone https://github.com/liblarch/liblarch ../liblarch
61 else
62 exit 1
63 fi
64 fi
65
62 if [ $profile -eq 1 ]; then66 if [ $profile -eq 1 ]; then
63 python -m cProfile -o gtg.prof ./gtg $args67 python -m cProfile -o gtg.prof ./gtg $args
64 python ./scripts/profile_interpret.sh68 python ./scripts/profile_interpret.sh

Subscribers

People subscribed via source and target branches

to status/vote changes: