Merge lp:~izidor/gtg/performance into lp:~gtg/gtg/old-trunk

Proposed by Izidor Matušov on 2012-05-03
Status: Merged
Merged at revision: 1177
Proposed branch: lp:~izidor/gtg/performance
Merge into: lp:~gtg/gtg/old-trunk
Diff against target: 319 lines (+59/-62) 10 files modified
To merge this branch: bzr merge lp:~izidor/gtg/performance
Reviewer Review Type Date Requested Status
Bertrand Rousseau (community) 2012-05-03 Approve on 2012-05-03
Review via email: mp+104525@code.launchpad.net

Description of the Change

I finally made my proposed performance optimization and hopefully, solve all new bugs I introduced. I did have to make changes to liblarch and there is a merge request as well: https://github.com/liblarch/liblarch/pull/2

How to test it? Run following commands:

mkdir fast-gtg && cd fast-gtg/
bzr branch lp:~izidor/gtg/performance gtg
git clone --branch=performance https://github.com/liblarch/liblarch liblarch
cd gtg/
./scripts/import_my_tasks_into_debug_tasks.sh
./scripts/debug.sh -l

Changes I made:
- added a new parameter -l which makes GTG to prefer a local installation of liblarch in ../liblarch rather than the system installation

- parameter -b works again! Run "time ./scripts/debug.sh -l -b" to test how fast your GTG boot up

- background colors are cached instead of recomputing every time. It makes GTG run smoother.

- quiet boot: GTG window is shown only when all tasks are loaded. By the size of your tasks, GTG would start in few seconds:

150 tasks: 4.6 seconds
My 450+ tasks: 10 seconds
Legendary Bryce set: 21 seconds

(The quiet boot should be okay because applications like Firefox takes some time to start too.)

- several improvement on side of liblarch

After merging this patch, GTG won't be perfectly fast. There is still room for improvement but it would make GTG more useable.

To post a comment you must log in.

Wow, the performance boost is amazing! It's great to have GTG feeling snappy! Congrats Izidor, you've done a great work there!

The quiet boot option is ok it's WAY better than having the tasks and tags appearing progressively, and still not be able to interact with GTG. I guess we could expect not to have more bryce's task count (I expect most people to have less than 100 tasks - btw, we could make a survey about this sometimes), so it should be allright. If it bothers people, there are ways to help with this (e.g. popup with loading bar, show GTG brower but make the lists inactive, etc.)

For me, your GTG patch is good to go!

review: Approve
Xuan (Sean) Hu (huxuan) wrote :

Really really fantastic!

I do see huge preformance improvement on my laptop with bryce's dataset.
Cause I even can not open bryce before (My laptop is a little old too) but now it's able to see all tasks even with tag sidebar and modification. It's sure to be more useable now.

A little problem is that there will be "* active tasks -" in the window title before "Getting Things GNOME!", but it's missing in this branch.

I looked into the code and found the window_title is updated by GTG.gtk.browser.browser._update_windows_title
And it's only called by callback of 'node-added-inview' and 'node-deleted-inview', maybe there need one more callback something like 'node-loaded-inview'

lp:~izidor/gtg/performance updated on 2012-05-04
1182. By Izidor Matušov on 2012-05-04

Refresh title on start

Izidor Matušov (izidor) wrote :

Xuan> thanks for the notice about the title! I updated code and the title is manually refreshed on startup - no regression anymore. Would anybody find other glitches?

Preview Diff

1=== modified file 'CHANGELOG'
2--- CHANGELOG 2012-04-22 15:16:24 +0000
3+++ CHANGELOG 2012-05-04 06:22:17 +0000
4@@ -18,6 +18,7 @@
5 * Reimplement the tag context menu as a widget, in order to - hopefully - be able to implement an enhanced context menu with, for instance, a color picker.
6 * Background Colors option was moved from View menu into preferences dialog
7 * Reorganised notification area menu (New task, Show browser, <tasks>, Quit)
8+ * Added an option to import local (development) version of liblarch instead of system installed one
9
10 2012-02-13 Getting Things GNOME! 0.2.9
11 * Big refractorization of code, now using liblarch
12
13=== modified file 'GTG/core/datastore.py'
14--- GTG/core/datastore.py 2012-03-22 14:19:18 +0000
15+++ GTG/core/datastore.py 2012-05-04 06:22:17 +0000
16@@ -39,7 +39,6 @@
17 from GTG.backends.genericbackend import GenericBackend
18 from GTG.tools import cleanxml
19 from GTG.backends.backendsignals import BackendSignals
20-from GTG.tools.synchronized import synchronized
21 from GTG.tools.borg import Borg
22 from GTG.core.search import parse_search_query, search_filter, InvalidQuery
23
24@@ -334,7 +333,6 @@
25 self.__tasks.add_node(task)
26 return task
27
28- @synchronized
29 def push_task(self, task):
30 '''
31 Adds the given task object to the task tree. In other words, registers
32@@ -647,19 +645,8 @@
33 self.to_set_timer = None
34
35 def start_get_tasks(self):
36- ''''
37- Maps the TaskSource to the backend and starts threading.
38- '''
39- self.start_get_tasks_thread = \
40- threading.Thread(target=self.__start_get_tasks)
41- self.start_get_tasks_thread.setDaemon(True)
42- self.start_get_tasks_thread.start()
43-
44- def __start_get_tasks(self):
45- '''
46- Loads all task from the backend and connects its signals afterwards.
47- Launched as a thread by start_get_tasks
48- '''
49+ """ Loads all task from the backend and connects its signals
50+ afterwards. """
51 self.backend.start_get_tasks()
52 self._connect_signals()
53 if self.backend.is_default():
54
55=== modified file 'GTG/core/plugins/api.py'
56--- GTG/core/plugins/api.py 2012-04-22 12:29:43 +0000
57+++ GTG/core/plugins/api.py 2012-05-04 06:22:17 +0000
58@@ -177,20 +177,19 @@
59 "%s" % e)
60
61 def set_bgcolor_func(self, func=None):
62- """ Set a function which defines a background color for each task """
63- # NOTE: this function is strongly dependend of browser structure
64- # after refaractoring browser, this might need to review
65+ """ Set a function which defines a background color for each task
66+
67+ NOTE: This function stronglye depend on browser and could be easily
68+ broken by changes in browser code
69+ """
70 browser = self.__ui
71
72 # set default bgcolor?
73 if func is None:
74 func = browser.tv_factory.task_bg_color
75- info_col = 'tags'
76- else:
77- info_col = 'task_id'
78
79 for pane in browser.vtree_panes.itervalues():
80- pane.set_bg_color(func, info_col)
81+ pane.set_bg_color(func, 'bg_color')
82 pane.basetree.get_basetree().refresh_all()
83
84 #=== file saving/loading ======================================================
85
86=== modified file 'GTG/gtk/browser/browser.py'
87--- GTG/gtk/browser/browser.py 2012-04-23 21:50:54 +0000
88+++ GTG/gtk/browser/browser.py 2012-05-04 06:22:17 +0000
89@@ -93,7 +93,7 @@
90
91 # Set up models
92 # Active Tasks
93- self.activetree.apply_filter('active',refresh=False)
94+ self.activetree.apply_filter('active')
95 # Tags
96 self.tagtree = None
97 self.tagtreeview = None
98@@ -131,6 +131,7 @@
99 #Update the title when a task change
100 self.activetree.register_cllbck('node-added-inview', self._update_window_title)
101 self.activetree.register_cllbck('node-deleted-inview', self._update_window_title)
102+ self._update_window_title()
103
104 ### INIT HELPER FUNCTIONS #####################################################
105 #
106@@ -209,6 +210,9 @@
107 self.on_tag_treeview_key_press_event)
108 self.sidebar_container.add(self.tagtreeview)
109
110+ # Refresh tree
111+ self.tagtree.reset_filters(transparent_only=True)
112+
113 # expanding search tag does not work automatically, request it
114 self.expand_search_tag()
115
116
117=== modified file 'GTG/gtk/browser/treeview_factory.py'
118--- GTG/gtk/browser/treeview_factory.py 2012-04-22 14:06:51 +0000
119+++ GTG/gtk/browser/treeview_factory.py 2012-05-04 06:22:17 +0000
120@@ -85,9 +85,9 @@
121 real_count = real_count + 1
122 return display_count < real_count
123
124- def task_bg_color(self,tags,bg):
125+ def task_bg_color(self, node, default_color):
126 if self.config.get('bg_color_enable'):
127- return colors.background_color(tags,bg)
128+ return colors.background_color(node.get_tags(), default_color)
129 else:
130 return None
131
132@@ -424,6 +424,13 @@
133 col['order'] = 0
134 desc[col_name] = col
135
136+ #invisible 'bg_color' column
137+ col_name = 'bg_color'
138+ col = {}
139+ col['value'] = [str, lambda node: None]
140+ col['visible'] = False
141+ desc[col_name] = col
142+
143 #invisible 'title' column
144 col_name = 'title'
145 col = {}
146@@ -463,6 +470,7 @@
147 desc[col_name] = col
148 return desc
149
150+
151 def build_task_treeview(self,tree,desc):
152 treeview = AutoExpandTreeView(tree,desc)
153 #Now that the treeview is done, we can polish
154@@ -470,7 +478,7 @@
155 treeview.set_expander_column('label')
156 treeview.set_dnd_name('gtg/task-iter-str')
157 #Background colors
158- treeview.set_bg_color(self.task_bg_color,'tags')
159+ treeview.set_bg_color(self.task_bg_color, 'bg_color')
160 # Global treeview properties
161 treeview.set_property("enable-tree-lines", False)
162 treeview.set_rules_hint(False)
163
164=== modified file 'GTG/plugins/urgency_color/urgency_color.py'
165--- GTG/plugins/urgency_color/urgency_color.py 2012-03-17 02:20:46 +0000
166+++ GTG/plugins/urgency_color/urgency_color.py 2012-05-04 06:22:17 +0000
167@@ -53,8 +53,7 @@
168 else:
169 return None
170
171- def bgcolor(self, node_id, standard_color):
172- node = self.req.get_task(node_id)
173+ def bgcolor(self, node, standard_color):
174 sdate = node.get_start_date()
175 ddate = node.get_due_date()
176 if (sdate != Date.no_date() != ddate):
177
178=== modified file 'GTG/tools/import_liblarch.py'
179--- GTG/tools/import_liblarch.py 2012-04-23 04:16:49 +0000
180+++ GTG/tools/import_liblarch.py 2012-05-04 06:22:17 +0000
181@@ -25,11 +25,11 @@
182 REQUIRED_LIBLARCH_API = "1.0"
183 GIT_CMD = "git clone https://github.com/liblarch/liblarch ../liblarch"
184
185-
186-def import_liblarch():
187+def import_liblarch(use_local=False):
188 """ Check if liblarch is installed and is compatible
189
190- If not, provide information how to obtain the newest version """
191+ If not, provide information how to obtain the newest version.
192+ If use_local, prioritize local (development) liblarch in ../liblarch"""
193
194 def check_liblarch():
195 """ Import liblarch and find out which one is missing """
196@@ -49,8 +49,12 @@
197
198 return has_libraries, " and ".join(missing)
199
200+ if use_local:
201+ sys.path.insert(0, "../liblarch")
202+
203 has_libraries, missing = check_liblarch()
204- if not has_libraries:
205+
206+ if not use_local and not has_libraries:
207 sys.path.append("../liblarch/")
208 has_libraries, missing = check_liblarch()
209
210@@ -83,7 +87,9 @@
211 return True
212
213 if __name__ == "__main__":
214- if import_liblarch():
215+ use_local = "-l" in sys.argv[1:] or "--local-liblarch" in sys.argv[1:]
216+
217+ if import_liblarch(use_local):
218 sys.exit(0)
219 else:
220 sys.exit(1)
221
222=== removed file 'GTG/tools/synchronized.py'
223--- GTG/tools/synchronized.py 2012-03-05 15:23:05 +0000
224+++ GTG/tools/synchronized.py 1970-01-01 00:00:00 +0000
225@@ -1,14 +0,0 @@
226-from __future__ import with_statement
227-from threading import Lock
228-
229-def synchronized(fun):
230- the_lock = Lock()
231-
232- def fwrap(function):
233- def newFunction(*args, **kw):
234- with the_lock:
235- return function(*args, **kw)
236-
237- return newFunction
238-
239- return fwrap(fun)
240
241=== modified file 'gtg'
242--- gtg 2012-04-30 05:56:11 +0000
243+++ gtg 2012-05-04 06:22:17 +0000
244@@ -50,6 +50,9 @@
245 default=False)
246 parser.add_option('-d', '--debug', action='store_true', dest='debug',
247 help="Enable debug output", default=False)
248+ parser.add_option('-l', '--local-liblarch', action='store_true',
249+ dest='local_liblarch', default=False,
250+ help="Use local liblarch located in ../liblarch if it is posible")
251 parser.add_option('-v', '--version', action='store_true',
252 dest='print_version', help="Print GTG's version number", default=False)
253 (options, args) = parser.parse_args()
254@@ -64,7 +67,7 @@
255 print "Could not open X display"
256 sys.exit(1)
257
258- elif import_liblarch():
259+ elif import_liblarch(options.local_liblarch):
260 from GTG import gtg
261 sys.exit(gtg.main(options, args))
262
263
264=== modified file 'scripts/debug.sh'
265--- scripts/debug.sh 2012-03-26 17:43:15 +0000
266+++ scripts/debug.sh 2012-05-04 06:22:17 +0000
267@@ -15,14 +15,18 @@
268 mkdir -p tmp
269
270 # Interpret arguments
271-while getopts bdnps: o
272+while getopts bdlnps: o
273 do case "$o" in
274 b) args="$args --boot-test";;
275 d) args="$args -d";;
276+ # Request usage local liblarch if it is possible
277+ l) args="$args -l"
278+ liblarchArgs="$liblarchArgs -l"
279+ ;;
280 n) norun=1;;
281 p) profile=1;;
282 s) set="$OPTARG";;
283- [?]) echo >&2 "Usage: $0 [-s dataset] [-b] [-d] [-n] [-p]"
284+ [?]) echo >&2 "Usage: $0 [-s dataset] [-b] [-d] [-l] [-n] [-p]"
285 exit 1;;
286 esac
287 done
288@@ -46,19 +50,19 @@
289 export XDG_CONFIG_HOME="./tmp/default/xdg/config"
290 fi
291
292-# Check for liblarch
293-if ! ./GTG/tools/import_liblarch.py ; then
294- echo
295- echo -n "Download latest liblarch? [y/N] "
296- read answer
297- if [ "$answer" = "y" -o "$answer" = "Y" -o "$answer" = "yes" ]; then
298- git clone https://github.com/liblarch/liblarch ../liblarch
299- else
300- exit 1
301- fi
302-fi
303-
304 if [ $norun -eq 0 ]; then
305+ # Check for liblarch
306+ if ! ./GTG/tools/import_liblarch.py $liblarchArgs; then
307+ echo
308+ echo -n "Download latest liblarch? [y/N] "
309+ read answer
310+ if [ "$answer" = "y" -o "$answer" = "Y" -o "$answer" = "yes" ]; then
311+ git clone https://github.com/liblarch/liblarch ../liblarch
312+ else
313+ exit 1
314+ fi
315+ fi
316+
317 if [ $profile -eq 1 ]; then
318 python -m cProfile -o gtg.prof ./gtg $args
319 python ./scripts/profile_interpret.sh

Subscribers

People subscribed via source and target branches

to status/vote changes: