Merge lp:~elopio/autopilot/fix1257055-slow_drag into lp:autopilot

Proposed by Leo Arias
Status: Merged
Approved by: Thomi Richards
Approved revision: 435
Merged at revision: 440
Proposed branch: lp:~elopio/autopilot/fix1257055-slow_drag
Merge into: lp:autopilot
Prerequisite: lp:~elopio/autopilot/no_uinput_side-effects
Diff against target: 418 lines (+280/-28)
5 files modified
autopilot/input/_X11.py (+19/-4)
autopilot/input/__init__.py (+63/-7)
autopilot/input/_uinput.py (+39/-14)
autopilot/tests/functional/test_input_stack.py (+2/-1)
autopilot/tests/unit/test_input.py (+157/-2)
To merge this branch: bzr merge lp:~elopio/autopilot/fix1257055-slow_drag
Reviewer Review Type Date Requested Status
Thomi Richards (community) Approve
PS Jenkins bot continuous-integration Approve
Leo Arias Pending
Review via email: mp+205702@code.launchpad.net

This proposal supersedes a proposal from 2014-01-19.

Commit message

Added Mouse, Touch and Pointer drags with rate.

To post a comment you must log in.
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote : Posted in a previous version of this proposal
review: Needs Fixing (continuous-integration)
Revision history for this message
Thomi Richards (thomir-deactivatedaccount) wrote : Posted in a previous version of this proposal

Hi,

I'm not sure if this MP is still relevant. If it's not, please delete the proposal to merge. If it is, then:

39 - def drag(self, x1, y1, x2, y2):
40 + def drag(self, x1, y1, x2, y2, rate=10):

If you're adding (or removing, or changing) a parameter to a public method, please document it in the docstrings. What exactly is 'rate' measured in anyway?

82 - def drag(self, x1, y1, x2, y2):
83 + def drag(self, x1, y1, x2, y2, rate=10):

You introduce a new parameter, and then don't use it at all in the method.

In general this MP makes me feel rather uneasy. You're introducing a new parameter, and some code changes, and I can't see any justification for it.

If this is really needed, I think it might be worth moving the algorithm to a common location, and making both the X11 and uinput backends be able to use it. That way, you can test it without needing to mock out anything.

Anyway, please talk to me on IRC about this. We need to either clean it up and merge it, or remove it from the review queue.

review: Needs Fixing
Revision history for this message
Thomi Richards (thomir-deactivatedaccount) wrote : Posted in a previous version of this proposal

Hi,

In order to make our review queue a little more sane, I'm setting this to WIP. If/when you need a new review, please set it back to 'Needs Review', and (optionally) ping someone on the AP team.

Thanks.

Revision history for this message
Leo Arias (elopio) wrote :

What exactly is 'rate' measured in anyway?

It's the number of pixels you advance per iteration. This is really confusing for me, I spent a couple of hours deciphering the algorithm; but unfortunately, that's what you chose for Mouse API. I would prefer something like the number of seconds that the drag will take, but at this point we can't change the Mouse API and the best we can do is to make the Touch API similar.

Revision history for this message
Leo Arias (elopio) wrote :

> If you're adding (or removing, or changing) a parameter to a public method, please document it in the docstrings.

I did my best. Please check it.

> In general this MP makes me feel rather uneasy. You're introducing a new parameter, and some code changes, and I can't see any justification for it.

The justification is on the linked bug. We need a way to slow down the drags. If we do them with the current hard-coded rate, when we try to make a new item visible on a list, we end up on the bottom of the list instead.

> If this is really needed, I think it might be worth moving the algorithm to a common location, and making both the X11 and uinput backends be able to use it. That way, you can test it without needing to mock out anything.

That's not actually possible at the moment, because the X11 relies on the move method, and so, it uses the current position of the pointer. On Touch, we don't have a current position, so there's no clean way to do a slow drag. Of course, I plan to keep cleaning things up, and that's something I will try in the following weeks.

But for now, I am losing my head with the number of workarounds I have to do on every place where the drag fails. This is an improvement from the current status, that has a hard-coded value of 100 steps per drag on the touch and 10 pixels per iteration on the mouse, which is inconsistent and comes without tests.

Lets talk tomorrow on IRC.

For now, I'll update the tests to use the new testability that comes from the branch I've added as a prerequisite.

Revision history for this message
Leo Arias (elopio) wrote :

Sorry, where it says: so there's no clean way to do a slow drag
It should say: so there's no clean way to do a slow move

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
434. By Leo Arias

We can't use the real Mouse, so switch to touch backend for now.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
435. By Leo Arias

Updated the fake.

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Needs Fixing (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Revision history for this message
Thomi Richards (thomir-deactivatedaccount) wrote :

Hi,

What's confusing me is that the new parameter 'rate' isn't used in this code:

193 + current_x, current_y = x1, y1
194 + while current_x != x2 or current_y != y2:
195 + dx = abs(x2 - current_x)
196 + dy = abs(y2 - current_y)
197 +
198 + intx = float(dx) / max(dx, dy)
199 + inty = float(dy) / max(dx, dy)
200 +
201 + step_x = min(rate * intx, dx)
202 + step_y = min(rate * inty, dy)
203 +
204 + if x2 < current_x:
205 + step_x *= -1
206 + if y2 < current_y:
207 + step_y *= -1
208 +
209 + current_x += step_x
210 + current_y += step_y
211 + self._device.finger_move(current_x, current_y)
212 +
213 + sleep(time_between_events)

So I don't see the point in adding the parameter if we're not going to use it.

review: Needs Information
Revision history for this message
Leo Arias (elopio) wrote :

201 + step_x = min(rate * intx, dx)
202 + step_y = min(rate * inty, dy)

I'm using it!

Actually, you are using it, as most of this code comes from X11 move. Here we are making sure that if the distance is smaller than the rate (which happens on the last iteration), we will make a move that will get us to the destination point and not farther. I added tests for this condition.

Revision history for this message
Thomi Richards (thomir-deactivatedaccount) wrote :

huh, sorry, I must be blind.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'autopilot/input/_X11.py'
2--- autopilot/input/_X11.py 2013-11-07 05:53:36 +0000
3+++ autopilot/input/_X11.py 2014-02-11 12:23:36 +0000
4@@ -1,7 +1,7 @@
5 # -*- Mode: Python; coding: utf-8; indent-tabs-mode: nil; tab-width: 4 -*-
6 #
7 # Autopilot Functional Test Tool
8-# Copyright (C) 2012-2013 Canonical
9+# Copyright (C) 2012, 2013, 2014 Canonical
10 #
11 # This program is free software: you can redistribute it and/or modify
12 # it under the terms of the GNU General Public License as published by
13@@ -455,16 +455,31 @@
14 x, y = coord["root_x"], coord["root_y"]
15 return x, y
16
17- def drag(self, x1, y1, x2, y2):
18- """Performs a press, move and release.
19+ def drag(self, x1, y1, x2, y2, rate=10, time_between_events=0.01):
20+ """Perform a press, move and release.
21
22 This is to keep a common API between Mouse and Finger as long as
23 possible.
24
25+ The pointer will be dragged from the starting point to the ending point
26+ with multiple moves. The number of moves, and thus the time that it
27+ will take to complete the drag can be altered with the `rate`
28+ parameter.
29+
30+ :param x1: The point on the x axis where the drag will start from.
31+ :param y1: The point on the y axis where the drag will starts from.
32+ :param x2: The point on the x axis where the drag will end at.
33+ :param y2: The point on the y axis where the drag will end at.
34+ :param rate: The number of pixels the mouse will be moved per
35+ iteration. Default is 10 pixels. A higher rate will make the drag
36+ faster, and lower rate will make it slower.
37+ :param time_between_events: The number of seconds that the drag will
38+ wait between iterations.
39+
40 """
41 self.move(x1, y1)
42 self.press()
43- self.move(x2, y2)
44+ self.move(x2, y2, rate=rate, time_between_events=time_between_events)
45 self.release()
46
47 @classmethod
48
49=== modified file 'autopilot/input/__init__.py'
50--- autopilot/input/__init__.py 2013-09-20 19:01:27 +0000
51+++ autopilot/input/__init__.py 2014-02-11 12:23:36 +0000
52@@ -369,12 +369,27 @@
53 """
54 raise NotImplementedError("You cannot use this class directly.")
55
56- def drag(self, x1, y1, x2, y2):
57- """Performs a press, move and release.
58+ def drag(self, x1, y1, x2, y2, rate=10, time_between_events=0.01):
59+ """Perform a press, move and release.
60
61 This is to keep a common API between Mouse and Finger as long as
62 possible.
63
64+ The pointer will be dragged from the starting point to the ending point
65+ with multiple moves. The number of moves, and thus the time that it
66+ will take to complete the drag can be altered with the `rate`
67+ parameter.
68+
69+ :param x1: The point on the x axis where the drag will start from.
70+ :param y1: The point on the y axis where the drag will starts from.
71+ :param x2: The point on the x axis where the drag will end at.
72+ :param y2: The point on the y axis where the drag will end at.
73+ :param rate: The number of pixels the mouse will be moved per
74+ iteration. Default is 10 pixels. A higher rate will make the drag
75+ faster, and lower rate will make it slower.
76+ :param time_between_events: The number of seconds that the drag will
77+ wait between iterations.
78+
79 """
80 raise NotImplementedError("You cannot use this class directly.")
81
82@@ -466,8 +481,28 @@
83 """Release a previously pressed finger"""
84 raise NotImplementedError("You cannot use this class directly.")
85
86- def drag(self, x1, y1, x2, y2):
87- """Perform a drag gesture from (x1,y1) to (x2,y2)"""
88+ def drag(self, x1, y1, x2, y2, rate=10, time_between_events=0.01):
89+ """Perform a drag gesture.
90+
91+ The finger will be dragged from the starting point to the ending point
92+ with multiple moves. The number of moves, and thus the time that it
93+ will take to complete the drag can be altered with the `rate`
94+ parameter.
95+
96+ :param x1: The point on the x axis where the drag will start from.
97+ :param y1: The point on the y axis where the drag will starts from.
98+ :param x2: The point on the x axis where the drag will end at.
99+ :param y2: The point on the y axis where the drag will end at.
100+ :param rate: The number of pixels the finger will be moved per
101+ iteration. Default is 10 pixels. A higher rate will make the drag
102+ faster, and lower rate will make it slower.
103+ :param time_between_events: The number of seconds that the drag will
104+ wait between iterations.
105+
106+ :raises RuntimeError: if the finger is already pressed.
107+ :raises RuntimeError: if no more finger slots are available.
108+
109+ """
110 raise NotImplementedError("You cannot use this class directly.")
111
112
113@@ -641,9 +676,30 @@
114 else:
115 return (self._x, self._y)
116
117- def drag(self, x1, y1, x2, y2):
118- """Performs a press, move and release."""
119- self._device.drag(x1, y1, x2, y2)
120+ def drag(self, x1, y1, x2, y2, rate=10, time_between_events=0.01):
121+ """Perform a press, move and release.
122+
123+ This is to keep a common API between Mouse and Finger as long as
124+ possible.
125+
126+ The pointer will be dragged from the starting point to the ending point
127+ with multiple moves. The number of moves, and thus the time that it
128+ will take to complete the drag can be altered with the `rate`
129+ parameter.
130+
131+ :param x1: The point on the x axis where the drag will start from.
132+ :param y1: The point on the y axis where the drag will starts from.
133+ :param x2: The point on the x axis where the drag will end at.
134+ :param y2: The point on the y axis where the drag will end at.
135+ :param rate: The number of pixels the mouse will be moved per
136+ iteration. Default is 10 pixels. A higher rate will make the drag
137+ faster, and lower rate will make it slower.
138+ :param time_between_events: The number of seconds that the drag will
139+ wait between iterations.
140+
141+ """
142+ self._device.drag(
143+ x1, y1, x2, y2, rate=rate, time_between_events=time_between_events)
144 if isinstance(self._device, Touch):
145 self._x = x2
146 self._y = y2
147
148=== modified file 'autopilot/input/_uinput.py'
149--- autopilot/input/_uinput.py 2014-02-11 12:23:36 +0000
150+++ autopilot/input/_uinput.py 2014-02-11 12:23:36 +0000
151@@ -521,8 +521,23 @@
152 """
153 self._device.finger_move(x, y)
154
155- def drag(self, x1, y1, x2, y2):
156- """Perform a drag gesture from (x1,y1) to (x2,y2).
157+ def drag(self, x1, y1, x2, y2, rate=10, time_between_events=0.01):
158+ """Perform a drag gesture.
159+
160+ The finger will be dragged from the starting point to the ending point
161+ with multiple moves. The number of moves, and thus the time that it
162+ will take to complete the drag can be altered with the `rate`
163+ parameter.
164+
165+ :param x1: The point on the x axis where the drag will start from.
166+ :param y1: The point on the y axis where the drag will starts from.
167+ :param x2: The point on the x axis where the drag will end at.
168+ :param y2: The point on the y axis where the drag will end at.
169+ :param rate: The number of pixels the finger will be moved per
170+ iteration. Default is 10 pixels. A higher rate will make the drag
171+ faster, and lower rate will make it slower.
172+ :param time_between_events: The number of seconds that the drag will
173+ wait between iterations.
174
175 :raises RuntimeError: if the finger is already pressed.
176 :raises RuntimeError: if no more finger slots are available.
177@@ -531,18 +546,28 @@
178 logger.debug("Dragging from %d,%d to %d,%d", x1, y1, x2, y2)
179 self._device.finger_down(x1, y1)
180
181- # Let's drag in 100 steps for now...
182- dx = 1.0 * (x2 - x1) / 100
183- dy = 1.0 * (y2 - y1) / 100
184- cur_x = x1 + dx
185- cur_y = y1 + dy
186- for i in range(0, 100):
187- self._device.finger_move(int(cur_x), int(cur_y))
188- sleep(0.002)
189- cur_x += dx
190- cur_y += dy
191- # Make sure we actually end up at target
192- self._device.finger_move(x2, y2)
193+ current_x, current_y = x1, y1
194+ while current_x != x2 or current_y != y2:
195+ dx = abs(x2 - current_x)
196+ dy = abs(y2 - current_y)
197+
198+ intx = float(dx) / max(dx, dy)
199+ inty = float(dy) / max(dx, dy)
200+
201+ step_x = min(rate * intx, dx)
202+ step_y = min(rate * inty, dy)
203+
204+ if x2 < current_x:
205+ step_x *= -1
206+ if y2 < current_y:
207+ step_y *= -1
208+
209+ current_x += step_x
210+ current_y += step_y
211+ self._device.finger_move(current_x, current_y)
212+
213+ sleep(time_between_events)
214+
215 self._device.finger_up()
216
217
218
219=== modified file 'autopilot/tests/functional/test_input_stack.py'
220--- autopilot/tests/functional/test_input_stack.py 2014-02-11 12:23:36 +0000
221+++ autopilot/tests/functional/test_input_stack.py 2014-02-11 12:23:36 +0000
222@@ -487,7 +487,8 @@
223 def __init__(self):
224 pass
225
226- def drag(self, x1, y1, x2, y2):
227+ def drag(self, x1, y1, x2, y2, rate='dummy',
228+ time_between_events='dummy'):
229 pass
230
231 p = Pointer(FakeTouch())
232
233=== modified file 'autopilot/tests/unit/test_input.py'
234--- autopilot/tests/unit/test_input.py 2014-02-11 12:23:36 +0000
235+++ autopilot/tests/unit/test_input.py 2014-02-11 12:23:36 +0000
236@@ -19,14 +19,15 @@
237
238 import testscenarios
239 from evdev import ecodes, uinput
240-from mock import ANY, call, patch, Mock
241+from mock import ANY, call, Mock, patch
242 from six import StringIO
243 from testtools import TestCase
244 from testtools.matchers import Contains, raises
245
246+import autopilot.input
247+from autopilot import utilities
248 from autopilot.input import _uinput
249 from autopilot.input._common import get_center_point
250-from autopilot import utilities
251
252
253 class Empty(object):
254@@ -698,6 +699,54 @@
255 touch.move(10, 10)
256 self.assertEqual(expected_calls, touch._device.mock_calls)
257
258+ def test_drag_must_call_finger_down_move_and_up(self):
259+ expected_calls = [
260+ call.finger_down(0, 0),
261+ call.finger_move(10, 10),
262+ call.finger_up()
263+ ]
264+
265+ touch = self.get_touch_with_mocked_backend()
266+ touch.drag(0, 0, 10, 10)
267+ self.assertEqual(expected_calls, touch._device.mock_calls)
268+
269+ def test_drag_must_move_with_specified_rate(self):
270+ expected_calls = [
271+ call.finger_down(0, 0),
272+ call.finger_move(5, 5),
273+ call.finger_move(10, 10),
274+ call.finger_move(15, 15),
275+ call.finger_up()]
276+
277+ touch = self.get_touch_with_mocked_backend()
278+ touch.drag(0, 0, 15, 15, rate=5)
279+
280+ self.assertEqual(
281+ expected_calls, touch._device.mock_calls)
282+
283+ def test_drag_without_rate_must_use_default(self):
284+ expected_calls = [
285+ call.finger_down(0, 0),
286+ call.finger_move(10, 10),
287+ call.finger_move(20, 20),
288+ call.finger_up()]
289+
290+ touch = self.get_touch_with_mocked_backend()
291+ touch.drag(0, 0, 20, 20)
292+
293+ self.assertEqual(
294+ expected_calls, touch._device.mock_calls)
295+
296+ def test_drag_to_same_place_must_not_move(self):
297+ expected_calls = [
298+ call.finger_down(0, 0),
299+ call.finger_up()
300+ ]
301+
302+ touch = self.get_touch_with_mocked_backend()
303+ touch.drag(0, 0, 0, 0)
304+ self.assertEqual(expected_calls, touch._device.mock_calls)
305+
306
307 class MultipleUInputTouchBackend(_uinput._UInputTouchDevice):
308
309@@ -731,3 +780,109 @@
310 self.addCleanup(finger1.release)
311
312 self.assertFalse(finger2.pressed)
313+
314+
315+class DragUInputTouchTestCase(testscenarios.TestWithScenarios, TestCase):
316+
317+ scenarios = [
318+ ('drag to top', dict(
319+ start_x=50, start_y=50, stop_x=50, stop_y=30,
320+ expected_moves=[call.finger_move(50, 40),
321+ call.finger_move(50, 30)])),
322+ ('drag to bottom', dict(
323+ start_x=50, start_y=50, stop_x=50, stop_y=70,
324+ expected_moves=[call.finger_move(50, 60),
325+ call.finger_move(50, 70)])),
326+ ('drag to left', dict(
327+ start_x=50, start_y=50, stop_x=30, stop_y=50,
328+ expected_moves=[call.finger_move(40, 50),
329+ call.finger_move(30, 50)])),
330+ ('drag to right', dict(
331+ start_x=50, start_y=50, stop_x=70, stop_y=50,
332+ expected_moves=[call.finger_move(60, 50),
333+ call.finger_move(70, 50)])),
334+
335+ ('drag to top-left', dict(
336+ start_x=50, start_y=50, stop_x=30, stop_y=30,
337+ expected_moves=[call.finger_move(40, 40),
338+ call.finger_move(30, 30)])),
339+ ('drag to top-right', dict(
340+ start_x=50, start_y=50, stop_x=70, stop_y=30,
341+ expected_moves=[call.finger_move(60, 40),
342+ call.finger_move(70, 30)])),
343+ ('drag to bottom-left', dict(
344+ start_x=50, start_y=50, stop_x=30, stop_y=70,
345+ expected_moves=[call.finger_move(40, 60),
346+ call.finger_move(30, 70)])),
347+ ('drag to bottom-right', dict(
348+ start_x=50, start_y=50, stop_x=70, stop_y=70,
349+ expected_moves=[call.finger_move(60, 60),
350+ call.finger_move(70, 70)])),
351+
352+ ('drag less than rate', dict(
353+ start_x=50, start_y=50, stop_x=55, stop_y=55,
354+ expected_moves=[call.finger_move(55, 55)])),
355+
356+ ('drag with last move less than rate', dict(
357+ start_x=50, start_y=50, stop_x=65, stop_y=65,
358+ expected_moves=[call.finger_move(60, 60),
359+ call.finger_move(65, 65)])),
360+ ]
361+
362+ def setUp(self):
363+ super(DragUInputTouchTestCase, self).setUp()
364+ # Mock the sleeps so we don't have to spend time actually sleeping.
365+ self.addCleanup(utilities.sleep.disable_mock)
366+ utilities.sleep.enable_mock()
367+
368+ def get_touch_with_mocked_backend(self):
369+ touch = _uinput.Touch(device_class=Mock)
370+ touch._device.mock_add_spec(
371+ _uinput._UInputTouchDevice, spec_set=True)
372+ return touch
373+
374+ def test_drag_moves(self):
375+ touch = self.get_touch_with_mocked_backend()
376+
377+ touch.drag(
378+ self.start_x, self.start_y, self.stop_x, self.stop_y)
379+
380+ # We don't check the finger down and finger up. They are already
381+ # tested.
382+ expected_calls = [ANY] + self.expected_moves + [ANY]
383+ self.assertEqual(
384+ expected_calls, touch._device.mock_calls)
385+
386+
387+class PointerWithTouchBackendTestCase(TestCase):
388+
389+ def get_pointer_with_touch_backend_with_mock_device(self):
390+ touch = _uinput.Touch(device_class=Mock)
391+ touch._device.mock_add_spec(
392+ _uinput._UInputTouchDevice, spec_set=True)
393+ pointer = autopilot.input.Pointer(touch)
394+ return pointer
395+
396+ def test_drag_with_rate(self):
397+ pointer = self.get_pointer_with_touch_backend_with_mock_device()
398+ with patch.object(pointer._device, 'drag') as mock_drag:
399+ pointer.drag(0, 0, 20, 20, rate='test')
400+
401+ mock_drag.assert_called_once_with(
402+ 0, 0, 20, 20, rate='test', time_between_events=0.01)
403+
404+ def test_drag_with_time_between_events(self):
405+ pointer = self.get_pointer_with_touch_backend_with_mock_device()
406+ with patch.object(pointer._device, 'drag') as mock_drag:
407+ pointer.drag(0, 0, 20, 20, time_between_events='test')
408+
409+ mock_drag.assert_called_once_with(
410+ 0, 0, 20, 20, rate=10, time_between_events='test')
411+
412+ def test_drag_with_default_parameters(self):
413+ pointer = self.get_pointer_with_touch_backend_with_mock_device()
414+ with patch.object(pointer._device, 'drag') as mock_drag:
415+ pointer.drag(0, 0, 20, 20)
416+
417+ mock_drag.assert_called_once_with(
418+ 0, 0, 20, 20, rate=10, time_between_events=0.01)

Subscribers

People subscribed via source and target branches