Merge lp:~oif-team/grail/trunk.touch into lp:grail

[Henrik Rydberg]

This patchset introduces gesture touch events.

[Chase Douglas (chasedouglas)]

I'm guessing the functionality is correct, but the header file is missing the TOUCH event attributes (just like there are separate DRAG, PINCH, ROTATE, etc. attributes).

review: Needs Fixing

[Henrik Rydberg (rydberg)]

Seems like a mid-air collision - it seems they are present now.

[Henrik Rydberg (rydberg)]

Removed a stale variable and fixed one whitespace error, looks all good to me now.

[Chase Douglas (chasedouglas)]

In the grail-types.h file there's an old comment that should be removed:

+/* touch event types use drag properties */

I now seem unable to trigger any non-touch gestures when a touch gesture is performed, though...

review: Needs Fixing

[Stephen M. Webb (bregma)]

The code looks OK stylistically and seems to work as I expected through geistest (once installed correctly). I do not have a problem with this.

review: Approve

[Henrik Rydberg (rydberg)]

There were some confusion as to what was a bug and what was expected behavior, but hopefully that is all straightened out now. This version seem to solve the known issues. Also see 744391 for another bugfix that goes on top of these.

[Chase Douglas (chasedouglas)]

Still has the touch types comment that should be removed :).

review: Needs Fixing

[lp:~oif-team/grail/trunk.touch]

133. By Henrik Rydberg on 2011-03-28

Introduce touch gesture events

Touch gesture events follow the same logic as drags, except they
cannot be held back but are emitted immediately.

Signed-off-by: Henrik Rydberg <email address hidden>

134. By Henrik Rydberg on 2011-03-28

Decouple timeout from discard

Move the actual gesture discard from the recognition phase
to the instantiation phase.

Signed-off-by: Henrik Rydberg <email address hidden>

135. By Henrik Rydberg on 2011-03-28

Do not timeout gestures overlapping touch gestures

The notion of listening to touch gestures implies that the
touches of those gestures never timeout. Hence, any pending
gesture that overlaps those touches should not timeout.

Signed-off-by: Henrik Rydberg <email address hidden>

136. By Henrik Rydberg on 2011-03-28

Never active gestures below activation threshold

Currently, all gestures activate after a certain time, such that
the smallest change result in gesture events. This was designed
to minimize disruption in pointer movement. Since grail no longer
handles pointers, the logic can be simplified to only activate when
the threshold is reached.

Signed-off-by: Henrik Rydberg <email address hidden>

137. By Henrik Rydberg on 2011-03-28

Do not treat single finger activation differently

Single-finger drags should no longer behave differently from
any other drag gesture. Fixed with this patch.

Signed-off-by: Henrik Rydberg <email address hidden>

138. By Henrik Rydberg on 2011-03-28

Do not hold back active gestures

Gestures that has once been sent should never be held back. The
current code would under some circumstance hold the gesture end
event and later discard it. This oneliner solves the problem, which
has been around since 241c5ec, Aug 2010. LP: #744391.

Signed-off-by: Henrik Rydberg <email address hidden>

[Chase Douglas (chasedouglas)]

All my comments have been addressed, and it appears to be working properly through gesturetest and geistest (with the new and improved utouch-geis :).

review: Approve

Unmerged revisions

=== modified file 'include/grail-types.h'
--- include/grail-types.h	2010-08-19 20:36:11 +0000
+++ include/grail-types.h	2011-03-28 20:03:33 +0000
@@ -59,11 +59,20 @@
 
 #define GRAIL_TYPE_SYSFLAG1	26	/* reserved system flag */
 
+#define GRAIL_TYPE_TOUCH1	32	/* one-finger touch */
+#define GRAIL_TYPE_TOUCH2	33	/* two-finger touch */
+#define GRAIL_TYPE_TOUCH3	34	/* three-finger touch */
+#define GRAIL_TYPE_TOUCH4	35	/* four-finger touch */
+#define GRAIL_TYPE_TOUCH5	36	/* five-finger touch */
+#define GRAIL_TYPE_ETOUCH	37	/* three-finger environment touch */
+#define GRAIL_TYPE_MTOUCH	38	/* four-finger meta touch */
+
 #define GRAIL_MAIN_DRAG		0
 #define GRAIL_MAIN_PINCH	1
 #define GRAIL_MAIN_ROTATE	2
 #define GRAIL_MAIN_TAP		3
 #define GRAIL_MAIN_SYSFLAG	4
+#define GRAIL_MAIN_TOUCH	5
 
 #define GRAIL_PROP_DRAG_DX	0	/* horizontal position delta */
 #define GRAIL_PROP_DRAG_DY	1	/* vertical position delta */
@@ -160,5 +169,24 @@
 #define GRAIL_PROP_TAP_X_T4	20
 #define GRAIL_PROP_TAP_Y_T4	21
 
+#define GRAIL_PROP_TOUCH_X1	0	/* bounding box x1 */
+#define GRAIL_PROP_TOUCH_Y1	1	/* bounding box y1 */
+#define GRAIL_PROP_TOUCH_X2	2	/* bounding box x2 */
+#define GRAIL_PROP_TOUCH_Y2	3	/* bounding box y2 */
+#define GRAIL_PROP_TOUCH_ID_T0	4	/* first touch id */
+#define GRAIL_PROP_TOUCH_X_T0	5	/* first touch horizontal position */
+#define GRAIL_PROP_TOUCH_Y_T0	6	/* first touch vertical position */
+#define GRAIL_PROP_TOUCH_ID_T1	7
+#define GRAIL_PROP_TOUCH_X_T1	8
+#define GRAIL_PROP_TOUCH_Y_T1	9
+#define GRAIL_PROP_TOUCH_ID_T2	10
+#define GRAIL_PROP_TOUCH_X_T2	11
+#define GRAIL_PROP_TOUCH_Y_T2	12
+#define GRAIL_PROP_TOUCH_ID_T3	13
+#define GRAIL_PROP_TOUCH_X_T3	14
+#define GRAIL_PROP_TOUCH_Y_T3	15
+#define GRAIL_PROP_TOUCH_ID_T4	16
+#define GRAIL_PROP_TOUCH_X_T4	17
+#define GRAIL_PROP_TOUCH_Y_T4	18
 
 #endif

=== modified file 'src/Makefile.am'
--- src/Makefile.am	2011-01-02 12:08:42 +0000
+++ src/Makefile.am	2011-03-28 20:03:33 +0000
@@ -17,6 +17,7 @@
 	grail-inserter.h \
 	grail-gestures.c \
 	grail-gestures.h \
+	gestures-touch.c \
 	gestures-drag.c \
 	gestures-pinch.c \
 	gestures-rotate.c \

=== modified file 'src/gestures-drag.c'
--- src/gestures-drag.c	2011-03-17 09:48:13 +0000
+++ src/gestures-drag.c	2011-03-28 20:03:33 +0000
@@ -70,10 +70,8 @@
 	}
 	if ((move->timeout & fm_mask) == fm_mask) {
 		if (state->active) {
-			gin_gid_discard(ge, state->gid);
-			state->active = 0;
+			gin_gid_timeout(ge, state->gid);
 		}
-		return 0;
 	}
 	if (!state->active) {
 		int type = getype[move->ntouch];
@@ -84,7 +82,7 @@
 	}
 	if (!(move->tickle & mask))
 		return 0;
-	if (!move->single && !(move->active & fm_mask))
+	if (!(move->active & fm_mask))
 		return 0;
 	set_props(ge->gin, state, move, frame);
 	gru_event(ge, state->gid, move, state->prop, state->nprop);
@@ -107,10 +105,8 @@
 	}
 	if ((move->timeout & fm_mask) == fm_mask) {
 		if (state->active) {
-			gin_gid_discard(ge, state->gid);
-			state->active = 0;
+			gin_gid_timeout(ge, state->gid);
 		}
-		return 0;
 	}
 	if (!state->active) {
 		if (move->ntouch == 4) {
@@ -131,7 +127,7 @@
 	}
 	if (!(move->tickle & mask))
 		return 0;
-	if (!move->single && !(move->active & fm_mask))
+	if (!(move->active & fm_mask))
 		return 0;
 	set_props(ge->gin, state, move, frame);
 	gru_event(ge, state->gid, move, state->prop, state->nprop);

=== modified file 'src/gestures-pinch.c'
--- src/gestures-pinch.c	2011-03-17 09:48:13 +0000
+++ src/gestures-pinch.c	2011-03-28 20:03:33 +0000
@@ -64,10 +64,8 @@
 	}
 	if ((move->timeout & fm_mask) == fm_mask) {
 		if (state->active) {
-			gin_gid_discard(ge, state->gid);
-			state->active = 0;
+			gin_gid_timeout(ge, state->gid);
 		}
-		return 0;
 	}
 	if (!(move->tickle & mask))
 		return 0;
@@ -102,10 +100,8 @@
 	}
 	if ((move->timeout & fm_mask) == fm_mask) {
 		if (state->active) {
-			gin_gid_discard(ge, state->gid);
-			state->active = 0;
+			gin_gid_timeout(ge, state->gid);
 		}
-		return 0;
 	}
 	if (!(move->tickle & mask))
 		return 0;

=== modified file 'src/gestures-rotate.c'
--- src/gestures-rotate.c	2011-02-24 22:51:33 +0000
+++ src/gestures-rotate.c	2011-03-28 20:03:33 +0000
@@ -63,10 +63,8 @@
 	}
 	if ((move->timeout & fm_mask) == fm_mask) {
 		if (state->active) {
-			gin_gid_discard(ge, state->gid);
-			state->active = 0;
+			gin_gid_timeout(ge, state->gid);
 		}
-		return 0;
 	}
 	if (!(move->tickle & mask))
 		return 0;
@@ -101,10 +99,8 @@
 	}
 	if ((move->timeout & fm_mask) == fm_mask) {
 		if (state->active) {
-			gin_gid_discard(ge, state->gid);
-			state->active = 0;
+			gin_gid_timeout(ge, state->gid);
 		}
-		return 0;
 	}
 	if (!(move->tickle & mask))
 		return 0;

=== added file 'src/gestures-touch.c'
--- src/gestures-touch.c	1970-01-01 00:00:00 +0000
+++ src/gestures-touch.c	2011-03-28 20:03:33 +0000
@@ -0,0 +1,94 @@
+/*****************************************************************************
+ *
+ * grail - Gesture Recognition And Instantiation Library
+ *
+ * Copyright (C) 2010 Canonical Ltd.
+ * Copyright (C) 2010 Henrik Rydberg <[e-mail]>
+ *
+ * This program is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation, either version 3 of the License, or (at your
+ * option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ ****************************************************************************/
+
+#include "grail-recognizer.h"
+#include <math.h>
+#include <stdio.h>
+
+static const int getype[DIM_TOUCH + 1] = {
+	-1,
+	GRAIL_TYPE_TOUCH1,
+	GRAIL_TYPE_TOUCH2,
+	GRAIL_TYPE_TOUCH3,
+	GRAIL_TYPE_TOUCH4,
+	GRAIL_TYPE_TOUCH5,
+};
+
+int gru_touch(struct grail *ge,
+	      const struct utouch_frame *frame)
+{
+	struct gesture_recognizer *gru = ge->gru;
+	struct combo_model *state = &gru->touch;
+	struct move_model *move = &gru->move;
+	if (frame->slot_revision != frame->prev->slot_revision) {
+		if (state->active) {
+			gru_end(ge, state->gid, move,
+				state->prop, state->nprop);
+			state->active = 0;
+		}
+	}
+	if (!state->active) {
+		int type = getype[move->ntouch];
+		if (type <= 0)
+			return 0;
+		state->gid = gin_gid_begin(ge, type, -PRIO_GESTURE, frame);
+		state->active = 1;
+	}
+	state->nprop = gin_add_contact_props(ge->gin, state->prop, frame);
+	gru_event(ge, state->gid, move, state->prop, state->nprop);
+	return 1;
+}
+
+int gru_wintouch(struct grail *ge,
+		 const struct utouch_frame *frame)
+{
+	struct gesture_recognizer *gru = ge->gru;
+	struct combo_model *state = &gru->wintouch;
+	struct move_model *move = &gru->move;
+	if (frame->slot_revision != frame->prev->slot_revision) {
+		if (state->active && out_of_bounds(state, move)) {
+			gru_end(ge, state->gid, move,
+				state->prop, state->nprop);
+			state->active = 0;
+		}
+	}
+	if (!state->active) {
+		if (move->ntouch == 4) {
+			state->gid = gin_gid_begin(ge, GRAIL_TYPE_MTOUCH,
+						   -PRIO_META, frame);
+			state->mintouch = 1;
+			state->maxtouch = 4;
+			state->active = 1;
+		} else if (move->ntouch == 3) {
+			state->gid = gin_gid_begin(ge, GRAIL_TYPE_ETOUCH,
+						   -PRIO_ENV, frame);
+			state->mintouch = 1;
+			state->maxtouch = 3;
+			state->active = 1;
+		} else {
+			return 0;
+		}
+	}
+	state->nprop = gin_add_contact_props(ge->gin, state->prop, frame);
+	gru_event(ge, state->gid, move, state->prop, state->nprop);
+	return 1;
+}

=== modified file 'src/grail-gestures.c'
--- src/grail-gestures.c	2011-03-17 09:48:13 +0000
+++ src/grail-gestures.c	2011-03-28 20:03:33 +0000
@@ -133,15 +133,14 @@
 		m->tickle &= ~(1 << i);
 	if (m->active & (1 << i))
 		return;
-	fm->action_delta = x - fm->original;
-	if (t - fm->original_ms > fm->hold_ms &&
-	    fabs(fm->action_delta) > fm->bar) {
-		m->active |= (1 << i);
+	fm->action_delta = 0;
+	if (fabs(x - fm->original) > fm->bar) {
+		if (t - fm->original_ms > fm->hold_ms) {
+			m->active |= (1 << i);
+			fm->action_delta = x - fm->original;
+		}
 	} else if (t - fm->original_ms > fm->bar_ms) {
-		m->active |= (1 << i);
 		m->timeout |= (1 << i);
-	} else {
-		fm->action_delta = 0;
 	}
 }
 

=== modified file 'src/grail-gestures.h'
--- src/grail-gestures.h	2011-02-24 22:51:17 +0000
+++ src/grail-gestures.h	2011-03-28 20:03:33 +0000
@@ -77,6 +77,8 @@
 	grail_prop_t prop[DIM_GRAIL_PROP];
 };
 
+int gru_touch(struct grail *ge,
+	      const struct utouch_frame *frame);
 int gru_drag(struct grail *ge,
 	     const struct utouch_frame *frame);
 int gru_pinch(struct grail *ge,
@@ -90,6 +92,8 @@
 	return m->ntouch < s->mintouch || m->ntouch > s->maxtouch;
 }
 
+int gru_wintouch(struct grail *ge,
+		 const struct utouch_frame *frame);
 int gru_windrag(struct grail *ge,
 		const struct utouch_frame *frame);
 int gru_winpinch(struct grail *ge,

=== modified file 'src/grail-inserter.c'
--- src/grail-inserter.c	2011-03-17 09:48:13 +0000
+++ src/grail-inserter.c	2011-03-28 20:03:33 +0000
@@ -38,6 +38,18 @@
 	return -1;
 }
 
+static int mask_overlap(const grail_mask_t *a, const grail_mask_t *b,
+			int bytes)
+{
+	int i;
+
+	for (i = 0; i < bytes; i++)
+		if (a[i] & b[i])
+			return 1;
+
+	return 0;
+}
+
 // todo: spanning tree for multi-user case
 static void setup_new_gestures(struct grail *ge,
 			       const struct utouch_frame *frame)
@@ -114,33 +126,50 @@
 void gin_frame_end(struct grail *ge, const struct utouch_frame *frame)
 {
 	struct gesture_inserter *gin = ge->gin;
-	int i, hold = 0, discard = 0;
+	grail_mask_t keep[DIM_TOUCH_BYTES];
+	int i, hold[2] = { 0, 0 }, discard[2] = { 0, 0 };
 
+	memset(keep, 0, sizeof(keep));
 	setup_new_gestures(ge, frame);
 
 	grail_mask_foreach(i, gin->used, sizeof(gin->used)) {
 		struct slot_state *s = &gin->state[i];
 		if (!s->nclient)
 			continue;
-		if (s->priority > hold)
-			hold = s->priority;
+		if (s->priority > hold[s->slice])
+			hold[s->slice] = s->priority;
 		if (s->status != GRAIL_STATUS_UPDATE)
 			continue;
-		if (s->priority > discard)
-			discard = s->priority;
+		if (s->priority > discard[s->slice])
+			discard[s->slice] = s->priority;
 	}
 
 	grail_mask_foreach(i, gin->used, sizeof(gin->used)) {
 		struct slot_state *s = &gin->state[i];
-		if (!s->nclient || s->priority < discard)
+		if (!s->nclient || s->priority < discard[s->slice])
 			gin_gid_discard(ge, s->id);
 	}
 
 	grail_mask_foreach(i, gin->used, sizeof(gin->used)) {
 		struct slot_state *s = &gin->state[i];
+		if (s->slice == 1)
+			grail_mask_set_mask(keep, s->span, sizeof(keep));
+	}
+
+	grail_mask_foreach(i, gin->used, sizeof(gin->used)) {
+		struct slot_state *s = &gin->state[i];
+		if (!s->timeout)
+			continue;
+		if (mask_overlap(keep, s->span, sizeof(keep)))
+			continue;
+		gin_gid_discard(ge, s->id);
+	}
+
+	grail_mask_foreach(i, gin->used, sizeof(gin->used)) {
+		struct slot_state *s = &gin->state[i];
 		struct gesture_event ev;
 		grail_mask_set(gin->types, s->type);
-		if (s->priority < hold)
+		if (s->priority < hold[s->slice])
 			continue;
 		while (!gebuf_empty(&s->buf)) {
 			gebuf_get(&s->buf, &ev);
@@ -166,7 +195,14 @@
 		return -1;
 	s = &gin->state[i];
 	s->type = type;
-	s->priority = priority;
+	if (priority < 0) {
+		s->priority = -priority;
+		s->slice = 1;
+	} else {
+		s->priority = priority;
+		s->slice = 0;
+	}
+	s->timeout = 0;
 	s->id = gin->gestureid++ & MAX_GESTURE_ID;
 	s->status = GRAIL_STATUS_BEGIN;
 	s->nclient = 0;
@@ -193,6 +229,13 @@
 	grail_mask_set(gin->unused, i);
 }
 
+void gin_gid_timeout(struct grail *ge, int gid)
+{
+	int i = find_gslot(ge->gin, gid);
+	if (i >= 0)
+		ge->gin->state[i].timeout = 1;
+}
+
 void gin_gid_event(struct grail *ge, int gid,
 		   float x, float y, int ntouch,
 		   const grail_prop_t *prop, int nprop,

=== modified file 'src/grail-inserter.h'
--- src/grail-inserter.h	2011-03-17 09:48:13 +0000
+++ src/grail-inserter.h	2011-03-28 20:03:33 +0000
@@ -37,6 +37,8 @@
 struct slot_state {
 	int type;
 	int priority;
+	int slice;
+	int timeout;
 	int id;
 	int status;
 	int nclient;
@@ -78,6 +80,7 @@
 int gin_gid_begin(struct grail *ge, int type, int priority,
 		  const struct utouch_frame *frame);
 void gin_gid_discard(struct grail *ge, int gid);
+void gin_gid_timeout(struct grail *ge, int gid);
 
 void gin_gid_event(struct grail *ge, int gid,
 		   float x, float y, int ntouch,

=== modified file 'src/grail-recognizer.c'
--- src/grail-recognizer.c	2011-01-02 12:08:08 +0000
+++ src/grail-recognizer.c	2011-03-28 20:03:33 +0000
@@ -47,9 +47,11 @@
 	if (!ge->gin || !ge->gru)
 		return;
 	gru_motion(ge, frame);
+	gru_touch(ge, frame);
 	gru_drag(ge, frame);
 	gru_pinch(ge, frame);
 	gru_rotate(ge, frame);
+	gru_wintouch(ge, frame);
 	gru_windrag(ge, frame);
 	gru_winpinch(ge, frame);
 	gru_winrotate(ge, frame);

=== modified file 'src/grail-recognizer.h'
--- src/grail-recognizer.h	2011-02-24 22:49:22 +0000
+++ src/grail-recognizer.h	2011-03-28 20:03:33 +0000
@@ -27,9 +27,11 @@
 
 struct gesture_recognizer {
 	struct move_model move;
+	struct combo_model touch;
 	struct combo_model drag;
 	struct combo_model pinch;
 	struct combo_model rotate;
+	struct combo_model wintouch;
 	struct combo_model windrag;
 	struct combo_model winpinch;
 	struct combo_model winrotate;