Mir

Merge lp:~mir-team/mir/android-input-transport-scatter-send-event into lp:mir

Proposed by Brandon Schaefer
Status: Rejected
Rejected by: Brandon Schaefer
Proposed branch: lp:~mir-team/mir/android-input-transport-scatter-send-event
Merge into: lp:mir
Diff against target: 316 lines (+165/-9)
2 files modified
3rd_party/android-input/android/frameworks/base/include/androidfw/InputTransport.h (+23/-2)
3rd_party/android-input/android/frameworks/base/services/input/InputTransport.cpp (+142/-7)
To merge this branch: bzr merge lp:~mir-team/mir/android-input-transport-scatter-send-event
Reviewer Review Type Date Requested Status
Mir CI Bot continuous-integration Approve
Alan Griffiths Needs Fixing
Daniel van Vugt Needs Fixing
PS Jenkins bot (community) continuous-integration Approve
Alberto Aguirre (community) Approve
Chris Halse Rogers Approve
Review via email: mp+279387@code.launchpad.net

Commit message

Move to using a scatter/gather when sending input message. This will make sending a uint8_t* mac + uint32_t mac_size much simpler (and better).

Description of the change

Move to using a scatter/gather when sending input message. This will make sending a uint8_t* mac + uint32_t mac_size much simpler (and better).

This is the beginning stages. All the casting to unit8_t* from a uint64_t location will be removed and replaced with a std::vector.data() call. Makes the core changes seen (with out bloating to much).

To post a comment you must log in.
3160. By Brandon Schaefer

* Merge trunk

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Revision history for this message
Chris Halse Rogers (raof) wrote :

+ /* Anything after the Pointer struct WILL BE CUT OFF if you send the entire
+ * struct. As the size() function returns only valid mouse pointers (leaving the rest off)
+ * So if you add to the struct insert before or manually re-construct the extra bits.
+ */

Could probably be reworded to be a bit clearer?

Maybe:
/* As an optimisation sendMessage() only sends the valid members of the pointers[]
 * array. This is implemented by size() only counting pointerCount members of
 * the pointers[] array.
 *
 * As a side effect, everything after the pointers[] array will NOT be reliably sent
 * in sendMessage(). Any members placed after pointers[] will need to be reconstructed
 * in recieveMessage().
 */

Otherwise seems fine.

review: Approve
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

This is an input protocol break (remember last time you changed these?).

If we intentionally break part of the protocol then there are version macros that need updating. Although I would recommend not breaking it at all if you can help it.

review: Needs Fixing
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Hmm, last time we changed these structures we had to ask app developers to rebuild re-release their click packages to match the new input protocol.

This needs a deep full-team discussion, if not deferring/rejecting for now...

Revision history for this message
Chris Halse Rogers (raof) wrote :

On Monday, 7 December 2015 7:35:59 PM AEDT, Daniel van Vugt wrote:
> Hmm, last time we changed these structures we had to ask app
> developers to rebuild re-release their click packages to match
> the new input protocol.
>
> This needs a deep full-team discussion, if not
> deferring/rejecting for now...

My memory of the outcome last time was “embedding libmirclient is not
supported; when we break it you get to keep both pieces”. Has anything
changrd to make it different this time?

--
Sent using Dekko from my Ubuntu device

Revision history for this message
Daniel van Vugt (vanvugt) wrote :

I'm not sure we have yet convinced/coerced app developers to stop shipping their own libmirclient.

Assuming we haven't, we should probably pause for a moment to consider the plan of attack. Because most things are better than breaking apps already in the store (again).

Disapproved for safety, till we're absolutely sure the same pain isn't going to happen all over again.

review: Disapprove
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

I'm not familiar with the term "scatter/gather" in this context. What is the motivation here? If we're changing the serialization, then it might make sense to switch to the protobuf serialization we use elsewhere.

review: Needs Information
Revision history for this message
Brandon Schaefer (brandontschaefer) wrote :

@duflu

From what I know we do not support the protocol. ie. We are free to break it, and make no guarantee it. From a public point of view.

Though if apps in the store are using it (against our guarantee). The yes we should approach a different plan of attack. Not sure the best one. As we *will* need to break the protocol again to support 160bit hash. How ever may it be.

@alan

The point of this scatter/gather is so we can send an arbitrary size hash over the protocol. As the 160bit hash is subject to being changed. We are looking at moving to a std::vector to hold the hash.

Issue being unions in InputMessage stop us from just using a vector. This is the approach around that. There will also be another issue in event_private, which we are moving MirEvent over to use boost::varaint so we can have a tagged union (which will then support std::vector!).

As far as moving to protobuf for serialization here... not sure. Im happy to do the best thing :). So Ill talk with RAOF. Though is protobuf the best for events? (Not really sure)

So right now the goal of this is:
1) Support sending a uint8_t* through the android socket to the client (this branch)
2) Move to using boost::variant in place of MirEvent (an attempt tagged union atm). This will allow us to store non-trivial objects in the MirEvent structs. As well as makes things type-safe (yay for us).
3) Move to using std::vector as a way to hold the MAC hash so we dont hard code 160bits. (subject to change)
4) Move to actually pushing the 160bit SHA-1 hash through to the vectors (which at this point will be support!)

Possibly best I write up a proposal?

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> As far as moving to protobuf for serialization here... not sure. Im happy to
> do the best thing :). So Ill talk with RAOF. Though is protobuf the best for
> events? (Not really sure)

On the basis of our testing protobuf is unlikely to be "best" for performance, it would just be consistent (and does have, for example, some support for compatibility between versions). I'm sure RAOF will give you the background story.

3161. By Brandon Schaefer

* Change comment (reads much better)

Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

Hey don't shoot the messenger. I'm just the poor bugger who untangled the problem last time so I remember it probably more than others.

That said, input protocol changes might be OK now providing (and we should verify) that the offending click packages:

1. use libmirclient9 (and only nine); and
2a. do not contain their own copy of libmirclient9; or
2b. will use the newer system version of libmirclient9 we provide instead of their own one.

Sounds feasible so we might be free to break the input protocol now.

review: Needs Information
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

I don't have reason to block, but I've not the time to look closer (I'm on vacation the rest of this week).

review: Abstain
Revision history for this message
Brandon Schaefer (brandontschaefer) wrote :

Not trying to shoot :). So first thing first (before we look at approving this). Is to test if we would offend any click packages.

We also talked about rejecting click packages that bundle their own libmirclient. Which if that were the case, we can break away?

Ill try to talk with you later to try and figure out if this is OK to do atm, or figure out what will be needed to be done!

Thanks!

Revision history for this message
Daniel van Vugt (vanvugt) wrote :

I just had to find the history of the problem again. It's all in --> bug 1507982

Sounds like it probably won't happen again so long as no packages are using libmirclient8 or earlier. The affected apps of bug 1507982 have been rebuilt and re-released against libmirclient9. So that should be OK. The only remaining concern is that the problem could still happen in theory even if people don't package their own libmirclient, because bug 1498281 is still open.

In summary, it sounds like we are free to break the input protocol now. Just so long as we don't bump to libmirclient10 in the foreseeable future. This way all existing native apps that use libmirclient9 will just get the new libmirclient9 and protocol changes will be transparent.

review: Abstain
Revision history for this message
Alberto Aguirre (albaguirre) wrote :

Comments inline.

review: Needs Fixing
3162. By Brandon Schaefer

* Address reviewer

3163. By Brandon Schaefer

* Rework the using of the empty_mac so we set the correct size

Revision history for this message
Alberto Aguirre (albaguirre) wrote :

LGTM

review: Approve
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Revision history for this message
PS Jenkins bot (ps-jenkins) wrote :
review: Approve (continuous-integration)
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

(1) Is 1000 a sensible number? That would allow up to 8000 bits which is a strange number. You might want something that is (2^n/8) bytes to avoid wastage.

(2) I just realized; having a (relative) pointer in an over-the-wire structure adds unnecessary complexity and risk:
+ uint8_t* mac;
The structure is meant to represent what goes over the wire, but it can't now and you have an unnecessary dance to support it.

Please just change it to a fixed size like "uint8_t mac[512]; // supports up to 4096 bits".

That will nicely restore clear over-the-wire opaque copy support, and so simplify the code a lot too.

review: Needs Fixing
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

(2) Or if you really do need variable size support, make it clearer using "ptrdiff_t mac_offset" and add a getter method that is "uint8_t* mac() const".

Revision history for this message
Alberto Aguirre (albaguirre) wrote :

> (2) I just realized; having a (relative) pointer in an over-the-wire structure
> adds unnecessary complexity and risk:
> + uint8_t* mac;
> The structure is meant to represent what goes over the wire, but it can't now
> and you have an unnecessary dance to support it.
>
> Please just change it to a fixed size like "uint8_t mac[512]; // supports up
> to 4096 bits".
>

Umm but that's the whole point of this MP, to use the scatter/gather support in the kernel so that you don't have to copy the mac into the structure when sending.

Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

OK codewise, but note this is a protocol break that isn't a client ABI bump.

Before release we need to update mir::protobuf::oldest_compatible_protocol_version() and possibly mir::protobuf::current_protocol_version() (the latter only if we don't add any client API functions and bump MIR_CLIENT_MINOR_VERSION).

mir::protobuf::oldest_compatible_protocol_version() should return protocol_version(MIR_CLIENT_MAJOR_VERSION, 3);

Otherwise we'll accept connections from older libmirclient binaries but send input events they cannot parse and (if we don't bump MIR_CLIENT_MINOR_VERSION) successfully connect new clients to 0.18 servers which do not supply the correct events.

We can either make these changes here, or accept a promise elsewhere but we need to be sure of the approach before landing this branch.

review: Needs Information
Revision history for this message
Alan Griffiths (alan-griffiths) wrote :

> OK codewise, but note this is a protocol break that isn't a client ABI bump.

Actually, the easiest approach would be to bump the epoch default value in mir::protobuf::protocol_version() - and that can be done here very simply.

review: Needs Fixing
Revision history for this message
Daniel van Vugt (vanvugt) wrote :

> Umm but that's the whole point of this MP, to use the scatter/gather support
> in the kernel so that you don't have to copy the mac into the structure when
> sending.

Maybe so, but I think my suggestion is significantly simpler. Smaller code and easier to follow and maintain.

Revision history for this message
Brandon Schaefer (brandontschaefer) wrote :

Im fine for using either. If we using a uint8_t[N] vs a uint8_t*. After all it looks we will want to replace with proper serialization anyway. So, the reason for supporting scatter/gather was with the idea that it would be useful for the future. If we want to rip this out, then maybe the simplest smallest way would be the best?

Though im not sure how long it would take to move to proper serialization :). It would be on my list of goals, and looking at it *shouldnt* be to hard.

Revision history for this message
Chris Halse Rogers (raof) wrote :

Ok. Since we apparently intend to do this correctly in the not-too-distant future, and we're capable of breaking protocol at this level at will, we might as well shove a simple
uint8_t mac[20];
into the relevant structures and preserve the blittability at the cost of mac size changes being a protocol break.

Revision history for this message
Mir CI Bot (mir-ci-bot) wrote :

PASSED: Continuous integration, rev:3163
https://mir-jenkins.ubuntu.com/job/mir-ci/19/
Executed test runs:
    None: https://mir-jenkins.ubuntu.com/job/generic-update-mp/19/console

Click here to trigger a rebuild:
https://mir-jenkins.ubuntu.com/job/mir-ci/19/rebuild

review: Approve (continuous-integration)

Unmerged revisions

3163. By Brandon Schaefer

* Rework the using of the empty_mac so we set the correct size

3162. By Brandon Schaefer

* Address reviewer

3161. By Brandon Schaefer

* Change comment (reads much better)

3160. By Brandon Schaefer

* Merge trunk

3159. By Brandon Schaefer

* Switch to using recvmsg so we can check if we get a truncated message! (We cant have those)
* Also added a comment to why we need to put values before the pointer[] if we want to copy those values.
  the mac* is handled in its own buffer (the address is copied) soo its fine after the pointer[]

3158. By Brandon Schaefer

* Move to using a unit8_t* for the mac + a uint32_t mac_size to pass a variable length mac hash
  through the fd

3157. By Brandon Schaefer

* Move to using a scatter/gather method for sending events.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '3rd_party/android-input/android/frameworks/base/include/androidfw/InputTransport.h'
2--- 3rd_party/android-input/android/frameworks/base/include/androidfw/InputTransport.h 2015-08-17 18:33:38 +0000
3+++ 3rd_party/android-input/android/frameworks/base/include/androidfw/InputTransport.h 2015-12-11 21:17:18 +0000
4@@ -45,6 +45,8 @@
5 * Intermediate representation used to send input events and related signals.
6 */
7 struct InputMessage {
8+ static uint32_t const MAC_PADDING = 1000;
9+
10 InputMessage();
11
12 enum {
13@@ -62,7 +64,6 @@
14 struct Key {
15 uint32_t seq;
16 int64_t eventTime;
17- uint64_t mac;
18 int32_t deviceId;
19 int32_t source;
20 int32_t action;
21@@ -72,6 +73,8 @@
22 int32_t metaState;
23 int32_t repeatCount;
24 int64_t downTime;
25+ uint32_t mac_size;
26+ uint8_t* mac;
27
28 inline size_t size() const {
29 return sizeof(Key);
30@@ -81,7 +84,6 @@
31 struct Motion {
32 uint32_t seq;
33 int64_t eventTime;
34- uint64_t mac;
35 int32_t deviceId;
36 int32_t source;
37 int32_t action;
38@@ -95,10 +97,20 @@
39 float xPrecision;
40 float yPrecision;
41 size_t pointerCount;
42+ uint32_t mac_size;
43+ /* As an optimisation sendMessage() only sends the valid members of the pointers[]
44+ * array. This is implemented by size() only counting pointerCount members of
45+ * the pointers[] array.
46+ *
47+ * As a side effect, everything after the pointers[] array will NOT be reliably sent
48+ * in sendMessage(). Any members placed after pointers[] will need to be reconstructed
49+ * in recieveMessage().
50+ */
51 struct Pointer {
52 PointerProperties properties;
53 PointerCoords coords;
54 } pointers[MAX_POINTERS];
55+ uint8_t* mac;
56
57 int32_t getActionId() const {
58 uint32_t index = (action & AMOTION_EVENT_ACTION_POINTER_INDEX_MASK)
59@@ -120,6 +132,15 @@
60 return sizeof(Finished);
61 }
62 } finished;
63+
64+ /* Use a 1KB padding for the MAC
65+ *
66+ * Reserve enough space to receive a MAC in
67+ * receiveMessage(...)
68+ */
69+ struct Padding {
70+ uint8_t mac_padding[MAC_PADDING];
71+ };
72 } body;
73
74 bool isValid(size_t actualSize) const;
75
76=== modified file '3rd_party/android-input/android/frameworks/base/services/input/InputTransport.cpp'
77--- 3rd_party/android-input/android/frameworks/base/services/input/InputTransport.cpp 2015-10-20 03:30:00 +0000
78+++ 3rd_party/android-input/android/frameworks/base/services/input/InputTransport.cpp 2015-12-11 21:17:18 +0000
79@@ -27,7 +27,9 @@
80 #include <unistd.h>
81 #include <sys/types.h>
82 #include <sys/socket.h>
83+#include <sys/uio.h>
84 #include <math.h>
85+#include <stdexcept>
86
87
88 namespace android {
89@@ -38,6 +40,8 @@
90 // behind processing touches.
91 static const size_t SOCKET_BUFFER_SIZE = 32 * 1024;
92
93+static const uint64_t EMPTY_MAC = 0;
94+
95 // Nanoseconds per milliseconds.
96 static constexpr const std::chrono::nanoseconds NANOS_PER_MS = std::chrono::nanoseconds(1000000);
97
98@@ -61,6 +65,19 @@
99 return a + alpha * (b - a);
100 }
101
102+size_t getMacSizeForMessage(const InputMessage* msg)
103+{
104+ switch(msg->header.type)
105+ {
106+ case InputMessage::TYPE_KEY:
107+ return msg->body.key.mac_size;
108+ case InputMessage::TYPE_MOTION:
109+ return msg->body.motion.mac_size;
110+ }
111+
112+ return 0;
113+}
114+
115 // --- InputMessage ---
116
117 InputMessage::InputMessage()
118@@ -69,7 +86,8 @@
119 }
120
121 bool InputMessage::isValid(size_t actualSize) const {
122- if (size() == actualSize) {
123+ size_t const mac_size = getMacSizeForMessage(this);
124+ if (size() + mac_size == actualSize) {
125 switch (header.type) {
126 case TYPE_KEY:
127 return true;
128@@ -140,11 +158,74 @@
129 return OK;
130 }
131
132+bool messageHasMac(const InputMessage* msg)
133+{
134+ switch(msg->header.type) {
135+ case InputMessage::TYPE_KEY:
136+ case InputMessage::TYPE_MOTION:
137+ return true;
138+ }
139+
140+ return false;
141+}
142+
143+uint8_t* macFromMessage(const InputMessage* msg)
144+{
145+ switch(msg->header.type) {
146+ case InputMessage::TYPE_KEY:
147+ return msg->body.key.mac;
148+ case InputMessage::TYPE_MOTION:
149+ return msg->body.motion.mac;
150+ }
151+
152+ throw std::runtime_error("Invalid message type: " + msg->header.type);
153+}
154+
155+size_t fillMac(const InputMessage* msg, iovec* vec)
156+{
157+ auto const mac = static_cast<void*>(macFromMessage(msg));
158+ auto const size = getMacSizeForMessage(msg);
159+
160+ if (mac && size == sizeof(uint64_t))
161+ {
162+ vec->iov_base = mac;
163+ vec->iov_len = size;
164+ }
165+ else
166+ {
167+ vec->iov_base = static_cast<void*>(const_cast<uint64_t*>(&EMPTY_MAC));
168+ vec->iov_len = sizeof(EMPTY_MAC);
169+ }
170+
171+ return vec->iov_len;
172+}
173+
174 status_t InputChannel::sendMessage(const InputMessage* msg) {
175+ std::array<iovec, 2> msg_parts;
176 size_t msgLength = msg->size();
177+ size_t msg_parts_size = 1;
178+
179+ msg_parts[0].iov_base = static_cast<void*>(const_cast<InputMessage*>(msg));
180+ msg_parts[0].iov_len = msgLength;
181+
182+ if (messageHasMac(msg)) {
183+ msgLength += fillMac(msg, &msg_parts[1]);
184+ msg_parts_size++;
185+ }
186+
187+ msghdr const message = {
188+ nullptr,
189+ 0,
190+ msg_parts.data(),
191+ msg_parts_size,
192+ nullptr,
193+ 0,
194+ 0
195+ };
196+
197 ssize_t nWrite;
198 do {
199- nWrite = ::send(mFd, msg, msgLength, MSG_DONTWAIT | MSG_NOSIGNAL);
200+ nWrite = ::sendmsg(mFd, &message, MSG_DONTWAIT | MSG_NOSIGNAL);
201 } while (nWrite == -1 && errno == EINTR);
202
203 if (nWrite < 0) {
204@@ -176,12 +257,64 @@
205 return OK;
206 }
207
208+uint8_t* getMacArrayFromMessageOffset(const InputMessage* msg, uint32_t mac_size)
209+{
210+ if (mac_size == sizeof(uint64_t)) {
211+ return reinterpret_cast<uint8_t*>(const_cast<InputMessage*>(msg)) + msg->size();
212+ }
213+
214+ return reinterpret_cast<uint8_t*>(const_cast<uint64_t*>(&EMPTY_MAC));
215+}
216+
217+void setMacAddressFromMessage(InputMessage* msg)
218+{
219+ switch (msg->header.type)
220+ {
221+ case InputMessage::TYPE_KEY:
222+ {
223+ msg->body.key.mac = getMacArrayFromMessageOffset(msg, msg->body.key.mac_size);
224+ break;
225+ }
226+ case InputMessage::TYPE_MOTION:
227+ {
228+ msg->body.motion.mac = getMacArrayFromMessageOffset(msg, msg->body.motion.mac_size);
229+ break;
230+ }
231+ }
232+}
233+
234 status_t InputChannel::receiveMessage(InputMessage* msg) {
235+ iovec recv_msg;
236+
237+ recv_msg.iov_base = msg;
238+ recv_msg.iov_len = sizeof(InputMessage);
239+
240+ msghdr message = {
241+ nullptr,
242+ 0,
243+ &recv_msg,
244+ 1,
245+ nullptr,
246+ 0,
247+ 0
248+ };
249+
250 ssize_t nRead;
251 do {
252- nRead = ::recv(mFd, msg, sizeof(InputMessage), MSG_DONTWAIT);
253+ nRead = ::recvmsg(mFd, &message, MSG_DONTWAIT);
254 } while (nRead == -1 && errno == EINTR);
255
256+ if (messageHasMac(msg)) {
257+ setMacAddressFromMessage(msg);
258+ }
259+
260+ if (message.msg_flags & MSG_TRUNC) {
261+#if DEBUG_CHANNEL_MESSAGES
262+ ALOGD("channel '%s' ~ received message was truncated undefined behaviour", c_str(mName));
263+#endif
264+ return BAD_VALUE;
265+ }
266+
267 if (nRead < 0) {
268 int error = errno;
269 #if DEBUG_CHANNEL_MESSAGES
270@@ -264,9 +397,10 @@
271 msg.body.key.scanCode = scanCode;
272 msg.body.key.metaState = metaState;
273 msg.body.key.repeatCount = repeatCount;
274- msg.body.key.mac = mac;
275 msg.body.key.downTime = downTime.count();
276 msg.body.key.eventTime = eventTime.count();
277+ msg.body.key.mac = reinterpret_cast<uint8_t*>(&mac);
278+ msg.body.key.mac_size = sizeof(uint64_t);
279 return mChannel->sendMessage(&msg);
280 }
281
282@@ -325,7 +459,6 @@
283 msg.body.motion.yOffset = yOffset;
284 msg.body.motion.xPrecision = xPrecision;
285 msg.body.motion.yPrecision = yPrecision;
286- msg.body.motion.mac = mac;
287 msg.body.motion.downTime = downTime.count();
288 msg.body.motion.eventTime = eventTime.count();
289 msg.body.motion.pointerCount = pointerCount;
290@@ -333,6 +466,8 @@
291 msg.body.motion.pointers[i].properties.copyFrom(pointerProperties[i]);
292 msg.body.motion.pointers[i].coords.copyFrom(pointerCoords[i]);
293 }
294+ msg.body.motion.mac = reinterpret_cast<uint8_t*>(&mac);
295+ msg.body.motion.mac_size = sizeof(uint64_t);
296 return mChannel->sendMessage(&msg);
297 }
298
299@@ -882,7 +1017,7 @@
300 msg->body.key.scanCode,
301 msg->body.key.metaState,
302 msg->body.key.repeatCount,
303- msg->body.key.mac,
304+ *(reinterpret_cast<uint64_t*>(msg->body.key.mac)),
305 std::chrono::nanoseconds(msg->body.key.downTime),
306 std::chrono::nanoseconds(msg->body.key.eventTime));
307 }
308@@ -908,7 +1043,7 @@
309 msg->body.motion.yOffset,
310 msg->body.motion.xPrecision,
311 msg->body.motion.yPrecision,
312- msg->body.motion.mac,
313+ *(reinterpret_cast<uint64_t*>(msg->body.motion.mac)),
314 std::chrono::nanoseconds(msg->body.motion.downTime),
315 std::chrono::nanoseconds(msg->body.motion.eventTime),
316 pointerCount,

Subscribers

People subscribed via source and target branches