Merge lp:~clint-fewbar/gearmand/document-protocol into lp:gearmand

Proposed by Clint Byrum on 2015-03-17
Status: Needs review
Proposed branch: lp:~clint-fewbar/gearmand/document-protocol
Merge into: lp:gearmand
Diff against target: 116 lines (+68/-3)
1 file modified
PROTOCOL (+68/-3)
To merge this branch: bzr merge lp:~clint-fewbar/gearmand/document-protocol
Reviewer Review Type Date Requested Status
Tangent Trunk 2015-03-17 Pending
Review via email: mp+253140@code.launchpad.net

Description of the change

Document new undocumented commands in PROTOCOL

To post a comment you must log in.
910. By Clint Byrum on 2015-04-16

Add documentation for the purpose of Unique ID

Unmerged revisions

910. By Clint Byrum on 2015-04-16

Add documentation for the purpose of Unique ID

909. By Clint Byrum on 2015-03-17

Add basic documentation of undocumented protocol pieces.

It seems that the protocol added a few more commands and responses in
recent times, and they should at least be documented. The way that REDUCER
is used is still somewhat confusing and needs better documentation as
to what is expected, but at least an implementor can handle the commands
with a graceful error now.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'PROTOCOL'
2--- PROTOCOL 2012-05-04 03:46:04 +0000
3+++ PROTOCOL 2015-04-16 00:11:24 +0000
4@@ -102,6 +102,13 @@
5 34 SUBMIT_JOB_LOW_BG REQ Client
6 35 SUBMIT_JOB_SCHED REQ Client
7 36 SUBMIT_JOB_EPOCH REQ Client
8+ 37 SUBMIT_REDUCE_JOB REQ Client
9+ 38 SUBMIT_REDUCE_JOB_BACKGROUND REQ Client
10+ 39 GRAB_JOB_ALL REQ Worker
11+ 40 JOB_ASSIGN_ALL RES Worker
12+ 41 GET_STATUS_UNIQUE REQ Client
13+ 42 STATUS_RES_UNIQUE RES Client
14+
15
16 4 byte size - A big-endian (network-order) integer containing
17 the size of the data being sent after the header.
18@@ -171,9 +178,26 @@
19 always take precedence, and jobs submitted with the normal versions
20 take precedence over the LOW versions.
21
22- Arguments:
23- - NULL byte terminated function name.
24- - NULL byte terminated unique ID.
25+ The unique ID can be used by the server to reduce queue length. If a
26+ job with the same Unique ID has already been submitted, the server
27+ may attach this request to the already existing job. This includes
28+ jobs already in progress, in which case non-background jobs will be
29+ sent the same result as background jobs. This is known commonly as
30+ "coalescing".
31+
32+ Arguments:
33+ - NULL byte terminated function name.
34+ - NULL byte terminated unique ID.
35+ - Opaque data that is given to the function as an argument.
36+
37+SUBMIT_REDUCE_JOB, SUBMIT_REDUCE_JOB_BACKGROUND
38+
39+ Works like the other SUBMIT_JOB commands, but adds a reducer argument.
40+
41+ Arguments:
42+ - NULL byte terminated function name.
43+ - NULL byte terminated unique ID.
44+ - NULL byte terminated reducer.
45 - Opaque data that is given to the function as an argument.
46
47 SUBMIT_JOB_SCHED
48@@ -209,6 +233,13 @@
49 Arguments:
50 - Job handle that was given in JOB_CREATED packet.
51
52+GET_STATUS_UNIQUE
53+
54+ A client issues this to get status information for a submitted job.
55+
56+ Arguments:
57+ - Unique value that was given when job was submitted.
58+
59 OPTION_REQ
60
61 A client issues this to set an option for the connection in the
62@@ -255,6 +286,22 @@
63 - NULL byte terminated percent complete numerator.
64 - Percent complete denominator.
65
66+STATUS_RES_UNIQUE
67+
68+ This is sent in response to a GET_STATUS_UNIQUE request. This is
69+ used by clients that have submitted a job with SUBMIT_JOB_BG to see
70+ if the job has been completed, and if not, to get the percentage
71+ complete.
72+
73+ Arguments:
74+ - NULL byte terminated job handle.
75+ - NULL byte terminated known status, this is 0 (false) or 1 (true).
76+ - NULL byte terminated running status, this is 0 (false) or 1
77+ (true).
78+ - NULL byte terminated percent complete numerator.
79+ - NULL byte terminated percent complete denominator.
80+ - Count of clients waiting.
81+
82 OPTION_RES
83
84 Successful response to the OPTION_REQ request.
85@@ -329,6 +376,13 @@
86 Arguments:
87 - None.
88
89+GRAB_JOB_ALL
90+
91+ Just like GRAB_JOB_UNIQ, but return JOB_ASSIGN_ALL when there is a job.
92+
93+ Arguments:
94+ - None.
95+
96 WORK_DATA
97
98 This is sent to update the client with data from a running job. A
99@@ -456,6 +510,17 @@
100 - NULL byte terminated unique ID.
101 - Opaque data that is given to the function as an argument.
102
103+JOB_ASSIGN_ALL
104+
105+ This is given in response to a GRAB_JOB_ALL request and acts
106+ just like JOB_ASSIGN_UNIQ but with the reducer returned.
107+
108+ Arguments:
109+ - NULL byte terminated job handle.
110+ - NULL byte terminated function name.
111+ - NULL byte terminated unique ID.
112+ - NULL byte terminated reducer.
113+ - Opaque data that is given to the function as an argument.
114
115 Administrative Protocol
116 -----------------------

Subscribers

People subscribed via source and target branches

to all changes: