Merge lp:~rogpeppe/juju-core/479-desired-peer-group into lp:~go-bot/juju-core/trunk

Proposed by Roger Peppe
Status: Merged
Approved by: Roger Peppe
Approved revision: no longer in the source branch.
Merged at revision: 2328
Proposed branch: lp:~rogpeppe/juju-core/479-desired-peer-group
Merge into: lp:~go-bot/juju-core/trunk
Diff against target: 658 lines (+648/-0)
2 files modified
worker/peergrouper/desired.go (+290/-0)
worker/peergrouper/desired_test.go (+358/-0)
To merge this branch: bzr merge lp:~rogpeppe/juju-core/479-desired-peer-group
Reviewer Review Type Date Requested Status
Juju Engineering Pending
Review via email: mp+201245@code.launchpad.net

Commit message

worker/peergrouper: new package

This will be a worker that maintains the replica set
with respect to the state.

The function in this CL implements the core functionality - it's
a stateless function that looks at a representation of the
current state of affairs and decides what the replicaset
member list should look like.

It may well change when faced with reality (although I've been
trying to sanity check with experimental code),
but I think it should be reasonable to check in now
as a staging post.

There are current some extraneous changes in this
branch (everything outside worker/peergrouper)
which are made redundant by other MPs. Please
ignore for the purposes of this review; I'll remove
before submitting.

https://codereview.appspot.com/49920045/

Description of the change

worker/peergrouper: new package

This will be a worker that maintains the replica set
with respect to the state.

The function in this CL implements the core functionality - it's
a stateless function that looks at a representation of the
current state of affairs and decides what the replicaset
member list should look like.

It may well change when faced with reality (although I've been
trying to sanity check with experimental code),
but I think it should be reasonable to check in now
as a staging post.

There are current some extraneous changes in this
branch (everything outside worker/peergrouper)
which are made redundant by other MPs. Please
ignore for the purposes of this review; I'll remove
before submitting.

https://codereview.appspot.com/49920045/

To post a comment you must log in.
Revision history for this message
Roger Peppe (rogpeppe) wrote :

Reviewers: mp+201245_code.launchpad.net,

Message:
Please take a look.

Description:
worker/peergrouper: new package

This will be a worker that maintains the replica set
with respect to the state.

The function in this CL implements the core functionality - it's
a stateless function that looks at a representation of the
current state of affairs and decides what the replicaset
member list should look like.

It may well change when faced with reality (although I've been
trying to sanity check with experimental code),
but I think it should be reasonable to check in now
as a staging post.

There are current some extraneous changes in this
branch (everything outside worker/peergrouper)
which are made redundant by other MPs. Please
ignore for the purposes of this review; I'll remove
before submitting.

https://code.launchpad.net/~rogpeppe/juju-core/479-desired-peer-group/+merge/201245

(do not edit description out of merge proposal)

Please review this at https://codereview.appspot.com/49920045/

Affected files (+684, -38 lines):
   A [revision details]
   M replicaset/replicaset.go
   M replicaset/replicaset_test.go
   M testing/mgo.go
   A worker/peergrouper/desired.go
   A worker/peergrouper/desired_test.go
   M worker/uniter/uniter_test.go

Revision history for this message
John A Meinel (jameinel) wrote :

I didn't finish the review before I'm on the next call, but some
thoughts I didn't want to lose.

https://codereview.appspot.com/49920045/diff/20001/worker/peergrouper/desired.go
File worker/peergrouper/desired.go (right):

https://codereview.appspot.com/49920045/diff/20001/worker/peergrouper/desired.go#newcode32
worker/peergrouper/desired.go:32: //func getPeerGroupInfo(st
*state.State, ms []*state.Machine) (*peerGroupInfo, error) {
this function is commented out. Generally either we should delete it, or
have it. I'm guessing this is just part of the process as you're getting
there.

https://codereview.appspot.com/49920045/diff/20001/worker/peergrouper/desired.go#newcode58
worker/peergrouper/desired.go:58: return j
Is there really nothing like min that we can just use?

https://codereview.appspot.com/49920045/diff/20001/worker/peergrouper/desired.go#newcode66
worker/peergrouper/desired.go:66: func desiredPeerGroup(info
*peerGroupInfo) ([]replicaset.Member, error) {
Your description makes it sound like this function mixes a query (what
peer group do you want?) with an action (set the voting status of all
the machines).

It would be best to split informational queries from side effects if
possible.

https://codereview.appspot.com/49920045/

Revision history for this message
Roger Peppe (rogpeppe) wrote :

Please take a look.

https://codereview.appspot.com/49920045/diff/20001/worker/peergrouper/desired.go
File worker/peergrouper/desired.go (right):

https://codereview.appspot.com/49920045/diff/20001/worker/peergrouper/desired.go#newcode32
worker/peergrouper/desired.go:32: //func getPeerGroupInfo(st
*state.State, ms []*state.Machine) (*peerGroupInfo, error) {
On 2014/01/13 11:37:12, jameinel wrote:
> this function is commented out. Generally either we should delete it,
or have
> it. I'm guessing this is just part of the process as you're getting
there.

It is. I suppose I should probably delete it for this CL and bring it
back in the next one.

https://codereview.appspot.com/49920045/diff/20001/worker/peergrouper/desired.go#newcode58
worker/peergrouper/desired.go:58: return j
On 2014/01/13 11:37:12, jameinel wrote:
> Is there really nothing like min that we can just use?

nope. it's not considered worth adding to the standard library.
(well, it wouldn't be one - there would be 12 of them).

https://codereview.appspot.com/49920045/diff/20001/worker/peergrouper/desired.go#newcode66
worker/peergrouper/desired.go:66: func desiredPeerGroup(info
*peerGroupInfo) ([]replicaset.Member, error) {
On 2014/01/13 11:37:12, jameinel wrote:
> Your description makes it sound like this function mixes a query (what
peer
> group do you want?) with an action (set the voting status of all the
machines).

> It would be best to split informational queries from side effects if
possible.

I've changed it so that it returns a map of values rather than setting
the machine fields in place. We want the caller to set the votes of the
machines after calling this function, and it's much easier to work out
that information here, rather than have the caller try to work out the
info from the replicaset members, which may not include all the machines
passed in.

https://codereview.appspot.com/49920045/

Revision history for this message
Roger Peppe (rogpeppe) wrote :
Revision history for this message
Nate Finch (natefinch) wrote :

LGTM overall. I'd like to see comments on the functions even if they're
not exported... makes it a little clearer what each one is doing.

https://codereview.appspot.com/49920045/diff/80001/worker/peergrouper/desired.go
File worker/peergrouper/desired.go (right):

https://codereview.appspot.com/49920045/diff/80001/worker/peergrouper/desired.go#newcode38
worker/peergrouper/desired.go:38: func desiredPeerGroup(info
*peerGroupInfo) ([]replicaset.Member, map[*machine]bool, error) {
This function is pretty long, would be a little easier to read if it
were broken up more, I think.

https://codereview.appspot.com/49920045/diff/80001/worker/peergrouper/desired.go#newcode199
worker/peergrouper/desired.go:199: found = m
break here?

https://codereview.appspot.com/49920045/

Revision history for this message
Roger Peppe (rogpeppe) wrote :

Please take a look.

https://codereview.appspot.com/49920045/diff/80001/worker/peergrouper/desired.go
File worker/peergrouper/desired.go (right):

https://codereview.appspot.com/49920045/diff/80001/worker/peergrouper/desired.go#newcode38
worker/peergrouper/desired.go:38: func desiredPeerGroup(info
*peerGroupInfo) ([]replicaset.Member, map[*machine]bool, error) {
On 2014/01/13 18:47:15, nate.finch wrote:
> This function is pretty long, would be a little easier to read if it
were broken
> up more, I think.

Yeah, I'd toyed with that idea but not mustered up the necessary energy.

Done; definitely a good idea, thanks.

https://codereview.appspot.com/49920045/diff/80001/worker/peergrouper/desired.go#newcode199
worker/peergrouper/desired.go:199: found = m
On 2014/01/13 18:47:15, nate.finch wrote:
> break here?

Done.

https://codereview.appspot.com/49920045/

Revision history for this message
Nate Finch (natefinch) wrote :

LGTM The refactor makes a big difference.

https://codereview.appspot.com/49920045/

Revision history for this message
Roger Peppe (rogpeppe) wrote :
Revision history for this message
Go Bot (go-bot) wrote :
Download full text (465.0 KiB)

The attempt to merge lp:~rogpeppe/juju-core/479-desired-peer-group into lp:juju-core failed. Below is the output from the failed tests.

ok launchpad.net/juju-core/agent 1.262s
ok launchpad.net/juju-core/agent/tools 0.279s
ok launchpad.net/juju-core/bzr 8.061s
ok launchpad.net/juju-core/cert 3.420s
ok launchpad.net/juju-core/charm 0.609s
? launchpad.net/juju-core/charm/hooks [no test files]
ok launchpad.net/juju-core/cloudinit 0.033s
ok launchpad.net/juju-core/cloudinit/sshinit 1.067s
ok launchpad.net/juju-core/cmd 0.224s
ok launchpad.net/juju-core/cmd/charm-admin 0.848s
? launchpad.net/juju-core/cmd/charmd [no test files]
? launchpad.net/juju-core/cmd/charmload [no test files]
ok launchpad.net/juju-core/cmd/juju 174.789s

----------------------------------------------------------------------
FAIL: machine_test.go:273: MachineSuite.TestManageEnviron

[LOG] 30.61946 DEBUG juju.environs.configstore Making /tmp/gocheck-6129484611666145821/45/home/ubuntu/.juju/environments
[LOG] 30.73054 DEBUG juju.environs.tools reading v1.* tools
[LOG] 30.73059 INFO juju environs/testing: uploading FAKE tools 1.17.1-precise-amd64
[LOG] 30.73153 DEBUG juju.environs.tools no architecture specified when finding tools, looking for any
[LOG] 30.73155 DEBUG juju.environs.tools no series specified when finding tools, looking for any
[LOG] 30.73162 DEBUG juju.environs.simplestreams fetchData failed for "http://127.0.0.1:52068/dummyenv/private/tools/streams/v1/index.sjson": file "tools/streams/v1/index.sjson" not found not found
[LOG] 30.73165 DEBUG juju.environs.simplestreams cannot load index "http://127.0.0.1:52068/dummyenv/private/tools/streams/v1/index.sjson": invalid URL "http://127.0.0.1:52068/dummyenv/private/tools/streams/v1/index.sjson" not found
[LOG] 30.73169 DEBUG juju.environs.simplestreams fetchData failed for "http://127.0.0.1:52068/dummyenv/private/tools/streams/v1/index.json": file "tools/streams/v1/index.json" not found not found
[LOG] 30.73171 DEBUG juju.environs.simplestreams cannot load index "http://127.0.0.1:52068/dummyenv/private/tools/streams/v1/index.json": invalid URL "http://127.0.0.1:52068/dummyenv/private/tools/streams/v1/index.json" not found
[LOG] 30.73198 INFO juju.environs.tools Writing tools/streams/v1/index.json
[LOG] 30.73202 INFO juju.environs.tools Writing tools/streams/v1/com.ubuntu.juju:released:tools.json
[LOG] 30.73218 INFO juju.environs.bootstrap bootstrapping environment "dummyenv"
[LOG] 30.73221 DEBUG juju.environs.bootstrap looking for bootstrap tools: series="precise", arch=<nil>, version=1.17.1
[LOG] 30.73223 INFO juju.environs.tools reading tools with major.minor version 1.17
[LOG] 30.73224 INFO juju.environs.tools filtering tools by version: 1.17.1
[LOG] 30.73225 INFO juju.environs.tools filtering tools by series: precise
[LOG] 30.73227 DEBUG juju.environs.tools no architecture specified when finding tools, looking for any
[LOG] 30.73231 DEBUG juju.environs.simplestreams fetchData failed for "http://127.0.0.1:52068/dummyenv/private/tools/streams/v1/index.sjson": file "tools/streams/v1/index.sjson" not found not found
[LOG] 30.73233 DEBUG juju.environs.simplestreams cannot load index "http:/...

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== added directory 'worker/peergrouper'
=== added file 'worker/peergrouper/desired.go'
--- worker/peergrouper/desired.go 1970-01-01 00:00:00 +0000
+++ worker/peergrouper/desired.go 2014-02-14 12:14:58 +0000
@@ -0,0 +1,290 @@
1package peergrouper
2
3import (
4 "fmt"
5 "sort"
6
7 "launchpad.net/juju-core/replicaset"
8 "launchpad.net/loggo"
9)
10
11var logger = loggo.GetLogger("juju.worker.peergrouper")
12
13// peerGroupInfo holds information that may contribute to
14// a peer group.
15type peerGroupInfo struct {
16 machines []*machine // possibly map[id] *machine
17 statuses []replicaset.MemberStatus
18 members []replicaset.Member
19}
20
21// machine represents a machine in State.
22type machine struct {
23 id string
24 wantsVote bool
25 hostPort string
26}
27
28// desiredPeerGroup returns the mongo peer group according to the given
29// servers and a map with an element for each machine in info.machines
30// specifying whether that machine has been configured as voting. It may
31// return (nil, nil, nil) if the current group is already correct.
32func desiredPeerGroup(info *peerGroupInfo) ([]replicaset.Member, map[*machine]bool, error) {
33 changed := false
34 members, extra, maxId := info.membersMap()
35
36 // We may find extra peer group members if the machines
37 // have been removed or their state server status removed.
38 // This should only happen if they had been set to non-voting
39 // before removal, in which case we want to remove it
40 // from the members list. If we find a member that's still configured
41 // to vote, it's an error.
42 // TODO There are some other possibilities
43 // for what to do in that case.
44 // 1) leave them untouched, but deal
45 // with others as usual "i didn't see that bit"
46 // 2) leave them untouched, deal with others,
47 // but make sure the extras aren't eligible to
48 // be primary.
49 // 3) remove them "get rid of bad rubbish"
50 // 4) bomb out "run in circles, scream and shout"
51 // 5) do nothing "nothing to see here"
52 for _, member := range extra {
53 if member.Votes == nil || *member.Votes > 0 {
54 return nil, nil, fmt.Errorf("voting non-machine member found in peer group")
55 }
56 changed = true
57 }
58
59 toRemoveVote, toAddVote, toKeep := possiblePeerGroupChanges(info, members)
60
61 // Set up initial record of machine votes. Any changes after
62 // this will trigger a peer group election.
63 machineVoting := make(map[*machine]bool)
64 for _, m := range info.machines {
65 if member := members[m]; member != nil && isVotingMember(member) {
66 machineVoting[m] = true
67 }
68 }
69 setVoting := func(m *machine, voting bool) {
70 setMemberVoting(members[m], voting)
71 machineVoting[m] = voting
72 changed = true
73 }
74 adjustVotes(toRemoveVote, toAddVote, setVoting)
75
76 addNewMembers(members, toKeep, maxId, setVoting)
77 if updateAddresses(members, info.machines) {
78 changed = true
79 }
80 if !changed {
81 return nil, nil, nil
82 }
83 var memberSet []replicaset.Member
84 for _, member := range members {
85 memberSet = append(memberSet, *member)
86 }
87 return memberSet, machineVoting, nil
88}
89
90func isVotingMember(member *replicaset.Member) bool {
91 return member.Votes == nil || *member.Votes > 0
92}
93
94// possiblePeerGroupChanges returns a set of slices
95// classifying all the existing machines according to
96// how their vote might move.
97// toRemoveVote holds machines whose vote should
98// be removed; toAddVote holds machines which are
99// ready to vote; toKeep holds machines with no desired
100// change to their voting status (this includes machines
101// that are not yet represented in the peer group).
102func possiblePeerGroupChanges(
103 info *peerGroupInfo,
104 members map[*machine]*replicaset.Member,
105) (toRemoveVote, toAddVote, toKeep []*machine) {
106 statuses := info.statusesMap(members)
107
108 for _, m := range info.machines {
109 member := members[m]
110 isVoting := member != nil && isVotingMember(member)
111 switch {
112 case m.wantsVote && isVoting:
113 toKeep = append(toKeep, m)
114 case m.wantsVote && !isVoting:
115 if status, ok := statuses[m]; ok && isReady(status) {
116 toAddVote = append(toAddVote, m)
117 } else {
118 toKeep = append(toKeep, m)
119 }
120 case !m.wantsVote && isVoting:
121 toRemoveVote = append(toRemoveVote, m)
122 case !m.wantsVote && !isVoting:
123 toKeep = append(toKeep, m)
124 }
125 }
126 // sort machines to be added and removed so that we
127 // get deterministic behaviour when testing. Earlier
128 // entries will be dealt with preferentially, so we could
129 // potentially sort by some other metric in each case.
130 sort.Sort(byId(toRemoveVote))
131 sort.Sort(byId(toAddVote))
132 sort.Sort(byId(toKeep))
133 return toRemoveVote, toAddVote, toKeep
134}
135
136// updateAddresses updates the members' addresses from the machines' addresses.
137// It reports whether any changes have been made.
138func updateAddresses(members map[*machine]*replicaset.Member, machines []*machine) bool {
139 changed := false
140 // Make sure all members' machine addresses are up to date.
141 for _, m := range machines {
142 if m.hostPort == "" {
143 continue
144 }
145 // TODO ensure that replicaset works correctly with IPv6 [host]:port addresses.
146 if m.hostPort != members[m].Address {
147 members[m].Address = m.hostPort
148 changed = true
149 }
150 }
151 return changed
152}
153
154// adjustVotes adjusts the votes of the given machines, taking
155// care not to let the total number of votes become even at
156// any time. It calls setVoting to change the voting status
157// of a machine.
158func adjustVotes(toRemoveVote, toAddVote []*machine, setVoting func(*machine, bool)) {
159 // Remove voting members if they can be replaced by
160 // candidates that are ready. This does not affect
161 // the total number of votes.
162 nreplace := min(len(toRemoveVote), len(toAddVote))
163 for i := 0; i < nreplace; i++ {
164 from := toRemoveVote[i]
165 to := toAddVote[i]
166 setVoting(from, false)
167 setVoting(to, true)
168 }
169 toAddVote = toAddVote[nreplace:]
170 toRemoveVote = toRemoveVote[nreplace:]
171
172 // At this point, one or both of toAdd or toRemove is empty, so
173 // we can adjust the voting-member count by an even delta,
174 // maintaining the invariant that the total vote count is odd.
175 if len(toAddVote) > 0 {
176 toAddVote = toAddVote[0 : len(toAddVote)-len(toAddVote)%2]
177 for _, m := range toAddVote {
178 setVoting(m, true)
179 }
180 } else {
181 toRemoveVote = toRemoveVote[0 : len(toRemoveVote)-len(toRemoveVote)%2]
182 for _, m := range toRemoveVote {
183 setVoting(m, false)
184 }
185 }
186}
187
188// addNewMembers adds new members from toKeep
189// to the given set of members, allocating ids from
190// maxId upwards. It calls setVoting to set the voting
191// status of each new member.
192func addNewMembers(
193 members map[*machine]*replicaset.Member,
194 toKeep []*machine,
195 maxId int,
196 setVoting func(*machine, bool),
197) {
198 for _, m := range toKeep {
199 if members[m] == nil && m.hostPort != "" {
200 // This machine was not previously in the members list,
201 // so add it (as non-voting). We maintain the
202 // id manually to make it easier for tests.
203 maxId++
204 member := &replicaset.Member{
205 Tags: map[string]string{
206 "juju-machine-id": m.id,
207 },
208 Id: maxId,
209 }
210 members[m] = member
211 setVoting(m, false)
212 }
213 }
214}
215
216func isReady(status replicaset.MemberStatus) bool {
217 return status.Healthy && (status.State == replicaset.PrimaryState ||
218 status.State == replicaset.SecondaryState)
219}
220
221func setMemberVoting(member *replicaset.Member, voting bool) {
222 if voting {
223 member.Votes = nil
224 member.Priority = nil
225 } else {
226 votes := 0
227 member.Votes = &votes
228 priority := 0.0
229 member.Priority = &priority
230 }
231}
232
233type byId []*machine
234
235func (l byId) Len() int { return len(l) }
236func (l byId) Swap(i, j int) { l[i], l[j] = l[j], l[i] }
237func (l byId) Less(i, j int) bool { return l[i].id < l[j].id }
238
239// membersMap returns the replica-set members inside info keyed
240// by machine. Any members that do not have a corresponding
241// machine are returned in extra.
242// The maximum replica-set id is returned in maxId.
243func (info *peerGroupInfo) membersMap() (members map[*machine]*replicaset.Member, extra []replicaset.Member, maxId int) {
244 maxId = -1
245 members = make(map[*machine]*replicaset.Member)
246 for _, member := range info.members {
247 member := member
248 var found *machine
249 if mid, ok := member.Tags["juju-machine-id"]; ok {
250 for _, m := range info.machines {
251 if m.id == mid {
252 found = m
253 break
254 }
255 }
256 }
257 if found != nil {
258 members[found] = &member
259 } else {
260 extra = append(extra, member)
261 }
262 if member.Id > maxId {
263 maxId = member.Id
264 }
265 }
266 return members, extra, maxId
267}
268
269// statusesMap returns the statuses inside info keyed by machine.
270// The provided members map holds the members keyed by machine,
271// as returned by membersMap.
272func (info *peerGroupInfo) statusesMap(members map[*machine]*replicaset.Member) map[*machine]replicaset.MemberStatus {
273 statuses := make(map[*machine]replicaset.MemberStatus)
274 for _, status := range info.statuses {
275 for m, member := range members {
276 if member.Id == status.Id {
277 statuses[m] = status
278 break
279 }
280 }
281 }
282 return statuses
283}
284
285func min(i, j int) int {
286 if i < j {
287 return i
288 }
289 return j
290}
0291
=== added file 'worker/peergrouper/desired_test.go'
--- worker/peergrouper/desired_test.go 1970-01-01 00:00:00 +0000
+++ worker/peergrouper/desired_test.go 2014-02-14 12:14:58 +0000
@@ -0,0 +1,358 @@
1package peergrouper
2
3import (
4 "fmt"
5 "sort"
6 "strconv"
7 "strings"
8 stdtesting "testing"
9
10 gc "launchpad.net/gocheck"
11 "launchpad.net/juju-core/replicaset"
12 jc "launchpad.net/juju-core/testing/checkers"
13 "launchpad.net/juju-core/testing/testbase"
14)
15
16func TestPackage(t *stdtesting.T) {
17 gc.TestingT(t)
18}
19
20type desiredPeerGroupSuite struct {
21 testbase.LoggingSuite
22}
23
24var _ = gc.Suite(&desiredPeerGroupSuite{})
25
26const mongoPort = 1234
27
28var desiredPeerGroupTests = []struct {
29 about string
30 machines []*machine
31 statuses []replicaset.MemberStatus
32 members []replicaset.Member
33
34 expectMembers []replicaset.Member
35 expectVoting []bool
36 expectErr string
37}{{
38 about: "single machine, no change",
39 machines: mkMachines("11v"),
40 members: mkMembers("1v"),
41 statuses: mkStatuses("1p"),
42 expectVoting: []bool{true},
43 expectMembers: nil,
44}, {
45 about: "extra member with nil Vote",
46 machines: mkMachines("11v"),
47 members: mkMembers("1v 2vT"),
48 statuses: mkStatuses("1p 2s"),
49 expectVoting: []bool{true},
50 expectErr: "voting non-machine member found in peer group",
51}, {
52 about: "extra member with >1 votes",
53 machines: mkMachines("11v"),
54 members: append(mkMembers("1v"), replicaset.Member{
55 Id: 2,
56 Votes: newInt(2),
57 Address: "0.1.2.12:1234",
58 }),
59 statuses: mkStatuses("1p 2s"),
60 expectVoting: []bool{true},
61 expectErr: "voting non-machine member found in peer group",
62}, {
63 about: "new machine with no associated member",
64 machines: mkMachines("11v 12v"),
65 members: mkMembers("1v"),
66 statuses: mkStatuses("1p"),
67 expectVoting: []bool{true, false},
68 expectMembers: mkMembers("1v 2"),
69}, {
70 about: "one machine has become ready to vote (-> no change)",
71 machines: mkMachines("11v 12v"),
72 members: mkMembers("1v 2"),
73 statuses: mkStatuses("1p 2s"),
74 expectVoting: []bool{true, false},
75 expectMembers: nil,
76}, {
77 about: "two machines have become ready to vote (-> added)",
78 machines: mkMachines("11v 12v 13v"),
79 members: mkMembers("1v 2 3"),
80 statuses: mkStatuses("1p 2s 3s"),
81 expectVoting: []bool{true, true, true},
82 expectMembers: mkMembers("1v 2v 3v"),
83}, {
84 about: "two machines have become ready to vote but one is not healthy (-> no change)",
85 machines: mkMachines("11v 12v 13v"),
86 members: mkMembers("1v 2 3"),
87 statuses: mkStatuses("1p 2s 3sH"),
88 expectVoting: []bool{true, false, false},
89 expectMembers: nil,
90}, {
91 about: "three machines have become ready to vote (-> 2 added)",
92 machines: mkMachines("11v 12v 13v 14v"),
93 members: mkMembers("1v 2 3 4"),
94 statuses: mkStatuses("1p 2s 3s 4s"),
95 expectVoting: []bool{true, true, true, false},
96 expectMembers: mkMembers("1v 2v 3v 4"),
97}, {
98 about: "one machine ready to lose vote with no others -> no change",
99 machines: mkMachines("11"),
100 members: mkMembers("1v"),
101 statuses: mkStatuses("1p"),
102 expectVoting: []bool{true},
103 expectMembers: nil,
104}, {
105 about: "two machines ready to lose vote -> votes removed",
106 machines: mkMachines("11 12v 13"),
107 members: mkMembers("1v 2v 3v"),
108 statuses: mkStatuses("1p 2p 3p"),
109 expectVoting: []bool{false, true, false},
110 expectMembers: mkMembers("1 2v 3"),
111}, {
112 about: "machines removed as state server -> removed from members",
113 machines: mkMachines("11v"),
114 members: mkMembers("1v 2 3"),
115 statuses: mkStatuses("1p 2s 3s"),
116 expectVoting: []bool{true},
117 expectMembers: mkMembers("1v"),
118}, {
119 about: "a candidate can take the vote of a non-candidate when they're ready",
120 machines: mkMachines("11v 12v 13 14v"),
121 members: mkMembers("1v 2v 3v 4"),
122 statuses: mkStatuses("1p 2s 3s 4s"),
123 expectVoting: []bool{true, true, false, true},
124 expectMembers: mkMembers("1v 2v 3 4v"),
125}, {
126 about: "several candidates can take non-candidates' votes",
127 machines: mkMachines("11v 12v 13 14 15 16v 17v 18v"),
128 members: mkMembers("1v 2v 3v 4v 5v 6 7 8"),
129 statuses: mkStatuses("1p 2s 3s 4s 5s 6s 7s 8s"),
130 expectVoting: []bool{true, true, false, false, false, true, true, true},
131 expectMembers: mkMembers("1v 2v 3 4 5 6v 7v 8v"),
132}, {
133 about: "a changed machine address should propagate to the members",
134 machines: append(mkMachines("11v 12v"), &machine{
135 id: "13",
136 wantsVote: true,
137 hostPort: "0.1.99.13:1234",
138 }),
139 statuses: mkStatuses("1s 2p 3p"),
140 members: mkMembers("1v 2v 3v"),
141 expectVoting: []bool{true, true, true},
142 expectMembers: append(mkMembers("1v 2v"), replicaset.Member{
143 Id: 3,
144 Address: "0.1.99.13:1234",
145 Tags: memberTag("13"),
146 }),
147}}
148
149func (*desiredPeerGroupSuite) TestDesiredPeerGroup(c *gc.C) {
150 for i, test := range desiredPeerGroupTests {
151 c.Logf("\ntest %d: %s", i, test.about)
152 info := &peerGroupInfo{
153 machines: test.machines,
154 statuses: test.statuses,
155 members: test.members,
156 }
157 members, voting, err := desiredPeerGroup(info)
158 if test.expectErr != "" {
159 c.Assert(err, gc.ErrorMatches, test.expectErr)
160 c.Assert(members, gc.IsNil)
161 continue
162 }
163 sort.Sort(membersById(members))
164 c.Assert(members, jc.DeepEquals, test.expectMembers)
165 if len(members) == 0 {
166 continue
167 }
168 for i, m := range info.machines {
169 c.Assert(voting[m], gc.Equals, test.expectVoting[i], gc.Commentf("machine %s", m.id))
170 }
171 // Assure ourselves that the total number of desired votes is odd in
172 // all circumstances.
173 c.Assert(countVotes(members)%2, gc.Equals, 1)
174
175 // Make sure that when the members are set as
176 // required, that there's no further change
177 // if desiredPeerGroup is called again.
178 info.members = members
179 members, voting, err = desiredPeerGroup(info)
180 c.Assert(members, gc.IsNil)
181 c.Assert(voting, gc.IsNil)
182 c.Assert(err, gc.IsNil)
183 }
184}
185
186func countVotes(members []replicaset.Member) int {
187 tot := 0
188 for _, m := range members {
189 v := 1
190 if m.Votes != nil {
191 v = *m.Votes
192 }
193 tot += v
194 }
195 return tot
196}
197
198func newInt(i int) *int {
199 return &i
200}
201
202func newFloat64(f float64) *float64 {
203 return &f
204}
205
206// mkMachines returns a slice of *machine based on
207// the given description.
208// Each machine in the description is white-space separated
209// and holds the decimal machine id followed by an optional
210// "v" if the machine wants a vote.
211func mkMachines(description string) []*machine {
212 descrs := parseDescr(description)
213 ms := make([]*machine, len(descrs))
214 for i, d := range descrs {
215 ms[i] = &machine{
216 id: fmt.Sprint(d.id),
217 hostPort: fmt.Sprintf("0.1.2.%d:%d", d.id, mongoPort),
218 wantsVote: strings.Contains(d.flags, "v"),
219 }
220 }
221 return ms
222}
223
224func memberTag(id string) map[string]string {
225 return map[string]string{"juju-machine-id": id}
226}
227
228// mkMembers returns a slice of *replicaset.Member
229// based on the given description.
230// Each member in the description is white-space separated
231// and holds the decimal replica-set id optionally followed by the characters:
232// - 'v' if the member is voting.
233// - 'T' if the member has no associated machine tags.
234// Unless the T flag is specified, the machine tag
235// will be the replica-set id + 10.
236func mkMembers(description string) []replicaset.Member {
237 descrs := parseDescr(description)
238 ms := make([]replicaset.Member, len(descrs))
239 for i, d := range descrs {
240 machineId := d.id + 10
241 m := replicaset.Member{
242 Id: d.id,
243 Address: fmt.Sprintf("0.1.2.%d:%d", machineId, mongoPort),
244 Tags: memberTag(fmt.Sprint(machineId)),
245 }
246 if !strings.Contains(d.flags, "v") {
247 m.Priority = newFloat64(0)
248 m.Votes = newInt(0)
249 }
250 if strings.Contains(d.flags, "T") {
251 m.Tags = nil
252 }
253 ms[i] = m
254 }
255 return ms
256}
257
258var stateFlags = map[rune]replicaset.MemberState{
259 'p': replicaset.PrimaryState,
260 's': replicaset.SecondaryState,
261}
262
263// mkStatuses returns a slice of *replicaset.Member
264// based on the given description.
265// Each member in the description is white-space separated
266// and holds the decimal replica-set id optionally followed by the
267// characters:
268// - 'H' if the instance is not healthy.
269// - 'p' if the instance is in PrimaryState
270// - 's' if the instance is in SecondaryState
271func mkStatuses(description string) []replicaset.MemberStatus {
272 descrs := parseDescr(description)
273 ss := make([]replicaset.MemberStatus, len(descrs))
274 for i, d := range descrs {
275 machineId := d.id + 10
276 s := replicaset.MemberStatus{
277 Id: d.id,
278 Address: fmt.Sprintf("0.1.2.%d:%d", machineId, mongoPort),
279 Healthy: !strings.Contains(d.flags, "H"),
280 State: replicaset.UnknownState,
281 }
282 for _, r := range d.flags {
283 if state, ok := stateFlags[r]; ok {
284 s.State = state
285 }
286 }
287 ss[i] = s
288 }
289 return ss
290}
291
292type descr struct {
293 id int
294 flags string
295}
296
297func isNotDigit(r rune) bool {
298 return r < '0' || r > '9'
299}
300
301var parseDescrTests = []struct {
302 descr string
303 expect []descr
304}{{
305 descr: "",
306 expect: []descr{},
307}, {
308 descr: "0",
309 expect: []descr{{id: 0}},
310}, {
311 descr: "1foo",
312 expect: []descr{{id: 1, flags: "foo"}},
313}, {
314 descr: "10c 5 6443arble ",
315 expect: []descr{{
316 id: 10,
317 flags: "c",
318 }, {
319 id: 5,
320 }, {
321 id: 6443,
322 flags: "arble",
323 }},
324}}
325
326func (*desiredPeerGroupSuite) TestParseDescr(c *gc.C) {
327 for i, test := range parseDescrTests {
328 c.Logf("test %d. %q", i, test.descr)
329 c.Assert(parseDescr(test.descr), jc.DeepEquals, test.expect)
330 }
331}
332
333// parseDescr parses white-space separated fields of the form
334// <id><flags> into descr structures.
335func parseDescr(s string) []descr {
336 fields := strings.Fields(s)
337 descrs := make([]descr, len(fields))
338 for i, field := range fields {
339 d := &descrs[i]
340 i := strings.IndexFunc(field, isNotDigit)
341 if i == -1 {
342 i = len(field)
343 }
344 id, err := strconv.Atoi(field[0:i])
345 if err != nil {
346 panic(fmt.Errorf("bad field %q", field))
347 }
348 d.id = id
349 d.flags = field[i:]
350 }
351 return descrs
352}
353
354type membersById []replicaset.Member
355
356func (l membersById) Len() int { return len(l) }
357func (l membersById) Swap(i, j int) { l[i], l[j] = l[j], l[i] }
358func (l membersById) Less(i, j int) bool { return l[i].Id < l[j].Id }

Subscribers

People subscribed via source and target branches

to status/vote changes: