1
=== added file 'doc/how-to-write-tests.txt'
2
--- doc/how-to-write-tests.txt	1970-01-01 00:00:00 +0000
3
+++ doc/how-to-write-tests.txt	2014-05-08 22:33:29 +0000
4
@@ -0,0 +1,304 @@
5
1
How to write tests
6
2
==================
7
3
8
4
On the whole, new or updated code will not pass review unless there are tests
9
5
associated with the code.  For code additions, the tests should cover as much
10
6
of the new code as practical, and for code changes, either the tests should be
11
7
updated, or at least the tests that already exist that cover the refactored
12
8
code should be identified when requesting a review to show that there is already
13
9
test coverage, and that the refactoring didn't break anything.
14
10
15
11
16
12
go test and gocheck
17
13
-------------------
18
14
19
15
The `go test` command is used to run the tests.  Juju uses the `gocheck` package
20
16
("launchpad.net/gocheck") to provide a checkers and assert methods for the test
21
17
writers.  The use of gocheck replaces the standard `testing` library.
22
18
23
19
Across all of the tests in juju-core, the gocheck package is imported
24
20
with a shorter alias, because it is used a lot.
25
21
26
22
```go
27
23
import (
28
24
	// system packages
29
25
30
26
	gc "launchpad.net/gocheck"
31
27
32
28
	// juju packages
33
29
)
34
30
```
35
31
36
32
37
33
setting up tests for new packages
38
34
---------------------------------
39
35
40
36
Lets say we are creating a new provider for "magic" cloud, and we have a package
41
37
called "magic" that lives at "launchpad.net/juju-core/provider/magic".  The
42
38
general approach for testing in juju is to have the tests in a separate package.
43
39
Continuing with this example the tests would be in a package called "magic_test".
44
40
45
41
A common idiom that has occurred in juju is to setup to gocheck hooks in a special
46
42
file called `package_test.go` that would look like this:
47
43
48
44
49
45
```go
50
46
// Copyright 2014 Canonical Ltd.
51
47
// Licensed under the AGPLv3, see LICENCE file for details.
52
48
53
49
package magic_test
54
50
55
51
import (
56
52
	"testing"
57
53
58
54
	gc "launchpad.net/gocheck"
59
55
)
60
56
61
57
func Test(t *testing.T) {
62
58
	gc.TestingT(t)
63
59
}
64
60
```
65
61
66
62
or
67
63
68
64
```go
69
65
// Copyright 2014 Canonical Ltd.
70
66
// Licensed under the AGPLv3, see LICENCE file for details.
71
67
72
68
package magic_test
73
69
74
70
import (
75
71
	stdtesting "testing"
76
72
77
73
	"launchpad.net/juju-core/testing"
78
74
)
79
75
80
76
func Test(t *stdtesting.T) {
81
77
	testing.MgoTestPackage(t)
82
78
}
83
79
```
84
80
85
81
The key difference here is that the first one just hooks up `gocheck`
86
82
so it looks for the `gocheck` suites in the package.  The second makes 
87
83
sure that there is a mongo available for the duration of the package tests.
88
84
89
85
A general rule is not to setup mongo for a package unless you really
90
86
need to as it is extra overhead.
91
87
92
88
93
89
writing the test files
94
90
----------------------
95
91
96
92
Normally there will be a test file for each file with code in the package.
97
93
For a file called `config.go` there should be a test file called `config_test.go`.
98
94
99
95
The package should in most cases be the same as the normal files with a "_test" suffix.
100
96
In this way, the tests are testing the same interface as any normal user of the
101
97
package.  It is reasonably common to want to modify some internal aspect of the package
102
98
under test for the tests.  This is normally handled by a file called `export_test.go`.
103
99
Even though the file ends with `_test.go`, the package definition is the same as the
104
100
normal source files. In this way, for the tests and only the tests, additional
105
101
public symbols can be defined for the package and used in the tests.
106
102
107
103
Here is an annotated extract from `provider/local/export_test.go`
108
104
109
105
```go
110
106
// The package is the "local" so it has access to the package symbols
111
107
// and not just the public ones.
112
108
package local
113
109
114
110
import (
115
111
	"github.com/juju/testing"
116
112
	gc "launchpad.net/gocheck"
117
113
118
114
	"launchpad.net/juju-core/environs/config"
119
115
)
120
116
121
117
var (
122
118
	// checkIfRoot is a variable of type `func() bool`, so CheckIfRoot is
123
119
	// a pointer to that variable so we can patch it in the tests.
124
120
	CheckIfRoot      = &checkIfRoot
125
121
	// providerInstance is a pointer to an instance of a private structure.
126
122
	// Provider points to the same instance, so public methods on that instance
127
123
	// are available in the tests.
128
124
	Provider         = providerInstance
129
125
)
130
126
131
127
// ConfigNamespace is a helper function for the test that steps through a
132
128
// number of private methods or variables, and is an alternative mechanism
133
129
// to provide functionality for the tests.
134
130
func ConfigNamespace(cfg *config.Config) string {
135
131
	env, _ := providerInstance.Open(cfg)
136
132
	return env.(*localEnviron).config.namespace()
137
133
}
138
134
```
139
135
140
136
Suites and Juju base suites
141
137
---------------------------
142
138
143
139
With gocheck tests are grouped into Suites. Each suite has distinct
144
140
set-up and tear-down logic.  Suites are often composed of other suites
145
141
that provide specific set-up and tear-down behaviour.
146
142
147
143
There are three main suites:
148
144
149
145
  * /testing/testbase.LoggingSuite (testing/testbase/log.go)
150
146
  * /testing.FakeHomeSuite (testing/environ.go)
151
147
  * /juju/testing.JujuConnSuite (juju/testing/conn.go)
152
148
153
149
The second two have the LoggingSuite functionality included through
154
150
composition.  The LoggingSuite is also composed of the LoggingSuite from 
155
151
github.com/juju/testing, which brings in the CleanupSuite from the same.
156
152
The CleanupSuite has the functionality around patching environment
157
153
variables and normal variables for the duration of a test. It also
158
154
provides a clean-up stack that gets called when the test teardown happens.
159
155
160
156
The FakeHomeSuite creates a temporary directory and sets the HOME environment
161
157
variable to it. It also creates ~/.juju and a simple environments.yaml file,
162
158
~/.ssh with a fake id_rsa.pub key, it isolates the test from the JUJU_HOME,
163
159
JUJU_ENV, and JUJU_LOGGING_CONFIG environment variables.
164
160
165
161
The JujuConnSuite does this and more. It also sets up a state server and api
166
162
server.  This is one problem with the JujuConnSuite, it almost always does a
167
163
lot more than you actually want or need.  This should really be broken into
168
164
smaller component parts that make more sense.  If you can get away with not
169
165
using the JujuConnSuite, you should try.
170
166
171
167
To create a new suite composed of one or more of the suites above, you can do
172
168
something like:
173
169
174
170
```go
175
171
type ToolsSuite struct {
176
172
	testbase.LoggingSuite
177
173
	dataDir string
178
174
}
179
175
180
176
var _ = gc.Suite(&ToolsSuite{})
181
177
182
178
```
183
179
184
180
If there is no extra setup needed, then you don't need to specify any
185
181
set-up or tear-down methods as the LoggingSuite has them, and they are
186
182
called by default.
187
183
188
184
If you did want to do something, say, create a directory and save it in
189
185
the dataDir, you would do something like this:
190
186
191
187
```go
192
188
func (t *ToolsSuite) SetUpTest(c *gc.C) {
193
189
	t.LoggingSuite.SetUpTest(c)
194
190
	t.dataDir = c.MkDir()
195
191
}
196
192
```
197
193
198
194
If the test suite has multiple contained suites, please call them in the
199
195
order that they are defined, and make sure something that is composed from
200
196
the LoggingSuite is first.  They should be torn down in the reverse order.
201
197
202
198
Even if the code that is being tested currently has no logging in it, it
203
199
is a good idea to use the LoggingSuite as a base for two reasons:
204
200
 * it brings in something composed of the CleanupSuite
205
201
 * if someone does add logging later, it is captured and doesn't polute
206
202
   the logging output
207
203
208
204
209
205
Patching variables and the environment
210
206
--------------------------------------
211
207
212
208
Inside a test, and assuming that the Suite has a CleanupSuite somewhere
213
209
in the composition tree, there are a few very helpful functions.
214
210
215
211
```go
216
212
217
213
var foo int
218
214
219
215
func (s *someTest) TestFubar(c *gc.C) {
220
216
	// The TEST_OMG environment value will have "new value" for the duration
221
217
	// of the test.
222
218
	s.PatchEnvironment("TEST_OMG", "new value")
223
219
224
220
	// foo is set to the value 42 for the duration of the test
225
221
	s.PatchValue(&foo, 42)
226
222
}
227
223
```
228
224
229
225
PatchValue works with any matching type. This includes function variables.
230
226
231
227
232
228
Checkers
233
229
--------
234
230
235
231
Checkers are a core concept of `gocheck` and will feel familiar to anyone
236
232
who has used the python testtools.  Assertions are made on the gocheck.C
237
233
methods.
238
234
239
235
```go
240
236
c.Check(err, gc.IsNil)
241
237
c.Assert(something, gc.Equals, somethingElse)
242
238
```
243
239
244
240
The `Check` method will cause the test to fail if the checker returns
245
241
false, but it will continue immediately cause the test to fail and will
246
242
continue with the test. `Assert` if it fails will cause the test to
247
243
immediately stop.
248
244
249
245
For the purpose of further discussion, we have the following parts:
250
246
251
247
	`c.Assert(observed, checker, args...)`
252
248
253
249
The key checkers in the `gocheck` module that juju uses most frequently are:
254
250
255
251
	* `IsNil` - the observed value must be `nil`
256
252
	* `NotNil` - the observed value must not be `nil`
257
253
	* `Equals` - the observed value must be the same type and value as the arg,
258
254
	  which is the expected value
259
255
	* `DeepEquals` - checks for equality for more complex types like slices,
260
256
	  maps, or structures. This is DEPRECATED in favour of the DeepEquals from
261
257
	  the `github.com/juju/testing/checkers` covered below
262
258
	* `ErrorMatches` - the observed value is expected to be an `error`, and
263
259
	  the arg is a string that is a regular expression, and used to match the
264
260
	  error string
265
261
	* `Matches` - a regular expression match where the observed value is a string
266
262
    * `HasLen` - the expected value is an integer, and works happily on nil
267
263
      slices or maps
268
264
269
265
270
266
Over time in the juju project there were repeated patterns of testing that
271
267
were then encoded into new and more complicated checkers.  These are found
272
268
in `github.com/juju/testing/checkers`, and are normally imported with the
273
269
alias `jc`.
274
270
275
271
The matchers there include (not an exclusive list):
276
272
277
273
	* `IsTrue` - just an easier way to say `gc.Equals, true`
278
274
	* `IsFalse` - observed value must be false
279
275
	* `GreaterThan` - for integer or float types
280
276
	* `LessThan` - for integer or float types
281
277
	* `HasPrefix` - obtained is expected to be a string or a `Stringer`, and
282
278
	  the string (or string value) must have the arg as start of the string
283
279
	* `HasSuffix` - the same as `HasPrefix` but checks the end of the string
284
280
	* `Contains` - obtained is a string or `Stringer` and expected needs to be
285
281
	  a string. The checker passes if the expected string is a substring of the
286
282
	  obtained value.
287
283
	* `DeepEquals` - works the same way as the `gocheck.DeepEquals` except
288
284
	  gives better errors when the values do not match
289
285
	* `SameContents` - obtained and expected are slices of the same type,
290
286
	  the checker makes sure that the values in one are in the other. They do
291
287
	  not have the be in the same order.
292
288
	* `Satisfies` - the arg is expected to be `func(observed) bool`
293
289
	  often used for error type checks
294
290
	* `IsNonEmptyFile` - obtained is a string or `Stringer` and refers to a
295
291
	  path. The checker passes if the file exists, is a file, and is not empty
296
292
	* `IsDirectory` - works in a similar way to `IsNonEmptyFile` but passes if
297
293
	  the path element is a directory
298
294
	* `DoesNotExist` - also works with a string or `Stringer`, and passes if
299
295
	  the path element does not exist
300
296
301
297
302
298
303
299
Good tests
304
300
----------
305
301
306
302
Good tests should be:
307
303
  * small and obviously correct
308
304
  * isolated from any system or environment values that may impact the test
Status:	Merged
Approved by:	Tim Penhey on 2014-05-08
Approved revision:	no longer in the source branch.
Merged at revision:	2711
Proposed branch:	lp:~thumper/juju-core/testing-docs
Merge into:	lp:~go-bot/juju-core/trunk
Diff against target:	308 lines (+304/-0) 1 file modified doc/how-to-write-tests.txt (+304/-0)
To merge this branch:	bzr merge lp:~thumper/juju-core/testing-docs
Related bugs:	Link a bug report
Reviewer	Review Type	Date Requested	Status
Juju Engineering		2014-05-08	Pending
Review via email: mp+218732@code.launchpad.net