Merge lp:~fgimenez/snappy/fix-expected-yaml-parse-error into lp:~snappy-dev/snappy/snappy-moved-to-github

[Federico Gimenez]

Fixed expected error message.

Fixed expected error message.

Maybe we could read the actual error messages from somewhere? If not, when all the cases will be covered, with every change of the messages the tests are going to break.

[Leo Arias (elopio)]

We'll need this only for 15.04, that's the old message. So please keep this branch for now, and we'll merge it into the branch that backports the tests to 15.04.

Once we do the backport, and we start running the tests on MPs, then every change in the code that breaks a test will have to update the test. This in turn will make things easy to backport, because code changes will be backported together with test changes.

Reading the expected values from the source code is a slippery road, as sometimes we will miss errors because a change in the code will automatically change the test to keep passing. We lose the option to start with a failing test. In this case it could be a good experiment, because the error message itself should be verified on a unit test, and on the integration suite we just need to confirm that it fails.

So we could have in the code:

msg := "err"

In the unit test:

assert output Equals "err"

In the integration test:

assert output Equals src.msg.

Then if we want to change "err" to "foo", we can start with a failing unit test that does

assert output Equals "foo"

review: Disapprove

[Federico Gimenez (fgimenez)]

I've proposed lp:~fgimenez/snappy/generalize-build-tests-across-versions that generalizes this one with valid messages for both releases, this branch can be deleted.

Unmerged revisions

569. By Federico Gimenez on 2015-07-08

Fixed error message

=== modified file '_integration-tests/tests/latest/build_test.go'
--- _integration-tests/tests/latest/build_test.go	2015-07-04 05:49:13 +0000
+++ _integration-tests/tests/latest/build_test.go	2015-07-08 16:05:35 +0000
@@ -73,7 +73,7 @@
 }
 
 func (s *buildSuite) TestBuildWrongYamlSnapOnSnappy(c *C) {
-	commonWrongTest(c, wrongYamlSnapName, "can not parse package.yaml:.*\n")
+	commonWrongTest(c, wrongYamlSnapName, ".*Can not parse .*\n")
 }
 
 func (s *buildSuite) TestBuildMissingReadmeSnapOnSnappy(c *C) {