I got integration tests to run by changing this in the Makefile: -build: deps develop +build: deps But then I got a bunch of legit failures from the tests: 2 tvansteenburgh@trusty-vm:/tmp/charm-tools> make check tests_functional/helpers/helpers.sh || sh -x tests_functional/helpers/helpers.sh timeout Fri Aug 28 11:04:24 EDT 2015: Testing ch_apparmor_load...PASS Fri Aug 28 11:04:24 EDT 2015: Testing ch_type_hash...PASS Fri Aug 28 11:04:24 EDT 2015: Testing ch_is_url...PASS Fri Aug 28 11:04:24 EDT 2015: Testing Testing ch_is_ip...PASS Fri Aug 28 11:04:24 EDT 2015: Testing Testing ch_get_ip...PASS Fri Aug 28 11:04:24 EDT 2015: Starting SimpleHTTPServer in /tmp/charm-helper-srv.BlmbCS on port 8999 to test fetching files. Fri Aug 28 11:04:24 EDT 2015: Looping wget until webserver responds... Fri Aug 28 11:04:25 EDT 2015: Attempt 1 succeeded. Fri Aug 28 11:04:25 EDT 2015: Creating temp data file Fri Aug 28 11:04:25 EDT 2015: creating gzipped test data Fri Aug 28 11:04:25 EDT 2015: Testing ch_get_file...PASS Fri Aug 28 11:04:25 EDT 2015: Shutting down webserver...DONE Fri Aug 28 11:04:25 EDT 2015: Printing server log 127.0.0.1 - - [28/Aug/2015 11:04:25] "GET / HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt.gz HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt.gz HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt.gz HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:25] "GET /testdata.txt HTTP/1.1" 200 - Fri Aug 28 11:04:25 EDT 2015: Testing ch_unit_name...PASS Fri Aug 28 11:04:25 EDT 2015: Testing ch_unit_id...PASS Fri Aug 28 11:04:25 EDT 2015: Testing ch_my_unit_id...PASS Test shell helpers with bash bash tests_functional/helpers/helpers.sh \ || bash -x tests_functional/helpers/helpers.sh timeout Fri Aug 28 11:04:25 EDT 2015: Testing ch_apparmor_load...PASS Fri Aug 28 11:04:25 EDT 2015: Testing ch_type_hash...PASS Fri Aug 28 11:04:25 EDT 2015: Testing ch_is_url...PASS Fri Aug 28 11:04:25 EDT 2015: Testing Testing ch_is_ip...PASS Fri Aug 28 11:04:25 EDT 2015: Testing Testing ch_get_ip...PASS Fri Aug 28 11:04:25 EDT 2015: Starting SimpleHTTPServer in /tmp/charm-helper-srv.TOq187 on port 8999 to test fetching files. Fri Aug 28 11:04:25 EDT 2015: Looping wget until webserver responds... Fri Aug 28 11:04:26 EDT 2015: Attempt 1 succeeded. Fri Aug 28 11:04:26 EDT 2015: Creating temp data file Fri Aug 28 11:04:26 EDT 2015: creating gzipped test data Fri Aug 28 11:04:26 EDT 2015: Testing ch_get_file...PASS Fri Aug 28 11:04:26 EDT 2015: Shutting down webserver...DONE Fri Aug 28 11:04:26 EDT 2015: Printing server log 127.0.0.1 - - [28/Aug/2015 11:04:26] "GET / HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:26] "GET /testdata.txt HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:26] "GET /testdata.txt HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:26] "GET /testdata.txt HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:26] "GET /testdata.txt.gz HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:26] "GET /testdata.txt.gz HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:26] "GET /testdata.txt.gz HTTP/1.1" 200 - 127.0.0.1 - - [28/Aug/2015 11:04:26] "GET /testdata.txt HTTP/1.1" 200 - Fri Aug 28 11:04:26 EDT 2015: Testing ch_unit_name...PASS Fri Aug 28 11:04:26 EDT 2015: Testing ch_unit_id...PASS Fri Aug 28 11:04:26 EDT 2015: Testing ch_my_unit_id...PASS tests_functional/helpers/helpers.bash || sh -x tests_functional/helpers/helpers.bash timeout Fri Aug 28 11:04:26 EDT 2015: Creating fake local repos... Fri Aug 28 11:04:26 EDT 2015: Testing ch_detect_vcs...PASS Fri Aug 28 11:04:26 EDT 2015: Cleaning up fake repos... Test shell helpers with bash bash tests_functional/helpers/helpers.bash \ || bash -x tests_functional/helpers/helpers.bash timeout Fri Aug 28 11:04:26 EDT 2015: Creating fake local repos... Fri Aug 28 11:04:26 EDT 2015: Testing ch_detect_vcs...PASS Fri Aug 28 11:04:26 EDT 2015: Cleaning up fake repos... Test charm proof tests_functional/proof/test.sh tests_functional/proof/record.sh: 13: tests_functional/proof/record.sh: tests_functional/proof/../../bin/python: not found tests_functional/proof/record.sh: 13: tests_functional/proof/record.sh: tests_functional/proof/../../bin/python: not found tests_functional/proof/record.sh: 13: tests_functional/proof/record.sh: tests_functional/proof/../../bin/python: not found tests_functional/proof/record.sh: 13: tests_functional/proof/record.sh: tests_functional/proof/../../bin/python: not found tests_functional/proof/record.sh: 13: tests_functional/proof/record.sh: tests_functional/proof/../../bin/python: not found tests_functional/proof/record.sh: 13: tests_functional/proof/record.sh: tests_functional/proof/../../bin/python: not found tests_functional/proof/record.sh: 13: tests_functional/proof/record.sh: tests_functional/proof/../../bin/python: not found tests_functional/proof/record.sh: 13: tests_functional/proof/record.sh: tests_functional/proof/../../bin/python: not found tests_functional/proof/record.sh: 13: tests_functional/proof/record.sh: tests_functional/proof/../../bin/python: not found tests_functional/proof/record.sh: 13: tests_functional/proof/record.sh: tests_functional/proof/../../bin/python: not found tests_functional/proof/record.sh: 13: tests_functional/proof/record.sh: tests_functional/proof/../../bin/python: not found diff -ur tests_functional/proof/expected/broken-categories tests_functional/proof/results/broken-categories --- tests_functional/proof/expected/broken-categories 2015-08-27 15:30:45.151720000 -0400 +++ tests_functional/proof/results/broken-categories 2015-08-28 11:04:26.920998203 -0400 @@ -1,3 +0,0 @@ -W: Categories metadata must be a list of one or more of: applications, app-servers, databases, file-servers, cache-proxy, misc -W: Categories are being deprecated in favor of tags. Please rename the "categories" field to "tags". -I: missing recommended hook config-changed diff -ur tests_functional/proof/expected/broken-config tests_functional/proof/results/broken-config --- tests_functional/proof/expected/broken-config 2015-08-27 15:30:45.151720000 -0400 +++ tests_functional/proof/results/broken-config 2015-08-28 11:04:26.924996153 -0400 @@ -1,2 +0,0 @@ -I: missing recommended hook config-changed -W: config.yaml: option foo has an invalid type (integer) diff -ur tests_functional/proof/expected/broken-maintainer tests_functional/proof/results/broken-maintainer --- tests_functional/proof/expected/broken-maintainer 2015-08-27 15:45:57.054552000 -0400 +++ tests_functional/proof/results/broken-maintainer 2015-08-28 11:04:26.924996153 -0400 @@ -1,12 +0,0 @@ -I: metadata name (test) must match directory name (broken-maintainer) exactly for local deployment. -W: Maintainer format should be "Name ", not "test@testhost" -W: no copyright file -W: Includes template README.ex file -W: README.ex includes boilerplate: Describe the intended usage of this charm and anything unique about how this charm relates to others here. -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -I: missing recommended hook config-changed diff -ur tests_functional/proof/expected/broken-subordinate tests_functional/proof/results/broken-subordinate --- tests_functional/proof/expected/broken-subordinate 2015-08-27 15:30:45.151720000 -0400 +++ tests_functional/proof/results/broken-subordinate 2015-08-28 11:04:26.924996153 -0400 @@ -1,2 +0,0 @@ -E: subordinates must have at least one scope: container relation -I: missing recommended hook config-changed diff -ur tests_functional/proof/expected/broken-subordinate2 tests_functional/proof/results/broken-subordinate2 --- tests_functional/proof/expected/broken-subordinate2 2015-08-27 15:30:45.151720000 -0400 +++ tests_functional/proof/results/broken-subordinate2 2015-08-28 11:04:26.924996153 -0400 @@ -1,2 +0,0 @@ -E: subordinates must have at least one scope: container relation -I: missing recommended hook config-changed diff -ur tests_functional/proof/expected/empty-requires tests_functional/proof/results/empty-requires --- tests_functional/proof/expected/empty-requires 2015-08-27 15:30:45.151720000 -0400 +++ tests_functional/proof/results/empty-requires 2015-08-28 11:04:26.928994102 -0400 @@ -1,2 +0,0 @@ -I: all charms should provide at least one thing -I: missing recommended hook config-changed diff -ur tests_functional/proof/expected/icon-template tests_functional/proof/results/icon-template --- tests_functional/proof/expected/icon-template 2015-08-27 15:45:57.054552000 -0400 +++ tests_functional/proof/results/icon-template 2015-08-28 11:04:26.928994102 -0400 @@ -1,15 +0,0 @@ -I: Includes template icon.svg file. -W: no copyright file -W: Includes template README.ex file -W: README.ex includes boilerplate: Describe the intended usage of this charm and anything unique about how this charm relates to others here. -E: template interface names should be changed: interface-name -E: Unknown relation field in relation relation-name - (baz) -E: template relations should be renamed to fit charm: relation-name -E: Unknown scope found in relation relation-name - (flouride) -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -E: relation non-map is not a map -I: relation non-map has no hooks -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -I: missing recommended hook config-changed diff -ur tests_functional/proof/expected/missing-maintainer tests_functional/proof/results/missing-maintainer --- tests_functional/proof/expected/missing-maintainer 2015-08-27 15:45:57.054552000 -0400 +++ tests_functional/proof/results/missing-maintainer 2015-08-28 11:04:26.928994102 -0400 @@ -1,12 +0,0 @@ -I: metadata name (test) must match directory name (missing-maintainer) exactly for local deployment. -E: Charm must have either a maintainer or maintainers field -W: no copyright file -W: Includes template README.ex file -W: README.ex includes boilerplate: Describe the intended usage of this charm and anything unique about how this charm relates to others here. -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -I: missing recommended hook config-changed diff -ur tests_functional/proof/expected/mod-spdy tests_functional/proof/results/mod-spdy --- tests_functional/proof/expected/mod-spdy 2015-08-27 15:30:45.151720000 -0400 +++ tests_functional/proof/results/mod-spdy 2015-08-28 11:04:26.932992052 -0400 @@ -1,3 +0,0 @@ -I: missing recommended hook start -I: missing recommended hook stop -I: missing recommended hook config-changed diff -ur tests_functional/proof/expected/test tests_functional/proof/results/test --- tests_functional/proof/expected/test 2015-08-27 15:45:57.054552000 -0400 +++ tests_functional/proof/results/test 2015-08-28 11:04:26.932992052 -0400 @@ -1,12 +0,0 @@ -W: Metadata missing required field "tags" -I: No icon.svg file. -W: no copyright file -W: Includes template README.ex file -W: README.ex includes boilerplate: Describe the intended usage of this charm and anything unique about how this charm relates to others here. -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -I: missing recommended hook config-changed diff -ur tests_functional/proof/expected/unknown-metadata tests_functional/proof/results/unknown-metadata --- tests_functional/proof/expected/unknown-metadata 2015-08-27 15:45:57.054552000 -0400 +++ tests_functional/proof/results/unknown-metadata 2015-08-28 11:04:26.932992052 -0400 @@ -1,16 +0,0 @@ -E: Unknown root metadata field (foo) -I: metadata name (test) must match directory name (unknown-metadata) exactly for local deployment. -W: no copyright file -W: Includes template README.ex file -W: README.ex includes boilerplate: Describe the intended usage of this charm and anything unique about how this charm relates to others here. -E: template interface names should be changed: interface-name -E: Unknown relation field in relation relation-name - (baz) -E: template relations should be renamed to fit charm: relation-name -E: Unknown scope found in relation relation-name - (flouride) -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -E: relation non-map is not a map -I: relation non-map has no hooks -E: template interface names should be changed: interface-name -E: template relations should be renamed to fit charm: relation-name -I: missing recommended hook config-changed Differences detected (see above). Abort! make: *** [integration] Error 1