Merge ~vultaire/charm-kubernetes-service-checks:remove-charmcraft-python-dep into charm-kubernetes-service-checks:master

[Paul Goins]

Update for more recent charmcraft

Changes include:
* Remove dependency on charmcraft Python lib in preference of the snap
* Update Makefile to work with charmcraft 1.1.0 builds.

The "fixes" here are intended as a bridge until this charm is migrated
to CharmHub, after which we may want to leverage the new build features
allowing for building targeted charms with different
channels/architectures via charmcraft.yaml.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:516d641d389a844c9b5319162877a9e4e19de255
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/9/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-unit/141/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/351/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/9//rebuild

review: Approve (continuous-integration)

[Paul Goins (vultaire)]

After reviewing charmcraft.yaml (a new config file used by the charmcraft snap), I think this set of patches was the wrong way to go about things.

"charmcraft build" can generate multiple charms now. And I'm not sure how this works together with the charm store, or if it forces moving to charmhub. This needs deeper review.

review: Needs Fixing

[Paul Goins (vultaire)]

This may be an OK "mid-term" solution which builds toward the long-term solution of porting this charm over to CharmHub. While the build artifact here is indeed built with some platform-specific bits, they all have fallbacks to pure-Python versions, and so we can effectively use these changes together with the newer charmcraft to continue our current flow.

Thus, I'm reverting my Needs Fixing, and adding an extra reviewer slot.

review: Approve

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:64a2c3c2f2651baf739800c966146270dafeef5b
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/11/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-unit/148/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/366/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/11//rebuild

review: Approve (continuous-integration)

3e899b1... by Paul Goins on 2021-07-29

Update for CharmHub and recent charmcraft

* Replace charmcraft Python dependency with charmcraft snap

* Added per-series builds

* Added .jujuignore per "charmcraft init" to avoid including certain
  files during the build process

* Updated .gitignore to ignore *.charm files

* Updated Makefile appropriately

* Updated functional tests to use the per-arch charms

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:80f378b6e98389076697e086b1a70ba13d2102b6
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/12/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-unit/149/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/367/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/12//rebuild

review: Approve (continuous-integration)

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:7a9cfa4910da24a2e85bbc1dfb1c977f656cdb25
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/13/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-unit/150/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/368/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/13//rebuild

review: Approve (continuous-integration)

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[Paul Goins (vultaire)]

I've been doing some rework here; this isn't quite ready yet. The functional tests need to be updated to use the new per-arch builds.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:a79f87219887fbc126a8ee2ca456a32e408ccb61
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/14/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-unit/151/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/369/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/14//rebuild

review: Approve (continuous-integration)

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:3e899b18a4ce1b8c86a04510a1a8db9d7905a4f7
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/15/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-unit/152/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/370/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-charm-kubernetes-service-checks-ci/15//rebuild

review: Approve (continuous-integration)

[Peter Sabaini (peter-sabaini)]

Hey Paul, based on your last comment I'm setting this to WIP, hope thats ok -- naturally feel free to set it back!

[Paul Goins (vultaire)]

No; that's totally OK!

And it seems there's not much of a hurry here, either. We're effectively blocked from pushing direct to CharmHub for now; legacy charms are copied over read-only for now. This will eventually change though. (See this thread: https://discourse.charmhub.io/t/charmhub-groups-support/4913)

[Paul Goins (vultaire)]

Adding context to ensure intent is clear: as we can't (yet) upload directly to CharmHub, we can't yet leverage the multiple per-target charm builds generated by the charmcraft tool.

Unmerged commits

3e899b1... by Paul Goins on 2021-07-29

Update for CharmHub and recent charmcraft

* Replace charmcraft Python dependency with charmcraft snap

* Added per-series builds

* Added .jujuignore per "charmcraft init" to avoid including certain
  files during the build process

* Updated .gitignore to ignore *.charm files

* Updated Makefile appropriately

* Updated functional tests to use the per-arch charms

diff --git a/.gitignore b/.gitignore
index 5237a11..5581913 100644
--- a/.gitignore
+++ b/.gitignore
@@ -40,3 +40,4 @@ dist/
 # Builds
 .build/
 build/
+/*.charm
diff --git a/.jujuignore b/.jujuignore
new file mode 100644
index 0000000..6ccd559
--- /dev/null
+++ b/.jujuignore
@@ -0,0 +1,3 @@
+/venv
+*.py[cod]
+*.charm
diff --git a/Makefile b/Makefile
index cb7ac8b..be3c2f4 100644
--- a/Makefile
+++ b/Makefile
@@ -1,4 +1,5 @@
 PYTHON := /usr/bin/python3
+TEMPDIR := /var/tmp
 
 PROJECTPATH=$(dir $(realpath $(MAKEFILE_LIST)))
 ifndef CHARM_BUILD_DIR
@@ -28,7 +29,7 @@ clean:
 	@echo "Cleaning files"
 	@git clean -ffXd -e '!.idea'
 	@echo "Cleaning existing build"
-	@rm -rf ${CHARM_BUILD_DIR}/${CHARM_NAME}
+	@rm -f ${CHARM_BUILD_DIR}/${CHARM_NAME}*.charm
 
 submodules:
 	@echo "Cloning submodules"
@@ -39,27 +40,24 @@ submodules-update:
 	@git submodule update --init --recursive --remote --merge
 
 build:
-	@echo "Building charm to base directory ${CHARM_BUILD_DIR}/${CHARM_NAME}"
+	@echo "Building charm into ${CHARM_BUILD_DIR}..."
 	@-git rev-parse --abbrev-ref HEAD > ./repo-info
 	@-git describe --always > ./version
 	@mkdir -p ${CHARM_BUILD_DIR}
 	@tox -e build
-	@mv ${CHARM_NAME}.charm ${CHARM_BUILD_DIR}
+	@cp -v $$(ls -1rt ${CHARM_NAME}_*.charm | tail -n1) ${TEMPDIR}/${CHARM_NAME}_proof.charm
+	@mv -v ${CHARM_NAME}_*.charm ${CHARM_BUILD_DIR}
 
 
-release: clean build unpack
-	@echo "Charm is built at ${CHARM_BUILD_DIR}/${CHARM_NAME}"
+release: clean build
+	@echo; echo "Charm is built at the following locations:"
+	@bash -c 'ls -1 "${CHARM_BUILD_DIR}/"${CHARM_NAME}_*.charm'
 
 unpack: build
-	@-rm -rf ${CHARM_BUILD_DIR}/${CHARM_NAME}
-	@mkdir -p ${CHARM_BUILD_DIR}/${CHARM_NAME}
-	@echo "Unpacking built .charm into ${CHARM_BUILD_DIR}/${CHARM_NAME}"
-	@cd ${CHARM_BUILD_DIR}/${CHARM_NAME} && unzip -q ${CHARM_BUILD_DIR}/${CHARM_NAME}.charm
-	# until charmcraft copies READMEs in, we need to publish charms with readmes in them.
-	@cp ${PROJECTPATH}/README.md ${CHARM_BUILD_DIR}/${CHARM_NAME}
-	@cp ${PROJECTPATH}/copyright ${CHARM_BUILD_DIR}/${CHARM_NAME}
-	@cp ${PROJECTPATH}/repo-info ${CHARM_BUILD_DIR}/${CHARM_NAME}
-	@cp ${PROJECTPATH}/version ${CHARM_BUILD_DIR}/${CHARM_NAME}
+	@-rm -rf ${TEMPDIR}/${CHARM_NAME}
+	@mkdir -p ${TEMPDIR}/${CHARM_NAME}
+	@echo "Unpacking built .charm into ${TEMPDIR}/${CHARM_NAME}"
+	@cd ${TEMPDIR}/${CHARM_NAME} && unzip -q ${TEMPDIR}/${CHARM_NAME}_proof.charm
 
 lint:
 	@echo "Running lint checks"
@@ -71,7 +69,7 @@ black:
 
 proof: unpack
 	@echo "Running charm proof"
-	@charm proof ${CHARM_BUILD_DIR}/${CHARM_NAME}
+	@charm proof ${TEMPDIR}/${CHARM_NAME}
 
 unittests:
 	@echo "Running unit tests"
diff --git a/charmcraft.yaml b/charmcraft.yaml
new file mode 100644
index 0000000..d8e00a9
--- /dev/null
+++ b/charmcraft.yaml
@@ -0,0 +1,6 @@
+type: "charm"
+bases:
+  - name: "ubuntu"
+    channel: "18.04"
+  - name: "ubuntu"
+    channel: "20.04"
diff --git a/tests/functional/tests/bundles/overlays/bionic.yaml.j2 b/tests/functional/tests/bundles/overlays/bionic.yaml.j2
index 65b2826..9a8a77d 100644
--- a/tests/functional/tests/bundles/overlays/bionic.yaml.j2
+++ b/tests/functional/tests/bundles/overlays/bionic.yaml.j2
@@ -1 +1,4 @@
 series: bionic
+applications:
+  {{ charm_name }}:
+    charm: {{ CHARM_BUILD_DIR }}/{{ charm_name }}_ubuntu-18.04-amd64.charm
diff --git a/tests/functional/tests/bundles/overlays/focal.yaml.j2 b/tests/functional/tests/bundles/overlays/focal.yaml.j2
index d3f46ab..2355df1 100644
--- a/tests/functional/tests/bundles/overlays/focal.yaml.j2
+++ b/tests/functional/tests/bundles/overlays/focal.yaml.j2
@@ -1 +1,4 @@
 series: focal
+applications:
+  {{ charm_name }}:
+    charm: {{ CHARM_BUILD_DIR }}/{{ charm_name }}_ubuntu-20.04-amd64.charm
diff --git a/tests/functional/tests/bundles/overlays/local-charm-overlay.yaml.j2 b/tests/functional/tests/bundles/overlays/local-charm-overlay.yaml.j2
deleted file mode 100644
index 7f59ee2..0000000
--- a/tests/functional/tests/bundles/overlays/local-charm-overlay.yaml.j2
+++ /dev/null
@@ -1,3 +0,0 @@
-applications:
-  {{ charm_name }}:
-    charm: {{ CHARM_BUILD_DIR }}/{{ charm_name }}.charm
diff --git a/tox.ini b/tox.ini
index a435b08..89d4442 100644
--- a/tox.ini
+++ b/tox.ini
@@ -33,7 +33,7 @@ passenv =
   OS_IDENTITY_API_VERSION
 
 [testenv:build]
-deps = charmcraft<1.1.0
+whitelist_externals = /snap/bin/charmcraft
 commands = charmcraft build
 
 [testenv:lint]