Merge ~lgp171188/launchpad:charming-extract-launchpad-build into launchpad:master

[Guruprasad]

Create a basic reactive charm skeleton for Launchpad

This WIP charm uses the ols layer to
* Download and extract the latest Launchpad build artifact from
  the SWIFT storage. It also does the symlink switching for
  deploying a different build.
* Install the requirements.

[Jürgen Gmach (jugmac00)]

> This WIP charm now uses to the ols layer to

=>

> This WIP charm now uses the ols layer to

[Guruprasad (lgp171188)]

Jürgen, thanks for catching this typo. Fixed now.

[Colin Watson (cjwatson)]

Just deploying "Launchpad" isn't really a coherent thing on its own - we'll want to have `launchpad-appserver`, `launchpad-librarian`, `launchpad-buildd-manager`, etc., and each of those would consist of some common code to put the actual Launchpad code in place plus some service-specific code to do whatever extra bits are needed for each particular deployment type.

To achieve that, I expect we may want to turn this skeleton into a local layer on top of layer:ols (or possibly inline the necessary bits into each service charm if it turns out that the common code is too trivial to be worth bothering with a common layer, but it's probably worth avoiding copying even a dozen lines of code into each service charm if we can). As such I expect we won't retain `charm/launchpad/` in the long run. However, this is fine to land as a starting point for future work.

review: Approve

[Guruprasad (lgp171188)]

> each of those would consist of some common code to put the actual Launchpad code in place plus some service-specific code to do whatever extra bits are needed for each particular deployment type.
>
> To achieve that, I expect we may want to turn this skeleton into a local layer on top of layer:ols
> (or possibly inline the necessary bits into each service charm if it turns out that the common
> code is too trivial to be worth bothering with a common layer, but it's probably worth avoiding
> copying even a dozen lines of code into each service charm if we can).

The current charm directory does not have any code to do the common service setup and `charm/launchpad/reactive/launchpad.py` just adds a hook for `ols.configured` to loop it in and then just logs 'Hello world!'. The actual work is all done by the configuration passed to the 'ols' layer. So I do not know if we have to add a layer on top of the ols layer to do the same things, unless we want to hard-code some configuration in it to abstract it away from the consumers of the layer.

diff --git a/charm/.gitignore b/charm/.gitignore
new file mode 100644
index 0000000..da831d3
--- /dev/null
+++ b/charm/.gitignore
@@ -0,0 +1,4 @@
+*.charm
+bundle.yaml
+dist
+tmp
diff --git a/charm/Makefile b/charm/Makefile
new file mode 100644
index 0000000..de55a73
--- /dev/null
+++ b/charm/Makefile
@@ -0,0 +1,96 @@
+# The charmcraft tool is shipped as a snap, so make sure it's on $PATH.
+export PATH := $(PATH):/snap/bin
+
+APP_NAME := launchpad
+
+BUILDDIR := $(CURDIR)/dist
+TMPDIR := $(CURDIR)/tmp
+CHARM_LAYERS_DIR := $(TMPDIR)/deps/ols-layers/layer
+CHARM_INTERFACES_DIR := $(TMPDIR)/deps/ols-layers/interface
+
+CHARM_SERIES ?= 18.04
+ARCH := $(shell dpkg --print-architecture)
+charm_file = $(1)_ubuntu-$(CHARM_SERIES)-$(ARCH).charm
+
+BUILD_LABEL = $(shell git rev-parse HEAD)
+TARBALL = $(APP_NAME).tar.gz
+ASSET = ../build/$(BUILD_LABEL)/$(TARBALL)
+
+CHARMS := launchpad
+
+all: ## alias to build
+all: build
+
+help: ## display this help message
+	@echo "Please use \`make <target>' where <target> is one of"
+	@grep '^[a-zA-Z]' $(MAKEFILE_LIST) | sort | awk -F ':.*?## ' 'NF==2 {printf "\033[36m  %-25s\033[0m %s\n", $$1, $$2}'
+
+$(BUILDDIR) $(TMPDIR):
+	@mkdir -p $@
+
+# We have to clone our dependencies by hand rather than letting charmcraft
+# do it, since some of them are in private repositories and charmcraft
+# doesn't have suitable credentials.
+CHARM_DEPS := $(CHARM_LAYERS_DIR)/.done $(CHARM_INTERFACES_DIR)/.done
+$(CHARM_DEPS): $(CURDIR)/dependencies.txt | $(TMPDIR)
+	@echo "Fetching dependencies..."
+	@mkdir -p $(TMPDIR)/deps
+	@cd $(TMPDIR)/deps && codetree $<
+	@touch $(CHARM_DEPS)
+
+build: ## build all the charms
+build: $(foreach charm,$(CHARMS),build-$(charm))
+
+build-launchpad: ## build the launchpad charm
+build-launchpad: dist/$(call charm_file,launchpad)
+
+dist/%_ubuntu-$(CHARM_SERIES)-$(ARCH).charm: $(CHARM_DEPS) | $(BUILDDIR)
+	@echo "Building $*..."
+	@rm -rf $*/tmp
+	@cp -a tmp $*/tmp
+	@cd $* && charmcraft pack
+	@cp -a $*/$(call charm_file,$*) dist/
+	@rm -rf $*/tmp
+
+clean: ## clean the build environment
+clean: $(foreach charm,$(CHARMS),clean-$(charm))
+	@find . -name \*.pyc -delete
+	@find . -depth -name '__pycache__' -exec rm -rf '{}' \;
+	@rm -f bundle.yaml
+	@rm -f layer/*/codetree-collect-info.yaml
+	@rm -rf $(BUILDDIR) $(TMPDIR)
+
+clean-%:
+	@echo "Cleaning $*..."
+	@cd $* && charmcraft clean
+	@rm -f dist/$(call charm_file,$*)
+
+bundle.yaml: ## create the bundle.yaml file from the bundle.yaml.in template
+bundle.yaml: bundle.yaml.in
+	sed \
+	    -e 's/%BUILD_LABEL%/$(BUILD_LABEL)/g' \
+	    bundle.yaml.in >bundle.yaml
+
+deploy: ## deploy the built charm
+deploy: build bundle.yaml
+	@echo "Deploying $(APP_NAME)..."
+	@juju deploy ./bundle.yaml
+
+payload: ## build a launchpad tarball
+payload: $(ASSET)
+$(ASSET):
+	@echo "Building asset for $(BUILD_LABEL)..."
+	@$(MAKE) -C .. build-tarball
+
+
+setup-jenkaas: ## prepare a Jenkins-as-a-service container for charm building
+setup-jenkaas:
+	sudo systemctl stop snapd.socket
+	sudo systemctl stop snapd
+	echo SNAPPY_STORE_NO_CDN=1 | sudo tee -a /etc/environment >/dev/null
+	echo SNAPPY_TESTING=1 | sudo tee -a /etc/environment >/dev/null
+	sudo systemctl start snapd.socket
+	sudo snap install --classic charm
+
+.PHONY: $(foreach charm,$(CHARMS),build-$(charm))
+.PHONY: all build clean deploy payload setup-jenkaas
diff --git a/charm/bundle.yaml.in b/charm/bundle.yaml.in
new file mode 100644
index 0000000..82244f1
--- /dev/null
+++ b/charm/bundle.yaml.in
@@ -0,0 +1,10 @@
+series: bionic
+description: "Launchpad development bundle"
+applications:
+  launchpad:
+    charm: ./dist/launchpad_ubuntu-18.04-amd64.charm
+    num_units: 1
+    options:
+      build_label: "%BUILD_LABEL%"
+      swift_storage_url: "https://radosgw.ps5.canonical.com/swift/v1/AUTH_318a449e4c8e40e2865c120a597a9644"
+      swift_container_name: "launchpad-builds"
diff --git a/charm/dependencies.txt b/charm/dependencies.txt
new file mode 100644
index 0000000..a4a6420
--- /dev/null
+++ b/charm/dependencies.txt
@@ -0,0 +1,2 @@
+ols-layers		git+ssh://git.launchpad.net/~ubuntuone-pqm-team/ols-charm-deps/+git/ols-layers;revno=df20c87d
+charm-wheels		git+ssh://git.launchpad.net/~ubuntuone-hackers/ols-charm-deps/+git/wheels;revno=fe523e25
diff --git a/charm/launchpad/charmcraft.yaml b/charm/launchpad/charmcraft.yaml
new file mode 100644
index 0000000..5b50137
--- /dev/null
+++ b/charm/launchpad/charmcraft.yaml
@@ -0,0 +1,20 @@
+type: charm
+bases:
+  - build-on:
+    - name: ubuntu
+      channel: "18.04"
+    run-on:
+    - name: ubuntu
+      channel: "18.04"
+parts:
+  launchpad:
+    source: .
+    plugin: reactive
+    build-snaps: [charm]
+    build-environment:
+      - CHARM_LAYERS_DIR: tmp/deps/ols-layers/layer
+      - CHARM_INTERFACES_DIR: tmp/deps/ols-layers/interface
+      - PIP_NO_INDEX: "true"
+      - PIP_FIND_LINKS: tmp/deps/charm-wheels
+    stage:
+      - -tmp
diff --git a/charm/launchpad/config.yaml b/charm/launchpad/config.yaml
new file mode 100644
index 0000000..adc4e2c
--- /dev/null
+++ b/charm/launchpad/config.yaml
@@ -0,0 +1,8 @@
+options:
+  # layer-apt
+  install_sources:
+    default: |
+      - ppa:launchpad/ppa
+  install_keys:
+    default: |
+      - null  # PPA keys securely added via Launchpad.
diff --git a/charm/launchpad/icon.svg b/charm/launchpad/icon.svg
new file mode 100644
index 0000000..b2889cc
--- /dev/null
+++ b/charm/launchpad/icon.svg
@@ -0,0 +1 @@
+<svg id="Layer_1" data-name="Layer 1" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 165.39062 165.39062"><defs><style>.cls-1{fill:#e9500e;}.cls-2{fill:#fff;}</style></defs><rect class="cls-1" width="165.39062" height="165.39062"/><path class="cls-2" d="M29.63876,57.97189C43.189,67.692,61.13456,69.25577,77.65457,62.15038c16.25576-6.87157,27.74036-21.43444,29.97828-38.0075.04663-.34331.11016-.81367-1.59861-1.24044l-10.10934-2.197c-.3254-.04494-.79136-.04967-1.15258,1.22455C91.37844,36.07384,84.34062,45.04243,72.6347,50.1123c-11.77316,5.10029-23.18748,4.05279-35.91893-3.29386-.58119-.27843-.91909-.26086-1.45568.52577l-5.77947,8.65163A1.34512,1.34512,0,0,0,29.63876,57.97189Z" transform="translate(0.39062 0.39062)"/><path class="cls-2" d="M79.86106,139.66026l10.3631.565c1.74155.03446,1.79122-.42981,1.83717-.77312,2.23826-16.5734-4.97222-33.66107-18.81739-44.59422C59.196,83.62132,41.47815,80.36935,25.83365,86.14747a1.33956,1.33956,0,0,0-.67918,1.85373l3.28,9.88226c.30952.90153.62816,1.011,1.26443.89409,14.22464-3.70543,25.50717-1.68748,35.50635,6.3512,9.94174,7.9934,14.34865,18.50754,13.86883,33.08867C79.08524,139.50144,79.53735,139.615,79.86106,139.66026Z" transform="translate(0.39062 0.39062)"/><path class="cls-2" d="M86.50488,70.59048a10.50817,10.50817,0,0,0-1.39587-.09461A9.35237,9.35237,0,0,0,79.39915,72.382a9.61981,9.61981,0,1,0,7.10573-1.79156Z" transform="translate(0.39062 0.39062)"/><path class="cls-2" d="M138.26869,53.18923,133.457,43.97736c-.68628-1.51583-1.22793-1.36985-1.79594-1.17657-15.382,6.63165-25.99848,21.22156-28.40434,39.03776-2.40755,17.82971,3.97169,34.72681,17.0647,45.19906a1.177,1.177,0,0,0,.90794.32844,1.48362,1.48362,0,0,0,.99546-.54l6.76175-8.11166c.62342-.78393.35783-1.18333.0321-1.52461-10.60639-10.44454-14.5764-20.81677-12.84905-33.60769,1.73682-12.86121,8.51918-22.08254,21.34457-29.019C138.52854,53.95289,138.36533,53.421,138.26869,53.18923Z" transform="translate(0.39062 0.39062)"/></svg>
\ No newline at end of file
diff --git a/charm/launchpad/layer.yaml b/charm/launchpad/layer.yaml
new file mode 100644
index 0000000..ae16aa0
--- /dev/null
+++ b/charm/launchpad/layer.yaml
@@ -0,0 +1,15 @@
+includes:
+  - layer:basic
+  - layer:ols
+options:
+  apt:
+    packages:
+      - launchpad-dependencies
+  ols:
+    service_name: launchpad
+    config_filename: service.conf
+    user: launchpad
+    tarball_payload: true
+    symlink_switch_payload: true
+    python_bin: /usr/bin/python3
+repo: https://git.launchpad.net/launchpad
diff --git a/charm/launchpad/metadata.yaml b/charm/launchpad/metadata.yaml
new file mode 100644
index 0000000..79f06eb
--- /dev/null
+++ b/charm/launchpad/metadata.yaml
@@ -0,0 +1,13 @@
+name: launchpad
+display-name: launchpad
+summary: Launchpad web application
+maintainer: Guruprasad <[e-mail]>
+description: |
+  Launchpad is an open source suite of tools that help people and teams
+  to work together on software projects.
+tags:
+  # https://juju.is/docs/charm-metadata#heading--charm-store-fields
+  - network
+series:
+  - bionic
+subordinate: false
diff --git a/charm/launchpad/reactive/launchpad.py b/charm/launchpad/reactive/launchpad.py
new file mode 100644
index 0000000..7cdf698
--- /dev/null
+++ b/charm/launchpad/reactive/launchpad.py
@@ -0,0 +1,47 @@
+# Copyright 2022 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+import subprocess
+
+from charmhelpers.core import hookenv
+from charms.reactive import (
+    remove_state,
+    set_state,
+    when,
+    when_not,
+    )
+from ols import base
+
+
+# Monkey-patch layer:ols.
+def create_virtualenv(wheels_dir, codedir, python_exe):
+    subprocess.run(
+        ['make', 'compile', 'PYTHON={}'.format(python_exe)],
+        cwd=codedir, check=True)
+
+
+base.create_virtualenv = create_virtualenv
+
+
+@when('ols.configured')
+@when_not('service.configured')
+def configure():
+    hookenv.log('Hello world!')
+    set_state('service.configured')
+
+
+@when('service.configured')
+def check_is_running():
+    hookenv.status_set('active', 'Ready')
+
+
+@when('config.changed.build_label')
+def build_label_changed():
+    remove_state('ols.service.installed')
+    remove_state('ols.configured')
+    remove_state('service.configured')
+
+
+@when('config.changed')
+def config_changed():
+    remove_state('service.configured')
diff --git a/charm/packages.txt b/charm/packages.txt
new file mode 100644
index 0000000..c33fce5
--- /dev/null
+++ b/charm/packages.txt
@@ -0,0 +1,3 @@
+flake8
+python-codetree
+squashfuse