Merge ~barryprice/wordpress-k8s-image-builder/+git/wordpress-k8s-image-builder:master into wordpress-k8s-image-builder:master

[Barry Price]

Create a Docker image to run the latest Wordpress, with our whitelisted plugins (upstream .zip and LP branches) and themes (LP branches only) already included.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[Tom Haddon (mthaddon)]

Some comments inline.

e0f49dc... by Barry Price on 2020-01-13

Always use the latest Wordpress, don't purge any packages for now

[Barry Price (barryprice)]

Retargetted to a new repository under the charm-k8s-wordpress project:

https://code.launchpad.net/~barryprice/charm-k8s-wordpress/+git/wordpress-k8s-image-builder/+merge/377498

review: Needs Resubmitting

Unmerged commits

e0f49dc... by Barry Price on 2020-01-13

Always use the latest Wordpress, don't purge any packages for now

122294d... by Barry Price on 2020-01-10

Use .latest-stable plugin URLs, drop BS/requests

967a385... by Barry Price on 2020-01-10

Add progress indicators to fetcher

93454b1... by Barry Price on 2020-01-10

Cleaner Makefile, strip historical wp-plugin- or wp-theme- prefixes

133eef4... by Barry Price on 2020-01-10

Use bzr/git directly instead of charmhelpers.fetch, clean up Makefile, add deps

fefc781... by Barry Price on 2020-01-10

Add tox/Makefile support (n.b. fetcher is currently failing to run 'bzr' via tox...

2aac1f5... by Barry Price on 2020-01-10

Initial commit

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..ac6bb88
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,3 @@
+plugins/
+themes/
+.tox/
diff --git a/Dockerfile b/Dockerfile
new file mode 100644
index 0000000..38866f4
--- /dev/null
+++ b/Dockerfile
@@ -0,0 +1,55 @@
+FROM ubuntu:bionic
+
+RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
+
+RUN apt-get update && apt-get -y dist-upgrade \
+    && apt-get --purge autoremove -y \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+
+ENV APACHE_CONFDIR=/etc/apache2
+ENV APACHE_ENVVARS=/etc/apache2/envvars
+
+RUN apt-get update \
+    && apt-get install -y --no-install-recommends apache2 \
+    && rm -rf /var/lib/apt/lists/* \
+    && sed -ri 's/^export ([^=]+)=(.*)$/: ${\1:=\2}\nexport \1/' "$APACHE_ENVVARS" \
+    && . "$APACHE_ENVVARS" \
+    && for dir in "$APACHE_LOCK_DIR" "$APACHE_RUN_DIR" "$APACHE_LOG_DIR"; do rm -rvf "$dir"; mkdir -p "$dir"; chown "$APACHE_RUN_USER:$APACHE_RUN_GROUP" "$dir"; chmod 777 "$dir";  done \
+    && ln -sfT /dev/stderr "$APACHE_LOG_DIR/error.log" \
+    && ln -sfT /dev/stdout "$APACHE_LOG_DIR/access.log" \
+    && ln -sfT /dev/stdout "$APACHE_LOG_DIR/other_vhosts_access.log" \
+    && chown -R --no-dereference "$APACHE_RUN_USER:$APACHE_RUN_GROUP" "$APACHE_LOG_DIR"
+
+RUN echo '<FilesMatch \.php$>' > "$APACHE_CONFDIR/conf-available/docker-php.conf" \
+    && echo '\tSetHandler application/x-httpd-php' >> "$APACHE_CONFDIR/conf-available/docker-php.conf" \
+    && echo '</FilesMatch>' >> "$APACHE_CONFDIR/conf-available/docker-php.conf" \
+    && echo >> "$APACHE_CONFDIR/conf-available/docker-php.conf" \
+    && echo 'DirectoryIndex disabled' >> "$APACHE_CONFDIR/conf-available/docker-php.conf" \
+    && echo 'DirectoryIndex index.php index.html' >> "$APACHE_CONFDIR/conf-available/docker-php.conf" \
+    && echo >> "$APACHE_CONFDIR/conf-available/docker-php.conf" \
+    && echo '<Directory /var/www/>' >> "$APACHE_CONFDIR/conf-available/docker-php.conf" \
+    && echo '\tOptions -Indexes' >> "$APACHE_CONFDIR/conf-available/docker-php.conf" \
+    && echo '\tAllowOverride All' >> "$APACHE_CONFDIR/conf-available/docker-php.conf" \
+    && echo '</Directory>' >> "$APACHE_CONFDIR/conf-available/docker-php.conf" \
+    && a2enconf docker-php
+
+RUN apt-get update && apt-get install -y curl php libapache2-mod-php php-mysql php-gd \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN a2dismod mpm_event \
+    && a2enmod mpm_prefork
+
+RUN curl -o wordpress.tar.gz -fSL "https://wordpress.org/latest.tar.gz" \
+    && tar -xzf wordpress.tar.gz -C /usr/src/ \
+    && rm wordpress.tar.gz \
+    && chown -R www-data:www-data /usr/src/wordpress \
+    && rm -rf /var/www/html \
+    && mv /usr/src/wordpress /var/www/html
+
+COPY --chown=www-data:www-data ./plugins/ /var/www/html/wp-content/plugins/
+COPY --chown=www-data:www-data ./themes/ /var/www/html/wp-content/themes/
+
+EXPOSE 80
+CMD apachectl -D FOREGROUND
diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..14a5b83
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,27 @@
+build: lint deps
+	@echo "Fetching plugins and themes."
+	@tox -e fetch
+	@echo "Building the image."
+	@docker build . -t wordpress
+
+deps:
+	@echo "Checking dependencies are present"
+	which bzr || sudo apt-get install -y bzr
+	which git || sudo apt-get install -y git
+
+lint: clean
+	@echo "Normalising python layout with black."
+	@tox -e black
+	@echo "Running flake8"
+	@tox -e lint
+
+clean:
+	@echo "Cleaning files"
+	@rm -rf ./.tox
+	@rm -rf ./.pytest_cache
+	@rm -rf ./plugins/*
+	@rm -rf ./themes/*
+	@mkdir -p plugins
+	@mkdir -p themes
+
+.PHONY: build lint clean
diff --git a/fetcher.py b/fetcher.py
new file mode 100755
index 0000000..2fe005c
--- /dev/null
+++ b/fetcher.py
@@ -0,0 +1,136 @@
+#!/usr/bin/env python3
+
+import os
+import shutil
+import subprocess
+import urllib.request
+import zipfile
+
+
+zip_plugins_to_get = {
+    # please keep these in alphabetical order
+    '404page',
+    'all-in-one-event-calendar',
+    'coschedule-by-todaymade',
+    'elementor',
+    'essential-addons-for-elementor-lite',
+    'favicon-by-realfavicongenerator',
+    'feedwordpress',
+    'fruitful-shortcodes',
+    'genesis-columns-advanced',
+    'line-break-shortcode',
+    'no-category-base-wpml',
+    'openid',
+    'post-grid',
+    'powerpress',
+    'redirection',
+    'relative-image-urls',
+    'rel-publisher',
+    'safe-svg',
+    'show-current-template',
+    'simple-301-redirects',
+    'simple-custom-css',
+    'social-media-buttons-toolbar',
+    'so-widgets-bundle',
+    'svg-support',
+    'syntaxhighlighter',
+    'wordpress-importer',
+    'wordpress-seo',
+    'wp-font-awesome',
+    'wp-lightbox-2',
+    'wp-markdown',
+    'wp-mastodon-share',
+    'wp-polls',
+    'wp-statistics',
+}
+
+branch_plugins_to_get = {
+    # please keep these in alphabetical order
+    'launchpad-integration': {'url': 'lp:wordpress-launchpad-integration'},
+    'openstack-objectstorage': {'url': 'lp:~canonical-sysadmins/wordpress/openstack-objectstorage'},
+    'teams-integration': {'url': 'lp:wordpress-teams-integration'},
+    'xubuntu-team-members': {'url': 'lp:~canonical-sysadmins/wordpress/wp-plugin-xubuntu-team-members'},
+}
+
+branch_themes_to_get = {
+    # please keep these in alphabetical order
+    'fruitful': {'url': 'https://git.launchpad.net/~canonical-sysadmins/wordpress/+git/wp-theme-fruitful'},
+    'light-wordpress-theme': {'url': 'lp:ubuntu-community-webthemes/light-wordpress-theme'},
+    'mscom': {'url': 'lp:~canonical-sysadmins/wordpress/wp-theme-mscom'},
+    'twentyeleven': {'url': 'lp:~canonical-sysadmins/wordpress/wp-theme-twentyeleven'},
+    'ubuntu-cloud-website': {'url': 'lp:ubuntu-cloud-website'},
+    'ubuntu-community': {'url': 'lp:~canonical-sysadmins/wordpress/wp-theme-ubuntu-community'},
+    'ubuntu-community-wordpress-theme': {'url': 'lp:ubuntu-community-wordpress-theme'},
+    'ubuntu-fi-new': {'url': 'lp:~canonical-sysadmins/wordpress/wp-theme-ubuntu-fi'},
+    'ubuntu-light': {'url': 'lp:~canonical-sysadmins/wordpress/wp-theme-ubuntu-light'},
+    'ubuntustudio-wp': {'url': 'lp:~canonical-sysadmins/wordpress/wp-theme-ubuntustudio-wp'},
+    'wordpress_launchpad': {'url': 'lp:~canonical-sysadmins/wordpress/wp-theme-launchpad'},
+    'xubuntu-theme': {'url': 'lp:~canonical-sysadmins/wordpress/wp-theme-xubuntu-website'},
+}
+
+
+def get_plugins(zip_plugins, branch_plugins):
+    total_zips = len(zip_plugins)
+    current_zip = 0
+    for zip_plugin in zip_plugins:
+        current_zip = current_zip + 1
+        print('Downloading zipped plugin {} of {}...'.format(current_zip, total_zips), end="\r")
+        url = 'https://downloads.wordpress.org/plugin/{}.latest-stable.zip'.format(zip_plugin)
+        file_name = os.path.join(os.getcwd(), 'plugins', os.path.basename(url))
+        with urllib.request.urlopen(url) as response, open(file_name, 'wb') as out_file:
+            shutil.copyfileobj(response, out_file)
+        with zipfile.ZipFile(file_name, 'r') as zip_ref:
+            zip_ref.extractall(os.path.join(os.getcwd(), 'plugins'))
+        os.remove(file_name)
+    print()
+
+    total_branches = len(branch_plugins)
+    current_branch = 0
+    for branch_plugin in branch_plugins:
+        current_branch = current_branch + 1
+        print('Downloading branched plugin {} of {}...'.format(current_branch, total_branches), end="\r")
+        url = branch_plugins[branch_plugin].get('url')
+        basename = os.path.basename(url)
+        if basename.startswith('lp:'):
+            basename = basename[3:]
+        if basename.startswith('wp-plugin-'):
+            basename = basename[10:]
+        dest = os.path.join(os.getcwd(), 'plugins/', basename)
+        if url.startswith('lp:'):
+            cmd = ['bzr', 'branch', url, dest]
+        elif url.startswith('https://git'):
+            cmd = ['git', 'clone', url, dest]
+        else:
+            print("ERROR: Don't know how to clone {}".format(url))
+            exit(1)
+        _ = subprocess.check_output(cmd, universal_newlines=True, stderr=subprocess.PIPE)
+    print()
+
+
+def get_themes(branch_themes):
+    total_branches = len(branch_themes)
+    current_branch = 0
+    for branch_theme in branch_themes:
+        current_branch = current_branch + 1
+        print('Downloading branched theme {} of {}...'.format(current_branch, total_branches), end="\r")
+        url = branch_themes[branch_theme].get('url')
+        basename = os.path.basename(url)
+        if basename.startswith('lp:'):
+            basename = basename[3:]
+        if basename.startswith('wp-theme-'):
+            basename = basename[9:]
+        dest = os.path.join(os.getcwd(), 'themes/', basename)
+        if url.startswith('lp:'):
+            cmd = ['bzr', 'branch', url, dest]
+        elif url.startswith('https://git'):
+            cmd = ['git', 'clone', url, dest]
+        else:
+            print("ERROR: Don't know how to clone {}".format(url))
+            exit(1)
+        _ = subprocess.check_output(cmd, universal_newlines=True, stderr=subprocess.PIPE)
+    print()
+
+
+if __name__ == '__main__':
+    get_plugins(zip_plugins_to_get, branch_plugins_to_get)
+    get_themes(branch_themes_to_get)
diff --git a/tox.ini b/tox.ini
new file mode 100644
index 0000000..92befc4
--- /dev/null
+++ b/tox.ini
@@ -0,0 +1,32 @@
+[tox]
+skipsdist=True
+envlist = build
+skip_missing_interpreters = True
+
+[testenv]
+basepython = python3
+setenv =
+  PYTHONPATH = .
+
+[testenv:black]
+commands = {envbindir}/black --skip-string-normalization --line-length=120 .
+deps = black
+
+[testenv:lint]
+commands = {envbindir}/flake8
+deps = flake8
+
+[testenv:fetch]
+commands = ./fetcher.py
+setenv =
+  BZR_HOME = /tmp
+
+[flake8]
+exclude =
+    .git,
+    __pycache__,
+    .tox,
+    plugins/
+    themes/
+max-line-length = 120
+max-complexity = 10