Merge ~ballot/jenkins-plugin-manager/+git/jenkins-plugin-manager:master into jenkins-plugin-manager:master

[Benjamin Allot]

* Initial commit
* Change name into jenkins_plugin_manager and add debian package

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[Benjamin Allot (ballot)]

The JSON files can be ignored because they are a snapshot of https://updates.jenkins.io/stable/update-center.json and https://updates.jenkins.io/plugin-versions.json

TODO in others MP:
* Download jenkins package from cli (only the library is implmented for now)
* Checksum of the downloaded package
* add various option in the "download" subcommand to allow optional dependencies and without if needed
* symlink the latest downloaded version if desired
* document the format of the plugin --from-file format

[Tom Haddon (mthaddon)]

Some comments inline

[Stuart Bishop (stub)]

All looks great and in a good style. Some inline comments in addition to Toms. Giving this a preemptive approve, although maybe the branch is being handed over instead of landed per discussion with Tom.

review: Approve

[Benjamin Allot (ballot)]

Inline comments of both addressed.

Basically took all of them into account except the try/import mock , because this would be needed for a python 3.2 support (the one on Precise). I added a comment related to that.

Also added a few tests on the JenkinsCore class.

[Tom Haddon (mthaddon)]

No need for precise/python3.2 support, and looks like the url property of setup.py needs updating.

[Tom Haddon (mthaddon)]

Looks good, thanks.

review: Approve

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Change successfully merged at revision 421dc1082891d2215f1198de9eef377b82fe458d

diff --git a/.flake8 b/.flake8
new file mode 100644
index 0000000..9a6d4d5
--- /dev/null
+++ b/.flake8
@@ -0,0 +1,4 @@
+[flake8]
+exclude = .venv
+max-complexity = 10
+max-line-length = 120
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..e238002
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,9 @@
+*.pyc
+*.swp
+*~
+.venv
+/.coverage
+/.pybuild
+/.tox
+__pycache__
+debian/files
diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..67498c0
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,67 @@
+PROJECTPATH = $(dir $(realpath $(firstword $(MAKEFILE_LIST))))
+
+VENV := .venv
+VENV_PIP := $(PROJECTPATH)/$(VENV)/bin/pip
+VENV_PYTHON := $(PROJECTPATH)/$(VENV)/bin/python3
+
+BLACK := $(VENV_PYTHON) -m black
+FLAKE8 := $(VENV_PYTHON) -m flake8
+
+PROJECT := jenkins-plugin-manager
+PKG_NAME := python3-$(PROJECT)
+
+all: lint test
+
+clean: clean-python clean-venv clean-tox
+
+clean-python:
+	@find $(PROJECTPATH) -name "__pycache__" -type d -exec rm -rf {} +
+	@find $(PROJECTPATH) -name "pytest_cache" -type d -exec rm -rf {} +
+
+clean-tox:
+	rm -rf $(PROJECTPATH)/.tox
+
+clean-venv:
+	rm -rf $(PROJECTPATH)/$(VENV)
+
+dch:
+	gbp dch --debian-tag='%(version)s' -D bionic --git-log --first-parent
+
+deb-src:
+	debuild -S -sa -I.git -I.tox -I$(VENV)
+
+deb:
+	debuild -us -uc -sa -I.git -I.tox -I$(VENV)
+
+/var/cache/pbuilder/base-bionic-amd64.cow:
+	ARCH=amd64 DIST=bionic git-pbuilder create
+
+pbuilder-deb-src: /var/cache/pbuilder/base-bionic-amd64.cow
+	GIT_PBUILDER_OPTIONS='--source-only-changes' \
+		ARCH=amd64 DIST=bionic git-pbuilder -sa -I.git -I.tox -I$(VENV)
+	@echo "Now use debsign on $(PROJECTPATH)../$(PKG_NAME)_x.y_source.changes"
+
+install-build-depends:
+	sudo apt install \
+	    debhelper \
+	    git-buildpackage \
+	    virtualenv
+
+lint: lint-python
+
+# See .flake8 for config options.
+lint-python:
+	tox -e black
+	tox -e lint
+
+test: test-python
+
+test-python:
+	$(VENV_PYTHON) -m unittest discover -v tests
+
+$(VENV):
+	virtualenv -p python3 $(PROJECTPATH)/$(VENV)
+	$(VENV_PIP) install -I -r requirements.txt
+
+.PHONY: test lint lint-python test-python all lint lint-python clean clean-python clean-tox clean-venv deb-src dch
+
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..fd3882f
--- /dev/null
+++ b/README.md
@@ -0,0 +1,48 @@
+# Jenkins Plugin Manager
+
+This repository contains the jenkins\_plugin\_manager python module.
+this module allows a management of the Jenkins plugins and their dependencies as well as getting them
+
+The Launchpad project can be found
+[here](https://launchpad.net/jenkins-plugin-manager).
+
+The code for this repository can be cloned with:
+
+  git clone git+ssh://git.launchpad.net/jenkins-plugin-manager
+
+## Code Location
+
+The Python code can be found at `./jenkins_plugin_manager.py`, with tests in the
+`tests` directory. Lint configuration can be found in `./.flake8`.
+
+## Testing Code
+
+To get started, you need `make` and `tox`.
+
+  make install-build-depends
+  make test
+
+## Building a Release
+
+The release process for a project would normally be something like:
+
+  [edit setup.py with the right version number (if applicable)]
+  make dch
+  [check and edit debian/changelog]
+  git commit -m 'x.y.z release'  debian/changelog
+  export GPG\_TTY=$(tty)
+  git tag -s -u <keyid> -m 'x.y.z release' x.y.z
+  git push --tags
+  make deb-src
+  dput ppa:jenkins-plugin-manager-devs/jpm-devel ../python3-jenkins-plugin-manager\_x.y.z\_source.changes
+
+Where `x.y.z` is the appropriate version number.
+This will upload the debian source package to the
+[devel PPA](https://launchpad.net/~jenkins-plugin-manager-devs/+archive/ubuntu/jpm-devel)
+for building.
+
+Once the new packages have been successfully tested, they can be
+[copied](https://launchpad.net/~jenkins-plugin-manager-devs/+archive/ubuntu/jpm-devel/+copy-packages)
+as binary packages (not rebuilt from source) to the
+[production  PPA](https://launchpad.net/~jenkins-plugin-manager-devs/+archive/ubuntu/jpm)
+The production environment can then be updated from this PPA.
diff --git a/debian/changelog b/debian/changelog
new file mode 100644
index 0000000..5a48582
--- /dev/null
+++ b/debian/changelog
@@ -0,0 +1,19 @@
+python3-jenkins-plugin-manager (0.3) bionic; urgency=medium
+
+  * Address inline comments
+  * 0.3 release
+
+ -- Benjamin Allot (ballot@Canonical) <[e-mail]>  Mon, 11 Nov 2019 23:12:30 +0100
+
+python3-jenkins-plugin-manager (0.2ubuntu1) bionic; urgency=medium
+
+  * 0.2ubuntu1 release
+
+ -- Benjamin Allot (ballot@Canonical) <[e-mail]>  Fri, 08 Nov 2019 10:02:45 +0100
+
+python3-jenkins-plugin-manager (0.1ubuntu1) bionic; urgency=medium
+
+  * Initial commit
+  * Change name into jenkins_plugin_manager and add debian package
+
+ -- Benjamin Allot (ballot@Canonical) <[e-mail]>  Fri, 08 Nov 2019 03:49:08 +0100
diff --git a/debian/compat b/debian/compat
new file mode 100644
index 0000000..f599e28
--- /dev/null
+++ b/debian/compat
@@ -0,0 +1 @@
+10
diff --git a/debian/control b/debian/control
new file mode 100644
index 0000000..0b7ec32
--- /dev/null
+++ b/debian/control
@@ -0,0 +1,14 @@
+Source: python3-jenkins-plugin-manager
+Section: misc
+Priority: optional
+Maintainer: Benjamin Allot <[e-mail]>
+Build-Depends: debhelper (>= 11), dh-python, python3-all, python3-setuptools, virtualenv
+Standards-Version: 3.9.7
+X-Python3-Version: >= 3.2
+
+Package: python3-jenkins-plugin-manager
+Architecture: any
+Depends: ${misc:Depends}, ${python3:Depends}
+Description: Jenkins Plugin Manager
+ A python module used for managing Jenkins plugins dependencies
+ and downloading them.
diff --git a/debian/rules b/debian/rules
new file mode 100755
index 0000000..b867466
--- /dev/null
+++ b/debian/rules
@@ -0,0 +1,22 @@
+#!/usr/bin/make -f
+
+# Determine the current release
+# # Use specific control and changelog files for Precise
+#
+codename ?= $(shell lsb_release -cs)
+module := jenkins_plugin_manager
+pkg_name := python3-$(module)
+
+export DH_VERBOSE=1
+export PYBUILD_NAME=$(module)
+
+%:
+	dh $@ --with python3 --buildsystem=pybuild
+
+override_dh_auto_test:
+
+override_dh_auto_clean:
+	dh_auto_clean
+	rm -rf build
+	rm -rf *.egg-info
+
diff --git a/debian/source/format b/debian/source/format
new file mode 100644
index 0000000..d3827e7
--- /dev/null
+++ b/debian/source/format
@@ -0,0 +1 @@
+1.0
diff --git a/jenkins_plugin_manager/__init__.py b/jenkins_plugin_manager/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/jenkins_plugin_manager/__init__.py
diff --git a/jenkins_plugin_manager/core.py b/jenkins_plugin_manager/core.py
new file mode 100644
index 0000000..8b25ad0
--- /dev/null
+++ b/jenkins_plugin_manager/core.py
@@ -0,0 +1,94 @@
+"""
+Jenkins Core module.
+
+This module loads a file like "https://updates.jenkins.io/stable-2.190/latestCore.txt"
+to determine the latest "stable" or "current" Jenkins version.
+It then provides methods through a class to retrieve the latest jenkins package
+on "https://pkg.jenkins.io/".
+"""
+
+import logging
+import os
+import urllib.request
+
+logger = logging.getLogger(__name__)
+
+
+DEFAULT_JENKINS_CORE_URL = "https://updates.jenkins.io"
+DEFAULT_JENKINS_REPO = "https://pkg.jenkins.io"
+DEFAULT_JENKINS_RELEASE = "stable"
+
+
+class JenkinsCore(object):
+    """Represents Jenkins core manager."""
+
+    def __init__(
+        self,
+        jenkins_core_url=DEFAULT_JENKINS_CORE_URL,
+        jenkins_repo=DEFAULT_JENKINS_REPO,
+        jenkins_release=DEFAULT_JENKINS_RELEASE,
+    ):
+        """Instantiate the JenkinsCore class.
+
+        Load the latest version of the `jenkins_release` available.
+
+        :param str jenkins_core_url: Url where the latest Jenkins core version is set.
+        :param str jenkins_repo: Url where the Jenkins binary packages are stored.
+        :param str jenkins_release: Flavor of Jenkins. Values are "stable" or "current".
+        """
+        # TODO: Use python-apt to use the real repository data (and manage correct checksum)
+        # Right now this manage only the official jenkins repository layout, which is non-standard
+        self.jenkins_repo = jenkins_repo
+        if jenkins_release not in ("current", "stable"):
+            err = "Wrong jenkins_release parameter. Valid values are [current|stable]"
+            logger.error(err)
+            raise ValueError(err)
+        self.jenkins_release = jenkins_release
+        self.latest_core_url = "{0}/{1}/latestCore.txt".format(jenkins_core_url, jenkins_release)
+        self.core_version = self.load_latest_core_version(self.latest_core_url)
+
+    def load_latest_core_version(self, core_url):
+        """Load the latest version available.
+
+        :param str core_url: Url of the latest Jenkins core
+        """
+        logger.info("Loading %s", core_url)
+        with urllib.request.urlopen(core_url) as core_data:
+            core_version = core_data.read().decode("utf-8")
+            logger.debug("Latest jenkins core version: %s", core_version)
+        return core_version
+
+    def get_binary_package(self, dst_dir, distribution="debian", version=None, arch="amd64"):
+        """Get a Jenkins binary package.
+
+        Download a binary package of the said distribution and of the latest Jenkins
+        release if not specified.
+
+        :param str dst_dir: Target directory for the download
+        :param str distribution: Distribution family to retrieve package for (debian, redhat)
+        :param str version: The version you want to download. Default to the latest Core if not specified
+        :param str arch: Architecture to download. Default to amd64
+        """
+
+        assert distribution in ("debian", "redhat")
+        if self.jenkins_release == "stable":
+            repo_dist = "{0}-stable".format(distribution)
+        else:
+            repo_dist = distribution
+
+        if not version:
+            target_version = self.core_version
+        else:
+            target_version = version
+        binary_package_url = "{url}/{dist}/binary/jenkins_{version}_all.deb".format(
+            url=self.jenkins_repo, dist=repo_dist, version=target_version
+        )
+        package_file = os.path.basename(binary_package_url)
+        package_file_path = os.path.join(dst_dir, package_file)
+        with urllib.request.urlopen(binary_package_url) as binary_pkg:
+            if not os.path.exists(os.path.dirname(package_file_path)):
+                os.makedirs(os.path.dirname(package_file_path))
+            logger.info("Downloading %s to %s", binary_package_url, package_file_path)
+            with open(package_file_path, "wb") as target_pkg:
+                target_pkg.write(binary_pkg.read())
+            logger.info("%s downloaded", package_file)
diff --git a/jenkins_plugin_manager/exceptions.py b/jenkins_plugin_manager/exceptions.py
new file mode 100644
index 0000000..e7871bb
--- /dev/null
+++ b/jenkins_plugin_manager/exceptions.py
@@ -0,0 +1,19 @@
+"""Jenkins Plugin Manager exceptions."""
+
+
+class InvalidPluginVersionError(Exception):
+    """Exception when a desired plugin is higher than the most recent one in the Update Center."""
+
+    pass
+
+
+class InvalidPluginError(Exception):
+    """Exception when the desired plugin is invalid or not available."""
+
+    pass
+
+
+class PluginMinCoreVersionError(Exception):
+    """Exception when a plugin need a higher jenkins core version than the one set as the current one."""
+
+    pass
diff --git a/jenkins_plugin_manager/main.py b/jenkins_plugin_manager/main.py
new file mode 100755
index 0000000..155dde0
--- /dev/null
+++ b/jenkins_plugin_manager/main.py
@@ -0,0 +1,122 @@
+#!/usr/bin/env python3
+"""
+Entrypoint for the jenkins-plugin-manager module
+"""
+
+import argparse
+import logging
+import sys
+
+from jenkins_plugin_manager.plugin import UpdateCenter
+
+
+logger = logging.getLogger("jenkins_plugin_manager")
+
+
+def parse_args():
+    """Parse the arguments provided to the CLI."""
+    parser = argparse.ArgumentParser(prog="jenkins-plugin-manager")
+    parser.add_argument("--log-level", action="store", help="Logging level", default="WARNING")
+    parser.add_argument("--log-file", action="store", help="Log file")
+    parser.add_argument("--log-file-level", action="store", help="File logging level", default="INFO")
+    subparsers = parser.add_subparsers(help="sub-command help")
+
+    # Sub command "plugin"
+    parser_plugin = subparsers.add_parser("plugin", help="Manage Jenkins plugins")
+    parser_plugin.add_argument("--from-file", action="store", help="Load a list of plugin from a file")
+    subparser_plugin = parser_plugin.add_subparsers(help="plugin sub-command help")
+
+    # Sub command "plugin list-urls"
+    parser_plugin_list = subparser_plugin.add_parser("list-urls", help="List Jenkins plugins url")
+    parser_plugin_list.add_argument("args", nargs=argparse.REMAINDER, help="List of plugin")
+    parser_plugin_list.set_defaults(func=plugin_list_urls)
+
+    # Sub command "plugin download"
+    parser_plugin_download = subparser_plugin.add_parser("download", help="Download Jenkins plugins")
+    parser_plugin_download.add_argument(
+        "--dst", action="store", required=True, help="Directory for the downloaded plugins"
+    )
+    parser_plugin_download.add_argument("args", nargs=argparse.REMAINDER, help="List of plugin")
+    parser_plugin_download.set_defaults(func=plugin_download)
+
+    return parser.parse_args()
+
+
+def configure_logging(args):
+    """Configure the logging.
+
+    :param :py:class:argparse.ArgumentParser args: Arguments provided
+    """
+    formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")
+    logger.setLevel(logging.DEBUG)
+    log_level = getattr(logging, args.log_level.upper(), None)
+    if not isinstance(log_level, int):
+        raise ValueError("Invalid log-level level: %s" % args.log_level)
+    console = logging.StreamHandler()
+    console.setLevel(log_level)
+    console.setFormatter(formatter)
+    logger.addHandler(console)
+    if args.log_file:
+        log_file_level = getattr(logging, args.log_file_level.upper(), None)
+        if not isinstance(log_file_level, int):
+            raise ValueError("Invalid --log-file-level level: %s" % args.log_file_level)
+        logfile_handler = logging.FileHandler(args.log_file)
+        logfile_handler.setLevel(log_file_level)
+        logfile_handler.setFormatter(formatter)
+        logger.addHandler(logfile_handler)
+
+
+def plugin_load(args):
+    """Return the list of plugin provided by the options.
+
+    :param :py:class:argparse.ArgumentParser args: Arguments provided
+    """
+    plugins = []
+    if args.from_file:
+        try:
+            with open(args.from_file, "r") as plugin_file:
+                plugins.extend(plugin_file.read().splitlines())
+        except FileNotFoundError:
+            logger.error("File %s: Not found", args.from_file)
+            sys.exit(1)
+    if args.args:
+        plugins.extend(args.args)
+    if not plugins:
+        logger.error("No plugin provided.")
+        sys.exit(1)
+    return plugins
+
+
+def plugin_download(args):
+    """Download the given plugins.
+
+    :param :py:class:argparse.ArgumentParser args: Arguments provided
+    """
+    plugins = plugin_load(args)
+    uc = UpdateCenter()
+    for plugin in uc.get_plugins(set(plugins)):
+        print(uc.download_plugin(plugin, args.dst))
+
+
+def plugin_list_urls(args):
+    """Sub-command "plugin list-urls".
+
+    Print the url of each plugin that need is specified.
+
+    :param :py:class:argparse.ArgumentParser args: Arguments provided
+    """
+    plugins = plugin_load(args)
+    uc = UpdateCenter()
+    for plugin in uc.get_plugins(set(plugins)):
+        print(uc.get_plugin_data(plugin)["url"])
+
+
+def main():
+    """Main entrypoint for jenkins-plugin-manager CLI tool."""
+    args = parse_args()
+    configure_logging(args)
+    args.func(args)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/jenkins_plugin_manager/plugin.py b/jenkins_plugin_manager/plugin.py
new file mode 100644
index 0000000..84b6c19
--- /dev/null
+++ b/jenkins_plugin_manager/plugin.py
@@ -0,0 +1,216 @@
+#!/usr/bin/env python3
+
+"""
+Jenkins Plugin Manager plugin module.
+
+This module loads the JSON file available in jenkins Update Center
+(default "https://updates.jenkins.io/stable/update-center.json") and load it for further computation.
+"""
+
+import base64
+import codecs
+import hashlib
+import json
+import logging
+import os
+import urllib.request
+
+from pkg_resources import parse_version
+
+from jenkins_plugin_manager.exceptions import (
+    InvalidPluginVersionError,
+    InvalidPluginError,
+    PluginMinCoreVersionError,
+)
+
+logger = logging.getLogger(__name__)
+
+
+class UpdateCenter(object):
+    """Represents the UpdateCenter data."""
+
+    def __init__(
+        self, uc_url="https://updates.jenkins.io/stable/update-center.json",
+    ):
+        """Instantiate the UpdateCenter class.
+
+        ..code:: python
+
+          uc = jenkins_plugin_manager.UpdateCenter()
+          for plugin in uc.get_plugins(["ansicolor", "docker-workflow"]):
+            uc.download_plugin(plugin, "/var/lib/jenkins/plugins", with_version=False
+
+        :param str uc_url: The url of the UpdateCenter JSON file. Default to the official stable one.
+        """
+        self.uc_data = self.load_update_center_data(uc_url)
+
+    def load_update_center_data(self, uc_url):
+        """Load the content of the Update Center JSON file.
+
+        :param str uc_url: Update Center JSON file url.
+        """
+        logger.info("Loading Jenkins Update Center data from %s", uc_url)
+        with urllib.request.urlopen(uc_url) as uc_data:
+            # Remove the "updateCenter.post(\n ... \n);" around the json payload
+            json_payload = uc_data.read().decode("utf-8").lstrip("updateCenter.post(\n").rstrip("\n);")
+            return json.loads(json_payload)
+
+    def get_plugin_data(self, plugin):
+        """Return the specific plugin data.
+
+        The plugin can be specificed with "latest" of it's version in the form "<plugin>:(<version>|latest)".
+        However, only the latest plugin is pulled. A check is performed to ensure the desired version is inferior or
+        equal to the latest available.
+
+        :param str plugin: The name of the plugin, possibly with the version or "latest". e.g  <plugin>:<version>.
+        :returns: The dictionary of the plugin's data as seen in the Update Center JSON file.
+        :rtype: dict
+        """
+        if ":" in plugin:
+            plugin_name, plugin_version = plugin.split(":")
+        else:
+            plugin_name = plugin
+            plugin_version = "latest"
+
+        try:
+            if plugin_version != "latest":
+                # Check that the desired version is <= to the one available in the latest verison of the plugin
+                latest_plugin_version = self.uc_data["plugins"][plugin_name]["version"]
+                if parse_version(plugin_version) > parse_version(latest_plugin_version):
+                    err = "Plugin {0} version {1} is higher than the latest available version {2}".format(
+                        plugin_name, plugin_version, latest_plugin_version
+                    )
+                    logger.error(err)
+                    raise InvalidPluginVersionError(err)
+                elif parse_version(plugin_version) < parse_version(latest_plugin_version):
+                    logger.info("Getting %s:%s instead of %s", plugin_name, latest_plugin_version, plugin)
+
+            plugin_data = self.uc_data["plugins"][plugin_name]
+            logger.debug("Plugin %s data: %s", plugin_name, plugin_data)
+        except KeyError:
+            err = "Cannot find plugin {0}".format(plugin_name)
+            logger.error(err)
+            raise InvalidPluginError(err) from None
+        return plugin_data
+
+    def _verify_sha256sum(self, plugin_file_path, sha256sum):
+        """Verify the sha256sum of the given file.
+
+        :param str plugin_file_path: Path of the downloaded file.
+        :param str sha256sum: A base64 encoding of the sha256sum of the plugin.
+        :returns: True if the file has the right content, False otherwise.
+        :rtype: bool
+        """
+        # The sha256 given is a binary base64 that need to be encoded to hex then converted into a string
+        # See https://github.com/jenkins-infra/update-center2/blob/master/src/main/java/org/jvnet/hudson/update_center/IndexHtmlBuilder.java#L82 # noqa
+        sha256sum_hex = codecs.decode(codecs.encode(base64.b64decode(sha256sum), "hex"), "utf-8")
+        try:
+            with open(plugin_file_path, "rb") as plugin_file:
+                content = plugin_file.read()
+            sha256sum_check = hashlib.sha256(content).hexdigest()
+            if sha256sum_check != sha256sum_hex:
+                logger.error("SHA256sum check failed, found %s (expected %s)", sha256sum_check, sha256sum_hex)
+                return False
+        except FileNotFoundError:
+            logger.error("Cannot open the file %s", plugin_file_path)
+            return False
+        return True
+
+    def _check_min_core_version(self, core_version, plugin_min_core_version):
+        """Check the plugin required Jenkins core version.
+
+        :param str core_version: Version of Jenkins core.
+        :param str plugin_min_core_version: The minimum version of Jenkins core required for this plugin.
+        :returns: True if the Jenkins core version is sufficient, False otherwise.
+        :rtype: bool
+        """
+        if parse_version(core_version) < parse_version(plugin_min_core_version):
+            logger.error(
+                "Jenkins core version %s or above is required (%s found)", plugin_min_core_version, core_version
+            )
+            return False
+        return True
+
+    def download_plugin(self, plugin, dst_dir, plugin_url=None, with_version=True):
+        """Download the given plugin in the "dst_dir" directory.
+
+        A checksum is done on the files downloaded.
+
+        :param str plugin: The name of the plugin, possibly with the version or "latest". e.g  <plugin>:<version>.
+        :param str dst_dir: A path where the files are downloaded.
+        :param str plugin_url: If set, use this url to retrieve the plugin instead of the one in the UpdateCenter
+            JSON file. This is useful when you use a in-house HTTP server hosting the plugins without having
+            a Jenkins updateCenter.
+        :param bool with_version: Append the version of the plugin to the name in the target file if True.
+            Only the name of the plugin if False.
+        :returns: The path of the downloaded file, False if it fails.
+        :rtype: str
+        """
+        plugin_data = self.get_plugin_data(plugin)
+        plugin_name = plugin_data["name"]
+
+        if not plugin_url:
+            plugin_url = plugin_data["url"]
+
+        logger.info("Downloading plugin %s from %s", plugin_name, plugin_url)
+        with urllib.request.urlopen(plugin_url) as final_plugin_url:
+            if with_version:
+                plugin_file_path = os.path.join(dst_dir, "{0}-{1}.jpi".format(plugin_name, plugin_data["version"]))
+            else:
+                plugin_file_path = os.path.join(dst_dir, "{0}.jpi".format(plugin_name))
+            if not os.path.exists(os.path.dirname(plugin_file_path)):
+                os.makedirs(os.path.dirname(plugin_file_path))
+            with open(plugin_file_path, "wb") as plugin_file:
+                plugin_file.write(final_plugin_url.read())
+        if not self._verify_sha256sum(plugin_file_path, plugin_data["sha256"]):
+            return False
+        logger.info("Plugin %s downloaded to %s", plugin_name, plugin_file_path)
+        return plugin_file_path
+
+    def get_plugins_dependencies(self, plugin, optional=False):
+        """Get the list of the plugin's dependencies.
+
+        :param str plugin: The name of the plugin, possibly with the version or "latest". e.g  <plugin>:<version>.
+        :param str dst: A path where the files are downloaded.
+        :param bool optional: Get the optional dependencies.
+        :returns: A set of the plugin's dependency.
+        :rtype: set
+        """
+        plugin_data = self.get_plugin_data(plugin)
+        dependencies_to_fetch = set()
+        for dependency in plugin_data["dependencies"]:
+            # Skip optional dependency  or if already in the list
+            if dependency["optional"] and not optional:
+                continue
+            # Since the constraints are always >=, we don't care to pick up the latest dependency
+            dependencies_to_fetch.add(dependency["name"])
+        return dependencies_to_fetch
+
+    def get_plugins(self, plugins, current_core_version=None, with_dependency=True, optional=False):
+        """Get a list of plugins with their dependencies.
+
+        You can also add optional dependencies if needed.
+
+        :param list plugins: List of plugins to download. Each plugin is a string containing the name of the plugin and
+            possibly a colon (":") with the version of the plugin or "latest". If omitted, "latest" is assumed.
+        :param str current_core_version: Jenkins core version running.
+        :param bool optional: Add the optional dependencies if True.
+        :returns: A set of plugins to download.
+        :rtype: set
+        """
+        plugins_to_fetch = set()
+        dependencies_to_fetch = set()
+        for plugin in plugins:
+            plugin_data = self.get_plugin_data(plugin)
+            plugin_name = plugin_data["name"]
+            # Check if the Jenkins Core version is enough
+            if current_core_version:
+                logger.info("Checking minimum Jenkins core version required for %s", plugin)
+                if not self._check_min_core_version(current_core_version, plugin_data["requiredCore"]):
+                    raise PluginMinCoreVersionError(
+                        "Plugin {0} requires Jenkins core version {1}".format(plugin_name, plugin_data["requiredCore"])
+                    )
+            plugins_to_fetch.add(plugin_data["name"])
+            if with_dependency:
+                dependencies_to_fetch = dependencies_to_fetch.union(self.get_plugins_dependencies(plugin, optional))
+        return plugins_to_fetch.union(dependencies_to_fetch)
diff --git a/requirements.txt b/requirements.txt
new file mode 100644
index 0000000..4e92b9d
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1,2 @@
+black
+flake8
diff --git a/setup.py b/setup.py
new file mode 100644
index 0000000..3659cd2
--- /dev/null
+++ b/setup.py
@@ -0,0 +1,44 @@
+"""Jenkins Plugin Manager module."""
+
+# Always prefer setuptools over distutils
+from setuptools import setup, find_packages
+from os import path
+
+here = path.abspath(path.dirname(__file__))
+
+# Get the long description from the README file
+with open(path.join(here, "README.md"), encoding="utf-8") as f:
+    long_description = f.read()
+
+# Arguments marked as "Required" below must be included for upload to PyPI.
+# Fields marked as "Optional" may be commented out.
+
+setup(
+    name="jenkins-plugin-manager",  # Required
+    version="0.3",  # Required
+    description="Manage Jenkins plugin (get dependencies, download them)",  # Optional
+    long_description=long_description,  # Optional
+    long_description_content_type="text/markdown",  # Optional (see note above)
+    url="https://launchpad.net/jenkins-plugin-manager",  # Optional
+    author="Benjamin Allot",  # Optional
+    author_email="[e-mail]",  # Optional
+    classifiers=[  # Optional
+        "Development Status :: 3 - Alpha",
+        "Intended Audience :: Developers",
+        "Topic :: Software Development :: Build Tools",
+        "License :: OSI Approved :: MIT License",
+        "Programming Language :: Python :: 3",
+    ],
+    keywords="jenkins",  # Optional
+    packages=find_packages(exclude=["tests"]),  # Required
+    install_requires="setuptools",
+    python_requires=">=3.3",
+    extras_require={"test": ["coverage"]},  # Optional
+    # For example, the following would provide a command called `sample` which
+    # executes the function `main` from this package when invoked:
+    entry_points={"console_scripts": ["jenkins-plugin-manager=jenkins_plugin_manager.main:main"]},  # Optional
+    project_urls={  # Optional
+        "Bug Reports": "https://bugs.launchpad.net/jenkins-plugin-manager",
+        "Source": "https://code.launchpad.net/jenkins-plugin-manager",
+    },
+)
diff --git a/tests/test_jenkins_plugin_manager_core.py b/tests/test_jenkins_plugin_manager_core.py
new file mode 100644
index 0000000..79d1119
--- /dev/null
+++ b/tests/test_jenkins_plugin_manager_core.py
@@ -0,0 +1,52 @@
+import unittest
+
+from unittest import mock
+
+from jenkins_plugin_manager.core import (
+    JenkinsCore,
+    DEFAULT_JENKINS_RELEASE,
+    DEFAULT_JENKINS_CORE_URL,
+)
+
+
+class TestJenkinsCore(unittest.TestCase):
+    """Tests for the JenkinsCore class."""
+
+    LATEST_CORE_VERSION = "2.1.190"
+
+    def setUp(self):
+        """Setup fxture for each test."""
+        core_version_patch = mock.patch("jenkins_plugin_manager.core.JenkinsCore.load_latest_core_version")
+        self.core_version_mock = core_version_patch.start()
+        self.addCleanup(core_version_patch.stop)
+        self.addCleanup(self.core_version_mock.reset_mock)
+        self.core_version_mock.return_value = self.LATEST_CORE_VERSION
+        self.jenkins_core = JenkinsCore()
+
+    def test_jenkinscore_attr(self):
+        """Check the varios attributes of the class."""
+        # Check default attribute
+        self.assertEqual(self.jenkins_core.jenkins_release, DEFAULT_JENKINS_RELEASE)
+        self.assertEqual(
+            self.jenkins_core.latest_core_url,
+            "{0}/{1}/latestCore.txt".format(DEFAULT_JENKINS_CORE_URL, DEFAULT_JENKINS_RELEASE),
+        )
+        self.assertEqual(self.jenkins_core.core_version, self.LATEST_CORE_VERSION)
+
+        # Check custom repository
+        jenkins_repo = "http://pkg.jenkins.canonical.com"
+        jenkins_release = "current"
+        jenkins_core_url = "http://updates.jenkins.canonical.com"
+        custom_jc = JenkinsCore(
+            jenkins_release=jenkins_release, jenkins_repo=jenkins_repo, jenkins_core_url=jenkins_core_url
+        )
+        self.assertEqual(custom_jc.jenkins_release, jenkins_release)
+        self.assertEqual(custom_jc.latest_core_url, "{0}/{1}/latestCore.txt".format(jenkins_core_url, jenkins_release))
+
+        # Check exception
+        jenkins_release = "custom"
+        with self.assertRaises(ValueError) as cm:
+            custom_jc = JenkinsCore(jenkins_release=jenkins_release)
+            self.assertEqual(
+                cm.exception, ValueError("Wrong jenkins_release parameter. Valid values are [current|stable]")
+            )
diff --git a/tests/test_jenkins_plugin_manager_plugin.py b/tests/test_jenkins_plugin_manager_plugin.py
new file mode 100644
index 0000000..768a7ff
--- /dev/null
+++ b/tests/test_jenkins_plugin_manager_plugin.py
@@ -0,0 +1,176 @@
+import json
+import os
+import tempfile
+import unittest
+
+from unittest import mock
+
+import jenkins_plugin_manager
+
+from jenkins_plugin_manager.plugin import UpdateCenter
+
+UC_DATA_FILE = "update_center_data.json"
+
+
+class TestUpdateCenterLoad(unittest.TestCase):
+    """Tests for the UpdateCenter methods loading data."""
+
+    dir_path = os.path.dirname(os.path.realpath(__file__))
+    uc_data_file = os.path.join(dir_path, UC_DATA_FILE)
+    with open(uc_data_file, "r") as uc_data:
+        uc_data = uc_data.read()
+
+    @mock.patch("urllib.request.urlopen", autospec=True)
+    def test_load_update_center_data(self, urllib_mock):
+        cm = mock.MagicMock()
+        cm.read.return_value = "updateCenter.post(\n{0}\n);".format(self.uc_data).encode("utf-8")
+        cm.__enter__.return_value = cm
+        urllib_mock.return_value = cm
+        uc = UpdateCenter()
+        self.assertEqual(uc.uc_data, json.loads(self.uc_data))
+
+
+class TestUpdateCenter(unittest.TestCase):
+    """Tests for the UpdateCenter class and methods."""
+
+    dir_path = os.path.dirname(os.path.realpath(__file__))
+    uc_data_file = os.path.join(dir_path, UC_DATA_FILE)
+    with open(uc_data_file, "r") as uc_data:
+        uc_data = json.loads(uc_data.read())
+
+    def setUp(self):
+        """Load samples of the Jenkins UpdateCenter JSON files."""
+        uc_data_patch = mock.patch("jenkins_plugin_manager.plugin.UpdateCenter.load_update_center_data")
+        self.uc_data_mock = uc_data_patch.start()
+        self.addCleanup(self.uc_data_mock.reset_mock)
+        self.addCleanup(uc_data_patch.stop)
+        self.uc_data_mock.return_value = self.uc_data
+        self.uc = UpdateCenter()
+
+    def test_default_load_data(self):
+        """Test the default JSON files."""
+        self.uc_data_mock.assert_called_once_with("https://updates.jenkins.io/stable/update-center.json")
+
+        self.uc_data_mock.reset_mock()
+
+    @mock.patch("jenkins_plugin_manager.plugin.UpdateCenter.load_update_center_data")
+    def test_custom_load_data(self, custom_uc_data_mock):
+        """Test custom urls for JSON files."""
+        custom_uc_data_mock.return_value = self.uc_data
+        UpdateCenter(uc_url="http://archive.admin.canonical.com/others/stable/update-center.json",)
+        custom_uc_data_mock.assert_called_once_with(
+            "http://archive.admin.canonical.com/others/stable/update-center.json"
+        )
+
+    def test_get_plugin_data(self):
+        """Test the plugin data and the exception raised."""
+        plugin_test = "ansible"
+        plugin_data = self.uc.get_plugin_data(plugin_test)
+        self.assertEqual(plugin_data, self.uc_data["plugins"][plugin_test])
+        plugin_test = "PluginThatDoesNotExist"
+        with self.assertRaises(jenkins_plugin_manager.exceptions.InvalidPluginError):
+            self.uc.get_plugin_data(plugin_test)
+        plugin_test = "docker-workflow:999999"
+        with self.assertRaises(jenkins_plugin_manager.exceptions.InvalidPluginVersionError):
+            plugin_data = self.uc.get_plugin_data(plugin_test)
+        plugin_test = "docker-workflow:1.20"
+        with self.assertLogs(logger="jenkins_plugin_manager", level="INFO") as cm:
+            plugin_data = self.uc.get_plugin_data(plugin_test)
+        self.assertEqual(plugin_data["version"], "1.21")
+        self.assertEqual(
+            cm.output,
+            ["INFO:jenkins_plugin_manager.plugin:Getting docker-workflow:1.21 instead of docker-workflow:1.20"],
+        )
+
+    def test_verify_sha256sum(self):
+        """Test the weird sha256sum.
+
+        The sha256sum provided in the Update Center is a weird one.
+        See https://github.com/jenkins-infra/update-center2/blob/master/src/main/java/org/jvnet/hudson/update_center/IndexHtmlBuilder.java#L82 # noqa
+        Fake a file containing "Jenkins Plugin Manager Rocks!\n" for the check.
+        The sha256sum is "0290683d8fa6746e9d6de82d0ab8753a5e4078d5c2a9a40aa08f1a0c05429748".
+        The sha256sum expected as argument is "ApBoPY+mdG6dbegtCrh1Ol5AeNXCqaQKoI8aDAVCl0g=".
+        """
+        with tempfile.TemporaryDirectory() as test_dir:
+            stub_file = os.path.join(test_dir, "check")
+            with open(stub_file, "w") as test_file:
+                test_file.write("Jenkins Plugin Manager Rocks!\n")
+            check = self.uc._verify_sha256sum(stub_file, "ApBoPY+mdG6dbegtCrh1Ol5AeNXCqaQKoI8aDAVCl0g=")
+            self.assertTrue(check)
+
+    def test_check_min_core_version(self):
+        """Test the required Jenkins core version is available."""
+        with self.assertLogs(logger="jenkins_plugin_manager.plugin", level="ERROR") as cm:
+            self.assertFalse(self.uc._check_min_core_version("2.1", "2.190.1"))
+            self.assertEqual(
+                cm.output,
+                ["ERROR:jenkins_plugin_manager.plugin:Jenkins core version 2.190.1 or above is required (2.1 found)"],
+            )
+        self.assertTrue(self.uc._check_min_core_version("2.190.1", "2.190"))
+
+    @mock.patch("jenkins_plugin_manager.plugin.UpdateCenter.get_plugin_data")
+    @mock.patch("jenkins_plugin_manager.plugin.UpdateCenter._verify_sha256sum", autospec=True)
+    @mock.patch("urllib.request.urlopen", autospec=True)
+    def test_download_plugin(self, urllib_mock, checksum_mock, get_plugin_data_mock):
+        """Test the download of a plugin."""
+
+        def get_plugin_data(plugin):
+            return self.uc_data["plugins"][plugin]
+
+        cm = mock.MagicMock()
+        cm.read.return_value = b"This is a plugin"
+        cm.__enter__.return_value = cm
+        urllib_mock.return_value = cm
+        checksum_mock.return_value = True
+        get_plugin_data_mock.side_effect = get_plugin_data
+        with tempfile.TemporaryDirectory() as test_dir:
+            plugin_file = os.path.join(test_dir, "jenkins", "ansicolor-0.6.2.jpi")
+            with self.assertLogs(logger="jenkins_plugin_manager.plugin", level="INFO") as cm:
+                filepath = self.uc.download_plugin("ansicolor", os.path.join(test_dir, "jenkins"))
+            self.assertTrue(os.path.exists(os.path.join(test_dir, "jenkins")))
+            self.assertEqual(filepath, plugin_file)
+            self.assertEqual(
+                cm.output,
+                [
+                    "INFO:jenkins_plugin_manager.plugin:Downloading plugin {0} from {1}".format(
+                        "ansicolor", "http://updates.jenkins-ci.org/download/plugins/ansicolor/0.6.2/ansicolor.hpi"
+                    ),
+                    "INFO:jenkins_plugin_manager.plugin:Plugin ansicolor downloaded to {0}".format(plugin_file),
+                ],
+            )
+            # Check that when verify_sha256sum fails, return False
+            checksum_mock.return_value = False
+            with self.assertLogs(logger="jenkins_plugin_manager.plugin", level="INFO") as cm:
+                self.assertFalse(
+                    self.uc.download_plugin(
+                        "ant",
+                        os.path.join(test_dir, "jenkins"),
+                        plugin_url="http://dummy.canonical.com/ant.hpi",
+                        with_version=False,
+                    )
+                )
+            self.assertEqual(
+                cm.output,
+                [
+                    "INFO:jenkins_plugin_manager.plugin:Downloading plugin {0} from {1}".format(
+                        "ant", "http://dummy.canonical.com/ant.hpi"
+                    )
+                ],
+            )
+            self.assertFalse(os.path.exists(os.path.join(test_dir, "jenkins", "ant-1.10.jpi")))
+            self.assertTrue(os.path.exists(os.path.join(test_dir, "jenkins", "ant.jpi")))
+
+    def test_get_plugin_dependencies(self):
+        """Test the list of dependencies for a given plugin."""
+        # Should be empty because all dependencies are optional
+        deps = self.uc.get_plugins_dependencies("ansicolor")
+        self.assertEqual(set(), deps)
+        deps = self.uc.get_plugins_dependencies("ansicolor", optional=True)
+        self.assertEqual({"workflow-api", "workflow-step-api"}, deps)
+
+    def test_get_plugins(self):
+        """Test the list of plugins to fetch and the check with the current Jenkins core version."""
+        with self.assertRaises(jenkins_plugin_manager.exceptions.PluginMinCoreVersionError):
+            self.uc.get_plugins(["ansicolor"], current_core_version="2.144")
+        plugins = self.uc.get_plugins(["ant"])
+        self.assertEqual(plugins, {"ant", "structs"})
diff --git a/tox.ini b/tox.ini
new file mode 100644
index 0000000..e94d726
--- /dev/null
+++ b/tox.ini
@@ -0,0 +1,36 @@
+[tox]
+skipsdist=True
+envlist = unit
+skip_missing_interpreters = True
+
+[testenv]
+basepython = python3
+setenv =
+  PYTHONPATH = {toxinidir}
+
+[testenv:unit]
+commands =
+    pytest \
+      {posargs:-v  --cov=jenkins_plugin_manager --cov-report=term-missing --cov-branch}
+deps =
+    -r{toxinidir}/requirements.txt
+    pytest
+    pytest-coverage
+
+[testenv:black]
+commands =
+    black --skip-string-normalization --line-length=120 {toxinidir}
+deps = black
+
+[testenv:lint]
+commands =
+    flake8 --exclude=.svn,CVS,.bzr,.hg,.git,__pycache__,.tox,.eggs,*.egg,.venv
+deps = flake8
+
+[flake8]
+exclude =
+    .git,
+    __pycache__,
+    .tox,
+max-line-length = 120
+max-complexity = 10