Merge ~graylog-charmers/charm-graylog:feature/multi-version into ~graylog-charmers/charm-graylog:master

[Kevin W Monroe]

Enable snap 'channel' config so users can switch between graylog versions.

This enables the charm operator to configure the major graylog version used when installing graylog from the snap store.

This is still a WIP as the upgrade from 2->3 hasn't been fully tested. It is, however, ready for early feedback. Folks that want to kick the tires can deploy ~graylog-charmers/graylog/43:

https://jaas.ai/u/graylog-charmers/graylog/43

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Unable to determine commit message from repository - please click "Set commit message" and enter the commit message manually.

[Stuart Bishop (stub)]

Looks good. Some minor comments inline, which you might want to change before this lands.

web_listen_uri, web_endpoint_uri, rest_transport_uri... oh my. I don't think this stuff should be in the config.yaml, instead the charm setting good defaults using the Juju network information. But they already exist, so reasonable that this is working with what we have rather than proposing a major backwards incompatible change :)

review: Approve

[Kevin W Monroe (kwmonroe)]

@stub, I agree that a lot of this charm config is a mess. Perhaps when we eventually make v3 the default, we can work on smarter bindings and deprecate at least a handful of these opts.

Thanks for the is_v2 helper idea! It simplified both reactive and lib code nicely. I implemented the channel handler so snap refresh/reconfig work like they should. And what the heck, i threw in some unit tests.

One important thing to note: I've made graylog.snap a 0-byte resource for the edge and beta channels. I felt it better for the default behavior to be a snap store install using a configured channel. The alternative would be to continue hosting the 150MB graylog.snap (v2), then requiring users to pull down and attach a 150MB v3 snap to get v3 deployed.

If this change in default behavior irks anyone, please chime in. Latest changes can be found in rev 44, currently in beta:

https://jaas.ai/u/graylog-charmers/graylog/44

[Paul Goins (vultaire)]

Overall, looks pretty nice.

I've provided a few comments for review. Note that only some of these "require" changes I think, but please look at all 4.

Additionally... In Graylog 3, a few APIs also changed, and I don't see those reflected in this change set. Specifically, I'm referring to these: https://git.launchpad.net/~vultaire/graylog-charm/diff/lib/charms/layer/graylog/logextract.py?h=graylog3&id=0b6162815e80e3b8ba73542b2d9dd326f74fa307

(Note the above is from a quick-and-dirty, backwards-incompatible fork of the Graylog charm I wrote for testing Graylog 3; it cannot be used as is since we need to call the correct API based on the graylog version.)

review: Needs Fixing

[Paul Goins (vultaire)]

Withdrew one of my comments.

[Rodrigo Barbieri (rodrigo-barbieri2010)]

Updated with the review feedback of Paul Groins.

Changed

@when_not('snap.installed.graylog')
def install_graylog():

to

@when('snap.installed.core')
def install_graylog():

[Paul Goins (vultaire)]

Re: my previous comments regarding v2/v3 APIs: as I don't think those bits are actually being used (since the config key that exposes them isn't exposed via config.yaml yet), they can be fixed later.

My only remaining concern remains regarding the description of the web_listen_uri config value; see my previously-posted comment on config.yaml from 2019-09-03.

review: Needs Fixing

[Rodrigo Barbieri (rodrigo-barbieri2010)]

Thanks again Paul. I updated the config option description.

[Paul Goins (vultaire)]

review: Approve

[Kevin W Monroe (kwmonroe)]

@when('snap.installed.core')
def install_graylog():

Can't do that ^^. Reactive will run 'install_graylog' every 5 minutes (every update_status hook) if we do.

review: Needs Fixing

[Kevin W Monroe (kwmonroe)]

On Wed, Sep 4, 2019 at 1:58 AM Paul Goins <email address hidden> wrote:

> > diff --git a/lib/charms/layer/graylog/__init__.py
> b/lib/charms/layer/graylog/__init__.py
> > index e69de29..38bffbe 100644
> > --- a/lib/charms/layer/graylog/__init__.py
> > +++ b/lib/charms/layer/graylog/__init__.py
> > @@ -0,0 +1,39 @@
> > +from charmhelpers.core.hookenv import log
>
> Purely stylistic; don't consider this a -1. (At least not from me, as I'm
> somewhat new to reactive charms.)
>
> Per charms.reactive docs, this type of layer lib file seems intended for
> base layers rather than charm layers. And I do see similar helper
> functions directly in the reactive/graylog.py module.
>
> Is the intent for this to be reused by other charms? Or simply as a "lib"
> module separate from the handlers?
>

The latter; this module is meant to be helpful funcs for graylog that
aren't reactive. The helpers you see in reactive/gl.py should also be in
this lib. Ideally, only @reactive funcs would appear in reactive/*.py, but
we didn't start that way, so that's not how it is today. That said, we
don't have to keep going down the wrong road. There's an opportunity for
better organization in this charm, but that's beyond the scope of the v3
changes proposed here.

> diff --git a/reactive/graylog.py b/reactive/graylog.py
> > index 0d64f26..4a718bd 100644
> > --- a/reactive/graylog.py
> > +++ b/reactive/graylog.py
> > @@ -55,15 +67,56 @@ rotation_strategies = {
> > }
> >
> >
> > +@when_not('snap.installed.graylog')
>
> I've got a little concern here. If I understand the charms.reactive docs
> correctly; it doesn't seem that there's a guaranteed order that multiple
> matching handlers get executed in. (I'm relatively new to reactive charms
> so I may be wrong; correct me if so!)
>
> I think there's no guarantee here whether the snap layer's install()
> function or this install_graylog() function will be run first.
>
> While it feels kind of dirty, I think one way to guarantee this runs after
> the snap layer w/o requiring changes to it would be to add a
> @when('snap.installed.core') decorator, as the snap layer will set that
> once it runs its install() function. (I'd prefer a clearer flag that the
> snap install handler has run, but this should work I think.)
>

The problem with @when('snap.installed.core') is that once it's set, it's
set forever. That means every hook will trigger that handler until the end
of time. Yes, it's not a big handler -- we'd call "snap install graylog"
on the system and it would come back with "hey, it's already installed".
Still, it's burning cycles needlessly. I understand your concern, but snap
is smart enough that we don't actually have to wait for 'core'. If you do
a 'snap install hello-world' and 'core' isn't there, snapd will bring it
(and any other prereqs) down first.

It's safe to do @when_not(snap.installed.graylog) to guard the install
regardless of prereqs like 'core' being installed. In fact, layer-snap's
install function will set 'snap.installed.core' if it's not already:

https://git.launchpad.net/layer-snap/tree/lib/charms/layer/snap.py#n60

[Paul Goins (vultaire)]

Hi Kevin,

Thanks for the explanation - you corrected some misunderstandings I had about reactive charm flow. I was treating everything similar to how you'd treat e.g. "config.changed", i.e. only set when something has changed.

[Paul Goins (vultaire)]

The above being said, the intent of my review comment wasn't to remove "@when_not('snap.installed.graylog')", but merely to add "@when('snap.installed.core')", e.g.:

  @when('snap.installed.core')
  @when_not('snap.installed.graylog')
  def install_graylog():
      [....]

However, I did approve the change where the @when_not decorator was removed, which was a miss.

I think if both decorators are there, it'll run only once *and* only when Graylog isn't installed via the snap layer.

What do you think?

[Kevin W Monroe (kwmonroe)]

I addressed previous comments in this last round of commits. Specifically:

- Guard install_graylog with @when_not(snap.installed.graylog). While I appreciate the idea of @when(core); @when_not(graylog), it's not needed for a couple reasons: (1) snap.install will ensure all prereqs are installed. (2) if 'core' ever changed names or graylog grew a new prereq, we'd have to come back and update the install_graylog decorators. Let's let snap handle the install just like we would apt.

- The beats input was being reconfigured every hook. There's no need for that. Instead, only reconfigure when config changes.

- From an earlier __init__.py comment, I decided to refactor the lib/charms/layer/graylog package. Move utils to a proper utils.py and make the relevant submodule bits known in __init__.py.

- From an earlier logextract.py comment, set the right URL depending on which version of graylog is installed. (Big Thanks for this heads up -- i didn't realize the URLs had changed)

- Test and Readme updates.

Interested parties can kick the tires with cs:~graylog-charmers/graylog-45, which has been released through candidate.

review: Approve

[Kevin W Monroe (kwmonroe)]

Latest round of updates addresses how we acknowledge the snap channel when upgrading from a charm that doesn't include that option. Also leverages leader unit data so we don't configure graylog until leader passwords are setup (we need all units to agree on the passwords).

And finally, i noticed the nagios handler wouldn't change the api url/port on upgrade. This has been fixed.

Pushed through candidate in cs:~graylog-charmers/graylog-49

[Kevin W Monroe (kwmonroe)]

IS deploy/upgrade tests were successful:

https://jenkins.canonical.com/is-mojo-ci/job/live-is-charm-test-graylog-bionic/113/

I also ran this through a round of manual charmed k8s integration tests which passed (graylog v3 showed extended k8s metadata in the logs).

Thanks to all for the reviews/suggestions. Rev cs:~graylog-charmers/graylog-49 is now in stable.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Change has no commit message, setting status to needs review.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Change successfully merged at revision 7cb4b82ab266da4e755142dbff3b5836b537bc38

diff --git a/README.md b/README.md
index a8174c7..94a4b69 100644
--- a/README.md
+++ b/README.md
@@ -1,65 +1,101 @@
 # Overview
 
-The charm installs [Graylog](https://www.graylog.org/) using the [snap package](https://uappexplorer.com/snap/ubuntu/graylog).
-The charm must be related to elasticsearch and mongodb in order to be a fully functioning installation.
+The charm installs [Graylog](https://www.graylog.org/) using the [snap package](https://snapcraft.io/graylog).
 
-# Usage
+## Usage
 
+```bash
 juju deploy cs:~graylog-charmers/graylog
-juju run-action graylog/X show-admin-password
-juju show-action-output <action ID>
+juju run-action --wait graylog/X show-admin-password
+```
 
 Graylog requires MongoDB to run and Elasticsearch to be useful.
 
-juju deploy cs:mongodb
+```bash
+juju deploy cs:~mongodb-charmers/mongodb
 juju relate graylog:mongodb mongodb:database
+
 juju deploy cs:~elasticsearch-charmers/elasticsearch
 juju relate graylog:elasticsearch elasticsearch:client
+```
 
-You can then browse to http://ip-address:9000 and log in as the user "admin".
-The password is by default a random value so 'juju run-action --wait graylog/X show-admin-password' must be run for admin access to the installation.
+You can then browse to `http://ip-address:9000` and log in as the user "admin".
+The password is by default a random value so `juju run-action --wait graylog/X show-admin-password` must be run for
+admin access to the installation.
 
 ## Reverseproxy Relation
-Graylog supports advertising its web and api ports to an application acting as a reverseproxy using the http relation.
-The port of the webUI is exposed over the relation as is the port for both the webUI and API in the all_services variable of the relation.
-More details on using this are in the reverseproxy instructions for the [Apache2 charm](https://jujucharms.com/apache2/).
 
-For example, you could use the following as a graylog vhost template for the apache2 charm.
-Note that you'll need to update the GRAYLOG_UNIT_IP in the template below to match the IP of your graylog/X unit.
+Graylog supports advertising its ports to an application acting as a reverseproxy using the `http` relation. The port
+of the webUI is exposed over the relation in the `all_services` variable of the relation.
 
-```
+>**Note**: For Graylog version 2, the API port is also exposed over the `http` relation. Graylog version 3 hard codes
+the `/api/` location and uses the default port (9000) for both webUI and API.
+
+More details on using this are in the reverseproxy instructions for the
+[Apache2 charm](https://jujucharms.com/apache2/).
+
+Sample Graylog 2 vhost template for the apache2 charm:
+
+```bash
 $ cat graylog-vhost.tmpl
 <Location "/">
     RequestHeader set X-Graylog-Server-URL "http://{{servername}}/api/"
-    ProxyPass http://GRAYLOG_UNIT_IP:9000/
-    ProxyPassReverse http://GRAYLOG_UNIT_IP:9000/
+    ProxyPass http://{{graylog_web}}/
+    ProxyPassReverse http://{{graylog_web}}/
 </Location>
 
 <Location "/api/">
-    ProxyPass http://GRAYLOG_UNIT_IP:9001/api/
-    ProxyPassReverse http://GRAYLOG_UNIT_IP:9001/api/
+    ProxyPass http://{{graylog_api}}/api/
+    ProxyPassReverse http://{{graylog_api}}/api/
 </Location>
 ```
 
-Now deploy and configure apache2 as your graylog reverse proxy:
+Sample Graylog 3 vhost template for the apache2 charm:
 
+```bash
+$ cat graylog-vhost.tmpl
+<Location "/">
+    RequestHeader set X-Graylog-Server-URL "http://{{servername}}/"
+    ProxyPass http://{{graylog_web}}/
+    ProxyPassReverse http://{{graylog_web}}/
+</Location>
 ```
+
+Now deploy and configure apache2 as your graylog reverse proxy:
+
+```bash
 juju deploy apache2
 juju config apache2 "enable_modules='headers proxy_html proxy_http'"
-juju config apache2 "vhost_http_template=$(base64 < graylog-vhost.tmpl)"
+juju config apache2 "vhost_http_template=$(base64 ./graylog-vhost.tmpl)"
 juju expose apache2
 juju relate apache2:reverseproxy graylog:website
 ```
 
-Visit http://[apache2-public-ip] to access the Graylog interface.
+Visit `http://<apache2-public-ip>` to access the Graylog web interface.
 
 ## Scale out Usage
 
-The MongoDB and Elasticsearch applications can both be scaled to clusters and Graylog will adapt to using the cluster.
+The MongoDB and Elasticsearch applications can both be scaled up or down. Graylog will reconfigure itself as needed.
 The Graylog charm does not yet support clustering of multiple units.
 
-# Configuration
+## Configuration
+
+Depending on the Elasticsearch charm used, the cluster name may not be passed to Graylog. In this case, the
+`elasticsearch_cluster_name` config option should be set.
+
+## Upgrade
+
+Graylog may be upgraded to a different snap version by setting the `channel` config option. For example, switch to the
+latest version 3 edge snap with the following:
+
+```bash
+juju config graylog channel='3/edge'
+```
 
-The administrator password is by default a random value so 'juju run-action --wait graylog/X show-admin-password' must be run for admin access to the installation.
+>**Note**: When upgrading from Graylog version 2 to version 3, please consult the
+[upgrade guide](https://docs.graylog.org/en/3.1/pages/upgrade.html) to ensure your environment meets the minimum
+requirements.
 
-Depending on the Elasticsearch charm used the cluster name may not be passed in the relation in which case it the `elasticsearch_cluster_name` config option should be set.
+If a new `channel` config option results in a new snap being installed, the charm will backup the previous
+configuration file on the graylog unit in `/var/snap/graylog/common/server.conf.$prev`. This may be useful if
+graylog needs to be reverted to a previous version in the future.
diff --git a/config.yaml b/config.yaml
index 77bc734..d0d2649 100644
--- a/config.yaml
+++ b/config.yaml
@@ -1,4 +1,11 @@
 options:
+  channel:
+    type: string
+    default: "2/stable"
+    description: |
+      Snap channel used to install/refresh the graylog snap.
+
+      This option has no effect when a valid graylog.snap resource is attached.
   elasticsearch_cluster_name:
     type: string
     default: ""
@@ -6,24 +13,38 @@ options:
   web_listen_uri:
     type: string
     default: "http://0.0.0.0:9000/"
-    description: The uri the web interface will be available at.
+    description: |
+      The uri the web interface will be available at. In version 3 and higher,
+      this is used for and converted to the appropriate format for
+      the 'http_bind_address' config value.
   web_endpoint_uri:
     type: string
     default: ""
-    description: >
-      If set, this will be published as the external address for connecting REST
-      API of the Graylog server. Web interface clients need to be able to
-      connect to this for the web interface to work.
-      By default, will use `rest_transport_uri` if defined, or will select one
-      of the node's allocated ips on port 9001. Example: http://10.0.0.1:9001/
+    description: |
+      If set, this will be published as the external address for connecting to
+      the REST API of the Graylog server. Web interface clients need to be able
+      to connect to this for the web interface to work.
+
+      In version 2, Graylog will set the default value to 'rest_transport_uri'
+      if defined; otherwise it will select the first non-loopback IPv4 address
+      on port 9001. Example: http://10.0.0.1:9001/
+
+      In version 3, Graylog refers to this option as 'http_external_uri' with
+      the default value being 'rest_transport_uri' if defined; otherwise it will
+      select the first non-loopback IPv4 address on port 9000.
+      Example: http://10.0.0.1:9000/
   rest_transport_uri:
     type: string
     default: ""
-    description: >
+    description: |
       If set, this will be promoted in the cluster discovery APIs. You will need
-      to define this, if your Graylog server is running behind a HTTP proxy that
-      is rewriting the scheme, host name or URI. This must not contain a
-      wildcard address (0.0.0.0). Example: http://192.168.1.1:9001/
+      to define this if your Graylog server is running behind a HTTP proxy that
+      is rewriting the scheme, host name, or URI. This must not contain a
+      wildcard address (0.0.0.0).
+
+      For Graylog 2, this usually takes the form http://10.0.0.1:9001/api/.
+      For Graylog 3 and higher, this is known as the 'http_publish_uri' and looks
+      like http://10.0.0.1:9000/.
   index_replicas:
     type: int
     default: 0
diff --git a/layer.yaml b/layer.yaml
index d4fa469..414c56e 100644
--- a/layer.yaml
+++ b/layer.yaml
@@ -8,6 +8,11 @@ includes:
   - 'interface:http'
   - 'interface:mongodb-cluster'
   - 'interface:nrpe-external-master'
+exclude:
+  - .coverage
+  - .tox
+  - .unit-state.db
+  - __pycache__
 options:
   basic:
     packages:
@@ -15,5 +20,3 @@ options:
   snap:
     core:
       channel: stable
-    graylog:
-      channel: stable
diff --git a/lib/charms/layer/graylog/__init__.py b/lib/charms/layer/graylog/__init__.py
index e69de29..5399010 100644
--- a/lib/charms/layer/graylog/__init__.py
+++ b/lib/charms/layer/graylog/__init__.py
@@ -0,0 +1,3 @@
+from .api import GraylogApi  # NOQA
+from .logextract import GraylogPipelines, GraylogRules, GraylogStreams, LogExtractPipeline  # NOQA
+from .utils import *  # NOQA
diff --git a/lib/charms/layer/graylog/logextract.py b/lib/charms/layer/graylog/logextract.py
index 2447da4..5d1ae27 100644
--- a/lib/charms/layer/graylog/logextract.py
+++ b/lib/charms/layer/graylog/logextract.py
@@ -3,6 +3,7 @@ extraction
 """
 
 import re
+from .utils import is_v2
 
 
 class GraylogTitleSourceItems:
@@ -47,11 +48,13 @@ class GraylogTitleSourceItems:
 
 
 class GraylogPipelines(GraylogTitleSourceItems):
-    url = '/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/pipeline'
+    url = '{}/system/pipelines/pipeline'.format(
+        '/plugins/org.graylog.plugins.pipelineprocessor' if is_v2() else '')
 
 
 class GraylogRules(GraylogTitleSourceItems):
-    url = '/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/rule'
+    url = '{}/system/pipelines/rule'.format(
+        '/plugins/org.graylog.plugins.pipelineprocessor' if is_v2() else '')
 
 
 class GraylogStreams:
@@ -78,7 +81,9 @@ class GraylogStreams:
         :param pipeline title
         :param stream title
         """
-        url = "/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/connections/to_stream"
+        url = '{}/system/pipelines/connections/to_stream'.format(
+            '/plugins/org.graylog.plugins.pipelineprocessor' if is_v2() else '')
+
         streamid = self.get_stream_id_for(stream)
         pipes = GraylogPipelines(self.graylogapi)
         pipeline = pipes.get_for_title(pipeline)
diff --git a/lib/charms/layer/graylog/utils.py b/lib/charms/layer/graylog/utils.py
new file mode 100644
index 0000000..cf89b61
--- /dev/null
+++ b/lib/charms/layer/graylog/utils.py
@@ -0,0 +1,42 @@
+from charmhelpers.core import hookenv
+from charms.layer import snap
+
+SNAP_NAME = 'graylog'
+
+
+def get_api_port():
+    return '9001' if is_v2() else '9000'
+
+
+def get_api_url():
+    return 'http://127.0.0.1:{}/api/'.format(get_api_port())
+
+
+def is_v2():
+    version = snap.get_installed_version(SNAP_NAME)
+    # If the snap isn't installed yet, base our version off the charm config
+    if not version:
+        version = hookenv.config('channel')
+    return True if version.startswith('2') else False
+
+
+def validate_api_uri(uri):
+    """Ensure the given API URI is valid based on the Graylog version.
+
+    For v2, api-related options should end with '/api/'. In v3, the graylog
+    server hard codes '/api/' to the end of these options and should therefore
+    be removed if included in the charm config.
+    """
+    if not uri.endswith('/'):
+        uri += '/'
+
+    if is_v2():
+        if not uri.endswith('/api/'):
+            hookenv.log("Appending 'api/' to the configured REST API options")
+            uri += 'api/'
+    else:
+        if uri.endswith('/api/'):
+            hookenv.log("Removing 'api/' from the configurted REST API options")
+            uri = uri[:-4]
+
+    return uri
diff --git a/reactive/graylog.py b/reactive/graylog.py
index b07e376..17427b4 100644
--- a/reactive/graylog.py
+++ b/reactive/graylog.py
@@ -1,23 +1,31 @@
 import hashlib
 import os
 import re
+import shutil
 import time
 import yaml
-from subprocess import check_output, CalledProcessError
 from urllib.parse import urlparse
 
-from charms.reactive import hook, when, when_not, is_state, remove_state, set_state
+from charms.reactive import hook, when, when_any, when_not, when_not_all, is_state, remove_state, set_state
 from charms.reactive.helpers import data_changed
 from charmhelpers.core import host, hookenv, unitdata
 from charmhelpers.contrib.charmsupport import nrpe
 import charms.leadership
 
-from charms.layer.graylog import logextract
-from charms.layer.graylog.api import GraylogApi
+from charms.layer import snap
+from charms.layer.graylog import (
+    GraylogApi,
+    LogExtractPipeline,
+    SNAP_NAME,
+    get_api_port,
+    get_api_url,
+    is_v2,
+    validate_api_uri
+)
+
 
-API_PORT = '9001'  # This is the default set by the snap
-API_URL = 'http://127.0.0.1:9001/api/'  # This is the default set by the snap
 CONF_FILE = '/var/snap/graylog/common/server.conf'
+SNAP_CONF_FILE = '/snap/graylog/current/etc/graylog/server/server.conf'
 # /snap/graylog is a read-only squashfs so we use the initial settings
 # and write out an override to SERVER_DEFAULT_CONF_FILE
 SHIPPED_SNAP_SERVER_DEFAULT_CONF_FILE = '/snap/graylog/current/etc/default/graylog-server'
@@ -32,7 +40,13 @@ def upgrade_charm():
 
 
 @when('config.changed')
+@when_not('config.changed.channel')
 def update_config():
+    """Allow new runtime config options to be processed.
+
+    Reset flags to allow new runtime config options to be written. Deployment
+    config options (e.g. 'channel') are handled separately.
+    """
     remove_state('graylog.configured')
 
 
@@ -55,63 +69,105 @@ rotation_strategies = {
 }
 
 
+@when_not('snap.installed.graylog')
+def install_graylog():
+    """Install the graylog snap.
+
+    The snap layer will first try to install a valid 'graylog.snap' resource.
+    If a valid resource is not found, the snap will be installed from the snap
+    store according to the configured 'channel' option.
+    """
+    channel = hookenv.config('channel')
+
+    hookenv.status_set('maintenance', 'Installing graylog snap')
+    snap.install(SNAP_NAME, channel=channel)
+
+
+@when('snap.installed.graylog')
+@when('config.changed.channel')
+def refresh_graylog():
+    """Refresh the graylog snap.
+
+    A change to the 'channel' config option may warrant a snap refresh. If so,
+    stop the service, backup the current config, and refresh.
+    """
+    channel = hookenv.config('channel')
+    cur_snap = new_snap = snap.get_installed_version(SNAP_NAME)
+
+    if channel:
+        host.service_stop(SERVICE_NAME)
+        # backup config file using the current version as a suffix
+        if os.path.exists(CONF_FILE):
+            shutil.copy2(CONF_FILE, '{path}.{ext}'.format(path=CONF_FILE, ext=cur_snap))
+
+        hookenv.status_set('maintenance', 'Refreshing graylog snap {}'.format(channel))
+        snap.refresh(SNAP_NAME, channel=channel)
+        new_snap = snap.get_installed_version(SNAP_NAME)
+    else:
+        hookenv.log("Cannot refresh snap when the 'channel' config option is missing")
+
+    # When the snap version changes, use the config file from the snap and
+    # adjust states to ensure appropriate re-config happens.
+    if cur_snap != new_snap:
+        shutil.copy2(SNAP_CONF_FILE, CONF_FILE)
+        data_changed('elasticsearch.relation', None)
+        data_changed('mongodb.uri', None)
+        remove_state('graylog.configured')
+
+    report_status()
+
+
 @when('snap.installed.graylog')  # noqa: C901
 @when_not('graylog.configured')
 def configure_graylog():
     db = unitdata.kv()
     if not os.path.exists(CONF_FILE):
-        hookenv.log('Configuration file "{}" missing, skipping configuration run.'.format(CONF_FILE))
-        hookenv.status_set('waiting', 'Waiting for snap config: {}'.format(CONF_FILE))
+        hookenv.status_set('maintenance', 'Waiting for {}'.format(CONF_FILE))
         return
     conf = hookenv.config()
+
     admin_password = db.get('admin_password')
-    if not admin_password:
-        admin_password = host.pwgen(18)
-        db.set('admin_password', admin_password)
+    password_secret = db.get('password_secret')
+    if not (admin_password and password_secret):
+        hookenv.status_set('maintenance', 'Waiting for leader passwords')
+        return
     pw_hash = hashlib.sha256(admin_password.encode('utf-8')).hexdigest()
     set_conf('root_password_sha2', pw_hash)
+    set_conf('password_secret', password_secret)
 
-    if conf['web_listen_uri']:
-        url = urlparse(conf['web_listen_uri'])
-        webport = db.get('web_listen_port')
+    api_port = get_api_port()
+    if is_v2():
+        if conf['web_listen_uri']:
+            url = urlparse(conf['web_listen_uri'])
+            webport = db.get('web_listen_port')
 
-        if webport == API_PORT:
-            msg = "Config 'web_listen_uri' is using port {} which conflicts with default REST port {}".format(
-                webport, API_PORT
-            )
-            hookenv.status_set('blocked', msg)
-            return
+            if webport == api_port:
+                msg = "Config 'web_listen_uri' is using port {} which conflicts with default REST port {}".format(
+                    webport, api_port
+                )
+                hookenv.status_set('blocked', msg)
+                return
+
+            set_conf('web_listen_uri', conf['web_listen_uri'])
+            if url.port != webport:
+                if webport:
+                    hookenv.close_port(webport)
+                db.set('web_listen_port', url.port)
+                hookenv.open_port(url.port)
 
-        set_conf('web_listen_uri', conf['web_listen_uri'])
-        if url.port != webport:
-            if webport:
-                hookenv.close_port(webport)
-            db.set('web_listen_port', url.port)
-            hookenv.open_port(url.port)
-
-    if conf['rest_transport_uri']:
-        set_conf('rest_transport_uri', _validate_api_uri(conf['rest_transport_uri']))
-
-    if conf['web_endpoint_uri']:
-        set_conf('web_endpoint_uri', _validate_api_uri(conf['web_endpoint_uri']))
-
-    hookenv.open_port(API_PORT)
-
-    # Set application version
-    snap_name = "graylog"
-    snap_version = ""
-    cmd = ['snap', 'list', snap_name]
-    try:
-        out = check_output(cmd).decode('UTF-8')
-    except CalledProcessError:
-        pass
+        if conf['rest_transport_uri']:
+            set_conf('rest_transport_uri', validate_api_uri(conf['rest_transport_uri']))
+
+        if conf['web_endpoint_uri']:
+            set_conf('web_endpoint_uri', validate_api_uri(conf['web_endpoint_uri']))
     else:
-        lines = out.split('\n')
-        for line in lines:
-            if snap_name in line:
-                # Second item in list is Version
-                snap_version = line.split()[1]
-    hookenv.application_version_set(snap_version)
+        if conf['web_listen_uri']:
+            url = urlparse(conf['web_listen_uri'])
+            set_conf('http_bind_address', url.netloc)
+        if conf['rest_transport_uri']:
+            set_conf('http_publish_uri', validate_api_uri(conf['rest_transport_uri']))
+        if conf['web_endpoint_uri']:
+            set_conf('http_external_uri', validate_api_uri(conf['rest_transport_uri']))
 
     if conf.get('logextraction-rules'):
         rules_config = yaml.safe_load(conf.get('logextraction-rules'))
@@ -120,31 +176,32 @@ def configure_graylog():
 
     set_jvm_heap_size(conf['jvm_heap_size'])
 
+    hookenv.application_version_set(snap.get_installed_version(SNAP_NAME))
+    hookenv.open_port(api_port)
+
     remove_state('graylog_api.configured')
     set_state('graylog.configured')
     report_status()
 
 
-def _validate_api_uri(uri):
-    if not uri.endswith('/'):
-        uri += '/'
-    if not uri.endswith('/api/'):
-        uri += 'api/'
-    return uri
-
-
 @when('leadership.is_leader')
-@when_not('leadership.set.admin_password')
-def generate_admin_password():
-    admin_password = host.pwgen(18)
-    charms.leadership.leader_set(admin_password=admin_password)
+@when_not_all('leadership.set.admin_password', 'leadership.set.password_secret')
+def generate_leader_passwords():
+    if not is_state('leadership.set.admin_password'):
+        admin_password = host.pwgen(18)
+        charms.leadership.leader_set(admin_password=admin_password)
+    if not is_state('leadership.set.password_secret'):
+        password_secret = host.pwgen(96)
+        charms.leadership.leader_set(password_secret=password_secret)
 
 
-@when('leadership.changed.admin_password')
-def get_leader_admin_password():
+@when_any('leadership.changed.admin_password', 'leadership.changed.password_secret')
+def get_leader_passwords():
     db = unitdata.kv()
     admin_password = charms.leadership.leader_get('admin_password')
     db.set('admin_password', admin_password)
+    password_secret = charms.leadership.leader_get('password_secret')
+    db.set('password_secret', password_secret)
     remove_state('graylog.configured')
 
 
@@ -226,17 +283,21 @@ def report_status():
 @when_not('graylog.needs_restart')
 @when_not('graylog_api.configured')
 def configure_graylog_api(*discard):
+    """Adjust states to ensure API dependents will be correctly configured."""
+    remove_state('beat.setup')
     remove_state('graylog_index_sets.configured')
     remove_state('graylog_inputs.configured')
+    remove_state('graylog_nagios.configured')
     set_state('graylog_api.configured')
 
 
-@when('graylog_api.configured')  # noqa: C901
+@when('graylog.configured')  # noqa: C901
+@when('graylog_api.configured')
 @when_not('graylog_index_sets.configured')
 def configure_index_sets(*discard):
     conf = hookenv.config()
     db = unitdata.kv()
-    g = GraylogApi(base_url=API_URL,
+    g = GraylogApi(base_url=get_api_url(),
                    username='admin',
                    password=db.get('admin_password'),
                    token_name='graylog-charm')
@@ -289,11 +350,11 @@ def configure_index_sets(*discard):
 
 def _configure_logextraction(rules_config):
     db = unitdata.kv()
-    g = GraylogApi(base_url=API_URL,
+    g = GraylogApi(base_url=get_api_url(),
                    username='admin',
                    password=db.get('admin_password'),
                    token_name='graylog-charm')
-    logext = logextract.LogExtractPipeline(g)
+    logext = LogExtractPipeline(g)
     logext.set_rules(rules_config)
 
 
@@ -344,7 +405,7 @@ def _remove_old_inputs(inputs, new_inputs):
     UI.
     """
     db = unitdata.kv()
-    g = GraylogApi(base_url=API_URL,
+    g = GraylogApi(base_url=get_api_url(),
                    username='admin',
                    password=db.get('admin_password'),
                    token_name='graylog-charm')
@@ -373,8 +434,7 @@ def graylog_remove_filebeat_input():
     hookenv.log("Removing filebeats input")
 
     db = unitdata.kv()
-
-    g = GraylogApi(base_url=API_URL,
+    g = GraylogApi(base_url=get_api_url(),
                    username='admin',
                    password=db.get('admin_password'),
                    token_name='graylog-charm')
@@ -393,22 +453,17 @@ def graylog_remove_filebeat_input():
     remove_state('beat.setup')
 
 
+@when('graylog.configured')
+@when('graylog_api.configured')
 @when('beats.connected')
-def provide_beats_port(filebeat):
-    beats_port = hookenv.config('beats_port')
-    filebeat.provide_data(beats_port)
-    set_state('beat.setup')
-
-
-@when('beat.setup')
-def setup_beats_input():
-
+@when_not('beat.setup')
+def setup_beats_input(filebeat):
     port = hookenv.config('beats_port')
     hookenv.log("Setting up beats input on port {}".format(port))
+    filebeat.provide_data(port)
 
     db = unitdata.kv()
-
-    g = GraylogApi(base_url=API_URL,
+    g = GraylogApi(base_url=get_api_url(),
                    username='admin',
                    password=db.get('admin_password'),
                    token_name='graylog-charm')
@@ -444,7 +499,10 @@ def setup_beats_input():
     else:
         hookenv.log("beats input already configured")
 
+    set_state('beat.setup')
+
 
+@when('graylog.configured')
 @when('graylog_api.configured')
 @when_not('graylog_inputs.configured')
 def configure_inputs(*discard):
@@ -453,7 +511,7 @@ def configure_inputs(*discard):
     """
     conf = hookenv.config()
     db = unitdata.kv()
-    g = GraylogApi(base_url=API_URL,
+    g = GraylogApi(base_url=get_api_url(),
                    username='admin',
                    password=db.get('admin_password'),
                    token_name='graylog-charm')
@@ -569,8 +627,9 @@ def update_reverseproxy_config(website):
         url = urlparse(conf['web_listen_uri'])
         website.configure(port=url.port)
         services += "- {service_name: web, service_port: " + str(url.port) + "}\n"
-
-    services += "- {service_name: api, service_port: " + API_PORT + "}\n"
+    if is_v2():
+        # NB: there is no revproxy to the api endpoint in v3+.
+        services += "- {service_name: api, service_port: " + get_api_port() + "}\n"
     website.set_remote(all_services=services)
 
 
@@ -692,16 +751,17 @@ def set_jvm_heap_size(heap_size='1G', conf_path=SERVER_DEFAULT_CONF_FILE):
 
 @when('graylog.configured')
 @when('nrpe-external-master.available')
+@when_not('graylog_nagios.configured')
 def configure_nagios(nagios):
-    if hookenv.hook_name() == 'update-status':
-        return
-
     db = unitdata.kv()
     nagios_password = db.get('nagios_password')
     if not nagios_password:
         nagios_password = host.pwgen(18)
         db.set('nagios_password', nagios_password)
 
+    api_port = get_api_port()
+    api_url = get_api_url()
+
     # Ask charmhelpers.contrib.charmsupport's nrpe to work out our hostname
     hostname = nrpe.get_nagios_hostname()
     nrpe_setup = nrpe.NRPE(hostname=hostname, primary=True)
@@ -728,11 +788,11 @@ def configure_nagios(nagios):
     nrpe_setup.add_check(
         'graylog_api',
         'Greylog API check',
-        '/usr/lib/nagios/plugins/check_http -I 127.0.0.1 -p {} -u /api -s "{}"'.format(API_PORT, check_string)
+        '/usr/lib/nagios/plugins/check_http -I 127.0.0.1 -p {} -u /api -s "{}"'.format(api_port, check_string)
     )
 
     # For nagios checks via API, we need to create a user and token
-    g = GraylogApi(base_url=API_URL,
+    g = GraylogApi(base_url=api_url,
                    username='admin',
                    password=db.get('admin_password'),
                    token_name='graylog-charm')
@@ -740,7 +800,7 @@ def configure_nagios(nagios):
     g.user_permissions_set('nagios', ['users:tokenlist', 'users:tokencreate',
                                       'indexercluster:read', 'indices:failures',
                                       'notifications:read', 'journal:read'])
-    g = GraylogApi(base_url=API_URL,
+    g = GraylogApi(base_url=api_url,
                    username='nagios',
                    password=nagios_password,
                    token_name='nagios')
@@ -764,7 +824,8 @@ def configure_nagios(nagios):
             os.path.join(nagios_plugins, 'check_graylog_health.py'),
             conf.get('nagios_uncommitted_warn', 0),
             conf.get('nagios_uncommitted_crit', 100),
-            API_URL)
+            api_url)
     )
 
     nrpe_setup.write()
+    set_state('graylog_nagios.configured')
diff --git a/unit_tests/test_graylog.py b/unit_tests/test_graylog.py
index 38ac6fa..37fb78d 100644
--- a/unit_tests/test_graylog.py
+++ b/unit_tests/test_graylog.py
@@ -3,20 +3,169 @@ import sys
 import tempfile
 import unittest
 from unittest import mock
+from charms.reactive import set_state, is_state
 
-from charms.layer.graylog.api import GraylogApi
-
-# charms.leadership only exists in the built charm; mock it out before
-# the graylog imports since those depend on charms.leadership
-layer_mock = mock.Mock()
-sys.modules['charms.leadership'] = layer_mock
+# some dep layers only exists in the built charm; mock them out before
+# the graylog imports since those depend on post-build charms.* layers
+leader_mock = mock.Mock()
+sys.modules['charms.leadership'] = leader_mock
+snap_mock = mock.Mock()
+sys.modules['charms.layer.snap'] = snap_mock
 
 from reactive.graylog import (
+    GraylogApi,
+    configure_graylog,
+    install_graylog,
+    refresh_graylog,
     set_conf,
     set_jvm_heap_size,
+    update_config,
+    upgrade_charm,
     _check_input_exists)  # noqa: E402
 from files import check_graylog_health  # noqa: E402
 
+
+class TestInstallUpdateUpgrade(unittest.TestCase):
+    """Test that install / update / upgrade clears our configured flag."""
+
+    @mock.patch('reactive.graylog.snap')
+    @mock.patch('reactive.graylog.hookenv.config')
+    def test_install(self, mock_config, mock_snap):
+        class Config(dict):
+            def __init__(self, *args, **kw):
+                super(Config, self).__init__(*args, **kw)
+
+        mock_config.return_value = Config({'channel': '2/stable'})
+        install_graylog()
+        self.assertTrue(mock_snap.install.called)
+        self.assertFalse(is_state('graylog.configured'))
+
+    def test_update(self):
+        set_state('graylog.configured')
+        update_config()
+        self.assertFalse(is_state('graylog.configured'))
+
+    def test_upgrade(self):
+        set_state('graylog.configured')
+        upgrade_charm()
+        self.assertFalse(is_state('graylog.configured'))
+
+
+class TestRefresh(unittest.TestCase):
+    """Test the snap refresh handler."""
+
+    @mock.patch('reactive.graylog.snap')
+    @mock.patch('reactive.graylog.hookenv.config')
+    def test_no_channel(self, mock_config, mock_snap):
+        """Ensure a no-op if there is no 'channel' config."""
+        mock_config.return_value = ''
+        mock_snap.get_installed_version.return_value = '2.foo'
+        set_state('graylog.configured')
+        refresh_graylog()
+        self.assertFalse(mock_snap.refresh.called)
+        self.assertTrue(is_state('graylog.configured'))
+
+    @mock.patch('reactive.graylog.shutil')
+    @mock.patch('reactive.graylog.os')
+    @mock.patch('reactive.graylog.host')
+    @mock.patch('reactive.graylog.snap')
+    @mock.patch('reactive.graylog.hookenv.config')
+    def test_no_change(self, mock_config, mock_snap, mock_host, mock_os, mock_shutil):
+        """Ensure no reconfiguration if the snap version does not change."""
+        mock_config.return_value = 'foo'
+        mock_snap.get_installed_version.side_effect = ['2.foo', '2.foo']
+        mock_os.path.exists.return_value = True
+        mock_shutil.copy2.return_value = 'foo'
+        set_state('graylog.configured')
+        refresh_graylog()
+        self.assertTrue(mock_snap.refresh.called)
+        self.assertTrue(is_state('graylog.configured'))
+
+    @mock.patch('reactive.graylog.data_changed')
+    @mock.patch('reactive.graylog.shutil.copy2')
+    @mock.patch('reactive.graylog.os')
+    @mock.patch('reactive.graylog.host')
+    @mock.patch('reactive.graylog.snap')
+    @mock.patch('reactive.graylog.hookenv.config')
+    def test_change(self, mock_config, mock_snap, mock_host, mock_os, mock_shutil, mock_data_changed):
+        """Ensure calls and states when the snap has changed."""
+        mock_config.return_value = 'foo'
+        mock_snap.get_installed_version.side_effect = ['2.foo', '3.foo']
+        mock_os.path.exists.return_value = True
+        mock_shutil.return_value = 'foo'
+        set_state('graylog.configured')
+        refresh_graylog()
+        self.assertTrue(mock_host.service_stop.called)
+        self.assertTrue(mock_snap.refresh.called)
+        self.assertEqual(mock_shutil.call_count, 2)
+        self.assertEqual(mock_data_changed.call_count, 2)
+        self.assertFalse(is_state('graylog.configured'))
+
+
+class TestConfig(unittest.TestCase):
+    """Test the configure_graylog handler."""
+
+    @mock.patch('reactive.graylog.hookenv')
+    @mock.patch('reactive.graylog.os')
+    def test_no_conf_file(self, mock_os, mock_hookenv):
+        """Ensure a status message if there is no config file present."""
+        mock_os.path.exists.return_value = False
+        configure_graylog()
+        self.assertTrue(mock_hookenv.status_set.called)
+
+    @mock.patch('reactive.graylog.set_jvm_heap_size')
+    @mock.patch('reactive.graylog.validate_api_uri')
+    @mock.patch('reactive.graylog.is_v2')
+    @mock.patch('reactive.graylog.set_conf')
+    @mock.patch('reactive.graylog.hookenv')
+    @mock.patch('reactive.graylog.os')
+    @mock.patch('reactive.graylog.unitdata')
+    def test_v2_config(self, mock_ud, mock_os, mock_hookenv, mock_set_conf, mock_v2, mock_validate_uri, mock_jvm_heap):
+        """Ensure calls and states with various config values for v2."""
+        class Config(dict):
+            def __init__(self, *args, **kw):
+                super(Config, self).__init__(*args, **kw)
+
+        mock_ud.kv.return_value = {'admin_password': 'password', 'password_secret': 'secret'}
+        mock_os.path.exists.return_value = True
+        mock_hookenv.config.return_value = Config({'jvm_heap_size': '1G',
+                                                   'web_listen_uri': None,
+                                                   'rest_transport_uri': 'foo/api',
+                                                   'web_endpoint_uri': 'bar/api'})
+        mock_v2.return_value = True
+        configure_graylog()
+        self.assertEqual(mock_validate_uri.call_count, 2)
+        self.assertTrue(mock_jvm_heap.called)
+        self.assertTrue(mock_hookenv.application_version_set.called)
+        self.assertTrue(mock_hookenv.open_port.called)
+
+    @mock.patch('reactive.graylog.set_jvm_heap_size')
+    @mock.patch('reactive.graylog.validate_api_uri')
+    @mock.patch('reactive.graylog.is_v2')
+    @mock.patch('reactive.graylog.set_conf')
+    @mock.patch('reactive.graylog.hookenv')
+    @mock.patch('reactive.graylog.os')
+    @mock.patch('reactive.graylog.unitdata')
+    def test_v3_config(self, mock_ud, mock_os, mock_hookenv, mock_set_conf, mock_v2, mock_validate_uri, mock_jvm_heap):
+        """Ensure calls and states with various config values for v3."""
+        class Config(dict):
+            def __init__(self, *args, **kw):
+                super(Config, self).__init__(*args, **kw)
+
+        mock_ud.kv.return_value = {'admin_password': 'password', 'password_secret': 'secret'}
+        mock_os.path.exists.return_value = True
+        mock_hookenv.config.return_value = Config({'jvm_heap_size': '1G',
+                                                   'web_listen_uri': None,
+                                                   'rest_transport_uri': 'foo/api',
+                                                   'web_endpoint_uri': 'bar/api'})
+        mock_v2.return_value = False
+        configure_graylog()
+        self.assertEqual(mock_validate_uri.call_count, 2)
+        self.assertTrue(mock_jvm_heap.called)
+        self.assertTrue(mock_hookenv.application_version_set.called)
+        self.assertTrue(mock_hookenv.open_port.called)
+
+
 initial_conf = u"""#key1 = value1
 key2 = value2
 key2=value3
diff --git a/unit_tests/test_lib.py b/unit_tests/test_lib.py
new file mode 100644
index 0000000..565ffc1
--- /dev/null
+++ b/unit_tests/test_lib.py
@@ -0,0 +1,36 @@
+import unittest
+from unittest import mock
+
+from charms.layer.graylog import (
+    get_api_port,
+    get_api_url,
+    validate_api_uri,
+)
+
+
+class TestLibraryUtils(unittest.TestCase):
+    """Test lib.charms.layer.graylog utils."""
+
+    @mock.patch('charms.layer.graylog.snap.get_installed_version')
+    def test_api(self, mock_snap):
+        """Validate the api port/url match what we expect for v2 and v3."""
+        mock_snap.return_value = '2.foo'
+        self.assertEqual(get_api_port(), '9001')
+        self.assertEqual(get_api_url(), 'http://127.0.0.1:9001/api/')
+
+        mock_snap.return_value = '3.bar'
+        self.assertEqual(get_api_port(), '9000')
+        self.assertEqual(get_api_url(), 'http://127.0.0.1:9000/api/')
+
+    @mock.patch('charms.layer.graylog.snap.get_installed_version')
+    def test_validate_api_url(self, mock_snap):
+        """Validate the URI suffix is valid for v2 and v3."""
+        mock_snap.return_value = '2.foo'
+        self.assertEqual(validate_api_uri('foo'), 'foo/api/')
+        self.assertEqual(validate_api_uri('foo/'), 'foo/api/')
+        self.assertEqual(validate_api_uri('foo/api'), 'foo/api/')
+
+        mock_snap.return_value = '3.bar'
+        self.assertEqual(validate_api_uri('foo'), 'foo/')
+        self.assertEqual(validate_api_uri('foo/'), 'foo/')
+        self.assertEqual(validate_api_uri('foo/api/'), 'foo/')
diff --git a/unit_tests/test_logextract.py b/unit_tests/test_logextract.py
index d3ea072..5bae816 100644
--- a/unit_tests/test_logextract.py
+++ b/unit_tests/test_logextract.py
@@ -1,7 +1,7 @@
 import unittest
 from unittest import mock
 
-from charms.layer.graylog import logextract
+from charms.layer.graylog import GraylogPipelines, GraylogRules, GraylogStreams, LogExtractPipeline
 
 
 testdata_simple_pipeline_source = '\npipeline "testpipe"\nstage 1 match either\n  rule "test-rule";\nend\n'
@@ -47,14 +47,14 @@ class TestGraylogPipelines(unittest.TestCase):
     @mock.patch('charms.layer.graylog.api.GraylogApi')
     def test_get_pipeline_for(self, mock_api):
         mock_api.request.return_value = testdata_simple_pipeline
-        gpipes = logextract.GraylogPipelines(mock_api)
+        gpipes = GraylogPipelines(mock_api)
         test = gpipes.get_for_title('juju-logextract-pipeline')
         self.assertEqual(test['title'], 'juju-logextract-pipeline')
 
     @mock.patch('charms.layer.graylog.api.GraylogApi')
     def test_set_pipeline(self, mock_api):
         mock_api.request.return_value = testdata_simple_pipeline
-        gpipes = logextract.GraylogPipelines(mock_api)
+        gpipes = GraylogPipelines(mock_api)
         gpipes.set('juju-logextract-pipeline', testdata_simple_pipeline_source)
         self.assertEqual(mock_api.request.call_args_list[1][1]['data']['title'], 'juju-logextract-pipeline')
 
@@ -64,7 +64,7 @@ class TestGraylogRules(unittest.TestCase):
     @mock.patch('charms.layer.graylog.api.GraylogApi')
     def test_get_rules(self, mock_api):
         mock_api.request.return_value = testdata_rules
-        grules = logextract.GraylogRules(mock_api)
+        grules = GraylogRules(mock_api)
         rules = grules.get()
         self.assertEqual(rules[0]['title'], "from ceph mon")
 
@@ -74,7 +74,7 @@ class TestGraylogStreams(unittest.TestCase):
     @mock.patch('charms.layer.graylog.api.GraylogApi')
     def test_get_stream_id_for(self, mock_api):
         mock_api.request.return_value = testdata_streams
-        gstreams = logextract.GraylogStreams(mock_api)
+        gstreams = GraylogStreams(mock_api)
         allmsgs = gstreams.get_stream_id_for("All messages")
         self.assertEqual(allmsgs, testdata_streams['streams'][0]['id'])
         nope = gstreams.get_stream_id_for("I don't exist")
@@ -88,13 +88,13 @@ class TestLogExtractPipeline(unittest.TestCase):
         mock_api.request.return_value = testdata_simple_pipeline
 
     def test_parse_rule_title(self):
-        logext = logextract.LogExtractPipeline()
+        logext = LogExtractPipeline()
         t = logext.parse_rule_title(testdata_rules[0]['source'])
         self.assertEqual(t, 'from ceph mon')
 
     @mock.patch('charms.layer.graylog.api.GraylogApi')
     def test_set_rules(self, mock_api):
-        logext = logextract.LogExtractPipeline(mock_api)
+        logext = LogExtractPipeline(mock_api)
         logext.set_rules([testdata_rules[0]['source']])
         exp = [mock.call('/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/pipeline')]
         mock_api.request.assert_has_calls(exp)
diff --git a/wheelhouse.txt b/wheelhouse.txt
new file mode 100644
index 0000000..d555101
--- /dev/null
+++ b/wheelhouse.txt
@@ -0,0 +1 @@
+requests>=2.0.0,<3.0.0