Merge ~martin-kalcok/juju-verify:ceph-mon into ~canonical-solutions-engineering/juju-verify:master

[Martin Kalcok]

Add verifier for ceph-mon

This change adds a CephMon verifier for the ceph-mon charm. It uses
existing actions to ensure the cluster has HEALTH_OK status.

The change relies on ceph-mon get-quorum-status action, introduced in
https://review.opendev.org/c/openstack/charm-ceph-mon/+/778837.

Also uses Juju changes in 2.8.10, and introduces a dependency on that
version.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Unable to determine commit message from repository - please click "Set commit message" and enter the commit message manually.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:afd975b31203e8b1e122843d43ef1743c39025f7
https://jenkins.canonical.com/bootstack/job/juju-verify-test/58/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/generic-make-test/79/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/58/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/juju-verify-test/58//rebuild

review: Approve (continuous-integration)

[Martin Kalcok (martin-kalcok)]

Code-wise this change is LGTM, thanks. I just have one in-line suggestion that I think is worth considering.

General question: I looked through the documentation and I see that odd number of monitors is not necessary for quorum, but it's strongly recommended [1]. Do you think it would be worth it to add warning output (via logger) that would warn user that the action would result in sub-optimal configuration of ceph-mon? (i.e. having 2 mons is not much better than 1, as it provides no fault tolerance).

[1] https://docs.ceph.com/en/latest/rados/operations/add-or-rm-mons/

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:f12707b94e2e68b7c633d71358225b63e331898b
https://jenkins.canonical.com/bootstack/job/juju-verify-test/63/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/generic-make-test/84/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/63/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/juju-verify-test/63//rebuild

review: Approve (continuous-integration)

[Xav Paice (xavpaice)]

This will want a rebase and functest update once the functest change is finalized.

[Robert Gildein (rgildein)]

Overall, it looks good, the only problem is using "\n".

[Robert Gildein (rgildein)]

review: Needs Fixing

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:8b3228b1e672d71954f180f649bd1511f75feb6e
https://jenkins.canonical.com/bootstack/job/lp-juju-verify-ci/39/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-full/31/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/161/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-verify-ci/39//rebuild

review: Approve (continuous-integration)

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:15f5c2d8f5bf0e3a50ee73234ad57a7dcfbd92ad
https://jenkins.canonical.com/bootstack/job/lp-juju-verify-ci/41/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-full/33/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/163/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-verify-ci/41//rebuild

review: Approve (continuous-integration)

[Alvaro Uria (aluria)]

Overall, change LGTM. I've added a suggestion to avoid multiple calls to the "get-health" action. Besides, some documentation could be introduced underlying what is the expected status of ceph-mons to give the OK (i.e. what is checked).

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:27201e23b8239e2fb10f1f841f3bd235334f3583
https://jenkins.canonical.com/bootstack/job/lp-juju-verify-ci/43/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-full/35/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/171/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-verify-ci/43//rebuild

review: Approve (continuous-integration)

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:ca7393f1f2db3c85c6914f7e2f029281dd4e9642
https://jenkins.canonical.com/bootstack/job/lp-juju-verify-ci/44/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-full/36/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/172/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-verify-ci/44//rebuild

review: Approve (continuous-integration)

[Robert Gildein (rgildein)]

LGTM, only two nitpicks and one suggestion for pylint ignore.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

A CI job is currently in progress. A follow up comment will be added when it completes.

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:fbbac866c9ade5d961ce40c9c0a8fb4d0ad671af
https://jenkins.canonical.com/bootstack/job/lp-juju-verify-ci/49/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-full/42/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/182/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-verify-ci/49//rebuild

review: Approve (continuous-integration)

[Robert Gildein (rgildein)]

LGTM

review: Approve

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Change successfully merged at revision e3a7cfa95e66b493ad11a1a165b8d65fbe389ea7

diff --git a/docs/verifiers/ceph-mon.rst b/docs/verifiers/ceph-mon.rst
new file mode 100644
index 0000000..43b7cab
--- /dev/null
+++ b/docs/verifiers/ceph-mon.rst
@@ -0,0 +1,33 @@
+Ceph-mon verifier
+=================
+
+So far, the "reboot" and "shutdown" actions are supported and they both
+perform the same set of checks.
+
+* check minimum Juju version
+* check Ceph Monitor quorum
+* check Ceph clusters health
+
+
+check Juju version
+------------------
+
+Ceph-mon verification relies on Juju features introduced in 2.8.10. If this minimum
+version requirement is not met, the verification will stop and return ``Failed`` result
+immediately.
+
+check Ceph Monitor quorum
+-------------------------
+
+This check verifies that intended action won't remove more than half of monitors in each
+affected ceph-mon cluster. Majority of Ceph monitors must be kept alive after the change
+to maintain quorum.
+
+check Ceph clusters health
+--------------------------
+
+This check runs ``get-health`` action on one of the targeted ``ceph-mon`` units to get
+cluster's health. If targeted units belong to multiple Juju applications, ``get-health``
+action is run on one unit per application. A cluster is considered healthy if the
+action's output contains HEALTH_OK. All affected clusters must be healthy for
+verification to succeed.
diff --git a/juju_verify/verifiers/__init__.py b/juju_verify/verifiers/__init__.py
index a7a518b..36233c0 100644
--- a/juju_verify/verifiers/__init__.py
+++ b/juju_verify/verifiers/__init__.py
@@ -25,7 +25,7 @@ from juju.unit import Unit
 from juju_verify.exceptions import CharmException
 from juju_verify.utils.unit import parse_charm_name
 from juju_verify.verifiers.base import BaseVerifier
-from juju_verify.verifiers.ceph import CephOsd
+from juju_verify.verifiers.ceph import CephOsd, CephMon
 from juju_verify.verifiers.nova_compute import NovaCompute
 from juju_verify.verifiers.result import Result, Severity
 
@@ -35,6 +35,7 @@ logger = logging.getLogger(__name__)
 SUPPORTED_CHARMS = {
     'nova-compute': NovaCompute,
     'ceph-osd': CephOsd,
+    'ceph-mon': CephMon,
 }
 
 
diff --git a/juju_verify/verifiers/ceph.py b/juju_verify/verifiers/ceph.py
index 4258125..1c5b56e 100644
--- a/juju_verify/verifiers/ceph.py
+++ b/juju_verify/verifiers/ceph.py
@@ -21,6 +21,7 @@ from collections import defaultdict
 from typing import Dict, Optional, Any, List
 
 from juju.unit import Unit
+from packaging.version import Version, InvalidVersion
 
 from juju_verify.utils.action import data_from_action
 from juju_verify.utils.unit import (
@@ -31,6 +32,7 @@ from juju_verify.utils.unit import (
 )
 from juju_verify.verifiers.base import BaseVerifier
 from juju_verify.verifiers.result import aggregate_results, Result, Severity
+from juju_verify.exceptions import CharmException
 
 logger = logging.getLogger()
 
@@ -99,6 +101,7 @@ class CephCommon(BaseVerifier):  # pylint: disable=W0223
         verify_charm_unit("ceph-mon", *units)
         result = Result()
         action_map = run_action_on_units(list(units), "get-health")
+
         for unit, action in action_map.items():
             cluster_health = data_from_action(action, "message")
             logger.debug("Unit (%s): Ceph cluster health '%s'", unit, cluster_health)
@@ -178,7 +181,7 @@ class CephCommon(BaseVerifier):  # pylint: disable=W0223
 class CephOsd(CephCommon):
     """Implementation of verification checks for the ceph-osd charm."""
 
-    NAME = 'ceph-osd'
+    NAME = "ceph-osd"
 
     def __init__(self, units: List[Unit]):
         """Ceph-osd charm verifier."""
@@ -333,3 +336,80 @@ class CephOsd(CephCommon):
     def verify_shutdown(self) -> Result:
         """Verify that it's safe to shutdown selected ceph-osd units."""
         return self.verify_reboot()
+
+
+class CephMon(CephCommon):
+    """Implementation of verification checks for the ceph-mon charm."""
+
+    NAME = "ceph-mon"
+
+    def verify_reboot(self) -> Result:
+        """Verify that it's safe to reboot selected ceph-mon units."""
+        version_check = self.check_version()
+        if not version_check.success:
+            return version_check
+
+        # Get one ceph-mon unit per each application
+        app_map = {unit.application: unit for unit in self.units}
+        unique_app_units = app_map.values()
+
+        return aggregate_results(
+            version_check,
+            self.check_quorum(),
+            self.check_cluster_health(*unique_app_units)
+        )
+
+    def verify_shutdown(self) -> Result:
+        """Verify that it's safe to shutdown selected units."""
+        return self.verify_reboot()
+
+    def check_quorum(self) -> Result:
+        """Check that the shutdown does not result in <50% mons alive."""
+        result = Result()
+
+        action_name = "get-quorum-status"
+        action_results = self.run_action_on_all(action_name)
+
+        affected_hosts = {unit.machine.hostname for unit in self.units}
+
+        for unit_id, action in action_results.items():
+            # run this per unit because we might have multiple clusters
+            known_mons = set(json.loads(data_from_action(action, "known-mons")))
+            online_mons = set(json.loads(data_from_action(action, "online-mons")))
+
+            mon_count = len(known_mons)
+            mons_after_change = len(online_mons - affected_hosts)
+
+            if mons_after_change <= mon_count // 2:
+                result.add_partial_result(Severity.FAIL, f"Removing unit {unit_id} will"
+                                                         f" lose Ceph mon quorum")
+
+        if result.empty:
+            result.add_partial_result(Severity.OK, 'Ceph-mon quorum check passed.')
+
+        return result
+
+    def check_version(self) -> Result:
+        """Check minimum required version of juju agents on units.
+
+        Ceph-mon verifier requires that all the units run juju agent >=2.8.10 due to
+        reliance on juju.Machine.hostname feature.
+        """
+        min_version = Version('2.8.10')
+        result = Result()
+        for unit in self.units:
+            juju_version = unit.safe_data.get('agent-status', {}).get('version', '')
+            try:
+                if Version(juju_version) < min_version:
+                    fail_msg = (f'Juju agent on unit {unit.entity_id} has lower than '
+                                f'minumum required version. {juju_version} < '
+                                f'{min_version}')
+                    result.add_partial_result(Severity.FAIL, fail_msg)
+            except InvalidVersion as exc:
+                raise CharmException(f'Failed to parse juju version from '
+                                     f'unit {unit.entity_id}.') from exc
+
+        if result.empty:
+            result.add_partial_result(Severity.OK, 'Minimum juju version check passed.')
+
+        return result
diff --git a/setup.cfg b/setup.cfg
index 1eb1e8e..b117068 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -25,7 +25,8 @@ include_package_data = True
 python_requires = >=3.6
 packages = find:
 install_requires =
-    juju
+    juju >= 2.8.6
+    packaging
 
 [options.extras_require]
 dev =
diff --git a/tests/functional/requirements.txt b/tests/functional/requirements.txt
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/tests/functional/requirements.txt
diff --git a/tests/functional/tests/bundles/ceph.yaml b/tests/functional/tests/bundles/ceph.yaml
index 94ea2d8..b29b301 100644
--- a/tests/functional/tests/bundles/ceph.yaml
+++ b/tests/functional/tests/bundles/ceph.yaml
@@ -1,7 +1,7 @@
 series: bionic
 applications:
   ceph-mon:
-    charm: cs:~rgildein/ceph-mon-2
+    charm: cs:~rgildein/ceph-mon-3
     num_units: 3
     options:
       expected-osd-count: 3
diff --git a/tests/functional/tests/bundles/hacluster.yaml b/tests/functional/tests/bundles/hacluster.yaml
index ec0bf13..95ce5e9 100644
--- a/tests/functional/tests/bundles/hacluster.yaml
+++ b/tests/functional/tests/bundles/hacluster.yaml
@@ -1,7 +1,7 @@
 series: focal
 applications:
   dashboard-hacluster:
-    charm: cs:~xavpaice/hacluster-0
+    charm: cs:~xavpaice/hacluster
   keystone:
     charm: cs:keystone
     num_units: 1
diff --git a/tests/functional/tests/test_ceph.py b/tests/functional/tests/test_ceph.py
index 4fce49e..0e84632 100644
--- a/tests/functional/tests/test_ceph.py
+++ b/tests/functional/tests/test_ceph.py
@@ -1,6 +1,7 @@
 """Test deployment and functionality of juju-verify."""
 
 from juju import loop
+import zaza.model
 from tests.base import BaseTestCase
 
 from juju_verify import juju_verify
@@ -24,3 +25,31 @@ class CephTests(BaseTestCase):
         verifier = get_verifier(unit_objects)
         result = verifier.verify(check)
         self.assertTrue(result.success is True)
+
+    def test_single_mon_unit(self):
+        """Test that shutdown of a single mon unit returns OK."""
+        # juju-verify shutdown --units ceph-mon/0
+
+        units = ['ceph-mon/0']
+        log_level = 'info'
+        check = 'shutdown'
+        juju_verify.config_logger(log_level)
+        model = loop.run(juju_verify.connect_model(zaza.model.CURRENT_MODEL))
+        unit_objects = loop.run(juju_verify.find_units(model, units))
+        verifier = get_verifier(unit_objects)
+        result = verifier.verify(check)
+        self.assertTrue(result.success is True)
+
+    def test_two_mon_unit(self):
+        """Test that shutdown of multiple mon units fails."""
+        # juju-verify shutdown --units ceph-mon/0 ceph-mon/1
+
+        units = ['ceph-mon/0', 'ceph-mon/1']
+        log_level = 'info'
+        check = 'shutdown'
+        juju_verify.config_logger(log_level)
+        model = loop.run(juju_verify.connect_model(zaza.model.CURRENT_MODEL))
+        unit_objects = loop.run(juju_verify.find_units(model, units))
+        verifier = get_verifier(unit_objects)
+        result = verifier.verify(check)
+        self.assertTrue(result.success is False)
diff --git a/tests/unit/verifiers/test_ceph.py b/tests/unit/verifiers/test_ceph.py
index 7d539ca..cf4a87e 100644
--- a/tests/unit/verifiers/test_ceph.py
+++ b/tests/unit/verifiers/test_ceph.py
@@ -17,15 +17,23 @@
 """CephOsd verifier class test suite."""
 import json
 from unittest import mock
-from unittest.mock import MagicMock
+from unittest.mock import MagicMock, PropertyMock
 
 import pytest
+from juju.model import Model
+from juju.unit import Unit
 
 from juju_verify.exceptions import CharmException
-from juju_verify.verifiers.ceph import AvailabilityZone, CephCommon, CephOsd
+from juju_verify.verifiers.ceph import AvailabilityZone, CephCommon, CephOsd, CephMon
 from juju_verify.verifiers.result import Result, Severity
 
 
+CEPH_MON_QUORUM_OK = "Ceph-mon quorum check passed."
+CEPH_MON_QUORUM_FAIL = "Removing unit {} will lose Ceph mon quorum"
+JUJU_VERSION_ERR = ("The machine for unit {} does not have a hostname attribute, "
+                    "please ensure that Juju controller is 2.8.10 or higher.")
+
+
 @pytest.mark.parametrize("az_info, ex_az", [
     ({"root": "default", "host": "test"}, "root=default,host=test"),
     ({"root": "default", "rack": "nova", "row": "row1", "host": "test"},
@@ -365,3 +373,149 @@ def test_verify_shutdown(
     mock_check_ceph_cluster_health.assert_called_once_with()
     mock_check_replication_number.assert_called_once_with()
     mock_check_availability_zone.assert_called_once_with()
+
+
[e-mail](
+    "action_return_value, severity, msg, hostname",
+    [('["host0", "host1", "host2"]', Severity.OK, CEPH_MON_QUORUM_OK, "host1"),
+     ('["host0", "host1"]', Severity.FAIL, CEPH_MON_QUORUM_FAIL, "host1"),
+     ('["host0", "host1", "host2"]', Severity.OK, CEPH_MON_QUORUM_OK, "host5"),
+     ],
+)
+def test_check_ceph_mon_quorum(mocker, action_return_value, severity, msg, hostname):
+    """Test Ceph quorum verification on CephMon."""
+    unit_to_remove = 'ceph-mon/0'
+    unit = Unit(unit_to_remove, Model())
+    verifier = CephMon([unit])
+    mocker.patch.object(verifier, "run_action_on_all").return_value = {
+        unit_to_remove: action_return_value,
+    }
+    unit.machine = mock.Mock(hostname=hostname)
+    mocker.patch(
+        "juju_verify.verifiers.ceph.data_from_action"
+    ).return_value = action_return_value
+    expected_msg = msg if severity == Severity.OK else msg.format(unit_to_remove)
+    expected_result = Result(severity, expected_msg)
+    result = verifier.check_quorum()
+    assert result == expected_result
+
+
[e-mail]('juju_version, expected_severity', [
+    pytest.param('2.8.0', Severity.FAIL, id='version-low'),
+    pytest.param('2.8.10', Severity.OK, id='version-match'),
+    pytest.param('2.8.11', Severity.OK, id='version-high'),
+])
+def test_ceph_mon_version_check(juju_version, expected_severity, mocker):
+    """Test expected results of ceph-mon juju version check."""
+    mock_unit_data = {'agent-status': {'version': juju_version}}
+    mocker.patch.object(Unit, 'safe_data',
+                        new_callable=PropertyMock(return_value=mock_unit_data))
+    unit_name = 'ceph-mon/0'
+    if expected_severity == Severity.OK:
+        expected_result = Result(Severity.OK, 'Minimum juju version check passed.')
+    else:
+        fail_msg = (f'Juju agent on unit {unit_name} has lower than '
+                    f'minumum required version. {juju_version} < 2.8.10')
+        expected_result = Result(Severity.FAIL, fail_msg)
+
+    verifier = CephMon([Unit(unit_name, Model())])
+    result = verifier.check_version()
+
+    assert result == expected_result
+
+
+def test_ceph_mon_fail_version_parsing(mocker):
+    """Test that exception is raised if parsing of juju version fails."""
+    bogus_version = 'foo'
+    unit_name = 'ceph-mon/0'
+    mock_unit_data = {'agent-status': {'version': bogus_version}}
+    mocker.patch.object(Unit, 'safe_data',
+                        new_callable=PropertyMock(return_value=mock_unit_data))
+
+    verifier = CephMon([Unit(unit_name, Model())])
+
+    with pytest.raises(CharmException) as exc:
+        _ = verifier.check_version()
+
+    assert str(exc.value) == f'Failed to parse juju version from unit {unit_name}.'
+
+
+def test_verify_ceph_mon_shutdown(mocker):
+    """Test that verify_shutdown links to verify_reboot."""
+    mocker.patch.object(CephMon, "verify_reboot")
+    unit = Unit("ceph-mon/0", Model())
+    verifier = CephMon([unit])
+    verifier.verify_shutdown()
+    verifier.verify_reboot.assert_called_once()
+
+
[e-mail]("juju_verify.verifiers.ceph.CephCommon.check_cluster_health")
[e-mail]("juju_verify.verifiers.ceph.CephMon.check_quorum")
[e-mail]("juju_verify.verifiers.ceph.CephMon.check_version")
+def test_verify_ceph_mon_reboot(mock_version, mock_quorum, mock_health):
+    """Test reboot verification on CephMon."""
+    unit = Unit("ceph-mon/0", Model())
+    mock_health.return_value = Result(Severity.OK, "Ceph cluster is healthy")
+    mock_quorum.return_value = Result(Severity.OK, "Ceph-mon quorum check passed.")
+    mock_version.return_value = Result(Severity.OK,
+                                       "Minimum juju version check passed.")
+    verifier = CephMon([unit])
+    result = verifier.verify_reboot()
+
+    expected_result = Result()
+    expected_result.add_partial_result(Severity.OK,
+                                       "Minimum juju version check passed.")
+    expected_result.add_partial_result(Severity.OK, 'Ceph-mon quorum check passed.')
+    expected_result.add_partial_result(Severity.OK, 'Ceph cluster is healthy')
+
+    assert result == expected_result
+
+
[e-mail]("juju_verify.verifiers.ceph.CephCommon.check_cluster_health")
[e-mail]("juju_verify.verifiers.ceph.CephMon.check_quorum")
[e-mail]("juju_verify.verifiers.ceph.CephMon.check_version")
+def test_verify_ceph_mon_reboot_stops_on_failed_version(mock_version, mock_quorum,
+                                                        mock_health):
+    """Test that if ceph-mon version check fails, not other checks are performed."""
+    expected_result = Result(Severity.FAIL, 'version too low')
+    mock_version.return_value = expected_result
+
+    verifier = CephMon([Unit('ceph-mon/0', Model())])
+    result = verifier.verify_reboot()
+
+    assert result == expected_result
+    mock_version.assert_called_once()
+    mock_quorum.assert_not_called()
+    mock_health.assert_not_called()
+
+
[e-mail]("juju_verify.verifiers.ceph.CephCommon.check_cluster_health")
[e-mail]("juju_verify.verifiers.ceph.CephMon.check_quorum")
[e-mail]("juju_verify.verifiers.ceph.CephMon.check_version")
+def test_verify_ceph_mon_reboot_checks_health_once(mock_version, mock_quorum,
+                                                   mock_health, mocker):
+    """Test that ceph-mon verification runs 'health check' only once per application."""
+    model = Model()
+    app_1 = [Unit('ceph-mon/0', model), Unit('ceph-mon/1', model)]
+    app_2 = [Unit('ceph-mon-extra/0', model), Unit('ceph-mon-extra/1', model)]
+    expected_health_check_units = [app_1[-1], app_2[-1]]
+
+    for unit in app_1:
+        mocker.patch.object(unit, 'data',
+                            new_callable=PropertyMock(
+                                return_value={'application': 'ceph-mon'})
+                            )
+
+    for unit in app_2:
+        mocker.patch.object(unit, 'data',
+                            new_callable=PropertyMock(
+                                return_value={'application': 'ceph-mon-extra'})
+                            )
+
+    verifier = CephMon(app_1 + app_2)
+    result = verifier.verify_reboot()
+
+    assert result.success
+    mock_version.assert_called_once()
+    mock_quorum.assert_called_once()
+    mock_health.assert_called_with(*expected_health_check_units)
diff --git a/tox.ini b/tox.ini
index e58986e..1138ca2 100644
--- a/tox.ini
+++ b/tox.ini
@@ -34,7 +34,7 @@ commands =
     flake8 {toxinidir}/juju_verify/ {toxinidir}/tests/
     mypy {toxinidir}/juju_verify/
     pylint {toxinidir}/juju_verify/
-    pylint {toxinidir}/tests/ --disable=E1101,R0913,R0914,W0212
+    pylint {toxinidir}/tests/ --disable=E1101,R0913,R0914,W0212,R0801
 
 [testenv:build]
 basepython = python3