Merge ~gabrielcocenza/juju-lint:bug/1988851 into juju-lint:master

[Gabriel Cocenza]

check path_mtu values depending on the networking solution (ovs, ovn)

- added the possibility of adding custom messages and log level for
  config checks (isset, eq, neq, gte and search)

Closes-Bug: #1988851

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

[Eric Chen (eric-chen)]

Do we need the function to adding custom messages in this PR? it seems an additional feature to me.

review: Needs Information

[Eric Chen (eric-chen)]

Leave similar quiestion inline

review: Needs Information

[Gabriel Cocenza (gabrielcocenza)]

Yes, it's actually two new features: ability of controlling log-level and custom message on every application config.

I think it's a good versatility to have this option of customizing messages and control the log-level on every application configuration. Let's get the neutron-api configuration of path-mtu for example:

- If we can't insert a custom message, the user would receive an error message saying:
Application neutron-api has config for path-mtu which is bigger than 1550: 2000
This standard message doesn't say much what actually this value can cause

The problem with this is:
- having a value bigger than 1550 it's not considered "wrong" depending on the networking solution, so it makes more sense send a warn message explaining what can happen if the value is bigger.

I can see this customization been useful on every "operator". Maybe using isset to False in a configuration is opinionated and not necessarily wrong.

pastebin of tests (lint, unit, func) https://pastebin.canonical.com/p/QZ6BT8HZxW/

[Nobuto Murata (nobuto)]

review: Needs Fixing

[Gabriel Cocenza (gabrielcocenza)]

Thanks for your review Nobuto.

I've updated the patch including your suggestions.
Here[0] is the logs of unit, lint and functional tests.

[0] https://pastebin.canonical.com/p/tbCSh5Hs8t/

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:f33c606a7fb298ccabfd9b14b73cfb711499ef0a
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/2/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-unit/303/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/1461/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/2//rebuild

review: Approve (continuous-integration)

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

FAILED: Continuous integration, rev:dcf5cae98379b519dce348c4a2fedac97ea672c2
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/5/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/bootstack/job/lp-charm-test-functest/1073/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/1492/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/5//rebuild

review: Needs Fixing (continuous-integration)

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

FAILED: Continuous integration, rev:dcf5cae98379b519dce348c4a2fedac97ea672c2
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/6/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/bootstack/job/lp-charm-test-functest/1076/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/1493/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/6//rebuild

review: Needs Fixing (continuous-integration)

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

FAILED: Continuous integration, rev:dcf5cae98379b519dce348c4a2fedac97ea672c2
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/7/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/bootstack/job/lp-charm-test-functest/1077/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/1494/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/7//rebuild

review: Needs Fixing (continuous-integration)

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

FAILED: Continuous integration, rev:dcf5cae98379b519dce348c4a2fedac97ea672c2
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/8/
Executed test runs:
    FAILURE: https://jenkins.canonical.com/bootstack/job/lp-charm-test-functest/1078/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/1495/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/8//rebuild

review: Needs Fixing (continuous-integration)

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:dcf5cae98379b519dce348c4a2fedac97ea672c2
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/9/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-functest/1079/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/1496/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/9//rebuild

review: Approve (continuous-integration)

[Nobuto Murata (nobuto)]

Left a nit-picking comment, other than that it looks good to me.

review: Approve

[🤖 prod-jenkaas-bootstack (prod-jenkaas-bootstack)]

PASSED: Continuous integration, rev:d520551fb9eb9060f7c7b00ce6688bfe9aa61958
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/10/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/bootstack/job/lp-charm-test-functest/1085/
    None: https://jenkins.canonical.com/bootstack/job/lp-update-mp/1501/

Click here to trigger a rebuild:
https://jenkins.canonical.com/bootstack/job/lp-juju-lint-ci/10//rebuild

review: Approve (continuous-integration)

[Eric Chen (eric-chen)]

review: Approve

[🤖 Canonical IS Merge Bot (canonical-is-mergebot)]

Change successfully merged at revision 3684a5c1f7e315832720e859f0e4a78c757e7d9a

diff --git a/contrib/includes/networking/ovn.yaml b/contrib/includes/networking/ovn.yaml
index b4f4140..bba04d5 100644
--- a/contrib/includes/networking/ovn.yaml
+++ b/contrib/includes/networking/ovn.yaml
@@ -1,9 +1,17 @@
 openstack config networking: &openstack-config-networking
- ovn-central:
-  ovsdb-server-election-timer:
-   gte: 4
+  ovn-central:
+    ovsdb-server-election-timer:
+      gte: 4
+  neutron-api:
+    path-mtu:
+      gte: 1558
+      log-level: warning
+      custom-message: >
+          Usable MTU for a tenant network is less than 1500.
+    global-physnet-mtu:
+      eq: 9000
 
 openstack mandatory charms networking: &openstack-mandatory-charms-networking
- - ovn-central
- - ovn-chassis
- - neutron-api-plugin-ovn
+  - ovn-central
+  - ovn-chassis
+  - neutron-api-plugin-ovn
diff --git a/contrib/includes/networking/ovs.yaml b/contrib/includes/networking/ovs.yaml
index 493195e..4cba606 100644
--- a/contrib/includes/networking/ovs.yaml
+++ b/contrib/includes/networking/ovs.yaml
@@ -1,4 +1,12 @@
-openstack config networking: &openstack-config-networking {}
+openstack config networking: &openstack-config-networking
+  neutron-api:
+      path-mtu:
+        gte: 1550
+        log-level: warning
+        custom-message: >
+          Usable MTU for a tenant network is less than 1500.
+      global-physnet-mtu:
+        eq: 9000
 
 openstack mandatory charms networking: &openstack-mandatory-charms-networking
- - neutron-openvswitch
+  - neutron-openvswitch
diff --git a/contrib/includes/openstack.yaml b/contrib/includes/openstack.yaml
index 01317e2..86af4da 100644
--- a/contrib/includes/openstack.yaml
+++ b/contrib/includes/openstack.yaml
@@ -14,11 +14,6 @@ openstack config base: &openstack-config-base
   glance:
     ceph-osd-replication-count:
       eq: 3
-  neutron-api:
-    path-mtu:
-      eq: 9000
-    global-physnet-mtu:
-      eq: 9000
   nova-compute:
     cpu-model:
       neq: ""
diff --git a/jujulint/lint.py b/jujulint/lint.py
index 8bbcff4..b8fa535 100755
--- a/jujulint/lint.py
+++ b/jujulint/lint.py
@@ -40,8 +40,14 @@ from jujulint.model_input import input_handler
 from jujulint.relations import RelationError, RelationsRulesBootStrap
 
 VALID_CONFIG_CHECKS = ("isset", "eq", "neq", "gte", "search")
-
-# Generic named tuple to represent the binary config operators (eq,neq,gte)
+VALID_LOG_LEVEL = {
+    "debug": logging.DEBUG,
+    "info": logging.INFO,
+    "warning": logging.WARNING,
+    "error": logging.ERROR,
+}
+
+# Generic named tuple to represent the binary config operators (eq, neq, gte)
 ConfigOperator = collections.namedtuple(
     "ConfigOperator", "name repr check error_template"
 )
@@ -212,7 +218,15 @@ class Linter:
 
         return _int * conv[suffix.lower()]
 
-    def isset(self, name, check_value, rule, config):
+    def isset(
+        self,
+        name,
+        check_value,
+        rule,
+        config,
+        log_level=logging.ERROR,
+        custom_message="",
+    ):
         """Check if value is set per rule constraints."""
         if rule in config:
             if check_value is True:
@@ -225,7 +239,7 @@ class Linter:
                 )
                 return True
             actual_value = config[rule]
-            self.handle_error(
+            self.message_handler(
                 {
                     "id": "config-isset-check-false",
                     "tags": ["config", "isset"],
@@ -233,10 +247,12 @@ class Linter:
                     "application": name,
                     "rule": rule,
                     "actual_value": actual_value,
-                    "message": "Application {} has config for {}: {}.".format(
+                    "message": custom_message
+                    or "Application {} has config for {}: {}.".format(
                         name, rule, actual_value
                     ),
-                }
+                },
+                log_level=log_level,
             )
             return False
         elif check_value is False:
@@ -247,19 +263,29 @@ class Linter:
                 )
             )
             return True
-        self.handle_error(
+        self.message_handler(
             {
                 "id": "config-isset-check-true",
                 "tags": ["config", "isset"],
                 "description": "Checks for config condition 'isset' true",
                 "application": name,
                 "rule": rule,
-                "message": "Application {} has no config for {}.".format(name, rule),
-            }
+                "message": custom_message
+                or "Application {} has no config for {}.".format(name, rule),
+            },
+            log_level=log_level,
         )
         return False
 
-    def eq(self, app_name, check_value, config_key, app_config):
+    def eq(
+        self,
+        app_name,
+        check_value,
+        config_key,
+        app_config,
+        log_level=logging.ERROR,
+        custom_message="",
+    ):
         """Check if value matches the provided value or regex, autodetecting regex."""
         operator = ConfigOperator(
             name="eq",
@@ -269,10 +295,24 @@ class Linter:
         )
 
         return self.check_config_generic(
-            operator, app_name, check_value, config_key, app_config
+            operator,
+            app_name,
+            check_value,
+            config_key,
+            app_config,
+            log_level,
+            custom_message,
         )
 
-    def neq(self, app_name, check_value, config_key, app_config):
+    def neq(
+        self,
+        app_name,
+        check_value,
+        config_key,
+        app_config,
+        log_level=logging.ERROR,
+        custom_message="",
+    ):
         """Check if value does not match a the config."""
         operator = ConfigOperator(
             name="neq",
@@ -282,10 +322,24 @@ class Linter:
         )
 
         return self.check_config_generic(
-            operator, app_name, check_value, config_key, app_config
+            operator,
+            app_name,
+            check_value,
+            config_key,
+            app_config,
+            log_level,
+            custom_message,
         )
 
-    def gte(self, app_name, check_value, config_key, app_config):
+    def gte(
+        self,
+        app_name,
+        check_value,
+        config_key,
+        app_config,
+        log_level=logging.ERROR,
+        custom_message="",
+    ):
         """Check if value is greater than or equal to the check value."""
 
         def operator_gte_check(check_value, actual_value):
@@ -302,10 +356,24 @@ class Linter:
         )
 
         return self.check_config_generic(
-            operator, app_name, check_value, config_key, app_config
+            operator,
+            app_name,
+            check_value,
+            config_key,
+            app_config,
+            log_level,
+            custom_message,
         )
 
-    def search(self, app_name, check_value, config_key, app_config):
+    def search(
+        self,
+        app_name,
+        check_value,
+        config_key,
+        app_config,
+        log_level=logging.ERROR,
+        custom_message="",
+    ):
         """Scan through the charm config looking for a match using the regex pattern."""
         if config_key in app_config:
             actual_value = app_config.get(config_key)
@@ -319,7 +387,7 @@ class Linter:
                     )
                 )
                 return True
-            self.handle_error(
+            self.message_handler(
                 {
                     "id": "config-search-check",
                     "tags": ["config", "search"],
@@ -328,13 +396,15 @@ class Linter:
                     "rule": config_key,
                     "expected_value": check_value,
                     "actual_value": actual_value,
-                    "message": "Application {} has an invalid config for '{}': regex {} not found at {}".format(
+                    "message": custom_message
+                    or "Application {} has an invalid config for '{}': regex {} not found at {}".format(
                         app_name,
                         config_key,
                         repr(check_value),
                         repr(actual_value),
                     ),
-                }
+                },
+                log_level=log_level,
             )
             return False
 
@@ -349,7 +419,14 @@ class Linter:
         return False
 
     def check_config_generic(
-        self, operator, app_name, check_value, config_key, app_config
+        self,
+        operator,
+        app_name,
+        check_value,
+        config_key,
+        app_config,
+        log_level=logging.ERROR,
+        custom_message="",
     ):
         """Apply the provided config operator to the configuration."""
         # First check if the config key is present
@@ -380,7 +457,7 @@ class Linter:
             )
             return True
         else:
-            self.handle_error(
+            self.message_handler(
                 {
                     "id": "config-{}-check".format(operator.name),
                     "tags": ["config", operator.name],
@@ -391,13 +468,15 @@ class Linter:
                     "rule": config_key,
                     "expected_value": check_value,
                     "actual_value": actual_value,
-                    "message": operator.error_template.format(
+                    "message": custom_message
+                    or operator.error_template.format(
                         app_name,
                         config_key,
                         repr(check_value),
                         repr(actual_value),
                     ),
-                }
+                },
+                log_level=log_level,
             )
         return False
 
@@ -428,11 +507,21 @@ class Linter:
                     )
                     continue
 
+            custom_message = rules[rule].pop("custom-message", "")
+            log_level = rules[rule].pop("log-level", "error").lower()
+
             for check_op, check_value in rules[rule].items():
                 # check_op should be the operator name, e.g. (eq, neq, gte, isset)
                 if check_op in VALID_CONFIG_CHECKS:
                     check_method = getattr(self, check_op)
-                    check_method(app_name, check_value, rule, config)
+                    check_method(
+                        app_name,
+                        check_value,
+                        rule,
+                        config,
+                        VALID_LOG_LEVEL.get(log_level, logging.ERROR),
+                        custom_message,
+                    )
                 else:
                     self._log_with_header(
                         "Application {} has unknown check operation for {}: {}.".format(
@@ -635,7 +724,7 @@ class Linter:
         for rule in relations_rules:
             for endpoint, applications in rule.missing_relations.items():
                 if applications:
-                    self.handle_error(
+                    self.message_handler(
                         {
                             "id": "missing-relations",
                             "tags": ["relation", "missing"],
@@ -646,7 +735,7 @@ class Linter:
                     )
             if rule.not_exist_error:
                 for relation in rule.not_exist_error:
-                    self.handle_error(
+                    self.message_handler(
                         {
                             "id": "relation-exist",
                             "tags": ["relation", "exist"],
@@ -657,7 +746,7 @@ class Linter:
                     )
 
             if rule.missing_machines:
-                self.handle_error(
+                self.message_handler(
                     {
                         "id": "missing-machine",
                         "tags": ["missing", "machine"],
@@ -697,7 +786,7 @@ class Linter:
         """Check we recognise the charms which are in the model."""
         for charm in self.model.charms:
             if charm not in self.lint_rules["known charms"]:
-                self.handle_error(
+                self.message_handler(
                     {
                         "id": "unrecognised-charm",
                         "tags": ["charm", "unrecognised"],
@@ -710,11 +799,11 @@ class Linter:
         for charm in self.lint_rules["operations mandatory"]:
             error = self.check_charms_ops_mandatory(charm)
             if error:
-                self.handle_error(error)
+                self.message_handler(error)
         if self.cloud_type == "openstack":
             for charm in self.lint_rules["openstack mandatory"]:
                 if charm not in self.model.charms:
-                    self.handle_error(
+                    self.message_handler(
                         {
                             "id": "openstack-charm-missing",
                             "tags": [
@@ -731,7 +820,7 @@ class Linter:
                     )
             for charm in self.lint_rules["operations openstack mandatory"]:
                 if charm not in self.model.charms:
-                    self.handle_error(
+                    self.message_handler(
                         {
                             "id": "openstack-ops-charm-missing",
                             "tags": [
@@ -752,7 +841,7 @@ class Linter:
         elif self.cloud_type == "kubernetes":
             for charm in self.lint_rules["kubernetes mandatory"]:
                 if charm not in self.model.charms:
-                    self.handle_error(
+                    self.message_handler(
                         {
                             "id": "kubernetes-charm-missing",
                             "tags": [
@@ -769,7 +858,7 @@ class Linter:
                     )
             for charm in self.lint_rules["operations kubernetes mandatory"]:
                 if charm not in self.model.charms:
-                    self.handle_error(
+                    self.message_handler(
                         {
                             "id": "kubernetes-ops-charm-missing",
                             "tags": [
@@ -895,7 +984,7 @@ class Linter:
 
         message = "Space binding mismatch: {}".format(mismatch)
         if error:
-            self.handle_error(
+            self.message_handler(
                 {
                     "id": "space-binding-mismatch",
                     "tags": ["mismatch", "space", "binding"],
@@ -912,7 +1001,7 @@ class Linter:
         if self.model.missing_subs:
             for sub in self.model.missing_subs:
                 principals = ", ".join(sorted(self.model.missing_subs[sub]))
-                self.handle_error(
+                self.message_handler(
                     {
                         "id": "ops-subordinate-missing",
                         "tags": ["missing", "ops", "charm", "mandatory", "subordinate"],
@@ -928,7 +1017,7 @@ class Linter:
         if self.model.extraneous_subs:
             for sub in self.model.extraneous_subs:
                 principals = ", ".join(sorted(self.model.extraneous_subs[sub]))
-                self.handle_error(
+                self.message_handler(
                     {
                         "id": "subordinate-extraneous",
                         "tags": ["extraneous", "charm", "subordinate"],
@@ -943,7 +1032,7 @@ class Linter:
         if self.model.duelling_subs:
             for sub in self.model.duelling_subs:
                 machines = ", ".join(sorted(self.model.duelling_subs[sub]))
-                self.handle_error(
+                self.message_handler(
                     {
                         "id": "subordinate-duplicate",
                         "tags": ["duplicate", "charm", "subordinate"],
@@ -962,7 +1051,7 @@ class Linter:
                 az_map = ", ".join(
                     ["{}: {}".format(az, az_counter[az]) for az in sorted(az_counter)]
                 )
-                self.handle_error(
+                self.message_handler(
                     {
                         "id": "AZ-unbalance",
                         "tags": ["AZ"],
@@ -987,7 +1076,7 @@ class Linter:
                 self.model.app_to_charm[app] = charm_name
                 self.model.charm_to_app.setdefault(charm_name, set()).add(app)
             else:
-                self.handle_error(
+                self.message_handler(
                     {
                         "id": "charm-not-mapped",
                         "tags": ["charm", "mapped", "parsing"],
@@ -1060,7 +1149,7 @@ class Linter:
                 return
 
             status_msg = status.get("message")
-            self.handle_error(
+            self.message_handler(
                 {
                     "id": "status-unexpected",
                     "tags": ["status"],
@@ -1162,7 +1251,7 @@ class Linter:
             azs.add(self.model.machines_to_az[machine])
         num_azs = len(azs)
         if num_azs != 3:
-            self.handle_error(
+            self.message_handler(
                 {
                     "id": "AZ-invalid-number",
                     "tags": ["AZ"],
@@ -1306,15 +1395,20 @@ class Linter:
                 "Model contains no applications, skipping.", level=logging.WARN
             )
 
-    def collect(self, error):
+    def collect(self, message):
         """Collect an error and add it to the collector."""
-        self.output_collector["errors"].append(error)
+        self.output_collector["errors"].append(message)
 
-    def handle_error(self, error):
-        """Collect an error and add it to the collector."""
-        self._log_with_header(error["message"], level=logging.ERROR)
-        if self.collect_errors:
-            self.collect(error)
+    def message_handler(self, message, log_level=logging.ERROR):
+        """Handle message from checks and append error messages to the collector."""
+        if "message" not in message:
+            log_level = logging.ERROR
+            message = {"message": "wrong message_handler format"}
+
+        self._log_with_header(message["message"], level=log_level)
+
+        if self.collect_errors and log_level == logging.ERROR:
+            self.collect(message)
 
     def _process_includes_in_rules(self, yaml_txt):
         """
diff --git a/tests/unit/test_jujulint.py b/tests/unit/test_jujulint.py
index 264b15e..77dbb5e 100644
--- a/tests/unit/test_jujulint.py
+++ b/tests/unit/test_jujulint.py
@@ -8,6 +8,7 @@ import pytest
 import yaml
 
 from jujulint import check_spaces, lint, relations
+from jujulint.lint import VALID_LOG_LEVEL
 
 
 class TestUtils:
@@ -536,22 +537,79 @@ applications:
         assert errors[0]["id"] == "kubernetes-ops-charm-missing"
         assert errors[0]["charm"] == "ntp"
 
-    def test_config_eq(self, linter, juju_status):
-        """Test the config condition 'eq'."""
+    @pytest.mark.parametrize(
+        "custom_message, log_level, generate_error",
+        [
+            ("", "", True),  # generate error, no custom message
+            ("custom message", "", True),  # generate error with custom message
+            ("", "foo", True),  # generate error, no custom message
+            (
+                "custom message",
+                "warning",
+                False,
+            ),  # doesn't generate error with custom message
+            (
+                "custom message",
+                "WARNING",
+                False,
+            ),  # doesn't generate error with custom message
+            ("", "info", False),  # doesn't generate error, no custom message
+            ("", "debug", False),  # doesn't generate error, no custom message
+        ],
+    )
+    def test_config_eq(
+        self, linter, juju_status, custom_message, log_level, generate_error
+    ):
+        """Test the config condition 'eq' with custom messages and log levels."""
         linter.lint_rules["config"] = {"ubuntu": {"fake-opt": {"eq": False}}}
         juju_status["applications"]["ubuntu"]["options"] = {"fake-opt": True}
+        if custom_message:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"][
+                "custom-message"
+            ] = custom_message
+        if log_level:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"]["log-level"] = log_level
+
         linter.do_lint(juju_status)
 
         errors = linter.output_collector["errors"]
-        assert len(errors) == 1
-        assert errors[0]["id"] == "config-eq-check"
-        assert errors[0]["application"] == "ubuntu"
-        assert errors[0]["rule"] == "fake-opt"
-        assert errors[0]["expected_value"] is False
-        assert errors[0]["actual_value"] is True
 
-    def test_config_eq_suffix_match(self, linter, juju_status):
-        """Test the config condition 'eq'. when suffix matches."""
+        if log_level.lower() != "error" and log_level.lower() in VALID_LOG_LEVEL:
+            assert not generate_error
+            assert len(errors) == 0
+
+        else:
+            assert len(errors) == 1
+            assert errors[0]["id"] == "config-eq-check"
+            assert errors[0]["application"] == "ubuntu"
+            assert errors[0]["rule"] == "fake-opt"
+            assert errors[0]["expected_value"] is False
+            assert errors[0]["actual_value"] is True
+
+    @pytest.mark.parametrize(
+        "custom_message, log_level, generate_error",
+        [
+            ("", "", True),  # generate error, no custom message
+            ("custom message", "", True),  # generate error with custom message
+            ("", "foo", True),  # generate error, no custom message
+            (
+                "custom message",
+                "warning",
+                False,
+            ),  # doesn't generate error with custom message
+            (
+                "custom message",
+                "WARNING",
+                False,
+            ),  # doesn't generate error with custom message
+            ("", "info", False),  # doesn't generate error, no custom message
+            ("", "debug", False),  # doesn't generate error, no custom message
+        ],
+    )
+    def test_config_eq_suffix_match(
+        self, linter, juju_status, custom_message, log_level, generate_error
+    ):
+        """Test the config condition 'eq'. when suffix matches with custom messages and log levels."""
         linter.lint_rules["config"] = {
             "ubuntu": {"fake-opt": {"eq": False, "suffixes": ["host", "physical"]}}
         }
@@ -559,31 +617,86 @@ applications:
         juju_status["applications"]["ubuntu-host"] = juju_status["applications"].pop(
             "ubuntu"
         )
+        if custom_message:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"][
+                "custom-message"
+            ] = custom_message
+        if log_level:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"]["log-level"] = log_level
+
         linter.do_lint(juju_status)
 
         errors = linter.output_collector["errors"]
-        assert len(errors) == 1
-        assert errors[0]["id"] == "config-eq-check"
-        assert errors[0]["application"] == "ubuntu-host"
-        assert errors[0]["rule"] == "fake-opt"
-        assert errors[0]["expected_value"] is False
-        assert errors[0]["actual_value"] is True
 
-    def test_config_eq_suffix_match_charm_name(self, linter, juju_status):
+        if log_level.lower() != "error" and log_level.lower() in VALID_LOG_LEVEL:
+            assert not generate_error
+            assert len(errors) == 0
+        else:
+            assert len(errors) == 1
+            assert errors[0]["id"] == "config-eq-check"
+            assert errors[0]["application"] == "ubuntu-host"
+            assert errors[0]["rule"] == "fake-opt"
+            assert errors[0]["expected_value"] is False
+            assert errors[0]["actual_value"] is True
+            if custom_message:
+                assert errors[0]["message"] == custom_message
+            else:
+                assert errors[0]["message"] != custom_message
+
+    @pytest.mark.parametrize(
+        "custom_message, log_level, generate_error",
+        [
+            ("", "", True),  # generate error, no custom message
+            ("custom message", "", True),  # generate error with custom message
+            ("", "foo", True),  # generate error, no custom message
+            (
+                "custom message",
+                "warning",
+                False,
+            ),  # doesn't generate error with custom message
+            (
+                "custom message",
+                "WARNING",
+                False,
+            ),  # doesn't generate error with custom message
+            ("", "info", False),  # doesn't generate error, no custom message
+            ("", "debug", False),  # doesn't generate error, no custom message
+        ],
+    )
+    def test_config_eq_suffix_match_charm_name(
+        self, linter, juju_status, custom_message, log_level, generate_error
+    ):
         """Test the config condition 'eq'. when suffix and base charm name."""
         linter.lint_rules["config"] = {
             "ubuntu": {"fake-opt": {"eq": False, "suffixes": ["host", "physical"]}}
         }
         juju_status["applications"]["ubuntu"]["options"] = {"fake-opt": True}
+        if custom_message:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"][
+                "custom-message"
+            ] = custom_message
+        if log_level:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"]["log-level"] = log_level
+
         linter.do_lint(juju_status)
 
         errors = linter.output_collector["errors"]
-        assert len(errors) == 1
-        assert errors[0]["id"] == "config-eq-check"
-        assert errors[0]["application"] == "ubuntu"
-        assert errors[0]["rule"] == "fake-opt"
-        assert errors[0]["expected_value"] is False
-        assert errors[0]["actual_value"] is True
+
+        if log_level.lower() != "error" and log_level.lower() in VALID_LOG_LEVEL:
+            assert not generate_error
+            assert len(errors) == 0
+
+        else:
+            assert len(errors) == 1
+            assert errors[0]["id"] == "config-eq-check"
+            assert errors[0]["application"] == "ubuntu"
+            assert errors[0]["rule"] == "fake-opt"
+            assert errors[0]["expected_value"] is False
+            assert errors[0]["actual_value"] is True
+            if custom_message:
+                assert errors[0]["message"] == custom_message
+            else:
+                assert errors[0]["message"] != custom_message
 
     def test_config_eq_suffix_skip(self, linter, juju_status):
         """Test the config condition 'eq'. when suffix doesn't match."""
@@ -599,22 +712,61 @@ applications:
         errors = linter.output_collector["errors"]
         assert not errors
 
-    def test_config_eq_no_suffix_check_all(self, linter, juju_status):
+    @pytest.mark.parametrize(
+        "custom_message, log_level, generate_error",
+        [
+            ("", "", True),  # generate error, no custom message
+            ("custom message", "", True),  # generate error with custom message
+            ("", "foo", True),  # generate error, no custom message
+            (
+                "custom message",
+                "warning",
+                False,
+            ),  # doesn't generate error with custom message
+            (
+                "custom message",
+                "WARNING",
+                False,
+            ),  # doesn't generate error with custom message
+            ("", "info", False),  # doesn't generate error, no custom message
+            ("", "debug", False),  # doesn't generate error, no custom message
+        ],
+    )
+    def test_config_eq_no_suffix_check_all(
+        self, linter, juju_status, custom_message, log_level, generate_error
+    ):
         """Test the config condition 'eq'. when no suffix all should be checked."""
         linter.lint_rules["config"] = {"ubuntu": {"fake-opt": {"eq": False}}}
         juju_status["applications"]["ubuntu"]["options"] = {"fake-opt": True}
         juju_status["applications"]["ubuntu-host"] = juju_status["applications"].pop(
             "ubuntu"
         )
+        if custom_message:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"][
+                "custom-message"
+            ] = custom_message
+        if log_level:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"]["log-level"] = log_level
+
         linter.do_lint(juju_status)
 
         errors = linter.output_collector["errors"]
-        assert len(errors) == 1
-        assert errors[0]["id"] == "config-eq-check"
-        assert errors[0]["application"] == "ubuntu-host"
-        assert errors[0]["rule"] == "fake-opt"
-        assert errors[0]["expected_value"] is False
-        assert errors[0]["actual_value"] is True
+
+        if log_level.lower() != "error" and log_level.lower() in VALID_LOG_LEVEL:
+            assert not generate_error
+            assert len(errors) == 0
+
+        else:
+            assert len(errors) == 1
+            assert errors[0]["id"] == "config-eq-check"
+            assert errors[0]["application"] == "ubuntu-host"
+            assert errors[0]["rule"] == "fake-opt"
+            assert errors[0]["expected_value"] is False
+            assert errors[0]["actual_value"] is True
+            if custom_message:
+                assert errors[0]["message"] == custom_message
+            else:
+                assert errors[0]["message"] != custom_message
 
     def test_config_neq_valid(self, linter, juju_status):
         """Test the config condition 'neq'."""
@@ -625,46 +777,164 @@ applications:
         errors = linter.output_collector["errors"]
         assert len(errors) == 0
 
-    def test_config_neq_invalid(self, linter, juju_status):
-        """Test the config condition 'neq', valid."""
+    @pytest.mark.parametrize(
+        "custom_message, log_level, generate_error",
+        [
+            ("", "", True),  # generate error, no custom message
+            ("custom message", "", True),  # generate error with custom message
+            ("", "foo", True),  # generate error, no custom message
+            (
+                "custom message",
+                "warning",
+                False,
+            ),  # doesn't generate error with custom message
+            (
+                "custom message",
+                "WARNING",
+                False,
+            ),  # doesn't generate error with custom message
+            ("", "info", False),  # doesn't generate error, no custom message
+            ("", "debug", False),  # doesn't generate error, no custom message
+        ],
+    )
+    def test_config_neq_invalid(
+        self, linter, juju_status, custom_message, log_level, generate_error
+    ):
+        """Test the config condition 'neq', invalid with custom messages and log levels."""
         linter.lint_rules["config"] = {"ubuntu": {"fake-opt": {"neq": ""}}}
         juju_status["applications"]["ubuntu"]["options"] = {"fake-opt": ""}
+        if custom_message:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"][
+                "custom-message"
+            ] = custom_message
+        if log_level:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"]["log-level"] = log_level
+
         linter.do_lint(juju_status)
 
         errors = linter.output_collector["errors"]
-        assert len(errors) == 1
-        assert errors[0]["id"] == "config-neq-check"
-        assert errors[0]["application"] == "ubuntu"
-        assert errors[0]["rule"] == "fake-opt"
-        assert errors[0]["expected_value"] == ""
-        assert errors[0]["actual_value"] == ""
 
-    def test_config_gte(self, linter, juju_status):
-        """Test the config condition 'gte'."""
+        if log_level.lower() != "error" and log_level.lower() in VALID_LOG_LEVEL:
+            assert not generate_error
+            assert len(errors) == 0
+
+        else:
+            assert len(errors) == 1
+            assert errors[0]["id"] == "config-neq-check"
+            assert errors[0]["application"] == "ubuntu"
+            assert errors[0]["rule"] == "fake-opt"
+            assert errors[0]["expected_value"] == ""
+            assert errors[0]["actual_value"] == ""
+            if custom_message:
+                assert errors[0]["message"] == custom_message
+            else:
+                assert errors[0]["message"] != custom_message
+
+    @pytest.mark.parametrize(
+        "custom_message, log_level, generate_error",
+        [
+            ("", "", True),  # generate error, no custom message
+            ("custom message", "", True),  # generate error with custom message
+            ("", "foo", True),  # generate error, no custom message
+            (
+                "custom message",
+                "warning",
+                False,
+            ),  # doesn't generate error with custom message
+            (
+                "custom message",
+                "WARNING",
+                False,
+            ),  # doesn't generate error with custom message
+            ("", "info", False),  # doesn't generate error, no custom message
+            ("", "debug", False),  # doesn't generate error, no custom message
+        ],
+    )
+    def test_config_gte(
+        self, linter, juju_status, custom_message, log_level, generate_error
+    ):
+        """Test the config condition 'gte' with custom messages and log levels."""
         linter.lint_rules["config"] = {"ubuntu": {"fake-opt": {"gte": 3}}}
         juju_status["applications"]["ubuntu"]["options"] = {"fake-opt": 0}
+        if custom_message:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"][
+                "custom-message"
+            ] = custom_message
+        if log_level:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"]["log-level"] = log_level
+
         linter.do_lint(juju_status)
 
         errors = linter.output_collector["errors"]
-        assert len(errors) == 1
-        assert errors[0]["id"] == "config-gte-check"
-        assert errors[0]["application"] == "ubuntu"
-        assert errors[0]["rule"] == "fake-opt"
-        assert errors[0]["expected_value"] == 3
-        assert errors[0]["actual_value"] == 0
 
-    def test_config_isset_false_fail(self, linter, juju_status):
-        """Test error handling if config condition 'isset'=false is not met."""
+        if log_level.lower() != "error" and log_level.lower() in VALID_LOG_LEVEL:
+            assert not generate_error
+            assert len(errors) == 0
+
+        else:
+            assert generate_error
+            assert len(errors) == 1
+            assert errors[0]["id"] == "config-gte-check"
+            assert errors[0]["application"] == "ubuntu"
+            assert errors[0]["rule"] == "fake-opt"
+            assert errors[0]["expected_value"] == 3
+            assert errors[0]["actual_value"] == 0
+            if custom_message:
+                assert errors[0]["message"] == custom_message
+            else:
+                assert errors[0]["message"] != custom_message
+
+    @pytest.mark.parametrize(
+        "custom_message, log_level, generate_error",
+        [
+            ("", "", True),  # generate error, no custom message
+            ("custom message", "", True),  # generate error with custom message
+            ("", "foo", True),  # generate error, no custom message
+            (
+                "custom message",
+                "warning",
+                False,
+            ),  # doesn't generate error with custom message
+            (
+                "custom message",
+                "WARNING",
+                False,
+            ),  # doesn't generate error with custom message
+            ("", "info", False),  # doesn't generate error, no custom message
+            ("", "debug", False),  # doesn't generate error, no custom message
+        ],
+    )
+    def test_config_isset_false_fail(
+        self, linter, juju_status, custom_message, log_level, generate_error
+    ):
+        """Test message handling if config condition 'isset'=false is not met."""
         linter.lint_rules["config"] = {"ubuntu": {"fake-opt": {"isset": False}}}
         juju_status["applications"]["ubuntu"]["options"] = {"fake-opt": 0}
+        if custom_message:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"][
+                "custom-message"
+            ] = custom_message
+        if log_level:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"]["log-level"] = log_level
+
         linter.do_lint(juju_status)
 
         errors = linter.output_collector["errors"]
-        assert len(errors) == 1
-        assert errors[0]["id"] == "config-isset-check-false"
-        assert errors[0]["application"] == "ubuntu"
-        assert errors[0]["rule"] == "fake-opt"
-        assert errors[0]["actual_value"] == 0
+
+        if log_level.lower() != "error" and log_level.lower() in VALID_LOG_LEVEL:
+            assert not generate_error
+            assert len(errors) == 0
+
+        else:
+            assert len(errors) == 1
+            assert errors[0]["id"] == "config-isset-check-false"
+            assert errors[0]["application"] == "ubuntu"
+            assert errors[0]["rule"] == "fake-opt"
+            assert errors[0]["actual_value"] == 0
+            if custom_message:
+                assert errors[0]["message"] == custom_message
+            else:
+                assert errors[0]["message"] != custom_message
 
     def test_config_isset_false_pass(self, linter, juju_status):
         """Test handling if config condition 'isset'=false is met."""
@@ -675,17 +945,56 @@ applications:
         errors = linter.output_collector["errors"]
         assert len(errors) == 0
 
-    def test_config_isset_true_fail(self, linter, juju_status):
-        """Test error handling if config condition 'isset'=true is not met."""
+    @pytest.mark.parametrize(
+        "custom_message, log_level, generate_error",
+        [
+            ("", "", True),  # generate error, no custom message
+            ("custom message", "", True),  # generate error with custom message
+            ("", "foo", True),  # generate error, no custom message
+            (
+                "custom message",
+                "warning",
+                False,
+            ),  # doesn't generate error with custom message
+            (
+                "custom message",
+                "WARNING",
+                False,
+            ),  # doesn't generate error with custom message
+            ("", "info", False),  # doesn't generate error, no custom message
+            ("", "debug", False),  # doesn't generate error, no custom message
+        ],
+    )
+    def test_config_isset_true_fail(
+        self, linter, juju_status, custom_message, log_level, generate_error
+    ):
+        """Test message handling if config condition 'isset'=true is not met."""
         linter.lint_rules["config"] = {"ubuntu": {"fake-opt": {"isset": True}}}
         juju_status["applications"]["ubuntu"]["options"] = {}
+        if custom_message:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"][
+                "custom-message"
+            ] = custom_message
+        if log_level:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"]["log-level"] = log_level
+
         linter.do_lint(juju_status)
 
         errors = linter.output_collector["errors"]
-        assert len(errors) == 1
-        assert errors[0]["id"] == "config-isset-check-true"
-        assert errors[0]["application"] == "ubuntu"
-        assert errors[0]["rule"] == "fake-opt"
+
+        if log_level.lower() != "error" and log_level.lower() in VALID_LOG_LEVEL:
+            assert not generate_error
+            assert len(errors) == 0
+
+        else:
+            assert len(errors) == 1
+            assert errors[0]["id"] == "config-isset-check-true"
+            assert errors[0]["application"] == "ubuntu"
+            assert errors[0]["rule"] == "fake-opt"
+            if custom_message:
+                assert errors[0]["message"] == custom_message
+            else:
+                assert errors[0]["message"] != custom_message
 
     def test_config_isset_true_pass(self, linter, juju_status):
         """Test handling if config condition 'isset'=true is met."""
@@ -709,23 +1018,60 @@ applications:
         errors = linter.output_collector["errors"]
         assert len(errors) == 0
 
-    def test_config_search_invalid(self, linter, juju_status):
+    @pytest.mark.parametrize(
+        "custom_message, log_level, generate_error",
+        [
+            ("", "", True),  # generate error, no custom message
+            ("custom message", "", True),  # generate error with custom message
+            ("", "foo", True),  # generate error, no custom message
+            (
+                "custom message",
+                "warning",
+                False,
+            ),  # doesn't generate error with custom message
+            (
+                "custom message",
+                "WARNING",
+                False,
+            ),  # doesn't generate error with custom message
+            ("", "info", False),  # doesn't generate error, no custom message
+            ("", "debug", False),  # doesn't generate error, no custom message
+        ],
+    )
+    def test_config_search_invalid(
+        self, linter, juju_status, custom_message, log_level, generate_error
+    ):
         """Test the config condition 'search' when invalid."""
         linter.lint_rules["config"] = {
-            "ubuntu": {"fake_opt": {"search": "\\W\\*, \\W\\*, 25000, 27500"}}
+            "ubuntu": {"fake-opt": {"search": "\\W\\*, \\W\\*, 25000, 27500"}}
         }
         juju_status["applications"]["ubuntu"]["options"] = {
-            "fake_opt": "[[/, queue1, 10, 20], [\\*, \\*, 10, 20]]"
+            "fake-opt": "[[/, queue1, 10, 20], [\\*, \\*, 10, 20]]"
         }
+        if custom_message:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"][
+                "custom-message"
+            ] = custom_message
+        if log_level:
+            linter.lint_rules["config"]["ubuntu"]["fake-opt"]["log-level"] = log_level
+
         linter.do_lint(juju_status)
 
         errors = linter.output_collector["errors"]
-        assert len(errors) == 1
-        assert errors[0]["id"] == "config-search-check"
-        assert errors[0]["application"] == "ubuntu"
-        assert errors[0]["rule"] == "fake_opt"
-        assert errors[0]["expected_value"] == "\\W\\*, \\W\\*, 25000, 27500"
-        assert errors[0]["actual_value"] == "[[/, queue1, 10, 20], [\\*, \\*, 10, 20]]"
+
+        if log_level.lower() != "error" and log_level.lower() in VALID_LOG_LEVEL:
+            assert not generate_error
+            assert len(errors) == 0
+
+        else:
+            assert len(errors) == 1
+            assert errors[0]["id"] == "config-search-check"
+            assert errors[0]["application"] == "ubuntu"
+            assert errors[0]["rule"] == "fake-opt"
+            assert errors[0]["expected_value"] == "\\W\\*, \\W\\*, 25000, 27500"
+            assert (
+                errors[0]["actual_value"] == "[[/, queue1, 10, 20], [\\*, \\*, 10, 20]]"
+            )
 
     def test_config_search_missing(self, linter, mocker):
         """Test the config search method logs warning if the config option is missing."""
@@ -1245,12 +1591,12 @@ applications:
     @pytest.mark.parametrize("input_file_type", ["juju-status", "juju-bundle"])
     def test_check_relations(self, linter, input_files, mocker, input_file_type):
         """Ensure that check_relation pass."""
-        mock_handle_error = mocker.patch("jujulint.lint.Linter.handle_error")
+        mock_message_handler = mocker.patch("jujulint.lint.Linter.message_handler")
         linter.lint_rules["relations"] = [
             {"charm": "nrpe", "check": [["nrpe:juju-info", "ubuntu:juju-info"]]}
         ]
         linter.check_relations(input_files[input_file_type])
-        mock_handle_error.assert_not_called()
+        mock_message_handler.assert_not_called()
 
     @pytest.mark.parametrize(
         "input_file_type",
@@ -1260,11 +1606,11 @@ applications:
         ],
     )
     def test_check_relations_exception_handling(
-        self, linter, juju_status, mocker, input_file_type, input_files
+        self, linter, mocker, input_file_type, input_files
     ):
         """Ensure that handle error if relation rules are in wrong format."""
         mock_log = mocker.patch("jujulint.lint.Linter._log_with_header")
-        mock_handle_error = mocker.patch("jujulint.lint.Linter.handle_error")
+        mock_message_handler = mocker.patch("jujulint.lint.Linter.message_handler")
         linter.lint_rules["relations"] = [
             {"charm": "ntp", "check": [["ntp", "ubuntu"]]}
         ]
@@ -1275,17 +1621,17 @@ applications:
         expected_exception = relations.RelationError(expected_msg)
 
         linter.check_relations(input_files[input_file_type])
-        mock_handle_error.assert_not_called()
+        mock_message_handler.assert_not_called()
         mock_log.assert_has_calls(
             [mocker.call(expected_exception.message, level=logging.ERROR)]
         )
 
     @pytest.mark.parametrize("input_file_type", ["juju-status", "juju-bundle"])
     def test_check_relations_missing_relations(
-        self, linter, juju_status, mocker, input_file_type, input_files
+        self, linter, mocker, input_file_type, input_files
     ):
         """Ensure that check_relation handle missing relations."""
-        mock_handle_error = mocker.patch("jujulint.lint.Linter.handle_error")
+        mock_message_handler = mocker.patch("jujulint.lint.Linter.message_handler")
         # add a relation rule that doesn't happen in the model
         linter.lint_rules["relations"] = [
             {
@@ -1294,7 +1640,7 @@ applications:
             }
         ]
         linter.check_relations(input_files[input_file_type])
-        mock_handle_error.assert_called_with(
+        mock_message_handler.assert_called_with(
             {
                 "id": "missing-relations",
                 "tags": ["relation", "missing"],
@@ -1307,7 +1653,7 @@ applications:
     @pytest.mark.parametrize("input_file_type", ["juju-status", "juju-bundle"])
     def test_check_relations_exist(self, linter, input_files, mocker, input_file_type):
         """Ensure that check_relation handle not exist error."""
-        mock_handle_error = mocker.patch("jujulint.lint.Linter.handle_error")
+        mock_message_handler = mocker.patch("jujulint.lint.Linter.message_handler")
         not_exist_relation = [
             "nrpe-host:nrpe-external-master",
             "elasticsearch:nrpe-external-master",
@@ -1317,7 +1663,7 @@ applications:
             {"charm": "nrpe", "not-exist": [not_exist_relation]}
         ]
         linter.check_relations(input_files[input_file_type])
-        mock_handle_error.assert_called_with(
+        mock_message_handler.assert_called_with(
             {
                 "id": "relation-exist",
                 "tags": ["relation", "exist"],
@@ -1338,7 +1684,7 @@ applications:
         input_file.machines_data.update(new_machines)
         # map file again
         input_file.map_file()
-        mock_handle_error = mocker.patch("jujulint.lint.Linter.handle_error")
+        mock_message_handler = mocker.patch("jujulint.lint.Linter.message_handler")
         linter.lint_rules["relations"] = [
             {
                 "charm": "nrpe",
@@ -1348,7 +1694,7 @@ applications:
 
         expected_missing_machines = ["2", "3"]
         linter.check_relations(input_file)
-        mock_handle_error.assert_called_with(
+        mock_message_handler.assert_called_with(
             {
                 "id": "missing-machine",
                 "tags": ["missing", "machine"],
@@ -1358,3 +1704,44 @@ applications:
                 ),
             }
         )
+
+    def test_path_mtu_for_ovs_ovn_rules(self, linter, rules_files):
+        """Test that ovs and ovn rules set right value of path-mtu."""
+        for rule in [rules_file for rules_file in rules_files if "fcb" in rules_file]:
+            linter.filename = rule
+            linter.read_rules()
+            path_mtu = linter.lint_rules["openstack config"]["neutron-api"]["path-mtu"]
+            if "ovs" in rule:
+                assert path_mtu["gte"] == 1550
+            else:
+                assert path_mtu["gte"] == 1558
+
+            assert path_mtu["log-level"] == "warning"
+            assert path_mtu["custom-message"]
+
+    @pytest.mark.parametrize(
+        "message, log_level, error",
+        [
+            ({}, None, True),  # log wrong message_handler format as error message
+            ({}, logging.INFO, True),  # log wrong message_handler format as error
+            ({"message": "my message"}, None, True),  # log error message
+            ({"message": "my message"}, logging.INFO, False),  # log info message
+            ({"message": "my message"}, logging.DEBUG, False),  # log debug message
+            ({"message": "my message"}, logging.WARNING, False),  # log warning message
+            ({"message": "my message"}, logging.ERROR, True),  # log error message
+        ],
+    )
+    def test_message_handler(self, linter, mocker, message, log_level, error):
+        """Test message_handler method."""
+        logger_mock = mocker.patch.object(linter, "_log_with_header")
+        expected_message = message.get("message", "wrong message_handler format")
+        if message and not error:
+            linter.message_handler(message, log_level)
+            logger_mock.assert_has_calls(
+                [mocker.call(expected_message, level=log_level)]
+            )
+        else:
+            linter.message_handler(message)
+            logger_mock.assert_has_calls(
+                [mocker.call(expected_message, level=logging.ERROR)]
+            )