NRPE Charm

Neutron netns check CRIT/CRITICAL mismatch

Bug #2003402 reported by Barry Price on 2023-01-20

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	NRPE Charm	Won't Fix	Low	Unassigned
	OpenStack Neutron Gateway Charm	New	Undecided	Unassigned

Bug Description

The included check_netns.sh script for checking neutron gateway netns issues uses "CRIT" to report critical errors, but check_status_file.py expects "CRITICAL".

The former script is here, and writes its output to a file when called by cron, so the "exit $STATE_CRIT" never reaches the check, which can only read the contents of the log file:

https://git.launchpad.net/charm-nrpe/tree/files/plugins/check_netns.sh

An error state will be recorded in the log file as e.g. "CRIT: [...] aren't responding".

But the script used by the check, check_status_file.py, onlny looks for "CRITICAL" in the output, so doesn't match "CRIT":

https://git.launchpad.net/charm-nrpe/tree/files/plugins/check_status_file.py

Tags:

Related branches

~barryprice/charm-nrpe/+git/charm-nrpe:master

On hold for merging into charm-nrpe:master

🤖 prod-jenkaas-bootstack (community): Needs Fixing (continuous-integration) on 2023-01-20

BootStack Reviewers: Pending requested 2023-01-20

Revision history for this message

Barry Price (barryprice) wrote on 2023-01-20 (last edit on 2023-01-20):

Marking this as also affecting the neutron-gateway charm, as that's where I came across it.

If desired, this could be worked around in that charm's code with something like:

- check_cmd='check_status_file.py -f /var/lib/nagios/netns-check.txt'
+ check_cmd='check_status_file.py -c CRIT -f /var/lib/nagios/netns-check.txt'

https://opendev.org/openstack/charm-neutron-gateway/src/branch/master/hooks/neutron_hooks.py#L347

Happy to propose that fix if desired.

Revision history for this message

Barry Price (barryprice) wrote on 2023-01-20:

Actually coming around to the idea that this may be a bad check, I've found it in CRIT state on other clouds which appear to be working fine.

Have marked the linked MP as WIP while we investigate further, but it may be that a better fix is to either remove or update check_netns.sh rather than to trust its current output.

Eric Chen (eric-chen) on 2023-01-20

tags:

added: bseng-761

Revision history for this message

Barry Price (barryprice) wrote on 2023-01-22:

I don't think Launchpad allows blocking one bug against another, but I believe the linked bug here should be addressed before we decide what to do with this one:

https://bugs.launchpad.net/charm-nrpe/+bug/2003641

Eric Chen (eric-chen) on 2023-05-06