snmpd fails to recognize Docker's overlay filesystem

Thanks for the steps to reproduce and link to the patch. It looks like this would affect kinetic and lunar as well, since the fix looks newer than the current 5.9.3 version included in those releases.

Hi István,

I've packaged the patch for lunar (Ubuntu 23.04) and posted to a PPA for testing:
    - https://launchpad.net/~bryce/+archive/ubuntu/net-snmp-sru-lp2007856

Currently lunar is in beta-freeze, but I think there'll be time to land this prior to the release, so we can get the fix in for the latest Ubuntu version.

Regarding SRU to older versions, there's some criteria to meet that I'm not certain this qualifies for but perhaps you can help. First, changes which enable features tend to raise red flags as far as the SRU approval goes, since even though this is just a one-line patch it enables a capability that itself might be buggy or might trigger bugs in other conditions that haven't been well tested. I'm not sure how to evaluate that criteria in this case, maybe the feature is sufficiently limited in scope it's a non-issue, however if it's a recent patch not widely tested that makes me concerned there could be undiscovered issues... The second important criteria is "impact", IOW how widespread is this problem and would it help a sufficiently large number of users to be worth the risk of adding the change? From your bug report it kind of sounds like the impact might be narrow in this case?

Anyway, I'd love to hear your thoughts on the above before we proceed with the SRU to past releases. This is a nice improvement that would be great to get out to users, but for SRU approval we'll need to ensure there's a strong case to make on the above points.

Meanwhile, I can continue the packaging work to get the patch available on the LTS releases via the PPA. That will at least ensure the fix is available for users wanting this, and may help on the testing front to verify the fix does work properly and reduce the regression concern levels.

Hi Bryce,

Thanks a lot for handling this issue!

I would be happy to have this backported to Jammy, so I don't have to maintain my own build of net-snmp. To be honest, I'm not sure it qualifies either.

Pros:
- This feature worked in an earlier version of net-snmp, so any other component that might encounter this behaviour should not be surprised.
- It could count as a fix, since it worked before, and now it doesn't.
- The code under the feature haven't changed much in the last few years, we're just reenabling it for overlay filesystems too.

Cons:
- The patch is very recent, I opened this bug days after I made and submitted the fix. There's no release yet that would contain the patch, and I guess no distros picked it up yet.
- I found no online mention of this issue at all, and it's been broken for several Ubuntu releases. I have to assume that either virtually nobody is using snmpd to query Docker container filesystem metrics, or they are still on 18.04 or similarly aged releases of other distributions. Both are equally possible.

I'm afraid I don't have a strong argument for an SRU. As I see, this fix impacts a very small number of users, is very small risk, and got very little testing.

Thanks for the quick reply István!

I've added builds to the PPA for kinetic and jammy. Hopefully you can make use of this for your needs. Ping this bug if the jammy build gets out of date and you want an update.

   https://launchpad.net/~bryce/+archive/ubuntu/net-snmp-sru-lp2007856

For focal, the patch looks a bit more involved to construct due to upstream's refactoring between 5.8 and 5.9.1, however offhand the fix still looks valid just will need to be coded up differently. Since at this time sounds like we wouldn't pursue SRU on this, I'll leave that backporting to a future task.

Thanks so much for your assessment, and I think that's a good take. What might swing things is if others run into this same problem, so let's leave this bug open and see if anyone else comments here as also affected, and then we'll re-evaluate.

Can someone please clarify what "monitoring inside a docker container fails" in the statement below:

"The new version did not recognize the /etc/mtab entry of "overlay" as a supported filesystem, so monitoring / inside a Docker container fails after Bionic (5.7.3), likely in every Ubuntu since Focal (5.8)."

What exactly fails? What is this scenario, is the docker container acting as a snmp server, and it won't start up? Or is it running snmpwalk against other snmp servers? Or is the snmpd server not listing the overlay filesystems in its filesystem status output, and that is just a missing piece of information, as all the others are? Or does it crash?

"Or is the snmpd server not listing the overlay filesystems in its filesystem status output, and that is just a missing piece of information, as all the others are?"

This is the one. An snmpd server running inside a Docker container won't see the filesystem related information, like the container root's free space for example.

Thanks for the feedback István, the proposed fix to Ubuntu lunar was approved and I've uploaded it. It should be available in lunar-proposed shortly, and if there are no migration issues should move from there to lunar-release. I'll follow up if there are migration problems.

I've also added a mention of this fix to the Ubuntu 23.04 Release Notes:
  - https://discourse.ubuntu.com/t/lunar-lobster-release-notes/31910

This bug was fixed in the package net-snmp - 5.9.3+dfsg-2ubuntu3

---------------
net-snmp (5.9.3+dfsg-2ubuntu3) lunar; urgency=medium

  * d/p/add-overlay-support.patch: Add Docker's "overlay" filesystem
    (LP: #2007856)

 -- Bryce Harrington <email address hidden> Wed, 22 Mar 2023 18:42:02 -0700