[Ubuntu 18.04][I2S][ethtool] -k options says 'Cannot get device udp-fragmentation-offload settings: Operation not supported'

Hi,
this is interesting.
Thanks for the ping.

Note: this needs no PPC or any other special arch, a VM guest is enough to test it.

Up until very recently we were still on kernel 4.13 where this did not occur.

With 4.13 - starting both off:
$ sudo ethtool -k ens3 | egrep 'udp-fragmentation-offload|generic-receive-offload'
  udp-fragmentation-offload: off
  generic-receive-offload: off
$ sudo ethtool -K ens3 gro on
$ sudo ethtool -k ens3 | egrep 'udp-fragmentation-offload|generic-receive-offload'
  udp-fragmentation-offload: off
  generic-receive-offload: on
$ sudo ethtool -K ens3 ufo on
$ sudo ethtool -k ens3 | egrep 'udp-fragmentation-offload|generic-receive-offload'
  udp-fragmentation-offload: on
  generic-receive-offload: on

So all was just fine.

ethtool release is rather tied to kernel version releases for just this reason.
But there was no update (yet) and we are on 4.11 for now.
Due to that the https://patchwork.ozlabs.org/patch/806563/ is missing still.

It is important that this is ONLY a warning and things work just fine.
Obviously setting UFO will fail, but setting GRO works (aside the annoying error messages).
So GRO is not functionally affected, it is just the older ethtool that tries to probe the related UFO and fails on that.

This is also the reason that upstream ONLY disabled the warning (but kept all else).

Therefore on 4.15 it behaves like this:
$ sudo ethtool -k ens3 | egrep 'udp-fragmentation-offload|generic-receive-offload'
  Cannot get device udp-fragmentation-offload settings: Operation not supported
  udp-fragmentation-offload: off
  generic-receive-offload: off
$ sudo ethtool -K ens3 gro on
  Cannot get device udp-fragmentation-offload settings: Operation not supported
  Cannot get device udp-fragmentation-offload settings: Operation not supported
$ echo $?
  0
$ sudo ethtool -k ens3 | egrep 'udp-fragmentation-offload|generic-receive-offload'
  Cannot get device udp-fragmentation-offload settings: Operation not supported
  udp-fragmentation-offload: off
  generic-receive-offload: on

So other than the warning it worked.
UFO doesn't of course.

$ sudo ethtool -K ens3 ufo on
  Cannot get device udp-fragmentation-offload settings: Operation not supported
  Cannot change udp-fragmentation-offload
  Cannot get device udp-fragmentation-offload settings: Operation not supported
  Could not change any device features
$ echo $?
  1
$ sudo ethtool -k ens3 | egrep 'udp-fragmentation-offload|generic-receive-offload'
  Cannot get device udp-fragmentation-offload settings: Operation not supported
  udp-fragmentation-offload: off
  generic-receive-offload: on

That said we have two options now:
1. we can pick the change from upstream and be good in regard to this bug.
   This can also be done post Feature Freeze as it is a bug fix.
2. We actually should - as we did with iproute2 recently - tie ethtool versions to kernel
   uploads. It is so dependent on the actual kernel, and might even more special for kernel
   backports, hwe kernels and so on.
   For example this will trigger on Xenial with a 4.15 HWE as well at some point.

I'll try to look into the doability of an ethtool 4.15 right now, but would like to ask the kernel ...

Ok, the merge was straight forward, no big issue of any kind.
It now looks like:

$ sudo ethtool -k ens3 | egrep 'udp-fragmentation-offload|generic-receive-offload'
  udp-fragmentation-offload: off
  generic-receive-offload: off
$ sudo ethtool -K ens3 gro on
$ sudo ethtool -k ens3 | egrep 'udp-fragmentation-offload|generic-receive-offload'
  udp-fragmentation-offload: off
  generic-receive-offload: on
$ sudo ethtool -K ens3 ufo on
  Cannot change udp-fragmentation-offload
  Could not change any device features

I asked the kernel Team (via mail) to take a look adopting that, especially in regard to HWE and backporting changes that might need an update here as well.

For now I'll test a bit more and push to Bionic later today (if nothing breaks).
FYI: Testing can be done from ppa https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3173

We had no Delta, the update is as said straight forward and works.
There are a few self-tests - all work on all architectures in the PPA build.

The message that was pointed out here can still show up if one really has no access (no sudo for example), but that is fine as it is correct in those cases.

Tested a few of the new features like --reset, so far all were fine (or my HW didn't support, but then it at least behaved correctly).

The openvswitch test suite has plenty of ethtool calls to test all sorts of features it provides.
Build time OVS tests in ppa: ok
Dep8 Tests against ppa: ok

If there are issue found later on we need to fix them, but to have ethtool 4.15 before FF is the right thing since we have kernel 4.15.

So pushing an upload to bionic now ...

See https://launchpad.net/ubuntu/+source/ethtool/1:4.15-0ubuntu1
Tracking migration on Monday ...

This bug was fixed in the package ethtool - 1:4.15-0ubuntu1

---------------
ethtool (1:4.15-0ubuntu1) bionic; urgency=medium

  * New upstream release 4.15 (LP: #1750989)

 -- Christian Ehrhardt <email address hidden> Fri, 23 Feb 2018 12:58:14 +0100

------- Comment From <email address hidden> 2018-06-19 00:30 EDT-------
With latest ethtool package the problem is fixed.

root@ltciofvtr-bostonlc1:~# dpkg -l | grep ethtool
ii ethtool 1:4.15-0ubuntu1 ppc64el display or change Ethernet device settings

root@ltciofvtr-bostonlc1:~# ethtool -k enP52p1s0f0
Features for enP52p1s0f0:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: on
tx-gre-csum-segmentation: on
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: on
tx-udp_tnl-csum-segmentation: on
tx-gso-partial: on
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: on
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: on [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: on
root@ltciofvtr-bostonlc1:~# ethtool -K enP52p1s0f0 gro on
root@ltciofvtr-bostonlc1:~# ethtool -K enP52p1s0f0 gro off

We can close this bug as fixed verified

Thanks for all your support !

I'm also seeing this on 16.04 systems with linux-generic-hwe-16.04 packaged kernel running. Will there be a ethtool update for 16.04 also?

There are a few issues with very kernel dependent tools like ethtool but also e.g. ip.
Some would want to get the newest version, especially to work better with HWE Kernels.
But clearly that would be a huge regression risk due to e.g. changed output breaking scripts.
There is no perfect solution yet, we discussed about an hwe-ethtool that conflicts with base ethtool and adding a suggest on the linux-hwe kernel.
But all that is currently only an open discussion and nothing is committed yet.

@smb - FYI this is one of the cases I meant when bringing up the discussion

@paelzer: New features in ethtool are certainly dependent on kernel version, but it always used to stay backward compatible with (much) older kernel versions. When I maintained it I would test on old versions before every release.

It's possible that this backward compatibility is no longer maintained, particularly around UFO control, but I think it's worth testing whether the new version could work for all supported kernel versions.

Hi Ben - yeah I agree it is generally working great across versions - the biggest pain IMHO would be subtle things - not an obvious Feature that breaks works.
Instead an uncommon option that now has a different effect or a slight change in the output. Both could seriously affect scripting/automation that people have already in place, but might be hard to spot even when testing.
Due to that I have suggested the opt-in hwe-ethtool style package which seems to provide the best of both worlds.

I have put it on my list of things to discuss with the owning Team on the next sprint, so if nothing happens until then I can discuss it there more in detail.