lp:sosreport

Created by Adam Stokes on 2013-04-17 and last modified on 2020-11-23
Get this branch:
bzr branch lp:sosreport

Branch merges

Related bugs

Related blueprints

Branch information

Owner:
sosreport
Project:
sosreport
Status:
Development

Import details

Import Status: Reviewed

This branch is an import of the HEAD branch of the Git repository at git://github.com/sosreport/sosreport.git.

The next import is scheduled to run in 4 hours.

Last successful import was 1 hour ago.

Import started 1 hour ago on alnitak and finished 1 hour ago taking 20 seconds — see the log
Import started 7 hours ago on alnitak and finished 7 hours ago taking 15 seconds — see the log
Import started 14 hours ago on alnitak and finished 14 hours ago taking 20 seconds — see the log
Import started 20 hours ago on alnitak and finished 20 hours ago taking 15 seconds — see the log
Import started on 2020-11-28 on alnitak and finished on 2020-11-28 taking 20 seconds — see the log
Import started on 2020-11-28 on alnitak and finished on 2020-11-28 taking 20 seconds — see the log
Import started on 2020-11-27 on alnitak and finished on 2020-11-27 taking 20 seconds — see the log
Import started on 2020-11-27 on alnitak and finished on 2020-11-27 taking 20 seconds — see the log
Import started on 2020-11-27 on alnitak and finished on 2020-11-27 taking 20 seconds — see the log
Import started on 2020-11-26 on alnitak and finished on 2020-11-26 taking 20 seconds — see the log

Recent revisions

3413. By Jake Hunsaker <email address hidden> on 2020-11-23

[readme] Update badging

Update the build badge for the repo README to point to Cirrus-CI instead
of Travis now that we've switched CI platforms.

Signed-off-by: Jake Hunsaker <email address hidden>

3412. By Mauricio Faria de Oliveira on 2020-11-23

[networking] Remove superfluous if/out_ns and re-indent per-ns code

Split this into its own commit as it's mostly indentation changes.

From:
```
        if self.g_n_n():
            out_ns = self.g_n_n(<filters>)
            for namespace in out_ns:
            ...

            if self.get_option('ethtool_namespaces'):
                for namespace in out_ns:
                ...
```

To:
```
        for namespace in self.g_n_n(<filters>):
              ...

        if self.get_option('ethtool_namespaces'):
            for namespace in self.g_n_n(<filters>):
            ...
```

Resolves: #2308

Signed-off-by: Mauricio Faria de Oliveira <email address hidden>
Signed-off-by: Jake Hunsaker <email address hidden>

3411. By Mauricio Faria de Oliveira on 2020-11-23

[Plugin,networking] Move namespace filtering to get_network_namespaces()

Copy the logic over from the networking plugin to `filter_namespaces()`,
which is generic, taking any list of namespaces; and then wrap it in
`get_network_namespaces()` for the particular network namespace type.

Now other plugins may be able to filter namespaces as well, and
not only network namespaces, but other types that may be added
(which can get convenience wrappers `get_X_namespaces()` then.)

In order to make the review simpler, the changes for the networking
plugin are intentionally small (keep the optional `if self.g_n_n():`
and only filter in `out_ns` which is used in `ethtool_namespaces`.)
The next patch addresses that, with the larger indentation changes.

Testing with the networking plugins' options to filter namespaces
by a regex pattern, and limit by the number:

```
    $ ip netns
    ns3
    ns2
    ns1
```

Option `networking.namespace_pattern` (only namespaces with trailing 2):
```
    $ sudo ./bin/sos report -o networking \
        -k networking.namespace_pattern='.*2' --batch

    $ find sosreport-*/sos_commands/networking \
        | cut -d/ -f2- | grep ip_netns_exec
    sos_commands/networking/ip_netns_exec_ns2_iptables-save
    sos_commands/networking/ip_netns_exec_ns2_ip_-s_-s_neigh_show
    sos_commands/networking/ip_netns_exec_ns2_ip_address_show
    sos_commands/networking/ip_netns_exec_ns2_ip_route_show_table_all
    sos_commands/networking/ip_netns_exec_ns2_ethtool_-i_lo
    sos_commands/networking/ip_netns_exec_ns2_ethtool_-k_lo
    sos_commands/networking/ip_netns_exec_ns2_ethtool_-S_lo
    sos_commands/networking/ip_netns_exec_ns2_ethtool_lo
    sos_commands/networking/ip_netns_exec_ns2_ip6tables-save
    sos_commands/networking/ip_netns_exec_ns2_netstat_-W_-agn
    sos_commands/networking/ip_netns_exec_ns2_netstat_-W_-neopa
    sos_commands/networking/ip_netns_exec_ns2_ip_rule_list
    sos_commands/networking/ip_netns_exec_ns2_netstat_-s
```

Option `networking.namespaces` (only one namespace):
```
    $ sudo ./bin/sos report -o networking \
        -k networking.namespaces=1 --batch

    $ grep 'Limiting namespace iteration' sosreport-*/sos_logs/sos.log
    2020-11-18 22:48:00,263 WARNING: [plugin:networking] \
    Limiting namespace iteration to first 1 namespaces found

    $ find sosreport-*/sos_commands/networking \
        | cut -d/ -f2- | grep ip_netns_exec
    sos_commands/networking/ip_netns_exec_ns3_ethtool_-i_lo
    sos_commands/networking/ip_netns_exec_ns3_ip_rule_list
    sos_commands/networking/ip_netns_exec_ns3_ip_route_show_table_all
    sos_commands/networking/ip_netns_exec_ns3_netstat_-W_-neopa
    sos_commands/networking/ip_netns_exec_ns3_netstat_-s
    sos_commands/networking/ip_netns_exec_ns3_ethtool_-k_lo
    sos_commands/networking/ip_netns_exec_ns3_netstat_-W_-agn
    sos_commands/networking/ip_netns_exec_ns3_iptables-save
    sos_commands/networking/ip_netns_exec_ns3_ip_address_show
    sos_commands/networking/ip_netns_exec_ns3_ip_-s_-s_neigh_show
    sos_commands/networking/ip_netns_exec_ns3_ip6tables-save
    sos_commands/networking/ip_netns_exec_ns3_ethtool_-S_lo
    sos_commands/networking/ip_netns_exec_ns3_ethtool_lo
```

Signed-off-by: Mauricio Faria de Oliveira <email address hidden>
Signed-off-by: Jake Hunsaker <email address hidden>

3410. By Mauricio Faria de Oliveira on 2020-11-23

[report,conntrack,ebpf,networking] Introduce get_network_namespaces()

Put the list of network namespaces in `commons['namespaces']['network']`
as with eg, existing block devices in `commons['devices']['block']`.

Then provide the shortcut function `get_network_namespaces()` for such.

And convert the conntrack/ebpf/networking plugins to use it, which
deduplicates their code and simplifies the plugins per-ns handling.

```
    $ ip netns
    ns3
    ns2
    ns1

    $ sudo python3
    >>> from sos import SoS
    >>> from sos.report.plugins import Plugin
    >>>
    >>> sos = SoS(['report'])
    >>> plugin = Plugin(sos._component.get_commons())

    >>> plugin.commons['namespaces']['network']
    ['ns3', 'ns2', 'ns1']

    >>> plugin.commons['devices']['block']
    ['loop1', 'vdb', 'vda', ...]
```

The contents of the `sos_commands/{contrack,ebpf,networking}`
directories are identical before/after this patch (no changes).

Before and After:
```
    $ sudo ./bin/sos report -o conntrack,ebpf,networking --batch
```
Compare:
```
    $ find sosreport-<OLD>/sos_commands/ | cut -d/ -f2- | sort \
        > find-sosreport.old

    $ find sosreport-<NEW>/sos_commands/ | cut -d/ -f2- | sort \
        > find-sosreport.new

    $ diff find-sosreport.{old,new}
    $

    $ md5sum find-sosreport.{old,new}
    6fb6d8db55b4bc7197de3c65358c3576 find-sosreport.old
    6fb6d8db55b4bc7197de3c65358c3576 find-sosreport.new
```

For reference, listing the commands run for `ns1`:
```
    $ find sosreport-sos-g-2020-11-18-ryrcztt/sos_commands/ \
       | cut -d/ -f2- | grep ns1
    sos_commands/networking/ip_netns_exec_ns1_ip_rule_list
    sos_commands/networking/ip_netns_exec_ns1_netstat_-W_-neopa
    sos_commands/networking/ip_netns_exec_ns1_netstat_-s
    sos_commands/networking/ip_netns_exec_ns1_iptables-save
    sos_commands/networking/ip_netns_exec_ns1_ip_address_show
    sos_commands/networking/ip_netns_exec_ns1_ethtool_lo
    sos_commands/networking/ip_netns_exec_ns1_ip_route_show_table_all
    sos_commands/networking/ip_netns_exec_ns1_netstat_-W_-agn
    sos_commands/networking/ip_netns_exec_ns1_ip6tables-save
    sos_commands/networking/ip_netns_exec_ns1_ip_-s_-s_neigh_show
    sos_commands/networking/ip_netns_exec_ns1_ethtool_-i_lo
    sos_commands/networking/ip_netns_exec_ns1_ethtool_-k_lo
    sos_commands/networking/ip_netns_exec_ns1_ethtool_-S_lo
    sos_commands/conntrack/ip_netns_exec_ns1_conntrack_-S
    sos_commands/conntrack/ip_netns_exec_ns1_conntrack_-L_-o_extended
    sos_commands/ebpf/ip_netns_exec_ns1_bpftool_net_list
```

Signed-off-by: Mauricio Faria de Oliveira <email address hidden>
Signed-off-by: Jake Hunsaker <email address hidden>

3409. By Mauricio Faria de Oliveira on 2020-11-23

[conntrack] gather per-namespace data

Run conntrack commands per-namespace, as the ebpf/networking plugins:

Test-case:
```
    # ip netns
    ns3
    ns2
    ns1
```

Before:
```
    # ./bin/sos report -o conntrack,ebpf,networking --batch

    # tar tf /tmp/sosreport-*.tar.xz | grep ip_netns_exec
    .../sos_commands/ebpf/ip_netns_exec_ns1_bpftool_net_list
    .../sos_commands/ebpf/ip_netns_exec_ns2_bpftool_net_list
    .../sos_commands/ebpf/ip_netns_exec_ns3_bpftool_net_list
    ...
    .../sos_commands/networking/ip_netns_exec_ns1_ip_address_show
    ...
    .../sos_commands/networking/ip_netns_exec_ns2_ip_address_show
    ...
    .../sos_commands/networking/ip_netns_exec_ns3_ip_address_show
    ...
```

After:

```
    # ./bin/sos report -o conntrack,ebpf,networking --batch

    # tar tf /tmp/sosreport-*.tar.xz | grep ip_netns_exec
    .../sos_commands/conntrack/ip_netns_exec_ns1_conntrack_-L_-o_extended
    .../sos_commands/conntrack/ip_netns_exec_ns1_conntrack_-S
    .../sos_commands/conntrack/ip_netns_exec_ns2_conntrack_-L_-o_extended
    .../sos_commands/conntrack/ip_netns_exec_ns2_conntrack_-S
    .../sos_commands/conntrack/ip_netns_exec_ns3_conntrack_-L_-o_extended
    .../sos_commands/conntrack/ip_netns_exec_ns3_conntrack_-S
    .../sos_commands/ebpf/ip_netns_exec_ns1_bpftool_net_list
    .../sos_commands/ebpf/ip_netns_exec_ns2_bpftool_net_list
    .../sos_commands/ebpf/ip_netns_exec_ns3_bpftool_net_list
    ...
    .../sos_commands/networking/ip_netns_exec_ns1_ip_address_show
    ...
    .../sos_commands/networking/ip_netns_exec_ns2_ip_address_show
    ...
    .../sos_commands/networking/ip_netns_exec_ns3_ip_address_show
    ...
```

Signed-off-by: Mauricio Faria de Oliveira <email address hidden>
Signed-off-by: Jake Hunsaker <email address hidden>

3408. By Mauricio Faria de Oliveira on 2020-11-23

[networking] Ignore stderr from ip netns for list of namespaces

The sosreport of a system with this issue:
```
    # ip netns
    Error: Peer netns reference is invalid.
    Error: Peer netns reference is invalid.
    test-ns
```

Shows that the networking plugin runs commands for the `Error:` lines:
(note the ebpf plugin does not due to differences addressed in PR #2306)
```
    # ./bin/sos report -o ebpf,networking --batch

    # tar tf /tmp/sosreport-*.tar.xz | grep ip_netns_exec
    .../sos_commands/ebpf/ip_netns_exec_test-ns_bpftool_net_list
    .../sos_commands/networking/ip_netns_exec_Error_ip6tables-save
    .../sos_commands/networking/ip_netns_exec_Error_ip6tables-save.1
    ...
    .../sos_commands/networking/ip_netns_exec_test-ns_ip6tables-save
    ...
```

This happens because the networking plugin calls `collect_cmd_output()`
with default `stderr=True` and does not handle such line type.

For the purposes of getting the list of network namespaces, it is OK to
ignore `stderr` since it does not provide that information. However, we
do want it in the archive, so it is fully documented for analysis/debug.

So change the call from `collect_cmd_output()` to `add_cmd_output()` to
include both `stdout` and `stderr` in the archive, and call `exec_cmd()`
that ignores `stderr` to get the list of network namespaces.

Note that the plugin _currently_ does not need `exec_cmd(stderr=False)`
to ignore `stderr`, as described in PR#2306, but will once/if that is
applied. However, with the next patch, `stderr=False` won't be needed.

Before:
```
  # tar tf /tmp/sosreport-*.tar.xz | grep ip_netns_exec
  .../sos_commands/ebpf/ip_netns_exec_test-ns_bpftool_net_list
  .../sos_commands/networking/ip_netns_exec_Error_ip6tables-save
  .../sos_commands/networking/ip_netns_exec_Error_ip6tables-save.1
  ...
  .../sos_commands/networking/ip_netns_exec_test-ns_ip6tables-save
  ...
```

After:
```
  # tar tf /tmp/sosreport-*.tar.xz | grep ip_netns_exec
  .../sos_commands/ebpf/ip_netns_exec_test-ns_bpftool_net_list
  .../sos_commands/networking/ip_netns_exec_test-ns_ip6tables-save
  .../sos_commands/networking/ip_netns_exec_test-ns_ip_-s_-s_neigh_show
  ...
```

And the `ip netns` contents with `stderr` still remain in the archive:
```
    # tar xf /tmp/sosreport-*.tar.xz --to-stdout \
      --wildcards '*/sos_commands/networking/ip_netns'
    Error: Peer netns reference is invalid.
    Error: Peer netns reference is invalid.
    test-ns
```

Test suite:

```
    # ./tests/simple.sh
    ...
    Everything worked!
```

Signed-off-by: Mauricio Faria de Oliveira <email address hidden>
Signed-off-by: Jake Hunsaker <email address hidden>

3407. By Jake Hunsaker <email address hidden> on 2020-11-23

[grub2] Stylistic cleanup for grub2-mkconfig calls

Small stylistic cleanups for the `grub2-mkconfig` environment variable
settings and command formatting.

Closes: #1070
Resolves: #2317

Signed-off-by: Jake Hunsaker <email address hidden>

3406. By Jake Hunsaker <email address hidden> on 2020-11-23

[Plugin] Add method to get all containers matching a regex

Adds a new method, `Plugin.get_all_containers_by_regex()`, that will
return all known containers on the system with a name matching a
provided regex string. Optionally, this method may also return
terminated containers in addition to those that are active and running.

Closes: #2176
Resolves: #2319

Signed-off-by: Jake Hunsaker <email address hidden>

3405. By Jake Hunsaker <email address hidden> on 2020-11-23

[redhat] Add podman to valid container env vars

In newer versions of podman, the `container` env var set inside the
container will be set to `podman` instead of `oci`. Add this value to
the list of valid values to use to determine if sos is running inside a
container or not, for when we want to capture from the host system.

Resolves: #2320

Signed-off-by: Jake Hunsaker <email address hidden>

3404. By Pavel Moravec <email address hidden> on 2020-11-23

add lang files to sos.spec

Fix packit/copr build failures by adding lang files to sos.spec

Closes: #2318
Resolves: #2321

Signed-off-by: Pavel Moravec <email address hidden>
Signed-off-by: Jake Hunsaker <email address hidden>

Branch metadata

Branch format:
Branch format 7
Repository format:
Bazaar repository format 2a (needs bzr 1.16 or later)
This branch contains Public information 
Everyone can see this information.

Subscribers