src.ctf.fs: be less strict when encountering trailing byte at the end of packetized CTF 1.8 metadata
A client has some traces with CTF 1.8 packetized metadata that has an
unexpected 0x0A (\n) byte at the end. The metadata file consists of a
sequence of several well-formed metadata packets, followed by one 0x0A
byte. It is not known where this byte comes from, it could have been
inserted as the result of some post-processing of the metadata files.
While not a valid metadata stream, Babeltrace 2.0 accepted it without
complaining. The code in 2.0 looked like:
... where `header` represents a metadata packet header. Because the
number of remaining bytes to read was less than the size of `header`,
`fread` would return 0. Babeltrace would conclude that it had reached
the end of the file and would exit the loop, without ever noticing this
extra byte.
In 2.1, Babeltrace is more strict, and emits this error:
CAUSED BY ['source.ctf.fs'] (/home/smarchi/src/babeltrace/src/plugins/ctf/common/src/metadata/tsdl/metadata-stream-decoder.cpp:314)
Failed to decode metadata stream section: data-ptr=0x79d1d9fec900, data-len-bytes=4097
CAUSED BY ['source.ctf.fs'] (/home/smarchi/src/babeltrace/src/plugins/ctf/common/src/metadata/tsdl/metadata-stream-decoder.cpp:274)
Failed to read a metadata stream packet: offset-bytes=4096, pkt-idx=1
CAUSED BY ['source.ctf.fs'] (/home/smarchi/src/babeltrace/src/plugins/ctf/common/src/metadata/tsdl/metadata-stream-decoder.cpp:244)
Remaining buffer isn't large enough to hold a packet header.
Even though this isn't technically a valid metadata stream, and we don't
know where this mysterious 0x0A byte comes from, the fact that
Babeltrace 2.0 accepted it and 2.1 rejects it can be a problem for
real-world users, if traces like this are in the wild.
Since this extra byte doesn't really prevent Babeltrace from reading the
trace as it did before, I propose to change the error to a warning.
Add a test for this, which consists of a copy of the `2packets`, to
which I added a trailing 0x0A byte to the metadata file.
The result looks like this:
05-14 14:47:52.527 107232 107232 W PLUGIN/CTF/META/DECODER _textFromPacketizedMetadata@/home/smarchi/src/babeltrace/src/plugins/ctf/common/src/metadata/tsdl/metadata-stream-decoder.cpp:244 Remaining buffer isn't large enough to hold a packet header.
Note that this warning appears three times when relying on the CLI's
autodisovery feature, because the metadata is parsed three times in this
case:
- once during the "support info" query
- once during the "get supported MIP versions" method call
- once when instantiating the component
Change-Id: Ibaaacd52e8993e0b126691d3591dee67fb94b3a0
Signed-off-by: Simon Marchi <email address hidden>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/14630
Tested-by: jenkins <email address hidden>
Reviewed-by: Philippe Proulx <email address hidden>
(cherry picked from commit b83661e6d4f92c2f2bc230237f0109ba1fb743aa)
Reviewed-on: https://review.lttng.org/c/babeltrace/+/14869
Add the `bt_test_cli` test util function, whose purpose is to reduce the
boilerplate in tests. `bt_test_cli` runs `bt_cli`, checks some aspects
of the result, and records some TAP test results (hence the `test` in
the name). See its doc for more details.
I changed a bunch of tests to use it, just enough to validate that the
approach was flexible enough and can apply in a variety of situations.
There are certainly more tests that could be changed to use it, and we
can add more options later to cater to more use cases.
Change-Id: I54d3b4ae96d33b0a5c7eae374b30d7ef5880fe09
Signed-off-by: Simon Marchi <email address hidden>
Reviewed-on: https://review.lttng.org/c/babeltrace/+/14629
Reviewed-by: Philippe Proulx <email address hidden>
Tested-by: jenkins <email address hidden>
(cherry picked from commit 6ec27c7002ab37a5a0ac1eb42dea47c2544fa3c4)
Reviewed-on: https://review.lttng.org/c/babeltrace/+/14868
./tests/bindings/python/bt2/test_field_class.py:30:13: F824 `nonlocal field_class` is unused: name is never assigned in scope
./tests/bindings/python/bt2/test_field_class.py:31:13: F824 `nonlocal value_setter_fn` is unused: name is never assigned in scope
./tests/bindings/python/bt2/test_graph.py:291:17: F824 `nonlocal graph` is unused: name is never assigned in scope
./tests/bindings/python/bt2/test_graph.py:465:17: F824 `nonlocal raised_in_sink` is unused: name is never assigned in scope
./tests/bindings/python/bt2/test_graph.py:532:13: F824 `nonlocal calls` is unused: name is never assigned in scope
./tests/bindings/python/bt2/test_message_iterator.py:573:13: F824 `nonlocal input_port_iter_can_seek_beginning` is unused: name is never assigned in scope
./tests/bindings/python/bt2/test_message_iterator.py:696:17: F824 `nonlocal do_seek_beginning` is unused: name is never assigned in scope
./tests/bindings/python/bt2/test_port.py:801:17: F824 `nonlocal user_datas` is unused: name is never assigned in scope
./tests/bindings/python/bt2/test_port.py:835:17: F824 `nonlocal user_datas` is unused: name is never assigned in scope
./tests/bindings/python/bt2/test_port.py:877:17: F824 `nonlocal user_datas` is unused: name is never assigned in scope
./tests/bindings/python/bt2/test_query_executor.py:225:17: F824 `nonlocal interrupter2` is unused: name is never assigned in scope
./tests/bindings/python/bt2/test_value.py:1092:13: F824 `nonlocal value` is unused: name is never assigned in scope
./tests/bindings/python/bt2/utils.py:301:13: F824 `nonlocal field_class` is unused: name is never assigned in scope
./tests/bindings/python/bt2/utils.py:302:13: F824 `nonlocal field_value_setter_fn` is unused: name is never assigned in scope
Remove almost all the unnecessary nonlocals. When removing the
nonlocals in test_port.py, then flake8 would complain with:
tests/bindings/python/bt2/test_port.py:804:17: F821 undefined name 'user_datas'
2a5817e...
by
=?utf-8?q?J=C3=A9r=C3=A9mie_Galarneau?= <email address hidden>
Fix: Python: set local `typing` version to that of 3.5.2
The `typing` that is in-tree is from the 3.10 version of python.
Running ./tests/plugins/flt.utils.trimmer/test-trimming.sh on
Python 3.5.2, produces the following error:
Traceback (most recent call last):
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/__init__.py",
line 23, in <module>
from bt2.mip import get_maximal_mip_version,
get_greatest_operative_mip_version
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/mip.py",
line 9, in <module>
from bt2 import component_descriptor as bt2_component_descriptor
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/component_descriptor.py",
line 5, in <module>
from bt2 import value as bt2_value
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/value.py",
line 524, in <module>
class ArrayValue(_ArrayValueConst, _Container,
collections.abc.MutableSequence, _Value):
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/value.py",
line 530, in ArrayValue
def __init__(self, value: typing.Optional[typing.Iterable] = None):
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/local_typing.py",
line 686, in inner
return func(*args, **kwds)
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/local_typing.py",
line 850, in __getitem__
return Union[arg, type(None)]
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/local_typing.py",
line 686, in inner
return func(*args, **kwds)
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/local_typing.py",
line 807, in __getitem__
return self.__class__(parameters, origin=self, _root=True)
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/local_typing.py",
line 747, in __new__
parameters = _remove_dups_flatten(parameters)
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/local_typing.py",
line 651, in _remove_dups_flatten
for t2 in all_params - {t1}
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/local_typing.py",
line 653, in <genexpr>
t2.__origin__ is not None)):
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/local_typing.py",
line 1158, in __subclasscheck__
return super().__subclasscheck__(cls)
File "/usr/lib/python3.5/abc.py", line 207, in __subclasscheck__
ok = cls.__subclasshook__(subclass)
File
"/home/jenkins/jgalar-build/babeltrace/src/bindings/python/bt2/build/build_lib/bt2/local_typing.py",
line 888, in __extrahook__
if issubclass(subclass, scls):
File
"/home/jenkins/jgalar-build/babeltrace/tests/utils/python/typing/typing.py",
line 1152, in __subclasscheck__
raise TypeError("Parameterized generics cannot be used with class "
TypeError: Parameterized generics cannot be used with class or instance checks
Moving the in-tree version back to something more contemporary with
Python 3.4/3.5 fixes the issue.
The version used is the earliest one in the python tree that contains the
fix to the issues being worked around in 3.5.2 [1][2].
2e77a0a...
by
=?utf-8?q?J=C3=A9r=C3=A9mie_Galarneau?= <email address hidden>
Fix: Python: missing __contains__ method for _PluginComponentClasses
The python binding tests fail on Python 3.4 and 3.5 when attempting to look up a
component class with an invalid name
(`test_create_from_name_wrong_component_class_name_type`) as as unexpected
`KeyError` is raised instead of the expected `TypeError`.
In these older Python versions, `typing.Mapping` does not provide a
`__contains__` method, which is used by the `in` operator. Instead, the
interpreter falls back to using the iterator to sequentially compare the
requested key against all values.
In more recent Python versions, the `__contains__` method is provided by
`typing.Mapping` and the test relies on this method to call `__getitem__` and
fail with a `TypeError` since the name of a component class must be a string
(`_check_str` fails).
This commit adds a `__contains__` method to `_PluginComponentClasses` that uses
`__getitem__` to check for the presence of a key and raises the `TypeError` that
test excepts.
