Comment 4 for bug 1773150

Hi Lee,

Sorry for the slow reply, but I can still reproduce this - MAAS 2.4.2 testing a Ubuntu 18.04.01 system. Using your code;

Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> smart_support_regex = re.compile('SMART support is:\s+Available')
>>> output = open('smartctl.txt', 'rb').read()
>>> match = smart_support_regex.search(output.decode('utf-8'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x99 in position 377: invalid start byte

Adding replace to that line, it works;

Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import re
>>> smart_support_regex = re.compile('SMART support is:\s+Available')
>>> output = open('smartctl.txt', 'rb').read()
>>> match = smart_support_regex.search(output.decode('utf-8', 'replace'))
>>> match is not None
True

In terms of the MAAS testing script, if I change the match line to the one above (line 85) that line no longer fails. However, it crashes on a later line instead when it attempts to print out the results;

./smartctl-validate --storage /dev/sdc
INFO: Veriying SMART support for the following drive: /dev/sdc
INFO: Running command: sudo -n smartctl --all /dev/sdc

INFO: SMART support is available; continuing...
INFO: Verifying and/or validating SMART tests...
INFO: Running command: sudo -n smartctl --xall /dev/sdc

FAILURE: SMART tests have FAILED for: /dev/sdc
The test exited with return code 64! See the smarctl manpage for information on the return code meaning. For more information on the test failures, review the test output provided below.
---------------------------------------------------

Traceback (most recent call last):
  File "./smartctl-validate", line 189, in <module>
    sys.exit(run_smartctl(args.storage, test))
  File "./smartctl-validate", line 171, in run_smartctl
    print(output.decode('utf-8'))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x99 in position 377: invalid start byte

If I update line 171 to also have the replace argument (i.e. `print(output.decode('utf-8', 'replace'))` ), the script works and returns the test results (attached).