[SRU] powerpc-utils commands exhibits "command substitution: ignored null byte in input" warning message

Default Comment by Bridge

------- Comment From <email address hidden> 2017-06-20 07:27 EDT-------
let me know if this patch get merged into Ubuntu build.

Could someone from foundations team please cherry-pick this commit?

The attachment "powerpc-utils_1.3.2-1ubuntu2_1.3.2-1ubuntu3.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

Manoj, could you please confirm that the testcases I wrote in the description are the right testcases for this issue? If any of them are wrong, please update accordingly with a working one.

This bug was fixed in the package powerpc-utils - 1.3.2-1ubuntu3

---------------
powerpc-utils (1.3.2-1ubuntu3) artful; urgency=medium

  * d/p/scripts-Command-substituion-warning.patch: fix bash warning
    "command substitution: ignored null byte in input" in ls-veth,
    ls-vscsi, lsdevinfo, and ofpathname; fix "cd: too many arguments"
    in ofpathname. LP: #1692420.

 -- Tiago Stürmer Daitx <email address hidden> Thu, 24 Aug 2017 09:48:27 -0700

Hello bugproxy, or anyone else affected,

Accepted powerpc-utils into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/powerpc-utils/1.3.2-1ubuntu3~17.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

------- Comment From <email address hidden> 2017-09-04 07:43 EDT-------
(In reply to comment #40)
> Hello bugproxy, or anyone else affected,
>
> Accepted powerpc-utils into zesty-proposed. The package will build now and
> be available at
> https://launchpad.net/ubuntu/+source/powerpc-utils/1.3.2-1ubuntu3~17.04 in a
> few hours, and then in the -proposed repository.
>
> Please help us by testing this new package. See
> https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to
> enable and use -proposed.Your feedback will aid us getting this update out
> to other Ubuntu users.
>
> If this package fixes the bug for you, please add a comment to this bug,
> mentioning the version of the package you tested and change the tag from
> verification-needed-zesty to verification-done-zesty. If it does not fix the
> bug for you, please add a comment stating that, and change the tag to
> verification-failed-zesty. In either case, details of your testing will help
> us make a better decision.
>
> Further information regarding the verification process can be found at
> https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> advance!

Tried to install package from proposed, getting below error.

root@alp10:~# apt-get install powerpc-utils/zesty-proposed
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Release 'zesty-proposed' for 'powerpc-utils' was not found

Thanks,
Pavithra

------- Comment From <email address hidden> 2017-09-05 00:27 EDT-------
(In reply to comment #41)
> (In reply to comment #40)
> > Hello bugproxy, or anyone else affected,
> >
> > Accepted powerpc-utils into zesty-proposed. The package will build now and
> > be available at
> > https://launchpad.net/ubuntu/+source/powerpc-utils/1.3.2-1ubuntu3~17.04 in a
> > few hours, and then in the -proposed repository.
> >
> > Please help us by testing this new package. See
> > https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to
> > enable and use -proposed.Your feedback will aid us getting this update out
> > to other Ubuntu users.
> >
> > If this package fixes the bug for you, please add a comment to this bug,
> > mentioning the version of the package you tested and change the tag from
> > verification-needed-zesty to verification-done-zesty. If it does not fix the
> > bug for you, please add a comment stating that, and change the tag to
> > verification-failed-zesty. In either case, details of your testing will help
> > us make a better decision.
> >
> > Further information regarding the verification process can be found at
> > https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> > advance!
>
>
> Tried to install package from proposed, getting below error.
>
> root@alp10:~# apt-get install powerpc-utils/zesty-proposed
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> E: Release 'zesty-proposed' for 'powerpc-utils' was not found
>
> Thanks,
> Pavithra

Canonical, Can you check ?

On Mon, Sep 04, 2017 at 11:50:09AM -0000, bugproxy wrote:
> > Further information regarding the verification process can be found at
> > https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> > advance!

> Tried to install package from proposed, getting below error.

> root@alp10:~# apt-get install powerpc-utils/zesty-proposed
> Reading package lists... Done
> Building dependency tree
> Reading state information... Done
> E: Release 'zesty-proposed' for 'powerpc-utils' was not found

That is not valid apt syntax. Please
see https://wiki.ubuntu.com/QATeam/PerformingSRUVerification for guidance on
how to run tests for SRUs.

------- Comment From <email address hidden> 2017-09-27 03:23 EDT-------
(In reply to comment #43)
> On Mon, Sep 04, 2017 at 11:50:09AM -0000, bugproxy wrote:
> > > Further information regarding the verification process can be found at
> > > https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in
> > > advance!
>
> > Tried to install package from proposed, getting below error.
>
> > root@alp10:~# apt-get install powerpc-utils/zesty-proposed
> > Reading package lists... Done
> > Building dependency tree
> > Reading state information... Done
> > E: Release 'zesty-proposed' for 'powerpc-utils' was not found
>
> That is not valid apt syntax. Please
> see https://wiki.ubuntu.com/QATeam/PerformingSRUVerification for guidance on
> how to run tests for SRUs.

Tried the steps given in above link, still could not install the new package.

1. Added below in /etc/apt/sources.list

deb http://archive.ubuntu.com/ubuntu/ zesty-proposed main restricted universe

2. root@alp10:~# sudo apt-get update
Hit:1 http://us.ports.ubuntu.com/ubuntu-ports zesty InRelease
Get:2 http://us.ports.ubuntu.com/ubuntu-ports zesty-updates InRelease [89.2 kB]
Get:3 http://us.ports.ubuntu.com/ubuntu-ports zesty-backports InRelease [89.2 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports zesty-security InRelease [89.2 kB]
Get:5 http://archive.ubuntu.com/ubuntu zesty-proposed InRelease [240 kB]
Get:6 http://us.ports.ubuntu.com/ubuntu-ports zesty-updates/main ppc64el Packages [205 kB]
Ign:7 http://archive.ubuntu.com/ubuntu zesty-proposed/main ppc64el Packages
Get:8 http://archive.ubuntu.com/ubuntu zesty-proposed/main Translation-en [17.7 kB]
Ign:9 http://archive.ubuntu.com/ubuntu zesty-proposed/universe ppc64el Packages
Get:10 http://archive.ubuntu.com/ubuntu zesty-proposed/universe Translation-en [12.9 kB]
Ign:7 http://archive.ubuntu.com/ubuntu zesty-proposed/main ppc64el Packages
Ign:9 http://archive.ubuntu.com/ubuntu zesty-proposed/universe ppc64el Packages
Ign:7 http://archive.ubuntu.com/ubuntu zesty-proposed/main ppc64el Packages
Ign:9 http://archive.ubuntu.com/ubuntu zesty-proposed/universe ppc64el Packages
Err:7 http://archive.ubuntu.com/ubuntu zesty-proposed/main ppc64el Packages
404 Not Found [IP: 91.189.88.152 80]
Ign:9 http://archive.ubuntu.com/ubuntu zesty-proposed/universe ppc64el Packages
Fetched 743 kB in 1s (578 kB/s)
Reading package lists... Done
E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/zesty-proposed/main/binary-ppc64el/Packages 404 Not Found [IP: 91.189.88.152 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.
root@alp10:~# apt-get dist-upgrade^C
root@alp10:~# sudo apt-get install powerpc-utils
Reading package lists... Done
Building dependency tree
Reading state information... Done
powerpc-utils is already the newest version (1.3.2-1ubuntu2~17.04).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Thanks,
Pavithra

On Wed, Sep 27, 2017 at 07:29:56AM -0000, bugproxy wrote:
> > > root@alp10:~# apt-get install powerpc-utils/zesty-proposed
> > > Reading package lists... Done
> > > Building dependency tree
> > > Reading state information... Done
> > > E: Release 'zesty-proposed' for 'powerpc-utils' was not found

> > That is not valid apt syntax. Please
> > see https://wiki.ubuntu.com/QATeam/PerformingSRUVerification for guidance on
> > how to run tests for SRUs.

> Tried the steps given in above link, still could not install the new
> package.

> 1. Added below in /etc/apt/sources.list

> deb http://archive.ubuntu.com/ubuntu/ zesty-proposed main restricted
> universe

Ah - sorry, for ppc64el this needs to be:

 deb http://us.ports.ubuntu.com/ubuntu-ports/ zesty-proposed main restricted universe

I've updated the wiki documentation to match.

We are still awaiting verification of the package in -proposed.

------- Comment From <email address hidden> 2017-12-20 04:18 EDT-------
Installed powerpc-utils from - proposed. Issue is still observed.

~# sudo apt-get install powerpc-utils
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
powerpc-utils
1 upgraded, 0 newly installed, 0 to remove and 27 not upgraded.
Need to get 1,706 B of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://us.ports.ubuntu.com/ubuntu-ports zesty-proposed/main ppc64el powerpc-utils ppc64el 1.3.2-1ubuntu3~17.04 [1,706 B]
Fetched 1,706 B in 0s (8,144 B/s)
(Reading database ... 51522 files and directories currently installed.)
Preparing to unpack .../powerpc-utils_1.3.2-1ubuntu3~17.04_ppc64el.deb ...
Unpacking powerpc-utils (1.3.2-1ubuntu3~17.04) over (1.3.2-1ubuntu2~17.04) ...
Setting up powerpc-utils (1.3.2-1ubuntu3~17.04) ...
:~# lsdevinfo
/usr/sbin/lsdevinfo: line 233: warning: command substitution: ignored null byte in input
device:
name="ibmveth2"
uniquetype="adapter/vdevice/IBM,l-lan"
class="adapter"
subclass="vdevice"
type="IBM,l-lan"
prefix="eth"
driver="ibmveth"
status="1"

I think the upstream patch for lsdevinfo is incomplete as there is another call to cat. From the code:

254 # get the physical location
255 physloc=$($CAT $dev/ibm,loc-code)
256 type="$($OD -t x2 $dev/vendor-id $dev/device-id |
257 $CUT -f3,5 -d ' ' -s --output-delimiter='')"

Do the other commands that were problematic (ls-vscsi, ls-veth, and ofpathname) work?

------- Comment From <email address hidden> 2017-12-21 00:16 EDT-------
(In reply to comment #51)
> I think the upstream patch for lsdevinfo is incomplete as there is another
> call to cat. From the code:
>
> 254 # get the physical location
> 255 physloc=$($CAT $dev/ibm,loc-code)
> 256 type="$($OD -t x2 $dev/vendor-id $dev/device-id |
> 257 $CUT -f3,5 -d ' ' -s --output-delimiter='')"
>
> Do the other commands that were problematic (ls-vscsi, ls-veth, and
> ofpathname) work?

Issue is observed even with other commands.

:~# ls-vscsi
/usr/sbin/ls-vscsi: line 72: warning: command substitution: ignored null byte in input
host0 U8286.42A.1069B3T-V8-C17-T1
~# ls-veth
/usr/sbin/ls-veth: line 74: warning: command substitution: ignored null byte in input
ibmveth2 U8286.42A.1069B3T-V8-C2-T1
:~# ofpathname /dev/sda
/usr/sbin/ofpathname: line 812: warning: command substitution: ignored null byte in input
/usr/sbin/ofpathname: line 865: cd: too many arguments
/vdevice/v-scsi@30000011/disk@8100000000000000

The command substitution errors are happening in the same line number where the change from cat to tr was made. Could you confirm that the files installed in /usr/sbin/ are using the tr instead of cat?

------- Comment From <email address hidden> 2017-12-22 00:13 EDT-------
(In reply to comment #53)
> The command substitution errors are happening in the same line number where
> the change from cat to tr was made. Could you confirm that the files
> installed in /usr/sbin/ are using the tr instead of cat?

files installed in /usr/sbin/ still uses cat. but package installation was successful.

Thanks,
Pavithra

With regards to comment #17 I've opened the following issue upstream.

https://github.com/ibm-power-utilities/powerpc-utils/issues/24

------- Comment From <email address hidden> 2018-01-05 04:10 EDT-------
(In reply to comment #55)
> With regards to comment #17 I've opened the following issue upstream.
>
> https://github.com/ibm-power-utilities/powerpc-utils/issues/24

My commit already had those changes

https://github.com/ibm-power-utilities/powerpc-utils/commit/4fcc611a09caea70e6bca6a1275211c72ac4bb45

(In reply to comment #55)
> With regards to comment #17 I've opened the following issue upstream.
>
> https://github.com/ibm-power-utilities/powerpc-utils/issues/24

root@tuleta4u-lp7:/usr/sbin# dpkg -l | grep powerpc
ii powerpc-ibm-utils 1.3.2-1ubuntu3~17.04 ppc64el utilities for maintenance of IBM PowerPC platforms
ii powerpc-utils 1.3.2-1ubuntu3~17.04 ppc64el utilities for maintenance of PowerPC platforms (metapackage)
root@tuleta4u-lp7:/usr/sbin#

.
I am not able to see the issue in powerpc package 1.3.2-1ubuntu3~17.04.

Also could see files installed in /usr/sbin/ still uses tr.

.
Attaching the output of the commands.

------- Comment (attachment only) From <email address hidden> 2018-01-05 04:11 EDT-------

While I see that the commit did have the change to lsdevinfo around line 276 it looks like it somehow got reverted as trunk shows it still using cat.

https://github.com/ibm-power-utilities/powerpc-utils/blob/master/scripts/lsdevinfo#L276

------- Comment From <email address hidden> 2018-01-05 15:55 EDT-------
(In reply to comment #58)
> While I see that the commit did have the change to lsdevinfo around line 276
> it looks like it somehow got reverted as trunk shows it still using cat.
>
> https://github.com/ibm-power-utilities/powerpc-utils/blob/master/scripts/
> lsdevinfo#L276

Actually, Seeteena's patch only touched two places in the lsdevinfo script. Another patch hunk went in several months later that shifted the original line numbers, and is exposing to us that their are two other places in the script that use cat that weren't fixed. Namely, for PCI ethernet devices, and VFC devices.

[tyreld@oc6857751186 powerpc-utils]$ git blame scripts/lsdevinfo | grep "$dev/ibm,loc-code"
74715cc0 (Nathan Fontenot 2017-05-22 16:40:58 -0400 233) physloc=$(tr -d '\0' < $dev/ibm,loc-code)
4fcc611a (Seeteena Thoufeek 2017-01-11 15:36:33 +0530 254) physloc=$(tr -d '\0' < $dev/ibm,loc-code)
c600f291 (Santiago Leon 2012-04-16 08:49:18 -0500 276) physloc=$($CAT $dev/ibm,loc-code)
4fcc611a (Seeteena Thoufeek 2017-01-11 15:36:33 +0530 296) physloc=$(tr -d '\0' < $dev/ibm,loc-code)
8a8ff90e (Santiago Leon 2009-09-21 14:16:35 -0500 441) physloc=$(cat $dev/ibm,loc-code)

------- Comment From <email address hidden> 2018-01-05 16:19 EDT-------
I've sent a secondary patch out to the mailing list to fixup the two remaining problem spots in lsdevinfo.

https://groups.google.com/forum/#!topic/powerpc-utils-devel/QrB58JnT3Wk

If nobody has any issues I can pull it into the "next" branch immediately.

------- Comment From <email address hidden> 2018-01-10 03:27 EDT-------
(In reply to comment #60)
> I've sent a secondary patch out to the mailing list to fixup the two
> remaining problem spots in lsdevinfo.
>
> https://groups.google.com/forum/#!topic/powerpc-utils-devel/QrB58JnT3Wk
>
> If nobody has any issues I can pull it into the "next" branch immediately.

Thanks Tyrel.

------- Comment From <email address hidden> 2018-02-23 06:58 EDT-------
Canonical, please pick https://github.com/ibm-power-utilities/powerpc-utils/commit/3dcc222814ec1d661710eda320aced13566817f2

This bug was fixed in the package powerpc-utils - 1.3.4-0ubuntu2

---------------
powerpc-utils (1.3.4-0ubuntu2) bionic; urgency=medium

  * Cherrypick patch resolving more null byte command substitution warnings in
    lsdevinfo. (LP: #1692420)

 -- Brian Murray <email address hidden> Thu, 05 Apr 2018 14:08:00 -0700

Default Comment by Bridge

------- Comment From <email address hidden> 2018-04-18 02:50 EDT-------
Pavithra, Please verify now ?