False positive testsuite failures with grep 2.10

Bug #996483 reported by Laurynas Biveinis
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona XtraBackup moved to https://jira.percona.com/projects/PXB
Fix Released
High
Alexey Kopytov
1.6
Fix Released
High
Alexey Kopytov
2.0
Fix Released
High
Alexey Kopytov
2.1
Fix Released
High
Alexey Kopytov

Bug Description

When grep 2.10 or newer (e.g. Ubuntu 12.04) is used, a part of tests fail with a false positive

+ grep 'xtrabackup: using O_DIRECT' /home/laurynas/percona/src/xtrabackup-page-filters/test/results/bug606981
grep: input file `/home/laurynas/percona/src/xtrabackup-page-filters/test/results/bug606981' is also the output
+ vlog 'xtrabackup was not using O_DIRECT for the input file.'
++ date '+%F %T'
++ basename t/bug606981.sh
+ echo '2012-05-08 13:22:21: bug606981.sh: xtrabackup was not using O_DIRECT for the input file.'
2012-05-08 13:22:21: bug606981.sh: xtrabackup was not using O_DIRECT for the input file.
+ exit -1

The "xtrabackup: using O_DIRECT" is present in the file, however grep refuses to run due to input and output being identical, this change is documented in 2.10 release notes:

  grep now rejects a command like "grep -r pattern . > out",
  in which the output file is also one of the inputs,
  because it can result in an "infinite" disk-filling loop.
  [bug present since "the beginning"]

Related branches

Stewart Smith (stewart)
Changed in percona-xtrabackup:
importance: Undecided → High
Revision history for this message
Alexey Kopytov (akopytov) wrote :

We don't see such failures on Precise builds in Jenkins?

Changed in percona-xtrabackup:
status: Confirmed → Incomplete
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

I did see these failures on Jenkins. Will provide links to examples soon (or confirm that the failures have disappeared since).

Revision history for this message
Stewart Smith (stewart) wrote : Re: [Bug 996483] Re: False positive testsuite failures with grep 2.10+

On Wed, 13 Jun 2012 16:50:50 -0000, Alexey Kopytov <email address hidden> wrote:
> We don't see such failures on Precise builds in Jenkins?

Up until very recently we hadn't added precise to the XB jobs, this
could be why.

--
Stewart Smith

Revision history for this message
Alexey Kopytov (akopytov) wrote :

On 14.06.12 4:58, Stewart Smith wrote:
> On Wed, 13 Jun 2012 16:50:50 -0000, Alexey Kopytov <email address hidden> wrote:
>> We don't see such failures on Precise builds in Jenkins?
>
> Up until very recently we hadn't added precise to the XB jobs, this
> could be why.
>

OK, Precise builders were added yesterday and now show the problem:
http://jenkins.percona.com/view/XtraBackup/job/percona-xtrabackup-2.0/921/

Revision history for this message
Alexey Kopytov (akopytov) wrote : Re: False positive testsuite failures with grep 2.10+

It turns out the failures are a result of a bug in grep itself. All failing tests except 2 do the correct thing: they use -q which supresses the output, so grepping the same file where the output is redirected is not a problem, because grep itself generates no output.

However, grep version shipped with Ubuntu 12.04 is buggy and fails even when -q is specified. The bug was fixed in grep 2.11:

" grep no longer rejects "grep -qr . > out", i.e., when run with -q
  and an input file is the same as the output file, since with -q
  grep generates no output, so there is no risk of infinite loop or
  of an output-affecting race condition. Thus, the use of the following
  options also disables the input-equals-output failure:
    --max-count=N (-m) (for N >= 2)
    --files-with-matches (-l)
    --files-without-match (-L)
  [bug introduced in grep-2.10]"

So fixing those 2 tests that do not use -q will not help, as all tests using grep will still be failing on Ubuntu 12.04. The workaround is to redirect the output to /dev/null.

summary: - False positive testsuite failures with grep 2.10+
+ False positive testsuite failures with grep 2.10
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

A note to self with affected tests on the current 2.1 trunk:
t/bug606981.sh t/bug759225.sh t/bug766033.sh t/bug972169.sh t/bug999750.sh t/ib_empty_dir.sh t/xb_stats.sh

Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-329

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.