Bug #1000498 “fmod() incorrectly returns NaN for (some?) denorma...” : Bugs : eglibc package : Ubuntu

Revision history for this message

In Sourceware.org Bugzilla #14048, Jkummerow (jkummerow) wrote on 2012-05-02:

#1

Download full text (4.3 KiB)

I've come across a case where fmod() does not return the expected result. Reduced repro:
------
#include <math.h>
#include <stdio.h>
#include <iostream>

int main() {
  double x = 2.225073858507201e-308;
  double y = 5e-324;
  double z = fmod(x, y);
  // printf("result: %g\n", z); // see [1] below.
  std::cout << "result: " << z << std::endl;
  return 0;
}
------
Expected result: 0
Actual result: -nan

I can repro the bug on both a Gentoo (gcc-4.5.3, kernel 3.3.4) and an Ubuntu Precise (gcc-4.6.3, kernel 3.2.5) system, which both have glibc-2.15, and both are x86_64. It works correctly on Ubuntu Lucid (glibc-2.11, gcc-4.4.2, kernel 2.6.38.8).

Further, it works correctly when compiling with either of -O1, -O2, -O3. It also works correctly when removing the "std::cout << ..." and "#include <iostream>" lines, and using the "printf(..." line (marked [1]) instead.

I've looked at the generated machine code. In the buggy case, glibc's fmod is called directly:
main():
  400744: 55 push rbp
  400745: 48 89 e5 mov rbp,rsp
  400748: 48 83 ec 20 sub rsp,0x20
  40074c: 48 b8 ff ff ff ff ff movabs rax,0xfffffffffffff
  400753: ff 0f 00
  400756: 48 89 45 f8 mov QWORD PTR [rbp-0x8],rax
  40075a: b8 01 00 00 00 mov eax,0x1
  40075f: 48 89 45 f0 mov QWORD PTR [rbp-0x10],rax
  400763: f2 0f 10 4d f0 movsd xmm1,QWORD PTR [rbp-0x10]
  400768: f2 0f 10 45 f8 movsd xmm0,QWORD PTR [rbp-0x8]
  40076d: e8 de fe ff ff call 400650 <fmod@plt>
  400772: f2 0f 11 45 e8 movsd QWORD PTR [rbp-0x18],xmm0
  400777: f2 0f 10 45 e8 movsd xmm0,QWORD PTR [rbp-0x18]
  40077c: bf dc 08 40 00 mov edi,0x4008dc
  400781: b8 01 00 00 00 mov eax,0x1
  400786: e8 75 fe ff ff call 400600 <printf@plt>
  40078b: b8 00 00 00 00 mov eax,0x0
  400790: c9 leave
  400791: c3 ret

When I do any of the changes that make it work (e.g. remove the iostream include), g++ decides to inline an FPU-based implementation of fmod (which seems to work as expected) and only calls out to glibc as a fallback:
main():
  400604: 55 push rbp
  400605: 48 89 e5 mov rbp,rsp
  400608: 48 83 ec 40 sub rsp,0x40
  40060c: 48 b8 ff ff ff ff ff movabs rax,0xfffffffffffff
  400613: ff 0f 00
  400616: 48 89 45 f8 mov QWORD PTR [rbp-0x8],rax
  40061a: b8 01 00 00 00 mov eax,0x1
  40061f: 48 89 45 f0 mov QWORD PTR [rbp-0x10],rax
  400623: dd 45 f8 fld QWORD PTR [rbp-0x8]
  400626: dd 45 f0 fld QWORD PTR [rbp-0x10]
  400629: d9 c0 fld st(0)
  40062b: d9 c2 fld st(2)
  40062d: d9 f8 fprem
  40062f: df e0 fnstsw ax
  400631: f6 c4 04 test ah,0x4
  400634: 75 f7 jne 40062d <main+0x29>
  400636: dd d9 fstp st(1)
  400638: dd 5d d8 fstp QWORD PTR [rbp-0x28]
  40063b: f2 0f 10 45 d8 movsd xmm0,QWORD PTR [rbp-0x28]
  400640: 66 0f 2e c0 ucomisd xmm0,xmm0
...

I've come across a case where fmod() does not return the expected result. Reduced repro:
------
#include <math.h>
#include <stdio.h>
#include <iostream>

int main() {
  double x = 2.225073858507201e-308;
  double y = 5e-324;
  double z = fmod(x, y);
  // printf("result: %g\n", z);  // see [1] below.
  std::cout << "result: " << z << std::endl;
  return 0;
}
------
Expected result: 0
Actual result: -nan

I can repro the bug on both a Gentoo (gcc-4.5.3, kernel 3.3.4) and an Ubuntu Precise (gcc-4.6.3, kernel 3.2.5) system, which both have glibc-2.15, and both are x86_64. It works correctly on Ubuntu Lucid (glibc-2.11, gcc-4.4.2, kernel 2.6.38.8).

Further, it works correctly when compiling with either of -O1, -O2, -O3. It also works correctly when removing the "std::cout << ..." and "#include <iostream>" lines, and using the "printf(..." line (marked [1]) instead.

I've looked at the generated machine code. In the buggy case, glibc's fmod is called directly:
main():
  400744:	55                   	push   rbp
  400745:	48 89 e5             	mov    rbp,rsp
  400748:	48 83 ec 20          	sub    rsp,0x20
  40074c:	48 b8 ff ff ff ff ff 	movabs rax,0xfffffffffffff
  400753:	ff 0f 00 
  400756:	48 89 45 f8          	mov    QWORD PTR [rbp-0x8],rax
  40075a:	b8 01 00 00 00       	mov    eax,0x1
  40075f:	48 89 45 f0          	mov    QWORD PTR [rbp-0x10],rax
  400763:	f2 0f 10 4d f0       	movsd  xmm1,QWORD PTR [rbp-0x10]
  400768:	f2 0f 10 45 f8       	movsd  xmm0,QWORD PTR [rbp-0x8]
  40076d:	e8 de fe ff ff       	call   400650 <fmod@plt>
  400772:	f2 0f 11 45 e8       	movsd  QWORD PTR [rbp-0x18],xmm0
  400777:	f2 0f 10 45 e8       	movsd  xmm0,QWORD PTR [rbp-0x18]
  40077c:	bf dc 08 40 00       	mov    edi,0x4008dc
  400781:	b8 01 00 00 00       	mov    eax,0x1
  400786:	e8 75 fe ff ff       	call   400600 <printf@plt>
  40078b:	b8 00 00 00 00       	mov    eax,0x0
  400790:	c9                   	leave  
  400791:	c3                   	ret

When I do any of the changes that make it work (e.g. remove the iostream include), g++ decides to inline an FPU-based implementation of fmod (which seems to work as expected) and only calls out to glibc as a fallback:
main():
  400604:	55                   	push   rbp
  400605:	48 89 e5             	mov    rbp,rsp
  400608:	48 83 ec 40          	sub    rsp,0x40
  40060c:	48 b8 ff ff ff ff ff 	movabs rax,0xfffffffffffff
  400613:	ff 0f 00 
  400616:	48 89 45 f8          	mov    QWORD PTR [rbp-0x8],rax
  40061a:	b8 01 00 00 00       	mov    eax,0x1
  40061f:	48 89 45 f0          	mov    QWORD PTR [rbp-0x10],rax
  400623:	dd 45 f8             	fld    QWORD PTR [rbp-0x8]
  400626:	dd 45 f0             	fld    QWORD PTR [rbp-0x10]
  400629:	d9 c0                	fld    st(0)
  40062b:	d9 c2                	fld    st(2)
  40062d:	d9 f8                	fprem  
  40062f:	df e0                	fnstsw ax
  400631:	f6 c4 04             	test   ah,0x4
  400634:	75 f7                	jne    40062d <main+0x29>
  400636:	dd d9                	fstp   st(1)
  400638:	dd 5d d8             	fstp   QWORD PTR [rbp-0x28]
  40063b:	f2 0f 10 45 d8       	movsd  xmm0,QWORD PTR [rbp-0x28]
  400640:	66 0f 2e c0          	ucomisd xmm0,xmm0
  400644:	7a 06                	jp     40064c <main+0x48>
  400646:	66 0f 2e c0          	ucomisd xmm0,xmm0
  40064a:	74 17                	je     400663 <main+0x5f>
  40064c:	dd 5d c8             	fstp   QWORD PTR [rbp-0x38]
  40064f:	f2 0f 10 4d c8       	movsd  xmm1,QWORD PTR [rbp-0x38]
  400654:	dd 5d c8             	fstp   QWORD PTR [rbp-0x38]
  400657:	f2 0f 10 45 c8       	movsd  xmm0,QWORD PTR [rbp-0x38]
  40065c:	e8 af fe ff ff       	call   400510 <fmod@plt>
  400661:	eb 04                	jmp    400667 <main+0x63>
  400663:	dd d8                	fstp   st(0)
  400665:	dd d8                	fstp   st(0)
  400667:	f2 0f 11 45 e8       	movsd  QWORD PTR [rbp-0x18],xmm0
  40066c:	f2 0f 10 45 e8       	movsd  xmm0,QWORD PTR [rbp-0x18]
  400671:	bf 7c 07 40 00       	mov    edi,0x40077c
  400676:	b8 01 00 00 00       	mov    eax,0x1
  40067b:	e8 70 fe ff ff       	call   4004f0 <printf@plt>
  400680:	b8 00 00 00 00       	mov    eax,0x0
  400685:	c9                   	leave  
  400686:	c3                   	ret

So, it looks to me like glibc's fmod() doesn't handle this case (max denormalized double modulo min denormalized double) correctly. Am I barking up the wrong tree?

Revision history for this message

In Sourceware.org Bugzilla #14048, Bugdal (bugdal) wrote on 2012-05-02:

#2

Using the -fno-builtin option to gcc should make it easier to test/reproduce this.

Revision history for this message

In Sourceware.org Bugzilla #14048, Jkummerow (jkummerow) wrote on 2012-05-03:

#3

Indeed, with -fno-builtin the provided test case fails regardless of -O{1,2,3}.

Revision history for this message

In Sourceware.org Bugzilla #14048, Jsm28 (jsm28) wrote on 2012-06-01:

#4

Fixed for 2.16 by:

commit c5bfe3d5ba29d36563f1e4bd4f8d7336093ee6fc
Author: Joseph Myers <email address hidden>
Date: Fri Jun 1 19:05:46 2012 +0000

Fix fmod for subnormals (bug 14048).

Carlos, I think this should go on 2.15 branch as well.

Revision history for this message

In Sourceware.org Bugzilla #14048, Jkummerow (jkummerow) wrote on 2012-06-02:

#5

Thanks!

I have recompiled glibc-2.15 on my machine with that patch applied manually and
can confirm that it fixes the issue I was seeing.

Revision history for this message

In Sourceware.org Bugzilla #14048, Jsm28 (jsm28) wrote on 2012-06-08:

#6

Carlos, the 2.15 backport should also include

commit 1b671feb6115afbc90a7b37be4929d3e0394f311
Author: Adhemerval Zanella <email address hidden>
Date: Tue Jun 5 21:31:24 2012 -0300

Fix for wrong ldbl128-ibm fmodl commit

commit 34ae0b3270c67cae0c54ac98b693fdf7d010a206
Author: Adhemerval Zanella <email address hidden>
Date: Tue Jun 5 10:13:41 2012 -0300

Fix ldbl128ibm fmodl for subnormals.

to avoid the new tests failing for powerpc.

Revision history for this message

In Sourceware.org Bugzilla #14048, Jsm28 (jsm28) wrote on 2012-06-25:

#7

Created attachment 6481
Backport 1/3

Carlos, please review this series of three backports for 2.15 branch. Tested x86_64.

Revision history for this message

In Sourceware.org Bugzilla #14048, Jsm28 (jsm28) wrote on 2012-06-25:

#8

Created attachment 6482
Backport 2/3

Revision history for this message

In Sourceware.org Bugzilla #14048, Jsm28 (jsm28) wrote on 2012-06-25:

#9

Created attachment 6483
Backport 3/3

Revision history for this message

In Sourceware.org Bugzilla #14048, Carlos-odonell (carlos-odonell) wrote on 2012-06-25:

#10

(In reply to comment #6)
> Created attachment 6481 [details]
> Backport 1/3
>
> Carlos, please review this series of three backports for 2.15 branch. Tested
> x86_64.

All three backports look good to me for 2.15.

My only worry is that we break the testsuite for yet another target.

Shall we drop the testsuite addition from the backport?

Revision history for this message

In Sourceware.org Bugzilla #14048, Joseph-codesourcery (joseph-codesourcery) wrote on 2012-06-25:

#11

On Mon, 25 Jun 2012, carlos_odonell at mentor dot com wrote:

> Shall we drop the testsuite addition from the backport?

The most conservative backport would be just patch 1/3, without the
testsuite addition - 1/3 is fixing the regression in 2.15 (a regression
introduced by the addition of the wordsize-64 version), the other patches
are fixing a failure that showed up for powerpc with the new testcases but
as far as I know that powerpc failure is not a regression.

Revision history for this message

In Sourceware.org Bugzilla #14048, Carlos-odonell (carlos-odonell) wrote on 2012-06-25:

#12

(In reply to comment #10)
> On Mon, 25 Jun 2012, carlos_odonell at mentor dot com wrote:
>
> > Shall we drop the testsuite addition from the backport?
>
> The most conservative backport would be just patch 1/3, without the
> testsuite addition - 1/3 is fixing the regression in 2.15 (a regression
> introduced by the addition of the wordsize-64 version), the other patches
> are fixing a failure that showed up for powerpc with the new testcases but
> as far as I know that powerpc failure is not a regression.

The Power failure is still a failure and should be fixed.

Please checkin all three patches *without* the testsuite addition.

I want to avoid people working with a stable branch having testsuite failures show up out of thin air because we fixed a bug. That's OK for trunk, but not OK for a stable branch. The new testsuite failure is difficult and time-consuming for non-experts to diagnose. We hide this by fixing the bug, and using the regression test to verify the fix, but then dropping the testcase from the backport.

Revision history for this message

In Sourceware.org Bugzilla #14048, Jsm28 (jsm28) wrote on 2012-06-25:

#13

Fixed 2.15 (without testcase) by:

commit b640404bd8c9a281502ccc87ab2ed640c9b4c085
Author: Adhemerval Zanella <email address hidden>
Date: Tue Jun 5 21:31:24 2012 -0300

Fix for wrong ldbl128-ibm fmodl commit
(cherry picked from commit 1b671feb6115afbc90a7b37be4929d3e0394f311)

commit c95a1e5f35fa099eba9b2820b461edaaa7765539
Author: Adhemerval Zanella <email address hidden>
Date: Tue Jun 5 10:13:41 2012 -0300

Fix ldbl128ibm fmodl for subnormals.
(cherry picked from commit 34ae0b3270c67cae0c54ac98b693fdf7d010a206)

commit 2676061a91c99fa0b2633ceee881ea5bc31de4c2
Author: Joseph Myers <email address hidden>
Date: Fri Jun 1 19:05:46 2012 +0000

Fix fmod for subnormals (bug 14048).
(cherry picked from commit c5bfe3d5ba29d36563f1e4bd4f8d7336093ee6fc)

Bug Watch Updater (bug-watch-updater) on 2012-08-09

Changed in eglibc:
importance:	Unknown → Medium
status:	Unknown → Fix Released

Revision history for this message

Launchpad Janitor (janitor) wrote on 2012-08-09:

#14

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in eglibc (Ubuntu):
status:	New → Confirmed

Chris J Arges (arges) on 2012-08-13

Changed in eglibc (Ubuntu):
assignee:	nobody → Chris J Arges (christopherarges)
importance:	Undecided → Medium

Chris J Arges (arges) on 2012-08-16

Changed in eglibc (Ubuntu Precise):
status:	New → Confirmed
importance:	Undecided → Medium
assignee:	nobody → Chris J Arges (christopherarges)

Revision history for this message

Chris J Arges (arges) wrote on 2012-08-21:

#15

fix_lp1000498_quantal.debdiff Edit (12.9 KiB, text/plain)

Attached is a patch for quantal.

I have built a fix for testing in my PPA here:
https://launchpad.net/~christopherarges/+archive/ppa-test/+sourcepub/2614914/+listing-archive-extra

I've been able to run the testcase in #1 and before it results in NaN, and after the patch I see 0.

Changed in eglibc (Ubuntu Quantal):
status:	Confirmed → In Progress

Revision history for this message

Ubuntu Foundations Team Bug Bot (crichton) wrote on 2012-08-21:

#16

The attachment "fix_lp1000498_quantal.debdiff" of this bug report has been identified as being a patch in the form of a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. In the event that this is in fact not a patch you can resolve this situation by removing the tag 'patch' from the bug report and editing the attachment so that it is not flagged as a patch. Additionally, if you are member of the ubuntu-sponsors team please also unsubscribe the team from this bug report.

[This is an automated message performed by a Launchpad user owned by Brian Murray. Please contact him regarding any issues with the action taken in this bug report.]

tags:

added: patch

Revision history for this message

Chris J Arges (arges) wrote on 2012-08-27:

#17

Here is a fix for Precise in my PPA as well.
https://launchpad.net/~christopherarges/+archive/ppa-test/+sourcepub/2623069/+listing-archive-extra

Changed in eglibc (Ubuntu Precise):
status:	Confirmed → In Progress

Revision history for this message

Chris J Arges (arges) wrote on 2012-08-27:

#18

eglibc-precise-fix-lp1000498.debdiff Edit (6.9 KiB, text/plain)

Revision history for this message

Chris J Arges (arges) wrote on 2012-08-27:

#19

eglibc-quantal-fix-lp1000498.debdiff Edit (6.9 KiB, text/plain)

Revision history for this message

Bryce Harrington (bryce) wrote on 2012-09-13:

#20

The change looks sane but please fill in the required SRU data (I've pasted the fields in the description for you) so we can get this done for precise.

description:	updated
Changed in eglibc (Ubuntu Precise):
status:	In Progress → Incomplete

Revision history for this message

Chris J Arges (arges) wrote on 2012-09-13:

#21

Ok changes made. Thanks

description:	updated
Changed in eglibc (Ubuntu Precise):
status:	Incomplete → In Progress

Adam Conrad (adconrad) on 2012-10-03

Changed in eglibc (Ubuntu Precise):
assignee:	Chris J Arges (christopherarges) → Adam Conrad (adconrad)
Changed in eglibc (Ubuntu Quantal):
assignee:	Chris J Arges (christopherarges) → Adam Conrad (adconrad)

Adam Conrad (adconrad) on 2012-10-04

Changed in eglibc (Ubuntu Quantal):
status:	In Progress → Fix Committed

Revision history for this message

Launchpad Janitor (janitor) wrote on 2012-10-04:

#22

This bug was fixed in the package eglibc - 2.15-0ubuntu20

---------------
eglibc (2.15-0ubuntu20) quantal; urgency=low

* Backport fixes for dbl-64 and ldbl-128 issues (LP: #1000498)
* Backport another FMA support patch from glibc master branch.

eglibc (2.15-0ubuntu19) quantal-proposed; urgency=low

  * SECURITY UPDATE: stack buffer overflow in vfprintf handling
    (LP: #1031301)
    - debian/patches/any/CVE-2012-3406.patch: switch to malloc when
      array grows too large to handle via alloca extension
    - CVE-2012-3406
  * SECURITY UPDATE: stdlib strtod integer/buffer overflows
    - debian/patches/any/CVE-2012-3480.patch: rearrange calculations
      and modify types to void integer overflows
    - CVE-2012-3480
-- Adam Conrad <email address hidden> Wed, 03 Oct 2012 15:58:02 -0600

Changed in eglibc (Ubuntu Quantal):
status:	Fix Committed → Fix Released

Revision history for this message

Brian Murray (brian-murray) wrote on 2012-10-05: Please test proposed package

#23

Hello Thomas, or anyone else affected,

Accepted eglibc into precise-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/eglibc/2.15-0ubuntu10.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please change the bug tag from verification-needed to verification-done. If it does not, change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in eglibc (Ubuntu Precise):
status:	In Progress → Fix Committed
tags:	added: verification-needed

Revision history for this message

Adam Conrad (adconrad) wrote on 2012-10-08:

#24

Verified with the test-case that this is fixed in precise-proposed.

tags:

added: verification-done
removed: verification-needed

Revision history for this message

Martin Pitt (pitti) wrote on 2012-10-08:

#25

Unsubscribing sponsors.

Revision history for this message

Alec Warner (antarus) wrote on 2012-10-16:

#26

its verified, please release it :)

Revision history for this message

Colin Watson (cjwatson) wrote on 2012-10-18: Update Released

#27

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.

Revision history for this message

Launchpad Janitor (janitor) wrote on 2012-10-18:

#28

This bug was fixed in the package eglibc - 2.15-0ubuntu10.3

---------------
eglibc (2.15-0ubuntu10.3) precise; urgency=low

* Backport fixes for dbl-64 and ldbl-128 issues (LP: #1000498)
* Backport another FMA support patch from glibc master branch.

eglibc (2.15-0ubuntu10.2) precise-security; urgency=low

  * SECURITY UPDATE: stack buffer overflow in vfprintf handling
    (LP: #1031301)
    - debian/patches/any/CVE-2012-3406.patch: switch to malloc when
      array grows too large to handle via alloca extension
    - CVE-2012-3406
  * SECURITY UPDATE: stdlib strtod integer/buffer overflows
    - debian/patches/any/CVE-2012-3480.patch: rearrange calculations
      and modify types to void integer overflows
    - CVE-2012-3480

eglibc (2.15-0ubuntu10.1) precise; urgency=low

  * Backport fix from 2.16 to fix htons() conversion errors on non-x86
    architectures, by correctly casting to uint16_t (LP: #1016349)
  * Restore missing AT_EMPTY_PATH definition in fnctl.h (LP: #1010069)
  * Backport FMA4/AVX detection from glibc 2.16 (LP: #956051, #979003)
  * Backport fixups to AVX-using code to match the detection backport.
  * Backport fix from 2.16 for sscanf/realloc deadlock (LP: #1028038)
  * Backport for bogus FPE on underflow for exp(double) (LP: #1007457)
-- Adam Conrad <email address hidden> Wed, 03 Oct 2012 15:58:02 -0600

Changed in eglibc (Ubuntu Precise):
status:	Fix Committed → Fix Released

Revision history for this message

In Sourceware.org Bugzilla #14048, Jackie-rosen (jackie-rosen) wrote on 2014-02-16:

#29

*** Bug 260998 has been marked as a duplicate of this bug. ***
Seen from the domain http://volichat.com
Page where seen: http://volichat.com/adult-chat-rooms
Marked for reference. Resolved as fixed @bugzilla.

Ubuntu
eglibc package

fmod() incorrectly returns NaN for (some?) denormalized inputs

Bug Description

Related branches

CVE References

Other bug subscribers

Patches

Remote bug watches

	Status	Importance	Assigned to
eglibc	Fix Released	Medium	sourceware-bugs #14048
eglibc (Ubuntu)	Fix Released	Medium	Adam Conrad
Precise	Fix Released	Medium	Adam Conrad
Quantal	Fix Released	Medium	Adam Conrad

Ubuntueglibc package

fmod() incorrectly returns NaN for (some?) denormalized inputs

Bug Description

Related branches

CVE References

Other bug subscribers

Patches

Remote bug watches

Ubuntu
eglibc package