Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2020-02

Possible bug in fpc_round_real for softfloat?

6 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Alexander Hofmann

Dear all,

while investigating a bug in an application designed for ARM with floating point emulation enabled, I stumbled upon the following problem:

When rounding large numbers to int64, some actually get rounded to something negative, e.g:

 round(1.5000000000000000E+018) => 1500000000000000000  round(1.5000005497558139E+018) => -491856199680

Tested with trunk and 3.0.0 on a raspberry pi (armsf).

At first I though it's specific to ARM, but it can be reproduced also on e.g. x86_64 by copying fpc_round_real from rtl/genmath.inc and using this directly. By the way, the above fractional numbers differ by only one bit, which is bit 32 (or the sign bit in a 32bit number).

I think the culprit is in line 1342 of genmath.inc, i.e.

            result:=((int64(hx) shl 32) or float64low(d)) shl (j0-52);

float64low(d) will return the lower 32 bit of the float as longint, where the sign is negative in case of the second number. The compiler seems to expand this to a 64bit signed integer, by keeping the number not the bits - thus the invalid result. If this line is changed to

            result:=((int64(hx) shl 32) or *dword*(float64low(d))) shl
(j0-52);

both floats are rounded correctly. It needs to be an unsigned number, it does not work with an explicit cast to int64.

Note that this code-path will only be put into action if the exponent of the base-two fractional number is larger 51 and the float thus lacks the fractional part. In my point of view, the rest of the code is not prone to this error; in line 1361 of genmath.inc the result of float64low is shifted right first, which will always leave a 0 at the position of the "sign bit".

I am, however, not sure if this is because my compilers all have been compiled e.g. with the wrong switches (leading to some obscure optimization) or if this is indeed an error. I didn't find anything about this on the list or the net.

Attached is a little program illustrating the problem, most of the code is a 1:1 copy from genmath.inc

With best regards,

Alex

Reply by email

Francisco Glover
to Alexander Hofmann

Thanks...

On Tue, Feb 18, 2020 at 8:36 PM Alexander Hofmann via fpc-devel <

Show quoted text 58 lines
wrote: Dear all, while investigating a bug in an application designed for ARM with floating point emulation enabled, I stumbled upon the following problem: When rounding large numbers to int64, some actually get rounded to something negative, e.g: round(1.5000000000000000E+018) => 1500000000000000000 round(1.5000005497558139E+018) => -491856199680 Tested with trunk and 3.0.0 on a raspberry pi (armsf). At first I though it's specific to ARM, but it can be reproduced also on e.g. x86_64 by copying fpc_round_real from rtl/genmath.inc and using this directly. By the way, the above fractional numbers differ by only one bit, which is bit 32 (or the sign bit in a 32bit number). I think the culprit is in line 1342 of genmath.inc, i.e. result:=((int64(hx) shl 32) or float64low(d)) shl (j0-52); float64low(d) will return the lower 32 bit of the float as longint, where the sign is negative in case of the second number. The compiler seems to expand this to a 64bit signed integer, by keeping the number not the bits - thus the invalid result. If this line is changed to result:=((int64(hx) shl 32) or *dword*(float64low(d))) shl (j0-52); both floats are rounded correctly. It needs to be an unsigned number, it does not work with an explicit cast to int64. Note that this code-path will only be put into action if the exponent of the base-two fractional number is larger 51 and the float thus lacks the fractional part. In my point of view, the rest of the code is not prone to this error; in line 1361 of genmath.inc the result of float64low is shifted right first, which will always leave a 0 at the position of the "sign bit". I am, however, not sure if this is because my compilers all have been compiled e.g. with the wrong switches (leading to some obscure optimization) or if this is indeed an error. I didn't find anything about this on the list or the net. Attached is a little program illustrating the problem, most of the code is a 1:1 copy from genmath.inc With best regards, Alex _______________________________________________ fpc-devel maillist - https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel

Reply by email

Alexander Hofmann
to thaddy

Hi,

thaddy wrote
Raspberry Pi, what OS, because you write armsf and the default on Raspbian (and other major distro's) is armhf.

Raspbian, which is armhf, yes.

The application is based on the hard-float ABI (just checked with readelf -h). Still it is using the soft-float implementation at least for round().

I came across this by debugging the application at assembly level and saw the call of fpc_round_real(). I don't know very much about the ARM instruction set. I reckon there's "round to integer" available but I don't know in which version of the ARM specification, and if it's used in FPC.

Also, I have to check if I accidentally activated the "-Ce" switch, although I though I checked that...

In the end, as I was able to reproduce the error on Linux x86_64, I though I'd better ask. Last time I dealt specifically with floating point emulation was on DOS with Turbo Pascal :-)

Best,

Alex

Reply by email