I have this simple little test. My expected answer for all the calculations are 0 (zero), but both FPC and Delphi give different results. Java is the only one that seems consistent regarding the results.

Can anybody explain this, especially the first 3 very small negative numbers that Delphi and FPC produces? I know not all floating point values can be stored exactly (or something like that), and when doing calculations, it might do a auto data type conversion. But is any of that taking place here?

The only positive I can see, is that FPC and Delphi are consistent. :-)

Here is the Object Pascal test:

===============[ TestOperatorPrecedence.pas ]=================
program TestOperatorPrecedence;
{$IFDEF FPC}
   {$mode objfpc}{$H+}
{$ELSE}
   {$apptype console}
{$ENDIF}
var
   a,b,c,d: double;
   ans: extended;
begin
   a := 1.0;
   b := 51009.9;
   c := 51009.9;
   d := 51009.9;
   writeln(b);
   writeln(c);
   writeln(d);
   writeln('---');
   ans := c - b * c / d;
   writeln(ans);
   ans := c - (b * c) / d;
   writeln(ans);
   ans := c - (b * c / d);
   writeln(ans);
   ans := c - b * (c / d);
   writeln(ans);
   ans := c - (b * (c / d));
   writeln(ans);
end.
==============================================================

And the results are:
C:\devel\tests\OperatorPrecedence

TestOperatorPrecedence.exe

5.10099000000000E+0004
5.10099000000000E+0004
5.10099000000000E+0004
---
-3.55271367880050E-0015
-3.55271367880050E-0015
-3.55271367880050E-0015
0.00000000000000E+0000
0.00000000000000E+0000

And here is the Java equivalent...

===============[ TestOperatorPrecedence.java ]================ public class TestOperatorPrecedence {

	public static void main(String[] args) {
		double b = 51009.9;
		double c = 51009.9;
		double d = 51009.9;
		double ans;
		System.out.println(b);
		System.out.println(c);
		System.out.println(d);
		System.out.println("-----");
		ans = c - b * c / d;
		System.out.println(ans);
		ans = c - (b * c) / d;
		System.out.println(ans);
		ans = c - (b * c / d);
		System.out.println(ans);
		ans = c - b * (c / d);
		System.out.println(ans);
		ans = c - (b * (c / d));
		System.out.println(ans);
	}
}

==============================================================

And the results are:
C:\devel\tests\OperatorPrecedence

java TestOperatorPrecedence

51009.9
51009.9
51009.9
-----
0.0
0.0
0.0
0.0
0.0

Regards,
Graeme

Am 03.10.2018 um 10:40 schrieb

I have this simple little test. My expected answer for all the calculations are 0 (zero), but both FPC and Delphi give different results. Java is the only one that seems consistent regarding the results.

Compile/run on the same architecture, use the same data types (and not extended for Object Pascal and double for Java).

With 3.0.4 x86_64-win64 I get (on i386-win32 you get this if you use the same data types and compile with -Cfsse2):

5.1009900000000001E+004
5.1009900000000001E+004
5.1009900000000001E+004
---
0.0000000000000000E+000
0.0000000000000000E+000
0.0000000000000000E+000
0.0000000000000000E+000
0.0000000000000000E+000

Hi Florian,

Thanks for your reply. Initially I had all the data types as Double in the Object Pascal example. I then thought that maybe there is some auto data-type conversion, and tried storing the result in a Extended type instead. It made no difference.

From your example, the only time there is a difference is when the -Cfsse2 is specified. What exactly does that do, because with or without that, the results vary.

I'm running 64-bit Windows 10 here, but using FPC 3.0.4 32-bit compiler. In Delphi I was using XE3 32-bit compiler as well.

See here... I changed all the data types to Double before I did this test:

C:\devel\tests\OperatorPrecedence

fpc -Criot -Cfsse3 TestOperatorPrecedence.pas

Free Pascal Compiler version 3.0.2 [2017/02/13] for i386 Copyright (c) 1993-2017 by Florian Klaempfl and others Target OS: Win32 for i386
Compiling TestOperatorPrecedence.pas
TestOperatorPrecedence.pas(10,3) Note: Local variable "a" is assigned but never used
Linking TestOperatorPrecedence.exe
36 lines compiled, 0.1 sec, 35312 bytes code, 2436 bytes data 1 note(s) issued

C:\devel\tests\OperatorPrecedence

TestOperatorPrecedence.exe

5.1009900000000001E+004
5.1009900000000001E+004
5.1009900000000001E+004
---
0.0000000000000000E+000
0.0000000000000000E+000
0.0000000000000000E+000
0.0000000000000000E+000
0.0000000000000000E+000

Perfect, the result I expected. Now the same code, but compiled without -Cfsse2 and the results change.

C:\devel\tests\OperatorPrecedence

fpc -Criot TestOperatorPrecedence.pas

Free Pascal Compiler version 3.0.2 [2017/02/13] for i386 Copyright (c) 1993-2017 by Florian Klaempfl and others Target OS: Win32 for i386
Compiling TestOperatorPrecedence.pas
TestOperatorPrecedence.pas(10,3) Note: Local variable "a" is assigned but never used
Linking TestOperatorPrecedence.exe
36 lines compiled, 0.1 sec, 35232 bytes code, 2436 bytes data 1 note(s) issued

C:\devel\tests\OperatorPrecedence

TestOperatorPrecedence.exe

5.1009900000000001E+004
5.1009900000000001E+004
5.1009900000000001E+004
---
-3.5527136788005009E-015
-3.5527136788005009E-015
-3.5527136788005009E-015
0.0000000000000000E+000
0.0000000000000000E+000

Sorry, but this makes no sense to me.

Regards,
Graeme

The explanation for the results is as follows:

the first three codings do the multiplication first, then the division. the last two first division, then multiplication. That makes the difference.

When division is done first, you get an intermediate result of 1.0 exactly. The subsequent multiplication yields 51009.9 exactly, so the difference is zero.
No risk for rounding errors.

When the multiplication is done first, you get a very large result which cannot
be stored exactly in the machine representation. When divided by 51009.9, the result differs by a little epsilon from 1.0, so the difference is shown as the final result.

Depending on the FP representation chosen, you will see the difference or not.
When I tested the program with my New Stanford Pascal compiler, I saw the different computation sequence in the generated code, but I didn't get a difference. But that's pure luck.

Example:

Multiplication first:

 LOC 21
 LOD R,1,416
 LOD R,1,408
 LOD R,1,416
 MPR
 LOD R,1,424
 DVR
 SBR
 STR R,1,432

Division first:

 LOC 23
 LOD R,1,416
 LOD R,1,408
 LOD R,1,416
 LOD R,1,424
 DVR
 MPR
 SBR
 STR R,1,432

The result was zero in both cases.

Kind regards

Bernd

Am 03.10.2018 um 10:40 schrieb

Am 03.10.2018 um 11:28 schrieb

In 32 Bit by default the x87 CPU is used. The x87 FPU uses extended for calculations and intermediate results (this is basically a difference with respect to all other FPUs having any significance during the last >20 years) by default. There are no instructions to force operations with single/double precisions, there is only a status which allows to switch to another precision, see below.

b * c / d calculated with extended precision differs from the plain double value of c.

To get the same results with the x87 FPU, add

uses
  math;

......

  SetPrecisionMode(pmDouble);

to your program.

Or:

change all variables to extended, so the constants are really stored with extended precision representing exactly the value 51009.9.

Florian Klämpfl wrote

really stored with extended precision representing exactly the value 51009.9.

This was sloopy wording by myself: the representation is not exact for extended either, but it must be actually the best possible presentation with extended precision.

Some corrections, see below ...

This is correct, because the same two input values go into the division. 1.0 can be represented correctly as FP value.

The subsequent multiplication yields 51009.9 exactly, so the difference is zero.

Wording needs improvement; the value is not 51009.9 exactly, but it is the nearest machine number, which is the same as the other operand (the same number multiplied by 1.0) of the subtraction, so the result will be zero.

Should be:
the result may differ by a little epsilon from the original machine representation
of 51009.9; this epsilon may be shown instead of zero.

this is in fact the real P-Code produced by the New Stanford compiler for the hypothetical stack machine; different for the two cases.

I don't know why you want to compare two floats, but you'd better use currency type. Float is for calculus, but comparing float1=float2 (or float1>float2) is rolling the dice. Obviously, the more precision, the less errors. But an accurate, intuitive result, is not guaranteed.

People who play with maths may use something like
function equal(const f1,f2:Extended; const Error:extended=1E-6):boolean;
begin
  Result:=(abs(f1-f2)<error);
end;

But for most applications,  using a function like "equal" (and "less", "lessOrEq" etc) everywhere is a burden that makes not sense. I think that every programing language should include a fixed precision type.  Freepascal has at least currency, that is four decimals, use it.

What does java does? I don't know. Perhaps it just rounds the output, try  System.out.println(ans==0.0). Perhaps it uses a high precision that *in this case* gets always 0.

But the question is that floating point representation can't store accurately many numbers. To begin with,  0.1 in base 10, is periodic number in binary 0.0001...001..001... so it has to truncate it, and when you truncate, the result depends of the order of operations, no matter the language or precision. So, it is matter of probability to get 1.0000 or 1.0001 or 0.9999, if you expect 1, psychologically 1.0001 looks better result than 0.9999. Dephi or Freepascal are doing nothing wrong.

This is one of the most useless collection of floating point myths I have seen since a long time.

With rolling a dice you mean, that the comparisons are only randomly correct or what)? Since the floating-point numbers are well-defined and exact (yes they are, and truncation/rounding errors are the results from former computations and/or the rounding of non-representable numbers). All operations are predictable, so there is no chance for random.

This function is non-sense and I doubt that 'math people' will use it. First: it uses only absolute errors, so 1e-7 and 1e-4000 are equal. Second: A tolerance of 1e-6 is ridiculous, given that the machine epsilon for 80-bit extended is 1.0842e-19.

What does java does? I don't know. Perhaps it just rounds the output, try System.out.println(ans==0.0). Perhaps it uses a high precision that *in this case* gets always 0.

As already said, you can get the same values as Java, if you use the same data types (double) and the same precision (53-bit) with Free Pascal (even with the X87 FPU)

5.1009900000000001E+004
5.1009900000000001E+004
5.1009900000000001E+004
---
0.0000000000000000E+000
0.0000000000000000E+000
0.0000000000000000E+000
0.0000000000000000E+000
0.0000000000000000E+000

But the question is that floating point representation can't store accurately many numbers. To begin with, 0.1 in base 10, is periodic number in binary 0.0001...001..001... so it has to truncate it,

No is this only true if you use the truncate rounding mode, which is not default (default is round-to-nearest/even).

and when you truncate, the result depends of the order of operations, no matter the language or precision. So, it is matter of probability to get 1.0000 or 1.0001 or 0.9999

Yes, but it is predictable and deterministic and no matter of probability.

To conclude: The reason for the differences is the usage of intermediate elevated precision (and this can occur also e.g. for C where it is also allowed to use intermediate higher precision). For Delphi/Free Pascal this phenomenon does not occur if you always compute with the precision appropriate to the data type, as e.g. with 64-bit/SSE.

Regards,
Wolfgang

AM wrote

This is one of the most useless collection of floating point myths I have seen since a long time.

The random comes from the input, not the output. If you know the numbers going in, you know the numbers coming out, certainly. But you rarely know the numbers going in , and most people don't know the deep details of what happens behind the scenes, and so sometimes some of those numbers going in give you surprising numbers coming out. Different implementations or different languages have different details, as well, which can also be surprising.

So the issue isn't technical, it is language. What was being said was vernacular, colloquial, imprecise, just to give you the impression that __unless you pay attention to the details__ the output you get could look "random" or appear to be a "crap shoot". I don't think the original poster meant that the results were truly random at all, only that if you don't take care to account for the details, you might get that impression.

Jeff.

Bernd Oppolzer wrote

The explanation for the results is as follows:

Thank you Bernd and Florian for your explanations. They were very useful indeed.

Regards,
Graeme

Santiago A. wrote

I don't know why you want to compare two floats, but you'd better use currency type.

I fully understand that. We do financial calculation up to 6 decimal places, so can't use Currency data type.

Our real issue was the different results using the same calculation. I thought order of precedence would apply in all cases, but couldn't fully understand the outcomes we received. But after reading Florian and Bernd's replies I now understand. The other issue was that our application is 32-bit, where 64-bit would not have had this issue.

Regards,
Graeme

Santiago A. wrote

What does java does? I don't know. Perhaps it just rounds the output, try  System.out.println(ans==0.0). Perhaps it uses a high precision that *in this case* gets always 0.

I investigated that too. Under Java, double is always 64-bit based. So I guess the same reason 64-bit FPC had no issues with the calculations either.

"double: The double data type is a double-precision 64-bit IEEE 754 floating point."

6 decimals, no currency that's a problem ;-)

With 64 bits, reaching a 6 decimals error is more difficult, but it is still an issue. In your example you have done a couple of operations and you have got an error of 1E-11. After a hundred of operations, you could reach the fatal 1E-6.
Once I had a problem like that, or integers, or floats. The best is rounding a lot in intermediate results, rounding before comparing and specify clearly the order:
i.e. if you add a list of items with a discount, you can get different results if you apply the discount to each item and sum, than if you sum the items and apply the discount to the total. will the difference be less then 1E-6? Depend on the numbers, and how many items you sum. So you must specify : "Discount will be applied to each item". 64bits is a lot of precision, but don't be overconfident, even in such case errors can skyrocket with divisions with small divisors and/or a lot of operations. Comparing to zero is always dangerous, you'd better round the number to 6 decimals before comparing.

Santiago A. wrote

> places, so can't use Currency data type. 6 decimals, no currency that's a problem ;-)

Yeah, tell me about it.

The product I work on is Royalties and Licensing across the globe for very large and medium sized companies. A single contract can span millions to billions of accounting lines, then you have to calculate royalties on all those and make sure afterwards that adding them all up still equals the original amounts. A huge pain and very lengthy process.

If it was up to me, I would have opted to converted all amounts to integers as early as possible, do the calculations and then divide by the correct amount to get the decimals required for reporting purposes. But then, this product was developed long before I joined the company, and changing it now would be a mammoth task. So for now, I've got to work with what I've got - as most of us has to do. ;-)

Regards,
Graeme

I examined the rounding problem with floating point arithmetic on different platforms
in the 1995 time frame, when I had to find a solution for a customer of mine (a large
insurance company) for the following problem: the insurance math package should
yield the same results for computations using double float values, although the computation
is done (using C) on very different platforms (for example Intel and IBM mainframe),
where the floating point formats are not compatible. The source of the problem, BTW,
is the binary or hexadecimal coding of the mantissa of the floating point values,
which doesn't matter when doing scientific calculations, but which is a problem,
when dealing with commercial problems and decimal rounding.

With some help from a math specialist, I found a solution (a rounding function
for floating point values) which produces the same (decimal) results in most cases,
no matter what the platform and the floating point representation is.

I then built this function (converted to Pascal) into the runtime of my New Stanford Pascal compiler, so that every FP value is implicitly rounded before
output (that is: before WRITE or WRITELN) to the number of decimal places requested by this WRITE or WRITELN. This way, the output results of WRITE and WRITELN of FP values after certain computations were the same on all platforms (again: IBM mainframe and Windows PC, for example), which was not the case before my extension.

The story is documented in more detail here (including the Pascal source code of the rounding function):

http://bernd-oppolzer.de/job9i032.htm

Have fun!

Another option: there are some libraries which support BCD arithmetic (binary coded decimals) ... and even platforms which support decimal arithmetic
directly; but this will probably be no option for you.

Kind regards

Bernd

Bernd Oppolzer:

The story is documented in more detail here (including the Pascal source code of the rounding function): http://bernd-oppolzer.de/job9i032.htm

Wirth introduced the capitalisation of keywords, and you have decided to invert his style and capitalise everything except keywords?