Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2006-10

Rounding inconsistent (critical)

10 messages · 3 people ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Graeme Geldenhuys

Hi,

I created the follow fpcUnit tests. For those that don't know fpcUnit, the AssertEquals takes three parameters: Failing message, Expected Value, Actual Value.

Now from the tests below, I am formatting all values to 2 decimal values. I expected all tests to pass, but it doesn't. Test 9 fails and said it expected 0.00 which is wrong.

Rules for rounding. If a decimal is exactly halfway (5) or more, rounding goes up. If decimal is < 5, rounding goes down. This rules is proven by tests 5, 6, 7 and 8. So why is test 9 failing? Is this a bug?

  AssertEquals('Failing on 1', '0.10', FormatFloat('0.00', 0.1));
  AssertEquals('Failing on 2', '0.01', FormatFloat('0.00', 0.01));
  AssertEquals('Failing on 3', '0.00', FormatFloat('0.00', 0.001));
  AssertEquals('Failing on 4', '0.50', FormatFloat('0.00', 0.5));
  AssertEquals('Failing on 5', '0.01', FormatFloat('0.00', 0.0051));
  AssertEquals('Failing on 6', '0.05', FormatFloat('0.00', 0.049));
  AssertEquals('Failing on 7', '0.00', FormatFloat('0.00', 0.0049));
  AssertEquals('Failing on 8', '0.05', FormatFloat('0.00', 0.045));
  AssertEquals('Failing on 9', '0.01', FormatFloat('0.00', 0.005));

---------------- Test Output -------------------
[graemeg-linux] FloatToStr > ./fpcunit_formatfloat -a --format=plain Test: TTestFormatFloat.TestFormat1
Failing on 9 expected: <0.01> but was: <0.00>

List of failures:
Failure:
Message: TTestFormatFloat.TestFormat1: Failing on 9 expected: <0.01> but was: <0.00>
Exception class: EAssertionFailedError
Exception message: Failing on 9 expected: <0.01> but was: <0.00>

------------------ end -----------------------------

I am using FPC 2.1.1 (r4642) under Linux. Considering that I am writing Accounting Software, I am taking this as a critical bug!

Regards,
- Graeme -

Reply by email

Vincent Snijders
to Graeme Geldenhuys
Graeme Geldenhuys wrote 24 lines
Hi, I created the follow fpcUnit tests. For those that don't know fpcUnit, the AssertEquals takes three parameters: Failing message, Expected Value, Actual Value. Now from the tests below, I am formatting all values to 2 decimal values. I expected all tests to pass, but it doesn't. Test 9 fails and said it expected 0.00 which is wrong. Rules for rounding. If a decimal is exactly halfway (5) or more, rounding goes up. If decimal is < 5, rounding goes down. This rules is proven by tests 5, 6, 7 and 8. So why is test 9 failing? Is this a bug? AssertEquals('Failing on 1', '0.10', FormatFloat('0.00', 0.1)); AssertEquals('Failing on 2', '0.01', FormatFloat('0.00', 0.01)); AssertEquals('Failing on 3', '0.00', FormatFloat('0.00', 0.001)); AssertEquals('Failing on 4', '0.50', FormatFloat('0.00', 0.5)); AssertEquals('Failing on 5', '0.01', FormatFloat('0.00', 0.0051)); AssertEquals('Failing on 6', '0.05', FormatFloat('0.00', 0.049)); AssertEquals('Failing on 7', '0.00', FormatFloat('0.00', 0.0049)); AssertEquals('Failing on 8', '0.05', FormatFloat('0.00', 0.045)); AssertEquals('Failing on 9', '0.01', FormatFloat('0.00', 0.005));

It is not a bug, it is using bankers rounding. See http://lazarus-ccr.sourceforge.net/docs/rtl/system/round.html

Vincent

Reply by email

Graeme Geldenhuys
to Vincent Snijders
Vincent Snijders wrote
It is not a bug, it is using bankers rounding. See http://lazarus-ccr.sourceforge.net/docs/rtl/system/round.html Vincent

Exactly, but it is still wrong, except if I really don't know how Banker's rounding works and what they mean by Even Number.

Take these to new tests.

  AssertEquals('Failing on 10', '0.44', FormatFloat('0.00', 0.445));
  AssertEquals('Failing on 11', '0.46', FormatFloat('0.00', 0.455));

Test 10:
Fails and said it expected 0.45 instead of my answer 0.44. Why? This is how I came to my answer of 0.44. I ignore the "0." and only look at 445 and I want that to 2 decimal digits. Lets now do what the documentation says and go to the closest even number. 44 is even, 45 is not. So I would have thought it must round down to 0.44. It doesn't.

Test 11:
Fails and said it expected 0.45 instead of my answer 0.46. Again, the way I rounded it. Ignore the "0.' and only want it to 2 decimal digits. What is the closest to a even number 45 or 46. In this case, 46. Hence I expected it to round up and the final answer to be 0.46. It doesn't.

I'm confused!! :-)

---- Under Delph 7 I got this -----------
Failing on 9: Expected <0.00> but got <0.01>

Failing on 10: Expected <0.44> but got <0.45>
------------------ end --------------------

So what rounding does Delphi 7 use?

Regards,
- Graeme -

Reply by email

Graeme Geldenhuys
to Graeme Geldenhuys

Oh, I followed the rules as define for Banker's Rounding (round-to-even) show on Wikipedia

http://en.wikipedia.org/wiki/Rounding

Use that to round 0.445 and 0.455. I gather that Delphi 7 defaults to Common Rounding and not Banker's Rounding.

Should FPC follow the Delphi rules on this to stay compatible?

Regards,
- Graeme -

Graeme Geldenhuys wrote 43 lines
On 02/10/06, Vincent Snijders wrote: > It is not a bug, it is using bankers rounding. See > http://lazarus-ccr.sourceforge.net/docs/rtl/system/round.html > > Vincent Exactly, but it is still wrong, except if I really don't know how Banker's rounding works and what they mean by Even Number. Take these to new tests. AssertEquals('Failing on 10', '0.44', FormatFloat('0.00', 0.445)); AssertEquals('Failing on 11', '0.46', FormatFloat('0.00', 0.455)); Test 10: Fails and said it expected 0.45 instead of my answer 0.44. Why? This is how I came to my answer of 0.44. I ignore the "0." and only look at 445 and I want that to 2 decimal digits. Lets now do what the documentation says and go to the closest even number. 44 is even, 45 is not. So I would have thought it must round down to 0.44. It doesn't. Test 11: Fails and said it expected 0.45 instead of my answer 0.46. Again, the way I rounded it. Ignore the "0.' and only want it to 2 decimal digits. What is the closest to a even number 45 or 46. In this case, 46. Hence I expected it to round up and the final answer to be 0.46. It doesn't. I'm confused!! :-) ---- Under Delph 7 I got this ----------- Failing on 9: Expected <0.00> but got <0.01> Failing on 10: Expected <0.44> but got <0.45> ------------------ end -------------------- So what rounding does Delphi 7 use? Regards, - Graeme -
There's no place like 127.0.0.1

Reply by email

Vincent Snijders
to Graeme Geldenhuys: Exactly, but it is still wrong, except if I really don't know how Banker's rounding works…
Graeme Geldenhuys wrote 21 lines
On 02/10/06, Vincent Snijders wrote: > It is not a bug, it is using bankers rounding. See > http://lazarus-ccr.sourceforge.net/docs/rtl/system/round.html > > Vincent Exactly, but it is still wrong, except if I really don't know how Banker's rounding works and what they mean by Even Number. Take these to new tests. AssertEquals('Failing on 10', '0.44', FormatFloat('0.00', 0.445)); AssertEquals('Failing on 11', '0.46', FormatFloat('0.00', 0.455)); Test 10: Fails and said it expected 0.45 instead of my answer 0.44. Why? This is how I came to my answer of 0.44. I ignore the "0." and only look at 445 and I want that to 2 decimal digits. Lets now do what the documentation says and go to the closest even number. 44 is even, 45 is not. So I would have thought it must round down to 0.44. It doesn't.

by "expected 0.45" you mean "got 0.45", otherwise I don't understand your reasoning.

I don't have an explanation, maybe it is due to rounding errors. 0.445 can not be represented exactly using floating point binary numbers.

Can you see if
   AssertEquals('Failing on 10', '0.12', FormatFloat('0.00', 0.125));
and
   AssertEquals('Failing on 10', '0.38', FormatFloat('0.00', 0.375));

work? These number can be represented exactly.

Show quoted text 9 lines
Test 11: Fails and said it expected 0.45 instead of my answer 0.46. Again, the way I rounded it. Ignore the "0.' and only want it to 2 decimal digits. What is the closest to a even number 45 or 46. In this case, 46. Hence I expected it to round up and the final answer to be 0.46. It doesn't. I'm confused!! :-)

I

Show quoted text 8 lines
---- Under Delph 7 I got this ----------- Failing on 9: Expected <0.00> but got <0.01> Failing on 10: Expected <0.44> but got <0.45> ------------------ end -------------------- So what rounding does Delphi 7 use?

No idea.

Vincent

Reply by email

Graeme Geldenhuys
to Vincent Snijders
Vincent Snijders wrote
> AssertEquals('Failing on 10', '0.44', FormatFloat('0.00', 0.445)); > AssertEquals('Failing on 11', '0.46', FormatFloat('0.00', 0.455));

Sorry, I shouldn't have used the work expected in my description, as that does confuse things a bit. You are right with your assumption though.

Test 10: I expected 0.44 but got 0.45 from FormatFloat() Test 11: I expected 0.46 but got 0.45 from FormatFloat()

Now using your tests:

  AssertEquals('Failing on 6', '0.12', FormatFloat('0.00', 0.125));
  AssertEquals('Failing on 7', '0.38', FormatFloat('0.00', 0.375));

Test 6: Passed
Test 7: Failing on 7 expected: <0.38> but was: <0.37>

I gather that this confirms a bug in FPC's banker's rounding.

Regards,
- Graeme -

Reply by email

Graeme Geldenhuys
to Graeme Geldenhuys

I also ran the exact example given on the Wikipedia site and I also got errors.

  AssertEquals('Failing on 1', '3.02', FormatFloat('0.00', 3.016));
  AssertEquals('Failing on 2', '3.01', FormatFloat('0.00', 3.013));
  AssertEquals('Failing on 3', '3.02', FormatFloat('0.00', 3.015));
  AssertEquals('Failing on 4', '3.04', FormatFloat('0.00', 3.045));
  AssertEquals('Failing on 5', '3.05', FormatFloat('0.00', 3.04501));

Test 1: Passed
Test 2: Passed
Test 3: Passed
Test 4: Failing on 4 expected: <3.04> but was: <3.05> Test 5: Passed

Regards,
- Graeme -

Graeme Geldenhuys wrote 26 lines
On 02/10/06, Vincent Snijders wrote: > > AssertEquals('Failing on 10', '0.44', FormatFloat('0.00', 0.445)); > > AssertEquals('Failing on 11', '0.46', FormatFloat('0.00', 0.455)); Sorry, I shouldn't have used the work expected in my description, as that does confuse things a bit. You are right with your assumption though. Test 10: I expected 0.44 but got 0.45 from FormatFloat() Test 11: I expected 0.46 but got 0.45 from FormatFloat() Now using your tests: AssertEquals('Failing on 6', '0.12', FormatFloat('0.00', 0.125)); AssertEquals('Failing on 7', '0.38', FormatFloat('0.00', 0.375)); Test 6: Passed Test 7: Failing on 7 expected: <0.38> but was: <0.37> I gather that this confirms a bug in FPC's banker's rounding. Regards, - Graeme -
There's no place like 127.0.0.1

Reply by email

Chris Cheney
to Graeme Geldenhuys

Date sent: Mon, 2 Oct 2006 12:59:45 +0200
From: "Graeme Geldenhuys"
To: "FPC developers' list"
Subject: Re: [fpc-devel] Rounding inconsistent (critical) Send reply to: FPC developers' list

Show quoted text 14 lines
I also ran the exact example given on the Wikipedia site and I also got errors. AssertEquals('Failing on 1', '3.02', FormatFloat('0.00', 3.016)); AssertEquals('Failing on 2', '3.01', FormatFloat('0.00', 3.013)); AssertEquals('Failing on 3', '3.02', FormatFloat('0.00', 3.015)); AssertEquals('Failing on 4', '3.04', FormatFloat('0.00', 3.045)); AssertEquals('Failing on 5', '3.05', FormatFloat('0.00', 3.04501)); Test 1: Passed Test 2: Passed Test 3: Passed Test 4: Failing on 4 expected: <3.04> but was: <3.05> Test 5: Passed

See, for example,

http://support.microsoft.com/default.aspx?scid=kb;EN-US;q42980

(there are plenty of other web sites explaining this sort of thing)

The short version is that very few numbers expressed as a decimal fraction are representable ACCURATELY on a computer.

BTW Delphi uses Banker's Rounding by default.

Reply by email

Vincent Snijders
to Chris Cheney
Chris Cheney wrote 32 lines
Date sent: Mon, 2 Oct 2006 12:59:45 +0200 From: "Graeme Geldenhuys" To: "FPC developers' list" Subject: Re: [fpc-devel] Rounding inconsistent (critical) Send reply to: FPC developers' list > I also ran the exact example given on the Wikipedia site and I also got errors. > > AssertEquals('Failing on 1', '3.02', FormatFloat('0.00', 3.016)); > AssertEquals('Failing on 2', '3.01', FormatFloat('0.00', 3.013)); > AssertEquals('Failing on 3', '3.02', FormatFloat('0.00', 3.015)); > AssertEquals('Failing on 4', '3.04', FormatFloat('0.00', 3.045)); > AssertEquals('Failing on 5', '3.05', FormatFloat('0.00', 3.04501)); > > > Test 1: Passed > Test 2: Passed > Test 3: Passed > Test 4: Failing on 4 expected: <3.04> but was: <3.05> > Test 5: Passed See, for example, http://support.microsoft.com/default.aspx?scid=kb;EN-US;q42980 (there are plenty of other web sites explaining this sort of thing) The short version is that very few numbers expressed as a decimal fraction are representable ACCURATELY on a computer.

But 0.125 and 0.375 are representable ACCURATELY on a computer, aren't they?

Vincent

Reply by email

Graeme Geldenhuys
to Vincent Snijders
Vincent Snijders wrote
But 0.125 and 0.375 are representable ACCURATELY on a computer, aren't they? Vincent

Yes they can, so FPC still has a bug in Banker's Rounding.

What is really interresting is that I never really thought about irrational number and how they are represented in computers. From that article in MSDN, here is a few basic calulations that don't work on computers. Multiply is the only test that worked! All other tests didn't. :-)

var
  sumE: extended;
  sumD: double;
  sumS: single;
  i: integer;
  t1, t2, t3: single;
begin
  write('multiply: ');
  writeln(0.0001 * 10000);
  writeln('=========');
  // should be 1.0 in theory
  sumE := 0;
  for I := 1 to 10000 do
    sumE := sumE + 0.0001;
  write('sum E: ');
  writeln(sumE);
  writeln('=========');
  // should be 1.0 in theory
  sumD := 0;
  for I := 1 to 10000 do
    sumD := sumD + 0.0001;
  write('sum D: ');
  writeln(sumD);
  writeln('=========');
  // should be 1.0 in theory
  sumS := 0;
  for I := 1 to 10000 do
    sumS := sumS + 0.0001;
  write('sum S: ');
  writeln(sumS);
  writeln('=========');
  writeln(' ');
  writeln('testing the follow:  69.82 = 69.20 + 0.62 ');
  t1 := 69.82;
  t2 := 69.20;
  t3 := 0.62;
  writeln(t1);
  writeln(t2);
  writeln(t3);
  if t1 = (t2 + t3) then
    writeln('equality')
  else
    writeln('non equals');   <=== prints this!!!
  writeln('=========');

Regards,
- Graeme -

Reply by email