Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2024-03

Loss of precision with the overloaded Max() function

1 message · 1 person ·

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Alan Krause

An issue recently popped up that I thought I'd bring up as a potential issue. In short, if you call the Max() function with an integer and a double parameter (such as d := Max( 0, d ) ), then fpc appears to be using the Max( single, single ) overloaded function instead of Max( double, double ). This is causing a loss of precision once the double values get large enough. Here is a sample program which illustrates the issue:

program project1;

uses
Math
, SysUtils

  ;
var
  d : Double;
  s : Single;
begin
  d := 159279.49;
  writeln( '159279.49 stored in Double d = ' + FloatToStr( d ) );
  d := Max( 0, d );
  writeln( 'Max( 0, d ) stored in Double d = ' + FloatToStr( d ) );
  writeln();
  s := 159279.49;
  writeln( '159279.49 stored in Single s = ' + FloatToStr( s ) );
  s := Max( 0, s );
  writeln( 'Max( 0, s ) stored in Single s = ' + FloatToStr( s ) );
  writeln();
  d := s;
  writeln( 'Assigning the value of s to Double d = ' + FloatToStr( d ) );
end.

Doing a bit of research on this issue, I see that it was brought up on Stack Overflow back in 2018 here
<https://stackoverflow.com/questions/52020129/unexpected-result-from-max-function-in-freepascal>. We can work around the issue by sending in 0 as 0.0, which then calls the correct Max( double, double ) function.

Cheers,
Alan