Hi,

We have implemented a BaseObject class in the tiOPF project that can be reference counted or not. Based on the constructor used. What was really strange was when we started doing tests to see how it would affect our performance. Some of us using tiOPF have applications that creates 10's of 1000's of objects, so it does make a difference.

I've put together a console test application generating the results show below (the application creates as many objects as it can in 5 seconds). It also shows our interface implementation compared to TInterfacedObject. See attached files. One is the FPC program (*.lpr) and one is the Delphi 7 program (*.dpr) source.

Here is the strange behaviour we detected. Anybody got answers for these?

* The first two tests have opposite results under FPC vs Delphi. Why??

* What is so different in our interfaced class implementation that degrades performance that much!!??

* Why is our non-reference counted implementation faster than TObject under FPC? We even changed the order of tests to see if that makes a difference, but it didn't. See test result 3.

Speed test results....

***** FPC 2.2.0 under Linux (Ubuntu 7.10)
./performancetest
2029 iterations in 5 seconds (no reference counting) 3185 iterations in 5 seconds (reference counting)
2469 iterations in 5 seconds (no reference counting with MyBaseObject) 935 iterations in 5 seconds (reference counting with MyBaseObject)

***** Delphi 7 under Windows 2000
C:\programming\tests\performancetest>performancetest 2721 iterations in 5 seconds (no reference counting) 1309 iterations in 5 seconds (reference counting)
2255 iterations in 5 seconds (no reference counting with MyBaseObject) 652 iterations in 5 seconds (reference counting with MyBaseObject)

We even changed the order of tests to see if that makes a difference...

***** FPC 2.2.0 under Linux (Ubuntu 7.10)
./performancetest
2048 iterations in 5 seconds (no reference counting with MyBaseObject) 1874 iterations in 5 seconds (no reference counting) 2877 iterations in 5 seconds (reference counting)
566 iterations in 5 seconds (reference counting with MyBaseObject)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

* Why is our non-reference counted implementation faster

Reference counting needs some additional code that is executed without writing additional instructions in the main line code. So it's no wonder it costs some performance.

-Michael

Why are the iteration differences that big? 566 vs 935? There seems to be a problem with how you test/measure.

Mattias

Michael Schnell wrote

Reference counting needs some additional code that is executed without writing additional instructions in the main line code. So it's no wonder it costs some performance.

Yes that I understand and I'm not arguing the point. Well, that's actually what I expected. The thing is the performance is opposite when you compare FPC and Delphi using TInterfacedObject. Then looking at our MyBaseObject, the performance degraded a lot!!

* Why is our non-reference counted implementation faster than TObject under FPC? We even changed the order of tests to see if that makes a difference, but it didn't. See test result 3.

And again quoting my original mail. Why is our "non-reference counted" object which in theory should be the same as TObject, faster than TObject?? This result appears under FPC only.

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Mattias Gaertner wrote

Why are the iteration differences that big? 566 vs 935? There seems to be a problem with how you test/measure.

No idea?? They seem to fluctuate after each run, but the overall result (which tests are faster) are always constant. Have a look at the code, if there is a better way of testing this. The code is pretty simple. Just loop and create 10,000 objects in each loop.

./performancetest
2255 iterations in 5 seconds (no reference counting with MyBaseObject) 1857 iterations in 5 seconds (no reference counting) 2904 iterations in 5 seconds (reference counting)
318 iterations in 5 seconds (reference counting with MyBaseObject)

... waited 10 seconds and then ran it again...

./performancetest
2243 iterations in 5 seconds (no reference counting with MyBaseObject) 1890 iterations in 5 seconds (no reference counting) 2950 iterations in 5 seconds (reference counting)
416 iterations in 5 seconds (reference counting with MyBaseObject)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Your tests do not free the refcounted objects, but free the non-refcounted ones. This probably explains why 'standard refcounting' test is faster than 'standard non-refcounting'.
Still cannot say anything about custom BaseObject. Maybe you could figure out the showstopper by using callgrind.

Regards,
Sergei

Hi,

was just wondering if anyone was aware of a FreePascal (or delphi) port of either the Java or C++ RDesktop app that's used for making RDP connections ?

(www.rdesktop.org)

Cheers

Mark

Mattias Gaertner wrote

Why are the iteration differences that big? 566 vs 935? There seems to be a problem with how you test/measure.

Okay, I removed the 'for i := 1 to 10000' loop so that we only have a single loop (while ...) and create objects in that loop. Results seem to be more consistent now and the results of the MyBaseObject usage has changed dramatically. They are much closer together than with the 'for' loop. Weird that the 'for' loop would have made such a huge difference in timing!!

Another weird observation....

* The Delphi 7 test runs inside a VirtualBox session, which normally I would have thought would
make programs run slower. Even so, Delphi 7 in a VirtualBox creates 4+ million objects more
than FPC running in a native installed Ubuntu. Huge difference in speed!!!

* TInterfacedObject still beats TObject under FPC. (makes no sense) Not Delphi though.

* The results of TMyBaseObject with and without reference counting is opposite between the
two compilers. Again, it makes no sense.

What is going on??? :-)

Example test without for loop:

procedure TMyApplication.TestRefCountedObjects;
var
  LO: TInterfacedObject;
  LStart: Cardinal;
  LCount: Cardinal;
  i: integer;
begin
  LCount := 0;
  LStart := tiGetTickCount;
  while tiGetTickCount - LStart < (CTestRunTime * 1000) do
  begin
    LO := TInterfacedObject.Create;
    Inc(LCount);
  end;
  Log(Format('%s iterations in %d seconds (reference counting)',
      [IntToStr(LCount), CTestRunTime]));
end;

**** FPC 2.2.0 under Ubuntu 7.10 (native OS)

./performancetest
1782823 iterations in 5 seconds (no reference counting with TObject) 1989364 iterations in 5 seconds (reference counting with TInterfacedObject) 1878392 iterations in 5 seconds (no reference counting with MyBaseObject) 1848856 iterations in 5 seconds (reference counting with MyBaseObject)

./performancetest
1699145 iterations in 5 seconds (no reference counting with TObject) 1923287 iterations in 5 seconds (reference counting with TInterfacedObject) 1849223 iterations in 5 seconds (no reference counting with MyBaseObject) 1836972 iterations in 5 seconds (reference counting with MyBaseObject)

./performancetest
1876295 iterations in 5 seconds (no reference counting with TObject) 1868920 iterations in 5 seconds (reference counting with TInterfacedObject) 1840136 iterations in 5 seconds (no reference counting with MyBaseObject) 1835363 iterations in 5 seconds (reference counting with MyBaseObject)

**** Delphi 7 under Win2000 (in a VirtualBox session)

C:\programming\tests\performancetest>performancetest 5782111 iterations in 5 seconds (no reference counting with TObject) 5555045 iterations in 5 seconds (reference counting with TInterfacedObject) 4301854 iterations in 5 seconds (no reference counting with MyBaseObject) 4742419 iterations in 5 seconds (reference counting with MyBaseObject)

C:\programming\tests\performancetest>performancetest 6002846 iterations in 5 seconds (no reference counting with TObject) 5679972 iterations in 5 seconds (reference counting with TInterfacedObject) 4305841 iterations in 5 seconds (no reference counting with MyBaseObject) 4829425 iterations in 5 seconds (reference counting with MyBaseObject)

C:\programming\tests\performancetest>performancetest 6007751 iterations in 5 seconds (no reference counting with TObject) 5704604 iterations in 5 seconds (reference counting with TInterfacedObject) 4301758 iterations in 5 seconds (no reference counting with MyBaseObject) 4878739 iterations in 5 seconds (reference counting with MyBaseObject)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

I tested your tests and the results were completely random.
- They need so much memory that the OS page handling interferes.
- You used the slow 'Now' function.

Decrease the CTestRunTime to 1 and let the loops run 1000000 and you will get more reliable results:

$ ./performancetest
13 iterations in 1 seconds (no reference counting) 11 iterations in 1 seconds (reference counting)
9 iterations in 1 seconds (no reference counting with MyBaseObject) 8 iterations in 1 seconds (reference counting with MyBaseObject) $ ./performancetest
13 iterations in 1 seconds (no reference counting) 11 iterations in 1 seconds (reference counting)
9 iterations in 1 seconds (no reference counting with MyBaseObject) 7 iterations in 1 seconds (reference counting with MyBaseObject)

Mattias

Sergei Gorelkin wrote

> Your tests do not free the refcounted objects, but free the non-refcounted ones. This probably explains why 'standard refcounting' test is faster than 'standard non-refcounting'.

Valid point, so I did that. 2 out of 3 times TInterfacedObject still beat TObject. Using MyBaseObject stayed consistante. Non-Ref. being faster (which would be expected result).

./performancetest
1887748 iterations in 5 seconds (no reference counting with TObject) 1922471 iterations in 5 seconds (reference counting with TInterfacedObject) 1904732 iterations in 5 seconds (no reference counting with MyBaseObject) 1848976 iterations in 5 seconds (reference counting with MyBaseObject)

./performancetest
1852721 iterations in 5 seconds (no reference counting with TObject) 1887200 iterations in 5 seconds (reference counting with TInterfacedObject) 1877736 iterations in 5 seconds (no reference counting with MyBaseObject) 1729770 iterations in 5 seconds (reference counting with MyBaseObject)

./performancetest
1943921 iterations in 5 seconds (no reference counting with TObject) 1912681 iterations in 5 seconds (reference counting with TInterfacedObject) 1881897 iterations in 5 seconds (no reference counting with MyBaseObject) 1833352 iterations in 5 seconds (reference counting with MyBaseObject)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

A single program like the below leaks memory. To the object be freed automatically is necessary to declare Obj as IUnknown. I wonder if your test is leaking memory.

Luiz

program asmIntf;
{$Mode ObjFpc}
procedure DoIt;
var
  Obj: TInterfacedObject;
begin
  Obj := TInterfacedObject.Create;
end;
begin //Main
  DoIt;
end.

There used to be a front end by Tony Maro, I think ?

Michael.

Mattias Gaertner wrote

I tested your tests and the results were completely random. - They need so much memory that the OS page handling interferes. - You used the slow 'Now' function.

Okay, to eliminate both those, I changed the loop again. You were right about the OS page handling by the way. I heard the HDD page in some tests.

Test notes:
--------------------
- I free both Non-Ref. and Ref. counted objects.
- I now loop 1mil times and check the time taken.
- Now() gets called once before the loop and once after the loop to get the time taken.

As you can see the reference counted TInterfacedObject always seem to beat the non-ref. counted TObject. The use of MyBaseObject, yet again yields a predictable (what you would expect) and constant result.

./performancetest
1000000 iterations in 302 ms (no reference counting with TObject) 1000000 iterations in 247 ms (reference counting with TInterfacedObject) 1000000 iterations in 260 ms (no reference counting with MyBaseObject) 1000000 iterations in 336 ms (reference counting with MyBaseObject)

./performancetest
1000000 iterations in 316 ms (no reference counting with TObject) 1000000 iterations in 236 ms (reference counting with TInterfacedObject) 1000000 iterations in 253 ms (no reference counting with MyBaseObject) 1000000 iterations in 324 ms (reference counting with MyBaseObject)

./performancetest
1000000 iterations in 304 ms (no reference counting with TObject) 1000000 iterations in 245 ms (reference counting with TInterfacedObject) 1000000 iterations in 268 ms (no reference counting with MyBaseObject) 1000000 iterations in 317 ms (reference counting with MyBaseObject)

Example of test loop now.

procedure TMyApplication.TestRefCountedObjects;
var
  LO: TInterfacedObject;
  LStart: Cardinal;
  LCount: Cardinal;
  i: integer;
begin
  LCount := 0;
  LStart := tiGetTickCount;
  while LCount < 1000000 do
  begin
    LO := TInterfacedObject.Create;
    LO.Free;
    Inc(LCount);
  end;
  Log(Format('%s iterations in %d ms (reference counting with
TInterfacedObject)',
      [IntToStr(LCount), tiGetTickCount - LStart]));
end;

Any idea why Delphi in a virtualbox still beats FPC? Delphi seems considerably faster and it's in a virtual machine. Oh, and I'm using a stock standard single core Intel P4 CPU 2.40GHz and it hasn't got any special virtualization features built in.
Also not, that Delphi now yields a constant result. Reference counted classes are always slower.

C:\programming\tests\performancetest>performancetest 1000000 iterations in 181 ms (no reference counting with TObject) 1000000 iterations in 250 ms (reference counting with TInterfacedObject) 1000000 iterations in 221 ms (no reference counting with MyBaseObject) 1000000 iterations in 300 ms (reference counting with MyBaseObject)

C:\programming\tests\performancetest>performancetest 1000000 iterations in 180 ms (no reference counting with TObject) 1000000 iterations in 251 ms (reference counting with TInterfacedObject) 1000000 iterations in 229 ms (no reference counting with MyBaseObject) 1000000 iterations in 270 ms (reference counting with MyBaseObject)

C:\programming\tests\performancetest>performancetest 1000000 iterations in 189 ms (no reference counting with TObject) 1000000 iterations in 241 ms (reference counting with TInterfacedObject) 1000000 iterations in 210 ms (no reference counting with MyBaseObject) 1000000 iterations in 269 ms (reference counting with MyBaseObject)

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Luiz Americo Pereira Camara wrote

A single program like the below leaks memory. To the object be freed automatically is necessary to declare Obj as IUnknown. I wonder if your test is leaking memory.

That was an oversight, thanks! Point taken, I was using interfaced objects incorrectly.

Yet in the last few test modifications, I explicitly free both Reference Counted and non-Reference Counted objects, so that shouldn't cause memory leaks.

With all these tests, we have cleared up a few issues. One still outstanding is why TInterfacedObject performs better that TObject using FPC?

Regards,
- Graeme -

_______________________________________________
fpGUI - a cross-platform Free Pascal GUI toolkit
http://opensoft.homeip.net/fpgui/

Graeme Geldenhuys wrote

With all these tests, we have cleared up a few issues. One still outstanding is why TInterfacedObject performs better that TObject using FPC?

Probably something alignment-related which changes the cpu cache behaviour.

Jonas

Thanks, I've seen that in the past but I was after a port of the actual RDP client to save me reworking it completely unless someone know's of a 'magic' tool to convert the C++ code to Pascal (or most of it even) !

Quoting Michael Van Canneyt :

A tip: create a small app (like i provided) with interfaced (with automatic and manual free) and not interfaced objects and compile with -al. This will create a *.s file with the assembly code.

Another tip: if your gettickcount implementation is not using high definition counter (TSC), try using TZentimer of cpu unit (http://members.yline.com/~tom_at_work/index.html) or EpikTimer (from wiki)

PS: I was considering using Interfaced objects in a project but gave up after seeing the asm code.

Luiz

(Just an idea) You can cross-compile FPC test for Windows and run it under the same virtual machine. Then you'll have at least the same environment for both tests.

Sergei

Either you have still some measurement problems, or see Jonas hint about wrong alignment.

Attached is the current test.
Make sure you run the program one or two times before, so that the OS caching does not interfere:

$ ./performancetest
...OS caching polluted...
$ ./performancetest
...OS caching polluted...
$ ./performancetest
10000000 iterations in 1169 ms (no reference counting) 10000000 iterations in 815 ms (no reference counting) 10000000 iterations in 1005 ms (reference counting) 10000000 iterations in 1002 ms (reference counting) 10000000 iterations in 1160 ms (no reference counting with MyBaseObject) 10000000 iterations in 1154 ms (no reference counting with MyBaseObject) 10000000 iterations in 1404 ms (reference counting with MyBaseObject) 10000000 iterations in 1344 ms (reference counting with MyBaseObject) $ ./performancetest
10000000 iterations in 808 ms (no reference counting) 10000000 iterations in 803 ms (no reference counting) 10000000 iterations in 992 ms (reference counting) 10000000 iterations in 990 ms (reference counting) 10000000 iterations in 1294 ms (no reference counting with MyBaseObject) 10000000 iterations in 1172 ms (no reference counting with MyBaseObject) 10000000 iterations in 1447 ms (reference counting with MyBaseObject) 10000000 iterations in 1369 ms (reference counting with MyBaseObject)

Notes:
As you can see I had to run each 10 million times because FPC is so fast, even without optimizations and there is still an error of 0.1 ms from the various processes running on todays linux. Compiler flags: -S2cgi -OG1 -gl
Make also sure, that you have at least 500mb of free RAM. Otherwise you measure your disk speed.

Mattias