Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2016-01

Is the TFPGMap.KeyData()/TDictionary.TryGetValue() faster than TStringList.Values[]?

9 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

silvioprog

Hello,

I need two fast list features:

Add(Key: string, Value: string);
Get(Key: string, Value: string): string;

So, I took a look at the `TFPGMap` (and `TDictionary` on Delphi) class. It is really a very nice option. But I tested the `TStringList` class too, and I was suprised with its performance. Please, run this small test in your environment (my env. is: FPC 3.1.1 / Delphi Seattle, both generating a 32 bits EXE):

http://pastebin.com/vxwhD9W0

FPC result:

TStringList: 00:00:14.988
TMyList: 00:00:00.187
TMyList: 00:00:00.219
TStringList: 00:00:15.204
Press [ENTER] to exit ...

*(Delphi result:*
*TStringList: 00:00:10.859*
*TMyList: 00:00:00.016*
*TMyList: 00:00:00.003*
*TStringList: 00:00:11.453*
*Press [ENTER] to exit ...**)*

Is this test wrong, or is the `TStringList.Values[]` really slow?

Thank you!

Silvio Clécio
silvioprog
to silvioprog

On Wed, Jan 13, 2016 at 11:34 PM, silvioprog wrote: [...]

Show quoted text 5 lines
Please, run this small test in your environment (my env. is: FPC 3.1.1 / Delphi Seattle, both generating a 32 bits EXE): http://pastebin.com/vxwhD9W0

Please disconsider this link above, the correct link is: http://pastebin.com/SUxJ1kKc.

(New results:

FPC - TFPGMap:

TStringList: 00:00:03.677
TMyList: 00:00:00.915
TMyList: 00:00:00.915
TStringList: 00:00:03.693

Delphi - TDictionary:

TStringList: 00:00:02.703
TMyList: 00:00:00.003
TMyList: 00:00:00.002
TStringList: 00:00:02.859)

Silvio Clécio
Michael Van Canneyt
to silvioprog: I need two fast list features: Add(Key: string, Value: string); Get(Key: string, Value:…
silvioprog wrote 31 lines
Hello, I need two fast list features: Add(Key: string, Value: string); Get(Key: string, Value: string): string; So, I took a look at the `TFPGMap` (and `TDictionary` on Delphi) class. It is really a very nice option. But I tested the `TStringList` class too, and I was suprised with its performance. Please, run this small test in your environment (my env. is: FPC 3.1.1 / Delphi Seattle, both generating a 32 bits EXE): http://pastebin.com/vxwhD9W0 FPC result: TStringList: 00:00:14.988 TMyList: 00:00:00.187 TMyList: 00:00:00.219 TStringList: 00:00:15.204 Press [ENTER] to exit ... *(Delphi result:* *TStringList: 00:00:10.859* *TMyList: 00:00:00.016* *TMyList: 00:00:00.003* *TStringList: 00:00:11.453* *Press [ENTER] to exit ...**)* Is this test wrong, or is the `TStringList.Values[]` really slow?

TStringList.Values is extremely slow, this is known. It performs a simple linear search.

- THashedStringList from inifiles should be faster for name=value.
- for simple key=value pairs, contrns contains various hash lists.

You can include them in your tests to get a more complete picture.

Michael.

Marco van de Voort
to Maciej Izak: My results for FPC and TDictionary from Generics.Collections (…

In our previous episode, Maciej Izak said:

> My results for FPC and TDictionary from Generics.Collections ( https://github.com/dathox/generics.collections ):

If you need the in-order capacity of tstringlist then you can use

http://www.stack.nl/~marcov/genlight.pas

- It is iterator based though, no array access to keys.
- Slower than most map solutions.
- faster than (ordered) tstringlist.
- Some tests indicate that the TStringlist slowdown is postponed from a few 100k to say 5M elements. (then you would need another level of indirection, though blocksize tweaking might buy you an order more)
- iirc duplicates are always ignores. I need to fix that proprely sometime

silvioprog
to Maciej Izak: My results for FPC and TDictionary from Generics.Collections (…
Maciej Izak wrote 19 lines
Please disconsider this link above, the correct link is: > http://pastebin.com/SUxJ1kKc. > > My results for FPC and TDictionary from Generics.Collections ( https://github.com/dathox/generics.collections ): TStringList: 00:00:03.644 TMyList: 00:00:00.007 TMyList: 00:00:00.007 TStringList: 00:00:03.708 Delphi XE2 : TStringList: 00:00:02.938 TMyList: 00:00:00.002 TMyList: 00:00:00.003 TStringList: 00:00:03.063

Nice. This - useful - class should be added in FCL. :-)

Silvio Clécio
silvioprog
to Michael Van Canneyt: TStringList.Values is extremely slow, this is known. It performs a simple linear search.…

On Thu, Jan 14, 2016 at 4:43 AM, Michael Van Canneyt <

Show quoted text 19 lines
wrote: On Wed, 13 Jan 2016, silvioprog wrote: > Hello, > > I need two fast list features: > > Add(Key: string, Value: string); > Get(Key: string, Value: string): string; [...] TStringList.Values is extremely slow, this is known. It performs a simple linear search. - THashedStringList from inifiles should be faster for name=value. - for simple key=value pairs, contrns contains various hash lists.

Very nice, I did not know that this classes existed.

You can include them in your tests to get a more complete picture.

I'm going to do it and get back soon showing the results ... :-)

Silvio Clécio
silvioprog
to Marco van de Voort: In our previous episode, Maciej Izak said: If you need the in-order capacity of…
Marco van de Voort wrote 8 lines
In our previous episode, Maciej Izak said: > > > My results for FPC and TDictionary from Generics.Collections ( > https://github.com/dathox/generics.collections ): If you need the in-order capacity of tstringlist then you can use http://www.stack.nl/~marcov/genlight.pas

It's a nice unit, and it should be available in FCL.

- It is iterator based though, no array access to keys.

Show quoted text 6 lines
- Slower than most map solutions. - faster than (ordered) tstringlist. - Some tests indicate that the TStringlist slowdown is postponed from a few 100k to say 5M elements. (then you would need another level of indirection, though blocksize tweaking might buy you an order more) - iirc duplicates are always ignores. I need to fix that proprely sometime

I need to do some tests. Thanks for share it. :-)

Silvio Clécio
silvioprog
to silvioprog: On Thu, Jan 14, 2016 at 4:43 AM, Michael Van Canneyt < Very nice, I did not know that…
silvioprog wrote
On Thu, Jan 14, 2016 at 4:43 AM, Michael Van Canneyt < > wrote:

[...]

Show quoted text 5 lines
You can include them in your tests to get a more complete picture. I'm going to do it and get back soon showing the results ... :-)

[...]

Done.

It seems that the best options (to fix my problem) are:

FPC: `Contnrs.TFPStringHashTable` (http://pastebin.com/s9pjK7TX). Delphi: `System.Generics.Collections.TDictionary` ( http://pastebin.com/AE2V6HZJ).

Both tests uses about 100k iterations. (my project uses about ~2k)

Thanks for indicate it! :-)

Silvio Clécio