Hello all (and especially Paul Ishenin since it's your fix I'm referring to),

http://bugs.freepascal.org/view.php?id=7749

This was recently marked as fixed for gtk2 in the Lazarus trunk. I pulled from svn this morning and built everything and put together a quick test program.

Basically I created a form and dropped a TListView onto it and set the OwnerData property to True, then defined an OnData event like:

procedure TForm1.ListView1Data(Sender: TObject; Item: TListItem);
begin
  Item.Caption := 'Item ' + IntToStr(Item.Index + 1);
end;

Then in the FormCreate I set the Items.Count property to some value:

procedure TForm1.FormCreate(Sender: TObject);
begin
  ListView1.Items.Count := 10;
end;

I ran the program and sure enough, the list showed up with my items with the correct caption. Great!

However, for the purposes of testing, I then increased the count to 1000000, since a major purpose of a virtual list view is so you don't have to eat up all your memory storing the items.

To my dismay, when I ran the program, it ran correctly, but the memory usage of my program was huge compared to when I only had 10 items.

I put some writeln's in the OnData method and also discovered that that the OnData routine was called, on startup of my program, for each of my million items.

Am I doing something wrong? Is there some other way I'm supposed to set the number of items for my list rather than setting Items.Count? I tried looking at the revision changes
(http://svn.freepascal.org/cgi-bin/viewvc.cgi?view=rev&root=lazarus&revision=22789) and didn't see any other way.

I don't have a setup to test other widgetsets (windows, carbon, whatever). Is this the behavior for other widgetsets?

Anyway, I wanted to discuss it here before I comment on or reopen the issue to make sure I don't have a misunderstanding of the expected behavior. While it does technically work (it calls the OnData method) it seems one of the major draws of using a virtual control is defeated by the huge memory usage I'm seeing.

And, by the way, thanks so much for the work and effort of the Lazarus development team.

-SG

ListViews on gtk can't be absolutely virtual - they need to have all the items. But captions, checks, etc are not stored in it (in the case of virtual) - they are requested before the drawing. So you should have some memory reduction in comparition with the real listview.

Only windows has real virtual listview. Other widgetsets have only compatible interface (your first test), not the implementation (storage organisation - your second test).

Best regards,
Paul Ishenin.

Thanks for the explanation. If that's the way it is, that's the way it is!

-SG

I recently did a test where I wrote my own gtktreemodel (the container for gtk treeview/listview data) I wrapped it around a TStrings object so that the data was only stored in the StringList. This could be done for a listview as well with really only a small change.

Right now I think it works like this

1. GtkTreeView asks model (GtkListStore) for data
2. model finds pointer to TListItem and calls custom procedure to with (TListItem) to get the data it needs.
3. GtkTreeView paints item.

with our own treemodel it could be

1. View asks model(TLCLListViewModel) for data
2. Model acceses TListItems for the data it needs.
3. GtkTreeView paints item.

The only major benefit I see is there is not a secondary list stored of the TListItems.

I did a small test loading a text file with 50,000+ lines with my custom model and the GtkTreeView retrieved the value of each item in the list on startup. I assume it's to calculate the size for the scrollbar since the height of the listitems is not fixed but can vary per row in the GtkTreeView.

Would this help virtual treeview in any way?

Regards,

Andrew

Okay I found gtk_tree_view_set_fixed_height_mode which stops the View from retrieving all of the items values initially.

Regards

Andrew Haines

For virtual listview it is not needed to store any items anywhere - we only need to tell to treeview that we have 50,000+ items and return correct listitem when it is requested by index. Yes, it would help to store less data.

At the moment data is stored:
- our own structure which we want to represent in the listview
- gtk list model items - we are storing there 50,000+ items * 3 (checkbox, glyph, index) = 50,000+ * 9+ bytes = 450,000++ bytes

If we can remove gtk list then we of course save some memory.

Best regards,
Paul Ishenin.

I've already maded small tests with qt. Yes, it creates all items via setItemCount() like gtk do, but it does not need to be se in OwnerData=True mode.I've created TFPList via setItemCount(), and then create QTreeWidgetItems's only when painting virtual list (create only items for visible rows).This way needs a lot less memory , but need some more cpu cycles while creating visible items...

If we use our own gtkmodel, then it should be possible to use the data in the TListview, without storing itself

Reminds me that I still need to create out own "image" which can be dramw from an imagelist, so you don't need 50000+ instances of the same image data.

Marc

Okay In r22945 I have implemented a custom treemodel for the gtk2 listview. This should fix your memory problem.

I put some writeln's in the OnData method and also discovered that that the OnData routine was called, on startup of my program, for each of my million items.

This still happens because for columns to be autosized it is not allowed to set the gtktree(list)view to have fixed height rows.

If the new treemodel is not working well you can disable it by compiling the lcl with -dUseOrigTreeModel

Regards,

Andrew

I tried your update. Yes, the memory usage is somewhat better now (before it was something like 100MB with 1 million items in the list, now it's around 47MB).

However, when I fire up my program the CPU usage goes to 100% for several minutes before it comes back down to idle.

Also, when I click on an item in the list I get an access violation:

$0828D52C GTK2WSLV_ITEMSELECTED, line 222 of gtk2wscustomlistview.inc

-SG

Should be fixed now in r22947

Btw I set the count to 1000000 and it loaded almost instantly, so no huge ,

Andrew.

I just checked out rev 23015 and I'm not getting the crash any more when clicking on the list view item.

There is not a delay at startup for me either, but for about a minute after startup the cpu usage for the application does skyrocket, even though the application is already loaded and is responsive. A writeln in the ondata method confirmed that what it's doing is gtk2 is calling ondata for all of the 1000000 items in the list, but this is apparently done in the background or something since the app is responsive in the meantime and the cpu does go down eventually.

Here's what top shows for memory usage when I set the ListView1.Items.Count property to a million:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 10584 user 20 0 61448 47m 8696 R 89.4 4.8 0:08.10 project1

Compared to the following for 1000:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 10586 user 20 0 24228 11m 8752 S 0.0 1.2 0:00.27 project1

Anyway, thanks, it's a major improvement over what it was before.

-SG