I am using a TListbox component on a form for displaying debug data arriving over a serial line at 115200 baud.
The data are a set of MQTT telegram texts which arrive in packets of about 40 lines each time (once per 10 seconds).
I add them to the listbox as follows:

procedure THanSimulatorMain.OnRxData(Sender: TObject; const Data: TBytes);
var
  len, oldlen: integer;
  DataTxt: AnsiString;
begin
  len := Length(Data); //Incoming packet
  oldlen := Length(DataBuffer);
  SetLength(DataBuffer, oldlen + len);
  Move(Data[0], DataBuffer[oldlen], len);
  SetLength(DataTxt, Length(DataBuffer));
  Move(DataBuffer[0], DataTxt[1], Length(DataBuffer));
  lbxRxData.Items.Text := DataTxt;  //Add the text to the list
  lbxRxData.ItemIndex := lbxRxData.Items.Count -1; //To make it visible
end;

DataBuffer is a global TBytes container where the incoming data are stuffed as they arrive (it grows during the session).
You see that the buffer contains the complete log history from the start...

I have noticed that after a while the display becomes very sluggish when data arrives and I think that is due to the way the component operates.

Now I wonder if there is some way to do as I did when I worked in Delphi with TListView objects, where I could use the BeginUpdate and EndUpdate calls to make all screen updates wait until it was all put in place. This was MUCH faster!

But I can not find a BeginUpdate on the TListBox object, only BeginUpdateBounds, which does not tell me much....

Any suggestions?

El 11/11/22 a les 16:29, Bo Berglund via lazarus ha escrit:

I have noticed that after a while the display becomes very sluggish when data arrives and I think that is due to the way the component operates.

You could try using a TSynEdit instead of a TListBox: some years ago I switched from TMemo to TSynEdit to do some logging in a similar (though I add lines instead of assigning the complete text, also, I delete the oldest line when there are 5000 lines) because I found that, under windows, TSynEdit is an order of magnitude (or more) faster than TMemo. No difference under linux though, and maybe it's no longer true nowadays.

Bye

In fact I found TListbox.Items.Beginupdate/EndUpdate, but it did not do much to improve the handling, too slow anyway....

I have never used TSynEdit before so I tried dropping one onto my form and it looked a bit "strange" with a wide margin to the left... How does it work (especially the margin that steals space)?

Note that I do not need an editor as such, just somewhere to show the log lines, and possibly also a way to copy certain lines or text sections to paste somewhere else.

BTW this application is on Windows and I am using Lazarus 2.0.12 and FPC 3.2.0

Bo Berglund wrote

You see that the buffer contains the complete log history from the start...

When the first block of data has arrived I would store the length of the buffer at this time. Then next time when new data come in, I would extract the "new data", i.e. the bytes between the stored length and the new length of the buffer and add them to the listbox. And store the current length for the next round. Etc... This avoids moving all the data to the listbox, which it already has, again and again. Because extraction of the individual lines in lbxRxdata.Items.Text := DataTxt is a length process.

El 11/11/22 a les 17:56, Bo Berglund via lazarus ha escrit:

I have never used TSynEdit before so I tried dropping one onto my form and it looked a bit "strange" with a wide margin to the left... > How does it work (especially the margin that steals space)?

Play with the properties in the project inspector, you can remove the gutter (gutter.visible -> false), the margin (by setting RightEdge to a very big value, say 800) and customize much of its appearance..

Note that I do not need an editor as such, just somewhere to show the log lines, and possibly also a way to copy certain lines or text sections to paste somewhere else.

Me too. Just set the readonly property to true and that's it. Then just treat it like a TMemo (using the Lines property). It has some more goodies that aren't available in a Tmemo.
As I said, I was using a TMemo for logging and it was too slow under windows, I switched to a TSynEdit and had great improvement in performance.

BTW this application is on Windows and I am using Lazarus 2.0.12 and FPC 3.2.0

So you'll probably see an improvement by using a TSynEdit (or maybe not, the only way to know is to try it).

Bye

Luca Olivetti wrote

Play with the properties in the project inspector, you can remove the gutter (gutter.visible -> false), the margin (by setting RightEdge to a very big value, say 800) and customize much of its appearance..

I replaced the TListBox with a TSynEdit as suggested and I have "played with" the properties a bit.
Seems to work quite well so far.

I have a couple of questions:

1) Is there a way to make sure that when adding text the view shifts such that the last line is visible?
I.e. Automatically scroll the text up when new text is added.

2) The reason I am not adding incoming data to the listbox as tyhey arrive is that I don't know if the serial component provides the data at line endings or in the middle of a line, so I add the data to the old data in the TBytes array and then dump the content of the TBytes into the text property of the box.

What happens if one adds the text as a block to this synedit where the block does not end in a line feed?
Will the end of it be some way into the last line such that if I then add the next transmission it will start at tahat position and not on a new line?

If that is possible then the handling will become a lot easier since I just have to add the few new lines at the end directly in synedit...

I'll test that too.
But how to jump to the end of the text on screen?

On Fri, 11 Nov 2022 23:37:14 +0100, Bo Berglund via lazarus wrote:

Concerning TSynEdit:

This was my most acute problem with the synedit object, but see below...

2) The reason I am not adding incoming data to the listbox as they arrive is that I don't know if the serial component provides the data at line endings or in the middle of a line, so I add the data to the old data in the TBytes array and then dump the content of the TBytes into the text property of the box.

I have dug down into the serial component and found that it actually dumps data at a rather high pace with small packets every time...

Example:
One transmission contains 16 lines of data totalling 1130 bytes, but the serial port component triggered an ondata event for a lot less, more like less than 10 bytes so the transmission triggered a big number of events, each updating the listbox which totally overloaded the system.

So now I have changed two properties on the serial component:
- ReadPacketSize := 1500 (default is 16)
- ReadTimeout := 200 (default is 2 ms)

Since most transfers are shorter than 1500 the timeout of 200 ms will trigger and by then the complete message *is* available, hence only one call to update the screen display with new text.
This helps to make it more responsive...

What happens if one adds the text as a block to this synedit where the block does not end in a line feed? Will the end of it be some way into the last line such that if I then add the next transmission it will start at that position and not on a new line?

It is my belief that even though there is no data following the end of the last line (like an explicit linefeed), the next data added will start on a new line...

Unless of course one could do the following (pseudocode):

synedit.lines[lastline] := synedit.lines[lastline] + incomingdata;

This would extend the last line with the incoming data...

If that is possible then the handling will become a lot easier since I just have to add the few new lines at the end directly in synedit...

No just simply adding incoming data to the lines does not work, they will start on a new line (see above)...

But how to jump to the end of the text on screen?

Right now this was my one remaining issue, since the Lines property does not have a settable "ItemIndex" value that will move a line into the visible area.

But I found out that there is another useful property:

synRxData.CaretY := synRxData.Lines.Count;

This tries to put the cursor 1 line beyond the end and it does exactly what I need to do by showing the incoming new data and scrolling up the display visibly.
This is how it is now done in full:

  SetLength(DataTxt, Length(DataBuffer)); //Set string length
  Move(DataBuffer[0], DataTxt[1], Length(DataBuffer)); //Copy data to string
  {$ifdef USESYNEDIT}
    synRxData.Lines.Beginupdate;
    synRxData.Lines.Text := DataTxt;
    synRxData.CaretY := synRxData.Lines.Count;
    synRxData.Lines.Endupdate;
  {$else}
    lbxRxData.Items.Beginupdate;
    lbxRxData.Items.Text := DataTxt;
    lbxRxData.ItemIndex := lbxRxDataBak.Items.Count -1;
    lbxRxData.Items.Endupdate;
  {$endif}

El 11/11/22 a les 23:37, Bo Berglund via lazarus ha escrit:

But how to jump to the end of the text on screen?

TheSynEdit.TopLine:=TheSynEdit.Lines.Count-TheSynEdit.LinesInWindow+1

Bye

Did not work, but this does work:

synRxData.CaretY := synRxData.Lines.Count;

El 12/11/22 a les 12:13, Bo Berglund via lazarus ha escrit:

Strange, I've been using it for many years (and still use it) to ensure the last line is visible.

synRxData.CaretY := synRxData.Lines.Count;

Em sex., 11 de nov. de 2022 ?s 12:29, Bo Berglund via lazarus <

escreveu: I am using a TListbox component on a form for displaying debug data

arriving

over a serial line at 115200 baud. The data are a set of MQTT telegram texts which arrive in packets of

about 40

stuffed as

they arrive (it grows during the session). You see that the buffer contains the complete log history from the

start...

I have noticed that after a while the display becomes very sluggish when

data

arrives and I think that is due to the way the component operates. Now I wonder if there is some way to do as I did when I worked in Delphi

with

TListView objects, where I could use the BeginUpdate and EndUpdate calls

to make

all screen updates wait until it was all put in place. This was MUCH faster! But I can not find a BeginUpdate on the TListBox object, only

BeginUpdateBounds,

Hi,

What size is your data? The biggest problem regarding Windows' TListBox (and TMemo and others) was it used a contiguous buffer for its whole text and increased capacity by doubling the buffer or something, thus it got ugly easily; the parsing did seem to have some problems too. Microsoft alleviated this problem for TMemo at least at some point but I don`t know about TListBox.
When you set/replace the whole text for a control (ex. using ListBox1.Items.Text) there`s (usually) no need to use Begin/EndUpdate; if you're appending a bunch of lines, though, you should use ListBox1.Items.BeginUpdate.
Do you really need to keep 'DataBuffer' around for other reasons? Otherwise you could just:

procedure TForm1.OnRxData(Sender: TObject; const Data: TBytes);
var
  buf: TStringList;
  DataTxt: ansistring;
begin
  SetLength(DataTxt, Length(Data));
  Move(Data[0], DataTxt[1], Length(Data));
  buf := TStringList.Create;
  buf.Text := DataTxt;
  SynEdit1.Lines.AddStrings(buf);
  buf.Free;
  SynEdit1.CaretY := SynEdit1.Lines.Count;
end;

Too bad TStrings don`t have a method to add text parsing the linebreaks... IMHO even a PChar overload with offsets would be deserved.

Best regards,
Fl?vio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20221118/44227a4e/attachment.htm>

Em dom., 13 de nov. de 2022 ?s 18:36, Luca Olivetti via lazarus <

 It should work. Or even straight
"TheSynEdit.TopLine:=TheSynEdit.Lines.Count;'.

Best regards,
Fl?vio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20221118/90b713a3/attachment.htm>

My current version of my logging tool application uses this to display incoming data on the serial port (where it ends up in the TBytes buffer DataBuffer[] before displaying):

...
  SetLength(DataTxt, Length(DataBuffer)); //Set string length
  Move(DataBuffer[0], DataTxt[1], Length(DataBuffer)); //Copy data to string
  {$ifdef USESYNEDIT}
    synRxData.Lines.Beginupdate;
    synRxData.Lines.Text := DataTxt;
    synRxData.CaretY := synRxData.Lines.Count;
    synRxData.Lines.Endupdate;
    stxLineCount.Caption := IntToStr(synRxData.Lines.Count);
  {$else}
...

Right now it has been running since 2022-11-19 01:20:05 and contains 11936 log lines and whenever new data arrive it is almost instant on screen when it loads the new lines and displays the very bottom of the list...

Em dom., 20 de nov. de 2022 04:23, Bo Berglund via lazarus <

I see. My point is that you don't need to copy the whole DataBuffer and re-populate the whole SynEdit each time you receive new data. But this probably won't matter much unless you have hundreds of thousands of lines running on not so recent hardware.

Best regards,
Fl?vio

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20221120/d601448e/attachment.htm>

Fl?vio Etrusco wrote

I see. My point is that you don't need to copy the whole DataBuffer and re-populate the whole SynEdit each time you receive new data. But this probably won't matter much unless you have hundreds of thousands of lines running on not so recent hardware.

Well, I tried at first to add incoming data to the bottom of the window, but it turned out that when the transfer of data had been cut into parts by the serial buffer such that the OnRxdata() triggered several times during the transfer then the line at which the trigger happened was cut in two in the middle of the screen.

Apparently the extra line break that was not part of the transfer was added by the control when the data was added to the end. Next data added to it was not adjacent to the last previous character but placed on the start of the next line...

So the data in the synedit always ended with a linefeed even though there was not yet any such in the transfered data.

Therefore I decided to use a dynamic TBytes array, which I could expand the size of whenever the data arrived to fit the new packet and put it at the end of the buffer.
Then I just replaced the strings in the display control with the full buffer every time.
This way lines are not visibly cut in half or worse....

But it would be better if I could have kept the data in the SynEdit itself and just expanded with new data as they arrived by putting them at the end...

Not so much to transfer then.
Just new arriving text.

It wouldn't be that hard to write custom component based on TGraphicControl + TBitmap with built-in bitmap font. Copying bitmap letters will be faster than rendering from ttf. ?
V.
______________________________________________________________

Now I wonder if there is some way to do as I did when I worked in Delphi with TListView objects, where I could use the BeginUpdate and EndUpdate calls to make all screen updates wait until it was all put in place. This was MUCH faster!

But I can not find a BeginUpdate on the TListBox object, only BeginUpdateBounds, which does not tell me much....

Any suggestions?

Maybe you could add a buffer in your flow of data, storing the incoming data ?
When a line ending arrives, you can flush your buffer to Synedit up to the line ending, keeping in the buffer just the end of the data not yet terminated by a line ending.

Back to the first question. You can append your data with lb.items.text. Just prepare your data in a string like this: strtmp:=data1+lineending+data2+lineending+..... And append lb.items.text:=lb.items.text+lineending+strtmp Yes it is a workaround but do the trick like beginupdate Ciao

?Ottieni BlueMail per Android ?

Il giorno 11 Nov 2022, 16:29, alle ore 16:29, Bo Berglund via lazarus ha scritto:

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20221122/ca964670/attachment.htm>