Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2011-08

Proper implementation of TDataLink descendant

17 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Alexander Klenin

After researching http://bugs.freepascal.org/view.php?id=19887, I have found the following:

0) For context, here is TDbChartSource desctiption: http://wiki.lazarus.freepascal.org/TAChart_documentation#Database_source
1) My implementation of TDb?hartSource is incorrect, because during the chart drawing
it accesses dataset directly from the TDataLink descendant. Apparently this causes infinite loop, because such accesses generate DataSetChanged events,
which in turn invalidate the chart.
Even DisableControls/EnableControls do not help, since EnableControls calls DataSetChanged (!).
2) Looking at the code in TDBGrid, it seems that the correct way is to use DataLink's ActiveRecord property.
Unfortunately, in contrast to TDBGrid, TDb?hartSource needs to read the whole dataset.

  It seems that the only way to do this is to set BufferCount :=

RecordCount, i.e. buffer entire dataset in memory, which I do not like.

So, my questions are:

1) Is there a way to totally disable DataSetChanged events in the dataset?
2) Alternatively, is there a mechanism to read entire dataset via TDataLink without fully buffering it?

Alexander S. Klenin

Reply by email

Michael Van Canneyt
to Alexander Klenin
Alexander Klenin wrote 24 lines
After researching http://bugs.freepascal.org/view.php?id=19887, I have found the following: 0) For context, here is TDbChartSource desctiption: http://wiki.lazarus.freepascal.org/TAChart_documentation#Database_source 1) My implementation of TDb?hartSource is incorrect, because during the chart drawing it accesses dataset directly from the TDataLink descendant. Apparently this causes infinite loop, because such accesses generate DataSetChanged events, which in turn invalidate the chart. Even DisableControls/EnableControls do not help, since EnableControls calls DataSetChanged (!). 2) Looking at the code in TDBGrid, it seems that the correct way is to use DataLink's ActiveRecord property. Unfortunately, in contrast to TDBGrid, TDb?hartSource needs to read the whole dataset. It seems that the only way to do this is to set BufferCount := RecordCount, i.e. buffer entire dataset in memory, which I do not like. So, my questions are: 1) Is there a way to totally disable DataSetChanged events in the dataset?

No.

You only disable the treatment of the event while browsing.

Something like

If not Floading then
   LoadChart;
Procedure LoadChart;
begin
   Floading:=True;
   try
     // Browse data and build series.
   finally
     Floading:=False;
   end;
end;
2) Alternatively, is there a mechanism to read entire dataset via TDataLink without fully buffering it?

No, for obvious reasons. I think that for the tchart, you should not bother with the buffer mechanism. Just do a

   First;
   While not EOF do
     begin
     AddPointToSeries;
     Next;
     end;

A chart always sees the dataset as a whole. DB aware controls normally see only 1 record. Normally, a grid sees as much records as it currently displays rows.

But many advanced grids also browse the whole dataset when building rows;
they don't bother with Activebuffer and buffercount. (

Michael.

Reply by email

Alexander Klenin
to Michael Van Canneyt: No. You only disable the treatment of the event while browsing. Something like If not…

2011/8/5 Michael Van Canneyt :

Show quoted text 21 lines
> 1) Is there a way to totally disable DataSetChanged events in the dataset? No. You only disable the treatment of the event while browsing. Something like If not Floading then ?LoadChart; Procedure LoadChart; begin ?Floading:=True; ?try ? ?// Browse data and build series. ?finally ? ?Floading:=False; ?end; end;

Of course I already do that.
The problem is that it does not help -- see the bug report I linked to. I need to disable the events in the dataset itself. Will the patch adding this capability be accepted?

Show quoted text 12 lines
> 2) Alternatively, is there a mechanism to read entire dataset via > TDataLink without fully buffering it? No, for obvious reasons. I think that for the tchart, you should not bother with the buffer mechanism. Just do a ?First; ?While not EOF do ? ?begin ? ?AddPointToSeries; ? ?Next; ? ?end;

This is (almost) exactly what I did. You can look yourself at the TADbSource unit.
The problem is, it does not work, as I described in the first mail. What are the obvious reasons you refer to?
I did not yet study the dataset/datalink implementation fully, so they are not obvious to me.

Show quoted text 6 lines
A chart always sees the dataset as a whole. DB aware controls normally see only 1 record. Normally, a grid sees as much records as it currently displays rows. But many advanced grids also browse the whole dataset when building rows; they don't bother with Activebuffer and buffercount. (

Such grids either fail in the same way TDbChartSource does, or use some other mechanism I am not aware of.

Alexander S. Klenin

Reply by email

Alexander Klenin
to Luiz Americo Pereira Camara: Yes implement dsBlockRead in TDataset http://docwiki.embarcadero.com/VCL/en/DB.TDataSet.Bl…
Luiz Americo Pereira Camara wrote 8 lines
On 4/8/2011 13:16, Alexander Klenin wrote: > > 1) Is there a way to totally disable DataSetChanged events in the dataset? Yes implement dsBlockRead in TDataset http://docwiki.embarcadero.com/VCL/en/DB.TDataSet.BlockReadSize

Excellent! This seems to be what I need.
Will a patch implementing event disabling be accepted?

Also, the Borland's design seems very hackish to me. I would prefer to add DataEventMask: TDataEventSet property to allow user to mask events separately.
On top of that, dsBlockRead state can be implemented as setting DataEventMask to full set.
What do you think?

Alexander S. Klenin

Reply by email

Luiz Americo Pereira Camara
to Alexander Klenin
Alexander Klenin wrote 17 lines
On Fri, Aug 5, 2011 at 03:39, Luiz Americo Pereira Camara wrote: > On 4/8/2011 13:16, Alexander Klenin wrote: >> 1) Is there a way to totally disable DataSetChanged events in the dataset? > Yes implement dsBlockRead in TDataset > > http://docwiki.embarcadero.com/VCL/en/DB.TDataSet.BlockReadSize > Excellent! This seems to be what I need. Will a patch implementing event disabling be accepted? Also, the Borland's design seems very hackish to me. I would prefer to add DataEventMask: TDataEventSet property to allow user to mask events separately. On top of that, dsBlockRead state can be implemented as setting DataEventMask to full set. What do you think?

If i understand correct this would be a public property.

This can lead to conflict with the internal state.

User set Mask to some events than DisableControls set only some (different) events then BlockRead set to not send any events

Restoring the initial state is possible with a stack like structure but would add some complexity. So i would add the bare minimum to implement dsBlockRead

All in all, i agree that TDataset has it's age and it would be welcome a fpc data abstraction designed from the ground to be used by reports, templates, charts, data controls

Luiz

Reply by email

Luiz Americo Pereira Camara
to Alexander Klenin: On Fri, Aug 5, 2011 at 03:39, Luiz Americo Pereira Camara Excellent! This seems to be…
Alexander Klenin wrote 10 lines
On Fri, Aug 5, 2011 at 03:39, Luiz Americo Pereira Camara wrote: > On 4/8/2011 13:16, Alexander Klenin wrote: >> 1) Is there a way to totally disable DataSetChanged events in the dataset? > Yes implement dsBlockRead in TDataset > > http://docwiki.embarcadero.com/VCL/en/DB.TDataSet.BlockReadSize > Excellent! This seems to be what I need. Will a patch implementing event disabling be accepted?

Be aware that even if is accepted it will land in a fpc release from at least one year (or more), so you will have to workaround anyway

Luiz

Reply by email

Alexander Klenin
to Luiz Americo Pereira Camara: If i understand correct this would be a public property. This can lead to conflict with…
Luiz Americo Pereira Camara wrote 10 lines
If i understand correct this would be a public property. This can lead to conflict with the internal state. User set Mask to some events than DisableControls set only some (different) events then BlockRead set to not send any events Restoring the initial state is possible with a stack like structure but would add some complexity. So i would add the bare minimum to implement dsBlockRead

I suggest simply to declare that Mask is "volatile", i.e. it will be overwritten by changes to State.
Alternatively, there could be separate Mask and InternalMask, the union of which will be actually applied.

All in all, i agree that TDataset ?has it's age and it would be welcome a fpc data abstraction designed from the ground to be used by reports, templates, charts, data controls

Probably yes, but the enrty barrier for such solution is very high. Maybe TDataset can be slowly improved instead.

Alexander S. Klenin

Reply by email

Alexander Klenin
to Luiz Americo Pereira Camara: Be aware that even if is accepted it will land in a fpc release from at least one year…
Luiz Americo Pereira Camara wrote
Be aware that even if is accepted it will land in a fpc release from at least one year (or more), so you will have to workaround anyway

I agree, FPC development is glacial :-(
I can only hope that 2.6.2 will not take a whole year. But there is no workaround, so I have no choice.

Alexander S. Klenin

Reply by email

Alexander Klenin
to Alexander Klenin
Alexander Klenin wrote
But there is no workaround, so I have no choice.

Heh, I should never underestimate the power of ugly hacks ;-) I have invented and committed a workaround, but it is not pretty at all -- see r31864.

Alexander S. Klenin

Reply by email

Luiz Americo Pereira Camara
to Alexander Klenin: On Fri, Aug 5, 2011 at 11:52, Luiz Americo Pereira Camara I suggest simply to declare…
Alexander Klenin wrote 16 lines
On Fri, Aug 5, 2011 at 11:52, Luiz Americo Pereira Camara wrote: > If i understand correct this would be a public property. > > This can lead to conflict with the internal state. > > User set Mask to some events than DisableControls set only some (different) > events then BlockRead set to not send any events > > Restoring the initial state is possible with a stack like structure but > would add some complexity. So i would add the bare minimum to implement > dsBlockRead I suggest simply to declare that Mask is "volatile", i.e. it will be overwritten by changes to State. Alternatively, there could be separate Mask and InternalMask, the union of which will be actually applied.

I implemented BlockReadSize

See http://bugs.freepascal.org/view.php?id=19897

The patch is minimal and do what you (and I) need without much complexity

In fact it took more time to do the Delphi testing and to compile fpc trunk than the time to code the patch itself

Luiz

Reply by email

Alexander Klenin
to Luiz Americo Pereira Camara: I implemented BlockReadSize See http://bugs.freepascal.org/view.php?id=19897 The patch is…
Luiz Americo Pereira Camara wrote 8 lines
I implemented BlockReadSize See http://bugs.freepascal.org/view.php?id=19897 The patch is minimal and do what you (and I) need without much complexity In fact it took more time to do the Delphi testing and to compile fpc trunk than the time to code the patch itself

Thank you. I know the required part of functionality is very simple. Unfortunately, I have a history of rejected FPC patch because I implemented only the part I needed, not the whole spec. This is why I asked first instead of programming it myself. Anyway, I hope you are luckier than me, and your patch will be applied ;-)

The patch seems to have problems with the whitespace -- there are spaces at the ends of added lines and tabs instead of spaces are used for indentation sometimes.
Other than that, it looks fine, thanks again.

Alexander S. Klenin

Reply by email

Luiz Americo Pereira Camara
to Alexander Klenin
Alexander Klenin wrote 13 lines
On Fri, Aug 5, 2011 at 15:11, Luiz Americo Pereira Camara wrote: > I implemented BlockReadSize > > See http://bugs.freepascal.org/view.php?id=19897 > > The patch is minimal and do what you (and I) need without much complexity > > In fact it took more time to do the Delphi testing and to compile fpc trunk > than the time to code the patch itself Thank you. I know the required part of functionality is very simple. Unfortunately, I have a history of rejected FPC patch because I implemented only the part I needed, not the whole spec.

I tested the Delphi behavior before so AFAIK is complete.

Show quoted text 7 lines
This is why I asked first instead of programming it myself. Anyway, I hope you are luckier than me, and your patch will be applied ;-) The patch seems to have problems with the whitespace -- there are spaces at the ends of added lines and tabs instead of spaces are used for indentation sometimes. Other than that, it looks fine, thanks again.

I fixed that

Luiz

Reply by email

Alexander Klenin
to Luiz Americo Pereira Camara
Luiz Americo Pereira Camara wrote 6 lines
On 5/8/2011 01:34, Alexander Klenin wrote: > Thank you. I know the required part of functionality is very simple. > Unfortunately, I have a history of rejected FPC patch because > I implemented only the part I needed, not the whole spec. I tested the Delphi behavior before so AFAIK is complete.

I mean this part:
" setting BlockReadSize also causes the dataset to fetch database information in blocks of BlockReadSize records, which it then buffers."

Alexander S. Klenin

Reply by email

Alexander Klenin

Please, write to the mailing list instead of directly to me. Other people on the list might also be interested in, or contribute to, our discussion.

Jorge L?pez wrote 8 lines
What do you think about large datasets? My implementation is an historical trend of data recolected in a factory. A new row should be created on frequency basis and may be every minute or every second, obviously this generate many records. I implemented a SQL query to select a manageable quantity of records with datetime parameters. Load all records in memory should hang up my system. This is solution?

Yes, filtering data at the database level is a good plan.

Alexander S. Klenin

Reply by email

Luiz Americo Pereira Camara
to Alexander Klenin: On Fri, Aug 5, 2011 at 15:43, Luiz Americo Pereira Camara I mean this part: " setting…
Alexander Klenin wrote 11 lines
On Fri, Aug 5, 2011 at 15:43, Luiz Americo Pereira Camara wrote: > On 5/8/2011 01:34, Alexander Klenin wrote: >> Thank you. I know the required part of functionality is very simple. >> Unfortunately, I have a history of rejected FPC patch because >> I implemented only the part I needed, not the whole spec. > I tested the Delphi behavior before so AFAIK is complete. I mean this part: " setting BlockReadSize also causes the dataset to fetch database information in blocks of BlockReadSize records, which it then buffers.

This is specific for DBE descendants. Also, nothing stops other dataset descendants to use this property.

Luiz

Reply by email