Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2021-09

Playing with debuggers

16 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Joost van der Sluis

Hi all,

I'm playing with several debuging-concepts. And I want to share this one with you.

I've created a new Lazarus-debugger LazDabDebugger that can work with FpdServer. It depends on a package (LazDebugExtensionIntf) that adds another way of showing debugging variables inside Lazarus.

I can write tons of texts about the background and possibilities. But for now I'm just curious at your input.

Look at the screencast or install the debugger, and share your opinion(s).

Screencast: http://amira.cnoc.nl/fpc/LazDabDebuggerScreencast.gif

LazDabDebugger: https://gitlab.com/fpcprojects/LazDABDebugger/ FpdServer: https://gitlab.com/fpcprojects/fpdserver

Btw: if you need the fpdserver executable and do not want to compile it yourself, download it here:
https://gitlab.com/fpcprojects/fpdserver/-/pipelines/370065998

Regards,

Joost.

Reply by email

Martin Frb
to Joost van der Sluis
Joost van der Sluis wrote 8 lines
Hi all, I'm playing with several debuging-concepts. And I want to share this one with you. I've created a new Lazarus-debugger LazDabDebugger that can work with FpdServer. It depends on a package (LazDebugExtensionIntf) that adds another way of showing debugging variables inside Lazarus.

Great, the frontend was overdue for a remake.

I have not yet looked at the code.
But ideally this should just replace/extend the existing watches. Of course that also would need the debugger intf to be updated.  (That does need an overhaul, and yes it could mean all existing debuggers will need a bit of work to follow / ideas welcome)

Ideally the entire debugger frontend could move into a package of its own too (all the current windows).

I can write tons of texts about the background and possibilities. But for now I'm just curious at your input.

Currently the debugger intf allows to fetch watches as plain text, or as structure. But this has to be passed in as flag at the request time.

Any watch object should instead have methods to fetch "sub watches" so structures can unfold.
I see you pass the variable to the debugger, that is ok. But that may well get abstracted into the Watch, so the watch has its own ref to the debugger, and makes the needed calls.

Not sure about passing callbacks directly. Thinking of the current notification system (data monitors).
What does your new watch window do, if the user modifies a variable? This triggers re-eval of all watches (so they need to unfold again). Currently the debugger backend can trigger the re-eval, and through the data-monitors tell everyone about the updates.

Have to look at it in more detail....

Reply by email

Joost van der Sluis
to Martin Frb

Op 14-09-2021 om 02:18 schreef Martin Frb via lazarus:

Show quoted text 14 lines
On 14/09/2021 01:00, Joost van der Sluis via lazarus wrote: > I'm playing with several debuging-concepts. And I want to share this > one with you. > > I've created a new Lazarus-debugger LazDabDebugger that can work with > FpdServer. It depends on a package (LazDebugExtensionIntf) that adds > another way of showing debugging variables inside Lazarus. Great, the frontend was overdue for a remake. I have not yet looked at the code. But ideally this should just replace/extend the existing watches. Of course that also would need the debugger intf to be updated.  (That does need an overhaul, and yes it could mean all existing debuggers will need a bit of work to follow / ideas welcome)

What I did now, is add new dialogs, and created an debug-interface separate from the existing one. But with the idea that it could replace/be merged with the existing one.

What we could also do is make the debug-windows (watches, locals, evaluation pluggable, so that a debugger can choose which one to use.)

Ideally the entire debugger frontend could move into a package of its own too (all the current windows).

Exactly, that is why I created a new debug-intf package instead of adapting the existing one.

Show quoted text 8 lines
> I can write tons of texts about the background and possibilities. But > for now I'm just curious at your input. Currently the debugger intf allows to fetch watches as plain text, or as structure. But this has to be passed in as flag at the request time. Any watch object should instead have methods to fetch "sub watches" so structures can unfold.

What I have now is that each object has a 'reference', and there is a separate call to get the sub-watches that belong to the reference. That way the data (in the object) and the logic (function to retrieve sub-watches) are separated.

I don't have a solution for requesting the result as text(string) only. But that's not a bad idea.

I see you pass the variable to the debugger, that is ok. But that may well get abstracted into the Watch, so the watch has its own ref to the debugger, and makes the needed calls.

I don't follow exactly. But if there's a need for an extra abstraction layer before sending the variable to the debugger directly, this could be added.

Show quoted text 8 lines
Not sure about passing callbacks directly. Thinking of the current notification system (data monitors). What does your new watch window do, if the user modifies a variable? This triggers re-eval of all watches (so they need to unfold again). Currently the debugger backend can trigger the re-eval, and through the data-monitors tell everyone about the updates. Have to look at it in more detail....

I have something similar as the data-monitors.

Main difference is that the events are based on what the gui needs, instead of what the debugger does.

So: there is an event to tell the 'gui' that it is inpossible to show any data. (for example: the application has stopped or there is no debugger at all) There is an event for the case that debug-data has become available or unavailable. (Application paused or continued)

And I should add an event for the case that the debug-info should be re-evaluated. (context switch, or variable-change). And I can think of another event that can be send when the debug-data could have been changed, due to unforeseen effect, for example when a propperty with a getter has been evaluated.

The advantage of 'events' that are more geared towards the gui, is that it is easier to add catchy things. For example: to avoid flickering, when the gui receives an event that debug-data is not available anymore, it starts a timer of 200 mseconds, and only after that timer the data is removed from the screen. This way, when stepping through the code, there is no flickering in the small periods of time that the application runs.

It also made it easy to highlight variables that changed.

But atm they are only suitable for displaying variables. Stuff like the call-stack are not touched.

Regards,

Joost.

Reply by email

Michael Van Canneyt
to Joost van der Sluis: I'm playing with several debuging-concepts. And I want to share this one with you. I've…
Joost van der Sluis wrote 13 lines
Hi all, I'm playing with several debuging-concepts. And I want to share this one with you. I've created a new Lazarus-debugger LazDabDebugger that can work with FpdServer. It depends on a package (LazDebugExtensionIntf) that adds another way of showing debugging variables inside Lazarus. I can write tons of texts about the background and possibilities. But for now I'm just curious at your input. Look at the screencast or install the debugger, and share your opinion(s).

Impressive job...

Looking at the screencast, I think changing the current interface with the one presented in the screencast would be a definite improvement; regardless of the backend used...

Michael.

Reply by email

Martin Frb
to Joost van der Sluis: Op 14-09-2021 om 02:18 schreef Martin Frb via lazarus: What I did now, is add new…

I started about "sub watches" before reading your "every watch has a reference" ....

Joost van der Sluis wrote 12 lines
Op 14-09-2021 om 02:18 schreef Martin Frb via lazarus: > But ideally this should just replace/extend the existing watches. Of > course that also would need the debugger intf to be updated.  (That > does need an overhaul, and yes it could mean all existing debuggers > will need a bit of work to follow / ideas welcome) What I did now, is add new dialogs, and created an debug-interface separate from the existing one. But with the idea that it could replace/be merged with the existing one. What we could also do is make the debug-windows (watches, locals, evaluation pluggable, so that a debugger can choose which one to use.)

Actually all it would need is for the watch to have a new method, and to have a flag if it can be expanded. (a debugger that can not do that, will not set the flag).

However, it is a bit more....

1)
All watches should be part of the "watches" list. (or alternative means must be found). This is so the debugger can update them, if the user changes memory (set a new value to a variable).
This is also helpful for the history window, and for "idle eval" (if the watches window is closed but the debugger is idle, then eval starts anyway).

The problem of not being in the "watches" exists for "debug inspector". It does not always update correctly. Adding to the "watches" would however mean that each watch needs some sort of owner, so the watch-window does not display watches added by others. So that ends up more rework.... (and maybe there is a better way)

Of course instead of adding new watch to the "watches", those watches can be hold as a sublist by each watch.

Note that watches for structures already have a list, with a description of each member field. (if the watch request had the correct flag set). But those list should probably be reworked...

2)
Currently formatting  (hex vs dec) is done in the debugger. That is also nonsense. As much as can be should be moved to the frontend.

3)
Btw, if arrays can be expanded, precautions are needed. I don't think we want to expand 100000 child values.

----
If we go for the flag and sub-watches, then "debug history" needs to be aware of it.

Show quoted text 6 lines
> Ideally the entire debugger frontend could move into a package of its > own too (all the current windows). Exactly, that is why I created a new debug-intf package instead of adapting the existing one.

Well we may need to create an interface for the debugger frontend, and then add the frontend package. Though DebugManager already has parts of that.

On the other hand its not strictly mandatory yet. As I see it your new watches can simply replace the old one. (as soon as watches provide that flag, so old watches can be displayed too)

Show quoted text 10 lines
> Currently the debugger intf allows to fetch watches as plain text, or > as structure. But this has to be passed in as flag at the request time. > > Any watch object should instead have methods to fetch "sub watches" > so structures can unfold. What I have now is that each object has a 'reference', and there is a separate call to get the sub-watches that belong to the reference. That way the data (in the object) and the logic (function to retrieve sub-watches) are separated.

Ok, I have to important factor on the design of every debugger-object (eg watches) is "notify on free" or TRefCountedObject.

Any debugger backend holds on to the "watch object" while it evaluates the data needed. However the frontend may remove (and free/dereference) that watch during that time. The backend must know that. (and potentially even call the callback, if the object no longer exists.... / the editor hint code requires this )

I don't have a solution for requesting the result as text(string) only. But that's not a bad idea.

Currently we have (as param to eval)
  TDBGEvaluateFlag =
    (defNoTypeInfo,        // No Typeinfo object will be returned      defSimpleTypeInfo,    // Returns: Kind (skSimple, skClass, ..); TypeName (but does make no attempt to avoid an alias)      defFullTypeInfo,      // Get all typeinfo, resolve all anchestors      defClassAutoCast,     // Find real class of instance, and use, instead of declared class of variable
     defAllowFunctionCall
    );
  TDBGEvaluateFlags = set of TDBGEvaluateFlag;

Currently

   TWatchValue = class(TFreeNotifyingObject)
      property TypeInfo: TDBGType;

and
TDBGType = class(TObject)
  property Fields: TDBGFields;
which is a list of TDBGField

So if (full)typeinfo is request, you have a list of fields, and you can then get the sub-watches.

But that is not a good way....
That means you then have lots of dup data => each subwatch, has some data also in a field.

** The other downside is, that the gdb debugger needs more time to fill the fields. So not every debugger should provide them by default.....

** The upside is, that having type info (and fields) the frontend can do all the formatting (almost... / mem dump still needs to get data as byte stream)
If the backend delivers a point (x,y) as 2 fields, the frontend can decide to print "(x: 1; y:2)"  or "(1,2)". The frontend can also print the individual values in dec,hex,bin,oct....

So always having type info would be nice.

But for gdb only remote debug targets, in needs to be seen if it can be done, without slow down.
It should be possible with one limitations.

Currently each Field has type (tkInteger, tkClass, tkRecord, ....) and ClassName (indicating which parent class introduced that field). Getting those fields is time consuming.

===================================
So as a first draft I would aim for something like this:

TDBGField needs to be a TWatch. => avoid data duplication / and provide for recursive eval.

By default the (if no flags are set) the debugger should provide fields. But only field name, and a value (int or string).
All other info can be retrieved later, on request.

In fact, even the field info can be retrieved on request (editor hint may not need it, but if formatting code goes to the frontend, then editor hint also needs to do the formatting).
In the same way, that a watch has a "validity" field (dsEvaluating, dsValid, dsError, ...) it can have validity for typeinfo and fields.

On top of that validity for typeinfo/fields can be dsNotAvail. In which case there just is a default text value for the watch.

Show quoted text 8 lines
> I see you pass the variable to the debugger, that is ok. But that may > well get abstracted into the Watch, so the watch has its own ref to > the debugger, and makes the needed calls. I don't follow exactly. But if there's a need for an extra abstraction layer before sending the variable to the debugger directly, this could be added.

See above, if the subwatches are already in place of the current fields, then they have a ds...validity. And if accessed they will further evaluate.

Show quoted text 25 lines
> Not sure about passing callbacks directly. Thinking of the current > notification system (data monitors). > What does your new watch window do, if the user modifies a variable? > This triggers re-eval of all watches (so they need to unfold again). > Currently the debugger backend can trigger the re-eval, and through > the data-monitors tell everyone about the updates. > > Have to look at it in more detail.... I have something similar as the data-monitors. Main difference is that the events are based on what the gui needs, instead of what the debugger does. So: there is an event to tell the 'gui' that it is inpossible to show any data. (for example: the application has stopped or there is no debugger at all) There is an event for the case that debug-data has become available or unavailable. (Application paused or continued) And I should add an event for the case that the debug-info should be re-evaluated. (context switch, or variable-change). And I can think of another event that can be send when the debug-data could have been changed, due to unforeseen effect, for example when a propperty with a getter has been evaluated.

All of those are available already

Most of those are DebuggerState changes. They are currently propagated hardcoded to each of the existing listeners. So they may need an API to subscribe to.
But they also need to be done in a prioritized order. Certain data needs to be done in specific order (partly that is enforced in the backend anyway)

Context changes are subscribe-able.
   CallStackNotification.OnChange   := @CallStackChanged;
   CallStackNotification.OnCurrent  := @CallStackCurrent;
   BreakpointsNotification.OnAdd    := @BreakPointChanged;
   BreakpointsNotification.OnUpdate := @BreakPointChanged;
   BreakpointsNotification.OnRemove := @BreakPointChanged;

Note that the debugger is not the only thing that can change context. If the user selects a history entry, then all watches change.

Show quoted text 8 lines
The advantage of 'events' that are more geared towards the gui, is that it is easier to add catchy things. For example: to avoid flickering, when the gui receives an event that debug-data is not available anymore, it starts a timer of 200 mseconds, and only after that timer the data is removed from the screen. This way, when stepping through the code, there is no flickering in the small periods of time that the application runs.

That can be done with the above Notification too.

It also made it easy to highlight variables that changed.

Ah, but does it?

Say I have an object with a delegate (object / class based => that is pointer to heap).

TBar = class
   i: integer;
end;
TFoo = class
   bar: TBar;
end;
foo.bar := otherBar.

If otherBar has the same value in "i", and if the address of "bar" in tfoo is not displayed. => then the displayed result of the "foo" watch does not change. The value however has changed.

But atm they are only suitable for displaying variables. Stuff like the call-stack are not touched.

In generally there may be reasons to extend on the current monitor/notification.
However, I would avoid passing a callback to a function for that.

Rather adding a notification list [1] to each watch. So if a single watch change (updates any of its validities), we no longer need to call the global watch changed.
Downside, if all watches change (to invalid) we don't want to call hundreds of callbacks.

So there are some design choices to be made.
There needs to be a good design to split into notifications for individual watches, and for groups of watches (usually the whole list)

There could in future be more than one monitor.
One idea is to allow to open more than one watches-window. So you can display an history entry next to a live value. (or 2 values from different stack frames).
But then (even if not sensible) a user could  have both windows showing the same data too (the list of watches is in TWatches, it is not bound to the window).

Reply by email

Joost van der Sluis
to Michael Van Canneyt: Impressive job... Looking at the screencast, I think changing the current interface with…

Op 14-09-2021 om 14:26 schreef Michael Van Canneyt via lazarus:

Show quoted text 24 lines
On Tue, 14 Sep 2021, Joost van der Sluis via lazarus wrote: > Hi all, > > I'm playing with several debuging-concepts. And I want to share this > one with you. > > I've created a new Lazarus-debugger LazDabDebugger that can work with > FpdServer. It depends on a package (LazDebugExtensionIntf) that adds > another way of showing debugging variables inside Lazarus. > > I can write tons of texts about the background and possibilities. But > for now I'm just curious at your input. > > Look at the screencast or install the debugger, and share your > opinion(s). Impressive job... Looking at the screencast, I think changing the current interface with the one presented in the screencast would be a definite improvement; regardless of the backend used...

Yes, but that needs work on the backends. And I think we can better improve and fine-tune the interface first. At this moment this is easily done. Once there are multiple backends, that becomes more work.

Regards,

Joost.

Reply by email

Martin Frb
to Joost van der Sluis
Joost van der Sluis wrote 9 lines
> Looking at the screencast, I think changing the current interface > with the > one presented in the screencast would be a definite improvement; > regardless of the backend used... Yes, but that needs work on the backends. And I think we can better improve and fine-tune the interface first. At this moment this is easily done. Once there are multiple backends, that becomes more work.

If it is a flag, then each backend can choose to support it.

Reply by email

Michael Van Canneyt
to Joost van der Sluis: Op 14-09-2021 om 14:26 schreef Michael Van Canneyt via lazarus: Yes, but that needs work…
Joost van der Sluis wrote 31 lines
Op 14-09-2021 om 14:26 schreef Michael Van Canneyt via lazarus: > > > On Tue, 14 Sep 2021, Joost van der Sluis via lazarus wrote: > >> Hi all, >> >> I'm playing with several debuging-concepts. And I want to share this >> one with you. >> >> I've created a new Lazarus-debugger LazDabDebugger that can work with >> FpdServer. It depends on a package (LazDebugExtensionIntf) that adds >> another way of showing debugging variables inside Lazarus. >> >> I can write tons of texts about the background and possibilities. But >> for now I'm just curious at your input. >> >> Look at the screencast or install the debugger, and share your >> opinion(s). > > Impressive job... > > Looking at the screencast, I think changing the current interface with the > one presented in the screencast would be a definite improvement; > regardless of the backend used... Yes, but that needs work on the backends. And I think we can better improve and fine-tune the interface first. At this moment this is easily done. Once there are multiple backends, that becomes more work.

I think you indeed better fine-tune first. It could be that the fine-tuning necessitates changes in the API, so better to do this with 1 backend. Once the API is stable, expand to other backends.

Michael.

Reply by email

Joost van der Sluis
to Martin Frb: I started about "sub watches" before reading your "every watch has a reference" ....…

Op 14-09-2021 om 14:27 schreef Martin Frb via lazarus:

Show quoted text 14 lines
I started about "sub watches" before reading your "every watch has a reference" .... On 14/09/2021 11:00, Joost van der Sluis via lazarus wrote: > Op 14-09-2021 om 02:18 schreef Martin Frb via lazarus: 1) All watches should be part of the "watches" list. (or alternative means must be found). This is so the debugger can update them, if the user changes memory (set a new value to a variable). This is also helpful for the history window, and for "idle eval" (if the watches window is closed but the debugger is idle, then eval starts anyway).

Yes, as you look carefully at the screencast, you'll see that the history-option is removed. ;)

And idle... I didn't really thought about that. My idea was that the gui is the most 'responsible' place to determine when to request data. And in this case this is the dialog itself. The dialogs in the screencast fetch their data only when they have to be showed, just to make it fast.

Catching data in advance when the debugger is idle slightly contradict with that approach. Although, the 'gui'/dialogs can always decide to do so. But in that case a 'global cache' of retrieved variables might be useful, yes.

2) Currently formatting  (hex vs dec) is done in the debugger. That is also nonsense. As much as can be should be moved to the frontend.

Yes, I agree. (But for the fpdserver-backend I use the DAB-protocol which has all kind of issues with this)

3) Btw, if arrays can be expanded, precautions are needed. I don't think we want to expand 100000 child values.

The fpdserver has a solution for that, I only did not implement it on the Lazarus-side. But it basically adds paging, together with the load-on-demand features of the treeview, this should be no problem.

---- If we go for the flag and sub-watches, then "debug history" needs to be aware of it.

Yup, that one needs work.

Show quoted text 12 lines
>> Ideally the entire debugger frontend could move into a package of its >> own too (all the current windows). > > Exactly, that is why I created a new debug-intf package instead of > adapting the existing one. Well we may need to create an interface for the debugger frontend, and then add the frontend package. Though DebugManager already has parts of that. On the other hand its not strictly mandatory yet. As I see it your new watches can simply replace the old one. (as soon as watches provide that flag, so old watches can be displayed too)

I'll see if I can implement that flag.

Show quoted text 20 lines
>> Currently the debugger intf allows to fetch watches as plain text, or >> as structure. But this has to be passed in as flag at the request time. >> >> Any watch object should instead have methods to fetch "sub watches" >> so structures can unfold. > > What I have now is that each object has a 'reference', and there is a > separate call to get the sub-watches that belong to the reference. > That way the data (in the object) and the logic (function to retrieve > sub-watches) are separated. Ok, I have to look at that. Another important factor on the design of every debugger-object (eg watches) is "notify on free" or TRefCountedObject. Any debugger backend holds on to the "watch object" while it evaluates the data needed. However the frontend may remove (and free/dereference) that watch during that time. The backend must know that. (and potentially even call the callback, if the object no longer exists.... / the editor hint code requires this )

That's easy. The gui requests a variable using a reference. Once it retrieves the callback with the data that response contains the reference. If the gui does not need that reference anymore, just throw away the data.

The request to the backend only contains the reference. The response is a newly created object, that the receiving party gets ownership to. And, as long as there is no global-cache, a variable has only one 'owner'.

All the gui-related stuff goes into the main thread.

Show quoted text 34 lines
> > I don't have a solution for requesting the result as text(string) > only. But that's not a bad idea. Currently we have (as param to eval)   TDBGEvaluateFlag =     (defNoTypeInfo,        // No Typeinfo object will be returned      defSimpleTypeInfo,    // Returns: Kind (skSimple, skClass, ..); TypeName (but does make no attempt to avoid an alias)      defFullTypeInfo,      // Get all typeinfo, resolve all anchestors      defClassAutoCast,     // Find real class of instance, and use, instead of declared class of variable      defAllowFunctionCall     );   TDBGEvaluateFlags = set of TDBGEvaluateFlag; Currently   TWatchValue = class(TFreeNotifyingObject)      property TypeInfo: TDBGType; and TDBGType = class(TObject)   property Fields: TDBGFields; which is a list of TDBGField So if (full)typeinfo is request, you have a list of fields, and you can then get the sub-watches. But that is not a good way.... That means you then have lots of dup data => each subwatch, has some data also in a field. ** The other downside is, that the gdb debugger needs more time to fill the fields. So not every debugger should provide them by default.....

The big advantage of the tree-view, is that you do not evaluate sub-watches, only when they have to be shown. So only those fields that need to be shown have to be evaluated by gdb. (It also solves the endless loop you sometimes have)

Show quoted text 35 lines
** The upside is, that having type info (and fields) the frontend can do all the formatting (almost... / mem dump still needs to get data as byte stream) If the backend delivers a point (x,y) as 2 fields, the frontend can decide to print "(x: 1; y:2)"  or "(1,2)". The frontend can also print the individual values in dec,hex,bin,oct.... So always having type info would be nice. But for gdb only remote debug targets, in needs to be seen if it can be done, without slow down. It should be possible with one limitations. Currently each Field has type (tkInteger, tkClass, tkRecord, ....) and ClassName (indicating which parent class introduced that field). Getting those fields is time consuming. =================================== So as a first draft I would aim for something like this: TDBGField needs to be a TWatch. => avoid data duplication / and provide for recursive eval. By default the (if no flags are set) the debugger should provide fields. But only field name, and a value (int or string). All other info can be retrieved later, on request. In fact, even the field info can be retrieved on request (editor hint may not need it, but if formatting code goes to the frontend, then editor hint also needs to do the formatting). In the same way, that a watch has a "validity" field (dsEvaluating, dsValid, dsError, ...) it can have validity for typeinfo and fields. On top of that validity for typeinfo/fields can be dsNotAvail. In which case there just is a default text value for the watch.

Let's not over complicate things (immediately). But let's see which functionality we need, and then come up with a design for that.

Show quoted text 9 lines
>> I see you pass the variable to the debugger, that is ok. But that may >> well get abstracted into the Watch, so the watch has its own ref to >> the debugger, and makes the needed calls. > > I don't follow exactly. But if there's a need for an extra abstraction > layer before sending the variable to the debugger directly, this could > be added. See above, if the subwatches are already in place of the current fields, then they have a ds...validity. And if accessed they will further evaluate.

Ok, let's dive something more into this particular case. I used flags for variables, for example to indicate that they are 'loading'. But at this moment I do not have an error-flag. When the 'gui' requests a variable from the backend, it can receive an error. In that cast it just fills in the error-message as the value of the variable and it's done.

Ok, when we want to have an history, this might not be what you want, but in that case the 'gui' (the history being part of the gui) can decide to do otherwise.

Show quoted text 25 lines
>> Not sure about passing callbacks directly. Thinking of the current >> notification system (data monitors). >> What does your new watch window do, if the user modifies a variable? >> This triggers re-eval of all watches (so they need to unfold again). >> Currently the debugger backend can trigger the re-eval, and through >> the data-monitors tell everyone about the updates. >> >> Have to look at it in more detail.... > > I have something similar as the data-monitors. > > Main difference is that the events are based on what the gui needs, > instead of what the debugger does. > > So: there is an event to tell the 'gui' that it is inpossible to show > any data. (for example: the application has stopped or there is no > debugger at all) There is an event for the case that debug-data has > become available or unavailable. (Application paused or continued) > > And I should add an event for the case that the debug-info should be > re-evaluated. (context switch, or variable-change). And I can think of > another event that can be send when the debug-data could have been > changed, due to unforeseen effect, for example when a propperty with a > getter has been evaluated. All of those are available already

Yes, I know. Basicly everything I made is available already. But by changing a few paradigma's I hope that things can become less complex. (But time will tell)

Show quoted text 16 lines
Most of those are DebuggerState changes. They are currently propagated hardcoded to each of the existing listeners. So they may need an API to subscribe to. But they also need to be done in a prioritized order. Certain data needs to be done in specific order (partly that is enforced in the backend anyway) Context changes are subscribe-able.   CallStackNotification.OnChange   := @CallStackChanged;   CallStackNotification.OnCurrent  := @CallStackCurrent;   BreakpointsNotification.OnAdd    := @BreakPointChanged;   BreakpointsNotification.OnUpdate := @BreakPointChanged;   BreakpointsNotification.OnRemove := @BreakPointChanged; Note that the debugger is not the only thing that can change context. If the user selects a history entry, then all watches change.

I was thinking about a user thanging the current entry in a call-stack, but it's basivly the same, yes.

Show quoted text 29 lines
> The advantage of 'events' that are more geared towards the gui, is > that it is easier to add catchy things. For example: to avoid > flickering, when the gui receives an event that debug-data is not > available anymore, it starts a timer of 200 mseconds, and only after > that timer the data is removed from the screen. This way, when > stepping through the code, there is no flickering in the small periods > of time that the application runs. That can be done with the above Notification too. > > It also made it easy to highlight variables that changed. Ah, but does it? Say I have an object with a delegate (object / class based => that is pointer to heap). TBar = class   i: integer; end; TFoo = class   bar: TBar; end; foo.bar := otherBar. If otherBar has the same value in "i", and if the address of "bar" in tfoo is not displayed. => then the displayed result of the "foo" watch does not change. The value however has changed.

True, but I don't think users will matter much. And, the field that displays the class itself also includes the address, and that one *will* change.

Show quoted text 25 lines
> But atm they are only suitable for displaying variables. Stuff like > the call-stack are not touched. In generally there may be reasons to extend on the current monitor/notification. However, I would avoid passing a callback to a function for that. Rather adding a notification list [1] to each watch. So if a single watch change (updates any of its validities), we no longer need to call the global watch changed. Downside, if all watches change (to invalid) we don't want to call hundreds of callbacks. So there are some design choices to be made. There needs to be a good design to split into notifications for individual watches, and for groups of watches (usually the whole list) There could in future be more than one monitor. One idea is to allow to open more than one watches-window. So you can display an history entry next to a live value. (or 2 values from different stack frames). But then (even if not sensible) a user could  have both windows showing the same data too (the list of watches is in TWatches, it is not bound to the window).

Regards,

Joost.

Reply by email

Martin Frb
to Joost van der Sluis
Joost van der Sluis wrote 11 lines
> 1) > All watches should be part of the "watches" list. (or alternative > means must be found). This is so the debugger can update them, if the > user changes memory (set a new value to a variable). > This is also helpful for the history window, and for "idle eval" (if > the watches window is closed but the debugger is idle, then eval > starts anyway). Yes, as you look carefully at the screencast, you'll see that the history-option is removed. ;)

I am using the history a lot. So that needs to be fixed

Show quoted text 6 lines
And idle... I didn't really thought about that. My idea was that the gui is the most 'responsible' place to determine when to request data. And in this case this is the dialog itself. The dialogs in the screencast fetch their data only when they have to be showed, just to make it fast.

Well, with fpdebug eval is fast. But still, if you stop (maybe unexpected on an exception), it is good if the debugger already did the work.
And it is needed anyway for the history.

Catching data in advance when the debugger is idle slightly contradict with that approach. Although, the 'gui'/dialogs can always decide to do so. But in that case a 'global cache' of retrieved variables might be useful, yes.

In fact, IIRC, it is not the debugger that acts on IDLE. the debugger just triggers an event.
It is the history, that then starts evaluating.

Show quoted text 6 lines
> On the other hand its not strictly mandatory yet. As I see it your > new watches can simply replace the old one. (as soon as watches > provide that flag, so old watches can be displayed too) I'll see if I can implement that flag.

See also what to do about TDbgFields. I think they are crucial here. See below.

Show quoted text 11 lines
That's easy. The gui requests a variable using a reference. Once it retrieves the callback with the data that response contains the reference. If the gui does not need that reference anymore, just throw away the data. The request to the backend only contains the reference. The response is a newly created object, that the receiving party gets ownership to. And, as long as there is no global-cache, a variable has only one 'owner'. All the gui-related stuff goes into the main thread.

Well yes, thats because you no longer use the global TWatches list. (Though subwatches would be the same)

Current eval works by creating a TWatch on the IDE side, and pass that object to the debugger.
And that is important, because "eval" carries more than just the watch expression "SomeVar".
It may request a memory dump. Or allow function calls. Other current options (formatting hex/dec) may get dropped. But still....

Also the debugger backend should keep access. So it can invalidate a watch, when it continues running.
That way, invalidate can happen in a single place, and not be spread to each dialog that has its own set of watches.
(dialogs can still defer updating the display)

Show quoted text 7 lines
> > On top of that validity for typeinfo/fields can be dsNotAvail. In > which case there just is a default text value for the watch. Let's not over complicate things (immediately). But let's see which functionality we need, and then come up with a design for that.

Yep.

But even if we implement only one backend to start with, we need to keep the other in mind, to come up with an universal enough design. Also the design should allow different frontends.

IHMO, new TWatches must be created for each opened value in the tree (for arrays some sort of fly-weight object may be done).

Then the question is, if those watches should be (directly) added to the global "TWatches" list. (debug inspector may need such a feature anyway, so the global list may anyway end up with values not displayed on top-level watch window).

The other question is how to return what currently is in TDbgFields. So there is the least amount of data duplication.

Show quoted text 18 lines
>>> I see you pass the variable to the debugger, that is ok. But that >>> may well get abstracted into the Watch, so the watch has its own >>> ref to the debugger, and makes the needed calls. >> >> I don't follow exactly. But if there's a need for an extra >> abstraction layer before sending the variable to the debugger >> directly, this could be added. > See above, if the subwatches are already in place of the current > fields, then they have a ds...validity. And if accessed they will > further evaluate. Ok, let's dive something more into this particular case. I used flags for variables, for example to indicate that they are 'loading'. But at this moment I do not have an error-flag. When the 'gui' requests a variable from the backend, it can receive an error. In that cast it just fills in the error-message as the value of the variable and it's done.

Not sure I follow.
But that is why I advocate to create a TWatch on the gui site, and pass that TWatch to the debugger to be filled in.
That way you have the full "validity" property.

Show quoted text 19 lines
>> Main difference is that the events are based on what the gui needs, >> instead of what the debugger does. >> >> So: there is an event to tell the 'gui' that it is inpossible to >> show any data. (for example: the application has stopped or there is >> no debugger at all) There is an event for the case that debug-data >> has become available or unavailable. (Application paused or continued) >> >> And I should add an event for the case that the debug-info should be >> re-evaluated. (context switch, or variable-change). And I can think >> of another event that can be send when the debug-data could have >> been changed, due to unforeseen effect, for example when a propperty >> with a getter has been evaluated. > All of those are available already Yes, I know. Basicly everything I made is available already. But by changing a few paradigma's I hope that things can become less complex. (But time will tell)

One thing is that there are more backends than frontends. And that is likely to stay. So defining frontend specific events puts a lot of burden on all backends.
IMHO the backend driven events are the way to go.

Also in the end, the frontend events can only be translations of the backend events.
The only point is to make sure we make all relevant events available, and possible allow selective access (per watch, rather than for all watches)

Yet, if all watches change together, we don't want to fire all those events. There should be only one, saying all changed.

I am thinking along the lines that a listener can  subscribe for events (such as validity changed). It can do so on a per watch base (filter). It gets a message if any of the watches that it is subscribed to triggers that event.
There is an optional field SenderWatch, which will be set if only one changed.
If all or several changed, then SenderWatch is nil.

Or something  like that.

Reply by email

Joost van der Sluis
to Martin Frb

Op 14-09-2021 om 17:12 schreef Martin Frb via lazarus:

Show quoted text 13 lines
On 14/09/2021 16:11, Joost van der Sluis via lazarus wrote: > >> 1) >> All watches should be part of the "watches" list. (or alternative >> means must be found). This is so the debugger can update them, if the >> user changes memory (set a new value to a variable). >> This is also helpful for the history window, and for "idle eval" (if >> the watches window is closed but the debugger is idle, then eval >> starts anyway). > > Yes, as you look carefully at the screencast, you'll see that the > history-option is removed. ;) I am using the history a lot. So that needs to be fixed

;) I'll have to look into it, I never used it, really.

But the next two weeks I'm on vacation, so it will have to wait.

Regards,

Joost.

Reply by email

Joost van der Sluis
to Joost van der Sluis

Op 17-09-2021 om 21:19 schreef Joost van der Sluis via lazarus:

Show quoted text 18 lines
Op 14-09-2021 om 17:12 schreef Martin Frb via lazarus: > On 14/09/2021 16:11, Joost van der Sluis via lazarus wrote: >> >>> 1) >>> All watches should be part of the "watches" list. (or alternative >>> means must be found). This is so the debugger can update them, if >>> the user changes memory (set a new value to a variable). >>> This is also helpful for the history window, and for "idle eval" (if >>> the watches window is closed but the debugger is idle, then eval >>> starts anyway). >> >> Yes, as you look carefully at the screencast, you'll see that the >> history-option is removed. ;) > I am using the history a lot. So that needs to be fixed ;) I'll have to look into it, I never used it, really. But the next two weeks I'm on vacation, so it will have to wait.

Update: I've implemented watches and improved the list of threads and the callstack.

This discussion in mind myu first intention was to create some kind of global list with watches. But in the end I decided not to do something like that. After all, a 'watch' isn't something on it's own. There is a window that shows a few watches. So I made the list of watches part of the dialog.

Makes it also possible to show multiple dialogs with each it's own watches.

But only the history will get a challenge. But this also need some more thought.

Regards,

Joost.

Reply by email

Martin Frb
to Joost van der Sluis: I'm playing with several debuging-concepts. And I want to share this one with you. I've…
Joost van der Sluis wrote 5 lines
I've created a new Lazarus-debugger LazDabDebugger that can work with FpdServer. It depends on a package (LazDebugExtensionIntf) that adds another way of showing debugging variables inside Lazarus. I can write tons of texts about the background and possibilities. But for now I'm just curious at your input.

Ok, lets try to pick up on where we were.  / Going through the thread, picking out previous comments.

The main points seem to be:

1) Events to be more frontend centric.
2) Actions (eval watch) to be triggered by the frontend
3) Representation of an "watch"  (Should methods be on the Watch object, or the Debugger object)

I think, I will write individual mails on each of them.

Reply by email

Martin Frb
to Joost van der Sluis: I'm playing with several debuging-concepts. And I want to share this one with you. I've…
1) Events to be more frontend centric. 2) Actions (eval watch) to be triggered by the frontend 3) Representation of an "watch"  (Should methods be on the Watch object, or the Debugger object)

=====> 1) Events to be more frontend centric.

The current events and especially states are certainly in need of clean up.

I would not say that they are all backend centric. States like: dsRun, dsPause, dsStop  are very valid and meaningful for the frontends too.

On the other hand states like
- dsDestroying is backend only, and should not be exposed.
- dsInit, dsInternalPause are not well at your code, I do not agree with your new states.

Show quoted text 9 lines
  TLazExtEvaluationState = (     esNone,                  // No debugger available     esEvaluationImpossible,  // Evaluation of variables is impossible (Maybe because there is nu debugee? (stopped))     esEvaluationPaused,      // Evaluation of variables is temporarily disabled (Maybe because debugee is running?)     esEvaluationPossible     // Evaluation of variables is possible (Debugee is probably paused)   );

1 - First of all,
The above are either
a- not enough
b- to many

a) There may be different extends of evaluation in the frontend. So more info than the above few states is needed.
=> Of course, the states may be sufficient, as further info could be maintained within the frontend.
* I quite often use breakpoints, that do not "break" (continue automatically), but take a history snapshot.
Currently when this happens the watch/stack/.. windows actually do update (and that is a workaround to missing options). But ideally they should not. Only the history-snapshot should run.
* Or while viewing history => one can get existing values, but not evaluate new ones.

b) On the other hand, why differ between Paused and Impossible? They both mean "can't eval".
Your frontend may differ between the 2 reasons behind them. But other Frontends, relay on other factors.

After writing all the below, I realized that (probably) your frontend uses the 2 states to decide, if there should be a timeout before clearing the values?
But that is very specific to your frontend (well most frontends probably would benefit). But then this is just a duplication of the existing and further needed dsStop.

See idea on "CurrentCapacities" below.

2 - Second,
they already exist:
   WatchValue.Validity => ddsUnknown, ddsRequested, ddsEvaluating,  ddsValid, ddsInvalid, ddsError
Though they are only set, once a value gets requested. (Asking upfront, may be an idea, but one only need "function CanEval: boolean". / See idea on "CurrentCapacities" below.)

This falls partly under the topic (3) representation => should the state come via the watch or the debugger.

And then, if history is shown, there is no point of calling a function on the debugger. There is no data to be computed by the debugger.

3 - Third
Also, while your argument for a frontend-needs driven event system is compiling, we need to keep in mind, that the frontend (or plural frontends) may change/differ how they want to react to different backend states.
The backends can not actually know this. So the frontend still needs (most of) the DebuggerStates (dsPause, dsRun, dsStopped)

Yet, as mentioned above, (sub)states may need to be added for different "pause reasons".
This may be something (some/all) frontends can do on their own. But some may also belong into the backend. (Maybe such as, that the backend is going to continue the run/step)

If a breakpoint has "no break" or "auto-continue" (the latter differs from the first), then parts of the frontend may need to know. This can to some extend be managed in the frontend itself, by examining the current breakpoint(s) [1].
But it can't be entirely taken from the backend, because if the backends state is "mid single step", then the frontend can not sent a "step over". If maintaining some of the state info is moved to the frontend, this must avoid duplicating info that is bound to the backend. (to avoid them becoming out of sync).

[1] Btw, off topic, there may be more than one breakpoint on the same address, so there needs to be a list of breakpoints hit...

===> Going forward

It might be best to look at an actual list of new states and events.

- What do different parts of the frontend need to react to (events)?   And can/should the backend(s) know about each of those "causes".

- What states are important, from a frontend/users view.
- What mapping, between abilities and state do we need, and where to compute them.

For example the debugger already has
    function  GetCommands: TDBGCommands; virtual; // property Commands And this returns what is currently possible
    if dcEvaluate in Commands then {watches can be evaluated}
- Except, that may be returning wrong for dsInternalPause => so there is some fixing needed.
- The naming is bad, because its more than just commands. Maybe should be:    ~ dcEvaluate in CurrentCapacities
   ~ Debugger.Is[Currently]Capable(dcEvaluate)
   ~ dcEvaluate may be to narrow, if it includes ability to read/provide stack/thread/... Or maybe there can be separate enums.

As for events, here are the events that currently can trigger watches to be refreshed:

By the debugger (IIRC)
  WatchesNotification.OnUpdate    := @WatchUpdate;     // send by the debugger  WITH watch=NIL, equal to the callback in your code
    // send by the debugger  WITH watchNIL, equal to edeEvalutionStateChanged
  ThreadsNotification.OnCurrent   := @ContextChanged;   CallstackNotification.OnCurrent := @ContextChanged;

By the frontend itself
   WatchesNotification.OnAdd       := @WatchAdd;
   WatchesNotification.OnRemove    := @WatchRemove;
   SnapshotNotification.OnCurrent  := @SnapshotChanged;

WatchesNotification.OnUpdate could be split into 2 events. It is also the same for start/end availability of data. That may want to be extended. Though the info can already be retrieved from the debugger. (DebuggerState / Commands)

Yet it is not even needed to get the state. (Or should not, I have to check if it works).
The watchvalues validity should be enough.
If correct, it should be ddsInvalid.
If a "ddsEvalNotPossible" is added, then the info is avail (and the timeout can be set, depending on debuggerState)

The threads and callstack need to be separate notification, Different windows may subscribe to different sets of them.

So I am not sure, if indeed we need an entire new event system. One could argue that thread and stack should trigger WatchesNotification.OnUpdate.
But that would be wrong.
=> I have often thought it would be good to have a 2nd (or more) watch window, and have a setting for that window to stay at a fixed thread/stack, even if the selection in the stack window is changed.
- So such a window, would not change if the current stack in the stackwindow was changed
- But it would need to reload, if the user changed the value of a watch, and therefore modified the target memory. (which could affect other watches too)

So in the end, the watches window needs to be able to decide which events it wants.

Joost van der Sluis wrote 10 lines
I have something similar as the data-monitors. Main difference is that the events are based on what the gui needs, instead of what the debugger does. So: there is an event to tell the 'gui' that it is inpossible to show any data. (for example: the application has stopped or there is no debugger at all) There is an event for the case that debug-data has become available or unavailable. (Application paused or continued)

 We have an event, when watches can start to be evaled "WatchesNotification.OnUpdate".
See above.

Show quoted text 11 lines
The advantage of 'events' that are more geared towards the gui, is that it is easier to add catchy things. For example: to avoid flickering, when the gui receives an event that debug-data is not available anymore, it starts a timer of 200 mseconds, and only after that timer the data is removed from the screen. This way, when stepping through the code, there is no flickering in the small periods of time that the application runs. It also made it easy to highlight variables that changed.

Both can be done, with the existing system. Though some optimizations can be applied.

Reply by email

Martin Frb
to Joost van der Sluis: I'm playing with several debuging-concepts. And I want to share this one with you. I've…
1) Events to be more frontend centric. 2) Actions (eval watch) to be triggered by the frontend 3) Representation of an "watch"  (Should methods be on the Watch object, or the Debugger object)

=====> 2) Actions (eval watch) to be triggered by the frontend

The evaluation of a watch is triggered when the frontend attempts to access it. The backend then gets a call to      TWatchesSupplier.InternalRequestData(AWatchValue: TWatchValue) for that one watch.

For all I can see neither gdb, nor Fp backends will evaluate any watch, unless the frontend asks.

=====> 3) Representation of an "watch"  (Should methods be on the Watch object, or the Debugger object)

You introduced a new method to get a watch value.
  Debugger.EvaluateExpression(S, Debugger.GetCurrentFrameID, FLatestRef, @EvaluateCallback)

and then in the callback the data is in an object: TLazDebugExtensionVariable
TLazDebugExtensionVariable has a reference, where the debugger can store any data it wants: integer or object

I am trying to see, what advantage this should bring over the current system:

* TIdeWatch / TCurrentWatch
  => those can nicely store all the data a frontend needs to store with the watch (any options the user did chose).
  => this also stores (as TWatchValue) the result returned from the debugger.
  => Each frontend can subclass it as it wants.
* The frontend decides when it wants the data. The backend then receives a single call for this/each watch.
* The backend then send a notification for the frontend, when the data is there.

Same flow? Main difference is that the methods to invoke the evaluation are on the TWatch.

--------------
So far so good.
- The TWatch class is currently quite big. It might benefit from some cut down, but that is not related to any of the things you implemented.

- An advantage is, that a watch eval can be aborted.   Your concept is to discard unwanted results, but the backend still spends time on evaluating it. Of course you could add      dbg.AbortEval(reference);
  In the end, I prefer to encapsulate this into the TWatch.

- Watches are stored in a list.
That is needed by the frontend. Because the History wants to know about the watches.
However, this may not be needed by the backend. That can be reviewed.

--------------
Collapsing / Expanding

Currently watches can only be added by the frontend. That may need to be changed.
Each watch needs a way to ask the debugger for the TWatchValue data of its children.
=> For that the Watch could call the backend
  dbg.GetChildren(self);
=>  The backend would then for each child call
  watch.GetChildWatch
and the backend would put the data in.

Something like that.....

As for the TWatches list:
The watch window would know, which watches are visible (expanded) and evaluate them.
For the history, each watch could have a "parent" and/or "CurrentVisible" field. Or the watch has an "Owner" (the window), which can be asked.

 Joost van der Sluis via lazarus  wrote
The big advantage of the tree-view, is that you do not evaluate sub-watches, only when they have to be shown. So only those fields that need to be shown have to be evaluated by gdb. (It also solves the endless loop you sometimes have)

Well at latest when the debugger is idle, evaluation of watches "not scrolled in" should start.
So when the user starts scrolling, values can be shown immediately.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20211121/5d77d122/attachment.html>

Reply by email

Martin Frb
to Martin Frb
Martin Frb wrote 5 lines
- Watches are stored in a list. That is needed by the frontend. Because the History wants to know about the watches. However, this may not be needed by the backend. That can be reviewed.

Actually, there only is a list for the frontend.

It is currently part of the DbgIntf. But it should not be. And should be easily fixable.

The backend site has (through a forward handler) access to it. The only use that is made of it, is to invalidate all watch values, if they changed (actually this is only used, if the user modified a watch).

The debugger should in that case send an event (or call an abstract method in the base class). This can then clear the values.

And the entire TWatches list can move up, into the current frontend.

Reply by email