Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2008-12

The Data-Model for SynEdit / The Data Model for Lazarus [Code Structure / SourceEdit and SyneEdit]

10 messages · 5 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Martin Friebe

Currently SynEdit handles a lot of different DataSources. Such as:
- The raw text
- Bookmark
- folded/unfolded nodes
- ...

All of this should be collected into a single access point (Model). Such a Model could then be shared between 2 or more SynEdits (to view the same data, in more than one window).
- For the Viewing in different Windows there may be a need for individual SynEdits to have privatized parts of the public/shared Model. Such as different nodes may be folded.
- The public/shareable model would have to be ref-counted. Or explicit ownership would need to be assigned and the owner would need to keep track of all Shareholders.
- The Model can be serialized. Special classes would to this work. Therefore a model can be saved to one file, or to a set of files

Those are just a few points to start looking at design details for such a model.

So far this only concerns SynEdit.

But for Lazarus a benefit could be, that such a model could also contain data, on top of the information hold for synEdit. The Lfm (FormDesigner) info could become part of this.
The it would only need an appropriate serialize class to save all of them.

This may simplify the current implementation (even so I admit much based on assumption on fragmented knowledge) which has to retrieve this info from the various components. (the need for the source-editor or main ide to fetch the text from SynEdit)

The Model could also carry callback lists, so the anybody who needs to know that it was loaded or saved could register a callback Model.AddOnSerailizedCallback( AMethod )

AMethod may be called with the reason, why serialization took place (e.g saved to file)

Just starting some basic thought on it. It will be a very long way, and a very long time before any of this goes into implementation.

Best regards
Martin

Show quoted text 53 lines
In some places below, I am unsure what you meant by "component": - the "view" (as in MVC) - the overall editor (as in SynEdit) Hans-Peter Diettrich wrote: > Martin Friebe schrieb: > >> Model shall contain >> - the raw text >> - bookmarks, and other marks >> - foldable secitions >> - ... >> >> Other information can be local to a specific Component (in case >> multiply component display the same model in more than one window). >> An example would be which sections are folded. This info may or may >> not be part of the model >> > > Right. We should start with the persistent information, stored > together with the text on disk, then add all information that is > common to all (multiple) Views. > Serializing the Model for storage purposes (such as saving to a file or many files) for me comes after the decision how the model looks. There may be more than one way to serialize the model. As there are many ways to use SynEdit. - In Lazarus it is used a SourceEditor, so serializing should write the Text information into one file, and other information into another file. - In User Applications, people may wish to serialize the model into just one file (which can not be edited by other editors) >> We could differ between a public and a private model. (Probably not >> needed) >> > Most probably not needed in this form, default is the public Model. Actually we may need. Example folding. Having the same model displayed in many Windows, the user may want to fold different nodes in each Window. Or even use different highlighters, leading to different nodes being available. The design of this must leave the choice to the user, which information is hold in the public, and which in the private model. The private Model can store the same info as the public. The private model only stores info the user whises to be different from the public model, all other info will be forwarded. However concerning the design of the other Classes (Controller/View), the differentation between private or public model is not relevant. They access one Model. This Model knows what to do.

[.....]

Reply by email

Martin Friebe
to Mattias Gärtner
Mattias G?rtner wrote 16 lines
Zitat von Martin Friebe : > [...] > But for Lazarus a benefit could be, that such a model could also contain > data, on top of the information hold for synEdit. The Lfm (FormDesigner) > info could become part of this. > What has synedit to do with the form designer? I'm not sure how synedit should handle the unit concept. The lfm is associated with the unit. A unit has a starting file (e.g. unit1.pas) and an arbitrary list of include files, plus the po file of the project/package, the fpdoc xml file and .res files. Each of them work together and need very different code.

The other way round. Synedit has no interest in the other structures. But the IDE may have an interest in a structure holding the combined data.

As I said, I have not looked at how this is currently handled. Or if there is a common DataStructure for a "pascal Unit". I will inform my self (by reading up the source) when I have the time.

There was a discussion between you and Paul on the dev-list:

Show quoted text 6 lines
doable except one thing. I dont know where to hook before unit save > > event. We just dont have it. Moreover unit save is not that plain > > as one can expect. At least I expected sourceeditor have something > > like Save method. No, because the source editor is only one part of the IDE, so a save must be handled one level higher

"saving is not as plain as one can expect"

the Ide will or should have some structure holding the combined data. But at current it appears it has to go down to code-tools and/or SynEdit (or SynEdit via code-tools) to get the actual Text.

Synedits Data holding structure could either be more integrated into that IDE structure (if that is there, I will look for it later). Or It could be subclassed, extended and used for a higher purpose than just SynEdit.

I will have to look at the current structures to see if that makes any just threw it in as an Idea, seeing if I get some feedback.

Show quoted text 14 lines
> The it would only need an appropriate serialize class to save all of them. > > This may simplify the current implementation (even so I admit much based > on assumption on fragmented knowledge) which has to retrieve this info > from the various components. (the need for the source-editor or main ide > to fetch the text from SynEdit) > synedit is the 'lines' view of the sources. Most of the lazarus code don't work with lines, but with whole files or absolute positions.

So what if SynEdit could have a "model" (An object exposing it's data) in both ways?

This is not a proposal to throw over all the working code. This is first to see which directions make sense inside SynEdit, and maybe SourceEditor at some stage.
Once that has developed into a more modular form, and gained the ability to provide that functionality, it can be reviewed, if any other code could benefit from it.

Best regards
Martin

Reply by email

Mattias Gärtner
to Martin Friebe

Zitat von Martin Friebe :

Show quoted text 24 lines
Mattias G?rtner wrote: > Zitat von Martin Friebe : > > >> [...] >> But for Lazarus a benefit could be, that such a model could also contain >> data, on top of the information hold for synEdit. The Lfm (FormDesigner) >> info could become part of this. >> > > What has synedit to do with the form designer? > I'm not sure how synedit should handle the unit concept. > The lfm is associated with the unit. A unit has a starting file (e.g. > unit1.pas) > and an arbitrary list of include files, plus the po file of the > project/package, > the fpdoc xml file and .res files. Each of them work together and need very > different code. > The other way round. Synedit has no interest in the other structures. But the IDE may have an interest in a structure holding the combined data. As I said, I have not looked at how this is currently handled. Or if there is a common DataStructure for a "pascal Unit".

There is no single data structure that holds all data of one unit.

Show quoted text 11 lines
I will inform my self (by reading up the source) when I have the time. There was a discussion between you and Paul on the dev-list: > doable except one thing. I dont know where to hook before unit save > > > event. We just dont have it. Moreover unit save is not that plain > > > as one can expect. At least I expected sourceeditor have something > > > like Save method. > No, because the source editor is only one part of the IDE, so a save > must be handled one level higher "saving is not as plain as one can expect"

yes

the Ide will or should have some structure holding the combined data.

Sometimes only parts are known.
The glue class is TUnitInfo. It does not hold all data, but with it you get to everywhere else.

But at current it appears it has to go down to code-tools and/or SynEdit (or SynEdit via code-tools) to get the actual Text.

Correct. The actual file content is in the codetools. The actual lines as seen in the source editor is in synedit. The codehelp can hold the xml nodes. The form designer holds the lfm as component. ...

Show quoted text 26 lines
Synedits Data holding structure could either be more integrated into that IDE structure (if that is there, I will look for it later). Or It could be subclassed, extended and used for a higher purpose than just SynEdit. I will have to look at the current structures to see if that makes any sense at all. I just threw it in as an Idea, seeing if I get some feedback. > > >> The it would only need an appropriate serialize class to save all of them. >> >> This may simplify the current implementation (even so I admit much based >> on assumption on fragmented knowledge) which has to retrieve this info >> from the various components. (the need for the source-editor or main ide >> to fetch the text from SynEdit) >> > > synedit is the 'lines' view of the sources. Most of the lazarus code don't work > with lines, but with whole files or absolute positions. > > So what if SynEdit could have a "model" (An object exposing it's data) in both ways?

The codetools have that already.
But for speed you have to optimize in one way or the other. You can not have both fast.

Show quoted text 7 lines
This is not a proposal to throw over all the working code. This is first to see which directions make sense inside SynEdit, and maybe SourceEditor at some stage. Once that has developed into a more modular form, and gained the ability to provide that functionality, it can be reviewed, if any other code could benefit from it.

Mattias

Reply by email

Hans-Peter Diettrich
to Martin Friebe: Currently SynEdit handles a lot of different DataSources. Such as: - The raw text -…
- The public/shareable model would have to be ref-counted. Or explicit ownership would need to be assigned and the owner would need to keep track of all Shareholders.

The latter. The model must know about all views, to be notified of changes. This suggests a viewer chain, as used e.g. for Windows clipboard viewers.

- The Model can be serialized. Special classes would to this work. Therefore a model can be saved to one file, or to a set of files

A separation in multiple classes is dangerous, because changes to the model (number or type of values) had to be reflected in all related classes, in every project using that model.

The it would only need an appropriate serialize class to save all of them.

Or an accordingly overwritten virtual method.

The Model could also carry callback lists, so the anybody who needs to know that it was loaded or saved could register a callback Model.AddOnSerailizedCallback( AMethod )

The beforementioned list of view(er)s. When actions shall be monitored, then an implementation in the controller seems to make more sense to me.

DoDi

Reply by email

Hans-Peter Diettrich
to Mattias Gärtner: An embedded and charset-unspecified text was scrubbed... Name: not available Url:…
Show quoted text 5 lines
I'm not sure how synedit should handle the unit concept. The lfm is associated with the unit. A unit has a starting file (e.g. unit1.pas) and an arbitrary list of include files, plus the po file of the project/package, the fpdoc xml file and .res files. Each of them work together and need very different code.

No file should be opened more than once, so there must exist a centralized manager (file pool), keeping track of all open files and other related data collections. That manager should have a concept of units. In MVC terms the file pool is the Model, with various Views (editor, form designer, code explorer...), and everything is controlled by (an dedicated part of) the IDE.

synedit is the 'lines' view of the sources. Most of the lazarus code don't work with lines, but with whole files or absolute positions.

Right, but when a file is loaded, there must exist means to access its contents - no matter whether the contents in memory are organized in lines, there must exist means to map line numbers into the associated text. That mapping must be done in one dedicated place, so why not in the owner of the file itself?

DoDi

Reply by email

Martin Friebe
to Hans-Peter Diettrich: The latter. The model must know about all views, to be notified of changes. This suggests…
Hans-Peter Diettrich wrote 11 lines
Martin Friebe schrieb: > - The public/shareable model would have to be ref-counted. Or explicit > ownership would need to be assigned and the owner would need to keep > track of all Shareholders. > The latter. The model must know about all views, to be notified of changes. This suggests a viewer chain, as used e.g. for Windows clipboard viewers.

That are 2 different things.
Explicit Owner or ref-counting, both was about creation, and destruction of the model.

You are talking about notification on changes. Each "Synedit" that uses the model becomes an observer of it.

For this it can either enter itself on a list of known observers, or leave a callback reference (a reference to an object-method technically contains the reference to the object). Notifying all observers will then be easy.
It can be decided, if the Model should detect changes and trigger the notifications (after EndUpdate), or if the current Actor on the Model retrieves the List and sends the Notifications.

Show quoted text 8 lines
> - The Model can be serialized. Special classes would to this work. > Therefore a model can be saved to one file, or to a set of files > A separation in multiple classes is dangerous, because changes to the model (number or type of values) had to be reflected in all related classes, in every project using that model.

IMHO an addition to the model does not need to be reflected in all Classes dealing with it.
Take the idea where you can build your SynEdit with different plugins from the Component palette. You build two SynEdits that should display the same Model. One of the SynEdits is build with folding the other one is not.
- The one that is not will simply ignore the Folding Data in the model. That will be no The model will allow to register additional "sub-models" such as folding. therefore the Synedit with the folding will have no issue working with this model. It will simply create the sub-model (if not existent) and register it.

Show quoted text 5 lines
> The it would only need an appropriate serialize class to save all of them. > Or an accordingly overwritten virtual method.

I prefer an extra class. It will be more flexible

Show quoted text 8 lines
> The Model could also carry callback lists, so the anybody who needs to > know that it was loaded or saved could register a callback > Model.AddOnSerailizedCallback( AMethod ) > The beforementioned list of view(er)s. When actions shall be monitored, then an implementation in the controller seems to make more sense to me.

Sorry not exactly sure if I follow? "an implementation in the controller"? An implementation of what?

To make sure I am speaking of notifications between multiply SynEdits. Not between parts within one SynEdit.
- so the list of Callbacks must be on the model, as it is to be shared.
- the action of triggering the sending of the notifications? (happy to have it in the controller, but not seeing a problem with having them triggered by the model in some cases)
- the implementation of what happens as reaction to the notification? Yes that needs to be in the cotroller

Best Regards
Martin

Reply by email

Mattias Gaertner
to Hans-Peter Diettrich: No file should be opened more than once, so there must exist a centralized manager (file…
Show quoted text 14 lines
Mattias G?rtner schrieb: > I'm not sure how synedit should handle the unit concept. > The lfm is associated with the unit. A unit has a starting file > (e.g. unit1.pas) and an arbitrary list of include files, plus the > po file of the project/package, the fpdoc xml file and .res files. > Each of them work together and need very different code. No file should be opened more than once, so there must exist a centralized manager (file pool), keeping track of all open files and other related data collections. That manager should have a concept of units. In MVC terms the file pool is the Model, with various Views (editor, form designer, code explorer...), and everything is controlled by (an dedicated part of) the IDE.

The files itself are 'opened' by the codetools TCodeBuffer. The unit concept is first implemented in TPascalParserTool and extended till TCodeTool.
The TUnitInfo further extends the unit concept, as that not all units needs to be opened and can be part of a project.

Show quoted text 9 lines
> synedit is the 'lines' view of the sources. Most of the lazarus > code don't work with lines, but with whole files or absolute > positions. Right, but when a file is loaded, there must exist means to access its contents - no matter whether the contents in memory are organized in lines, there must exist means to map line numbers into the associated text. That mapping must be done in one dedicated place, so why not in the owner of the file itself?

TCodeBuffer has GetLine(Index: integer) and GetLineRange. But if you want to edit 'lines', then for speed reasons you should create your own TStrings copy and use the hooks (OnInsert, OnDelete, ...) like the source editor does with TSynEdit.

Mattias

Reply by email

Hans-Peter Diettrich
to Martin Friebe: That are 2 different things. Explicit Owner or ref-counting, both was about creation, and…
Show quoted text 15 lines
>> - The public/shareable model would have to be ref-counted. Or explicit >> ownership would need to be assigned and the owner would need to keep >> track of all Shareholders. >> > The latter. The model must know about all views, to be notified of > changes. This suggests a viewer chain, as used e.g. for Windows > clipboard viewers. > > That are 2 different things. Explicit Owner or ref-counting, both was about creation, and destruction of the model. You are talking about notification on changes. Each "Synedit" that uses the model becomes an observer of it.

A model is created and destroyed by some owner, e.g. the application. When the last observer (view) detaches from the model, the owner can decide whether to keep or destroy the model.

For this it can either enter itself on a list of known observers, or leave a callback reference (a reference to an object-method technically contains the reference to the object). Notifying all observers will then be easy.

I'd vote for a list of observers, but just running into one problem: of what type should the observers be?

A common base class is not very appropriate, unless only LazEdit observers are allowed. In so far a notification method reference would be more appropriate.

It can be decided, if the Model should detect changes and trigger the notifications (after EndUpdate), or if the current Actor on the Model retrieves the List and sends the Notifications.

ACK, with Actor = MVC Controller.

>> - The Model can be serialized. Special classes would to this work.

[...]

Show quoted text 6 lines
Take the idea where you can build your SynEdit with different plugins from the Component palette. You build two SynEdits that should display the same Model. One of the SynEdits is build with folding the other one is not. - The one that is not will simply ignore the Folding Data in the model. That will be no becomes problematic, when the non-folding component would try to save

the model himself. That's why *serialization* IMO must be done by the model, not by arbitrary helper classes.

- The model will allow to register additional "sub-models" such as folding. therefore the Synedit with the folding will have no issue working with this model. It will simply create the sub-model (if not existent) and register it.

It's okay when a model consists of several helper objects (sub-models, data containers), then these classes can implement the serialization of their own data - but only the model knows about all its data containers.

Show quoted text 5 lines
>> The it would only need an appropriate serialize class to save all of them. >> > Or an accordingly overwritten virtual method. > I prefer an extra class. It will be more flexible

I already noticed your preference ;-)

What about classes of a different implementation (data format...)? Who should determine, which of these classes saved some data, and consequentially has to be used for loading that data, later? That's why IMO every (sub-)model should have built-in code for it's own de/serialization.

Show quoted text 9 lines
>> The Model could also carry callback lists, so the anybody who needs to >> know that it was loaded or saved could register a callback >> Model.AddOnSerailizedCallback( AMethod ) >> > The beforementioned list of view(er)s. When actions shall be monitored, > then an implementation in the controller seems to make more sense to me. > Sorry not exactly sure if I follow? "an implementation in the controller"? An implementation of what?

The controller knows what has been changed, and can send the appropriate notifications. Also the controller knows about the views (of different kinds), the model doesn't know that. I'd restrict notifications to an single type of callback methods (observers in general), perhaps with a parameter indicating *what* has gone on.

To make sure I am speaking of notifications between multiply SynEdits.

The SynEdits (views in general) must not know about each other. The controller has to notify all views (observers) of all changes to the model.

Not between parts within one SynEdit. - so the list of Callbacks must be on the model, as it is to be shared.

ACK.

- the action of triggering the sending of the notifications? (happy to have it in the controller, but not seeing a problem with having them triggered by the model in some cases)

Well, that should do no harm.

- the implementation of what happens as reaction to the notification? Yes that needs to be in the cotroller

Oops? The *controller* sends the notifications, the *views* react on receipt of a notification. A view (or menu...) can send an action request (command) to the controller, which executes the action and notifies all views about consequential changes to the model.

DoDi

Reply by email

Martin
to Hans-Peter Diettrich

Late response / Away from home

Hans-Peter Diettrich wrote 19 lines
Martin Friebe schrieb: >>> - The Model can be serialized. Special classes would to this work. >>> [...] > Take the idea where you can build your SynEdit with different plugins > from the Component palette. You build two SynEdits that should display > the same Model. One of the SynEdits is build with folding the other one > is not. > - The one that is not will simply ignore the Folding Data in the model. > That will be no problem at all. > It becomes problematic, when the non-folding component would try to save the model himself. That's why *serialization* IMO must be done by the model, not by arbitrary helper classes.

Serialization classes are registered on the Model (e.g. By the models creator/owner). The are triggered by an actor (controller of a MVC = Synedit)

If a none folding component saves the model, it still works.

But it will be easier to switch between different serialization: binary, plain text/ini, xml,...

Show quoted text 11 lines
> - The model will allow to register additional "sub-models" such as > folding. therefore the Synedit with the folding will have no issue > working with this model. It will simply create the sub-model (if not > existent) and register it. > It's okay when a model consists of several helper objects (sub-models, data containers), then these classes can implement the serialization of their own data - but only the model knows about all its data containers.

ACK.

But each sub-model has a helper serialization class, which can be exchanged to produce a diff output format.

Show quoted text 17 lines
>>> The it would only need an appropriate serialize class to save all of them. >>> >>> >> Or an accordingly overwritten virtual method. >> >> > I prefer an extra class. It will be more flexible > I already noticed your preference ;-) What about classes of a different implementation (data format...)? Who should determine, which of these classes saved some data, and consequentially has to be used for loading that data, later? That's why IMO every (sub-)model should have built-in code for it's own de/serialization.

The owner/creator knows.

Lazarus-IDe is such an owner/creator. Lets say Lazarus saves always as plain text and a metadata xml-file. Then Lazarus always sets those serializers, and knows how to load the data.

If I write my own app, and want binary file-formats, then I use other classes for serialization.

Show quoted text 26 lines
>>> The Model could also carry callback lists, so the anybody who needs to >>> know that it was loaded or saved could register a callback >>> Model.AddOnSerailizedCallback( AMethod ) >>> >>> >> The beforementioned list of view(er)s. When actions shall be monitored, >> then an implementation in the controller seems to make more sense to me. >> >> > Sorry not exactly sure if I follow? "an implementation in the > controller"? An implementation of what? > The controller knows what has been changed, and can send the appropriate notifications. Also the controller knows about the views (of different kinds), the model doesn't know that. I'd restrict notifications to an single type of callback methods (observers in general), perhaps with a parameter indicating *what* has gone on. > To make sure I am speaking of notifications between multiply SynEdits. > The SynEdits (views in general) must not know about each other. The controller has to notify all views (observers) of all changes to the model.

ACK

Reply by email