Reading this, I just had an idea. (Nothing that would be done anytime soon, as it would be a major project)

In The current structure
SynEdit is the Visual control, and therefore also the control that takes all events such as mouse/key down/up/move. Often it is SynEdits work to react to this, but often it also needs to call back to SourceEditor.

It is at least worth reviewing if this order could/should be changed (I am not sure about it):
- SourceEditor could be a visual component with all the Key/Mouse event handlers.
- It would *not* inherit from SynEdit, but same as now it would have a SynEdit instance that it can make calls to. This SynEdit would not paint on it's own canvas, but rather paint on the SourceEditors canvas
- Instead of SynEdit making all the callbacks to SourceEditor, now all events go to SourceEditor first, and SourceEditor can decide what to forward.

On the other hand, it is probably not worth the amount of work. Well the future will show, if there is a use case for it...

Best Regards
Martin

Sigh, if we are dreaming anyway, here is my dream:

1) Logic should be separated from the presentation, so first there should be 'TAbstractSynEdit' class, concerning itself purely with text manipulations, such as text insertion/deletion, cursor position changes, text attribute calculation,
save/loading, codetools etc.
TAbstractSynEdit should not depend on any visual code, in partucular it should descend from TObject/TPersistent, not TControl.

2) Logic should be covered by automated tests, e.g. using fpcunit framework.

3) Presentation and user interaction should be the concern of TSynEdit class, aggregating TAbstractSynEdit and delegating all actual logic to the latter.

4) SourceEditor should be removed (or converted into a trivial wrapper), TSourceNotebook should reference TSynEdit directly.

On the other hand, it is probably not worth the amount of work. Well the future will show, if there is a use case for it...

True. Proper design requires lots of effort, and (especially short-term) benefits
might not outweight the costs.

Well I am in the process of breaking it up into smaller bits.
- The gutter drawing moved to its own class.
- Folding is partly abstracted
- Trim Trailing spaces has been abstracted into a class of its own. (a view of SynEditLines)
- Caret and block have been started

Of course, they do still have heavy dependencies. But those can not easily be removed yet, other parts have to be broken free first.

2) Logic should be covered by automated tests, e.g. using fpcunit framework.

Also started (mainly folding), so they need heavy cleanup. I cam from other test frame-works and forcefully attempted to abuse fpcunit...

But as far as I am concerned, you can not write code without having automated tests.

It would make the design more "pure" then as it is now. I have too been wondering why synedit needs all the knowledge of ide events.

Marc

I've a TCharGrid component, developed in the time when the SynEdit was too instable and unreliable. It handles all cursor moves, scrolling, selecting text (also in columns), tab widths, (syntax) coloring, and also has provisions for both vertical and horizontal gutters. It was intended as a more reliable base for e.g. SynEdit, in general for representing text in a monospaced font, as is convenient for source code. It was developed and tested with Delphi 7, i.e. for Windows, but should be widely platform independent. If somebody is interested in the source code...

DoDi

My vision deviates a bit from this in the sense that TAbstractSynEdit would just be an abstraction (or dedicated implementation) of a cleaner text manipulation interface (as TStrings won't cut it), which would also implement transparent/automatic/implicit 'undo' handling. Then each command would be a class, probably inheriting from TAction.

It would make the design more "pure" then as it is now. I have too been wondering why synedit needs all the knowledge of ide events.

We should "just" fix the TAction implementation to support multiple keystrokes ;-)

Best regards,
Fl?vio

It is not a deviation as much as detalization ;-)
However, it is also important not to overdo abstractions -- perhaps a separate class for every command is too much.

Anyway, I am glad that you (and Martin) agree with the general concept.

Looking at your signature, you are the Author of
http://wiki.lazarus.freepascal.org/Redesign_of_the_SynEdit_component ?

A couple of remarks:
-Individual drawer objects fro Gutter and TextArea (I will avoid Grid in the name, Grid is a specialization)

This has been started for the Gutter. It does need a lot of clean up still. It would also benefit from the Os-Handle and canvas being moved into a wrapper class. This would:
- avoid the need to callback synedit for Invalidates
- allow a Handle/Canvas of another Component being passed to SynEdit, and SynEdit painting the other component (e.g. SourceEditor)

Similar considerations for the Textarea. However the textdrawer will have to access a lot of other objects, and need ways to merge the result. there are
- the highlighter
- the MarkUpManager
- The TextLines itself (e.g. Highlighter token will return a tab as a tab, but the TestLines need to translate this into displayable chars. Either -- spaces
-- "show special chars" >" (and if tab is only one char, then maybe there is only one ">" followe by spaces?
This can only be done by the Lines, as they know the layout/tabwidth (see concept of Views/ TabView below)

The storage and view of the TextLines:
I think TSynEditCodeBuffer is a good start for this. Yet tabs are a specialisation, that should go into a ViewClass.
ViewClasses TSynEditStrings themself, that will modify how the stored text is seen. (TrimTrailingSpaces is an example. FoldedView too, so FoldedFiew does not yet fully follow the concept)

The following Views (and others) can apply to the text

- WordWrapView
- FoldView
- TabView or ElasticTabView (http://bugs.freepascal.org/view.php?id=9650) modifies Logical(byte) to Phisical (char on screen) calculations. probably still returns tabs, but offers conversation methods.
- replacing tabs with spaces in the Strings[] property, may complicate Highlighting and MarkUp and show special chars It would also impact the ability of keeping the caret from being placed in the middle of a tab (which currently can be done)
- TrimSpaceView
- TSynEditStringBuffer (holding the actual text)

Martin

Looking at your signature, you are the Author of http://wiki.lazarus.freepascal.org/Redesign_of_the_SynEdit_component ?

Right, this was the draft for the subsequent implementation.

A couple of remarks: -Individual drawer objects fro Gutter and TextArea (I will avoid Grid in the name, Grid is a specialization)

The Grid is a hint on the organisation of the canvas, in rectangular cells. I've spent a lot of time in the various coordinate systems and their mapping, for drawing purposes (relative to the window), document view (rows/columns, scrolling, line wrapping), and document storage (folding, tab expansion, UTF encoding). The need for properly anchored bookmarks was a big challenge. Similar, but finally easy to implement, was the preservation of the cursor column, when scrolling across lines of shorter lenght.

Here we may have quite different viewpoints, on the delegation of the responsibilities.

In my solution the highlighting (including hyperlinks) is implemented in derived classes, by overriding the line-painting method. I ended up in a single array, holding the scanner start state for every line - required for proper handling of multi-line comments. Hyperlinks are implemented as special highlighting information. When the mouse pointer moves, the according characters and attributes are obtained for the current line, from the document, then the line eventually is repainted when the "active" state of a hyperlink has changed.

Right, tab expansion is highly configurable in my solution :-)

This can only be done by the Lines, as they know the layout/tabwidth (see concept of Views/ TabView below)

Right, the handling of wrapped lines also was a challenge :-)

I've left folding to the document management, for any convenient implementation. The visual component manages visible lines only, the mapping between stored and visible lines must be implemented outside of it. Including notifications of the changed line count, when text blocks are collapsed or expanded. If desired, multiple views can have different blocks collapsed, different tab width, wrapping on different screen boundaries etc. The consideration of multiple views reveals clear frontiers for the various responsibilities (what feature to implement where).

IMO all this is already perfectly implemented in my CharGrid.

With regards to the ElasticTabView, I have my own opinion on a mix of an source code editor with a text processor or page layouter - it stinks :-(

I neither like subroutine arguments indented to the "(" of the call, nor block comments to the right of source code, sensitive to insertion or deletion of lines of code. I don't want to open a can of worms for people who put more emphasis on the appearance of their(?) source code, than on its functionality. It's more annoying when the caret disappears in the last line or column of the window, as I observed in the current SynEdit implementation.

DoDi

The only reason I did skip the "grid" in the name is that a generic painter base class will not define a griod (or maybe it will but based on pixels). So it does not block anyone from implementing a proportional painter

That canvas/handle holder does not replace the (Grid)Painter. It is used by the GridPainter(s), and used by the GutterPainter(s). It may even be thyat it does not need to exist, and canvas and handle can be passed to all the Painters in Form of their LCL classes.

As It currently stands all the info about everything is hold by the Main SynEdit Class, and all other classes need to ask the central SynEdit Class. That is undesirable.
I extracted 2 classes already (but the ove isn't complete)
- TSynEditCaret:
Storing all info about the caret. It will obviously need help from other modules, to deal with tabs, double-width chars, wrapped lines (that will probably be a specialized subclass, completely replacing the original), and other things
-TSynEditSelection:
To deal with the selected block. This one is not very related to this discussion. But it has the same needs as the Caret

I will have to add a TSynEditViewPortClass:
This will at least store the Coordinates of the screen in the text (as in TopLine/LeftChar - LinesInWindow/CharWidthOfScreen), maybe a bit more. To do so, it will need access to the Painter to get information about the grid (LineHeight, SingleCharWidth)

If I understand your description correct: This special array is currently part of SynEditCodeBuffer? In any case this is information for the highlighter. The (Grid)Drawer should never access this info directly. The grid drawer will ask the highlighter (In the current Synedit there may be a need to clean up the way the Highlighter is handled...)

For Hyper links, had you have a look at the MarkUp class? (SynEditMarkupCtrlMouseLink in
components\synedit\syneditmarkupctrlmouselink.pp; there is stil some remains in central Synedit that need moving)

Again this class does not replace the GridDrawer. It is to be used as a helper class by any GridDrawer.

I have no information about the internals of you r grid drawer, and how it delegates work and responsibilities, so I can not really comment on it. I don't know how your grid drawer deals with all the different tasks/responsibilities, it has to meet (especially within the LineDrawing Class). You talk a lot of sub classes that implement the individual bits. This may be needed (and probably is at least for WordWrapping).
But I believe that the Griddrawer (and it's LineDrawer) should solve a lot by delegate to helper classes?

Of course this is easily said. And I haven't yet got the full design for how I will/would do the GridDrawer.

In which way?
I'd like to think of the final Synedit as a collection of some mandatory and some optional Classes. If you need a feature, you instantiate the apprpriate Class, either as a kind of plugin, or as a replacement/wrapper for the default class.

So as far as I am concerned the GridDrawer should ask the TabHandlingClass how to deal with tabs.
In my current design the TabHandlingClass will be a ViewClass on the LineBuffer. This allows to hand it in to caret and other point-classes that need to do byte to screen-char conversation. (Have a look at the current caret class, it has some initial code for this)

Yes, It will be.
for the TextDrawer it will probably be a another TextView on the Linebuffer (very similar to the FoldedView). Dealing with the scrollbars correctly could be "interesting".
It may also need to substitute the caret class, or the view-point class, to deal with questions like: is the caret in the visible area.

Yes, that's true, multiply views need to be considered. But if you look at the concept of FoldedView, then several SynEdits viewing the same LineBuffer can have each there own FoldedView (or tabbedView / WordWrappedView, once they is there).

That's why I like the idea of plugins. Anyone who wants to have it can have it. Anybody else should not be affected :)

As I said I don't know anything about the internal structure of you code. One consideration is, I am not going to replace the whole current solution (or a major part of it) in a single step by anything new. (This applies for your code as well as any bold single step move to synedit version 2)
Even provided that your code is matured very well by now, just the work of integrating it will create a huge amount of bugs. This is unless it comes with a full test case for SynEdit providing test for each and every feature, and providing at least 95% coverage of visited code / condition / branches.

This does not reject your code. As I said I do hardly know about it. But as far as I am concerned it would mean to integrate it step by step. It may also mean changes to it, depended were we would meet on the various goals each of us has described. If you look at the current LazSynEdit (not the one you had when you started) and you think that you can provide patches (based on your work) to improve it, then I am happy to they would require changes to the class structure I had in mind, then it will be better to discuss those changes first. I also welcome any ideas how to improve the class idea I have in mind, so if you think any particular bit of it should be done different from what I said, then feel free to point it out.
It may even be possible to modularize SynEdit enough that it can chose between different internal frameworks. Your framework would then be one of it (admitting that getting SynEdit there is a bold target).

Anywhere, since I pointed you to a lot of structures in the current SynLazCode. Is your code available anywhere to be studied. (I can't promise when I will get to this).

Best Regards
Martin

The only reason I did skip the "grid" in the name is that a generic painter base class will not define a griod (or maybe it will but based on pixels). So it does not block anyone from implementing a proportional painter

Okay, but a proportional representation will have a very different coordinate mapping, so that most of the basic functions (mouse, cursor movements) are very different. My approach addressed exactly those low level tasks, with the CharGrid already being a specialized grid class, with cells containing characters, and added font and text properties.

My primary goal was a stable base component, that can be turned into any text viewer or editor, by adding specialized document interfaces in derived classes. For a proportional representation the design will have to be turned upside down, and the base class will have to be specialized and follow the design of the related content handler(s).

That canvas/handle holder does not replace the (Grid)Painter. It is used by the GridPainter(s), and used by the GutterPainter(s). It may even be thyat it does not need to exist, and canvas and handle can be passed to all the Painters in Form of their LCL classes.

Then my CharGrid is kind of a canvas holder, which performs the mapping between document (content) and viewport (painting) space. The painters do not have to know about the organization of the canvas, they only have to paint given information within their actual clipping area (part of a display line). The content holder (source file) can be switched at any time, whereupon the CharGrid adjusts the viewport (window) to the new content extent.

As It currently stands all the info about everything is hold by the Main SynEdit Class, and all other classes need to ask the central SynEdit Class. That is undesirable.

ACK. A MVC (model-view-controller) approach migth be better. The model holds the source files, the view manages painting and user interface (mouse and keyboard), and the controller updates the document upon input or other commands, and synchronizes the related view(s) afterwards.

The caret is private to every view, the outer world only has to retrieve or modify the logical (content based) caret position. The same for the selection, with content based row/col coordinates; the view will manage the display of either a sequential or column based block hightlighting, and merge the text attributes of all "block" sources (syntax, hyperlinks, selection).

BTW, just the requirement for column-based blocks discourages the use of an proportional font. Elastic tabs may allow for blocks with consistent left/right margins, but then the text will look strange in any other editor or viewer, what's not desireable with shareable source code.

This were my CharGrid, that translates everything between document (model) and view space. The interface between document and view can be a simple record, containing the document row count. My CharGrid then can determine everything else from the text itself. I delegated that mapping to the syntax highlighter, that already has the task of parsing the text. It also will be involved in the determination of foldable blocks, so that folding (list of blocks and their state) can be implemented inside that class.

TopLine and LeftChar are not of any interest outside the view. A ScrollIntoView method will be sufficient for the outer world, with document based coordinates, perhaps with an anchor (alTop, alBottom, alCenter).

My hiliter has several purposes:
- it parses a new document for it's line count, multi-line comments, longest line (maybe for foldable blocks as well).
- with folding, it translates between document line and visible row numbers (not yet implemented).
- it parses a given line for syntactic elements, setting the text attributes in the display line buffer.

Painting a line requires multiple actions:
- the viewport row is mapped into a document line (or vice versa).
- the according text is retrieved and stored in the line buffer, with tab expansion, special character substitution and UTF translation if required, bookmarks etc.
- the syntax highlighter fills in character attributes.
- hyperlink attributes are updated, according to the mouse position.
- the visible part of the line buffer is determined, depending on the viewport margins and wrapped lines.

Then the text and gutter painters are invoked for the supplied line buffer, if their painting area is not clipped. The text painter determines sequences of same text attributes, initializes the canvas (DC) and paints the according characters.

The prepared line buffers can be kept in an cache, so that a quick refresh can be made when the mouse moves over the hyperlinks in the current view.

BTW, the top gutter can hold file tabs, or an ruler with column numbers, tab and margin markers. It must not exist at all, but when such an object is assigned to the grid, it's invoked in the appropriate situations.

Multiple edit windows deserve an application wide (non-visual) file pool. A visual notebook can contain any subset of source files, so that both cannot be combined into one component.

So much for now
DoDi

Then how to you handle double width chars? This si one of the problems I still have to address.
Even in a proportional font, some chars (Chinese, and other) have twice the width of a normal char. They will 2 positions in the grid. Actually it is the same issue as tabs.

Each char needs to have an info, how many cells it will allocate. Now if that is the case then to display proportional fonts, all I need to to make the grid size 1 pixel.
-- I do *not* encourage that. I do not say this is a good thing for a source editor. But given how close SynEdit comes to proportional display anyway, I keep at least an open mind about a structure that would allow to implement it. --

Column Blocks, as well as horizontal caret movement have to deal with this anyway, since you may allow to be in the middle of a tab. But you can not permit to have the caret in the middle of a chinese char.

The question still is how do your painters to all that. IMHO the mapping into a grid requires a lot of info (folded/ word wrapped/ tabs,...) as well as highlighting info. The painter as I see it collects all this info, but the info is provided by other objects.

The View port for example does not do the painting, it is a helper class to map the right code into the grid. It is used by the painters, but also used outside.
It allows (without accessing the painter) to check if a char or the caret is in the visible area. ( There still is the question if it will be the painter or the viewport who defines the size of each grid cell (basically the font size))

The view IMHO is more than one class, that gradually apply the mapping from a Source-Holder (TStringList) to a char-grid. The Painter then transfers each char to from the grid to the canvas.

That is what I am currently trying to do

the painter looks at a "grid-provider", a grid provider may read either the source or another grid-provider as input. I currently call those grid-provider "View".

One thing must change, currently the PaintLines code, combines the highlight info with the grid-view result. But that means mapping the highlight info.
The highlight info must be applied to the unmodified source, and then share the way through the grid-providers
(That's actually something I realized from this discussion => good)

So If I display a text that has no tabs, no double width chars, no folds, no ...., then all I need is:

-source-buffer
-highlight info (does not re-organize the layout)
-viewport-grid
-painter

The view port grid, selects the correct lines, and within each line the correct substring. So if the text is horizontaly scrolled it cuts the beginning of each line, and in any case, it cuts any line that is to long. There a 2 objects:
- The TViewPort => which defines the corner points / the rectangle
- The TViewPortTextView => which reveals the text in the ViewPort. E.g. returns 20 lines, with 80 chars in each line (like a grid)

If I have tabs, I add them
-source-buffer
-highlight info
-tab expander
-viewport-grid
-painter

DoubleWidth char view would then fill an empty-helper cell, after each chinese char (and add drawing info to the cell with the DBL char indicating this is a DBL) It also provides helper methods for the Caret Navigation.

Again I do not know the exact organization of your grid. But for me the (as an example) the tab-view/expander is not a subclass of the painter (or grid). The tab-view/expander class is a class of it's own (inheriting from an abstract TextView/GridMapper).

All the individual view/grid/mapping classes are organized in a stack. You can at anytime add/exchange mebers of the stack to archive new functionality.

See above. You always speak of your Grid in singular, as one class. For me this is a list of classes (the stack), plus the helper classes (Highlighter and Markup)

Of course the Stack I am speaking of, will depended on the situation present itself though a single interface.

The View here being a SynEdit drawing a (possible shared) Textbuffer? True TopLine should not be needed outside, but it is needed for Caret Control.
Therefore I differ between the ViewPort (defining the rectangle) and The ViewportTextView using the rectangle to provide the grid of chars which is to be displayed.
It is possible that those two indeed become just one class in the end. But from where SynEdit is now, I'll start them as 2 classes. It allows to take smaller steps, and hopefully reduce the amount of bugs I may introduce during this task.

And I plan on a stack of Textviews, that will provide, collect and merge all the above properties

Best Regards
Martin

Are these characters inside the Unicode BMP?

Actually I did a translation into WideChar, so that only characters from the Unicode BMP can be processed. In this model I'd insert a dummy character into the output string, that is not displayed but compensates for the width of the preceding double-width character.

And what about RTL text? IMO there exist limits for the task of an source code editor, where it's easier to use or port some existing text processing component, instead of reinventing the wheel.

Full Unicode requires assistance by some sophisticated library, that can deal with all the oddities of character sequences (ligatures...), and that should be provided and maintained by the platform. Windows has such a Uniscribe library, see
<http://www.catch22.net/tuts/neatpad/11>
This editor tutorial convinced me to either stay with truly monospaced characters, or let an according library do all drawing. Nothing half-baked in between. The same for the *input* of Chinese or other exceptional character sets (IME editors...).

Column Blocks, as well as horizontal caret movement have to deal with this anyway, since you may allow to be in the middle of a tab. But you can not permit to have the caret in the middle of a chinese char.

No problem, my tabs have a special display encoding, that forces the caret to move to the next ordinary character cell.

Is it really still a column block, when a double-width character hangs out on it's right boundary?

The question still is how do your painters to all that. IMHO the mapping into a grid requires a lot of info (folded/ word wrapped/ tabs,...) as well as highlighting info. The painter as I see it collects all this info, but the info is provided by other objects.

Right, I left the implementation of the highlighters and folding to dedicated objects. The classes have to implement only the very slim interface of the base class, everything else is open end.

BTW, I stored "characteristic" info in fixed size records, which can easily saved and exchanged together with the source file or the global settings. No encapsulation, but easy to use, and little chances for coding errors.

In my model nobody has to know about the painting, all required information resides in the line buffer and viewport outline. Apart from the basic client and gutter size (in pixels) and the font size, the viewport outline contains the overall grid dimension, corresponding to the line/row count of the document after block folding, the visible area is described by a scrollable offset within the grid, or (0,0) for painting in client address space, and the current extent of the viewport (client area of the control), in both fully and partially visible characters. The separation into fully (page size) and partially visible (viewport size) characters allows to e.g. scroll by (fully visible) pages in both directions, while painting and display of the caret stops only at the viewport margins - the latter (caret display) is not properly implemented in the current LazEdit!

All that has to be encapsulated in every single view(er). When the code explorer is a view, it's internals have almost nothing in common with the text viewer. A gutter also is kind of an viewer, coupled with the text viewer only by a common TopLine, but independent otherwise.

Of course the various viewers are related to a distinct document and helper objects, from which they obtain all the information to be displayed, but the kind of required information depends on their individual tasks.

The document base class has (virtual) functions for the conversion between stored (file based) and visible (possibly folded) coordinates, so that the details of folding are encapsulated. When the gutter painter/viewer requires information about blocks, it also can obtain that information from the according methods in the document base class.

Okay - with the subtle difference that in my model the painter is provided by the grid-provider with all required information, eliminating the need for any callbacks to other objects. The extent of information, required by an painter, can be determined easily, and only that information has to be passed to the painter.

Did you realize that mutliple views of the same source file can have different TopLines, viewport sizes, word-wrap settings, and much more? I really cannot see how one grid-provider can fit these different needs of multiple views. A meaningful separation IMO were:
- file provider: access in file coordinates (by characters or lines).
- block manager: access to visible (expanded) blocks, in display lines/columns.
- view: mapping of visible lines into display (row/col) coordinates.
- painter: mapping of viewport into pixel coordinates.

The latter (pixel mapping) occurs in two places, in the grid itself from mouse coordinates in grid coordinates, and in the painter from grid coordinates into canvas coordinates (different directions). The required information is stored in the viewport outline.

That's why keep both the characters and their text attributes in the line buffer, passed to the painter. I started with separate regions, describing the begin and length of tokens, selection etc., but it turned out that the handling of overlapping regions is accomplished easier by a direct mapping of the text attributes to every single character. The text attributes then can be used to e.g. determine, whether the mouse is over a (character in a) hyperlink.

I see no need for according (separate) objects. The coordinate transformations are determined by the information in the grid outline. The outline is updated when the viewport size changes, the font size changes, or when the viewport is scrolled - i.e. by user actions. It also is updated when the current file changes, either to a different file, or by insertion/removal of text, or by block folding. When the information has been updated, the display is refreshed.

Line wrapping can occur only in a fixed width viewport with no horizontal scrolling capabilities. In this case the document lines can be broken into according display lines, which are stored in the line buffer cache as distinct lines, eventually containing continuation markers (characters). The mapping between physical (display) and logical (document based) lines can be stored in the line buffer records.

Again I do not know the exact organization of your grid.

Attached the source code and documentation, if I don't forget... [Too big for attachment, sorry]

But for me the (as an example) the tab-view/expander is not a subclass of the painter (or grid). The tab-view/expander class is a class of it's own (inheriting from an abstract TextView/GridMapper).

Why should tab expansion require a separate class, extending or descending from any other class? The tab settings are global (IDE wide), and can be reflected in a commonly used data structure or tab-expander singleton.

All the individual view/grid/mapping classes are organized in a stack. You can at anytime add/exchange mebers of the stack to archive new functionality.

Sounds good, but I doubt that this is feasable. The modules are so tigthly coupled, that an implementation in distinct units will be almost impossible. This way adding new functionality will require to edit the common unit, so that it doesn't matter in which class (common or separate) the functionality is implemented.

One such case is the docking manager, where I still don't see a chance to implement an different manager separately from Controls.pp. The anchor docking sample in fact lacks the drag-dock functionality, because the implementation would require access to and modification of the existing code base, hidden in the implementation section of Controls.pp. An extraction of the hidden classes leads to circular unit references all over, protected methods are inaccessible from other classes etc.

If you want to make LazEdit that modular and extensible, please supply a unit structure that really allows for such extensions. And don't forget proper object management, when a reference is changed to a different object - the docking manager implementation will result in memory leaks or other quirks, when the automatically created manager is replaced. When different viewers (or other objects) can share other objects, interfaces instead of classes may be a better solution for the lifetime management of the exchangeable objects.

See above. You always speak of your Grid in singular, as one class. For me this is a list of classes (the stack), plus the helper classes (Highlighter and Markup)

My design is bottom up, with open end. The base class implements a default behaviour, that can be modified in a derived class, as can be seen in the TTextViewer class. The base classes and the base unit(!) never must be touched when the functionality is extended.

A user of a CharGrid class, e.g. the Lazarus IDE, doesn't have to care about eventual related classes, it only uses the interface provided in the single (maybe derived) component class. Then it's also easy to hunt bugs, introduced by the implementation of extensions. Either the bug resides in the base class, then it can be fixed there, without having to wade through uncountable extensions, or it resides in the extension itself, and has to be fixed there. The more helper classes are put into the base component, the harder the maintenance and extension of such a class.

Of course the Stack I am speaking of, will depended on the situation present itself though a single interface.

I do not really understand why you need an stack? A pool or list of exchangeable object references looks more appropriate to me.

A shared text buffer with a shared caret or scroll position does make no sense to me. It is debatable whether bookmarks or block folding should be the same in multiple views of some file, with regards to the amount and management of such block trees, but the user must be allowed to move to different places in every view, select different parts of the text, have different (hyperlink) history lists, insert/overwrite modes etc.

Thus caret control has to be private to every view. Please try to separate all your intended helper objects, with regards to their later use, as being bound to a single document, a single view, or whatever else. Also keep in mind what has to be saved and restored when the user tabs through the file list of a notebook.

Therefore I differ between the ViewPort (defining the rectangle) and The ViewportTextView using the rectangle to provide the grid of chars which is to be displayed.

We agree to disagree.

DoDi

An embedded and charset-unspecified text was scrubbed... Name: not available
Url: http://www.lazarus.freepascal.org/pipermail/lazarus/attachments/20081216/f3061c7b/attachment.pl

What double, 4 bytes or double display width?

Double width on the display could be fixed by simply doubling their indicated width, if it applies to all characters in that codepage. Monospaced doesn't mean quadratic ;-)

[...]

What means 'text attributes to every single character'? How much memory is needed for a 10mb text?

The text attributes only are stored in the line buffer of the line to be painted, nowhere else.

> [...] The anchor docking lacks d&d because my plan is: first finish the restore layout and get manual things stable. Then implement d&d. But at the moment I have other priorities than docking.

Oh, that's you, too? :-)

DoDi

Zitat von Hans-Peter Diettrich :

double display width. The font is called 'monospace' but has two different widths. chinese characters are twice the width as the other characters in the font.

ok

Yes, the world is small.

Mattias

There are several meanings by the word "view" I have tried to put some clarification into this. Having finished this mail, I thought I copy 2 fragments up here to the top. If they are not clear in itself, please read on first. And let us find a common approach to name those things....

----
Maybe for clarifications. I started this using the word "view". But I can see there are 2 ways to read this word.
- "physical view": Like a painter. The final output of the combined text/style information. Most common drawing it to a canvas. But it could be a reader too.
- "logical view": (I guess what you called the grid?). A module that takes the raw-text, and converts it into a structure suitable for the "physical view", applying style/highlight info on the way.

----
I do use the word "view" to describe the logical-view of a source-file transformed into a grid.
I do not use the word "view" to describe the painter. ( I propose "physical view")
I do not use the word "view" to describe the high level dual visibility of the same source-file in 2 windows (or a splitted window) ( I propose "user view" ?? imho a weak description, not good)

Some of the answers where given before I understood that "view" in your text sometimes refers to "multi window display". I tried to amend them in a 2nd run through my answer. I hope I didn't miss any. If my answers do not seem to match your text, I probably read view in a different way from what you meant

Yes, and there are quite a few Lazarus users waiting for a solution. In utf16 they occupy 1 code point ( they are encoded in a single 16 bit word), but need display twice the display width. Actually it seems they are called "full width", while all others (like western chars) are called half width.

Full Unicode requires assistance by some sophisticated library, that can deal with all the oddities of character sequences (ligatures...), and

True, and so far we only talk about the display, it also needs to be edited. Anyway there is much other work before I get there, so I think I defer the details on this part.

Well it is the users option (with tabs) to either have ragged column blocks, or cut tabs into spaces or ....
With Chinese (and I believe Arabic, and a few other language) there are no options (the char can not be cut).

Right that sounds similar to my plans. The folded info is stored in a FoldTree (well at the moment it is split, and some is still stored on the raw-text-lines, work in progress)
The mapping is done in FoldedView.

From all I read the difference is, that I put a viewer-class into a stack. You seem to have this in your grid-class, or a specialized inherited grid-class.
Probably both approaches have their benefits. In order to compare them we would have to deeply analyse both of them.

You are speaking of the highlighting info? Maybe easiest to give a usecase or example? Also when you say characteristic, do you mean: the details of the characteristics (e.g. numbers are blue or comments are bold), or do you mean whih characteristic apply to a char/group of chars (e.g. format the next 3 chars with the format for numbers).

Sounds identical to my idea. Only the painters (gutter and text) will know about painting, and they get provided by a text fragment fully transformed into a grid, with all char and highlight info provided.

In other words I could replace the painters by other modules, saving a bitmap, voice-reading, writing html or richtext or pdf. (though some of them would probably prefer the text before it is transformed into a grid, but maybe with some of the other transformations (folding or wrapping?) already applied...)

Please fill in a bug (or feature) report for the caret stuff. Besides, I agree the way the caret is currently implemented is horrible.

The last paragraph sound very much like what I have started.

One of the problems in this mail-thread is that we had different association for the names each other used to describe their classes. And another problem (introduced by me) is that at some point I tried to simplify thinks by referring to my structures with the names I took from your description (that was when I switched from "view" to "grid-provider") that was likely more confusing than helpful.

Maybe for clarifications. I started this using the word "view". But I can see there are 2 ways to read this word.
- "physical view": Like a painter. The final output of the combined text/style information. Most common drawing it to a canvas. But it could be a reader too.
- "logical view": (I guess what you called the grid?). A module that takes the raw-text, and converts it into a structure suitable for the "physical view", applying style/highlight info on the way.

The 2 classes go hand in hand. The desired output of the logical-view is a grid-matrix, for the kind of physical-view we currently have in mind. (It may slightly differ for a reader (text to voice)). Ideally the logical-view should be divided into a part that is independent of the physical-view, and a part that is allowed to depend.

Rereading my last paragraph. I seem to have mixed up some terminology. Yes the painter is provided by the "logical viewer" (aka grid-provider) with all information it needs.

Did you realize that mutliple views of the same source file can have different TopLines, viewport sizes, word-wrap settings, and much more?

Yes, I realized that. (And I just realize you do not talk about view as i have read this before. I have read "logical view". You write "user view, in 2 actual windows)

See my comments above. When I moved to the wording "grid-provider" I attempted to adapt to your terminology. This went wrong and didn't help making my point clear.
In my initial description I spoke of views (meaning logical views, not the painter). I also included the actual raw-text (file-provider) in the list of views. Because in my stacked organization they share some part in their interface. A logical view reads it's input from either another logical view or from the file provider (hence the file provider must be able to look like a logical view)

For some clarification (now that I got this as "user view across 2 windows). Each "user-view" will have it's complete own stack of logical views. But each stack reading the same instance of the the file-buffer

The code for handling overlapping regions has yet to be refactored, it does already handle overlapping region. But yes this needs to and will move out of the painting code

You wrote above:

Did you realize that mutliple views of the same source file can have different TopLines, viewport sizes, word-wrap settings, and much more?

Which could include tab settings....
Anyway, there also is the quest for those elastic tabs (not really a source editor feature) but it doesn't hurt if it can be provided easily.

And collecting tab-handling code in a single place, makes maintenance easier too.

Well yes there will be problems to overcome. But I still think it is possible. And I also see a benefit in it, if where changes to the common unit can not be avoided, they can be minimized.

Extending by inheritance is exactly where I see the problem.

Lets say I provided to or more extension of how word wrapping should be handled. I put each of my extension into a new subclass of your class.

Now I want to provide 2 or more extensions to tab handling (or anything else). Each of those extensions should and could extend each of the word-wrap behaviours.

How do I do that? (Let's say I had 2 word-wrappings and 3 tab handling). Do I write 3 subclasses for each of the 2 ward wraps?

There must be a misunderstanding. I never said that a caret should be shared between 2 user-views of the same text.
Of course they must not.

I meant to say caret/topline a needed within each single SynEdit (where two SynEdits can be seen a 2 user-views of the same raw-text (file) buffer).
But being visible throughout each such instance (user-view), means caret/topline are visible outside the logical-views described earlier.

I do use the word "view" to describe the logical-view of a source-file transformed into a grid.
I do not use the word "view" to describe the painter. ( I propose "physical view")
I do not use the word "view" to describe the high level dual visibility of the same source-file in 2 windows (or a splitted window) ( I propose "user view" ?? imho a weak description, not good)

Imho 2 source files seen in 2 windows, should act like to independent instances of SynEdit (no matter how they are structured internally) that share only selected information. This selected information is the TextBuffer, containing the source file. They will update according to changes made by the other instance(s).
They may if the user wishes share additional info, such as folding or the selected block. But they do not by default.

Best regards
Martin

In the model-view-controller context, the view is the logical aspect.

Let me add another clarification:

My primary viewpoint is that of an component writer, designing components for general use, i.e. not bound to a specific application or context, unless specific to the functionality of the component itself.

Consequently my CharGrid is a general component, whose use in Lazarus may deserve extensions to the general functionality. The basic component has certain capabilities, and a specific implementation. Take it both as a general model, and a specific implementation as a proof of the concept.

Now we can discuss in general, *whether* it's possible to achive the Lazarus-specific functionality, based on the given component design. This was my context in the preceding discussion. I wanted to prove that my approach is suitable for that specific use, and find out where the implementation may deserve a redesign. The result only can be go/nogo.

Then we can discuss in detail, *how* the desired functionality can be achieved (implemented). In this part I can explain how I implemented certain functionalities, and how I would use and extend the functionality in general. Others can explain their model, how to implement and extend the functionality in their design and implementation, so that we can find out about the specific advantages and disadvantages of the different approaches. We seem to have just entered this part of the discussion.

When it comes to implementation details, like painting, then I'd prefer "component" for the overall (TLazEdit or TCharGrid) component. That component can consist of, or use, dedicated sub-components, wich we should address as text and gutter painters, syntax highlighters etc.

IMO we should restrict the term "view" to any unspecific visual representation of a model (document), in the abstract MVC context. Distinct from the context of a specific component or viewer.

[...]

The user selects an column block with the mouse, as a rectangle from top/left to bottom/right. Then we have to specify how to deal with unaligned double-width characters, at the left and right margins of that rectangle, in block highlighting and copy/paste operations.

With Chinese (and I believe Arabic, and a few other language) there are no options (the char can not be cut).

There exist more such language and Unicode specific restrictions. Take the more familiar case of an "?" (A-umlaut), which in Unicode can be represented as either a single code point "?" or as two code points for umlaut and A. Both encodings are displayed as the same glyph, by a "true" Unicode painter. Now we have a problem, because the internal representation of that glyph can consist of one or two code points, so what should happen when the user deletes that "character" from the text? In an Unicode editor the first "delete" may delete the umlaut, so that the glyph is not removed on screen, but instead is converted into the remaining "A".

That's why I dislike the use of Unicode with all related hazzles. Either we need a Unicode library, that implements everything, including the painting, and implement something like an text processor - or we deal with code points and display above umlaut-A combination as two adjacent glyphs.

In my approach I assumed (willingly) that there exists a 1:1 correspondence between stored code units (WideChar) and visible glyphs, and all glyphs are displayed left-to-right. For all unhandled cases I expect a detailed specification, from somebody really familiar with the handling of those cases in a specific language, how a component should deal with that case. I would be *willing* to implement such specifications, to some degree, but I can not take the *responsibility* for the result, whether it does make sense to every end user.

That said, I'm waiting for such a specification, for the case of double-width glyphs. When most glyphs in a specific code page are double-width, an option could be specified, that for a particular document (file) *all* characters are displayed as if they were double-width. Or we could switch between single- and double-width display on the fly (context menu), and replace in single-width mode all double-width characters by single-width placeholders, and expand all single-width characters in double-width view to double width.
Additionally string literals or comments can be treated like kind of hyperlinks, so that the most appropriate representation of such text is shown in a hint window, when the user places the mouse over such text. Hint windows also could be used with other encodings, e.g. strings with embedded escape sequences, like '%uuuu' or '#$xxxx'. Or a Unicode-aware edit field could be added to the component, holding the current line (with the cursor), where the user can view and edit the line in his usual way. When he confirms a modification in that edit control, its content replaces the current line. For multi-line text also the selection could be displayed and updated accordingly, instead of the current line.

These solutions were easy to implement but, as already mentioned, I don't know what might make sense to the user.

Just for clarification: I assume kind of a DocumentSource property in the editor (CharGrid) component, that is initialized with the appropriate object reference, by the controller (in MVC terms). In the simplest case the DocumentSource is kind of a TStringList, or it can be a FoldedView, or whatever is appropriate for the file type and context.

Even if Undo functionality is added to the document source, this is nothing what the visual component should have to know or care about. In the MVC model it's the task of the controller, to submit user actions to the right place (object and method), where Undo, Folding, Insert etc. is implemented. The controller must know about all those details, but the viewer component only has to know how to obtain the text to display. It even must *not* know, whether the text can be edited by the user at all
- it only must be aware of a Change message, indicating that the text has been changed, somehow.

The controller reference can become another property of the final component, but for the first steps I left it to an OnKeyPress handler, to translate keystrokes into actions. E.g. my test application translates the cursor keys into scroll actions, and calls the according methods of the CharGrid.

From all I read the difference is, that I put a viewer-class into a stack. You seem to have this in your grid-class, or a specialized inherited grid-class.

Right, so far. I've built the test implementation around an TStrings type, holding the text, just to keep the implementation simple. In a final version of the CharGrid an interface type could be used instead, eliminating the need for derived classes. I only didn't know in advance, how the interface would look like in the end.

Probably both approaches have their benefits. In order to compare them we would have to deeply analyse both of them.

I still don't understand your stack. Exchanging the document (file) only requires to change the DocumentSource property of the control. Writing to that property will trigger all actions, required to display the new text source.

Can you please explain what you have in mind, when you introduce the term "stack"? I associate an stack with push/pop actions, what seems not to make sense to me, in the context of an text editor.

Sorry, I obviously abandoned the record thing in later implementations. Please forget what I've said about it ;-)

[...]

Well, the painters should paint, not do anything else instead. The gutter painter at least has to clear the gutter pane...

It may help to leave the "physical view" as a view (output only), and to consider the "logical view" as a model+view+controller (data + painter + edit functions). While a "typical" edit control also holds (owns) the displayed text, the SynEdit text resides in the Lazarus notebook (file pool) - that's why I think that a separation into MVC makes sense.

The 2 classes go hand in hand. The desired output of the logical-view is a grid-matrix, for the kind of physical-view we currently have in mind. (It may slightly differ for a reader (text to voice)).

It should be 3 classes, according to MVC. The view (painter...) reads from the model, which holds the information (text), and the controller translates user interaction into commands for editing, scrolling etc. User actions can come from the view (mouse and keyboard input), but also from a menu, notebook tabs etc., or from the application code (after init, before shutdown).

Ideally the logical-view should be divided into a part that is independent of the physical-view, and a part that is allowed to depend.

From the designer viewpoint, the component palette should contain multiple related components: at least a general editor component, and a (non-visual) component representing the file pool. The current integration of the file pool into the SynEdit notebook is inappropriate for multiple edit windows - the current notebook (tabs) can be integrated into the editor/viewer component, or can become a slightly specialized derivate of the common tab control.

This structure is incompatible with MVC. The model (your file provider) doesn't know about or interact with anything else. It may include a list of active views, so that changes to the data can be made known to the views. I implemented a chain (linked list) of views, with the list header in the document (file provider) object, and every view containing a link to the next view of the same document. A separate controller becomes important, as soon as multiple views can send edit commands for the same source file, and must be notified of all changes to their shared data source.

The controller receives commands from a view or other source. From "other" source means that the command applies to the view in the active window. Then the controller sends scroll commands etc. (affecting only the view) immediately back to the view, translated into logical actions (scroll a page ahead, copy selection etc.). Edit commands (insert/delete) are sent to View.DataSource, or are performed immediatly on that document, and after completion the active views of the updated document are notified of the changes. When the controller is invoked with keyboard input, then it must look for the according action in the key-map, and if no action is associated, inserts the character into the file, at the current (caret) position of the view.

Do you understand now, why a multi-window editor deserves some strict logical (re)structuring of the current SynEdit component?

For some clarification (now that I got this as "user view across 2 windows). Each "user-view" will have it's complete own stack of logical views. But each stack reading the same instance of the the file-buffer

Right :-)

But file-buffer is not the best term, because it must include shared bookmarks, foldable blocks and more. All that has to be separated in the current implementation of the SynEdit, i.e. must be removed from the (physical and logical) view, and has to be encapsulated within the MVC "model".

If you want to implement that, where should these different tab settings be stored? Just one file with different tab settings is hard to manage, when Lazarus loads a project or session.

If ever, I'd show an ruler in the top gutter, where the user can adjust his preferred settings. Together with means to save and restore the current settings. A text processor would keep such settings in the paragraph format of a document, so that we could add paragraphs (blocks) to the source files, which hold the settings for every such range of lines. Then a document object should contain a (linked) list of objects, which have to be updated whenever a line is inserted or removed from the file (block tree, bookmark list...). In so far you are right, some non-visual objects also can be seen as views, which have to be notified of changes to the document; but a separation makes more sense, because the real (painting) views can be notified only after all other objects have updated their internal state and tables.

Anyway, there also is the quest for those elastic tabs (not really a source editor feature) but it doesn't hurt if it can be provided easily.

It could be done in an exchangable tab expander (class/object), or in the CharGrid GetLineText method, which currently does the tab expansion (and more).

And collecting tab-handling code in a single place, makes maintenance easier too.

I just indicated that place in the CharGrid ;-)

No. You derive one specialized (customized) class, where you override all affected virtual methods. When you want to delegate tab expansion to a dedicated class, you add the class definition to your unit, and put all code into your customized class, to initialize and invoke methods of the added helper object/class. All further changes go into the implementation of the helper class(es), in the same or in other units.

>> The View here being a SynEdit drawing a (possible shared) Textbuffer? >> True TopLine should not be needed outside, but it is needed for Caret >> Control.

[...]

There must be a misunderstanding. I never said that a caret should be shared between 2 user-views of the same text. Of course they must not.

You are right in so far, as the caret position is required for all insert/delete operations. We only should not confuse the physical caret position (in pixels) with the logical current position (in file coordinates).

I'm undecided about the "best" definition of file coordinates, as single file offset values, or as line/column pairs. Finally the implementation of the undo buffer will have the highest weight. File offsets have to be adjusted after every insert or deletion of a single character, whereas line/column pairs only have to be updated when entire lines are inserted or removed.

DoDi

An HTML attachment was scrubbed...
URL: http://www.lazarus.freepascal.org/pipermail/lazarus/attachments/20081217/51a3159d/attachment-0001.html -------------- next part --------------
A non-text attachment was scrubbed...
Name: textdrawer.gif
Type: image/gif
Size: 3421 bytes
Desc: not available
Url : http://www.lazarus.freepascal.org/pipermail/lazarus/attachments/20081217/51a3159d/attachment-0001.gif

Right. We should start with the persistent information, stored together with the text on disk, then add all information that is common to all (multiple) Views.

Folding has to be reflected in a component, both as a structure (block tree in the gutter) and as currently visible lines (for the text painter), so that a helper object/class is appropriate for retrieving all the information in an synchronized way. Then one instance can reside in the Model, holding the shared definition of the blocks, and another instance can be part of the Views, doing individual folding - that's only a matter of how a new View is initialized.

View.Folding := Model.Folding; //object reference
if FoldingPerView then begin
   PrivateFolding := TFoldingState.Create(View.Folding);
   View.Folding := PrivateFolding;
end;

[This is what you would consider as "pushing" another item on the stack, see below]

Information like the (logical) caret position are specific to each Component. This information is not part of the model.

ACK

We could differ between a public and a private model. (Probably not needed)

Most probably not needed in this form, default is the public Model.

This should apply to both (your grid and LazSynEdit). To the user it will present itself as one whole component. It should not expose it's inner structure. In the case where it is implemented using a structure of internal classes, it will act as Facade and have a single interface.

Right, that's why I addressed the component view.

Isn't LazSynEdit such a customized version of a SynEdit?

So the below will concern itself with the View of the MVC

[...]

Right. We now should agree about what to discuss, for what purpose.

The need for refactoring should be clear by now, when multiple edit windows shall become possible. I should know more about editing lcl code, so that it is at least syntax-checked in the IDE, and in the next step how to test the modifications, perhaps with a test application, or by rebuilding the IDE.

[ discussion about double-display-width and multi-code-point chars ] [ column mode block ] Again we can open a thread with implementation details

I would like to see see folded view to be "part of" the TextDrawer (char-grid ?) component.

See above. That's only a matter of the initialization and finalization of the component.

ACK. UTF-8 encoding, I suppose? Access should be routed through the folding object.

Wordwrap should be implemented in the component, because it depends on the width of the window. Folding should be reflected in a helper object, that allows to retrieve the visible lines. In the simplest case (source not foldable at all) that object does a 1:1 mapping of the unfolded lines.

I disagree in many details :-(

The physical viewport size is controlled by the user, who also controls all scrolling and folding. This should all be done in the component. Please specify the situations, where the contents should be scrolled from outside the component. Hyperlink or bookmark jumps should be handled in one ScrollIntoView method, that eventually also moves the caret to the new position.

The component then retrieves the correct part(s) of the raw source, transforms it and paints it to the canvas

This depends on the old and new position in the text and canvas. As long as the new position is alreday visible, no redraw is the component has all information available, to determine what really has to be done.

Does this description makes sense? And we open other mail-threads for other topics?

Please do so :-)

All that is a matter of mapping the character positions, from text (visible lines) into grid cells (in the entire grid, then to the current viewport).

The component should also concern the application of highlighting or Markup information, provided it can retrieve it from the appropriate classes. (This will allow multiply Components to display the same Model , using different highlighters)

It's a matter of invoking the methods of the right object in the right moment. The object references are initialized once, their methods are invoked when required.

Full ACK.

Now I understand :-)

The different to a list of transformers is: - in a list 2 neighbours would not necessarily know each others (they could, but that would almost make it a stack) - a list controller would be needed.

I'm used to see such constucts as pipes or chains of filters, according to the data flow. Every object *has* to know about its input object, so that it can invoke its methods. Kind of an controller has to be implemented in either case, with regards to the construction and destruction(!) of the objects.

ACK.

ACK.

As you can see in the image, I divide the TextDrawer into 2 sets of classes.

I see. It's a matter of taste whether the ScrollBars are separate elements, or are part of a ScrollableWindow - in either case they determine the logical origin of the canvas. IMO also the TextDrawer *implements* the Logical View, eventually using helper objects.

In terms of MVC this needs more clarification.

You mean, when MVC is applied to the internal structure of the TextDrawer, in contrast to the application viewpoint?

I did put folding into the LogicalView (and therefore into the TextDrawer, which is the View of the MVC)

IMO it's encapsulated in the stack, used to retrieve the visible lines from the data source. The amount of currently visible lines determines the overall "grid" size (height in lines), eventually increased by the amount of continuation lines - the WordWrapper would be one level in that stack. The gutter object manages both the painting and the mapping of viewport row numbers back into file based line numbers (to be displayed).

And it's the concern of the designer of the stack objects, to provide the most appropriate interface for the TextDrawer.

This gets as outside the Display-Model. A solution could be to have a ModularSynEdit (which has SourceModel Property) and an integrated SynEdit (which has it's own Model for compatibility with existing code)

Could we agree about an CustomTextDrawer (as a general base class) and an derived/customized LazSynEdit (as a readily usable component, presented in the component palette)?

The customization would consist primarily of the construction of the stacks (pipes between Model and View), and secondarily in the reflection of eventually modified interfaces of the stack objects, i.e. in the code to access these objects.

Seems we were discussing different things. I only discussed the internals of the TextDrawer.

The internals are important in so far, as they affect the interface of the Model, so that a View can do its job.

The file provider is seen in a special way from within each TextDrawer that accesses it. That should not limit it's interaction with any amount of Controllers.

The interface of every (Model anf View) object should be separated (logically) into a data and a control part. The View accesses the Model via its data interface, the controller uses the control interface of both.

Okay, the command classification deserves some definitions. I'd distinguish between e.g.:
- Edit commands, affecting the data in the Model.
- Position commands (jumps to bookmarks...), affecting an View.
- Attribute commands, with possibly Model-specific effects (defining blocks, setting bookmarks) or View-specific effects (tab width, expanding/collapsing blocks).
- Data commands, e.g. Copy (in copy/paste actions). Perhaps as Control commands in general, e.g. including loading/saving the data in the Model, exchanging the data source of an View.

The last two categories are mostly informal, could be summarized as "other" commands.

DoDi

In some places below, I am unsure what you meant by "component":
- the "view" (as in MVC)
- the overall editor (as in SynEdit)

Serializing the Model for storage purposes (such as saving to a file or many files) for me comes after the decision how the model looks.

There may be more than one way to serialize the model. As there are many ways to use SynEdit.
- In Lazarus it is used a SourceEditor, so serializing should write the Text information into one file, and other information into another file.
- In User Applications, people may wish to serialize the model into just one file (which can not be edited by other editors)

> We could differ between a public and a private model. (Probably not needed) > Most probably not needed in this form, default is the public Model.

Actually we may need. Example folding.
Having the same model displayed in many Windows, the user may want to fold different nodes in each Window.
Or even use different highlighters, leading to different nodes being available.

The design of this must leave the choice to the user, which information is hold in the public, and which in the private model. The private Model can store the same info as the public. The private model only stores info the user whises to be different from the public model, all other info will be forwarded.

However concerning the design of the other Classes (Controller/View), the differentation between private or public model is not relevant. They access one Model. This Model knows what to do.

Ack

My implementation probably looks slightly different. There will be no dedicated properties (on the View) for dedicated features.

  View.Model := TheModel;

Then the View can internally distribute this. The Model can be a Private Model, which acts as a wrapper to the public Model.

This way it is easier to add features, without the need to change the way the view is set up.

yes and no. SynLazarus is a fork off, of an early Synedit. Not done by inheritance.

For SynEdit in Lazarus, currently UTF8. Ideally most of the code should be agnostic to the encoding, and the remainder exchangeable. The design of the Classes involved most not be based on an encoding. (except maybe the design of the model)

Which Component ? The View (as in MVC)?

In my design both Folding and WordWrap will be a Class on the "Views" internal Stack.

If the MVC is correctly implement, then the only difference between Folding or not Folding is:
- An extension to the Model: This should be generic, by adding a class to it without the need of changing the Container.
- A Class added to the Views Stack. Again no other code in the View should change.

"no other code should change" of course this is not possible, but there are ways to get very close. You can have a list of enumerated Features (like ecCommand in Synedit)

const
  ftFoo = 1;
  ftFold = 2;
....
FoldData := TFoldData( Model.RetrieveFeature( ftFold ) );
Yet, while such a generic aproach is noce, for very common features it
may be better to add direct accessor ("FoldData := Model.FoldData;")

Have you taken into account the difference between the TextDrawer, and the decorated TextDrawer? The above comment was not refering to the full "View component". It was talking about the undecorated TextDrawer.

Yes ViewPort Management is part of the View. But not of the undecorated textDrawer.

Yes, but I wasn't going into that much detail yet. In that case we have to handle external invalidation too. The OS may tell us to redraw parts of the Window

Yes. That is what I thought we talked about in the earlier mails in this thread?

For me this still is a having the right Class(es) on the Stack (the stack inside the View). Those Classes will have access of the Model (or parts of the model; as they will have been initialize accordingly). Each Class performs one step of the transformation.
Any knowledge about those tranformations outside those classes should be minimized (or ideal not exist at all)

This does allow the stack to contain any kind of transformation class. Classes can be added without changing any other code. (or with a minimum of those changes)

In implementation the difference are probably very minor. In design it distinguishes the idea behind it. For the stack the controller is reduced to a set-up and tear-down method.

In the stack every class knows about it's next higher level (so it knows it input). In that it is similar to a linked list or pipe.

A linked list (or list in general), I would understand as something that allows me access (from the outside or controller / controller only if encapsulated) to elements in the middle. With a stack there should only be access to the bottom element (maybe in some implementations to the top element / I do not plan to access the top element)

Ack, it uses the Stack, with all the classes on it.

The canvas is passed in at setup time.

No, the MVC has no idea about those internals. This line was only to indicate that some of my considerations (including most of my considerations in previous emails) where not made in the context of the "View" only. (Meaning I was exploring the internals of a MVC View and not the Model or Controller / The presence or need of I was taking for granted at that time, leaving it out of discussion)

Therefore discussing some of this in the context of the full MVC some parts need further details (see below, distinguishing between various aspects of folding is needed, if looking across the borders of the view )

Yes that was what I was trying to say before. (but it failed on the differently used terminology).
The stack is part of the logical-view.

ACK

The TextDrawer (undecorated) is the Facade (component) to the following objects:
- the stack (with all it's classes)
- the painter

So did you mean: for the stack objects to provide the most appropriate interface for the Painter?

Yes, A SynEdit should be customizable by picking individual other SynHelper components from the palette. (alternatively see below)

It may be more convenient to provide a special User Interface for this. Something like the anchor editor. In which you can select parts of your synedit

On a complete different issue, but it will become important when we go to implementation. I'd like to change many things to be more even/callback driven.

In the current implementation, if something changes, this something often knows about everyone else who needs to react to this change, and makes direct calls to all of them. That's real bad. I' d like to revert that. Individual elements know, on who's changes they need to react, and register a callback handler. If the other one changes, it just makes all the callbacks. It has no need to know who receives the callback.

Example:
Interaction between Caret and Viewport.
- If the caret moves, the viewpoer may have to move, so currently the caret calls the viewport.
Better: the Viewport just registers on the carets OnChange callback List.

Best Regards
Martin

In some places below, I am unsure what you meant by "component": - the "view" (as in MVC) - the overall editor (as in SynEdit)

Now that you mention it, I'm also unsure. I meant the implementation of the component, without thinking of the embedding/outsorcing of the controller, or whether the control should include an default controller for a default (new text file) data source.

Serializing the Model for storage purposes (such as saving to a file or many files) for me comes after the decision how the model looks.

But we can decide in advance, which properties are to be stored at all, or will have to be saved and restored when the data source (file) is exchanged.

All that IMO is configurable, using according stacks. The component code uses a given helper object, without knowing whether it's private or shared.

Or the private Model is independent from outside data, and all shared properties have to be transferred into the View, by the (application wide) Controller. The public Model cannot know what a View has to do, when settings change, so that all changes must be signaled to the Views. In an stack, we need an back-channel for propagating such configuration changes to the other end.

This may be related to what I called "characteristic" data. The View contains an set of properties, for private use, and the Model contains another set of shared properties. When the shared properties change, a single change notification is sent to each View, with the changed property set, and every View has to check for changes and react as appropriate. That's easier to implement, and doesn't leave an View in an inconsistent state, when one of more related settings change.

I remember FontChanged and other methods in SynEdit, which may have to be embedded in a wider scope, e.g. guarded by BeginChange/EndChange calls. All intermediate changes are stored, but are recognized only on the final EndChange. When all acutal changes are collected in an set of ChangeFlags, the receiver can know which settings actually have changed, and which (other) ones may deserve an according readjustment.

I already planned to open another thread, on my design for such synchronizations, in detail for synchronizing scrolls and caret moves.

However concerning the design of the other Classes (Controller/View), the differentation between private or public model is not relevant. They access one Model. This Model knows what to do.

IMO the Model has to know nothing, except that it holds data, and possibly how to load/store these data in a customized class (overridden methods).

I should have used Self instead of View, to indicate that my code resides inside the View, in the View.Model property setter. Then we agree again, I assume?

[encoding]

For SynEdit in Lazarus, currently UTF8. Ideally most of the code should be agnostic to the encoding, and the remainder exchangeable. The design of the Classes involved most not be based on an encoding. (except maybe the design of the model)

The data exchange between the classes IMO should use fixed data types. When TStrings are used, an agreement must exist whether the strings are Ansi or UTF-8 encoded. In the CharGrid I used WideString in the interface (assuming UCS-2), and left it to the Model to translate the string encoding, between file format, internal storage, and viewer/contoller interface.

The component implementing an View.

IMO it's possible, as outlined above. An agreement must exists about the base class, that represents foldable text. The interface of that class allows to retrieve lines (maybe as "raw" or "folded"), and to retrieve information about the blocks (tree) at all. Everything is implemented for shared use (by multiple Views) in the Model.FoldedText (.FoldData) object. When folding should be synchronized in all Views, every View accesses that object directly. When folding shall be different for an specific View, it stacks his private folding object on top of the Model's object, and the private object (class) implements the private collapsed/expanded state of the shared blocks. Since the private class inherits from the common class, it has the same interface, and no other code deserves any adaptation; except for the initialization of the stack, of course.

[...]

Yet, while such a generic aproach is noce, for very common features it may be better to add direct accessor ("FoldData := Model.FoldData;")

...and the FoldData setter method will do whatever is required :-)

In the destructor a FoldData := Nil; statement will destroy an eventually created private object, so that the destructor deserves no special code or knowledge about private helper objects.

Have you taken into account the difference between the TextDrawer, and the decorated TextDrawer? The above comment was not refering to the full "View component". It was talking about the undecorated TextDrawer.

I do not distinguish between an un/decorated TextDrawer - one is the CustomDrawer class, the decorated one *is* the customized class. A separation into multiple classes IMO is restricted to very basic functionality, like a mere text painter. Just a gutter painter deserves information about the scrolled line numbers, so that a base class without access to such information (stored in or provided by the component/View class) looks useless to me.

Yes ViewPort Management is part of the View. But not of the undecorated textDrawer.

In my model your undecorated TextDrawer is reduced to a single (virtual) method, for painting a single row (of characters). An overridden method will prepare the actual text and attributes (line buffer), before calling the inherited method. The gutter painter is a self-contained object, that is already incorporated into the very basic component class (TBaseGrid), and deserves no further consideration in a derived class.

Right, that's why a component should tell the OS, which parts of the viewport deserve an refresh, and wait for the resulting paint message. The OS will collect such requests and send one or more paint messages, taking into account hidden parts of the window. A component should not try to figure out and manage all such details itself (reinventing the wheel ;-)

Until now I pointed out, how my CharGrid can take into account *all* possible extensions, by design. With the only restriction to character cells of a unique size. Even the case of double-width characters can be handled, somehow. I did so in order to prove the usability of my approach, so that the procedures (abstract, not subroutines) can be applied in an redesign of the SynEdit component.

With regards to encapsulation I see no special difference, between calling a (virtual) method of the same class, or of a helper class.

This does allow the stack to contain any kind of transformation class. Classes can be added without changing any other code. (or with a minimum of those changes)

With regards to (data) consistency I see possible problems in an configurable stack, because it fixes the order of the transformations. This may make sense with predefined protocol levels, but not necessarily with tab expansion, syntax highlighting etc., as required in the preparation of the text for painting. In detail the (hypothetical) handling of double-width characters or proportional fonts affects the caret positioning, transformation of mouse coordinates, and more, so that an according helper object must be accessible directly, for specific purposes apart from painting.

In the stack every class knows about it's next higher level (so it knows it input). In that it is similar to a linked list or pipe.

This was my misunderstanding, with regards to push/pop operations in an traditional stack, where the pushed objects do know nothing about their neighbourhood.

And as mentioned above, a need may occur to access distinct elements, apart from their placement in an stack or chain for particular operation sequences.

I see a lot of stacks, not only one :-(

In the case of scrollbars, there must be reserved space in the window, their ranges and positions have to be adjusted dynamically, so that I cannot see how they would fit into any stack.

[...]

I see multiple stacks, e.g. one for the text (painter), others for line numbers (folding...), bookmarks etc. (for gutter painter and UI). I'd prefer to view these as multiple pipes, connecting View and Model, as an extension to direct access to properties or helper objects of the Model.

Interesting idea. But I wonder how a SynEdit component shall know how to make use of such a dropped object, in detail when such objects are of the same or of a new class, not yet known to the SynEdit.

You mix up the component configuration at design- and runtime?

DoDi

Let me start an thread about the internal design of such components.

Martin Friebe schrieb:
[...]

IMO it makes sense to apply MVC also to the internals of a visual component.

[...]

I deny the usability of callbacks here, and present a different (proven) model, as I implemented in the CharGrid.

Even better:

Scrolling, folding, or caret movements (hyperlink or bookmark jumps...) can affect both the caret position and the viewport origin. In order to synchronize both, a centralized dispatcher comes into mind.

Every event is translated into a primary action, by the controller. The action e.g. describes whether the viewport should follow the caret move, or the caret should follow the viewport move, or should stay where it is.

The dispatcher (or local controller) then determines all *consequential* changes, required to reflect the *intended* effect. It initializes an set of flags, indicating what has to be done, what already has been considered, and what remains to do, to reflect the new state (display refresh).

Everything starts with a BeginUpdate, preventing premature operations. When the caret shall be moved, or the viewport shall be scrolled, then according flags prevent a later re-adjustment of these values. Then all further checks and adjustments are made, like clipping the caret and viewport to the extent of the actual text, scrolling the caret into view, or moving the caret into the new viewport, and how to adjust the scrollbars, when a block has been collapsed or expanded. Also dependencies between vertical and horizontal moves are checked. The sequence of the checks and adjustments can be fixed, at least in case of the CharGrid, so that it's easy to debug and re-order the various steps, if required.

A recursion into already invoked methods does no harm, so that a vertical scroll check can invoke an horizontal scroll check and vice vera, because already performed checks simply do nothing. Each procedure only must know about related checks, to be triggered, but must not assume anything about which other checks may have to be bypassed under certain circumstances. This makes a significant difference vs. setting individual properties one by one, where the setter methods might invoke each other, possibly causing infinite loops, or producing unintended inappropriate results.

When the new state has been determined, the last EndUpdate triggers the required refresh of all components (caption, scrollbars, status bar, text and gutter areas etc.), based on the flags in the action/state flag set. In that execution state some internal properties can be re-adjusted or updated, e.g. the physical (visible) caret position can be forced to the end of a short line, while the logical caret position stays in the the same column as before a vertical move. The affected display area is determined and invalidated, with chances for optimizations, as far as the OS and widget set allow. One such optimization (Win32) were scrolling the canvas immediately, and repainting only the uncovered client area.

Such a strict separation between initial commands, state changes, and final response, centralized in kind of an local Controller, will result in an almost bullet proof implementation. All modifications to the remaining code in the component class are allowed to use only the predefined dedicated methods.

DoDi

Just read this, ....

I had the same idea, and am in progress with something like this. I needed it to abstract layers of functionality. for example trimming trailing spaces. If it should work as a transparent plugin it needs access to undo/redo.

My current approach is:

- I separated SynEdit.Lines from the internal textbuffer (so textbuffer has no longer a need to be TStrings based (it still is, because i didn't take the time to change it))

- I add a few basic methods such as
InsertText(x,y, substring); DeleteText,(x,y, len) ; BreakLine, JoinLine (maybe InsertNewLine, DeleteEntireLine) [*]
- they can deal with undo and redo
- In SynEdit all operations can be reduced to the above (and don't care about undo/redo)

Since TrailingSpaces is already a wrapper around the TextBuffer, it can simply intercept those methods, and deal with them.

----
The only flaw I still have with this is, that currently all the SynEditText* (wrappers around the actual textbuffer, doing, spaces, tabs, folding....) are combining Display and Edit functionality.

It would be nice to be able to write a TSynDisplay class first, and inherit the editor from it....

Best Regards
Martin

[*]
of course insertText could detect newlines, but it adds complexity. It will be simpler if SynEdit keeps control of this level, and InsertText, only operates on the current line

IMO an abstract base class better should be a viewer, without text manipulation commands. This would allow to sort out problems related to undo/redo for debugging.

The TStrings approach isn't really bad. The virtual Get method can access any kind of buffer, and we don't have to introduce new names for the basic methods and properties, reducing possible confusion of the users of the component.

- I add a few basic methods such as InsertText(x,y, substring); DeleteText,(x,y, len) ; BreakLine, JoinLine (maybe InsertNewLine, DeleteEntireLine) [*]

In my CharGrid I use a single basic function for insertion and deletion, with the Len indicating insertion (> 0) or deletion (< 0). This delta can be used to adjust the text size, also with undo/redo operations.

It might be necessary to distinguish clearly between display and text positions, where display positions take into account folded blocks, tab expansion etc., which only affect the display. While display coordinates may be separated into row and column, other positions (bookmarks...) better should be given as single text/file offsets. [Where IMO 32 bits are sufficient for *text* file size/positions - a dedicated data type will allow for eventual customization]

- they can deal with undo and redo - In SynEdit all operations can be reduced to the above (and don't care about undo/redo)

The undo/redo capabilities can be encapsulated in the text buffer object implementation, only the related methods have to be added to the TStrings interface.

Since TrailingSpaces is already a wrapper around the TextBuffer, it can simply intercept those methods, and deal with them.

Such a wrapper IMO disallows to change the trim 'd implement it only in the display code.

FACK

[*] of course insertText could detect newlines, but it adds complexity. It will be simpler if SynEdit keeps control of this level, and InsertText, only operates on the current line

As mentioned above, a single text offset value is sufficient for all text (buffer) operations. The text buffer must not necessarily be organized by lines, the object *only* has to map between line numbers and text positions for display references. Two such mapping functions are required, one for absolute lines in the entire text (in the text buffer object), and another one for virtual (folded) display row numbers (in the block folding object).

DoDi

Synedit has such a wrapper, and it can be configured, or switched off at run time.