Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2019-01

TSynEdit highlighter with simple spell-checker (almost working nicely)

4 messages · 2 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

pascal_tom

I want to share a simple solution to spell-checking with TSynEdit (Lazarus) which I just implemented. It is super-simple, thus maybe trivial. However, in my internet search I found that many people are seeking spell-checking for Lazarus and Delphi, so this might help some people or even trigger the development of a TSynSpellcheck component for being included to Lazarus.

It works already nice with English text (no umlauts) and almost nice in German text (only words with umlauts are not highlighted).

Background and problem formulation:
As a university teacher I am building a small editor tool which should have parallel spell checking for german and

Solution:
======
I found I nice trick to simply create a language spell-check for English and German (should work for all languages basically):

1.) I use a TSynEdit (lazarus 1.8.4) and place a TSynAnySyn with it on my main Form.

2.) I downloaded the English and German dictionary from hunspell (LibreOffice), opened and merged them in Excel (with right codepage selected to preserve German umlauts). In Excel I split the hunspell dictionary in two columns, removed the unneeded second column and sorted the words alphabetically (this speeds up the approach enormously). Then I saved it to an UTF-8 encoded codepage to a file 'dictionary.txt' (double-checked this in atom editor).

3.) At the opening of my main form I load this dictionary.txt file into a TStringList which I then set as 'constants' or 'objects' in the corresponding TSynAnySyn property that is set as a highlighter in my TSynEdit. This process takes only 4-5 seconds on my 4 year old MacBookPro, so that can be covered nicely by a splash screen at the start up of the application.

Results:
======
Actually everything works out of the box, the text entering works fast and all known words are highlighted.

Possible improvements:
=================
I think it is easy to add an 'add word to dictionary' feature as a context menu. Maybe the reading of the dictionary can be even faster if the dictionary is compiled into the app's executable as a resource and then be read from there.

Still existing problems:
================
The only two problems I encountered and maybe some of the experts here may help to improve :

- Words with german umlauts are not recognized. I think this is due to the fact that these characters can not be added to the IdentifierChars property, they are just replaced by question marks if one edits them to the string of '!$&+,-./0123456789:;?ABCDEFGHIJKLMNOPQRSTUVXYZ_abcdefghijklmnopqrstuvwxyz' that is set as a default there. I tried to add those characters also in the code of TSynAnySyn, but it seems to be restricted to deal with only the first 256 characters of the ASCII (or ANSI) table. I have to admit I do not really know all these codepage-related things.

- It is a bit strange that the highlighter highlights correctly spelled words (found in the dictionary) and not the other way around (marking not-detected words with a little wave below, as usual). I found no way to achieve this.

Any help from the community is appreciated, I would love to contribute this approach as a coding example for lazarus, as it works at least nicely for English words, but I don't know how/where I could upload the example code.

Reply by email

Mattias Gaertner
to pascal_tom: I want to share a simple solution to spell-checking with TSynEdit (Lazarus) which I just…
Show quoted text 6 lines
I want to share a simple solution to spell-checking with TSynEdit (Lazarus) which I just implemented. It is super-simple, thus maybe trivial. However, in my internet search I found that many people are seeking spell-checking for Lazarus and Delphi, so this might help some people or even trigger the development of a TSynSpellcheck component for being included to Lazarus.

That is indeed a missing feature.

Show quoted text 6 lines
[...] I found I nice trick to simply create a language spell-check for English and German (should work for all languages basically): 1.) I use a TSynEdit (lazarus 1.8.4) and place a TSynAnySyn with it on my main Form.

Note: For the IDE a solution is needed that uses source marks instead of highlighter.

Show quoted text 7 lines
2.) I downloaded the English and German dictionary from hunspell (LibreOffice), opened and merged them in Excel (with right codepage selected to preserve German umlauts). In Excel I split the hunspell dictionary in two columns, removed the unneeded second column and sorted the words alphabetically (this speeds up the approach enormously). Then I saved it to an UTF-8 encoded codepage to a file 'dictionary.txt' (double-checked this in atom editor).

Under Linux it would be nice to use ispell or aspell. I believe Mac has spelling lib as well. Maybe you can design it so the backend can be overridden?

Show quoted text 6 lines
3.) At the opening of my main form I load this dictionary.txt file into a TStringList which I then set as 'constants' or 'objects' in the corresponding TSynAnySyn property that is set as a highlighter in my TSynEdit. This process takes only 4-5 seconds on my 4 year old MacBookPro, so that can be covered nicely by a splash screen at the start up of the application.

Or better: load it in a thread. So the user can already use the application.
Btw, how big is the txt file?

Show quoted text 5 lines
[...] - Words with german umlauts are not recognized. I think this is due to the fact that these characters can not be added to the IdentifierChars property, they are just replaced by question marks if one edits them to the string of

The IdentifierChars is set of chars.
To use UTF-8 I guess adding range #196..#247 should do the trick.

Show quoted text 5 lines
[...] - It is a bit strange that the highlighter highlights correctly spelled words (found in the dictionary) and not the other way around (marking not-detected words with a little wave below, as usual). I found no way to achieve this.

Search for TSynEditMarkupMark.
Maybe Martin knows a nice example how to use them.

Mattias

Reply by email

pascal_tom
to Mattias Gaertner

Thanks Mattias for your reply, please see in the following my response.

Under Linux it would be nice to use ispell or aspell. I believe Mac has spelling lib as well. Maybe you can design it so the backend can be overridden?

It is just a text file that contains the words. I described in my first post how
I produced this from hunspell with Excel (can surely be done in Libre Office Calc as well) in a few minutes by hand. I guess an automated import from an existing Libre Office installation would be possible too, maybe it will slow
down the application startup a bit as the processing and sorting takes time. However, sorting is only necessary if you (like me) add up several dictionaries,
as the hunspell dictionaries seemed to be sorted already. For my application it is good enough to have this text file produced once by hand and packaged together with the application (later maybe as a resource
file compiled with the executable). If the user wants to add words this may be
done via a second text file for user-added words (the sorting issue then reappears, however).

Or better: load it in a thread. So the user can already use the application. Btw, how big is the txt file?

Good idea. I will look into this. For each language the files are about 4.5 MB.

Show quoted text 8 lines
>[...] > - Words with german umlauts are not recognized. I think this is due > to the fact that these characters can not be added to the > IdentifierChars property, they are just replaced by question marks if > one edits them to the string of The IdentifierChars is set of chars. To use UTF-8 I guess adding range #196..#247 should do the trick.

I looked in the source code of TSynAnySyn and I see several things like 'Identifiers: array[#0..#255] of ByteBool' and loops like 'for I := #0 to #255 do',
thus I assumed that the character ranges up to #256 are already covered. However,
german Umlauts do not work as Constants or Objects, making the spell-check trick failing with such words. What is especially bad as spell checking is especially
helpful with those characters, e.g. "ss" versus the 'ß' (not sure if you will see
the second one correctly displayed in the post, I meant a 'scharfes s' in German).

Any help to get also those words working would be appreciated, maybe you are referring to other places in the source code than those that I have identified and
tested.

I also thought maybe one has to set somewhere the right codepage first and only then
the upper half (#128-#256) of the char range is used correctly? Maybe TSynAnySyn
or TSynEdit should have a 'codepage' property and use the set codepage then?

Search for TSynEditMarkupMark. Maybe Martin knows a nice example how to use them.

Thanks for the advice. I will look for that. It would have to mark all text as 'incorrect' by
default (e.g., red waves below the text) and then override this for correctly found words
(no red waves). Is this what you mean?

Mattias

Thomas.

Reply by email