Hello,

I was porting a Delphi project and I needed support for TRegExpr. Well, it seams that FPC doesn't already come with one, but it has a Regexpr unit, but it's docs are not very positive about how complete this library is:

http://wiki.lazarus.freepascal.org/Regexpr

But I asked Sorokin if he could relicense TRegExpr from RegExpStudio in the same modifyed LGPL as the FPC RTL and FCL and he agreed!

So, what do you think about adding this unit to FPC:

http://openpokergrid.svn.sourceforge.net/viewvc/openpokergrid/Components/regexpr/RegExpr.pas?revision=202&view=markup

The only issue is that it has a name colision with an already existing unit in FPC.

Hopefully now people won't say anymore that FPC does not have a good regular expression support =)

I tested this unit in Linux and it worked fine for my project.

thanks,

I knew I recognised the name of the author. This code is already used in lazarus and can be found in components/synedit/synregexpr.pas. I had to work on it because the original code doesn't run on cpu's requiring alignment. The patch is attached at http://bugs.freepascal.org/view.php?id=19109.

Ludo

Components/regexpr/RegExpr.pas?revision=202&view=markup

The only issue is that it has a name colision with an already existing unit in FPC.

Hopefully now people won't say anymore that FPC does not have a good regular expression support =)

I tested this unit in Linux and it worked fine for my project.

thanks,

I've been using that in Windows for years and it appears stable, although I've not necessarily been tracking updates.

2011/8/2 Ludo Brands :

Still, this is good news, in Lazarus the license is MPL or GPL, now it can be modified LGPL.

Vincent

For sure!

My comments were more on the fact that the original code isn't compatible with platforms that need aligned data and that a patch is available. I also attached to the issue mentioned a test unit for SynRegExpr that is a combination of Andrey Sorokin's tests in his original package and the tests in regexpr/examples/testreg1.pp.

Ludo

If Florian agrees (if I'm correct, he wrote the old unit), we can move the old regexpr to oldregexpr, and move this one into its place.

Michael.

In our previous episode, Michael Van Canneyt said:

There are more contenders
Yesterday, on IRC, sb (Rosseaux) offered a native regex unit with PCRE constructs:

19:52 < rosseaux> https://scm.fluktuation.net/svn/brre/ a feature-complete (with mostly all known regexp features from
perl/pcre/etc.) and Unicode8.0-conform-and-UTF8-capable (as optional work mode in addition to the
ansichar-bytewise-mode) bytecode-based regular expression engine for object pascal, it has two subengines, a backtracking NFA and a parallel threaded NFA (also called lazy-computed DFA), both engines are cascaded in
each another, so that ReDO
19:52 < rosseaux> S attacks are still possible but not more so easy exploitable as like in some other regex engines. It's licensed under the LGPL with static-linking-exception. 20:00 < rosseaux> and it includes shift-or and boyer-moore (if >32 subsearchs for static simple regex patterns

20:46 < fpk> rosseaux: did you do any speed comparisations? 20:46 < rosseaux> not yet
20:47 < rosseaux> only feature comparsion tests, but i'll do it in the next days
20:49 < fpk> nice work :
20:49 < fpk> )
20:49 < rosseaux> the parallel threaded non-backtracking NFA idea is based on the http://swtch.com/~rsc/regexp/regexp1.html
article, which I've found with google months ago,
20:49 < rosseaux> thanks :)
20:52 < rosseaux> the UTF8 decoder stuff in BRRE is also a DFA machine and is position-state-hold-based until the whle regexp stuff is in the process, so the UTF8 support should be faster than in some other regexp engines with
non-complete UTF8 support just as PCRE and so on.
20:55 < rosseaux> and a already-done-compiled BRRE regex can be used in multiple CPU threads at the same time, so it's
semi-threadsafe in this sense.
20:56 < rosseaux> so that
should working now

That being said, there is probably room for two packages.

Hmmm. Yes.

But the units would have to be named differently anyhow, to avoid the mess like we had with the apache headers.

In order to avoid future nameclashes, I would propose to prefix the 'native FPC' one with 'fp'.

Which one that is, is largely irrelevant to me.
I haven't had the need for regular expressions yet :)

Michael.

Michael Van Canneyt wrote

If Florian agrees (if I'm correct, he wrote the old unit), we can move the old regexpr to oldregexpr, and move this one into its place.

So, Florian, what do you think?

thanks,

I think mine is already replaced by something written by Joost?

I talked with Benjamin about including his regexpr engine as well and to me it looks very good. Maybe we can just offer multiple ones.

I think that would be best. Maybe some speed comparisions can be done.

Michael.

@Joost, so, what do you think about renaming your unit to clear the namespace? Which name do you prefer?

I propose: joostregexpr

oldregexpr is messy because there is already a old/regexpr

I added Sorokins RegExpr to packages in rev 18891 introducing the following incompatibility:

http://wiki.lazarus.freepascal.org/User_Changes_Trunk#regexpr.pp_was_renamed_to_oldregexpr.pp

The main file from Joost Reg Expresions was RegEx.pp so probably many users won't be affected.

Place for some docs:

http://wiki.lazarus.freepascal.org/Regexpr

I added Sorokins RegExpr to packages in rev 18891 introducing the following incompatibility:

Something went wrong with the commit. The rename worked but the new regex.pp isn't committed:
http://svn.freepascal.org/cgi-bin/viewvc.cgi/trunk/packages/regexpr/src/

U packages\regexpr\fpmake.pp
D packages\regexpr\src\regexpr.pp
A packages\regexpr\src\oldregexpr.pp
U packages\regexpr\examples\testreg1.pp
Updated to revision 18892.

Ludo

Ludo Brands wrote

Something went wrong with the commit. The rename worked but the new regex.pp

There is no new regex.pp, I did not modify this file, it is still Joost's version.

Sorokin's RegExpr is all in one file: RegExpr.pas therefore I did not see a need to touch regex.pp

And where is RegExpr.pas?

http://svn.freepascal.org/cgi-bin/viewvc.cgi?view=rev&revision=18891

Ludo

ops... fixed in rev 18893

ops... fixed in rev 18893

Got the file.

Several issues though:
- a lot of the integers should be ptrint for 64 bit processors. That work has been done already in SynRegExpr.pas in the LCL.
- the file lost its UTF8 characters somewhere. In RegExpr you'll find RusRangeLo = 'àáâãäå¸æçèéêëìíîïðñòóôõö÷øùúûüýþÿ' while in SynRegExpr.pas it says RusRangeLo = 'абвгдеёжзийклмнопрстуфхцчшщъыьэюя' .
- SynRegExpr.pas also contains some code clean up and some fixes not present in RegExpr.pas.
- the code crashes with cpu's requiring alignment. Patch for SynRegExpr.pas in http://bugs.freepascal.org/view.php?id=19109

Ludo

Try rev 18900

I think I fixed all issues

That was quick :)

Ludo

Why didn't you just give the sorokin tregexpr unit another name? This way, no incompatiblities would have been introduced.

Add: because there is also Benjamin Rosseax regexpr unit. If this one is added as well, sorokins will be renamed to old2regexpr?

Florian Klämpfl wrote

Why didn't you just give the sorokin tregexpr unit another name? This way, no incompatiblities would have been introduced.

Because:

1> the old regexpr.pas had something like 20 lines of code and it's own description said it doesn't even support most POSIX, so it didn't look very useful?
2> Most of Joost's code is in regex.pp which was not changed, not in oldregexpr.pp
3> Compatibility with Delphi projects which use regexpr.pas where it means Sorokin's RegExpr (I've already found a couple of those only in projects which I develop, I don't know if it was included in Delphi, but I think it is possible because I found some projects which use it but don't have it in the sources)
4> Beginners would most likely try to use regexpr.pas which has the most simple name, they are better off trying to use the Sorokin version.
5> Benjamin Rosseax regexpr is a new invention, not something well stablished and widely used like the Sorokin unit (even Lazarus uses the Sorokin version), so I recommend naming it something like benjaminregexpr or something like that.

If we had namespaces that could be 'renamed' inside the app...

Marcos Douglas

Just googled 'Benjamin Rosseax regexpr' and don't find anything that's trelevant! Where is it please?

John

John Lee wrote

Just googled 'Benjamin Rosseax regexpr' and don't find anything that's trelevant! Where is it please?

It might help for a start to get the name right, his last name is "Rosseaux"...

http://bero.freqvibez.net/public/BESEN/BESEN.pas

hth,

Ralf