Hi,

I cannot find any documentation about TMaskUtils.ValidateInput function (unit MaskUtils).
AFAICS Delphi does not have such a class?
(ref: http://docwiki.embarcadero.com/Libraries/XE6/en/System.MaskUtils)

There are some bugreports about this unit in Mantis which got me interested.

See the attached demo program.

It outputs:
C:\Users\Bart\LazarusProjecten\ConsoleProjecten\bugs\maskutils>mutest Mask : "###"
Value: "ABCD"
VI : " " (Shouldn't this fail?)

Mask : "999"
Value: "ABCD"
VI : " " (Shouldn't this fail?)

Mask : "000"
Value: "ABCD"
ValidateInput Failed

Mask : "LLL"
Value: "1234"
ValidateInput Failed

Mask : "lll"
Value: "1234"
VI : " " (Shouldn't this fail?)

Mask : "AAA"
Value: "[{|/"
ValidateInput Failed

Mask : "aaa"
Value: "[{|/"
VI : " " (Shouldn't this fail?)

I am the maintainer of the Lazarus MaskEdit unit.
I find the results rather confusing and inconsistent with Lazarus/Delphi MaskEdit behaviour.

Note:
When the mask contains mask-characters that mean "value must be of type XYZ, but not required" the ValidateInput accepts any input. The same characters in MaskEdit (both Lazarus and Delphi) interpret this as: "if you put anything in that place it must be of type XYZ, but you may leave this place empty (e.g. a blank)". Basically ValidateInput now treats '#', '9', 'l' and 'a' as if it were 'c' (any character allowed here). This seems wrong to me.

However, since the function lacks any documentation one can argue that current behaviour is correct.

To me the (IMO more logical) behaviour of the ValidateInput function could be described as the result of these actions in a LCL/VCL program:

- have a MaskEdit1: TMaskEdit
- do MaskEdit1.EditMask := Mask
- do MaskEdit1.Text := Value
- do MaskEdit1.ValidateEdit
- if no exception is raised then the result will be MaskEdit1.EditText, but with all "SpaceChar" repkaced by #32.

I.o.w. the function shall raise an exception it Value (after appllying Mask to it) does NOT match the Mask.

So, is it a bug?
If so, I'll open a ticket in Mantis for it (and the discussion should then move there).

Bart

I recently did some work on it; the class works completely wrong IMO, and I though to replace it completely with the functions found in the LCL.

Michael.

Michael Van Canneyt wrote

I recently did some work on it; the class works completely wrong IMO, and I though to replace it completely with the functions found in the LCL.

Yep, I was going that way too (and started coding it so). Currently FormatMaskText from maskedit (LCL) behaves better than the one from maskutils.

The LCL implementation handles UTF8, the fpc's will (also with code ported form LCL) only handle single byte encoding.

Maybe a version of maskutils unit based on UnicodeString should also be made?

Bart

Bart wrote

> I recently did some work on it; the class works completely wrong IMO, and

As a side note:
function ValidateInput : string;

The function raises an exception upon failure.
Since we did not have to follow Delphi here, a better declaration IMHO would have been:
function ValidateInput (out ValidatedString: string): Boolean;

or as a pair of functions (like the conversion routines for strings):

function ValidateInput : string;
function TryValidateInput (out ValidatedString: string): Boolean;

Bart

Bart wrote

Maybe a version of maskutils unit based on UnicodeString should also be made?

This seems so redundant. I still need to study the new FPC 3.0 string handling, but I was hoping such duplication of implementations would not be necessary with newer FPC versions. We don't see such rubbish [duplication of code with various string types] in Java or C# or Qt, so why must we put up with it in Object Pascal.

Regards,
Graeme

I think Bart meant that the maskutils unit should simply use unicodestring. Given that it almost surely will need to deal with $ and € etc, that seems like a better approach.

Michael.

Michael Van Canneyt wrote

I think Bart meant that the maskutils unit should simply use unicodestring. Given that it almost surely will need to deal with $ and € etc, that seems like a better approach.

Yep.
In UTF8 the 1 on 1 relation between the internal mask structure and the content of Value gets lost.
In LCL's MaskEdit unit this is solved by assuming all string data is UTF8. (We convert the result from the widgetset to UTF8 explicitely) The MaskEdit unit has routines (GetCodePoint/SetCodePoint) that rely on routines in LazUtf8 in order to match a position of a codepoint to a position in the internal mask.
This cannot be done in pure fpc, so a TMaskUtils based on AnsiChar can only function with single byte encoding (as it is now, so that doesn't break anything)..
(UTF8 might even work if no case conversion is used in the mask, I have not tested).

I attach an alterative _maskutils unit, which basically uses the same approach as TMaskEdit, but without the UTF8 support. Rewriting this to use WideChar and UnicodeString should be very easy.

The internal processing of the (edit)mask is the same as TMaskEdit. If that is broken somehow, it is also broken in TMaskEdit.

The only functions that may need adjustments are

* function FormatMaskText(const EditMask: string; const AValue: string): string;
* function FormatMaskInput(const EditMask: string): string;
* function MaskDoFormatText(const EditMask: string; const AValue: string;
                   ASpaceChar: Char): string;
* function TMaskUtils.TryValidateInput(out ValidatedString: String): Boolean;

But since there is no real documentation and there is no test suite, it's hard to tell if they do not function as they should or introduce regressions.

Note; I did not find any reference to use of MaskUtils in FreePascal itself, except for the uses clause in the db uit (where I did not find any reference to the above mentioned functions).

Q: Would changing the implementation to UnicodeString introduce regressions or compilation errors for existing programs using this unit?
(It'll probably give warnings about possible data loss due to implicit conversions.)

Bart

2016-04-18 13:47 GMT-03:00 Bart :

[..]

I propose to write a test suite first with the desired output, based on Delphi.

Given that the implementation becomes a detail, subject to changes in future.

I can take the duty of starting, just need to know the format. fpcunit or plain tests like most fpc tests?

Luiz

I agree 100%

fpcunit. 'most fpc tests' are for the compiler, not for unit functionality.

Michael.

Luiz Americo Pereira Camara wrote

I propose to write a test suite first with the desired output, based on Delphi.

Fantastic idea! Test Driven Development - the best way to write any software.

Regards,
Graeme

Luiz Americo Pereira Camara wrote

I propose to write a test suite first with the desired output, based on Delphi.

I agree with the concept, but:
Please note that Delphi (AFAICS) does not have a TMaskUtils class. The behaviour of TMaskEdit (and hence methods like ApplyMaskToText) has been tested quit vigorously agains the behaviour of Delphi 3. It is near 100% compatible.

So, since Delphi only publishes FormatMaskText function, this is all we can compare.
The other 2 functions we have in the interface are only used internally by Delphi if I understand correctly (from some old fpc-devel mails), so there behaviour is an imlementation detail.

The Delphi docs on FormatMaskText don't clarify much about how it must behave.

Bart

Michael Van Canneyt wrote

I think Bart meant that the maskutils unit should simply use unicodestring. Given that it almost surely will need to deal with $ and € etc, that seems like a better approach.

FWIW an unicode version for testing attached.

Bart

2016-04-18 15:26 GMT-03:00 Luiz Americo Pereira Camara <

I created a test suite that can be compiled with Delphi (dunit) and fpc (fpcunit).

Attached are 3 projects: Delphi, fpc (tests MaskUtils function), lazarus (tests LCL.EditMask function)

Compiled with Delphi 7 - all tests passes
FPC 3.0 - maskutils 7 failures 1 error
LCL trunk - 3 failures

There are some oddities in the Delphi 7 results (some of them are marked with //?? comment). I can't say if is a bug or the expected behavior.

Examples:
Res := FormatMaskText('00', 'test');
Returns 'te'. 00 is not supposed to be only numbers?

 Res := FormatMaskText('\!cc;0', 'test');
Returns 'te!st'. Should not be '!test' ?

Anyway i suggest to run the tests with a newer Delphi before settling the expected behavior

The projects are hosted at
https://bitbucket.org/blikblum/pascal-demos/src/c248883d46797e9b3760338367a0786198acbc47/delphi/FormatMaskText/?at=master

https://bitbucket.org/blikblum/pascal-demos/src/c248883d46797e9b3760338367a0786198acbc47/fpc/FormatMaskTextTests/?at=master

https://bitbucket.org/blikblum/pascal-demos/src/c248883d46797e9b3760338367a0786198acbc47/lazarus/FormatMaskTextLCLTests/?at=master

Luiz