In Delphi mode, each custom operator has to be associated with a record type, and must take or return such a record. It seems that the OBJFPC mode doesn’t have this restriction; for example, the Math unit exports a custom ** on Int64, which is a simple type.

However, I’ve been unable to define operators other than ** for non-record types, even if they do not conflict with the default interpretation. Examples include OPERATOR MOD (x, m: Double): Double and OPERATOR AND (x, y: Tristate): Tristate, where Tristate is an enumeration type.

Is this a bug, or is there any explanation for this?

Thanks.

JC Chu wrote

In Delphi mode, each custom operator has to be associated with a record type, and must take or return such a record. It seems that the OBJFPC mode doesn’t have this restriction; for example, the Math unit exports a custom ** on Int64, which is a simple type.

Global operator declarations were forbidden for mode Delphi as Delphi does not support them either (you can use them in Delphi modes though even if they were declared in non-Delphi modes). Global operators existed in FPC before Delphi introduced their operators in records concept.

Some parameter combinations are not allowed, because certain inline operators are already in place. I don't know their exact rulings, but ordinals (Integer values, floating point values), enums and sets are a bit restricted regarding overload (especially if it is a binary operator where left and right is of such a type).

It's best you simply try whether it's supported and if not you need to look for a different method... (or ask on the list why it's not allowed)

Regards,
Sven

Do you have some inside information on the extra limitations regarding ordinal, floating point, and set types?

As I understand, the only limitation on operator “overloading” should be that the operator do not have a default interpretation for the operands–result type combination.

The modulus operator, for example, is certainly not defined by default as (Double, Double) → Double; but when I try to define it, the compiler says it’s an “[i]mpossible operator overload”. The same goes for logical operators on enumeration types. (Although each enumeration element has an underlying numerical value, it requires the use of Ord() or an explicit cast to convert it into a numerical type accepted by bitwise logical operators.)

Maybe I should start preparing for a bug report?

The best source of information is the compiler source itself. The whole logic that decides whether an operator is overloadable or not is located in htypechk.pas, isoperatoracceptable and more imporantly also isbinaryoperatoroverloadable. From what I see everything is forbidden except what is written in there as an exception (and the ** operator which is always allowed).

Maybe I should start preparing for a bug report?

Maybe there were reasons to not allow them... You can of course prepare a bug report or wait till one of the other developers who are more experienced in FPC's overloading speak up.

Regards,
Sven

Anyway, here’s a patch for reviewing…

Sven Barth wrote

Maybe there were reasons to not allow them... You can of course prepare a bug report or wait till one of the other developers who are more experienced in FPC's overloading speak up.

Sven Barth wrote on Mon, 02 Jul 2012:

Maybe there were reasons to not allow them... You can of course prepare a bug report or wait till one of the other developers who are more experienced in FPC's overloading speak up.

I think most of operator overloading support was implemented by Peter, who no longer has time for FPC nowadays. I don't know whether anyone ever made a thorough analysis of what can be safely allowed and what not, and to what extent the current limitations are for implementation resp. semantic reasons.

Jonas

Apologies for the previous patch which I didn’t test carefully.

I’ve made a revised version which fixes the over-relaxation problem that led to a number of default operand type combinations to be treated as overloadable.

◦ internal_check() now handles enumeration, set, and floating-point types. ◦ Unnecessary restrictions on enumerations have been removed. ◦ Restrictions on the string-as-the-first-operand case have been relaxed.

There may be other valid cases not covered by this fix.

I’ve filed a bug report for the patch at
<http://bugs.freepascal.org/view.php?id=22359>.

JC Chu wrote

Anyway, here’s a patch for reviewing…

JC Chu wrote on Mon, 02 Jul 2012:

Going to a situation where overloading is allowed by default is not good. It's too easy to miss something (because there is no formal specification of the language anywhere), and then in subsequent releases you may have to have to break existing code because you have to disable certain overloads again.

By explicitly allowing particular operation overloads and forbidding the rest, the risk of having to break backward compatibility later on is much smaller (at least if every exception is carefully considered).

Jonas

Am 02.07.2012 16:51 schrieb "JC Chu" <jcchu @
acm.org >:

◦ Restrictions on the string-as-the-first-operand case have been relaxed.

I don't know whether you realized that already, but the isbinaryoperatoroverloadable function is called twice: once with the order given by the user and once with the order reversed as at least regarding the possibility of overloading the binary operators are commutative. So the checks for the first operand being a string indirectly also apply to the right one being a string.

Regards,
Sven

Yes. I made that error in my first patch. The new compiler wasn’t even able to compile itself…

I think someone should start making a complete chart for acceptance and rejection rules.

Jonas Maebe wrote

Going to a situation where overloading is allowed by default is not good. It's too easy to miss something (because there is no formal specification of the language anywhere), and then in subsequent releases you may have to have to break existing code because you have to disable certain overloads again.

isbinaryoperatoroverloadable() is called once. It first calls internal_check() with the default (LHS, RHS) order. internal_check() accepts or rejects the pair only if it recognizes the pair. Only if (LHS, RHS) is not recognized, will internal_check() be called again with the order reversed. If neither (LHS, RHS) nor (RHS, LHS) is recognized, isbinaryoperatoroverloadable() defaults to False. (Besides, commutativity is a semantic property which relies on the user implementation and cannot be deduced at compile time.)

Rejection rules are used to prevent conflicts with the default interpretation. The original implementation, however, rejects overloads based solely on the operand type combination, not taking into consideration the operator being used. For the string-as-the-first-operand case, this resulted in, say, * as a function on (string, PChar) being rejected, which is legal. What I did with it is restrict the operand to +, =, , <, <=, >, and >= (and, of course, removing the limits on the second operand being an enumeration).

For some operators, only one-sided protection is necessary. For example, IN must not be overloaded as a function on (<enumeration>, <set>).

For some operators, the set of accepted operand type pairs should be symmetric, and thus so should its complement, the set of rejected pairs. If acceptance and rejection rules are inconsistent, rejection becomes sensitive to the order of operands. For example, I didn’t change pointer-as-the-first-operand case to allow for (PChar, string) overloads, so * as a function on (PChar, string), despite legal, still gets rejected.

That is to say, the rules in internal_check() are not yet exhaustive.

Yes, I mean internal_check. Sorry. I hadn't the source available when I wrote you yesterday.

(Besides, commutativity is a semantic property which relies on the user implementation and cannot be deduced at compile time.)

For commutativity regarding the implementation you are right, but I was talking about commutativity regarding the possibility to overload. E.g. you should be able to overload "and" no matter whether it's argument types are "(String, LongInt)" or "(LongInt, String)".

Agreed so far. :)

So in the end "* (PChar, String)" should be allowed as well out of consistency, shouldn't it?

Nevertheless for pointers one needs to be careful, as I don't know exactly how far internal pointer arithmetic operators are available (AFAIK at least "+" and maybe "-" exist).

Regards,
Sven

JC Chu wrote

Yes. I made that error in my first patch. The new compiler wasn’t even able to compile itself…

I don't know whether you know this, but:
* testing the compiler compilation (as a first step) is best done using "make cycle" in the compiler subdirectory (optionally "make fullcycle" if you changed something were a subset of the supported platforms is affected)
* if the compiler can be compiled you should do a "make clean all TEST_FPC=path/to/your/newly/compiled/ppcXXX" in the "tests" directory and compare with a testrun of the same version without the modifications (it's sufficient to copy the "log" or "faillist" file from the "tests/output/yourcpu-youros" directory for comparison)

Regards,
Sven

Yes, it should. The bottomline is that cases treated as overloadable must not be defined by the default interpretation, because the overloaded interpretation will always take precedence. This is probably why the ld.typ = orddef case is not handled by either isbinaryoperatoroverloadable() or its unary counterpart. (But it’s really not true that ordinal types always make trouble—think of * as (Char, Integer) → string, for example.)

Speaking of unary operators… I did not change
isunaryoperatoroverloadable() to allow NOT to be defined on Double and enumeration types. I think NOT should be allowed on all types except ordinals (which include Booleans).

Our terminology seems a bit inconsistent, too, don’t you think?

PChar and PWideChar correspond to pointerdef, but behaves like strings---namely they can be manipulated and compared using operators +, =, , <, <=, >, and >= with another string-like entity (- doesn’t seem to belong there). The original implementation for the ld.typ = pointerdef case already guarantees that both the LHS and the RHS are string-like, so it remains only to add a limit on the operator so that only those mentioned above are forbidden.

The thing that worries me is that I’m not sure if the default interpretation depends on the compilation mode being used. If that is the case maybe the rules will become even more complicated.

Sven Barth wrote

I don't know whether you know this, but: * testing the compiler compilation (as a first step) is best done using "make cycle" in the compiler subdirectory (optionally "make fullcycle" if you changed something were a subset of the supported platforms is affected)

I did “make compiler” in $(source)\compiler and copied the resulting executable to my scratch directory. No wonder I didn’t find out the error. (ಠ_ಠ)

Could you elaborate on that? Do I need to run the test suite with the old compiler first?

"JC Chu" wrote

On Tue, Jul 3, 2012 at 3:55 PM, Sven Barth

> I don't know whether you know this, but: > * testing the compiler compilation (as a first step) is best done using > "make cycle" in the compiler subdirectory (optionally "make fullcycle"

if

> you changed something were a subset of the supported platforms is

affected)

I did “make compiler” in $(source)\compiler and copied the resulting executable to my scratch directory. No wonder I didn’t find out the error. (ಠ_ಠ)

Now you know a better way :)

> * if the compiler can be compiled you should do a "make clean all > TEST_FPC=path/to/your/newly/compiled/ppcXXX" in the "tests" directory

and

> compare with a testrun of the same version without the modifications

(it's

* first do a "make cycle" on a unmodified checkout, then run the testsuite.
* save the "log" and/or "faillog" file
* do your modifications (might involve some "make cycle" executions until you're satisfied with the results)
* run the testsuite again and compare the new "log" or "faillist" file against the backuped counterpart
* if there are no explainable changes in succeeding/failing tests: publish your patch
* otherwise correct your changes so that you don't introduce regressions and repeat at point 4

Regards,
Sven

Sven Barth wrote on Tue, 03 Jul 2012:

While "make cycle" is useful as a first test, for a complete testsuite run you also need the packages. So before running the testsuite, run a "make all" in the top level FPC directory (possibly preceded by an "rm build*" in case make says that nothing needs to be done for "all").

Also, if you do this regularly and have a multicore, you can speed up things significantly by using (replace "x" with the number of cores you have, or the number of cores+1 if you have a slow hard drive or are performing a cold build)
a) for the fpc make all: make all -j x FPMAKEOPT="-T x" b) for the testsuite: make full TEST_OPT="-O2" -j x

(the testsuite "full" target is equivalent to "clean all digest", but written in a way that makes all dependencies explicit so that parallel building works correctly)

Jonas

Thank you both for the tips.

I have just finished the patch and tested it. The patched version gives identical test results as the trunk.

The patch refines operator overloading acceptance and rejection rules for both unary and binary operators.

The new rules for unary operators allow NOT to be defined on enumeration and floating-point types.

The new rules for binary operators should allow for all operator overloads that do not conflict with the default interpretation (DI).* The rules were designed based on a classification of operators and operand types according to their roles in the DI, and achieved a high accuracy in the first trial.

*Note: An exception is that the operator IN cannot be overloaded with the signature (ordinal, set), where ordinal is any type covered by TOrdDef, even though not all such typs have a corresponding set type. This is unlikely to become a real obstacle, however, and can be relaxed very easily.

The patch will soon be added to the bug report at
<http://bugs.freepascal.org/view.php?id=22359>.