Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2020-03

modeswitch multihelpers precedence problem

15 messages · 7 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Anthony Walter

I started adding {$modeswitch multihelpers} to some of my code and noticed a problem. It would seem that the SysUtils unit is defining type helpers for the string type and this led me to discover that multiple type helpers takes precedence over other helpers in a way that is conflicts with other pascal rules.

To demonstrate, if I have MyUnit.pas that defines a string type helper with EndsWith() and an IntToStr() function then consider the following code:

program Test;
uses
  MyUnit, SysUtils;
begin
  'hello'.EndsWith('lo'); // invokes MyUnit.TStringHelper.EndsWith()
  IntToStr(12); // invokes SysUtils.IntToStr()
end.

But if I reverse the uses order:

program Test;
uses
  SysUtils, MyUnit;
begin
  'hello'.EndsWith('lo'); // invokes SysUtils.TStringHelper.EndsWith()
  IntToStr(12); // invokes MyUnit.IntToStr()
end.

What should happen is that both examples use the function from the last unit in the uses clause. Using the example above with "uses SysUtils, MyUnit;" the following should happen:

  'hello'.EndsWith('lo'); // invokes MyUnit.TStringHelper.EndsWith()
  IntToStr(12); // invokes MyUnit.IntToStr()

I know some of you might see this as a small problem, but it's an inconstancy that can lead to a situation where the wrong methods will be called without being obvious. The standard has long been that when an identifier of the same name is is found, pascal will choose to identify the name as the one coming from the last matching unit in the uses clause. Multiple type helpers should follow this same rule.

Please discuss and if we all agree I will file this as a bug in mantis. Thanks.

Michael Van Canneyt
to Anthony Walter
Anthony Walter wrote 44 lines
I started adding {$modeswitch multihelpers} to some of my code and noticed a problem. It would seem that the SysUtils unit is defining type helpers for the string type and this led me to discover that multiple type helpers takes precedence over other helpers in a way that is conflicts with other pascal rules. To demonstrate, if I have MyUnit.pas that defines a string type helper with EndsWith() and an IntToStr() function then consider the following code: program Test; uses MyUnit, SysUtils; begin 'hello'.EndsWith('lo'); // invokes MyUnit.TStringHelper.EndsWith() IntToStr(12); // invokes SysUtils.IntToStr() end. But if I reverse the uses order: program Test; uses SysUtils, MyUnit; begin 'hello'.EndsWith('lo'); // invokes SysUtils.TStringHelper.EndsWith() IntToStr(12); // invokes MyUnit.IntToStr() end. What should happen is that both examples use the function from the last unit in the uses clause. Using the example above with "uses SysUtils, MyUnit;" the following should happen: 'hello'.EndsWith('lo'); // invokes MyUnit.TStringHelper.EndsWith() IntToStr(12); // invokes MyUnit.IntToStr() I know some of you might see this as a small problem, but it's an inconstancy that can lead to a situation where the wrong methods will be called without being obvious. The standard has long been that when an identifier of the same name is is found, pascal will choose to identify the name as the one coming from the last matching unit in the uses clause. Multiple type helpers should follow this same rule.

I'm inclined to think you found the reason why Embarcadero disallows type helpers.

A type helper extends a type with some additional methods. Since the sysutils unit comes first in the uses clause, it is the first to extend the string type. Then comes the myunit helper, which tries to extend the same type with the same methods. The compiler sees the first attached method.

I agree the order of the units is relevant, but I think that it is acting more like unit initialization & class constructor order (if viewed as 'attaching methods to type') than like identifier scope (if viewed as 'looking for method identifier').

So while I agree it may be confusing, it's not necessarily wrong.

I suppose Sven can shed some light on the matter.

Michael.

Michael Van Canneyt
to Michael Van Canneyt
Michael Van Canneyt wrote 9 lines
> I know some of you might see this as a small problem, but it's an > inconstancy that can lead to a situation where the wrong methods will be > called without being obvious. The standard has long been that when an > identifier of the same name is is found, pascal will choose to identify the > name as the one coming from the last matching unit in the uses clause. > Multiple type helpers should follow this same rule. I'm inclined to think you found the reason why Embarcadero disallows type helpers.

That should be *multiple* type helpers, of course.

Michael.

Ryan Joseph
to Anthony Walter: I started adding {$modeswitch multihelpers} to some of my code and noticed a problem. It…
Show quoted text 14 lines
On Mar 10, 2020, at 11:57 AM, Anthony Walter via fpc-pascal wrote: To demonstrate, if I have MyUnit.pas that defines a string type helper with EndsWith() and an IntToStr() function then consider the following code: program Test; uses MyUnit, SysUtils; begin 'hello'.EndsWith('lo'); // invokes MyUnit.TStringHelper.EndsWith() IntToStr(12); // invokes SysUtils.IntToStr() end.

I think the problem here may be because SysUtils doesn't enable the mode switch. In that example above I would expect EndsWith to be used from the last unit, that is SysUtils. If you remake the example without SysUtils and define the type helpers in 2 units that both enable multihelpers does it work correctly?

Regards,
Ryan Joseph

Michael Van Canneyt
to Ryan Joseph
Ryan Joseph wrote 22 lines
> On Mar 10, 2020, at 11:57 AM, Anthony Walter via fpc-pascal wrote: > > To demonstrate, if I have MyUnit.pas that defines a string type helper with EndsWith() and an IntToStr() function then consider the following code: > > program Test; > > uses > MyUnit, SysUtils; > > begin > 'hello'.EndsWith('lo'); // invokes MyUnit.TStringHelper.EndsWith() > IntToStr(12); // invokes SysUtils.IntToStr() > end. > I think the problem here may be because SysUtils doesn't enable the mode switch. In that example above I would expect EndsWith to be used from the last unit, that is SysUtils. If you remake the example without SysUtils and define the type helpers in 2 units that both enable multihelpers does it work correctly?

In that case the mode switch would be wrong. As a developer of a type helper, you should not have to care about this: you just define a type helper. You should not have to say 'I will allow this type helper to be used with other type helpers'; That should always be so.

I imagine that the type helper is meant for the person wanting to use multiple type helpers: it says to the compiler 'please look for more type helpers'.

Michael.

Mattias Gaertner
to Michael Van Canneyt: I'm inclined to think you found the reason why Embarcadero disallows type helpers. A type…
[...] I'm inclined to think you found the reason why Embarcadero disallows type helpers.

That would imply E thought type helpers through. I doubt that, e.g. write access to read only properties.

Show quoted text 5 lines
A type helper extends a type with some additional methods. Since the sysutils unit comes first in the uses clause, it is the first to extend the string type. Then comes the myunit helper, which tries to extend the same type with the same methods. The compiler sees the first attached method.

In Delphi the type helper of the last unit wins. So I would expect the same for multi type helpers.

Mattias

Sven Barth
to Ryan Joseph: I think the problem here may be because SysUtils doesn't enable the mode switch. In that…

Ryan Joseph via fpc-pascal schrieb am
Di., 10. März 2020, 09:03:

Show quoted text 25 lines
> On Mar 10, 2020, at 11:57 AM, Anthony Walter via fpc-pascal < > wrote: > > To demonstrate, if I have MyUnit.pas that defines a string type helper with EndsWith() and an IntToStr() function then consider the following code: > > program Test; > > uses > MyUnit, SysUtils; > > begin > 'hello'.EndsWith('lo'); // invokes MyUnit.TStringHelper.EndsWith() > IntToStr(12); // invokes SysUtils.IntToStr() > end. > I think the problem here may be because SysUtils doesn't enable the mode switch. In that example above I would expect EndsWith to be used from the last unit, that is SysUtils. If you remake the example without SysUtils and define the type helpers in 2 units that both enable multihelpers does it work correctly?

As Michael said the unit providing the helper has nothing to do with this.

Regards,
Sven

Sven Barth
to Michael Van Canneyt: I'm inclined to think you found the reason why Embarcadero disallows type helpers. A type…

Michael Van Canneyt schrieb am Di., 10. März 2020, 08:11:

Show quoted text 70 lines
On Tue, 10 Mar 2020, Anthony Walter via fpc-pascal wrote: > I started adding {$modeswitch multihelpers} to some of my code and noticed > a problem. It would seem that the SysUtils unit is defining type helpers > for the string type and this led me to discover that multiple type helpers > takes precedence over other helpers in a way that is conflicts with other > pascal rules. > > To demonstrate, if I have MyUnit.pas that defines a string type helper > with EndsWith() and an IntToStr() function then consider the following code: > > program Test; > > uses > MyUnit, SysUtils; > > begin > 'hello'.EndsWith('lo'); // invokes MyUnit.TStringHelper.EndsWith() > IntToStr(12); // invokes SysUtils.IntToStr() > end. > > But if I reverse the uses order: > > program Test; > > uses > SysUtils, MyUnit; > > begin > 'hello'.EndsWith('lo'); // invokes SysUtils.TStringHelper.EndsWith() > IntToStr(12); // invokes MyUnit.IntToStr() > end. > > What should happen is that both examples use the function from the last > unit in the uses clause. Using the example above with "uses SysUtils, > MyUnit;" the following should happen: > > 'hello'.EndsWith('lo'); // invokes MyUnit.TStringHelper.EndsWith() > IntToStr(12); // invokes MyUnit.IntToStr() > > I know some of you might see this as a small problem, but it's an > inconstancy that can lead to a situation where the wrong methods will be > called without being obvious. The standard has long been that when an > identifier of the same name is is found, pascal will choose to identify the > name as the one coming from the last matching unit in the uses clause. > Multiple type helpers should follow this same rule. I'm inclined to think you found the reason why Embarcadero disallows type helpers. A type helper extends a type with some additional methods. Since the sysutils unit comes first in the uses clause, it is the first to extend the string type. Then comes the myunit helper, which tries to extend the same type with the same methods. The compiler sees the first attached method. I agree the order of the units is relevant, but I think that it is acting more like unit initialization & class constructor order (if viewed as 'attaching methods to type') than like identifier scope (if viewed as 'looking for method identifier'). So while I agree it may be confusing, it's not necessarily wrong. I suppose Sven can shed some light on the matter.

It should work as with normal identifiers and multiple helpers without the modeswitch (there the last one wins as well). So this is indeed a bug (though hopefully easy to solve).

@Anthony: would you report that, please?

Regards,
Sven

Anthony Walter
to Michael Van Canneyt: In that case the mode switch would be wrong. As a developer of a type helper, you should…

An issue has been submitted here
https://bugs.freepascal.org/view.php?id=36783 under the FPC compiler with tags "type helpers".

I just want to take a moment and thank you guys for your work and explain for a moment my STRONG support for multiple type helpers. Multiple type helpers have been a big savior to me, allowing me to elegantly solve a long standing design problem I've had with one of my projects. If you guys could take a little time and consider the problem I am about to describe I hope you can think about it and see why and how multiple type helpers have made a big difference to me.

Previously I have always had a disconcerting problem of how to handle assets in my game engine. An asset is an unknowable concept until such a time as when you come up with an idea for data and the code to use it when writing a game. Examples of assets are things like textures, sounds, shader programs, 3D models, and can be other things you invent, such as game levels, ai scripts, animation paths, or longs lines of dialog in text form. The point is you never know as a developer what kind of asset you might need for a particular game or project and always can invent new ones.

From a development point of view things "never know" issue presents an

issue. I do know I want a universal class to search for and manage assets, but I always often need to tie those assets to a context, for example both textures, shaders, and also 3D models, need to be aware of an OpenGL context in order generate and use assets correctly belong to a specific current context correctly.

Multiple type helpers have helper me solve this problem in an elegant manner. Any unit I write with my framework which needs to implement an asset type adds a type helper to a the TContext class. Then if I need to use an asset type in another unit I simply include that asset unit in my uses I have access to those assets managed and catalogs by the current context. Consider this snippet of code:

uses
{ Bring in texture asset extensions }
Codebot.Render.Textures

{ Later in code I can then access textures valid for that context using a type helper }
Tex := Context.Textures['smiley_face'];

Multiple type helpers allow other units to define their own assets and attach them directly to a context. Now with multiple type helpers many units can share assets belonging to a single context. In my case I've made all assets derive from a TContextManagedObject giving each asset type an owning context and a name to fit into a particular slot on on a context. Other type helpers for TContext in other units define extensions such as Context.Shaders[], Context.Models[], and Context.Fonts[]. In my opinion when presented with my asset problem, multiple type helpers is the ultimate solution to this "never know" issue.

Thanks again for those of you who took the time to read and consider my use case. I know there are a multitude of other design approaches to handling this problem, but for my work habits multiple type helpers brings in the most optimal way of handling things, and again this is in my opinion.

Anthony

stdreamer
to Mattias Gaertner: On Tue, 10 Mar 2020 00:57:12 -0400 Yes, that's how Pascal usually works and so it works…
Mattias Gaertner wrote 12 lines
On Tue, 10 Mar 2020 00:57:12 -0400 Anthony Walter via fpc-pascal wrote: > [...] > What should happen is that both examples use the function from the > last unit in the uses clause. Yes, that's how Pascal usually works and so it works in pas2js. Strange that fpc does it the other way round. Bug or feature?

No it does not. The first unit in the uses clause always take presents over the last. It always has it always should. Don't brake backwards compatibility.

Marco van de Voort
to Anthony Walter

Op 2020-03-10 om 14:46 schreef Anthony Walter via fpc-pascal:

Stdreamer, you are wrong. The last unit take precedence. Please test to verify.

Yes. He might be mixed up with the uses clause of the main program governing unit initialization.

Ryan Joseph
to Anthony Walter: An issue has been submitted here https://bugs.freepascal.org/view.php?id=36783 under the…
On Mar 10, 2020, at 8:09 PM, Anthony Walter wrote: An issue has been submitted here https://bugs.freepascal.org/view.php?id=36783 under the FPC compiler with tags "type helpers".

Can you make up an example that doesn't use SysUtils? Here's something I made up really quick but I don't see the bug.

All I did with multi helpers was continue the search which stopped at the first hit so all the normal overloading rules should apply as normal.

===================

{$mode objfpc}
{$modeswitch typehelpers}
{$modeswitch multihelpers}
program test;
uses
  umultihelpers_precedence0,
  umultihelpers_precedence1;
begin
  'hello'.EndsWith('lo');
  IntToStr(12);
end.

===================

{$mode objfpc}
{$modeswitch typehelpers}
{$modeswitch mutlihelpers}
unit umultihelpers_precedence0;
interface
type
  TStringHelper = type helper for string
    function EndsWith(str: string): boolean;
  end;
function IntToStr(int: integer): string;
implementation
function TStringHelper.EndsWith(str: string): boolean;
begin
  writeln('EndsWith - umultiplehelpers_precedence0');
end;
function IntToStr(int: integer): string;
begin
  writeln('IntToStr - umultiplehelpers_precedence0');
end;
end.

===================

{$mode objfpc}
{$modeswitch typehelpers}
{$modeswitch mutlihelpers}
unit umultihelpers_precedence1;
interface
type
  TStringHelper = type helper for string
    function EndsWith(str: string): boolean;
  end;
function IntToStr(int: integer): string;
implementation
function TStringHelper.EndsWith(str: string): boolean;
begin
  writeln('EndsWith - umultiplehelpers_precedence1');
end;
function IntToStr(int: integer): string;
begin
  writeln('IntToStr - umultiplehelpers_precedence1');
end;
end.

Regards,
Ryan Joseph