Hello

As the language is currently specified, an identifier is searched in, and used from, the last unit in the USES clause that exposes that identifier.

As some might know already, this exposes the user to future compatibility problems, stemming from the fact that the USES clause imports and "open namespace" into the user program.

Imagine my perfectly tested scientific computations program uses two third-party units: Math and Trig. Currently only the Math unit exposes a log() function that computes the base e (that Euler number) logarithm of the given argument. All is so good and nice and I get promoted or I get to another project or to another company and leave the code on the repository.

An year later a new version of Trig unit is available, the company promptly upgrades it and after some time someone else needs to make some changes to, or just re-compile for porting or other resons, the initial scietific computations program.

Now Trig also exposes a new log() function, which computes the base 10 logarithm for its given argument. Now Trig appears last in the USES list, after Math.

With no error messages, or even with no changes to the program since 1 and a half year in the repository, the scientific calculations are now all blown up, and program outputs only maintainer now curses and chases me for having the nerve to leave a program known as working in such a horrible mess in the repository.

Of course this is a particular case, and in the general case it is most likely I will get some compilation error, if a unit that appears later in the USES clause suddenly now defines a previous symbol, that I make use of in my program. But still, if the program never changes (on git/svn/cvs), it should still compile later, as it stands...

This story is inspired from a real case when *lots* of user code suddenly stopped compiling after JDK 1.2 was released, with a new List class, which users had to import from a different package until JDK 1.1. In this case is wasn't even a third-party library that triggered conflicts in existing code, it was the system run-time library. Since that story most Java programmers prefer to import and enumerate each and every class they use explicitly, instead of importing the package that provides the classes (I believe there is no equivalent feature in Pascal that imports only one symbol from a unit).

Is there any form of proposal or some alternative to the USES clause, that will keep all the imported symbol names qualified by some namespace name or by some prefix I choose ?

Something like:

	import System into sys, Math, Trigonometric into trig;

and then I would need to use trig.actg() as the only possible way to refer to actg() function in the Trigonometric unit, and to use Math.log() for logarithm ?

Yes, I know I can qualify names even now if I so choose, but I was thinking for a way to ensure that qualified names are the only possibility, and a way that will allow me some shorter qualification, because with the current qualified names, the qualified name can be really long.

Thank you,
Timothy Madden

Importing a single class in Java still requires you to specify the full package name, so how would that help preventing compilation failures in case a class is moved from one package to another?

(I believe there is no equivalent feature in Pascal that imports only one symbol from a unit).

Indeed.

Is there any form of proposal or some alternative to the USES clause, that will keep all the imported symbol names qualified by some namespace name or by some prefix I choose ?

Not that I know of.

Jonas

Very tedious, but you could create a wrapper unit and/or class for each library, and expose a prefixed name instead of the original.

Jeff,

My problem is about a new symbol in a new version of unit X, overriding an already-used symbol from unit Y, as soon as I upgrade to the new X and re-compile.

If a symbol is removed from unit X, as you say, that is obviously a problem (if not a mistake) with unit X (supposing the symbol was not deprecated already).

If a symbol is added to unit X, on the other hand, that is a legitimate (and expected) case, and here you might turn to the language itself for some help to ensure your future.

By the way, as it turns out C++ has a similar problem with its "using namespace std;" directive, but it stands a little better in that the directive is explicitly required before it can have any effect, and user can also rename the namespace instead, to some meaningfull prefix, to make it easier to prefix all symbols when used throughout the code, if they so wish.

Thank you,
Timothy Madden

DaWorm wrote

Very tedious, but you could create a wrapper unit and/or class for each library, and expose a prefixed name instead of the original.

Very ingeniuos, I believe this is as close to a solution as I can get for now.

But there are still a few probles I can see:
- there is no way to "prefix" symbols within a unit, that I know of. Symbols may only be prefixed by the unit name directly. Is there a sort of a "namespace" in Pascal ?
- if I make a wrapper unit around the third-party one, there is alredy little need for a prefix, since I can expose only the symbols I know of and that I use.
- I find it even more tedious to do the same thing with the System unit or the standard Pascal units.

And then, if wrapper unit publicly uses the third-party unit, doesn't that mean that all symbols in the thrid-party unit are now automatically exposed and public in the wrapper unit ?

If so, is there a way to expose and make public the types, classes, variables and functions from a private (implementation) unit ?

Thank you,
Timothy Madden

I am trying hard to think of a situation where unit prefixes are not enough, but its hard to see.

2012/8/21 Timothy Madden :

Nope, you can "uses" in the implementation section AND if a unit uses a third unit, the third unit identifiers are not visible in the program..

Unit a:
Interface
Uses b;
Implementation
End.
Unit b;
Interface
Type z=integer;
implementation
end.
program c;
uses a;
var
  h: z;
begin
end.

Type Z is not visible in program c, generating an error.

Types, consts, variables, classes, functions, procedures etc, are public if they are defined in the interface section, and private if they are defined in the implementation section.

A unit/program imports the interface definitions of another unit if it uses then in their own interface section.

"With no error messages, or even with no changes to the program since 1 and a half year in the repository, the scientific calculations are now all blown up, and program outputs only maintainer now curses and chases me for having the nerve to leave a program known as working in such a horrible mess in the repository."

Never tested, but doesnt the compiler warn of the colision ? if not, it should.

Usually i prefix my symbols with some meaningfull prefixes to avoid collision. I never have two symbols with the same name and i hardly used unit prefixes in the past.

But a collision warning from the compiler would be helpfull.

In our previous episode, Timothy Madden said:

Some languages related to pascal and maybe even other pascal dialects have ways to import an unit/module qualified. FPC doesn't.

E.g. under modula2

IMPORT IO;

will import everything from IO qualified.

Unqualified importing is possible, but only for specific symbols:

FROM IO IMPORT WrStr;

allows to use WrStr without IO.

functions to be exported unqualified when imported.

E.g.

an

EXPORT UNQUALIFIED WrStr;

in module IO would cause the first statement to add WrStr both as "WrStr" and IO.WrStr to the scope.

Note that FPC has other ways to disambiguate thoug. Since scope is stack based in Wirthian languages, order of units in the uses clause matters for which symbol is visible when similar named symbols clash.

Symbols may only be prefixed by the unit name directly. Is there a sort of a "namespace" in Pascal ?

An unit/module is a concept that includes a namespace. (since you can qualify identifiers with unitname)

- I find it even more tedious to do the same thing with the System unit or the standard Pascal units.

Why do you feel the need for a prefix? The rare clashes can be easily dealt with by uses list ordering and qualifying with unitname.

The only other thing I can imagine is that you might want to confine importing units with gigantic numbers of symbols (like Windows) into general units/modules, but confine that more OS specific and encapsulation units.

This because such units of course increase the chance of clashes significantly, and while that can be remedied as above, in this case it is better to avoid it.

And then, if wrapper unit publicly uses the third-party unit, doesn't that mean that all symbols in the thrid-party unit are now automatically exposed and public in the wrapper unit ?

Not standard no. If you "uses" an unit you only import the symbols from that unit itself, not what that unit imports.

Units provide a namespace, so UnitA.Foo is not the same as UnitB.Foo. Thus it is perfectly allowed and valid and sometimes also common to have a collision here. [also the compiler stops looking for further symbols if it has found the first one => one of the reasons for the high compilation speed]

Regards,
Sven

I already know that you can fully qualify, i believe you misread my post.

Its not that common to have name colisions if you use prefixes like i said.

I cant see how the added complexity of full name space support would help pascal in that regard.

2012/8/21 Sven Barth :

I wrote about why the compiler does not and should not warn because of a collision in naming of symbols in different units.

Regards,
Sven

I feel your pain.
A long time ago this topic was talked here, on the list, and many ideas were proposed... but none was accepted or because no one will implement it.
I proposed this sintaxe:
uses my_long_unit_name as my;
begin

  my.proc();

end

Simple, clean and easy. But I do not know how implement this on the compiler.

IMHO, without changing the language, only the compiler could use "simple names". All others third-party library -- even Lazarus -- should use prefixes in ALL identifiers in the unit, unfortunately.

Marcos Douglas

Marcos Douglas wrote

On Mon, Aug 20, 2012 at 1:15 PM, Timothy Madden wrote:

[...]

This is what I am also looking for, I think it is the only real solution.

Also, with this syntax, qualifing the symbol with "my." is the only way
to access the proc() symbol there, meaning that
begin
   proc();
end
will never work.

Sorry to hear no one will implement it, you would not say it is difficult to do at a first glance...

Thank you,
Timothy Madden

The problem is the language exposes my program to the risk of future collisions from the units I use.

A warning is a little beyond the scope of this problem, because my program is working perfectly as of now.

But next year, some unit I use will have reached a new version, and if I merely recompile my program I have already have a conflict, that was not there last year.

And the new version for the unit in question is fully compatible with the old one, so you would say my code should still compile.

The question here is if it would be possible for the language to prevent this problem, like python (and others) do.

Otherwise, I could never upgrade a library, even if the vendor announces full compatibility with the previous version. Because I will still expose my old code to the risk of new collisions.

Even worse, as the language currently stands, it is possible to upgrade a unit to a compatible version and get a different behaviour from my program. Without a single warning from the compiler.

Thank you,
Timothy Madden

"Never work", hum... ,maybe a compiler directive to ON/OFF this feature would be better.

Sorry to hear no one will implement it, you would not say it is difficult to do at a first glance...

I think the FPC core did not like or does not care. If has no one to implement it is a problem too. =(

Marcos Douglas

Yes.

Yes.

Obligatorily, not.
You can use a pratice to use unit.func always or, as I do, use a prefix in all identifiers:
unit: foo_unitname
proc/func: fooProc, fooFunc -- maybe use class procedure/function instead. classes: TfooClass
global var: DO NOT USE!
global consts: FOO_CONST_NAME -- maybe use class const instead.

Yes, Yes, even people saying otherwise.

Marcos Douglas

Timothy Madden wrote

But next year, some unit I use will have reached a new version, and if I merely recompile my program I have already have a conflict, that was not there last year.

I'd suggest that the first thing to do is to document in each unit what version of the compiler it's tested with. It's only marginally more difficult to add a warning (or even error) if the compiler version isn't as expected.

In our previous episode, Marcos Douglas said:

This doesn't protect any better, since
the new unit might also define "my".

True, but using this sintaxe I can use an alias to the third-party units so, I can use my own names to reference identifiers that I can not change.
The collision still can exists? Yes, but in that case the programmer would be wrong, not third-party unit names or because the compiler not helped.

The third-party could use a better and bigger name like "XyzNetSocket" but I could use just "net" (uses XyzNetSocket as net), for example.

IMHO this is more sophisticated than pure namespace.

Marcos Douglas

Instead of implementing the half-assed C++ namespace model, just add to the compiler a warning when it detects that there is a collision in the current scope (two functions with the same parameters from different units that can be called from the scope being inspected). I believe function overload alread provides the necessary hooks, but i cant do it because i have zero experience with FPC sources...

2012/8/21 Marcos Douglas :

Well, a good idea but this is another thing...
This is do not resolves the problem if I have a third-party unit with the same name that other.

Marcos Douglas

...continue:
If I can use an alias to ref an unit, the names could be bigger because I can ref them only using the alias.

Extend the concept:
if nevertheless we have same names in two libs (eg: foounit), we could use: uses
foounit as f1 in '/path_1/foounit.pas',

  foounit as f2 in '/path_2/foounit.pas';

IMHO, would be PERFECT.

Marcos Douglas

the alias is the fully qualified function name :

myunit1.function1()
myunit2.function2()

you dont need alias, cause you cant have two units with the same name, so you have two different fully qualified function names.

The warning would simply point out that there are two functions with the same name and parameters in the current scope, because thats the only possible conflict.

Requiring all functions to be allways fully qualified will make the most commom case worse than current implementation just to be better for the uncommon case. This is a step back, not forward...

2012/8/21 Marcos Douglas :

Requiring all functions to be allways fully qualified will make the

most commom case worse than current implementation just to be better for the uncommon case. This is a step back, not forward...

Indeed, and the way unit system works has an advantage: changing entry order / entry name in the uses clause can change the program behavior as intended without changing other parts. Say, I have GraphicsDirectX and GraphicsOpenGL unit, both having the same interface, simply changing this in the uses clause could do a backend switch easily.

Jorge Aldo G. de F. Junior wrote

the alias is the fully qualified function name : myunit1.function1() myunit2.function2()

Yes, we know..

you dont need alias, cause you cant have two units with the same name, so you have two different fully qualified function names.

Yes, we know too... but how prevent this using third-party libs? For example, what do you think if I have a project that have a file called lnet.pas? Can I use this identifier? I think so... WAIT, I can't because the lnet project already uses, right? But I didn't know before...ohh my... and now, I can't use this project because I will got a compiler error because "cant have two units with the same name". Do you think this is right?
Because of this conflict that I proposed the syntax: uses
foounit as f1 in '/path_1/foounit.pas',

   foounit as f2 in '/path_2/foounit.pas';

But this sintaxe is just to resolve a conflict, isn't default configuration, of course.

The warning would simply point out that there are two functions with the same name and parameters in the current scope, because thats the only possible conflict.

Yeah, I understood and I think this is a great idea (too).

Requiring all functions to be allways fully qualified will make the most commom case worse than current implementation just to be better for the uncommon case. This is a step back, not forward...

Timothy Madden want this, not me. ;-)

Marcos Douglas

Yes, this is cool but you can got problems, as Timothy Madden said in this thread.
I am not proposing break this feature -- because I use it too! -- but add one more.

Marcos Douglas

"Yes, we know too... but how prevent this using third-party libs? For example, what do you think if I have a project that have a file called lnet.pas? Can I use this identifier? I think so... WAIT, I can't because the lnet project already uses, right? But I didn't know before...ohh my... and now, I can't use this project because I will got a compiler error because "cant have two units with the same name". Do you think this is right?
Because of this conflict that I proposed the syntax:"

How many times this actually did happen ?

I never seen that before...

Depends how many third-party libs do you use in a single project. Ah, I'm not talking about just open and famous libs, but any one! For example: In a company we can have many departments; each department can have your own lib; each department can choose the names of your own identifiers and here is the problem.
I had many problems using Object Pascal (Delphi) a long time ago.

I never seen that before...

Do you use many third-party libs? ;-)

Marcos Douglas

Edit myunit.pas

Unit myunit.pas; <- backspace backspace backspace
Unit mynewunit.pas;

Alt-S Alt-X

Ren myunit.pas mynewunit.pas

No collisions now !

2012/8/21 Marcos Douglas :

Are you kidding me?
Third-party means: you can't change (can't, not want or not have access).

Marcos Douglas

In our previous episode, Marcos Douglas said:

Yes, we know too... but how prevent this using third-party libs? For example, what do you think if I have a project that have a file called lnet.pas? Can I use this identifier?

As much as any identifier. If you uses lnet; in the same unit where you import an unit with lnet symbol, you have to qualify it fully.

That only fixed it for that compilation unit,
if another unit just "uses foounit" that leads to an inconsistent state.

I have experience with it, and it is a nice feature if it is not mandatory. So you don't _have_ to qualify.

If you know that a certain unit contains a lot of fairly common symbol names you import it qualified. (like many large API units, typically you only uses an handfull symbols from e.g. Windows, and it is not mandatory for the other situations).

Modula2 went a step to far to only allow unqualified use if you name the symbols. That is too laboursome. Simply unqualified (what we have now) and qualified would go a long way.

... and all its effects and sideeffects are known.

The compiler can not detect such collisions, because it stops searching for a symbol if it found a suitable one. This also explains why you need the "overload" directives in certain cases of cross unit overloading (because then the compiler will search for more). This is one of the reasons why the compiler can compile code so fast: It does not need to check all loaded units of a unit (and there are units that contain many uses).

So from me this gets a definit -1 in big letters.

Regards,
Sven

Marco van de Voort wrote

In our previous episode, Marcos Douglas said:

[...]

This is necessary in order to guarantee thare can be no conflicts whatsoever. No matter how many units you want to USE in your program, no matter what versions of the units.

It is not that bad, python uses that from the start. You must call os.path.basename(), and there is not way for you to call basename() directly (well, there is one, but they have taken special measures for it, so no name conflicts are possible).

But for this to work, I need a way to alias the importing module, because not all units have a 2-letter name like "os", for example I need something like:
USES 'TrigonometricLibrary' AS 'trig' in 'pas/TrigonometricLibrary.pas'

Of course I will not remove the existing behavior for USES, that is programmers can still say:
USES charset, matrix, socket;
and they get the behaviour they know.

But if the programmer says:
USES charset AS cs, matrix AS matrix, socket AS socket; than no symbols are made directly visible, and all of them need their prefix now (one of cs. or matrix. or socket. ). These prefixes are chosen by the programmer so thay may not create conflicts, as long as programmer sticks to the new form of USES.

If both forms of USES are intermixed, like

   USES unit1, unit2, charset AS cs, matrix AS matrix, socket AS socket;

than there is a chance that prefix cs will conflict with some unit1.cs symbol (either now or in a future version of unit1, or unit2). In this case I think it is appropriate to make the prefix "cs.", introduced by "charset AS cs", to have precedence over unit1.cs. This is because the programmer must use the prefix "cs." a lot in the program ((s)he has no choice, since the prefix is the only way to access simbols from unit charset).

In a corporate environment there will probably be some convention saying programmers have to use the second USES syntax at all times, to ensure no future name conflicts are possible.

As I said many Java programmers have been bitten by this problem, and now they explicitly list every single symbol they need to import from every package, composing what is known as "imports lists". Which is surely not fun at all, but it is safe! Most Java IDEs othen try to automate the generation of such lists, and collapse them when the source file is opened, because the lists can get very long.

Regards,
Timothy Madden

All this discussion with a warning for duplicate symbols is not helpfull:
- it is not backward compatible
- as someone else in this thread said, it is possible to put the current lookup rules to good use
- it is not enough to solve the problem I am talking about in this thread (that is, eliminate the risk of colisions when the compiler or the libraries are upgraded).

Using aliases for imported units, and making them required on each symbol reference, will not affect compilation speed or lookup for unqualified names.

Lookup for qualified names is faster than lookup for unqualified names.

Regards
Timothy Madden

Recompilation when one of units/libraries was upgraded, almost always implies rebuild. Who on Earth is so trigger-happy to upgrade, rebuild and not test for regressions? This is the only sane way to get the process going - document it *as MML said below*;

This AFAIR is sane programming practice, EVERY change you make, you test. And your managers HAVE to allow room and time for that, no less, no more. If they don't, they shall not be IT managers. There is no room for 'program done, out the door' culture any more. /IF/ there is one, just treat the program with updated/upgraded units as completely new product, completely detached from its original, and show it as such to the managers, if they need explanations. (even in free-software/open-source, 'releases' always/usually carry the versions of libraries/sources they've been tested with, not just in Pascal, take a look at what your average C (including ++ and #), Java and other programmers do as 'dependencies'. Otherwise no one ever could wrap their head around the resulting heap of s*t.)

That's what /testing/ is for.
There must be a rule: no untested changes are published. If there is none in your company, then *you* make one for yourself and keep to it.

Thank you, Timothy Madden

HTH,
Lukasz

Lukasz Sokol wrote

Recompilation when one of units/libraries was upgraded, almost always implies rebuild. Who on Earth is so trigger-happy to upgrade, rebuild and not test for regressions? This is the only sane way to get the process going - document it *as MML said below*;

That's not the point. Why on earth should a programmer *not* get as much help as possible from a programming language/compiler? That's the genuine purpose of a programming language. Why stick with flaws if they are avoidable?
*If* the issue Timothy reported can be avoided why *not* doing it? It would help avoiding many problems and reduce testing time. Although I have not such a strong feeling about it I would still appreciate the suggested addition. It would add a function for safer programming without drawbacks (except the coding effort of course). Though I don't know whether it can be implemented easily (if at all).

Jürgen Hestermann wrote on Wed, 22 Aug 2012:

Indeed.

Though I don't know whether it can be implemented easily (if at all).

There are definitely some practical problems:
* class/record helpers. The last class/record helper for a particular type that has come into scope automatically adds its methods to the helped class/record. If symbols can only be accessed by explicitly qualifying them with a separately specified unit alias, I don't see how you would "activate" class helpers from aliased units. Would these be an exception to the rule? Or would this require a new language construct to import all desired helper types explicitly?
* Objective-Pascal categories are the same as class helpers, except that there an unlimited number of categories can be active for a single class at the same time. In the end, it's the same problem as above though.
* Objective-Pascal also has the special "id" type. It is possible to call any method of any class type (and associated categories) in scope on a variable of this type. Again, I don't see a way how the proposed mechanism could be used to tell the compiler which classes it should search and which not if it should only search fully qualified identifiers from aliased units.

There may be more issues, but these are the ones I can immediately think of.

Jonas

Feel free to extend the compiler with such a feature and provide a patch. If you give enough thought to it, provide enough tests and ensure that this won't influence old code too much the chances that your patch gets applied shouldn't be too bad.

If you don't implement it... well... it may sound harsh, but: there are already many (partly more important) features that we need/want to implement...

Note: I definitely prefer your solution instead of a warning ;)

Regards,
Sven

I said identifier, but I mean FILE and, in that case, I can't use it.

First, this is an idea, so should be improved.
Second, this is a problem of programmer; if he forget, sorry for him. Third, maybe this sintaxe should be used just on project (not unit) and in units only use the alias (f1 and f2 in that case).

I agree.

Marcos Douglas

At least you thought about it. :)
Well, must be a solution, but this (new) feature should please everyone first, I think.

Marcos Douglas

I proposed this feature, a couple years ago, I said now again and got the same answer: "Feel free to extend the compiler" :( I never worked with compilers so, I'm not the programmer for that task, at first, sorry.

If you don't implement it... well... it may sound harsh, but: there are already many (partly more important) features that we need/want to implement...

Well, I will continue waiting a couple of year... :)

Note: I definitely prefer your solution instead of a warning ;)

Thanks.

Marcos Douglas