Hi

There is lots of duplicate code and data definitions between Lazarus TProject and TLazPackage (and their ancestors).
I tried to improve the situation by creating a common base class for them. It is not possible because it ends up with a circular reference hell.

Almost 2 years ago I suggested to solve the problem in language syntax somehow.
http://lists.freepascal.org/lists/fpc-devel/2010-January/018909.html

The comments always tell to use interfaces and abstract base classes, and claim that this feature enforces good design.
Bollocks!
This feature enforces BAD design and implementation. Lazarus itself is an example of it.
Is copy-pasting code between Project and Package a sign of good design. IMO it is not.
Is the class structure somehow flawed if such problems happen. No it is not. Such things happen in big, real projects.

For the same reason there are many type-casts in Lazarus core units. A variable is type-casted from base class to the real class. Another common solution is to copy all code into one big file. It is not only a bad thing but it has some limits. You wouldn't want to copy most Lazarus code into one file for example.
Thus I found it strange that even Lazarus developers claimed they never encountered such problems. Yet, they have clearly worked hard to avoid circular references.
It is almost like in a religious sect, nobody dares to admit it was a mistake to join it.

I read the Bold application has abandoned Delphi native code and moved to .NET.
The reason was mostly circular reference problems, which I understand with such object framework tool.

I am not expecting any solution for this now. Even the latest Delphi has not solved it although it has many other syntax additions. I just feel strongly that the language will remain a niche, not interesting for most people, because of this one single problem. Pretty much all other features are good.
Pity!
I remember already years ago someone told me he used Delphi but it was not suitable for big projects because of circular reference problems, and he moved to other languages.
Then I didn't understand how serious the problem is but I learned it later.

If someone still wants to claim that there is absolutely no problem, please show me how to reuse code between Lazarus Project and Package thus avoiding the duplicate code, without copying most Lazarus code into one unit. Then I will believe there is no problem.
Or, explain me why it is a good thing to have all those type-casts in Lazarus core-units (connecting to Designer etc.).

Sorry for this rant. I like to solve problems but there is no way to solve this one, except for changing the language. Frustrated now...

Juha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20111004/82f6daa4/attachment.html>

introduce getter functions, or properties, and centralize the typecast. (see below)

IMHO it's a fine line.

I did encounter both, situation where I was forced to think about the design and ended up with a different and *better* design, because of this limit.
But also situation, where the opposite happened.

And while it may very well be, that in those cases where it was not an improvement, a better design might actually exist, the path to it appears blocked by tons of old code, grown over many years. Sure easy to say "refactor" (an I would love too), but resources are limited. In real live the amount of refactor one can do is limited, code grows over years, and not all of it is or will be as it should.

Anyway, having to accept the situation as it is, a few thinks I found helpful:

1)
Recent work on MainIDEIntf, which for some reasons is spread over at least 3 units. IIRC:
- MainIdeIntf => abstract base
- MainIDEBase => some implementations => this unit also has the global variable
- MainIDE => final implementation => this unit initializes the global var in MainIdeBase

Only I needed the variable in MainIdeIntf, as some units, which needed it, were themself already used by MainIdeBase
So I changed the variable in MainIdeBase to be a property (property outside of class declaration). That way it was easy to move it to another class, yet all code could still access it as wanted (and the property does the type cast once and for all. (Only needed to fix a few FreeAndNil as the property doesn't do for var param)

2)
The other problem I find, is that plenty of code requires circles, because some classes get to know about stuff they shouldn't. I did experience that in SourceEditor => for some reaon it knew about the fpdoc build-in editor. IMHO it should not, and I replaced all that with events, that fpdoc editor can hook.

This 2nd points shows exactly the problem: a lot of code that has grown, and has created dependencies in the wrong direction. But it is an almost indefinte amount of work to change all that.

But this 2nd point is also the example that helped improving design. By reducing the amount of knowledge that some classes now have about everything else, while they shouldn't have it..

2011/10/4 Juha Manninen :

While I agree that Lazarus contains some badly designed code, I am strongly convinced that adding even more circular references will make it worse, not better.
A proper alternative is moving to a good object-oriented design. Yes, that includes using interfaces and (sometimes abstract) base classes ;-)

There might be pains in transition, especially is there is a strong requirement for keeping some legacy API compatible (as was the case with my attempt to fix PropEdit).
However, I have never encountered a case where existing language features were not sufficient
for a proper design (as opposed to compatibly fixing existing design).

Please state more specifically what problems you encountered with TProject/TLazPackage and I am confident we will find a solution.

Now, as I stated above, the only real problem is that proper design often involves splitting gigantic units into a reasonable parts, which may pose a problem if there is lots of code relying on the previous monolitic design.

To combat this, I'd suggest "reexport" aka "unit inheritance" feature, where a unit is allowed to fully re-publish interface of another unit.

To illustrate:

unit OldBigUnit;
reexport Part1, Part1;
---
unit Part1;
interface
  type T1 = class ...
---
unit Part2;
interface
  type T2 = class ...
---
program LegacyCode;
uses OldBigUnit;
var
  a: T1; b: T2;

?

Juha Manninen hat am 4. Oktober 2011 um 12:01
geschrieben:

Hi There is lots of duplicate code and data definitions between Lazarus TProject and TLazPackage (and their ancestors).

?
True.?
On the other hand there are a lot of differences.
??

I tried to improve the situation by creating a common base class for them. It is not possible because it ends up with a circular reference hell.

Huh? We encountered lots of problems. See the devel list archives.? ?
?

?
OP has one big feature: type safety. In a language like javascript you don't need typecasts, but the drawback is that it is much more complicated to find a bug and to extend existing code, because you don't know what a change effects. It's one of the reasons why you can write one order of magnitude bigger projects in OP than in javascript.?
So, the typecasts are ugly, but they have an advantage. ? Of course it would be nice to reduce the amount of typecasts. ? Many typecasts are caused by deep class hierarchies.? Reasons for deep class hierarchies are
?
1. big classes were split into smaller ones. Logically it makes no sense, but the class and unit would be too big, so the class was split. The 'too big' depends mostly on the used tools aka IDE. In other words: It's nicer to work with ten 3000 lines units than with one big 30.000 lines unit.? ?
2. classes were split instead of using parameterization. Instead of adding a parameter to many methods, an abstract class and multiple descendants are defined. This almost always duplicates code, but the code itself has far less if-then-else blocks. OTOH it can create more if-then-else outside the split classes.?
?
3. information hiding. A class is split into the public part and the private part. Because the public part is used by third parties it must be carved in stone and therefore you must be very careful what to put there. ?
4. logical abstraction. In a big project like the IDE you have multiple definitions of 'public'. The debugger windows needs to access almost all properties/methods of the debugger, but the source editor should only access some of them. A developer working on the source editor should not need to understand the full debugger. That means some units should work tightly together and others less. There is no concept for this in OP, except splitting the class.?
?
That being said:
What duplicated method do you want to combine into one?? ?
Mattias
?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20111004/b24c23c1/attachment.html>

2011/10/4 Martin

That one looks interesting. I never used global properties myself.

Juha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20111004/129adf53/attachment.html>

2011/10/4 Juha Manninen :

I tried to improve the situation by creating a common base class for them. It is not possible because it ends up with a circular reference hell.

Just place the common base class in a separate unit.

2011/10/4 Juha Manninen :

I agree with you.
I found this example by Nikolai ZHUBR:
http://lists.freepascal.org/lists/fpc-devel/2010-January/018942.html I do not think this is better solution. For only one "circular reference" between 2 classes, I have to created 3 files! And if these references grow?!

These "base classes" with many virtual methods empty or raising Exceptions is, IMHO, bad design implementation too. These classes just exists to do the magic of polymorphism works... where the many many copy definition methods codes was the price.

Florian Klaempfl said more options here:
http://lists.freepascal.org/lists/fpc-devel/2010-January/018948.html They are:
- abstract base classes = I said about them...
- interfases = doubles the code many many times, just to work around the circular reference
- include files = doubles the code physically
- big units = hard to read...
- type casting (in the worst case) = yeah, the worst case...

Marcos Douglas

You misunderstood. Of course you do not have to create 3 files to use interfaces -- it is the reverse, interfaces allow you to split you code cleanly
into 3 (or more) files if/when it becomes too large for a single one.

In other words: It's nicer to work with ten 3000 lines units than

with one big 30.000 lines unit.

Why that? I would prefer the one unit including everything over having multiple files (i.e. when searching for something). If it has no other impact than I would always vote for having lesser files instead of cluttering all into many files.

2. classes were split instead of using parameterization. Instead of

adding a parameter to many methods, an abstract class and multiple descendants are defined. This almost always duplicates code, but the code itself has far less if-then-else blocks. OTOH it can create more if-then-else outside the split classes.

Code is much more readable (understandable) when if-then-else blocks and no OP-methods (like descendants) are used. You see all relevant code in one block and there is no need to gather it from multiple places.

3. information hiding. A class is split into the public part and the

private part. Because the public part is used by third parties it must be carved in stone and therefore you must be very careful what to put there.

True. But there must be also a way to evolve in time so it *must* be possible to change things. Of course, the less it's needed the better.

Hello,
Avoiding circular references is of particular importance for big projects. I don't understand your rant here. Several years back, when I was making professional software in Delphi, I noted two things:
- big projects in Delphi tended to be smaller than equivalent big projects in other languages (C++, Java)
- big projects in Delphi benefited much from the (very fast) compilation times of Delphi, itself resulting in part from the prevention of circular references.

Typical big project = 1M+ lines of code.

When I measure quality these days in C++ or Java programs, I find a <lot> of copy/paste, big classes, and many, many circular references, so allowing circular references obviously doesn't solve the problems mentioned here.

I noticed you haven't mentioned three other possible means of reducing copy/paste and sharing code:
- events
- notification/publication,
- generics.

Your comment about having difficulty to read "big units" is valid, of course, if the unit holds diverse classes with poor cohesion between them. But if the classes need to work together closely, as implied by the need for circular references, then it improves readability that they are close together in the same unit.

Looking at another language: Ada also prevents circular references between packages, with quite some success in its intended area. Admittedly, the latest versions of the language do provide very interesting capabilities for adding new features to an existing package, without recompiling/changing anything in the original package. Maybe these sort of features in pascal could help you, while keeping the safety net of the pascal language.

Best regards,
Thierry

...

I was talkin about that example... but doesn't matter if you create one or three files: your have to put all interfaces OR classes in same file using forward declarations if you want do use dependencies between these interfaces/classes... I can do this only using classes, but these was a problem said here, didn't?

If you do not use a single file for these interfaces/classes you should move up in the hierarchy creating over and over more abstractions... just because you have a single class A that uses class B and vice-versa. You do not consider it an unnecessary overhead?

Marcos Douglas

2011/10/4 Mattias Gaertner

No Mattias, this problem is not caused by type safety. It is caused by forbidden circular references.
Better comparisons are Java and the .NET languages C# and VB.NET, or even C++ (which has otherwise messy syntax).
In those languages you can refer to other classes as you like. One could argue that it allows more sloppy class design but in reality it IMPROVES the code.
No type-casts or interfaces just to get rid of circular references are needed, or any other such tricks.

True, the dynamic, weakly typed languages are not suitable for big applications but that is a whole different issue.

That being said:

What duplicated method do you want to combine into one?

All of them :)
I can send a list later, now I am at school in a C++ course.

Regards,
Juha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20111004/182f24f7/attachment-0001.html>

2011/10/4 Felipe Monteiro de Carvalho

:-)

Juha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20111004/4f6aaa59/attachment.html>

?

"J?rgen Hestermann" hat am 4. Oktober 2011 um 17:41 geschrieben:

It's matter of taste. I only gave one possible reason for splitting up classes. Everyone has to decide when it is appropriate and when not. Obviously the limits vary with every programmer.?
?
?

Well, that is not true in general. ?
?
?

[...]?

Mattias
?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20111004/1be7ccd2/attachment.html>

This sounds, as if units were to be split at random: Like start a new unit every 3000 lines... (Well I am exaggerating of course).

What belongs together into one unit, is major part of the design, and the size of units will be dictated by this. If you let your design being dictated by fixing the unit size, then you got an issue.

And btw, if the size of a file (not unit) is a problem: include files.

Still you should aim, for keeping as little as possible in a unit. It will improve code, by making sure, each class, has the minimum dependencies possible. Intruducing unecesary dependencies, by adding to much knowledge about other things into a class, is likely to cause problems at some time.

I do disagree. I really do disagree.
In fact, there are man y effort, to reduce "IF" blocks in code. Patterns like "State" or "visitor" for example.