I would say that that's a little naïve and dangerous to think like that. Sure, Windows might have the means to clean up memory after an application terminates, but not all platforms have such heap deallocation features (e.g. pure DOS, where certain procedures and interrupts remain in memory even after the application terminates... so-called memory-resident programs).

Even if all blocks are freed when the compiler finishes its work, memory leaks may still cause problems with sufficiently large projects that are being compiled in one go.  The more leaks you have, the more likely you're going to hit an out of memory situation, especially as you can't really control the target platforms in regards to how they have their physical and virtual memory and page files configured.
A lot of the issues do come about from the fact that Free Pascal has a lot of legacy code and started life as a 16-bit Turbo Pascal project (it was first made because Turbo Pascal was being discontinued so Borland could focus on Windows development, and it didn't have 32-bit support).  Free Pascal is advanced enough now that a lot of the features could be rewritten to use more robust and self-contained features, but it's going to be a long job.
I've made some suggestions to major improvements, especially to speed or reducing the size of the compiler's binary (and the size of the binaries that it generates), but Florian has turned down some of the patches either because it's too experimental (my Deep Optimizer prototype) or overcomplicates the codes (I replaced a large case block for the x86 peephole optimiser with a custom-made binary search system, which was slightly faster but wasn't obvious in its function and would silently break if it wasn't sorted properly).

I would say that trying to minimise the reliance on global variables would be a big improvement to the node builder, because that would allow it to be multi-threaded.  *makes note of a possible future area of research!*

Gareth aka. Kit

On Mon 30/07/18 13:40 , "Thorsten Engler" sent:...

Freeing memory is irrelevant if your process is going to exit anyway right after freeing it. The OS doesn’t care about any memory your process has allocated at that time. All pages go back into the available pool.

 ...

The compiler is not such program. You obviously need to choose for which programs you do this.

But having the memory freed properly would also enable the use of the compiler as a DLL in e.g. Lazarus.

Michael.

Hi,

While I generally agree, in practice it's usually a non-issue.

Pure DOS, as in the 16bit target is not supported to run the compiler, only via GO32V2, and having DPMI TSRs is... well... :)

On Amiga and similar systems, AROS, MorphOS for example, which indeed does not free up the memory after a program terminates, I implemented a memory pool in the FPC RTL, so all memory is freed anyway during the program's exit. Any application which wants to go beyond this behavior for whatever of its own reasons is free to use the OS allocation functions directly, or implement its own memory manager. The same with open files on this platform, actually, which wouldn't be closed without the RTL taking care of them. And I also have to manage our own stack, and lets not even mention threading... :)

The sideeffect is, that this approach silently "fix" the compiler's own leaks too, so FPC remains useable on these systems. In general it is much safer anyway, because expecting all the broken code out there to get fixed for such systems is sadly just naive.

Having this said, anyone who still fixes leaks win the compiler will be my hero! :)

Charlie

Not necessarily.  The Internal Errors are effectively assertions that should never be triggered. True, cleaning up the memory is a bonus, but here it's safe to assume that the internal state of the compiler is bad in some way.  Throwing an exception upon too many errors might be problematic because there's no telling where it might trigger, but otherwise, the compiler abort tends to occur after it's finished attempting to compile its current node trees, so most memory should be freed.  Still, it's something one might want to research to see if the exception handler can check and free some global structures.

Gareth aka. Kit

On Mon 30/07/18 16:24 , "Michael Van Canneyt" sent:

Sven Barth wrote

Michael Van Canneyt schrieb am Mo., 30. Juli 2018, 15:30: > Obviously provided you don't install another mechanism that does this

and

aborted

e.g. when too many errors occurred or an internal error is encountered.

This is bad news, because it in effect means that the try/finally is necessary
if people want to fix memleaks :(

This will considerably slow down the compiler.

Michael.

Compilers are one of those programs that fall under a very special, very exclusive set of projects where a lot of the standard rules may not apply, but if they can be followed where possible, brilliant. Sometimes, increasing complexity and maintainability in the name of speed is not a solution, as I found out with my binary search implementation.  However, a way around that is to see if we can program such a feature into the compiler itself if it detects it can gain a speed boost from it (in this situation, a case block with 36 values to choose from, but where each branch calls an identically-structured procedure... some of the values call the same routine, but otherwise the routines have the same parameters and return types).

If you can find a way to fix a memory leak without sacrificing performance too significantly, by all means submit a patch for review. If I had my way, if I wanted to make the compiler as fast as possible, I'd just drop into assembly language for the most critical parts, but that's a big no-no in this particular project, not because of how hard it is to maintain, debug and explain, but because of cross compilation issues.  You should be able to create a compiler that targets any platform of your choice.  I think the only time you can use assembly language is with some of the internal compiler routines like the Frac function.

Gareth aka. Kit

On Mon 30/07/18 17:42 , "Marcos Douglas B. Santos" sent:

All us like fast compiling...
However, IMHO, a compiler should use the same good practices that others programs must use.
If it cannot use them because it's very slow, what is the matter about good practices?

"Do what I'm telling you to do, but don't do what I do" ? :)

best regards,
Marcos Douglas

Correct me if I'm mistaken, but I believe one shortcoming with the Free Pascal Compiler, despite its speed, is that it isn't multi-threaded.  It can't compile multiple independent units at once, and probably won't be able to for a while due to the use of global variables in the node builder.  When it comes to building Lazarus and the compiler, this is gotten around by the Makefiles running several instances of the compiler on different processes, building independent packages that can later be linked.
I personally think that if the compiler can be made multi-threaded, then projects like OpenSceneGraph might see a huge improvement in compile times if the compiler is smart enough to identify units that don't depend on each other and compile them in parallel.  I'm not sure how cross-compatible it can be made, since different platforms approach multithreading in different ways and I've noticed that the compiler tends to avoid using SysUtils and Classes and the like (so no using TThread for now).

This is getting a bit off-subject from fixing memory leaks though. Gareth aka. Kit

On Mon 30/07/18 19:39 , sent:

yep, for sufficiently large projects... when we compile OSG (OpenSceneGraph)
over here (using -j 8) it takes 20+ minutes for the first time or if we reconfigure from debug to release mode or the other way around...

now, after the first time we compile it, the next times takes mere 10's of

seconds... then we move on to SceneGear and FlightGear when we're building that
whole project which uses OSG... 30-45 minutes for a complete, from the ground up
builds are not uncommon... it takes me back to the days of 30 years ago when
you'd start the (borland/turbo pascal) compile process and go get some c0ffee ;)

note: the machines these builds are being done on have 16Gig RAM, 1Tb HD, and a
4Ghz 8-core AMD FX-8350 CPU...

I thought I'd add a point about compiling and memory. On the Raspberry Pi, I know for a fact that building the compiler can fail after a while due to insufficient memory. When this happens the problem with no memory doesn't present itself as such. Rather you're left with an error indicating a problem with the source. I've gotten around this problem by making the compiler without a desktop environment running and re configuring the swap size.

Might this problem be related in some way to the discussion in this thread?

As mentioned before, the main problem with try...finally blocks is that they incur a massive speed penalty, so they're not things you want to enter and exit on a frequent basis. Whether that means we need to work on the compiler so it reduces this penalty on such blocks (such as internally removing the construct and copying the code in the 'finally' section into a local exception handler) I'm not sure.

I do agree though that we should manage memory with care, especially as Pascal gives us such fine control over it (compared to, say, C++'s RAII feature which is a little nebulous at times with its garbage collection).  I think first though we need to fix any obvious memory leaks, namely objects and blocks of memory that we forget to free, then build up from there.  At least that will help reduce the risk of an out of memory situation during normal operation.

Gareth aka. Kit

On Mon 30/07/18 21:27 , "Marcos Douglas B. Santos" sent:

I'm sorry, but we work with a language that expected programmers should manage their objects.
The language even provides a clean way (try-finall) to do that. I use try-finally even to write a snippet to show an example of code...

IMO, that must be a habit in such languages.

regards,
Marcos Douglas

TThread is not a necessity. It's functionality and the synchronization primitives are based on what TThreadManager provides. This manager resides in the System unit and is accessible through functions like BeginThread and such. Please note however that this might need an external thread manager on certain platforms (e.g. Unix-like systems) and the idea on those platforms is for the compiler *not* to rely on the C-library.

Regards,
Sven

Adding locally allocated structures to a global list is a pretty gnarly issue that will take some time to solve. The easier ones are simply using labels to easily manage exit conditions. It looks a lot nicer than nested if statements.

The issue with such code is it is fairly boring to write, so I understand if someone didn't do it initially.