Does FPC implement sub-allocation , in which the user program allocates a block of memory from the heap , and only "plays" in that sub-allocated block , such that if the user program has a serious memory leak , the user program may crash but it cannot exhaust the OS memory and cause the OS to crash?

... sorry for the long winded sentence.

Thanks in advance
Brian

"Brian" wrote

Does FPC implement sub-allocation , in which the user program allocates a block of memory from the heap , and only "plays" in that sub-allocated

block

, such that if the user program has a serious memory leak , the user

program

may crash but it cannot exhaust the OS memory and cause the OS to crash? ... sorry for the long winded sentence.

FPC's default heap manager allocates until the OS reports that the process' address space is exhausted which for 32-bit processes on Windows without LargeAddressAware is at around 2 GB. After that you'll receive EOutOfMemory exceptions.

You can however implement a heap manager that works differently.

Regards,
Sven

Thanks Sven.

Do you know how it behaves when running under Linux?

Brian wrote

Thanks Sven. Do you know how it behaves when running under Linux?

What do you mean with this question ?

Large memory blocks are allocated directly from the OS. For small memory blocks, pools are allocated from the OS, and the small memory blocks are then taken from the pool.

Michael.

2014-10-31 16:33 GMT+01:00 Michael Van Canneyt :

I think that Brian is asking if a memory leak could eat the system's memory irreversibly. IIUC in Windows, when the program is stopped, all it's memory is freed, even if the program leaked memory. I don't know about Linux, but I'd be surprised if it weren't the same.

This does not mean the OS will not be almost as bad as crashed: if your program leaks too much, you may trigger so much thrashing that your system will become unusable. This did happen quite frequently to me with Windows XP 32 bits. I now use Windows 7 64 bits, and it hasn't happened to me since, but I guess that's only because I haven't tried hard enough :-)

Frederic Da Vitoria wrote

Yes that is exactly what I was asking. For a high reliable system running 24/7 my fear using Object Oriented code vs procedural code is exactly what you mentioned . If there is a serious leak , or even a small one for an extended period of time , it can take down the OS. I have seen this happen with Sun OS and a 3rd party driver that leaked and caused the OS to crash after an extended period of time.

First, Linux will simply kill off processes if memory is exhausted, to ensure system stability.

Second: a driver is something special. Most drivers run in kernel (OS) space. If the kernel itself leaks memory, obviously it can not kill itself...

Third: on unix, you can tell the system how much memory a program is allowed to consume. If the program tries to consume more, it will simply not receive it. See the getrlimit/setrlimit calls.

So for a user-space program there really is nothing to worry about.

Michael.

An operating system certainly /shouldn't/ crash if user processes allocate excessive memory. The overall system might thrash which would make it unresponsive to e.g. new SSH sessions, but that's hardly the same thing.

I've got stuff here running 24/7 with uptime measured in months. Historically, the memory leaks weren't in application code but were in database backend stuff that managed its own heap, which made it very difficult to monitor.

It's easy enough to code such that when you perform a complex activity which /should/ free all memory on completion you check that it does so. It's also easy enough to put something in your UI that reports how much is currently on the heap, and if you see that it's growing you can investigate in a development environment using e.g. HeapTrace.

In extremis, it's usually possible to get a program to restart itself if something's obviously wrong. I don't know if it's still the case, but Apache used to limit the number of requests each process would handle to protect it from memory loss due to faulty CGI etc.

If necessary, you can use a custom heap manager e.g. if you don't want it to expand beyond some preallocated size. In principle, you could code yourself multiple heaps each with custom procedures/functions, but while these might be useful for stuff that you're writing yourself you'll not be able to use them with the standard OO classes.

I don't believe that there's an equivalent to mark/release in the standard heap managers, and from discussion elsewhere I don't believe it could be added in a compatible way. Some variant of Mark() that took a parameter suggesting how much memory could be allocated could be useful, but your focus should be on preventing leaks and runaway rather than constantly recovering from them.