Hi All!

I'm trying to use the FPC 3.2.0 beta + Lazarus 2.0.1 with fpCEF3. But when I try compile this package with FPC 3.2.0 I got this error:

Error: Local variables size exceeds supported limit

But the same package compiles fine with FPC 3.0.4 and 3.0.5.

Should I fill a bug report?

Ahh, in time:

FPC 3.2.0 beta buit with sources updated today (07 - mar - 2019) 64 bits Linux Mint 19.1 x86_64
Lazarus 2.0.1
fpCEF3 v 3029.1

Sorry if I forget to include some information.

It would be helpful if you provide a code snippet of a function where the error is triggered. Especially the part with local vars declarations, including all referenced custom types.

Yuriy.

FPC 3.2.0 limits the total size of local variables and parameter space for a single procedure/function to 2GB on 32 bit and 64 bit targets. On FPC 3.0.4 the limit was the maximum size supported by the address space.

The previous size check for the locals was not working correctly according to the commit message, but I don't know why this was changed like that for 64 bit targets.

Jonas

Hi Jonas!

Thanks by your feedback. Do you think that should I fill a bug report?

The datatype that throws this error is this (cef3types.pas:2416):

TCefCompositionUnderlineArray = array[0..(High(Integer) div
SizeOf(TCefCompositionUnderline)) - 1] of TCefCompositionUnderline;

The TCefCompositionUnderline is a record with 20 bytes of size (5x32 bits fields/vars into this record). So only this var exceeds this limit on 32 bits systems.

The error is throw on this procedure, in cef3ref.pas:1574

procedure TCefBrowserHostRef.ImeSetComposition(const text: ustring;
underlinesCount: TSize;
  underlines: TCefCompositionUnderlineArray; const replacementRange,
selectionRange: TCefRange);

I think that 2GB on 64 bits a very conservative limit.

Jonas, do you remember the revision number where this was changed?

Fabio Luis Girardi wrote

Thanks by your feedback. Do you think that should I fill a bug report?

That's up to you.

I think that 2GB on 64 bits a very conservative limit.

2GB stack space on is very large on any platform. Normally the heap is used for large allocations.

This probably mostly stems from 32 (and 16) bit platforms, where the issue was that it was hard to have large contiguous address ranges in the virtual memory space. OTOH,
* several systems still support limiting the stack size separately from the rest of the program's memory to more quickly catch infinite recursion errors. Thread stacks are also often limited in size.
* The offsets you can encode in a single instruction are always limited. E.g. on x86-64, offsets relative to the stack/frame pointer are limited to 2GB. So if you allocate more stack space than that, accessing local variables that are outside this addressable range will become slower because you'll need extra instructions (while local variables are often accessed a lot)

Jonas, do you remember the revision number where this was changed?

r39916

Jonas

Jonas Maebe wrote

r39916

Thanks. How fpcef3 isn't my project and I don't understood how it works (I don't understand why a very large array is needed), I'll switch back to a previous revision number and wait the FPC 3.2 to be released. After that I'll see if I report a bug to FPC or fpCEF3 team.

The best regards,

Fabio

FYI on 64-bit Windows the stack size limit is hard coded to 1GB. The same 1GB limit applies on 32-bit Windows. Probably Linux can be configured for larger stack.

But I agree with Jonas, that using huge local vars is pointless and inefficient.

So it is not the FPC issue, but the library design issue.

Yuriy.

Passing a 2GB array _by value_ as the "underlines" parameter for the procedure "ImeSetComposition" is totally wrong. A pointer to an array of "underlinesCount" size should be passed here.

You should file a bug report for this library.

Yuriy.

Yuriy Sydorov wrote

FYI on 64-bit Windows the stack size limit is hard coded to 1GB. The same 1GB limit applies on 32-bit Windows. Probably Linux can be configured for larger stack.

Linux needs to be specially configured to forbid a larger stack.

But I agree with Jonas, that using huge local vars is pointless and inefficient. So it is not the FPC issue, but the library design issue.

I disagree. We should not break things unless it is required. On Win64 we should probably give an error when the stack allocation size is larger than 1GB, but on other 64 bit platforms we should allow up to high(int64).

Jonas

If I recall correctly, currently the compiler uses longints as stack offsets internally in several places. That's why the limit is 2GB even for 64-bit targets.

The issue in this tread is caused by the incorrect declaration of procedure's params. See my other answer. In this case 2GB of stack is not suppose there are no real world apps which require 2GB of stack for local vars or parameters. Even if we consider future apps it is useless to support 64-bit stack offsets for local vars of a single procedure.

Yuriy.

Yuriy Sydorov wrote

If I recall correctly, currently the compiler uses longints as stack offsets internally in several places. That's why the limit is 2GB even for 64-bit targets.

That would be compiler bugs that need to be fixed. Ideally, the commit message would also have mentioned that :)

If the architecture and OS support it, I see no reason to forbid it. It's always possible to add a hint about efficiency in case the size of the locals grows beyond offsets that can be embedded in a single instruction, although this hint would trigger already at 32KB of locals on e.g. PowerPC.

Jonas

Hi,

Jonas Maebe wrote

It's always possible to add a hint about efficiency in case the size of the locals grows beyond offsets that can be embedded in a single instruction, although this hint would trigger already at 32KB of locals on e.g. PowerPC.

I'm all for such a hint in any case, even if that limit is quite low on many archs. It's useful also on m68k, where the "efficiency limit" is also 32K. One who ports code across many CPU archs, especially slower ones (embedded anyone?) such hint could be important to discover inefficiency hotspots.

Charlie

Sure :)
I've checked it right now. Indeed the compiler uses longint values as the stack offsets here and there. It need to be fixed in order to support 64-bit offsets. Earlier versions of FPC simply does not contain the proper checks and does not throw en error for local vars exceeding 2GB. But the resulting generated code is broken anyway. So no regression here.

Sure, if someone would volunteer to implement this :)

ARM also requires several instructions for offsets larger than 4KB and this is normal. Therefore the hint should be issued only for huge offsets related to the bitness of a target CPU. The warning would be even better, since in most cases usage of huge local vars/params means a bug or not desired effect, such as discussed in this topic.

Yuriy.