hi all,

i wanted to ask a question, before i spend time going down a road to implement it.

are there any know limits on how big a source file can be for the FPC compiler ?

i was thinking about experimenting with "code generating" a source file that would become a memory resident database of sorts.

this would make my application, implemented as a shared library "plugin" quite fast, and would potentially simplify the distribution of my shared library to my user base.

are there any thoughts on having a source file that might be megabytes in size that is basically just structured data implemented as source ?

tia,
Tony

are there any know limits on how big a source file can be for the FPC compiler ?

interesting question... hmm did you try a 3GB file which calls writeln('test') over and over and over again? That's the stupid way of doing tests - the smart way would be waiting for an answer from the FPC team.

Here are my prefered colors for your website (take it or leave it):

body bgcolor="#1E1E1E" text="#D7D7D7" link="#006699" alink="#006699" vlink="#006699"

Gotta love dark backgrounds.

hmmm website Sigs really do work because I went ahead and clicked on your site.

hmmm ... i'm sensing some sarcasm.

i'm not usually one to post dumb questions to a list because i'm lazy or have nothing better to do.

also, i thought this was the "users" list ... not the compiler developers list.

in any event ... i have no idea if putting 3GB's of "writeln" statements in a file is a good test.

in my case, the data would have to have structure.

it isn't just the size of the source file, but the size of the code as generated by the compiler into the DLL.

are there limits to the "code segment" or other such internals of the compiler and runtime file ?

i think the way i would have to lay it out would be to have 1000's of Pascal "record" types, with members that are a mixture of "constant" String and Double values.

i guess i would have to put all of these "record"s inside of a procedure that alloced and assigned them all as members of an array, upon invocation of the function at runtime ?

maybe i could put them in the initializer ?

it would actually take me some time to even lay the thing out to begin with ... only to find that it doesn't work ... or that I _assume_ it won't work ... because i took one path that didn't work, where there might have been another that would have ... had i known.

so you see ... i feel like it is maybe more complicated that just testing by echoing 3GB's of "writeln" statements into a source file and compiling it.

if no one has any thoughts on it ... i'll go to the next step ... but i didn't think it would hurt to ask the question.

but thanks for the help.

hmmm ... i just don't know if i'm reading into the intent of your comments very well.

btw ... the screenshots you see there are screenshots of a plugin i've written for a simulator i use from http://www.x-plane.com ... using Free Pascal.

-- L505

Tony

I think there are no hard limits on the source size. But you have to keep in mind that procedures (and maybe data) cannot become infinitely long, generating fast object code for them would be too complicated.

I would definitely try it.

BTW, lazarus is doing something similar, generating large strings that act as resource, the largest several 100's of KB. But they are statically linked.

Vincent

Correct, and extremely long sourcefiles will also use lots of memory, so there is also a more practical limit there.

IIRC the codegenerator was tested using 6MB large procedures, if you go over a few MBs per procedure you might get into trouble.

What means structured data? Source data size doesn't matter, they can be up to 2 GB, the important thing is the size of the resulting parsing pieces. I.e. very long procedures (>10k lines), statements, expressions call for trouble. If you create a 1M line file with 100k procedures or declarations, I don't expect trouble.

this is the answer that answers my question i think.

while i hadn't thought it all the way through, in fact, the picture i had in my head was one where there would be one massive function, that i would call when my DLL was initialized, that would create all of the "records" of data and add them to an array or collection for use by the might be able to break that function into several somewhat smaller functions, and aggregate the calls to each on via one.

procedure init();
begin
do100();
do200();
do300();
.
.
.
end;

procedure do100();
begin
{ add 1-100 of the N elements here }
.
.
.
end;

procedure do200();
begin
{ add 101-200 of the N elements here }
.
.
.
end;

thanks for the thoughts ...

Tony

> are there any know limits on how big a source file can be for the FPC compiler ? interesting question... hmm did you try a 3GB file which calls writeln('test')

over

None intended, I was serious.. personally, I tried the test and what will limit you is the memory on your pc. Thousands of writeln on my windows machine said "you are out of virtual memory". So this will only be your problem, as far as I tested..

i'm not usually one to post dumb questions to a list because i'm lazy or have nothing better to do.

It wasn't a dumb question - I'm a user too, I'm not a developer (yet).

in any event ... i have no idea if putting 3GB's of "writeln" statements in a file is a good test.

It sort of is, at least I found out something new.. the virtual memory blew up on my
PC at 180704 lines of source code ... with writeln('test');

in my case, the data would have to have structure.

That too, you could try something like a simple record over and over again. Your data wouldn't be over and over again, but it would still tell you quickly if FPC can handle 400MB of source code, 4GB, etc. A writeln('test') would still cause you to place thousands of strings into the exe and etc.

My idea was to let the FPC team tell you more about it than doing a silly test like this.. because I'm just a user and they know more. But I suppose it's not that silly because I learned something about virtual
memory when I tried it.

hmmm ... i just don't know if i'm reading into the intent of your comments very well.

I've experimented with dark backgrounds and found an "off black" to be nice on websites. I've found white on black is nice especially when it's an "off black" and an "off white". I was complimenting your site.. here I have written about this a while back:

http://z505.com/cgi-bin/qkcont/qkcont.cgi?p=black%20backgrounds

Only advantage with the old corporate black on white is that it is easier to send to the printer.

Tony Pelton said:

are there any know limits on how big a source file can be for the FPC compiler ?

Why don't you just compile a binary resource into your program? I assume that fpc can do the same thing as TP/Delphi somehow. That would keep your source file nice and small but do what you want. Editing large files is a considerable pain, they take ages to load.

Moz