http://www.freepascal.org/wiki/index.php/OS_aware_RTL

Tell me what you think

Ales

http://www.freepascal.org/wiki/index.php/OS_aware_RTL Tell me what you think

First: only Linux has the main problem. BSD never renumbers ABI calls, newer abi calls have a different prototype.

The pros: * ABI compatible binaries

As long as only syscall nrs change. This rarely happens. Usually something gets 64-bit, or has an extra param, or different structure etc.

* Fixes certain future problems (epoll/FPC_USE_LIBC)

That is a nice trick, but IMHO only important for that hand full of "problematic" calls that are urgently needed but recently added.

* It would be just cool since FPC would be the only compiler with proper ABI compat. on linux

ABI compat is way more than using the right syscall nrs.

* more complicated, thus more bugprone.
* case that this (except libc trick) actually saves anybody is IMHO near zero
* All binary compability must be prepared in the binaries that go onto the field. That makes them legacy binaries anyway.

So in short: yes, some of the techniques are usefull if there indeed is such huge problem with dynlinking, that can't be solved with some form of lazy linking.

But the workaround should be kept to those functions _only_, and only if there is a real chance that anything actually might go wrong.

First: only Linux has the main problem. BSD never renumbers ABI calls, newer abi calls have a different prototype.

No even BSD adds new stuff from time to time, like kqueue, altho that's older.

Well yes but there are other implications, see winsock1 vs winsock2 problem in windows.

Yes I agree.

ABI compat is way more than using the right syscall nrs.

Indeed. That's why I included the libc parts too. It just illustrates some uses.

Yes indeed. I should perhaps clearly state that it shouldn't be the standard for all units etc.

Ales Katona wrote

http://www.freepascal.org/wiki/index.php/OS_aware_RTL Tell me what you think

Something like this is already partly done for OS/2 (memory allocation above 512 MB which is only supported on newer OS/2 versions) and use of dynamically loaded imports for e.g. 64-bit file access (with an alternative using a stub relying on old 32-bit calls) has been decided for OS/2 target quite some time ago (although not fully implemented yet :-( ).

Tomas

This is manifestly wrong:

Sure, there are new syscall numbers in linux, but the old numbers still work as they always have. Proof:

The current set of numbers already works since 10 years.

I'm not saying you'll have the latest features with the old numbers, but that is irrelevant.

Windows has the same issue. For almost each and every OS XYZ call, there is a XYZex call. It's normal if you want to maintain backward compatibility, but also want to give new version numbers.

IMHO all we need to do is to decide which call (number) we want to use, and warn people that some things may not work on older systems.

Michael.

Yes but this is more than about syscalls. Ofcourse I didn't mean to say that the whole RTL should be done this way but stuff like winsock1/2 and epoll/kqueue calls are a nice example IMO. + it also fixes FPC_USE_LIBC for those things.

Ales

That's the main point, I guess. As it is now, we have to decide and either sacrifice the new features, or compatibility with slightly older platforms. My understanding is that the proposal of Ales was related to exactly this situation.

If I understand it correctly, his suggestion basically allows to extend the support for older platforms (i.e. keeping your binaries working properly on those platforms) while still supporting the new functionalities if they are available (i.e. if running on a newer system version).

It's obvious that this approach cannot work correctly under all circumstances. In some cases there's just no fallback solution available, so you might end up with an exception while trying to use the binary on some old system version. However, your application can still handle this situation and present it to the end-user in friendly way (e.g. notifying him of this fact, but still allowing him to use the other functionalities not relying on the particular new API call).

Tomas

Yes that's the idea. IMHO the question is not if such a solution is the right thing to do in these situations but if it's POSSIBLE to do. I've looked at the uname syscall (which logicly cannot be "modernized" this way and there's no need to either).
I must say POSIX sux hard. The people who made this standard could save everybody lots of trouble by not doing it. As it stands out, I don't think there's any reliable way of finding out a version of OS in the unix world. They simply didn't bother with format specification.

Even if a parser is made which will "fall back" to "compiled-by-version" if the numbers don't make sense, there's still a possibility that you end up parsing it wrong thinking it's right, making the resulting binary malfunction. This is the biggest problem I see.

I guess Unix won another round of Stupidity vs ABI compaibility.

10:0 so far.

Ales

Op Fri, 10 Mar 2006, schreef Ales Katona:

http://www.freepascal.org/wiki/index.php/OS_aware_RTL Tell me what you think

I don't like to do the abstraction at the syscall level, but one level higher, i.e. the Tstream implementation. The reason is that the operating system abstraction happens at this level:
- OS abstraction wis present here.
- Emulating missing system calls is often much more inefficient than a higher level workaround.

Adding an extra layer of abstraction without a good technical reason greatly increases the danger of overdesign and code bloat.

Daniël

I never wanted to emulate missing syscalls.

The reason is libfprtl.so

How does this make it a reason ?

libfprtl will always be specific to the distribution it was compiled on. If tuned, it should be tuned to that system.

Just like libc or any library close to the system is. Don't try to copy a binary libc.so from a SuSE to a Fedora system, it won't work. Just like you shouldn't copy kernel32.dll from Windows XP to Windows 2003.

Michael.

I don't think this is feasible. It would be if you'd get it into those distroes but people will want to take their own < 1mb libfprtl.so with them rather than copy 30mb fpc on various distroes with their apps.

Ofcourse the question is, do we want to utilize libfprtl.so at all? How do we want to cope with new features in OSes?

Ales

What about Lazarus/binary packages? I think libfprtl.so is required there. If you use syscall version of this lib it's ok. If you use FPC_USE_LIBC however, it will cause linking problems on older distroes for example. I wonder how do you plan to fix this little problem.

I've updated the wiki page to clarify what parts of RTL I wanted to be affected etc.

There's also the "oldselect" problem for example. Might seem like a triviality but select() is used for many things (sleep, file IO etc.) which might mean FPC binaries would be much slower on 2.6 for example.

Ales

That is exactly my point:
this <1mb libfprtl can only contain low-level stuff which can be expected to be the same on all distros...

Ofcourse the question is, do we want to utilize libfprtl.so at all? How do we want to cope with new features in OSes?

We don't.
We just make the interface available, that's it.
It shouldn't be used by default in libfprtl.

Michael.

Op Sun, 12 Mar 2006, schreef Ales Katona:

Using libc will by definition cause dependency problems on the libc version. Add to that that different libc's are more incompatible that just the linking errors. In other words if you want to support multiple distributions, you *must* use syscalls.

You can safely use the new select; it is at least present since Linux 2.2 and more likely 2.0.

Daniël

Hmm I'll update the RTL then. Thanks

Ales

I dropped a hint in an earlier mail, but the linker guru's didn't respond to that, so I looked it up:

To my knowledge there is something like lazy evaluation, iow symbols are not resolved until they are used (http://www.iecc.com/linker/linker10.html , search for lazy ). So it might be just setting some bits on the symbol in the linker.

IOW a manual solution is not 100% necessary.

There's also the "oldselect" problem for example. Might seem like a triviality but select() is used for many things (sleep, file IO etc.) which might mean FPC binaries would be much slower on 2.6 for example.

These things are only temporary. For BSD I sometimes put stuff like this under "FREEBSD5" directive, so that i can simply tell people that run into these problems to recompile their compiler using that directive.

For Linux this is a bit harder though, due to separate versioning of all components.

a: Old syscalls will be used on new OSes if the binary was compiled on old ones

Would this work?

b: Non-existing syscalls will be used on old OSes if the binary was compiled on new ones

Does this problem really exist on the rtl??? I distribute software in binary form for Linux and I never had trouble with old kernels.

Daniël Mantione wrote

Using libc will by definition cause dependency problems on the libc version. Add to that that different libc's are more incompatible that just the linking errors. In other words if you want to support multiple distributions, you *must* use syscalls.

I find a great advantage of Free Pascal that we are free from the libc nightmare. You can compile once and expect it to run on all distros and even the very old ones!

On c, because of glibc, you need to recompile for each libc version you are targeting =(

I find it very frustrating when I cannot upgrade the packages of my distro because once again the developers of libc broke the ABI and now I need to reinstall everything =/ And beliave me, they like to break the ABI often.

thanks,