Sign in Join
Wiki copytemporary

OS aware RTL: what changed

Revision 0 to 1. Removed lines have a minus sign; added lines have a plus sign.

Title: OS aware RTL

OldNewLine
1+## The problem
2+
3+Currently there's a runtime problem with binaries created by [[FPC]]. The problem lies in [[libc library|libC]]/syscall numbers but it spans to userland as I'll explain later.
4+
5+I'll talk mostly about the Linux [[RTL]] since it's the most chaotic OS out there but this really applies to any [[Operating System|OS]] including Windows.
6+
7+1. Syscall number changes
8+2. LibC [[Function|functions]] missing
9+
10+It's a known fact that syscall numbers change. This happens on all unices out there more or less frequently. The problem with FPC RTL is that it's smartlinked into the binary. Although this is also a big advantage, the drawback is that things like syscall numbers are hardcoded. This means one of two things for ABI compatibility:
11+
12+a: Old syscalls will be used on new OSes if the binary was compiled on old ones\
13+b: Non-existing syscalls will be used on old OSes if the binary was compiled on new ones
14+
15+There's more: as an example I'll use the newly added epoll* functions in linux.pp. These are used as syscalls if {\$FPC_USE_LIBC} is false. This way the binaries will compile even on 2.4 kernel where epoll doesn't exist. The problem comes if someone tries to create a libfprtl.so with {\$FPC_USE_LIBC} on these systems (or use one from 2.6) since smartlinking cannot "hide" the missing functions anymore.
16+
17+## The proposal
18+
19+Currently the only thing used to alleviate this is {\$ifdef} and good luck. This won't last forever and if fpc wants to both use latest features and be backwards ABI compatible, something which I call "OS aware binaries" must be created.
20+
21+Lets say we change the unit which defines syscall numbers on Linux to something like this: (note: I know it's chopped up to include file but let's keep it simple)
22+
23+``` pascal
24+unit
25+ Syscalls;
26+
27+interface
28+
29+var
30+ syscall_nr_epoll_create: Integer; // defined them as variables
31+ ...
32+
33+implementation
34+
35+uses
36+ Linux, Kernel24, Kernel26;
37+
38+initialization
39+ if VersionToInt(LinuxVersion()) > VersionToInt('2.5.66') then begin
40+ sycall_nr_epoll_create:=...;
41+ end else begin
42+ ...
43+ end;
44+
45+end.
46+```
47+
48+Another solution which applies to the epoll/libC problem is this:
49+
50+``` pascal
51+unit Linux;
52+
53+interface
54+...
55+function epoll_create(size: cint): cint;
56+...
57+implementation
58+
59+uses
60+ Syscalls, Epoll;
61+
62+var
63+ epoll_create_func: TEpoll_CreateFunction;
64+
65+function epoll_create(size: cint): cint;
66+begin
67+ epoll_create_func(size);
68+end;
69+
70+initialization
71+ if VersionToInt(LinuxVersion()) > VersionToInt('2.5.66') then
72+ epoll_create_func:=Epoll.epoll_create // assign as it's there
73+ else
74+ epoll_create_func:=@Epoll.epoll_create_dummy;
75+end.
76+```
77+
78+And epoll would look something like:
79+
80+``` pascal
81+unit Epoll;
82+
83+interface
84+
85+type
86+ TEpollCreateFunction = function(cint): cint;
87+ ...
88+
89+function epoll_create_dummy(size: cint): cint;
90+...
91+
92+var
93+ epoll_create: TEpollCreateFuncion;
94+ ...
95+
96+implementation
97+
98+uses
99+ dl;
100+
101+{$ifndef FPC_USE_LIBC}
102+function epoll_create_syscall(size: cint): cint;
103+begin
104+ // do syscall stuff
105+end;
106+{$endif}
107+
108+function epoll_create_dummy(size: cint): cint;
109+begin
110+ raise TOSMismatchException.Create; // we don't have this one here...
111+end;
112+
113+initialization
114+{$ifndef FPC_USE_LIBC}
115+ epoll_create:=dlopen();
116+{$else}
117+ epoll_create:=@epoll_create_syscall;
118+{$endif}
119+
120+end.
121+```
122+
123+This should enable libfprtl.so to be compilable and linkable on any Linux version. Of course it should be compiled on latest possible so all "features" are there.
124+
125+Of course there are cons and pros, and inherited problems in this solution.
126+
127+The pros:
128+
129+- ABI compatible binaries
130+- Fixes certain future problems (epoll/FPC_USE_LIBC)
131+- It would be just cool since FPC would be the only compiler with proper ABI compat. on linux
132+
133+The cons:
134+
135+- Overhead both memory and CPU (initialization slows down, smartlinking would be useless on these things)
136+- Detection of OS version is more than tricky even with standardized POSIX.1 functions like uname
137+- problem still only runtime detectable. (language exception instead of CPU detection)
138+- more code, several interfaces (2.4, 2.6, libc) involved, bigger chance that one develops a problem and pulls down the whole binary. More code to maintain, bigger chance that it will get old and not fixed, and we have the same thing over again.
139+
140+## Areas of use
141+
142+As was stated before there are basically two major problems which would be solved if this solution was implementable. There are a few additional problems which fall under this category however.
143+
144+1. Changes of Syscalls (changes also means new syscalls in recent versions)
145+2. libC linking problems over platforms with missing syscalls/functions
146+3. Use of effective syscalls. For example on linux 2.6 "oldselect" is currently used.
147+
148+The areas where this approach is viable are limited and few luckily. Only new syscalls and changed syscalls should be addressed this way. Same goes for functions/units like winsock1 vs winsock2. It should be clear now that this is not a system-wide solution or abstraction layer, but a solution to specific areas in RTL.
149+
150+The question now is, how should FPC RTL address ABI issues? There are three known possible ways known to me:
151+
152+1. Ignore new features/syscalls in OSes for the sake of compatibility.
153+2. Ignore backwards compatibility and use only new features/syscalls.
154+3. Try to stay compatible while using new features on new versions and vice versa.
155+
156+The pros and cons of these are:
157+
158+1. This is how it was done mostly in the unix world. Only common stuff was added to the RTL, units like \<OSName\> with OS/version specific stuff are rather new. This approach leads to use of old/obsolete syscalls which might result in FPC being unusable for highly optimized specialized code.
159+2. This is even worse than \#1. It brakes ABI compatibility and binaries will become "C-like". Smartlinking can alleviate this problem if RTL is linked in and users don't use special features, but it does not solve the "oldselect" problem.
160+3. This is what this solution was meant to be. Problem is implementation. At the moment there's no known way of knowing OS versions for sure in unix.
161+
162+---
163+*From the Free Pascal wiki, page "OS aware RTL", as it stood on 2020-06-28 after a change by Kai Burghardt, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).*
164+Category: FPC
165+Category: Proposals

Back to history