Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2008-09

ctypes and RTL

11 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Ivo Steinmann

Hello all

I messed a littlebit around with size_t and ctypes. Now I thought it's best to fix all the ctypes stuff in RTL. All that code is quite complicated and hard to maintain. To solve this, I created a new branch for the RTL

http://svn.freepascal.org/svn/fpc/branches/ctypes/rtl

the branch solves this problems:
* ctypes is seperated from the rest of the RTL now, no mess with include files
* ctypes.pp contains all the c-types now (nothing more)
* ctypes.pp may include a platform dependant ctypes.inc (if required!)
* baseunix, unixtype and unix no longer define ctypes (they were included of ptypes.inc)
* unixtype is seperated now from baseunix and unix

+ the advantage is, that there's a clear structure
- the disadvantage is, that you have to include ctypes and unixtype into several packages (fixed in most cases)

A discussion in #fpc between me, Florian and Marco resultet in the idea to define a
csize_t, pcsize_t in ctypes.pp (size_t = csize_t). Since you can define a csize_t per platform, this is a good solution.

-Ivo

Reply by email

Marco van de Voort
to Ivo Steinmann

In our previous episode, Ivo Steinmann said:

Show quoted text 19 lines
I messed a littlebit around with size_t and ctypes. Now I thought it's best to fix all the ctypes stuff in RTL. All that code is quite complicated and hard to maintain. To solve this, I created a new branch for the RTL http://svn.freepascal.org/svn/fpc/branches/ctypes/rtl the branch solves this problems: * ctypes is seperated from the rest of the RTL now, no mess with include files * ctypes.pp contains all the c-types now (nothing more) * ctypes.pp may include a platform dependant ctypes.inc (if required!) * baseunix, unixtype and unix no longer define ctypes (they were included of ptypes.inc) * unixtype is seperated now from baseunix and unix + the advantage is, that there's a clear structure - the disadvantage is, that you have to include ctypes and unixtype into several packages (fixed in most cases)

That breaks compatability, and is the main reason why the current situation is as it is. To avoid telling people ten times a day "but you must also include xxxx".

A discussion in #fpc between me, Florian and Marco resultet in the idea to define a csize_t, pcsize_t in ctypes.pp (size_t = csize_t). Since you can define a csize_t per platform, this is a good solution.

I never saw the need to fix this. Or at least I didn't understand it.

Reply by email

Ivo Steinmann
to Marco van de Voort

no problem,
size_t stuff reverted (was a commit around r11752)

I maintain my translated headers on my own svn server, it's simpler for me.

-Ivo

Show quoted text 40 lines
In our previous episode, Ivo Steinmann said: > I messed a littlebit around with size_t and ctypes. Now I thought it's > best to fix all the ctypes stuff in RTL. All that code is quite > complicated and hard to maintain. To solve this, I created a new branch > for the RTL > > http://svn.freepascal.org/svn/fpc/branches/ctypes/rtl > > the branch solves this problems: > * ctypes is seperated from the rest of the RTL now, no mess with include > files > * ctypes.pp contains all the c-types now (nothing more) > * ctypes.pp may include a platform dependant ctypes.inc (if required!) > * baseunix, unixtype and unix no longer define ctypes (they were > included of ptypes.inc) > * unixtype is seperated now from baseunix and unix > > + the advantage is, that there's a clear structure > - the disadvantage is, that you have to include ctypes and unixtype into > several packages (fixed in most cases) > That breaks compatability, and is the main reason why the current situation is as it is. To avoid telling people ten times a day "but you must also include xxxx". > A discussion in #fpc between me, Florian and Marco resultet in the idea > to define a > csize_t, pcsize_t in ctypes.pp (size_t = csize_t). Since you can define > a csize_t per platform, this is a good solution. > I never saw the need to fix this. Or at least I didn't understand it. _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

Reply by email

Florian Klaempfl
to Ivo Steinmann
no problem, size_t stuff reverted (was a commit around r11752) I maintain my translated headers on my own svn server, it's simpler for me.

Writing code satisfying everybody's need is indeed not easy :)

Reply by email

Marco van de Voort
to Florian Klaempfl

In our previous episode, Florian Klaempfl said:

Show quoted text 7 lines
Ivo Steinmann schrieb: > no problem, > size_t stuff reverted (was a commit around r11752) > > I maintain my translated headers on my own svn server, it's simpler for me. Writing code satisfying everybody's need is indeed not easy :)

Yup. The current situation took nearly a year (first renamings in may, finished with FPC_USE_LIBC in week between christmas and newyear)

Reply by email

Ivo Steinmann
to Marco van de Voort
Show quoted text 15 lines
In our previous episode, Florian Klaempfl said: > Ivo Steinmann schrieb: > >> no problem, >> size_t stuff reverted (was a commit around r11752) >> >> I maintain my translated headers on my own svn server, it's simpler for me. >> > Writing code satisfying everybody's need is indeed not easy :) > Yup. The current situation took nearly a year (first renamings in may, finished with FPC_USE_LIBC in week between christmas and newyear)

yes, and that's exactly the code looks like. not clearly structured (at least concerning ctypes.inc and ptypes.inc and the alias include files required for keeping it consistent). my changes are so small. for the users itself there's almost no change, except that they have to use UnixType unit also now, but that could be avoided by merging unixtype with baseunix and then dropping unixtype. If really somebody uses unixtype, they simply remove it from uses clause. in 99% of all cases unixtype is used together with baseunix.

if somebody want to use a c-type, then include ctypes into uses. in current situation you can simply include baseunix, then you have got your ctypes also. but then be consequent and add ctypes to windows unit also (and other oses). now for one os you have to use ctypes, for the other not -- really nice one.

same for unixtype, baseunix and unix... all these 3 units define the same types (while unixtype is the master, and baseunix and unix uses an alias include to it). what's the idea having 3 units defining the same types? why not making a clear situation and say:
1. unixtype defines all unixtypes and baseunix some base functionalty or
2. let's drop unixtype and therefore baseunix defines all base types and functionality.

for me it simply looks messy and dont tell me this was planned from beginning. nobody plans such a code structure. there were changes over a long time and this was the result.

BTW: my branch is linux only so far, for other oses an update of Makefile.fpc and some small changes in units are required.

-Ivo

Reply by email

Florian Klaempfl
to Ivo Steinmann
Show quoted text 5 lines
if somebody want to use a c-type, then include ctypes into uses. in current situation you can simply include baseunix, then you have got your ctypes also. but then be consequent and add ctypes to windows unit also (and other oses). now for one os you have to use ctypes, for the other not -- really nice one.

Not all OSes are as C centric as unix.

Reply by email

Marco van de Voort
to Ivo Steinmann: yes, and that's exactly the code looks like. not clearly structured (at least concerning…

In our previous episode, Ivo Steinmann said:

Show quoted text 8 lines
>> > > Yup. The current situation took nearly a year (first renamings in may, > finished with FPC_USE_LIBC in week between christmas and newyear) > yes, and that's exactly the code looks like. not clearly structured (at least concerning ctypes.inc and ptypes.inc and the alias include files required for keeping it consistent).

Sure. You won't be hearing from me that the situation is perfect. It had to keep compiling for two compiler versions during this transition. But a lot of that has been cleaned up. Ales and I still clean up little bits from time to time.

My point is more that there is more than meets the eye, and you have to be very careful, and don't expect instant gratification of a major restructure.

To start with, you need to abandon the one target only approach. To properly get this done you need to review several targets at once, since the situations are not plain copies of eachother.

Also, keep notes. We wouldn't have this conversation if I had, and I lost some of the correspondence with Michael about the design when my machine got stolen.

my changes are so small. for the users itself there's almost no change, except that they have to use UnixType unit also now, but that could be avoided by merging unixtype with baseunix and then dropping unixtype.

No, since unixtype afaik exports more than baseunix and would polute the namespace. There might also be circular dependancies involved. Fixing that will improve the structure, but lead to more code duplication and includefiles.

If really somebody uses unixtype, they simply remove it from uses clause. in 99% of all cases unixtype is used together with baseunix.

IIRC there were several reasons for unixtype

- Only one place where the .inc files were important to avoid type incompatabilities (e.g. when overloading)
- most importantly: platform dependant unit might use unixtypes BELOW baseunix level, like bsd/sysctl (which is used by baseunix), maybe syscall too. IOW system dependant units needed to implement baseunix. Did you btw have a look at the unixrtl doc? The (admitted butt ugly) graph demonstrates this. (http://www.stack.nl/~marcov/unixrtl.pdf). These were inclusions before but these led to said type incompats.
- I can vaguely remember this also allows strict control over the identifiers that baseunix/unix export. Some of the more platform specific constants in ptypes are not exported through the strictly portable baseunix interface. Keep an eye on what exports what!
- to avoid all users having to import

if somebody want to use a c-type, then include ctypes into uses.

Ah, but I also use the ctypes in the implementation of the system unit. That is the trick of the current situation. The includes are used twice, but only once in the interface.

current situation you can simply include baseunix, then you have got your ctypes also. but then be consequent and add ctypes to windows unit also (and other oses). now for one os you have to use ctypes, for the other not -- really nice one.

Because Windows doesn't have the concept of a system C compiler. Moreover due to the LLP64 choice, the 32-bit/64-bit types scale differently there.

(long is 32-bit on windows 64-bit iirc)

same for unixtype, baseunix and unix... all these 3 units define the same types (while unixtype is the master, and baseunix and unix uses an alias include to it). what's the idea having 3 units defining the same types? why not making a clear situation and say:

See above. A little bit of ugliness, but it hides a lot of code duplicatio n (for each target again), while maintaining type compability. If it riles you so much to import ctypes, you could investigate the ctypes types to baseunix aliases maybe.

1. unixtype defines all unixtypes and baseunix some base functionalty or 2. let's drop unixtype and therefore baseunix defines all base types and functionality.

That's not possible.

Reply by email

Ivo Steinmann
to Marco van de Voort
Show quoted text 21 lines
In our previous episode, Ivo Steinmann said: >>> >>> >> Yup. The current situation took nearly a year (first renamings in may, >> finished with FPC_USE_LIBC in week between christmas and newyear) >> >> > yes, and that's exactly the code looks like. not clearly structured (at > least concerning ctypes.inc and ptypes.inc and the alias include files > required for keeping it consistent). > Sure. You won't be hearing from me that the situation is perfect. It had to keep compiling for two compiler versions during this transition. But a lot of that has been cleaned up. Ales and I still clean up little bits from time to time. My point is more that there is more than meets the eye, and you have to be very careful, and don't expect instant gratification of a major restructure.

it's not that major restructure as it looks in the first shot. it's a very careful step.

Show quoted text 21 lines
To start with, you need to abandon the one target only approach. To properly get this done you need to review several targets at once, since the situations are not plain copies of eachother. Also, keep notes. We wouldn't have this conversation if I had, and I lost some of the correspondence with Michael about the design when my machine got stolen. > my changes are so small. for the > users itself there's almost no change, except that they have to use > UnixType unit also now, but that could be avoided by merging unixtype > with baseunix and then dropping unixtype. > No, since unixtype afaik exports more than baseunix and would polute the namespace. There might also be circular dependancies involved. Fixing that will improve the structure, but lead to more code duplication and includefiles.

true, unixtype exports more than baseunix. in fact, unixtype exports the content of ptypes.inc (platform dependant) in it's interface. most of those types are reimported into interface part of baseunix and unix over aliasptp.inc (platform independant).

Show quoted text 21 lines
> If really somebody uses unixtype, they simply remove it from uses clause. > in 99% of all cases unixtype is used together with baseunix. > IIRC there were several reasons for unixtype - Only one place where the .inc files were important to avoid type incompatabilities (e.g. when overloading) - most importantly: platform dependant unit might use unixtypes BELOW baseunix level, like bsd/sysctl (which is used by baseunix), maybe syscall too. IOW system dependant units needed to implement baseunix. Did you btw have a look at the unixrtl doc? The (admitted butt ugly) graph demonstrates this. (http://www.stack.nl/~marcov/unixrtl.pdf). These were inclusions before but these led to said type incompats. - I can vaguely remember this also allows strict control over the identifiers that baseunix/unix export. Some of the more platform specific constants in ptypes are not exported through the strictly portable baseunix interface. Keep an eye on what exports what! - to avoid all users having to import

yes, ptypes.inc are included only by UnixType and the system unit, while ctypes.inc is included by ptypes.inc. And that's exactly what I changed:

current situation:
ctypes.inc: all c types
aliasctp.inc: mapping of all ctypes defined in unixtype (over ptypes.inc) to locale scope
ptypes.inc: all unix platfrom (dependant) unix types. it also includes ctypes.inc
aliasptp.inc: mapping of all unix platfrom independant types + all c-types defined in UnixType to locale scope
unixtype: all unix platfrom (dpendant) unix types + all c-types baseunix: includes aliasptp.inc
unix: includes aliasptp.inc

1. I removed {$i ctypes.inc} from each ptypes.inc
2. I removed {$i aliasctp.inc} from ctypes.pp and aliasptp.inc and added {$i ctypes.inc} instead
3. I added ctypes in uses clause of unixtype, baseunix and unix
4. I added ctypes.inc to system unit (because it's no longer included of ptypes.inc)

situation after changes:
ctypes.inc: all c types
aliasctp.inc: removed
ctypes.pp: includes ctypes.inc
unixtype: all unix platfrom (dependant) unix types

and that maybe not good, as you explained above:
aliasptp.inc: removed
baseunix: base funtionality with no types (because of removed aliasptp.inc)

if you keep aliasptp.inc, then only ctypes are outsourced from unixtype, baseunix and unix.

Show quoted text 8 lines
> if somebody want to use a c-type, then include ctypes into uses. > Ah, but I also use the ctypes in the implementation of the system unit. That is the trick of the current situation. The includes are used twice, but only once in the interface.

yes

Show quoted text 12 lines
> current situation you can simply include baseunix, then you have got > your ctypes also. but then be consequent and add ctypes to windows unit > also (and other oses). now for one os you have to use ctypes, for the > other not -- really nice one. > Because Windows doesn't have the concept of a system C compiler. Moreover due to the LLP64 choice, the 32-bit/64-bit types scale differently there. (long is 32-bit on windows 64-bit iirc)

k

-Ivo

Reply by email

Daniël Mantione
to Ivo Steinmann: yes, and that's exactly the code looks like. not clearly structured (at least concerning…

Op Mon, 15 Sep 2008, schreef Ivo Steinmann:

my changes are so small. for the users itself there's almost no change, except that they have to use UnixType unit also now

Compatibility is a big issue. When breaking it this means about most code using the Unix units will suddenly no longer compile.

It would also create confusion with old documentation still in circulation.

I agree with the technical merit of your changes, but I haven't heard a strong argument for the need to break compatibility. A unit that becomes a little bit more elegant is too weak for the hassle that it causes.

Daniël

Reply by email

Marco van de Voort
to Ivo Steinmann: it's not that major restructure as it looks in the first shot. it's a very careful step.…

In our previous episode, Ivo Steinmann said:

true, unixtype exports more than baseunix. in fact, unixtype exports the content of ptypes.inc (platform dependant) in it's interface. most of those types are reimported into interface part of baseunix and unix over aliasptp.inc (platform independant).

And e.g. On FreeBSD sysctl imports unixtypes too, and baseunix imports sysctl. IOW, you can't have sysctl import baseunix for the types.

Show quoted text 21 lines
yes, ptypes.inc are included only by UnixType and the system unit, while ctypes.inc is included by ptypes.inc. And that's exactly what I changed: current situation: ctypes.inc: all c types aliasctp.inc: mapping of all ctypes defined in unixtype (over ptypes.inc) to locale scope ptypes.inc: all unix platfrom (dependant) unix types. it also includes ctypes.inc aliasptp.inc: mapping of all unix platfrom independant types + all c-types defined in UnixType to locale scope unixtype: all unix platfrom (dpendant) unix types + all c-types baseunix: includes aliasptp.inc unix: includes aliasptp.inc 1. I removed {$i ctypes.inc} from each ptypes.inc 2. I removed {$i aliasctp.inc} from ctypes.pp and aliasptp.inc and added {$i ctypes.inc} instead 3. I added ctypes in uses clause of unixtype, baseunix and unix 4. I added ctypes.inc to system unit (because it's no longer included of ptypes.inc)

(in the implemntation I hope)

Show quoted text 5 lines
situation after changes: ctypes.inc: all c types aliasctp.inc: removed ctypes.pp: includes ctypes.inc unixtype: all unix platfrom (dependant) unix types

But unixtype and baseunix doesn't export the ctypes anymore, which breaks compatibility.

A beginner program like

var c:cint;
begin
  <somebaseunixfunc>(c);

will nead adaptation.

Reply by email