I started investigating huge pointer support in 16-bit C compilers and discovered there are at least two different implementations:

Borland C++ 3.1:

Huge pointers aren't changed when converting from a far pointer. They are normalized as soon as some pointer arithmetic is performed on them. Normalization means changing the segment and the offset in such a way that it still points to the same linear address ( =segment*16+offset ), but the offset is always between $0000 and $000F. So:

HugePointer(Ptr($1234,$5678)) = $1234:$5678
HugePointer(Ptr($1234,$5678))+1 = $179B:$0009
HugePointer(Ptr($1234,$5678))-1 = $179B:$0007

Microsoft C/C++ 7.0 and Open Watcom 1.9:

Huge pointers aren't changed when converting from a far pointer (same as Borland). However, during pointer arithmetic, they aren't normalized like in Borland, but the segment is changed only when the offset overflows, i.e.:

HugePointer(Ptr($1234,$5678)) = $1234:$5678
HugePointer(Ptr($1234,$5678))+1 = $1234:$5679
HugePointer(Ptr($1234,$5678))-1 = $1234:$5677
HugePointer(Ptr($1234,$FFFF))+1 = $2234:$0000
HugePointer(Ptr($1234,$0000))-1 = $0234:$FFFF

Back in the old days, I never used huge pointers, because I wrote in Turbo Pascal, which didn't have them, so I don't have an opinion which way is better. So, which way should FPC for i8086 follow?

Nikolay

Sunday, April 6, 2014, Nikolay Nikolov wrote:

NN> Back in the old days, I never used huge pointers, because I wrote in Turbo Pascal, NN> which didn't have them, so I don't have an opinion which way is better. NN> So, which way should FPC for i8086 follow?

BP7 does not normalize pointers, since it targets DPMI16, where such normalization is senseless. Moreover, it uses special typed const "system.SelectorInc" to handle offset overflows. Under real mode, SelectorInc is barely $1000.
Under DPMI16 environment SelectorInc can be obtained by DPMI function INT31h/AX=0003h (usually, it returns 8). With this approach the whole pointer arithmetics is similar between real and protected modes.

---
WBR, Max.

Ok, I just added this constant to the msdos rtl (r27497). However, as far as BP7 compatibility is concerned, it doesn't really matter which huge pointer model we use, because BP7 doesn't have a huge pointer type and in fact, it doesn't even offer any kind of pointer arithmetic as far as I know. But it's a good point that the pointer normalization, used in Borland C works in real mode only. This means that if we follow that way, huge pointers must be either entirely disallowed or behave differently (i.e. like Microsoft and Watcom C, but by using System.SelectorInc) in 16-bit protected mode. However, the Borland C way, IMHO, does have some really nice advantages:

- If you pass a huge pointer to a function or interrupt routine that expects only a far (not huge) pointer, it'll always be able to access at least the next (64K-15) bytes of the object
- Free Pascal often generates code using [base + index + offset] references to access record/object/class fields or array elements. This will work nicely with normalized huge pointers, as long as the structure itself does not exceed 64kb (which is currently not allowed in the compiler). With MS/Watcom-style huge pointers, you could end up, after some pointer arithmetic at Ptr($1234:$FFFF) and then when you dereference the pointer and try to access some record member, you're screwed.

Nikolay

On Mon, April 7, 2014 10:56, Nikolay Nikolov wrote: .
.

.
.

BP7 already provided pointer arithmetics for PChars.

Tomas

You're right, I didn't know that. However, I just checked and it only increments/decrements the offset, so it's normal far pointer arithmetic, not huge pointer arithmetic.

Nikolay

Monday, April 7, 2014, Nikolay Nikolov wrote:

NN> Back in the old days, I never used huge pointers, because I wrote in NN> Turbo Pascal, which didn't have them, so I don't have an opinion which NN> way is better.

I've rechecked -- and yes, You are right, BP7 RTL itself uses SelectorInc only in DPMI16 environment for locating writable alias for the current code segment (it's just an internal borland EXE loader convention to create such aliases). For all other modes this variable just maintained [and it works, if you manually use it in, e.g., Win16 for accessing huge data block allocated with Win16 API, either in real or protected mode, since Win16 kernel exports constant "__AHIncr", which is exactly the same thing as SelectorInc, and is used by BP7 RTL for SelectorInc initialization].

For "real mode" normalization of (typed?) huge pointer probably could be done during converting it to regular far pointer. But for non-"real mode" segmented model access to the data structure crossing segment boundary always a challenge ending up with "stream it out to the safe place at first", if the structure is relatively small.

---
WBR, Max.

Hmm, then maybe we should support both and add a
{$HUGEPOINTERNORMALIZATION} directive?

Nikolay

Nikolay Nikolov wrote

Hmm, then maybe we should support both and add a {$HUGEPOINTERNORMALIZATION} directive?

Well, I don't know for sure..
But I think introducing huge pointers implies introduction of some minimal support routines, like [h]move(hugefrom,hugeto,hugelen), [h]fill(..), etc. Implementation of such routines is quite different among "normalized real" and "unnormalized common" modes, so going clean way sadly will require twice effort (and a quadruple maintenance, I suspect.. );

---
WBR, Max.

Yes, huge pointers will need RTL helper routines, but IMHO, not that many and they would be relatively simple. I think only pointer arithmetic will need helpers (i.e. adding a pointer with an integer and pointer difference). As for the other routines in the RTL like (h)FillChar and (h)Move, they will have to be made to work with unnormalized huge pointers (in order to support 16-bit protected mode) and this way they will automatically work with normalized huge pointers as well.

Nikolay

Yes, it looks feasible for now.. (:

---
WBR, Max.