Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2019-07

Implementation of variant records

10 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Bernd Oppolzer

This question is not directly related to FPC, but it is instead a Pascal question in general.

Take this record definition:

type S1 = record
             X : CHAR ( 27 ) ;
             case INTEGER of
               1 :
                 ( V : INTEGER ) ;
               2 :
                 ( W : REAL ) ;
               3 :
                 ( Z : CHAR ( 8 ) ) ;
           end ;

CHAR (n) is an abbreviation for /* packed */ array [1..n] of char which is accepted by a certain compiler.

Now the question:

X needs 27 bytes, so the variant part of the record starts at offset 27. But on a certain platform V has alignment 4 (integer), W has alignment 8 (real).
The compiler decided to put V on offset 28, W on offset 32 and Z (no alignment)
on offset 27, so the three variants don't overlay properly.

IMO, the variants in a variant record should always overlay correctly (like unions in C),
so the variant part should start at offset 32 in this case, and this is where all three
variants should start.

In other words: every record (in classical Pascal) consists of a fixed part and a variant
part and a fixed border between the two (that's how it should be, IMO). And this border
takes its alignment properties from the highest alignment of the fields in the variant part.

What do you think?

Why is this important to me? Because I am working on a WITH extension for record definitions,
which allows the integration of the components from other record and pointer types into the
current record type namespace, and this has to be fixed, before I can do this.

BTW: is there a forum to discuss general Pascal questions not directly related to the FPC product?
FPC-Pascal will sure be the largest auditorium available for such questions ...

Thank you, kind regards

Bernd

Tomas Hajny
to Bernd Oppolzer
Bernd Oppolzer wrote
This question is not directly related to FPC, but it is instead a Pascal question in general.

.
.

BTW: is there a forum to discuss general Pascal questions not directly related to the FPC product? FPC-Pascal will sure be the largest auditorium available for such questions ...

Obviously, fpc-other is the right list unless your question is directly applicable to FPC.

Tomas

Marco van de Voort
to Bernd Oppolzer: This question is not directly related to FPC, but it is instead a Pascal question in…

Op 2019-07-05 om 11:49 schreef Bernd Oppolzer:

Show quoted text 6 lines
IMO, the variants in a variant record should always overlay correctly (like unions in C), so the variant part should start at offset 32 in this case, and this is where all three variants should start.

This is not a guarantee case in the Pascal language, afaik many old compilers don't even try, but make fields sequential.

As soon as you have expections of overlaying, you are dialect and architecture specific.

Ralf Quint
to Marco van de Voort
Marco van de Voort wrote 14 lines
Op 2019-07-05 om 11:49 schreef Bernd Oppolzer: > > IMO, the variants in a variant record should always overlay correctly > (like unions in C), > so the variant part should start at offset 32 in this case, and this > is where all three > variants should start. This is not a guarantee case in the Pascal language, afaik many old compilers don't even try, but make fields sequential. As soon as you have expections of overlaying, you are dialect and architecture specific.

Shouldn't a PACKED Record guarantee that values are aligned at the byte level?

Ralf

---
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Marco van de Voort
to Ralf Quint

Op 2019-07-05 om 13:53 schreef Ralf Quint:

Show quoted text 16 lines
>> IMO, the variants in a variant record should always overlay >> correctly (like unions in C), >> so the variant part should start at offset 32 in this case, and this >> is where all three >> variants should start. > > This is not a guarantee case in the Pascal language, afaik many old > compilers don't even try, but make fields sequential. > > As soon as you have expections of overlaying, you are dialect and > architecture specific. Shouldn't a PACKED Record guarantee that values are aligned at the byte level?

In Borland and x86 pascals in general that is somewhat the norm.

But the original Pascal architecture was not byte, but word addressable, and "packed' there meant packing several fields into one word.  Access to packed fields meant loading the word, and shifting the values out.

From https://en.wikipedia.org/wiki/CDC_6600

---

The central processor had 60-bit <https://en.wikipedia.org/wiki/60-bit> words, while the peripheral processors had 12-bit
<https://en.wikipedia.org/wiki/12-bit> words. CDC used the term "byte" to refer to 12-bit entities used by peripheral processors; characters were 6-bit, and central processor instructions were either 15 bits, or 30 bits with a signed 18-bit address field, the latter allowing for a directly addressable memory space of 128K words of central memory (converted to modern terms, with 8-bit bytes, this is 0.94 MB).

----

Strings that were packed had 10 6-bit chars in a word etc. But for some array operations, operating on unpacked data (IOW one element per word) was easier. There were standard procedures (builtins?) PACK and UNPACK to convert between packed and unpacked arrays/records.

Packing was explicit and was meant as a generalized solution to counter the memory waste of such schemes (since one 6-bit char/60bits word is quite wastful), not to govern the memory layout precisely, that was architecture/implementation dependent. The architectures varied to wildly for that.

Other, newer, processors were less funky, but sometimes still had limitations. I can still remember the initial powerpc port having a problem on some (603) processors that couldn't load floating point values from unaligned addresses. (lfd/stfd or so)

And of course Alpha (older multias only?) that had exceptions on unaligned access iirc.

Ralf Quint
to Marco van de Voort
Marco van de Voort wrote 58 lines
Op 2019-07-05 om 13:53 schreef Ralf Quint: > >>> IMO, the variants in a variant record should always overlay >>> correctly (like unions in C), >>> so the variant part should start at offset 32 in this case, and >>> this is where all three >>> variants should start. >> >> This is not a guarantee case in the Pascal language, afaik many old >> compilers don't even try, but make fields sequential. >> >> As soon as you have expections of overlaying, you are dialect and >> architecture specific. > > Shouldn't a PACKED Record guarantee that values are aligned at the > byte level? > In Borland and x86 pascals in general that is somewhat the norm. But the original Pascal architecture was not byte, but word addressable, and "packed' there meant packing several fields into one word.  Access to packed fields meant loading the word, and shifting the values out. From https://en.wikipedia.org/wiki/CDC_6600 --- The central processor had 60-bit <https://en.wikipedia.org/wiki/60-bit> words, while the peripheral processors had 12-bit <https://en.wikipedia.org/wiki/12-bit> words. CDC used the term "byte" to refer to 12-bit entities used by peripheral processors; characters were 6-bit, and central processor instructions were either 15 bits, or 30 bits with a signed 18-bit address field, the latter allowing for a directly addressable memory space of 128K words of central memory (converted to modern terms, with 8-bit bytes, this is 0.94 MB). ---- Strings that were packed had 10 6-bit chars in a word etc. But for some array operations, operating on unpacked data (IOW one element per word) was easier. There were standard procedures (builtins?) PACK and UNPACK to convert between packed and unpacked arrays/records. Packing was explicit and was meant as a generalized solution to counter the memory waste of such schemes (since one 6-bit char/60bits word is quite wastful), not to govern the memory layout precisely, that was architecture/implementation dependent. The architectures varied to wildly for that. Other, newer, processors were less funky, but sometimes still had limitations. I can still remember the initial powerpc port having a problem on some (603) processors that couldn't load floating point values from unaligned addresses. (lfd/stfd or so) And of course Alpha (older multias only?) that had exceptions on unaligned access iirc.

That's rather odd. I have done a lot of data conversion between different architectures, always using Pascal, and can't recall where an explicit "packed" record element was not aligned at a byte level, and this was IIRC introduced with UCSD Pascal, which was based  on a 16bit p-code machine, regardless of the underlying CPU, so your CDC reference doesn't directly apply.

And the concept of a "packed record" (as coming from UCSD Pascal) should be different from the explicit PACK/UNPACK from ISO7165, which can also be applied arrays and sets, which I think is what you are likely referring to...

The only time I had to deal with PACK/UNPACK was on some HP (fka Compaq fka DEC) Pascal for (Open)VMS, and a packed record would work on a byte level, unless sets smaller than a byte where used, with the whole record then padded in memory (but not on storage) to the necessary alignment/register width.

Can't find my old Metrowerks CodeWarrior manuals right now, that is the only time I have worked with a (non-UCSD) Pascal on a PPC (and 68k) processor to check what how that was defined in there...

Ralf

---
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Florian Klaempfl
to Ralf Quint: Shouldn't a PACKED Record guarantee that values are aligned at the byte level? Ralf ---…
Ralf Quint wrote 18 lines
On 7/5/2019 3:27 AM, Marco van de Voort wrote: > > Op 2019-07-05 om 11:49 schreef Bernd Oppolzer: >> >> IMO, the variants in a variant record should always overlay correctly >> (like unions in C), >> so the variant part should start at offset 32 in this case, and this >> is where all three >> variants should start. > > This is not a guarantee case in the Pascal language, afaik many old > compilers don't even try, but make fields sequential. > > As soon as you have expections of overlaying, you are dialect and > architecture specific. Shouldn't a PACKED Record guarantee that values are aligned at the byte level?

It does in TP, but the ISO says only that it shall be economised.

Ralf Quint
to Florian Klaempfl
Florian Klaempfl wrote 5 lines
Am 05.07.2019 um 13:53 schrieb Ralf Quint: > Shouldn't a PACKED Record guarantee that values are aligned at the byte > level? It does in TP, but the ISO says only that it shall be economised.

Well, the ISO is probably what has done most of the damage to Pascal. AFAIK, the ISO doesn't not actually specify any "packed record" per se, but there is an explicit PACK and UNPACK procedures to more economically store sets and arrays (and possibly records, I would have to look up the ISO text to be certain). For example this would be something to create bit fields, pretty much the only feature of C that I sometimes miss in (Free)Pascal.

For me, the use of packed records is a feature that facilitates the easy exchange of stored data between architectures, something that a lot of people these days with their mania about 64bit this and 64bit that are often forgetting or chose to ignore...

Ralf

---
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Sven Barth
to Ralf Quint
Ralf Quint wrote 13 lines
On 7/6/2019 12:21 PM, Florian Klaempfl wrote: > Am 05.07.2019 um 13:53 schrieb Ralf Quint: >> Shouldn't a PACKED Record guarantee that values are aligned at the byte >> level? > It does in TP, but the ISO says only that it shall be economised. > Well, the ISO is probably what has done most of the damage to Pascal. AFAIK, the ISO doesn't not actually specify any "packed record" per se, but there is an explicit PACK and UNPACK procedures to more economically store sets and arrays (and possibly records, I would have to look up the ISO text to be certain). For example this would be something to create bit fields, pretty much the only feature of C that I sometimes miss in (Free)Pascal.

You mean like this?

=== code begin ===

struct Test
{
         uint8_t field1;
         uint16_t bfield1 : 1;
         uint16_t bfield2 : 1;
         uint16_t bfield3 : 3;
         uint16_t bfield4 : 5;
         uint16_t bfield5 : 1;
         uint16_t field2;
};

=== code end ===

=== code begin ===

type
   TTest = bitpacked record
     field1: UInt8;
     bfield1: 0..1;
     bfield2: 0..1;
     bfield3: 0..7;
     bfield4: 0..31;
     bfield5: 0..1;
     pad: UInt8; // needed due to bitpacked
     field2: UInt16;
   end;

=== code end ===

The size of both records is 6. :)

Regards,
Sven

Sven Barth
to Sven Barth
Sven Barth wrote 47 lines
Am 07.07.2019 um 02:55 schrieb Ralf Quint: > On 7/6/2019 12:21 PM, Florian Klaempfl wrote: >> Am 05.07.2019 um 13:53 schrieb Ralf Quint: >>> Shouldn't a PACKED Record guarantee that values are aligned at the >>> byte >>> level? >> It does in TP, but the ISO says only that it shall be economised. >> > Well, the ISO is probably what has done most of the damage to Pascal. > AFAIK, the ISO doesn't not actually specify any "packed record" per > se, but there is an explicit PACK and UNPACK procedures to more > economically store sets and arrays (and possibly records, I would > have to look up the ISO text to be certain). For example this would > be something to create bit fields, pretty much the only feature of C > that I sometimes miss in (Free)Pascal. You mean like this? === code begin === struct Test {         uint8_t field1;         uint16_t bfield1 : 1;         uint16_t bfield2 : 1;         uint16_t bfield3 : 3;         uint16_t bfield4 : 5;         uint16_t bfield5 : 1;         uint16_t field2; }; === code end === === code begin === type   TTest = bitpacked record     field1: UInt8;     bfield1: 0..1;     bfield2: 0..1;     bfield3: 0..7;     bfield4: 0..31;     bfield5: 0..1;     pad: UInt8; // needed due to bitpacked     field2: UInt16;   end; === code end ===

Small correction: there needs to be another "bpad: 0..31;" field between bfield5 and pad, cause otherwise the pad field will be directly behind the bfield5 one. This could of course be avoided by putting the bit fields into a subrecord, though then of course there'd be the need for an additional field name.

Regards,
Sven