Can freepascal be used on small processors that are 8 or 16 bits or does it expect that there are at least 32 bits for most things? I've been tempted lately to start making little circuits with microcontrollers on them. I realize that a great deal of the rtl/fcl would not be used on these little processors. Also since there is no OS on these devices does that mean that for sure fpc cannot work?

Regards,

Andrew Haines

A 32 bit microcontroller:
http://www.microcontroller.com/news/microchip_pic32.asp

Most of the RTl is written for 32- and 64-bit computing, as you correctly state. I would imagine that _some_ of it would work on 16-bit, and maybe 1-2% would work on 8-bit ;-)

However, you can choose to compile fpc without the system unit, using a special option on the command-line. This would enable you to code your own system routines (such as writeln), most likely in assembler.

However, the most obvious problem here is that (AFAIK) fpc does not support arbitrary 8- and 16-bit targets. You need to find an architecture that fpc supports, so you can compile programs for it.

Other people on this list probably know more about embedded systems than I do, so I hope someone else can answer with the specifics.

Andrew Haines wrote

Can freepascal be used on small processors that are 8 or 16 bits or does it expect that there are at least 32 bits for most things?

FPC only supports 32 and 64bits platforms. I recommend using commercial Pascal compilers for 16 bits / 8 bits. You may even be able to find some free software ones, but I'd inspect their quality first, because they may be unsuitable for commercial projects.

I've been tempted lately to start making little circuits with microcontrollers on them. I realize that a great deal of the rtl/fcl would not be used on these little processors.

If you want to use FPC then I'd recomend you to use a ARM board. If it's too expensive for your product, then you can choose a cheap microcrontroller.

It's trivial to find commercial Pascal compilers for microcontrolers, like 8051:

http://www.google.com/search?q=8051+pascal+compiler&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:pt-BR:official&client=firefox-a

Or PIC:

http://www.google.com/search?q=pic+pascal+compiler&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:pt-BR:official&client=firefox-a

If you still don't know which microcontroler to use I'd go for one of those options. PIC is by far the cheapest of all of them. The programming hardward cost is incredibly cheaper then for other ones. PIC is very popular one for cheap / simple projects, and I've been using it in project with great success. It also has a nice forum to ask questions, which was quite important for me.

If you want something a little better, then 8051 is very popular in the industry is various fields from automotive to CNC machines.

If you want even more power, then go for a ARM. There are even some embedded x86 microprocessors, but they are expensive if you are building something simple and cheap.

Also since there is no OS on these devices does that mean that for sure fpc cannot work?

I think this doesn't make any difference. The only thing is that if there is no OS then you also have no RTL, i.e. if you want one you have to write your own. Commercial compilers should provide a small RTL.

Christian Iversen wrote

However, the most obvious problem here is that (AFAIK) fpc does not support arbitrary 8- and 16-bit targets. You need to find an architecture that fpc supports, so you can compile programs for it.

Florian is working on an AVR port (http://en.wikipedia.org/wiki/Atmel_AVR ).

Jonas

IMHO, nowadays, 8 (and 16) bit processors only make sense for very small projects. Those are very hardware specific and thus using C here does make sense.

OTOH, 32 bit processors have become very cheap and handy. For embedded projects, IMHO, especially processors that are programmed as "ip-Cores" in FPGAs are very interesting, so that porting the FP compiler could make sense. I am considering to do this port for the NIOS CPU (that runs in Altera FPGAs). I don't intend to start that project very soon, but if someone is interested in starting the port, I of course will try to help.

-Michael

This is very blue-sky, but it would be an interesting enhancement to Lazarus to target ARM or Atmel AVR CPUs in the manner of the Processing language/IDE:

http://en.wikipedia.org/wiki/Processing_%28programming_language%29

It's how you create software for the Arduino boards that are so popular here in the US. I started out in embedded processors, and I can program in C-like languages, but they make me itch. Pascal would be better.

--73--

--JD--

Jeff Duntemann wrote

This is very blue-sky,

FPGA processors are used a lot in embedded projects (though of course usually with programming in C). As well Altera as Xilinx have µCLinux for their processors (full Linux to come very soon) These are the "primary" FPGA companies, but there are several other, too, that also have 32 bit processors (and Linux) for their chips.

Because of the very versatile usability of such projects (and dropping hardware prices) they increasingly gain acceptance in the market. Of course there are lots of SDKs, OEM / prototyping boards etc to play with, so you don't need to build any hardware for starting.

but it would be an interesting enhancement to Lazarus to target ARM or Atmel AVR CPUs in the manner of the Processing language/IDE:

Lazarus already can create ARM programs (as well for Linux as for WinCE). I did a (very basic) WinCE test and my program did run as expected as of the first upload without any debugging. As the NIOS instruction set is quite similar to ARM (thus dissimilar to x86) I think it's a viable task to do a NIOS port. And _if_ starting a port for embedded use at all, I would do it on a technology that is likely do be one of the winners in that world.

-Michael

Michael Schnell wrote

IMHO, nowadays, 8 (and 16) bit processors only make sense for very small projects. Those are very hardware specific and thus using C here does make sense.

I have to disagree with this one, depending with what "small" means. If you mean "small" in project complexity, I'd say this is very far from true. BOSCH uses 8-bit microcontrolers (8051) widely inside several sensors (tire pressure sensors for example) and I'm sure I wouldn't call those projects simple considering the number of workers and years involved on them. If you mean "small" in size, then maybe =) The sensors are really small.

Also considering how much time I saw the other internship student waste trying to understand the criptic C code for the microcontroler I'd also disagree that using C is necessarely the best choice.

Of course you are right. By small I meant the size of the task the processor is supposed to do and thus to some extend the estimated effort needed for programming.

If you plan a device that is to be sold in huge quantities, you select the cheapest possible processor whatsoever, completely disregarding the programming effort. I suppose here using Pascal is never considered anyway.

-Michael

Felipe Monteiro de Carvalho ha scritto:

If Pascal, or another decent language is available, C is *never* the best choice. :-)

Giuliano

If Pascal, or another decent language is available, C is *never* the best choice. :-)

Since with FP it is possible to use a preprocessor this is kind of true, as same can be use to cure shortcomings of the language used which might be problematic especially in embedded projects.

-Michael

Could you give use case examples why a preprocessor is mandatory for an embedded language?

You can avoid cryptic language constructs in the main source code.

I did not yet use the preprocessor in Pascal but a standard way to access memory mapped ports in C is something like this (from the top of my head):

#define _port = 0x87

#define ByteIO(p) *(((volatile unsigned char)*)(p))

#define port=ByteIO(_port)

now in the main code you go

 x = ByteIO(_port);

or just

 x = port;
 ByteIO(_port) = 0x55;

may look queer to some watchers, but

 port = 0x55;

should be nice.

-Michael

const _port=$87;

#define ByteIO(p) *(((volatile unsigned char)*)(p))

A bit cryptic, but I guess this is

function ByteIO(var p):puchar; inline;
begin
  result:=puchar(@p);
end;

#define port=ByteIO(_port)

const port = puchar(_port); // !??!?

port^:=$55;

Op Wed, 27 Feb 2008, schreef Marco van de Voort:

Nope, don't forget that we have a nice arsenal of constructs to do system programming, like the mem array, the port array, absolute etc. The above can be replaced by:

mem[_port]:=$55;

Daniël

I know, but I thouht it was mainly about the typing. Most importantly that you can access this bitwise using e.g. ptrs to bitpacked arrays. One sees this a lot in embedded C too.

Michael Schnell wrote

You can avoid cryptic language constructs in the main source code.

Or you can avoid cryptic languages altogether. :D

-- 8< -- snip --

with Interfaces;
with System.Storage_Elements;

procedure Foo is

You mean like ADA? Indeed, I agree :P

Micha

No, I meant languages which are not able to give me bit level control over a type and have to circumvent this restriction with either awkward "macro magic" or even can't do I mean to set the write buffer water mark of the SDRAM controller register, I'd rather write:

|SDRAM_Buffer_Control.WB_Watermark := Sixteen;

instead of

|SDRAMBufferControl := (SDRAMBufferControl and not WB_WATERMARK_MASK) or
|                      WB_WATER_MARK_16;

Not even to think of the much more cryptic C version.

Vinzent.

I don't see the point: pascal and also fpc know bitpacked records?

Op Wed, 27 Feb 2008, schreef Vinzent Hoefler:

Then use bitpacked records:

type TSDRAM_Buffer_Control=bitpacked record
{...}

       end;

var SDRAM_Buffer_Control:TSDRAM_Buffer_Control absolute [$xxxxxx];
begin
   {...}
   SDRAM_Buffer_Control.WB_Watermark := Sixteen;
   {...}

Just like Micha I'm not impressed about the Ada code, and it is off-topic, this is and remains a Pascal newsgroup. Comparisons between Pascal and other languages are on-topic, comparisons between C and Ada are off-topic.

Daniël

Well, TP didn't, and until now I found no real documentation on it (it sure wasn't mentioned in *my* copy of the programmer's manual). If it's similar to C-records then it's unusable anyway, as there are no guarantees. Not about endianess, not about padding. And what about volatile and/or atomic semantics?

Well, too many boil it down to this one: How else do you write it? The original layout declaration:

|  for SDRAM_Buffer_Control use
|   record
|      WB           at 0 range 0 .. 0;
|      WB_Flush     at 0 range 1 .. 1;
|      WB_Watermark at 0 range 2 .. 3;
|      Read_Ahead   at 0 range 4 .. 4;
|      --  bits [5:7] are reserved
|   end record;

It get's especially tricky, because in many cases the bit patterns don't fit the numbers:

|  type RAS_CAS_Delay is range 2 .. 4;
|  for  RAS_CAS_Delay'Size use 2;

or

|  type CAS_Latency is range 2 .. 3;
|  for  CAS_Latency'Size use 1;

I don't really want to write

|SDRAM_Timing_Control.CAS_Latency := 1; // Where 0 is 2 and 1 is 3.

Well, not that big a problem, because once the patterns get nonconsecutive, you'd need enums either way. ;)

Vinzent.

Daniël Mantione wrote

Then use bitpacked records:

If you'd document them properly?

-- 8< --

type
   TX = 1 .. 4;
   TY = 0 .. 3;
type
   Foo = bitpacked record
      A : TX;
      B : TY;
   end {record};
var
   X : Foo;
begin
   X.A := 1;
   X.B := 2;
   WriteLn (X.A);
   WriteLn (X.B);
   WriteLn (SizeOf (X));
end.

-- 8< --

Free Pascal Compiler version 2.0.0 [2005/09/09] for i386 Copyright (c) 1993-2005 by Florian Klaempfl
Target OS: Linux for i386
Compiling test.pas
test.pas(6,10) Error: Identifier not found "bitpacked"

It wasn't meant to impress anybody. It was meant to show that there's no preprocessor needed.

and it is off-topic, this is and remains a Pascal newsgroup.

Oh, I thought of it as a mailing list. ;)

Comparisons between Pascal and other languages are on-topic, comparisons between C and Ada are off-topic.

So we're on-topic again. You are comparing Pascal and Ada. I look at Ada as a more strict version of Pascal, is all. Too many Ada-features have crept into Pascal over the years. ;)

Vinzent.

Op Wed, 27 Feb 2008, schreef Vinzent Hoefler:

Well, TP didn't, and until now I found no real documentation on it (it sure wasn't mentioned in *my* copy of the programmer's manual).

That is correct, but the last version of TP appeared in 1992. Many Pascal compilers have supported bitpacking, since 2.2 we do too.

If it's similar to C-records then it's unusable anyway, as there are no guarantees. Not about endianess, not about padding.

This is simply a matter of documenting and standardizing. I.e. we can exchange packed records with TP/Delphi, what would be the problem with bitpacking? If I recall well, Jonas made our bitpacking compatible with GCC and GNU-Pascal.

And what about volatile and/or atomic semantics?

Pascal does not have volatile. It is not strictly needed either since the use of mem or absolute should provide sufficient information to the compiler it cannot optimize away reads/writes.

type  TSDRAM_Buffer_Control=bitpacked record
         WB:boolean; {Or 0..1, depending on your preference.}
         WB_Flush:boolean;
         WB_Watermark:0..3;
         Read_Ahead:boolean;
       end;

Daniël

Op Wed, 27 Feb 2008, schreef Vinzent Hoefler:

Already done afaik.

So we're on-topic again. You are comparing Pascal and Ada. I look at Ada as a more strict version of Pascal, is all.

Yes, we are on topic again. What I object to is your e-mails often look like Ada advertisements. This is only okay up to a certain point.

Comparisons with Ada are valid when designing language extensions; better copy well than do it bad yourself. On the other hand, Ada isn't perfect either.

Too many Ada-features have crept into Pascal over the years. ;)

You can look at it that way, but Ada wasn't the blueprint for Pascal extensions. Some come from C, some from Modula-2, some from Visual Basic. Bitpacking can be considered a "real" Pascal extension, early compilers in the eighties could already do it.

Daniël

I don't understand why it's not in the 2.2.0 docs, because the doc changes were committed to svn on 2007-06-29, which was before the 2.2.0 release (and there are no different branches for the documentation afaik).

"bitpacked" was introduced in version 2.2.0 (which is the latest official release).

That said, bitpacking is unsuited for hardware interfacing because it is opaque: there is guarantee whatsoever regarding the layout (just like with sets). FPC's current implementation on all platforms is mostly the same as GPC's bitpacking and structs declared in GCC using __attribute__((__packed__)), but not entirely (on some platforms and with some types, GCC nevertheless still inserts padding bits when using __attribute__((__packed__)), while we never do that currently).

The bit layout is also different on little and big endian systems.

Jonas

Daniël Mantione wrote

> Too many Ada-features have crept into Pascal over the years. ;) You can look at it that way, but Ada wasn't the blueprint for Pascal extensions.

Well, in the early days, it was quite the other way around, yes. Ada is derived (mostly) from Pascal. And other languages had their influences (and others later were influenced), too. Even computer languages have their Babel tower.

Some come from C, some from Modula-2, some from Visual Basic. Bitpacking can be considered a "real" Pascal extension, early compilers in the eighties could already do it.

Well, Ada's representation clauses also were known in 1983 already. And that was after the standardization procedure was through, so they surely appeared in the earlier 1979 drafts already. (And no objection please, but representation clauses are quite a bit more than just bitpacking.)
But as most features of Ada those probably appeared somewhere else before, I guess.

But for sure, such types (do they really call it "record" there or is that just my imagination?) appeared in C not before 1999. Together with other brain dead design decision, this makes (or at least "made") C about the worst choice for embedded, low level programming - and yet, it is the most successful.

Vinzent.

See, and that's a problem im production code. The project started at the time where 1.0.10 was current (and a couple of months later I used the 1.9.6 release candidate), but due to several show-stoppers (thread leaks, random XML parser crashes) I'm currently stuck to version 2.0.0.

Of course, I tried version 2.2.0 shortly after its release, but then the project didn't even compile anymore. Ok, my fixed up version of the tread manager implementation was expected to break, but that suddenly subroutines were moved to other units wasn't something I expected. (Yes, the Interlocked stuff is now in System where before it was in SysUtils - which generally is a good thing; when using such code, chances are high, you don't even want to pull in SysUtils - , but still it breaks my code). And there may have been other issues, I don't remember.

Of course, some day soon I'll probably do a branch to migrate to 2.2.0, but until then, sorry, "bitpacked" or not, documented or not, I can't use it, because it doesn't exist in the current setup.

That said, bitpacking is unsuited for hardware interfacing because it is opaque:

And that's why I'd rather go with Ada's representation clauses. If the hardware platform is unsuitable for the thing I want it to do, it simply doesn't compile. Which is justified, because in 99% of the times such RCs are hardware specific.

The bit layout is also different on little and big endian systems.

And let me guess: There's no way to specify how the Endianess of the bitpacked type should be interpreted regardless of the target system?

Vinzent.

Well, the first version of the software I am maintaining appeared around 1992, too. There still were IFDEFs for TP4 in it.

For me it's a matter of the hardware.

I.e. we can exchange packed records with TP/Delphi, what would be the problem with bitpacking? If I recall well, Jonas made our bitpacking compatible with GCC and GNU-Pascal.

Great. And surely the PCI-board registers are compatible with GCC (oh, and if so, which compiler version then?).

No. Global variables in threading context are just as volatile as any memory-mapped hardware might be. And how should the compiler know that a given pointer actually points to a memory-mapped hardware register if you can't tell it? Sure, so far I did not have problems with such (I don't use too many global variables, the embedded hardware is programmed in another language anyway, and I only (mis)use types as atomic, if I can sufficiently guarantee that in most circumstances the access will be atomic indeed).

Vinzent.

I simply said that "bitpacked", being a major new feature (in terms of changes to the code generator), was introduced in a newer major release than what you are testing (as described in 2.2.0 release notes). Major releases can indeed more easily introduce major new bugs and incompatibilities.

If you want to help reduce that to a minimum, you can test pre-release versions, in order to see whether there are regressions which aren't caught by our test suites. If you don't have the time for that, you can just test the releases when they are out and hope for the best. I can guarantee you that every single new release will probably break something different. Whether or not it's something you use, nobody can tell in advance (except you).

And no, I'm not saying it's your fault if something you use breaks in a newer release, I'm merely saying it's nobody else's fault in particular either (at least not most of the time).

And let me guess: There's no way to specify how the Endianess of the bitpacked type should be interpreted regardless of the target system?

As I said it's an opaque construct (so no control or guarantees whatsoever about the layout) and it was not written with hardware interfacing in mind (regardless of what other people like to promote it for).

Jonas

Jonas Maebe wrote

And no, I'm not saying it's your fault if something you use breaks in a newer release, I'm merely saying it's nobody else's fault in particular either (at least not most of the time).

Hey, I wasn't trying to blame anybody here.

I was merely reasoning why I can't use the latest bleeding edge compiler. ;)

So, basically it is not useful in the context I was mentioning (JFTR: neither do the similar C99 constructs). Things like memory controller register fall into the hardware category, I suppose. ;)

Vinzent.

C bitpacking is not "GCC-compatible" or not. C bitpacking is defined in the ABI for each platform (apart from the
__attribute__((__packed__)) case, where the packing /is/ GCC-specific).

And yes, that means it's unsuited for hardware interfacing. You can hack around it (I repeat, this is a hack) by
a) using cases which are equal on all current ABIs (although this is not guaranteed to remain so by anyone in any way) like a.o. not using bitfields which span a byte/word/dword boundary (depending on the size of the bitfield and the bitfields coming before it) b) taking into account the endianess of the host platform (since bitfields start counting from the most significant bit of big endian systems and from the least significant bit on little endian systems).

Jonas

And that's why I'd rather go with Ada's representation clauses. If the hardware platform is unsuitable for the thing I want it to do, it simply doesn't compile. Which is justified, because in 99% of the times such RCs are hardware specific.

I still don't see how ada could handle things like little endian cpus accessing big endian peripherial with bit patterns overlapping byte borders (no, this is not an academic case, just look at the brain dead arm fpa).

Well, I'm not sure what you want here, let me try:

|type Data is ...
|
|for Data'Bit_Order use High_Order_First;
|for Data use
|  record
|     Byte_3 at 0 range 0 .. 7;
|     Byte_2 at 1 range 0 .. 7;
|     Byte_1 at 2 range 0 .. 7;
|     Byte_0 at 3 range 0 .. 7;
|  end record;

Don't get it wrong. An Ada compiler does *not* automatically flip bytes for you, so if you really have some type spanning byte boundaries in the wrong endianess, you still have to flip it accordingly. IOW:

|type Data is:
|
|for Data'Bit_Order use High_Order_First;
|for Data use
| record
| Word_1 at 0 range 0 .. 15;
| Word_0 at 2 range 0 .. 15;
...

does not work as one might expect. The bytes would still be flipped on a little endian machine.

But at least it is possible to define a record representation totally endianess independent. It's a bit more complex than necessary, so it's probably better to read the whole stuff (although it might require some Ada experience to really see it through)

http://www.sigada.org/ada_letters/sept2005/Endian-Independent%20Paper.pdf

instead of letting me give you an example.

After all, for embedded programming the guarantees I get from Ada record representation clauses is usually enough to know it'll work.

Vinzent.

In the Linux source code preprocessor macros are used if data structures are accessed that require a certain endianess (usually "net"- data that are defined high byte first by TCP/IP).

Here clever tweaking provides that no additional code is created if the byte order already is correct with the processor the code is compiled for.

-Michael

As pointed out, this is possible without a preprocessor. Not in C, though.

Vinzent.

Op Wed, 27 Feb 2008, schreef Jonas Maebe:

As I said it's an opaque construct (so no control or guarantees whatsoever about the layout) and it was not written with hardware interfacing in mind (regardless of what other people like to promote it for).

Ok, I'll bite, we have something to discuss :)

Bitpacking can have two uses:
A) Saving the last bit of memory
B) Binary matching structures (which can be hardware)

Throw away (B) and half (most?) of the use of the feature is gone.

Of course the compiler should be internally consistent. A record written on i.e. Windows should be readable on i.e. Linux. Only if {$packrecords C} is used, we pack records according to ABI, as to interface with the OS.

There is no reason why bitpacked records should be any different from packed records: They should internally consistent. Therefore, the packing behaviour should be known, documented, and therefore usable for binary use. In other words, if there will be ever differences as to target platforms, this has to happen only under $packrecords C, or an equivalent directive for bitpacked records.

Daniël

Daniël Mantione wrote

There is no reason why bitpacked records should be any different from packed records: They should internally consistent. Therefore, the packing behaviour should be known, documented, and therefore usable for binary use.

They are already very different from regular packed records due to the fact that their structure is completely different on little and big endian systems (they are not byte swapped, but "bit swapped").

Locking the compiler into a particular implementation by documenting and supporting the implementation details mainly lead to trouble (and lack of optimization opportunities) later due to the legacy balast it causes. And don't worry: even without documenting and supporting it, people will still figure it out (well, it's documented by example in the source code's comments, so that's not that hard), still depend on the layout, and still file bugs if it changes for some reason.

And as Vinzent noted, simply making something "bitpacked" does not make it suited for binary interfacing. You need more control over the layout if you want to really use it. Bitpacked records are simply not fit for defining a particular layout (let alone access patterns: writing to a bitfield may involve loading the containing and several surrounding bytes, sometimes more than once in the generic implementation, and that's certainly not something you want to happen when accessing hardware).

In other words, if there will be ever differences as to target platforms, this has to happen only under $packrecords C, or an equivalent directive for bitpacked records.

There is no equivalent directive for bitpacked records, and supporting C-style bitpacking is impossible with regular Pascal syntax ("char c: 1" and "int c:1" are often laid out differently in C depending on the previous fields, but there is no way to express this difference using plain Pascal; you need extra attributes -- not to mention that those packing rules are often quite hairy, and it would be a mess to implement them all).

Jonas

Op Wed, 27 Feb 2008, schreef Jonas Maebe:

Okay, then we do not support "ABI" packed records, we can support it, but aren't after all required to support it.

As long as the compiler is consistent between platforms, it is okay. Differences between little/big endian are acceptable because this is the only situation where we require the coder to manually intervene and write two code paths (usually a simple endian conversion). We don't force the coder to make different code paths between i.e. Linux/Windows, nor should we.

Daniël

Of course the compiler should be internally consistent. A record written on i.e. Windows should be readable on i.e. Linux. Only if {$packrecords C} is used, we pack records according to ABI, as to interface with the OS.

Are any bitpacked structures used in the API of any OS ?

If hardware access is an issue: how should endianess be handled ? The compiler does know about the endianess of the processor, but it does not know about what endianess the user needs in the hardware. Thus it would be necessary to define it via a kind of compiler option.

-Michael

Bitpacked records are simply not fit for defining a particular layout

Thus maybe an additional option (supposedly three choices: "compiler endianess" (default = not binary portable = behavior as it is now), "low endian", and "high endian") could help.

-Michael

Op Thu, 28 Feb 2008, schreef Michael Schnell:

Probably not, but the "OS" should be interpreted broadly here, any C library can expose bitpacked structures, therefore they can be standardized in ABIs.

If hardware access is an issue: how should endianess be handled ? The compiler does know about the endianess of the processor, but it does not know about what endianess the user needs in the hardware. Thus it would be necessary to define it via a kind of compiler option.

The compiler can only care about processor endianness. Having a known binary structure is something different as being usable for hardware access.

Daniël

Op Thu, 28 Feb 2008, schreef Michael Schnell:

To my knowledge there is no problem with the current implementation. Endian conversion is already the reponsibility of the programmer. Therefore I don't see a need for changes on the compiler side.

Daniël

C-style bitpacking ("char c:1" and "int c:1" are often laid out differently in C depending on the previous fields,

Not only this. C defines the layout as implementation depended. I once was bitten by this when porting a networked project from a low endian processor to a high endian processor :( .

Thus if we want binary portability of the structures we need to be better than C (optionally defining the structure as high-endian or low endian on user request) !!!!

-Michael

To my knowledge there is no problem with the current implementation. Endian conversion is already the reponsibility of the programmer. Therefore I don't see a need for changes on the compiler side.

I don't understand your meaning here.

If a record with binary fields is defined and transferred to (e.g.) another instance of the same program running on another machine by network or within a file, or if it needs to be crafted according to a specification of a network block or file format or a hardware device, endianess needs to be taken care of, and it's not nice if the user needs to write active code to do this, if a "bitpacked" type is available that seemingly can be used for that issue.

You even might want to define a record that just holds a single 16 bit value. Here it would be good to be able do define endianess of the bitpacked structure to make it compatible with the communication partner.

-Michael

Op Thu, 28 Feb 2008, schreef Michael Schnell:

Why?

The only thing I want to guarantee is that blockwrite followed by blockread on a platform with the same endianness works and will work in the future.

This (combined with ifdef based endian conversion) guarantees portability of structures to any platform.

Or do I see this wrong?

Daniël

Daniël Mantione wrote

To my knowledge there is no problem with the current implementation. Endian conversion is already the reponsibility of the programmer. Therefore I don't see a need for changes on the compiler side.

Jonas said the bits were swapped, not the bytes. So PPC32 (1 shl 30) becomes (1 shl 6) on Intel (actual), while it should be (1 shl 1) (expected use by compiler). It's both the "second bit", bit it's in different places.

Micha

The compiler can only care about processor endianness. Having a known binary structure is something different as being usable for hardware access.

Right. AFAIK, even C does not do this in a language construct. But FP might be better than standard :) .

-Michael

Op Thu, 28 Feb 2008, schreef Michael Schnell:

Fixed endianness of records could be a language extension, it might even ease programmers from doing endian conversion (rather than doing ifdefs, just mention the endianness of your record), much nicer.

However, the situation as it is, is that we do not have such a language feature and rely on manual endian conversion by means of ifdefs. There is no difference between normal, packed, or bitpacked records, you have to endian convert them manually.

Daniël

To my knowledge there is no problem with the current implementation. Endian conversion is already the reponsibility of the programmer. Therefore I don't see a need for changes on the compiler side.

It might be possible do define the individual bytes of a certain value in a bitpacked record to be located at certain bit positions by some complicated syntax, but if that is required for binary compatibility, that is not "nice" at all. A compiler option to optionally define the endianess is much more handy.

-Michael

Op Thu, 28 Feb 2008, schreef Micha Nelissen:

Okay, but does this have impact on the discussion? I mean it makes manual endian conversion a bit more tricky (also need to swap bits around), but doesn't change the fact that you manually need to do endian conversion.

Daniël

IMHO, I do think this not enough.

ADAIK, there is an FP version for a high endian processor (68K); more can be crafted any time. It should be made easy to create communication systems independent of the architecture. Of course you can't have binary compatibility with all values by default (due to performance considerations), but if we do have a "bitpacked" type that is not optimized for speed but for structure, it should be possible to use it for that purpose.

Moreover communication via structures in a documented layout is very often needed (Network, files, hardware, ...). There should be an easy way to craft a record type according to such a documentation, may if be documented to hold it's multi-byte values in high or low endian representation. "bitpacked" did open this box of Pandora and it's an obvious request to go all the way :) .

(e.g.: TCP/IP defines high-endian, while PC's work low endian internally, so everything needs to be converted.)

-Michael

Well we have procedures to do byte swapping, but none to do bit swapping. It's also very inefficient AFAIK; while changing the compiler's definition of which bit to use is "free".

Micha

Micha Nelissen wrote

Well we have procedures to do byte swapping, but none to do bit swapping. It's also very inefficient AFAIK; while changing the compiler's definition of which bit to use is "free".

{$BITORDER LOW_ORDER_FIRST}
{$BITORDER HIGH_ORDER_FIRST}

?

{$BITORDER LOW_ORDER_FIRST} {$BITORDER HIGH_ORDER_FIRST}

Where can this be used ? What exactly does it mean ?

-Michael

Michael Schnell wrote

> {$BITORDER LOW_ORDER_FIRST} > {$BITORDER HIGH_ORDER_FIRST} Where can this be used ? What exactly does it mean ?

Well, call it proposal (of course, the names are strongly influenced by personal language preferences).

AFAICS, it would be useful for bitpacked records only, so it could appear anywhere where a {PACKRECORDS} directive or similar can appear currently.

Vinzent.

It's not about Linux vs. Windows, it's about FPC 2.2.0 vs FPC 3.4.0, coupled with the fact that bitpacked records as currently defined are not usable for defining a specific layout.

For that sort of functionality, you need extensions anyway. If someone wants that functionality, it's better to create such extensions so the programmer can in fact specify everything and implement that, rather than adding constraints to the implementation of bitpacked records.

Jonas

Op Thu, 28 Feb 2008, schreef Jonas Maebe:

It compeletely normal that a record written by a a program written in FPC 2.2 can be read be FPC 3.4. If you design a feature, after a grace time, it should be kept backward compatible. I haven't heard the argument why bitpacked records should be exempt from this.

Daniël

AFAICS, it would be useful for bitpacked records only, so it could appear anywhere where a {PACKRECORDS} directive or similar can appear currently.

IMHO it would only be useful (allowed with, regarded by) bitpacked record, as any other data representation is supposed to be optimized for speed according to the processor architecture.

-Michael

A regularly packed record, yes. "Non-packed" records: fact, their layout changed in some circumstances in FPC 2.3.1 compared to earlier versions. As to bitpacked records:

If you design a feature, after a grace time, it should be kept backward compatible.

Not if they are described like this in the manual (ref.tex, line 1204):

***
Note that the internals of the bitpacking are opaque: they can change at any time in the future. What is more: the internal packing depends on the endianness of the platform for which the compilation is done, and no conversion between platforms is possible. This makes bitpacked structures unsuitable for storing on disk or transport over networks. The format is however the same as the one used by the GNU Pascal Compiler, and we aim to retain this compatibility in the future. ***

The same goes for the internal format of sets (which also changed a while ago), and should also go for the layout (as far as the part which is normally invisible to the programmer goes) and reference counting of ansistrings/interfaces etc.

I haven't heard the argument why bitpacked records should be exempt from this.

Because they were not designed/implemented with binary portability/ compatibility in mind, and doing so allows freedom to optimize them, make them compatible on any platform with the custom format there if any (e.g. how debuggers expect them to be laid out), etc.

If you don't have to fix something in concrete, it's always a good idea not to do so because it'll only come back later to haunt you. That does not mean you have to actively try to change every opaque structure in every release in order to break backwards compatibility, but it does give you the freedom to do so when it's useful.

As I said before: if you want to define something which has a predictable layout, you have to do so specifically and give the programmer the means to do so. If it is not clear what the compiler will/may do from just reading the declaration and active compiler directives, it's almost by definition improper to rely on any current implementation details.

Jonas

Hmm, not necessarily. I frequently use enumeration types to express the meaning of a set of hardware bits. So thinking about it, what if I'd use enumerations in a bitpacked record? Maybe like this:

-- 8< --
type
   // 2 bits.
   Enable_Set = (Dont_Care := 0,  // 00
                 Disable   := 1,  // 01
                 Enable    := 3); // 11
type
   Control =
   bitpacked record
      Continuous_Mode   : Boolean;    // bit 0/rightmost bit
      Alternate_Compare : Boolean;    // bit 1
      ...
      Enable_Mode       : Enable_Set; // bit 14 .. 15/leftmost bits
   end;
-- 8< --

I don't know if FPC can pack enumerations into a bitpacked record at all, but if it does, it might consider the bit order here, too.

Consider something like:

-- 8< --
var
   My_Set    : Enable_Set;
   My_Record : Control;
My_Set := My_Record.Enable_Mode;
-- 8< --

How should the assignment be handled if the bit order for that bitpacked record is changed?

Vinzent.

Enable_Mode : Enable_Set; // bit 14 .. 15/leftmost bits

With an x86 the "leftmost bits" will be in the "rightmost" (second) of the two bytes,

with an 68K the "leftmost bits" will be in the "leftmost" (first) of the two bytes,

So the two can't communicate this record via files or via network.

If you want to have them understand each other, you need to define the edianess of the record independently of that of the processor. Enumerated types don't help here.

-Michael

Yes, bad example, because we already crossed the byte boundary. The real question about leftmost and rightmost was on which "data line" each bit would appear.

(Usually it's called most significant bit, but as we're talking about hardware bits, not numbers, I wouldn't use that term. In this context, no bit is necessarily more significant than the other.)

So the two can't communicate this record via files or via network. If you want to have them understand each other, you need to define the edianess of the record independently of that of the processor.

Yes. That's what {$BIT_ORDER} would stand for (still, it would not change *byte* order).

Enumerated types don't help here.

They weren't meant to solve the issue, they were meant to help to understand the issue I was trying to point out.

The question was, how to interpret the enumeration values, if their bit order could/would differ from that of the record they're in.

Should they be put in as is or swapped accordingly?

Vinzent.

Yes. That's what {$BIT_ORDER} would stand for (still, it would not change *byte* order).

I don't understand this. I don't think the bit order within a byte is to be considered changing.

I would call the issue "byte-order" and (thus I'd prefer something like
{$BIT_PACKED_BYTE_ORDER} or {$BIT_PACKED_ENDIAN}.

-Michael

Well, the question is, if the first bit in a record is the leftmost or the rightmost bit.

It's a matter of interpretation. But as Jonas pointed out, the order of the bits may change depending on the endianess (assuming I didn't misunderstand him).

I would call the issue "byte-order" and (thus I'd prefer something like {$BIT_PACKED_BYTE_ORDER} or {$BIT_PACKED_ENDIAN}.

It's not byte order.

If I declare:

|bitpacked record
|   X : Byte;
|   Y : Byte;
|end record;

X will still be at the lowest address and Y will be at @X + 1. The issue arises when I say:

|bitpacked record
|   X : Boolean;
|   Y : Boolean;
|   Z : Two_Bit_Enum;
|end;

Assuming, bit 0 is the LSB, does the compiler access bit 0 and 1 (low order first) for X and Y or does it choose bit 7 and 6 (high order first) then? And how would it interprete a specific value for Z? At least two interpretations are possible:

X:7, Y:6, Z[5:4] or X:0, Y:1, Z[3:2]

ASCII graphic:

|X|Y|Z|Z|-|-|-|-|
|-|-|-|-|Z|Z|Y|X|

Ok, I guess, the issue with the enum is none, because the LSB is still at the right place on the data bus, no matter how you forget that. ;)

Of course, there are more nasty things like

|bitpacked record
|   X : Boolean;
|   Y : Byte;
|end;

where a single value would cross the byte boundary... *headscratch* I guess, there's a reason, why endianess issues are not automatically handled by the compiler. :D

Vinzent.