Hello lazarus project,

This error is shown, when I try to SetLength (FItem, 1); where FItem is defined as:

TYPE TSomeItem = RECORD
                   First :single;
                   Second:Single;
                   Third :integer;
TYPE TSomethingArr = ARRAY OF TSomeItem;
TYPE TMyObject =class
                 private
                   FItem :TSomethingArr;
                 ...

Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_PROTECTION_FAILURE at address: 0x0000cbe0 0x00002890 in SYSTEM_CPUDECLOCKED$LONGINT$$BOOLEAN () (gdb)

I use up-to-date SVN versions of Lazarus/Fpc on MacOSX 10.5 Intel with Carbon.

What could be a reason?

this code should work:

var
  m : TMyObject;
...
  m := TMyObject.Create; // has been called first?
  SetLength(m.FItem, 1);

also, you don't need to write 'type' before every type declaration. It's better to use pascal style:

type
  TSomeItem = record
                  First :single;
                  Second:Single;
                  Third :integer;
  end;
  TSomethingArr = array of TSomeItem;
  TMyObject =class
  private
    FItem :TSomethingArr;
    ..
  end;

thanks,
dmitry

Hello Dmitry,

Thanks for your reply.

I call SetLength from public method of class TMyobject, what means that object is created and accessible.

About my style: I have just written a fast example which was written inside of email client without any style. :-)

TRoland;

<<< 23.4.2009 12:23 - dmitry boyarintsev "" >>>
db> also, you don't need to write 'type' before every type declaration. db> It's better to use pascal style:

db> type
db>   TSomeItem = record
db>                   First :single;
db>                   Second:Single;
db>                   Third :integer;
db>   end;
db>   TSomethingArr = array of TSomeItem;

db> TMyObject =class
db> private
db> FItem :TSomethingArr;
db> ..
db> end;

db> thanks,
db> dmitry

Why does calling from a public method mean that the object has been created?

Vincent

Hello Vincent,

Thanks for your reply.

Of course, I have checked the existence of object and it is created. The public method called contain lot of code which uses other objects created in constructor where the FItem is initializated with := NIL;

I still think, that it must be an error of compiler.

TRoland;

<<< 23.4.2009 12:52 - Vincent Snijders "" >>>

VS> Why does calling from a public method mean that the object has been created?

VS> Vincent

This code works perfectly fine (at least it does not crash) with fpc 2.2.2

constructor TMyObject.Create;
begin
  DebugLn('Create');
  FItem := nil;
  DebugLn('nil');
  SetLength(FItem,3);
  DebugLn('SetLength');
  FItem[0].First:= 1.0;
  DebugLn('FItem[0].First = ',DbgS(FItem[0].First));
end;

Bart

Hello Bart and others,

Thanks for your kindness to test my test case, which works fine in my case too, but I have found something what maybe more explain my problem.

TYPE TSomeItem = RECORD
                   First :single;
                   Second:Single;
                   Third :integer;
TYPE TSomethingArr = ARRAY OF TSomeItem;

TYPE TSomeHeader =RECORD
...

                    end;

TYPE TMyObject =class
                 private
                   FHeader:TSomeHeader;
                   FItems :TSomethingArr;

...

FUNCTION TMyObject.LoadFromStream (Stream:TStream):BOOLEAN;
...
  SetLength (FItems, 0); //!! for testing only -- this line works fine
  Stream.Read (FHeader, HeaderLen);
  SetLength (FItems, 0); //!! for testing only -- this line raises an exception
END;

If you see the declaration of TMyObject you will find that FHeader declaration is followed by FItems and the length of read memory block goes over size of FHeader and it affects FItems.

Hm?

TRoland;

<<< 23.4.2009 17:19 - Bart "" >>>
B> This code works perfectly fine (at least it does not crash) with fpc 2.2.2

B> constructor TMyObject.Create;
B> begin
B>   DebugLn('Create');
B>   FItem := nil;
B>   DebugLn('nil');
B>   SetLength(FItem,3);
B>   DebugLn('SetLength');
B>   FItem[0].First:= 1.0;
B>   DebugLn('FItem[0].First = ',DbgS(FItem[0].First));
B> end;

B> Bart

Hm?

TYPE TSomethingArr = ARRAY OF TSomeItem;

declares a dynamic arrays. Dynamic arrays are implicit pointers, so should be used carefully in streaming operations.

this should fix your problem:

FUNCTION TMyObject.LoadFromStream (Stream:TStream):BOOLEAN;
...
 SetLength (FItems, 0);
 // make sure, that length of (FHeader) is greater than zero,
 // otherwise it will cause another exception
 Stream.Read (FHeader[0], HeaderLen);
 SetLength (FItems, 0);
END;

thanks,
dmitry

No, FHeader is just a record and not an array. IMO the provided code should work, assuming
a) HeaderLen <= sizeof(TSomeHeader)
b) stream is a correctly initialized stream object c) myObject is initialized correctly.

However, just a few more lines to make a complete and compilable example that shows the error would be nice ...

regards
Burkhard

Roland Turcan pisze:

How is HeaderLen declared ?

Boguslaw

"maybe"? This does most certainly explain your problem, see below

Take a look of how thinks are organized in memory:

TMyObject allocates some memory. FHeaders (being a record) is embedded into the same memory, it is part of TMyObjects memory. But FItems (being an array) allocates it's own memory, somewhere completely different. It only stores a pointer inside TMyObject.

This means when you write past FHeaders, then you but a random value into the pointer. Next time you try to access the array it will try to follow this random value. This means you try to access memory that doesn't belong to the application. Even if the value pointed to an address inside the apps accessible mem, you would still override random other data and crash.

Best Regards
Martin

<<< 24.4.2009 15:30 - Bogus?aw Brandys "" >>>
BB> Roland Turcan pisze:

BB> How is HeaderLen declared ?

Stream.Read (HeaderLen, SIZEOF (HeaderLen));

where information header's length is stored into binary.

Stream.Read (HeaderLen, SIZEOF (HeaderLen)); where information header's length is stored into binary.

HeaderLen is integer, right?
have you checked HeaderLen being large that sizeof(TSomeHeader)

it's possible, especiialy for Mac files, because of PowerPC and some structures may use Big-endian byte order, rather than Intel (little-endian)

thanks,
dmitry

Hello Dmitry,

<<< 24.4.2009 16:17 - dmitry boyarintsev "" >>>

> Stream.Read (HeaderLen, SIZEOF (HeaderLen)); > > where information header's length is stored into binary.

db> HeaderLen is integer, right?
db> have you checked HeaderLen being large that sizeof(TSomeHeader)

Of course, that I know now that the problem is, that the binary which is parsed contains bigger size that size if Header is.

db> it's possible, especiialy for Mac files, because of PowerPC and some db> structures may use Big-endian byte order, rather than Intel db> (little-endian)

This I know, but are there other differences? I mean sizes of datatypes between Win&Linux and Mac (intel/ppc)?

Roland Turcan pisze:

hmm,I smell somthing related to not packed records and different size of record in different platforms ..

you should really check what is written to the file and what is read from stream.

Not sure, maybe it could be related ? : http://www.freepascal.org/port.var

Boguslaw

IMO the problem is HeaderLen. What if you use SizeOf(FHeader) instead?

DoDi

The you should verify that HeaderLen <= SizeOf(FHeader), before
   Stream.Read (FHeader, HeaderLen);

Otherwise this statement will overwrite the following FItem data, with the fatal consequences you already experienced.

When a header will ever change in size (or structure), it's wise to store a version number in the data files. Then you can read the stored header data into the exactly applicable header type (record), and convert that record into the current THeader definition, field by field.

DoDi

Hello Diettrich,

To tell the truth this code and style I got from previous developer and I really don't know why he decided to get the size of header from the binary instead of getting its size. The fact is, that this "optimistic variant" of coding is on many places. :-|

TRoland;

<<< 24.4.2009 19:56 - Hans-Peter Diettrich "" >>>

HPD> The you should verify that HeaderLen <= SizeOf(FHeader), before
HPD>    Stream.Read (FHeader, HeaderLen);

HPD> Otherwise this statement will overwrite the following FItem data, with HPD> the fatal consequences you already experienced.

HPD> When a header will ever change in size (or structure), it's wise to HPD> store a version number in the data files. Then you can read the stored HPD> header data into the exactly applicable header type (record), and HPD> convert that record into the current THeader definition, field by field.

HPD> DoDi

HPD> _______________________________________________ HPD> Lazarus mailing list
HPD>
HPD> http://www.lazarus.freepascal.org/mailman/listinfo/lazarus

Hello All,

I have found some new information about this problem:

1.

TYPE TMyOne =record
               First:integer;
               Second:integer;
             end;

... then the size of this object is 2*4 bytes.

2.

TYPE TMyOne =record
               First:integer;
               Second:double;
             END;

... where I would expect 4 + 8 bytes = 12 bytes
... but the size of this object is 16!

3.

TYPE TMyOne =record
               First:integer;
               Second:extended;
             END;

... where I would expect 4 + 16 bytes = 20 bytes
... but the size of this object is 32!

So this proves me, that compiler tries to align the data structure to multiplied size of biggest element to which fit all elements.

I have understood this behavior, but this happens me in these cases:

1. Delphi 7
2. Kylix 3
3. Lazarus on Ubuntu
4. Lazarus on PowerPC Mac OS X 10.5

but on Intel Based Mac OS X 10.5 Mini with Intel Duo 2 Core it DOESN'T. Therefore I have problems with parsing of binaries back to memory.

I don't expect CPU specific problem, otherwise it would not work active projects, therefore I think it must be compiler specific problem.

Do you have any idea?

Thanks in advance.

Greetings, TRoland;

<<< 25.4.2009 9:43 - Roland Turcan "" >>>
RT> Hello Diettrich,

RT> To tell the truth this code and style I got from previous developer RT> and I really don't know why he decided to get the size of header from RT> the binary instead of getting its size. The fact is, that this RT> "optimistic variant" of coding is on many places. :-|

RT> TRoland;

RT> <<< 24.4.2009 19:56 - Hans-Peter Diettrich "" >>>

HPD>> The you should verify that HeaderLen <= SizeOf(FHeader), before
HPD>>    Stream.Read (FHeader, HeaderLen);

HPD>> Otherwise this statement will overwrite the following FItem data, with HPD>> the fatal consequences you already experienced.

HPD>> When a header will ever change in size (or structure), it's wise to HPD>> store a version number in the data files. Then you can read the stored HPD>> header data into the exactly applicable header type (record), and HPD>> convert that record into the current THeader definition, field by field.

HPD>> DoDi

HPD>> _______________________________________________ HPD>> Lazarus mailing list
HPD>>
HPD>> http://www.lazarus.freepascal.org/mailman/listinfo/lazarus

use:
  TYPE TMyOne = packed record
              First:integer;
              Second:extended;
            END;
to be sure about 20 bytes boundary.

thanks,
dmitry

Have you checked packed records ?

Boguslaw

Hello Dmitry,

Thanks for your reply, but I think, that you haven't understood me what I am trying to explain.

I have binary contents which are copied from memory using by an application which is written in Delphi. I am porting an application to MacOS X 10.5, but I need to use the same structures as I have in Delphi and Kylix where "NOT PACKED" records are aligned.

The fact is, that FPC compiler on intel based Mac OS X builds up application which contains records "packed" by 4 bytes by default.

I need to get the same behavior in intel based mac os x as I have in other platforms.

What should I do to build FPC compiler which will behave the same on all platforms.

TRoland;

<<< 27.4.2009 17:04 - dmitry boyarintsev "" >>>
db> use:
db> TYPE TMyOne = packed record
db> First:integer;
db> Second:extended;
db> END;
db> to be sure about 20 bytes boundary.

db> thanks,
db> dmitry

db> Hello All,
db>
db> I have found some new information about this problem: db>
db> 1.
db>
db> TYPE TMyOne =record
db> First:integer;
db> Second:integer;
db> end;
db>
db> ... then the size of this object is 2*4 bytes. db>
db> 2.
db>
db> TYPE TMyOne =record
db> First:integer;
db> Second:double;
db> END;
db>
db> ... where I would expect 4 + 8 bytes = 12 bytes db> ... but the size of this object is 16!
db>
db>
db> 3.
db>
db> TYPE TMyOne =record
db> First:integer;
db> Second:extended;
db> END;
db>
db> ... where I would expect 4 + 16 bytes = 20 bytes db> ... but the size of this object is 32!
db>
db>
db> So this proves me, that compiler tries to align the data structure to db> multiplied size of biggest element to which fit all elements. db>
db> I have understood this behavior, but this happens me in these cases: db>
db> 1. Delphi 7
db> 2. Kylix 3
db> 3. Lazarus on Ubuntu
db> 4. Lazarus on PowerPC Mac OS X 10.5
db>
db> but on Intel Based Mac OS X 10.5 Mini with Intel Duo 2 Core it db> DOESN'T. Therefore I have problems with parsing of binaries back to db> memory.
db>
db> I don't expect CPU specific problem, otherwise it would not work db> active projects, therefore I think it must be compiler specific db> problem.
db>
db> Do you have any idea?
db>
db> Thanks in advance.
db>
db> Greetings, TRoland;
db>
db>
db> <<< 25.4.2009 9:43 - Roland Turcan "" >>>
RT>> Hello Diettrich,
db>
RT>> To tell the truth this code and style I got from previous developer RT>> and I really don't know why he decided to get the size of header from RT>> the binary instead of getting its size. The fact is, that this RT>> "optimistic variant" of coding is on many places. :-| db>
RT>> TRoland;
db>
RT>> <<< 24.4.2009 19:56 - Hans-Peter Diettrich "" >>> HPD>>> Roland Turcan schrieb:
db>

db>
HPD>>> The you should verify that HeaderLen <= SizeOf(FHeader), before HPD>>> Stream.Read (FHeader, HeaderLen);
db>
HPD>>> Otherwise this statement will overwrite the following FItem data, with HPD>>> the fatal consequences you already experienced. db>
HPD>>> When a header will ever change in size (or structure), it's wise to HPD>>> store a version number in the data files. Then you can read the stored HPD>>> header data into the exactly applicable header type (record), and HPD>>> convert that record into the current THeader definition, field by field. db>
HPD>>> DoDi

Check docs. Also http://wiki.freepascal.org/Mode_MacPas and http://bugs.freepascal.org/view.php?id=13187

Cannot help further, because don't know Mac OS.

Regards
Boguslaw

I'm not even sure that Extended has the same size on all platforms. I might of course be horribly wrong about this ;-) But if not, it might complicate things.

If you need to read those records, created with Delphi/Kylix now on MacOS and you cannot solve how to declare your record, you could as a last resort read x bytes (where x is the size of the delphi record on disk) and translate theis sequence of bytes to the values you want.

It's just one of those things with proprietary formats and cross-platform usage.

Bart

Use packed records and add gap variables.

This won't work. The other sources and libs expect another alignment. FPC uses the various alignments for good reason.

Mattias

I'm not even sure that Extended has the same size on all platforms.

What size might it have, other than the FPU defined size?

If you need to read those records, created with Delphi/Kylix now on MacOS and you cannot solve how to declare your record, you could as a last resort read x bytes (where x is the size of the delphi record on disk) and translate theis sequence of bytes to the values you want.

If nothing else helps, it should be sufficient to use packed records with inserted dummy fields for the original record layout.

DoDi

what FPU?

What if some FPU does not support 80 bits floating point, but does support 128 bits floating point, then the extended type may be 128 bits. Similar things happened with 'integer' too, it grew from 16 to 32 bits.

Vincent

I know that you know this, but for the others:
integer is 32bit on 64 bit machines too. AFAIK integer is only 16bit in mode tp.

And on some machine extended can be a double.

Mattias

Mattias Gaertner wrote

> > Bart schrieb: > > > >> I'm not even sure that Extended has the same size on all platforms. > >

[...]

And on some machine extended can be a double. Mattias

And so they probably do not have the same size ...?

Another thougth.
Are you gonna run int endian-ness problems when reading files on a Mac that are created on a PC?

Bart

Yes, see here
http://wiki.lazarus.freepascal.org/Multiplatform_Programming_Guide#Endianess

Mattias

<<< 27.4.2009 19:26 - Mattias Gaertner "" >>>
MG> Use packed records and add gap variables.

Uffff, look at this:

TYPE TPackedGap =PACKED RECORD
                   VarInteger :INTEGER;
                   Dummy1     :INTEGER;
                   VarDouble  :DOUBLE;
                   VarBoolean :BOOLEAN;
                   VarBoolean2:BOOLEAN;
                   Dummy2     :Word;
                   Dummy3     :INTEGER;
                 END;

... and I would expect, that the binary representation of this definition will keep the order of variables from definition, but it doesn't.

Could somebody explain what rules is applied to change the order of variables in memory?

TRoland;

<<< 27.4.2009 19:26 - Mattias Gaertner "" >>>
MG> On Mon, 27 Apr 2009 17:27:30 +0200

MG> Use packed records and add gap variables.

MG>

MG> This won't work. The other sources and libs expect another alignment. MG> FPC uses the various alignments for good reason.

MG> Mattias
MG> _______________________________________________ MG> Lazarus mailing list
MG>
MG> http://www.lazarus.freepascal.org/mailman/listinfo/lazarus

It does.

Could somebody explain what rules is applied to change the order of variables in memory?

You forgot to explain why you think that the order is wrong.

Mattias

Hello all,

I take back my previous mail, that the packed record doesn't use the same order as it is definition. I am in time hurry and made on error, that I was exporting the content of similar record to file to compare and therefore I looked me as wrong one.

So it is solved.

TRoland;

<<< 28.4.2009 15:23 - Roland Turcan "" >>>
RT> <<< 27.4.2009 19:26 - Mattias Gaertner "" >>>
MG>> Use packed records and add gap variables.

RT> Uffff, look at this:

RT> TYPE TPackedGap =PACKED RECORD
RT>                    VarInteger :INTEGER;
RT>                    Dummy1     :INTEGER;
RT>                    VarDouble  :DOUBLE;
RT>                    VarBoolean :BOOLEAN;
RT>                    VarBoolean2:BOOLEAN;
RT>                    Dummy2     :Word;
RT>                    Dummy3     :INTEGER;
RT>                  END;

RT> ... and I would expect, that the binary representation of this RT> definition will keep the order of variables from definition, but it RT> doesn't.

RT> Could somebody explain what rules is applied to change the order of RT> variables in memory?

RT> TRoland;

RT> <<< 27.4.2009 19:26 - Mattias Gaertner "" >>>
MG>> On Mon, 27 Apr 2009 17:27:30 +0200

MG>> Use packed records and add gap variables.

MG>>

MG>> This won't work. The other sources and libs expect another alignment. MG>> FPC uses the various alignments for good reason.

MG>> Mattias
MG>> _______________________________________________ MG>> Lazarus mailing list
MG>>
MG>> http://www.lazarus.freepascal.org/mailman/listinfo/lazarus