Redirected to fpc-devel for a larger audience.

-------- Original-Nachricht --------
Betreff: TMemDataset : pb & suggestions
Datum: Fri, 06 Oct 2006 11:34:51 +0200
Von: Oro06
Antwort an:
An: core

Hi,

i'm trying to use TMemDataset on arm/wince and
actually getting bus error/alignement issue.

1°) pb
in short :
all records are stored in a big memory stream
each record is accessed using base address+offset(multiple of RecSize where RecSize is the sum of all fields size)
then each field is also accessed using base record address+field offset

finally accessing datas give something like this : pChar(Buffer+MDSGetFieldOffset(I))^
and this give alignement errors
is there a proper way to make it mode cpu portable ?

2°)missing feature :
-this architecure can't handle blob/strings with unfixed length (with sqlight datasets, length for string fields is 0) -as indicated in sources, record delete is unefficient as it duplicate the full stream (see line 364 memds.pp)

3°)proposed changes
a-use of an internal TList or TLockList to navigate between records b-handle a second internal 'aligned size' longint for each fields used for storing datas in memory with internalalignedsize=usersize+some alignement unused bytes
c-strings are no more stored directly in the record buffer but allocated in the heap and only a pointer to it
is stored in record buffer
d-add of blob handling with same rule as strings

->i don't see maintener on snv log for this unit so i'm about to do that and of course
any comments regarding this and/or multicpu best practice appreciated

regards
or

re

Florian Klaempfl wrote

Redirected to fpc-devel for a larger audience.

actually this give an alignement error:

PRecInfo(Buffer+FRecInfoOffset)^.BookmarkFlag := Value;    (memds line
750, procedure TMemDataset.SetBookmarkFlag)
replacing with this :
  APRecInfo:=PRecInfo(Buffer+FRecInfoOffset);
  Unaligned(APRecInfo^).BookmarkFlag := Value;

give Internal error 20060521

regards
or

Can you create a short compiling/crashing example which shows the problem?

sample attached based on testpop (from memds dir)
run well on win32.

line 739 :   PRecInfo(Buffer+FRecInfoOffset)^.BookmarkFlag := Value;

give Bus error or misaligned data access TMemdataset.set_bookmarkflag, line 739

replacing line 739 with that :

  APRecInfo:=PREcInfo(Buffer+FRecInfoOffset);
  Unaligned(APRecInfo^).BookmarkFlag := Value;

give this at compile time:
Fatal: Internal error 20060521

regards
or

it seems that using move func give better results.

replacing this

function TMemDataset.GetBookmarkFlag(Buffer: PChar): TBookmarkFlag;
begin
  Result:=PRecInfo(Buffer+FRecInfoOffset)^.BookmarkFlag;
end;

by this :

function TMemDataset.GetBookmarkFlag(Buffer: PChar): TBookmarkFlag;
var ARecInfo: TMTRecInfo;
begin
  Move(PRecInfo(Buffer+FRecInfoOffset)^,ARecInfo,FRecInfoSize);
  Result:=ARecInfo.BookmarkFlag;
end;

same for TMemDataset.SetBookmarkFlag...

procedure TMemDataset.SetBookmarkFlag(Buffer: PChar; Value: TBookmarkFlag);
var APRecInfo: PRecInfo;
    ARecInfo: TMTRecInfo;
begin
  //Unaligned(PRecInfo(Buffer+FRecInfoOffset)^).BookmarkFlag := Value;
  APRecInfo:=PREcInfo(Buffer+FRecInfoOffset);
  Move(APRecInfo^,ARecInfo,FRecInfoSize);
  ARecInfo.BookmarkFlag:=Value;
  Move(ARecInfo,APRecInfo^,FRecInfoSize);
end;

regards
or

.patch for memds, tested on win32 and wince
+4 methods added to encapsulate RecInfo access (that was causing alignement troubles in several places under wince) +const added to define size (now 200) when string Field with zero size are copied from other datasets
+cache added for fields offset instead of recalculating sum(fileds size) everytime

regards
or

Applied the patch, thank you very much !

Michael.

We should fix the compiler instead of working around its bugs :)

We should fix the compiler instead of working around its bugs :)

lines 771, 782 and 794 are commented and ready to use when Fatal: Internal error 20060521 is fixed :)

Ok, should be fixed.