Hello,

Using TSdfDataset my records are cut to fit 255 chars, even when I change the new MaxRecordLength property to 2048. This is a very elusive bug, and I am searching all over for the cause without success =(

I searched everywhere for a shortstring, which could be the cause, but didn't find any. Any ideas?

I was not able to debug inside FCL, maybe the standard build comes without debug info? I was able to debug the sdfdata unit because I am using it locally in my project, because I am using FPC 2.2.2 and when the flow of events comes to TFIXEDFORMATDATASET__SETFIELDDATA the string is already cutted and in TDBDRAWINGCODEMEMO__UPDATEDATA it was correct, so it must have been cutted in the FCL somewhere in between.

My data is comming from a modifyed TDBMemo, and I already certified that my Memo is giving a correct string to the dataset.

Here is the chain of events that happen when I click in a button that will call Post for the dataset:

#0 TFIXEDFORMATDATASET__SETFIELDDATA((^TFIELD) 0x14f358, (POINTER) 0x6d430, (TFIXEDFORMATDATASET) 0xbede0) at sdfdata.pp:674 <<<--------- String is cutted to 255 chars
#1 DB_TDATASET_$__SETFIELDDATA$TFIELD$POINTER$BOOLEAN at :0 #2 DB_TFIELD_$__SETDATA$POINTER$BOOLEAN at :0
#3 DB_TFIELD_$__SETDATA$POINTER at :0
#4 DB_TSTRINGFIELD_$__SETASSTRING$ANSISTRING at :0 #5 TDBDRAWINGCODEMEMO__UPDATEDATA((^TOBJECT) 0x159490, (TDBDRAWINGCODEMEMO) 0x10a750) at dbcomponents.pas:340 <<<------ String is correct
#6 TFIELDDATALINK__UPDATEDATA((TFIELDDATALINK) 0x159490) at dbctrls.pp:1312 #7 DB_TDATALINK_$__UPDATERECORD at :0
#8 TDBDRAWINGCODEMEMO__EDITINGDONE((TDBDRAWINGCODEMEMO) 0x10a750) at dbcomponents.pas:372
#9 TCONTROL__MOUSEDOWN(MBLEFT, [SSLEFT], 8, 13, (TCONTROL) 0x11d280) at control.inc:3219
#10 TCUSTOMSPEEDBUTTON__MOUSEDOWN(MBLEFT, [SSLEFT], 8, 13, (TCUSTOMSPEEDBUTTON) 0x11d280) at speedbutton.inc:635

Possibly because FieldDefs[<n>].Size is too small. Did you setup TSdfDataset.Schema according to your needs? The format is <fieldname>=<fieldsize> AFAIK, MAXSTRLEN is a default value only. The new MaxRecordLength property is actually unnecessary and a little bit misleading.

I was not able to debug inside FCL, maybe the standard build comes without debug info?

Correct.

"
procedure TStringField.SetAsString(const AValue: string);
var Buf      : TStringFieldBuffer;
begin
  IF Length(AValue)=0 then
    begin
    Buf := #0;
    SetData(@buf);
    end
  else if FTransliterate then
    begin
    DataSet.Translate(@AValue[1],Buf,True);
    Buf[DataSize-1] := #0;
    SetData(@buf);
    end
  else
    begin
    // The data is copied into the buffer, since some TDataset descendents
copy
    // the whole buffer-length in SetData. (See bug 8477)
    Buf := AValue;
    // If length(AValue) > Datasize the buffer isn't terminated properly
    Buf[DataSize-1] := #0;           <<<<<<<<<<<<<-------- probably here
    SetData(@Buf);
    end;
end;
"

Martin

Martin Schreiber wrote

Possibly because FieldDefs[<n>].Size is too small. Did you setup TSdfDataset.Schema according to your needs? The format is <fieldname>=<fieldsize> AFAIK, MAXSTRLEN is a default value only. The new MaxRecordLength property is actually unnecessary and a little bit misleading.

Indeed, but I can't find what needs to be fixed. I simply didn't set my field sizes, and I would expect it to accept any sizes, and I cannot find why it's cutted in the special value of 255.

Also note that the schema for TSdfDataset is different then for TFixedDataset.

I think the problem is somewhere in TSdfDataSet.InternalInitFieldDefs, but the code is rather wierd and hard to understand. Even the high readability of Pascal doesn't help in a code with a lot of short whiles zero comments.

Even worse to debug, today it suddenly started working without any code changes!! =(

It also works even if I change MaxRecordLength to 1, which proves that this field is absolutely unnecessary.... I renamed it to DefaultRecordLength. Didn't yet commit.

Another wierd thing is that it absolutely does not work (nothing is read from database) if I don't set MaxRecordLength. The property has a default value of 250, which somehow does not work, so I removed the default in the property and initialized the value in the constructor, which works.

In our previous episode, Felipe Monteiro de Carvalho said:

That shoulds like a shortstring limit. Is everything in {$H+} or {$mode delphi} ?

Just for clarification : if you declare a property with a default value, you _have_ to set
that value to the property in the constructor. The compiler will _not_ set it for you.
For example :

TClass_a = class
public
  property a : integer read Fa write Fa default 1210;
  constructor Create();
end;
constructor TClass_a.Create();
begin
  Fa := 1210;
end;

Marco van de Voort wrote

That shoulds like a shortstring limit. Is everything in {$H+} or {$mode delphi} ?

This was my first idea, but I couldn't find anywhere a H+ missing ...

Also, as to my previous statement that it now started without without reason, it seams only to have moved to a higher limit. I am getting cut with 310 chars.

If anyone wishes to take a look at the full project:

svn co https://turbocircuit.svn.sourceforge.net/svnroot/turbocircuit turbocircuit

TurboCircuit is a circuit drawing program developed with lazarus. It should work in any lazarus supported OS. Be careful that there is a local copy of sdfdata to test changes to it. Opening the program, click in the menu Components --> Components Editor.

There will be a modifyed TDBMemo which holds the drawing instructions for the components: draw line, text, arc, etc, and this drawing code may be up to 2k chars long for complex components, so I need a really flexible size.

thanks,

Did you open an existing datafile?
"
procedure TFixedFormatDataSet.InternalInitFieldDefs;
var
  i, len, Maxlen :Integer;
  LstFields      :TStrings;
begin
  if not Assigned(FData) then
    exit;
  FRecordSize := 0;
  Maxlen := 0;
  FieldDefs.Clear;
  for i := FData.Count - 1 downto 0 do  // Find out the longest record <<<--
  begin
    len := Length(FData[i]);
    if len > Maxlen then
      Maxlen := len;                   <<<--
    FData.Objects[i] := TObject(Pointer(i+1));   // Fabricate Bookmarks
  end;
  if (Maxlen = 0) then
    Maxlen := MAXSTRLEN;
  LstFields := TStringList.Create;
  try
    LoadFieldScheme(LstFields, Maxlen);
    for i := 0 to LstFields.Count -1 do  // Add fields
    begin
      len := StrToIntDef(LstFields.Values[LstFields.Names[i]], Maxlen); <<<--
"
Martin

Martin Schreiber wrote

Did you open an existing datafile?

I am always working opening an existing datafile. So, you are thinking that it calculates the size for the fields upon loading and then it does not increase the field sizes when posting to the database?

It seems so if you don't define the field size in the TSdfDataset.Schema. Unlike tmsebufdataset, TSdfDataset stores the data in a record with fixed size which can't be changed after open. Is it possible to define the field size in the first data row? I never used TSdfDataset, I made the integration into MSEgui with tmsesdfdataset only so my knowledge about TSdfDataset is limited.
Is Sqlite3 no option for you? MSEgui and FPC sqldb both have a DB-connection component for SQLite3.

Martin

Martin Schreiber wrote

It seems so if you don't define the field size in the TSdfDataset.Schema. Unlike tmsebufdataset, TSdfDataset stores the data in a record with fixed size which can't be changed after open. Is it possible to define the field size in the first data row?

Didn't work:

ID,NAMEEN,NAMEPT,HEIGHT,WIDTH,PINS,DRAWINGCODE = 2048 chars

It hits the same limit as before.

I have found sdf better then sqlite3 (except for the bugs) for my small database needs, because I then I don't need an external library and I like the fact that i can easely edit the database with a text editor.

There are 2 ways:

1. When opening an existing file, the longest existing line length is used as the max record size if firstlineasschema is True. The lengths are not changed after that.

2. When creating a new file, the lengths are taken from the schema.

So:
I would imagine that in your case, you set firstlineasschema=False, and set the correct lengths in Schema. That should do it.

A possible extension is that the schema is defined in the FieldDefs property, and that this is used to parse the file.

Michael.

Michael Van Canneyt wrote

1. When opening an existing file, the longest existing line length is used as the max record size if firstlineasschema is True. The lengths are not changed after that.

That looks wrong. So it's basically impossible to have data of arbitrary size.

So: I would imagine that in your case, you set firstlineasschema=False, and set the correct lengths in Schema. That should do it.

This one is hard to test because nowhere is it documented what this Schema: TStringList should contain inside it.

A possible extension is that the schema is defined in the FieldDefs property, and that this is used to parse the file.

Ok, so it seams to me that the Schema should not actually exist, because all information it contains is redundant with FieldDefs. Setting a size in FieldDefs has no effect.

ID,NAMEEN,NAMEPT,HEIGHT,WIDTH,PINS,DRAWINGCODE = 2048 works for me, it returns the fieldsize of 53 (the character count of the first row) for ID, NAMEEN, NAMEPT, HEIGHT, WIDTH and PINS and 2048 for DRAWINGCODE.

MSEgui tstatfile or ttextdatastream are other options, probably Lazarus has similar components?

Martin

Is Sqlite3 no option for you? MSEgui and FPC sqldb both have a DB-connection component for SQLite3.

FWIW I had the problem of string fields limited to 255 characters with sqlite3 and zeos.
I hadn't time to test other combinations.
Since I only
needed the db aware components to display (not even edit) fields of less than 255 chars, I simply used direct queries to manipulate fields longer than 255 chars.
Like Felipe I couldn't trace the source of that
limitation.

Bye

There is no such limitation with MSEgui tmsesqlquery and tsqlite3connection AFAIK.

Martin