Hello,

something broke in sdfdataset with fpc 3.2.0 (a segfault that doesn't occur if I compile the same program with fpc 3.0.4) and I'm trying to debug it.

First I tried to compile just the fcl with debug information

cd fpc-3.2.0/packages
OPT=-gl make
sudo make install

but lazarus then complained that it couldn't find the DB unit (?), so I recompiled everything:

cd fpc-3.2.0
make distclean
OPT=-gl make all
sudo make install

but even so I'm missing some debug info. This is what I see in the call stack windows when I put a breakpoint on TFixedFormatDataSet.GetRecord (file fpc-3.2.0/packages/fcl-db/src/sdf/sdfdata.pp)

#0 GETRECORD(0x7fffe3949370, 0x0, GMNEXT, false) at fcl-db/src/sdf/sdfdata.pp:546
#1 ?? at :0
#2 ?? at :0
#3 ?? at :0
#4 ?? at :0
#5 ?? at :0
#6 ?? at :0

that's with "copy all", the content of the call stack window is different, i.e. (only the index, location and line columns for brevity)

0 sdfdata.pp (fcl-db/src/sdf/) 546
1 :7FFFFFFFDF58
2 :7FFFFFFFE0A0
3 :00000001
4 :7FFFE3949370
5 :7FFFFFFFE129
6 :00000000

Any hint?

Bye

El 15/7/20 a les 17:20, Luca Olivetti via lazarus ha escrit:

For the record, the segfault happens here

#0 SYSGETMEM_FIXED(544) at ../inc/heap.inc:963
#1 ?? at :0
#2 fpc_ansistr_decr_ref(<error reading variable: Cannot access memory at address 0x30>) at ../inc/astrings.inc:148
#3 ?? at :0

i.e. the call stack is even more wacky, but I know it comes from TFixedFormatDataSet.GetRecord

Bye

El 15/7/20 a les 17:31, Luca Olivetti via lazarus ha escrit:

[...]

> Any hint?

The optimization! I had to use "-g -gl -O-" to get the full call stack.

The memory is hosed, I still don't know why.
I managed to reproduce the problem with a simple lazarus program, however a plain pascal program that does the same doesn't trigger the it.

I'm attaching the lazarus project as well as the simple pascal program if somebody could look into it...

Bye

El 15/7/20 a les 18:26, Luca Olivetti via lazarus ha escrit:

I reported it here

https://bugs.freepascal.org/view.php?id=37370

Bye

El 16/7/20 a les 8:55, Luca Olivetti via lazarus ha escrit:

I found that reverting

https://svn.freepascal.org/cgi-bin/viewvc.cgi?view=revision&revision=43245

(in fact I just took sdfdata.pp from 3.0.4)

fixes the segfault.

Michael, what do you think?

Bye

Seems like a bug somewhere. I'll need to has something to do with the default codepage: Lazarus changes that.

Michael.

El 17/7/20 a les 9:51, Michael Van Canneyt ha escrit:

What worries me is the fact that it is so easy to mess up the heap. What other dragons are lurking ahead in 3.2.0?

For the record, I tried to set the codepage of the component to 'utf-8' and it still segfaults, so I don't think it's because of the default codepage.

Also the fact that some of the TFieldDefs.Add are procedure and some functions doesn't seem right.

Bye

Looking at the bugtracker, there could be several. I think we may need to release a 3.2.2 after a relatively short amount of time.

For the record, I tried to set the codepage of the component to 'utf-8' and it still segfaults, so I don't think it's because of the default codepage.

We'll see.
This is the only relevant change in the component, so that is the most likely candidate. What is more puzzling is that it only happens in Lazarus.

Also the fact that some of the TFieldDefs.Add are procedure and some functions doesn't seem right.

They are simply overloaded versions, but all boil down to the same. Some of these exist for Delphi compatibility. I doubt this is the cause.

Michael.

El 17/7/20 a les 10:26, Michael Van Canneyt via lazarus ha escrit:

In fact, if you look at the bug report, it's enough to use the old version of Fielddefs.Add (i.e. the one without the encoding) to "fix" the problem, but it's not because of the default (I changed it to 'utf-8', or maybe it's actually 'utf-8' that causes the problem).

What is more puzzling is that it only happens in Lazarus.

For the record, in the simple program I tried with 100000 iterations and it still doesn't segfaults.

I'm not saying it's the cause, it just seemed odd.

Bye

El 17/7/20 a les 11:46, Luca Olivetti via lazarus ha escrit:

Yes, setting it to '437' or 'iso8859-1' avoids the segfault.

Bye

Haha... The mystery is unraveling :-)

Note that these are single-byte codepages.
Are you using fixed-length of CSV ? Because in the case of fixed-length, UTF8 makes no sense...

Michael.

El 17/7/20 a les 12:12, Michael Van Canneyt via lazarus ha escrit:

I'm not using the CSV at all, I load an empty file and fill the dataset by other means (Dataset.Fields[f].AsString:=....). However I put a breakpoint on every SetAsString and none is triggered....(the assumption here is that a buffer of length n*4 is actually of length n, that would explain the heap corruption and I wanted to see where it happens).
I any case it doesn't matter if it makes sense or not, it should work, maybe with a wrong encoding but definitely it shouldn't segfault.

Bye

El 17/7/20 a les 12:32, Luca Olivetti via lazarus ha escrit:

However I put a breakpoint on every SetAsString and none is triggered....

And that's an issue with the debugger I think :-(

Bye

El 17/7/20 a les 12:45, Luca Olivetti via lazarus ha escrit:

Sorry, it was a PEBKAC, I was debugging under windows where the rtl/fcl has no debug information.
Back to linux...

Bye

El 17/7/20 a les 12:49, Luca Olivetti via lazarus ha escrit:

I found the problem and added a patch to the bugreport. This is no C folks, "Move" should be banned from the compiler, though I use it myself ;-)

https://bugs.freepascal.org/view.php?id=37370#c124122

Bye