Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2005-07

DB Bug in 2.0.1?

17 messages · 8 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Tony Maro

I've got an app built with 2.0.0 using TDbf 6.4.3. All of the date fields work fine.

I tried building this app with 2.0.1 from July 5th (and again from today's 2.0.1 snapshot) of FPC and all date fields are reading as "12/31/0000" when read as both a string and as a TDateTime. I use the same download of TDbf in both, not the TDbf that's packaged with FPC.

This happens both in code and within data aware components.

I've also noticed that the TDBGrid component in Lazarus no longer allows reliable editing of the data. Most of the time it just reverts the cell back to it's previous value, but sometimes it will let the change save (and I'm not sure why it's not consistent.) I'm assuming this might be related to the recent db subsystem changes?

Can someone confirm the date issue for me?

Thanks,
Tony Maro

Jonas Maebe
to Tony Maro
Tony Maro wrote
Can someone confirm the date issue for me?

I can't confirm it, but I do want to mention that I fixed all datetime-related routines for *nix platforms in sysutils a while ago. They operated based on the supposition that the dates they get are in Windows format, as opposed to the number of seconds since x/x/1970.

Maybe some databases always store the date in Windows format and maybe some database units counted on this bug...

Jonas

Tony Maro
to Jonas Maebe
Jonas Maebe wrote 13 lines
On 21 jul 2005, at 00:40, Tony Maro wrote: > Can someone confirm the date issue for me? I can't confirm it, but I do want to mention that I fixed all datetime-related routines for *nix platforms in sysutils a while ago. They operated based on the supposition that the dates they get are in Windows format, as opposed to the number of seconds since x/x/1970. Maybe some databases always store the date in Windows format and maybe some database units counted on this bug...

I wouldn't think so in this case - it's not just returning the wrong date, it's always returning the SAME wrong date, regardless of the value in the database table. But I won't rule it out as a possibility. I'll point Micha to this thread.

-Tony

Tony Maro
to Jonas Maebe: I can't confirm it, but I do want to mention that I fixed all datetime-related routines…
Jonas Maebe wrote 10 lines
On 21 jul 2005, at 00:40, Tony Maro wrote: > Can someone confirm the date issue for me? I can't confirm it, but I do want to mention that I fixed all datetime-related routines for *nix platforms in sysutils a while ago. They operated based on the supposition that the dates they get are in Windows format, as opposed to the number of seconds since x/x/1970.

I have a concern on that - and please keep in mind I've not even looked at the code, so I don't know if it's an issue or not.

My main purpose in using FPC is to write cross-platform applications. From what I gather, I'll now need to understand that TDateTime doesn't actually store things the same way on different OS's? I know, FPC's goal is not specifically Delphi compatibility, but...

Here's a quote I found online:
"In Delphi for Win32 (and Kylix), TDateTime is defined as a Double (64-bit floating-point number). Date and time information is stored as the count of days since midnight on 30-Dec-1899."

Here you can see that even Kylix uses the same methodology in storing TDateTime values.

I can see both perspectives, but if the idea is to make it easier to develop cross-platform software, I'd expect TDateTime to be stored the same way across platforms regardless of how the OS counts. A lot of software written for Delphi is going to be awfully hard to convert for Linux, Mac, etc if TDateTime is stored differently everywhere.

-Tony

Michael Van Canneyt
to Tony Maro
Tony Maro wrote 28 lines
Jonas Maebe wrote: > > On 21 jul 2005, at 00:40, Tony Maro wrote: > > > Can someone confirm the date issue for me? > > > I can't confirm it, but I do want to mention that I fixed all > datetime-related routines for *nix platforms in sysutils a while ago. > They operated based on the supposition that the dates they get are in > Windows format, as opposed to the number of seconds since x/x/1970. I have a concern on that - and please keep in mind I've not even looked at the code, so I don't know if it's an issue or not. My main purpose in using FPC is to write cross-platform applications. From what I gather, I'll now need to understand that TDateTime doesn't actually store things the same way on different OS's? I know, FPC's goal is not specifically Delphi compatibility, but... Here's a quote I found online: "In Delphi for Win32 (and Kylix), TDateTime is defined as a Double (64-bit floating-point number). Date and time information is stored as the count of days since midnight on 30-Dec-1899." Here you can see that even Kylix uses the same methodology in storing TDateTime values.

The TDateTime value itself is not the issue. It is the same on all platforms, and compatible to Delpi.

Show quoted text 6 lines
I can see both perspectives, but if the idea is to make it easier to develop cross-platform software, I'd expect TDateTime to be stored the same way across platforms regardless of how the OS counts. A lot of software written for Delphi is going to be awfully hard to convert for Linux, Mac, etc if TDateTime is stored differently everywhere.

It is not.

If I remember correctly, there have been some fixes on the internal Date/Time field buffer format of TDataset, and I suppose your problem is a result of that change.

Michael.

Michalis Kamburelis
to Tony Maro: I have a concern on that - and please keep in mind I've not even looked at the code, so I…
Tony Maro wrote 37 lines
Jonas Maebe wrote: > > On 21 jul 2005, at 00:40, Tony Maro wrote: > >> Can someone confirm the date issue for me? > > > > I can't confirm it, but I do want to mention that I fixed all > datetime-related routines for *nix platforms in sysutils a while ago. > They operated based on the supposition that the dates they get are in > Windows format, as opposed to the number of seconds since x/x/1970. I have a concern on that - and please keep in mind I've not even looked at the code, so I don't know if it's an issue or not. My main purpose in using FPC is to write cross-platform applications. From what I gather, I'll now need to understand that TDateTime doesn't actually store things the same way on different OS's? I know, FPC's goal is not specifically Delphi compatibility, but... Here's a quote I found online: "In Delphi for Win32 (and Kylix), TDateTime is defined as a Double (64-bit floating-point number). Date and time information is stored as the count of days since midnight on 30-Dec-1899." Here you can see that even Kylix uses the same methodology in storing TDateTime values. I can see both perspectives, but if the idea is to make it easier to develop cross-platform software, I'd expect TDateTime to be stored the same way across platforms regardless of how the OS counts. A lot of software written for Delphi is going to be awfully hard to convert for Linux, Mac, etc if TDateTime is stored differently everywhere.

AFAIK TDateTime internal format is the same for all platforms, and compatible to Delphi. I guess that Jonas meant his change at revision 444, see
svn diff -r 443:444
These fixes concern DateTimeToFileDate and FileDateToDateTime. The FileDate meaning is OS-dependent (and Jonas fixed DateTimeToFileDate and FileDateToDateTime to work accordingly).

(It's worth mentioning that good code shouldn't rely on internal TDateTime format anyway. Usually TDateTime should be treated as opaque type, and operated only using standard routines, like the ones from DateUtils unit.)

Michalis

Michael Van Canneyt
to Joost van der Sluis: It's an known issue. There was some discussion about which format to use internally for…
Joost van der Sluis wrote 16 lines
> I tried building this app with 2.0.1 from July 5th (and again from > today's 2.0.1 snapshot) of FPC and all date fields are reading as > "12/31/0000" when read as both a string and as a TDateTime. I use the > same download of TDbf in both, not the TDbf that's packaged with FPC. > This happens both in code and within data aware components. It's an known issue. There was some discussion about which format to use internally for date/time-fields. That the Delphi help stated that TDateTime should be used was not accepted as an argument then, so it's changed to something else. Now I did some tests and I know that Delphi 7 uses TDateTime internally, so it'll be changed back to that. I have the patches ready, but need to do more tests with all supported databases.

Because of the bug report I did some research on this as well. Up to and including D4, a TDateTimeRec was used internally. As of Delphi 5, TDateTime was used.

Since the first FPC TDataset implementation was based on a D3 spec, this explains the confusion.

At the same time I discovered some bugs in the rtl regarding TDateTime's, i'll send a fix for that too. I think it all will be done in the weekend.

Send me the fix if you want.

Michael.

Alexandrov Alexandru
to Michael Van Canneyt: The TDateTime value itself is not the issue. It is the same on all platforms, and…
Michael Van Canneyt wrote 55 lines
On Wed, 20 Jul 2005, Tony Maro wrote: > Jonas Maebe wrote: > > > > > On 21 jul 2005, at 00:40, Tony Maro wrote: > > > > > Can someone confirm the date issue for me? > > > > > > I can't confirm it, but I do want to mention that I fixed all > > datetime-related routines for *nix platforms in sysutils a while ago. > > They operated based on the supposition that the dates they get are in > > Windows format, as opposed to the number of seconds since x/x/1970. > > I have a concern on that - and please keep in mind I've not even looked at the > code, so I don't know if it's an issue or not. > > My main purpose in using FPC is to write cross-platform applications. From > what I gather, I'll now need to understand that TDateTime doesn't actually > store things the same way on different OS's? I know, FPC's goal is not > specifically Delphi compatibility, but... > > Here's a quote I found online: > "In Delphi for Win32 (and Kylix), TDateTime is defined as a Double (64-bit > floating-point number). Date and time information is stored as the count of > days since midnight on 30-Dec-1899." > > Here you can see that even Kylix uses the same methodology in storing > TDateTime values. The TDateTime value itself is not the issue. It is the same on all platforms, and compatible to Delpi. > > I can see both perspectives, but if the idea is to make it easier to develop > cross-platform software, I'd expect TDateTime to be stored the same way across > platforms regardless of how the OS counts. A lot of software written for > Delphi is going to be awfully hard to convert for Linux, Mac, etc if TDateTime > is stored differently everywhere. It is not. If I remember correctly, there have been some fixes on the internal Date/Time field buffer format of TDataset, and I suppose your problem is a result of that change. Michael. _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

I think you need at least dbf_common.inc from fpc package.

Alex

Joost van der Sluis
to Tony Maro: I've got an app built with 2.0.0 using TDbf 6.4.3. All of the date fields work fine. I…
Show quoted text 6 lines
I tried building this app with 2.0.1 from July 5th (and again from today's 2.0.1 snapshot) of FPC and all date fields are reading as "12/31/0000" when read as both a string and as a TDateTime. I use the same download of TDbf in both, not the TDbf that's packaged with FPC. This happens both in code and within data aware components.

It's an known issue. There was some discussion about which format to use internally for date/time-fields. That the Delphi help stated that TDateTime should be used was not accepted as an argument then, so it's changed to something else.

Now I did some tests and I know that Delphi 7 uses TDateTime internally, so it'll be changed back to that. I have the patches ready, but need to do more tests with all supported databases.

At the same time I discovered some bugs in the rtl regarding TDateTime's, i'll send a fix for that too. I think it all will be done in the weekend.

Show quoted text 5 lines
I've also noticed that the TDBGrid component in Lazarus no longer allows reliable editing of the data. Most of the time it just reverts the cell back to it's previous value, but sometimes it will let the change save (and I'm not sure why it's not consistent.) I'm assuming this might be related to the recent db subsystem changes?

Is that fpc-related or lazarus-related? (ie: does it work with fpc 2.0.0?) Several people including myself reported this, but Jesus Reyes can't reproduce it. (If it's fpc-related, i'll have a look at it)

Met vriendelijke groeten, Joost van der Sluis CNOC Informatiesystemen en Netwerken http://www.cnoc.nl
Micha Nelissen
to Michael Van Canneyt: Because of the bug report I did some research on this as well. Up to and including D4, a…
Show quoted text 6 lines
Because of the bug report I did some research on this as well. Up to and including D4, a TDateTimeRec was used internally. As of Delphi 5, TDateTime was used. Since the first FPC TDataset implementation was based on a D3 spec, this explains the confusion.

Yes, but FPC already used TDateTime internally, according to TDbf :-)

dbf_common.inc:

{$ifdef DELPHI_5}

...

  {$define SUPPORT_NEW_FIELDDATA}

...

SUPPORT_NEW_FIELDDATA means use TDateTime as internal format. Under {$ifdef FPC} I also have defined SUPPORT_NEW_FIELDDATA.

(I know: the SUPPORT_ should be named HAS_ or so...)

Regards from the not entirely ignorant tdbf maintainer as proven by this mail ;-)

Micha

Alexandrov Alexandru
to Micha Nelissen
Micha Nelissen wrote 32 lines
On Thu, 21 Jul 2005 06:51:12 +0200 (CEST) Michael Van Canneyt wrote: > Because of the bug report I did some research on this as well. > Up to and including D4, a TDateTimeRec was used internally. > As of Delphi 5, TDateTime was used. > > Since the first FPC TDataset implementation was based on a D3 spec, > this explains the confusion. Yes, but FPC already used TDateTime internally, according to TDbf :-) dbf_common.inc: {$ifdef DELPHI_5} ... {$define SUPPORT_NEW_FIELDDATA} ... SUPPORT_NEW_FIELDDATA means use TDateTime as internal format. Under {$ifdef FPC} I also have defined SUPPORT_NEW_FIELDDATA. (I know: the SUPPORT_ should be named HAS_ or so...) Regards from the not entirely ignorant tdbf maintainer as proven by this mail ;-) Micha _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

It is not about SUPPORT_NEW_FIELDDATA, but
SUPPORT_BACKWARD_FIELDDATA.

In fcp package, in dbf_common.inc it is defined
SUPPORT_BACKWARD_FIELDDATA but in
last package on sourceforge not.

Alex

Micha Nelissen
to Alexandrov Alexandru
Show quoted text 19 lines
On 7/21/05, Micha Nelissen wrote: > On Thu, 21 Jul 2005 06:51:12 +0200 (CEST) > Michael Van Canneyt wrote: > > > Because of the bug report I did some research on this as well. > > Up to and including D4, a TDateTimeRec was used internally. > > As of Delphi 5, TDateTime was used. > > > > Since the first FPC TDataset implementation was based on a D3 spec, > > this explains the confusion. > > Yes, but FPC already used TDateTime internally, according to TDbf :-) It is not about SUPPORT_NEW_FIELDDATA, but SUPPORT_BACKWARD_FIELDDATA. In fcp package, in dbf_common.inc it is defined SUPPORT_BACKWARD_FIELDDATA but in last package on sourceforge not.

I see, the backward compatible translation thingy. Why didn't you report this to me ?

'Official' TDbf has never had the SUPPORT_BACKWARD_FIELDDATA defined for FPC.

Micha

Alexandrov Alexandru
to Micha Nelissen
Micha Nelissen wrote 33 lines
On Thu, 21 Jul 2005 13:25:05 +0300 Alexandrov Alexandru wrote: > On 7/21/05, Micha Nelissen wrote: > > On Thu, 21 Jul 2005 06:51:12 +0200 (CEST) > > Michael Van Canneyt wrote: > > > > > Because of the bug report I did some research on this as well. > > > Up to and including D4, a TDateTimeRec was used internally. > > > As of Delphi 5, TDateTime was used. > > > > > > Since the first FPC TDataset implementation was based on a D3 spec, > > > this explains the confusion. > > > > Yes, but FPC already used TDateTime internally, according to TDbf :-) > > It is not about SUPPORT_NEW_FIELDDATA, but > SUPPORT_BACKWARD_FIELDDATA. > > In fcp package, in dbf_common.inc it is defined > SUPPORT_BACKWARD_FIELDDATA but in > last package on sourceforge not. I see, the backward compatible translation thingy. Why didn't you report this to me ? 'Official' TDbf has never had the SUPPORT_BACKWARD_FIELDDATA defined for FPC. Micha _______________________________________________ fpc-devel maillist - http://lists.freepascal.org/mailman/listinfo/fpc-devel

I have sent an patch (2 month ago) to improve the compatibility with some older tdataset components.
I have included corrections to all related components: interbase, mysql3 and 4, sqldb, tdbf.
I don't know who is maintainer for each component.

this was about tdbf:

--- dbase/dbf_common.inc	(revision 171)
+++ dbase/dbf_common.inc	(working copy)
@@ -170,6 +170,7 @@
   {$define SUPPORT_INT64}
   {$define SUPPORT_DEFAULT_PARAMS}
   {$define SUPPORT_NEW_TRANSLATE}
+  {$define SUPPORT_BACKWARD_FIELDDATA}
   {$define SUPPORT_NEW_FIELDDATA}
   {$define SUPPORT_FIELDDEF_TPERSISTENT}
   {$define SUPPORT_FIELDTYPES_V4}

Alex

Tony Maro
to Joost van der Sluis: It's an known issue. There was some discussion about which format to use internally for…
Joost van der Sluis wrote 12 lines
>I've also noticed that the TDBGrid component in Lazarus no longer allows >reliable editing of the data. Most of the time it just reverts the cell >back to it's previous value, but sometimes it will let the change save >(and I'm not sure why it's not consistent.) I'm assuming this might be >related to the recent db subsystem changes? > > Is that fpc-related or lazarus-related? (ie: does it work with fpc 2.0.0?) Several people including myself reported this, but Jesus Reyes can't reproduce it. (If it's fpc-related, i'll have a look at it)

I'm not sure to be honest with you. You can't compile the latest Lazarus CVS with FPC 2.0.0, and the problem doesn't exist in my 2.0.0 FPC + older Lazarus. It was introduced on my PC when I upgraded my FPC to 2.0.1 so I could update my Lazarus... - meaning I updated both then found the problem, so I don't know which causes it.

I was assuming that if the TDataSet components were changed recently that it might be the cause...

I'll see if I can't make an already-compiled demonstration app and send it to Jesus Reyes so he can see it happen.

Thanks,
Tony

Micha Nelissen
to Alexandrov Alexandru: I have sent an patch (2 month ago) to improve the compatibility with some older tdataset…
Show quoted text 5 lines
I have sent an patch (2 month ago) to improve the compatibility with some older tdataset components. I have included corrections to all related components: interbase, mysql3 and 4, sqldb, tdbf. I don't know who is maintainer for each component.

Ah, I didn't see it come by. But now you know who it is for TDbf anyway :-)

Micha

Tony Maro
to Tony Maro: I'm not sure to be honest with you. You can't compile the latest Lazarus CVS with FPC…
Tony Maro wrote 25 lines
Joost van der Sluis wrote: >> I've also noticed that the TDBGrid component in Lazarus no longer >> allows reliable editing of the data. Most of the time it just >> reverts the cell back to it's previous value, but sometimes it will >> let the change save (and I'm not sure why it's not consistent.) I'm >> assuming this might be related to the recent db subsystem changes? >> > > > Is that fpc-related or lazarus-related? (ie: does it work with fpc > 2.0.0?) Several people including myself reported this, but Jesus Reyes > can't reproduce it. (If it's fpc-related, i'll have a look at it) > I'm not sure to be honest with you. You can't compile the latest Lazarus CVS with FPC 2.0.0, and the problem doesn't exist in my 2.0.0 FPC + older Lazarus. It was introduced on my PC when I upgraded my FPC to 2.0.1 so I could update my Lazarus... - meaning I updated both then found the problem, so I don't know which causes it. I was assuming that if the TDataSet components were changed recently that it might be the cause... I'll see if I can't make an already-compiled demonstration app and send it to Jesus Reyes so he can see it happen.

Okay we found the issue - or at least how to reliably reproduce it. Talk about a bang your head against the wall problem. It has to do with HOW you start editing a field. I double-click the cell to get edit mode
- Jesus was single clicking and then typing.

-Tony