Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2011-03

RFC: customvariant handling in variants.pp and fmtbcd.pp

12 messages · 2 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

LacaK

Hi,
I would like ask for your opinion in this case.

I found, that in this sample program is raised "Invalid variant ..."

var bcd1: TBCD;
    v,v1: variant;
    s: string;
begin
     bcd1:=2;
     v1:=varfmtbcdcreate(bcd1);
     s:=v1; //assigment from customvariant to string is not handled in
sysvartolstr()
end;

I can fix it by following fixs:

1. fmtbcd_castto.diff ... added case when castto varString is requested ... then do not use cast throught varDouble (to avoid lost of precision), but convert directly from TBCD to string

2. variants.pp ... here we must add handling of customvariants into sysvartolstr ... I created "helper" function TryCastFromCustomVariant which can be used multiple times (now in sysvartolstr and sysvartoreal) ... I isolate in this function code which is required multiple times.

I am not sure if
1. is this optimal approach, or is better to put same code repeatedly in sysvartolstrm sysvartoreal and in future in others sysvarto... ?
2. is the name and place of such function good choosen ?

Can somebody look at it, and commit what is good and change what is not good ;-) ?
Thanks
-Laco.

Reply by email

Sergei Gorelkin
to LacaK

LacaK пишет:

Show quoted text 17 lines
I can fix it by following fixs: 1. fmtbcd_castto.diff ... added case when castto varString is requested ... then do not use cast throught varDouble (to avoid lost of precision), but convert directly from TBCD to string 2. variants.pp ... here we must add handling of customvariants into sysvartolstr ... I created "helper" function TryCastFromCustomVariant which can be used multiple times (now in sysvartolstr and sysvartoreal) ... I isolate in this function code which is required multiple times. I am not sure if 1. is this optimal approach, or is better to put same code repeatedly in sysvartolstrm sysvartoreal and in future in others sysvarto... ? 2. is the name and place of such function good choosen ?

Doing what you propose isn't good. Checking for custom variant type is expensive because it involves locking, therefore custom variant handling should be done *after* the standard types. The real problem is that the standard types are currently handled in VarUtils.VariantToX functions, which do not depend on Variants unit and cannot use custom variant stuff.

The correct solution would be to move the code of VariantToX functions to Variants unit, inlining them to sysvartoX functions. But if you try that, you'll discover very soon that VariantChangeTypeEx (non-Windows implementation in varutils.inc) depends on some of them. This is again not correct, because conversion functions called from VariantChangeTypeEx do not need to handle Pascal-specific types and by-reference variants. But a simplifed portion of conversion functions have to stay in VarUtils unit.

This is of course a major rewrite, but I'd stay on it, because with a number of quick-fixes we risk to end up with unmaintainable code.

Regards,
Sergei

Reply by email

LacaK
to Sergei Gorelkin

Hi Sergei,

Show quoted text 21 lines
> > I can fix it by following fixs: > > 1. fmtbcd_castto.diff ... added case when castto varString is > requested ... then do not use cast throught varDouble (to avoid lost > of precision), but convert directly from TBCD to string > > 2. variants.pp ... here we must add handling of customvariants into > sysvartolstr ... I created "helper" function TryCastFromCustomVariant > which can be used multiple times (now in sysvartolstr and > sysvartoreal) ... I isolate in this function code which is required > multiple times. > > I am not sure if > 1. is this optimal approach, or is better to put same code repeatedly > in sysvartolstrm sysvartoreal and in future in others sysvarto... ? > 2. is the name and place of such function good choosen ? > Doing what you propose isn't good. Checking for custom variant type is expensive because it involves locking, therefore custom variant handling should be done *after* the standard types.

ok

The real problem is that the standard types are currently handled in VarUtils.VariantToX functions, which do not depend on Variants unit and cannot use custom variant stuff.

yes

Show quoted text 13 lines
The correct solution would be to move the code of VariantToX functions to Variants unit, inlining them to sysvartoX functions. But if you try that, you'll discover very soon that VariantChangeTypeEx (non-Windows implementation in varutils.inc) depends on some of them. This is again not correct, because conversion functions called from VariantChangeTypeEx do not need to handle Pascal-specific types and by-reference variants. But a simplifed portion of conversion functions have to stay in VarUtils unit. This is of course a major rewrite, but I'd stay on it, because with a number of quick-fixes we risk to end up with unmaintainable code.

ok I agree, but I think, that this is too dificult for me, so I must wait if somebody will do it.

So my questions are:
1. is acceptable apply my patch (or some variation of it) to variants.pp as a temporary solution ? (you can add there hint "must be chaged, when ...")
2. fix for fmtbcd.pp does not affect above mentioned problem, so should you commit it (if is correct from your POV) to not forget about it ?

Thanks
-Laco.

Reply by email

Sergei Gorelkin
to LacaK

LacaK пишет:

Show quoted text 52 lines
Hi Sergei, >> >> I can fix it by following fixs: >> >> 1. fmtbcd_castto.diff ... added case when castto varString is >> requested ... then do not use cast throught varDouble (to avoid lost >> of precision), but convert directly from TBCD to string >> >> 2. variants.pp ... here we must add handling of customvariants into >> sysvartolstr ... I created "helper" function TryCastFromCustomVariant >> which can be used multiple times (now in sysvartolstr and >> sysvartoreal) ... I isolate in this function code which is required >> multiple times. >> >> I am not sure if >> 1. is this optimal approach, or is better to put same code repeatedly >> in sysvartolstrm sysvartoreal and in future in others sysvarto... ? >> 2. is the name and place of such function good choosen ? >> > Doing what you propose isn't good. Checking for custom variant type is > expensive because it involves locking, therefore custom variant > handling should be done *after* the standard types. ok > The real problem is that the standard types are currently handled in > VarUtils.VariantToX functions, which do not depend on Variants unit > and cannot use custom variant stuff. yes > > The correct solution would be to move the code of VariantToX functions > to Variants unit, inlining them to sysvartoX functions. But if you try > that, you'll discover very soon that VariantChangeTypeEx (non-Windows > implementation in varutils.inc) depends on some of them. This is again > not correct, because conversion functions called from > VariantChangeTypeEx do not need to handle Pascal-specific types and > by-reference variants. But a simplifed portion of conversion functions > have to stay in VarUtils unit. > > This is of course a major rewrite, but I'd stay on it, because with a > number of quick-fixes we risk to end up with unmaintainable code. > ok I agree, but I think, that this is too dificult for me, so I must wait if somebody will do it. So my questions are: 1. is acceptable apply my patch (or some variation of it) to variants.pp as a temporary solution ? (you can add there hint "must be chaged, when ...") 2. fix for fmtbcd.pp does not affect above mentioned problem, so should you commit it (if is correct from your POV) to not forget about it ?

Both committed in r17170, with some changes:

In fmtbcd.pp, I used VarDataFromStr method instead of assigning individual fields of TVarData;
if I understand things correctly, this method exist precisely for such cases.

In variants.pp, I used an individual function, with an out-parameter of type AnsiString. Such approach allows to avoid finalization of the intermediate Variant.

Regards,
Sergei

Reply by email

LacaK
to Sergei Gorelkin

Hi Sergei,

Show quoted text 5 lines
>> Doing what you propose isn't good. Checking for custom variant type >> is expensive because it involves locking, therefore custom variant >> handling should be done *after* the standard types. Both committed in r17170, with some changes:

great thanks!

In fmtbcd.pp, I used VarDataFromStr method instead of assigning individual fields of TVarData; if I understand things correctly, this method exist precisely for such cases.

ok thanks,
I have attached minor "rearangement", consider them please

Show quoted text 5 lines
In variants.pp, I used an individual function, with an out-parameter of type AnsiString. Such approach allows to avoid finalization of the intermediate Variant.

ok, thank you very much

I have looked again at function FindCustomVariantType and I see there :

 3590     Result:=(aVarType>=CMinVarType);
 3591     if Result then

So it seems to me, that for standard variant types is control immediately returned, so no performance problem should exist there with entering critical section and so on. Or I missed something ?

Laco.

Reply by email

Sergei Gorelkin
to LacaK

LacaK пишет:

I have attached minor "rearangement", consider them please

Will do. It's possible to go even further and completely exclude finalization and try..finally block, because the intermediate Variant is always of type Double, which doesn't need finalization.

Show quoted text 11 lines
I have looked again at function FindCustomVariantType and I see there : 3590 Result:=(aVarType>=CMinVarType); 3591 if Result then So it seems to me, that for standard variant types is control immediately returned, so no performance problem should exist there with entering critical section and so on. Or I missed something ?

Indeed. Probably, after I wrote a number of improvements and tests for that part, something was left in a locked state in my central processing unit aka brain, making me think otherwise :)

Regards,
Sergei

Reply by email

LacaK
to Sergei Gorelkin
Show quoted text 7 lines
> I have attached minor "rearangement", consider them please > Will do. It's possible to go even further and completely exclude finalization and try..finally block, because the intermediate Variant is always of type Double, which doesn't need finalization.

Thanks!
I will look forward for your patch.

L.

Reply by email

Sergei Gorelkin
to LacaK

12.04.2011 16:18, LacaK пишет:

Show quoted text 7 lines
>> I have attached minor "rearangement", consider them please >> > Will do. It's possible to go even further and completely exclude finalization and try..finally > block, because the intermediate Variant is always of type Double, which doesn't need finalization. See please attached patch if is it

Sorry for the delay, got stuck with other issues... Applied in r17319. Thank you for the patch!

Sergei

Reply by email

Sergei Gorelkin
to LacaK

15.04.2011 9:20, LacaK пишет:

Show quoted text 7 lines
Hi Sergei, >> > Sorry for the delay, got stuck with other issues... Applied in r17319. Thank you for the patch! It is ok ;-) Thanks. Could I ask why is there needed VarDataInit ? (only for my interest ;-))

A good question. Formally it is not needed, but in general initializing variables before use remains a good idea. Here the TVarData is passed to VarDataCastTo, and after initializing we can be sure that VarDataCastTo won't find garbage in remaining 4 bytes of structure if it accesses them for whatever reason.

Sergei

Reply by email

LacaK
to Sergei Gorelkin
Show quoted text 8 lines
> Could I ask why is there needed VarDataInit ? (only for my interest ;-)) > A good question. Formally it is not needed, but in general initializing variables before use remains a good idea. Here the TVarData is passed to VarDataCastTo, and after initializing we can be sure that VarDataCastTo won't find garbage in remaining 4 bytes of structure if it accesses them for whatever reason.

Thanks for explain

Laco.

Reply by email