Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2014-12

Log4D weirdness

10 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Mario R. Carro

Hello everyone, I've been debgging a weird problem I was having with a little app of mine. I'm using the latest stable Lazarus and FreePascal. The same project on Windows runs fine. On Linux (Debian Sid) fires an exception when running the finalization code.

I narrowed the problem to the use of Log4D (svn head from SF). Simply adding Log4D to the uses list is enough to make the application fail on exit.

Long-history-short, the culprit code is this:

constructor TLogLog.Create;
begin
   inherited Create('');
   Appender := ;
   AddAppender(TLogODSAppender.Create(''));
   InternalDebugging := False;
   Level             := Log4D.Debug;
end;

There's a LogLog variable that is destroyed in the finalization section (and fails). The problem is that the TLogODSAppender created in the ctor ends up destroyed inside the AddAppender call. This is the AddAppender code:

procedure TLogLogger.AddAppender(const Appender: ILogAppender);
begin
   LockLogger;
   try
     if FAppenders.IndexOf(Appender) = -1 then
     begin
       FAppenders.Add(Appender);
       if FHierarchy  nil then
         FHierarchy.FireAppenderEvent(True, Self, Appender);
     end;
   finally
     UnlockLogger;
   end;
end;

The appender dtor is called inside the call to IndexOf (it seems by the stack trace because it's use-count reaches zero). So: The appender is destroyed before is adding it to the list and later the TLogLog dtor fails when tries to destroy it again.

I already solved the problem, but don't know why. Changing the TLogLog.Create code by this one makes the problem disappear:

constructor TLogLog.Create;
var
   Appender: ILogAppender;
begin
   inherited Create('');
   Appender := TLogODSAppender.Create('');
   AddAppender(Appender);
   InternalDebugging := False;
   Level             := Log4D.Debug;
end;

Who is at fault here? Log4D? The compiler? Me?
Any hints will be appreciated...

/MRC

Reply by email

Marco van de Voort
to Mario R. Carro
Mario R. Carro wrote 18 lines
There's a LogLog variable that is destroyed in the finalization section (and fails). The problem is that the TLogODSAppender created in the ctor ends up destroyed inside the AddAppender call. This is the AddAppender code: procedure TLogLogger.AddAppender(const Appender: ILogAppender); begin LockLogger; try if FAppenders.IndexOf(Appender) = -1 then begin FAppenders.Add(Appender); if FHierarchy nil then FHierarchy.FireAppenderEvent(True, Self, Appender); end; finally UnlockLogger; end; end;

Yes, this is a known kind of buggy code. (buildin assumptions of Delphi code that only "happens" to work on Delphi). Delphi /usually/ destroys automatically created temps at
the end of a function, while FPC can also do it after a block.

Best is to explicitely keep a local variable reference, local variable references are kept to the end at the moment, as you already found out.

Who is at fault here? Log4D? The compiler? Me?

Hard to say, there is some opinion involved there.

Like many cases, Delphi does not specify this behaviour, and in general for Delphi not much is known what of the implementation is intended, or what is coincidence. This is aggrevated by the fact that too much Delphi code messing with interface refcounting seems to be created with trial and error.

However it has come up a zillion times in the bugtracker, and FPC compiler devels have stated that they won't fix it (since it could inhibit/complicate implementing certain optimizations in the future), and the code is shoddy to start with.

Best is to root out these kinds of bugs in the relevant projects (in this case Log4D)

Reply by email

Hans-Peter Diettrich
to Marco van de Voort
Marco van de Voort wrote 6 lines
On Fri, Dec 19, 2014 at 06:49:07PM -0300, Mario R. Carro wrote: > There's a LogLog variable that is destroyed in the finalization section > (and fails). The problem is that the TLogODSAppender created in the ctor > ends up destroyed inside the AddAppender call. This is the AddAppender code: > > procedure TLogLogger.AddAppender(const Appender: ILogAppender);

[...]

Show quoted text 13 lines
Yes, this is a known kind of buggy code. (buildin assumptions of Delphi code that only "happens" to work on Delphi). Delphi /usually/ destroys automatically created temps at the end of a function, while FPC can also do it after a block. Best is to explicitely keep a local variable reference, local variable references are kept to the end at the moment, as you already found out. > Who is at fault here? Log4D? The compiler? Me? Hard to say, there is some opinion involved there. Like many cases, Delphi does not specify this behaviour, and in general for Delphi not much is known what of the implementation is intended, or what is coincidence. This is aggrevated by the fact that too much Delphi code messing with interface refcounting seems to be created with trial and error.

Also strings had (have?) problems with refcounting of *const* parameters in Delphi. As a first try I remove the "const", in case of unexpected trouble.

DoDi

Reply by email

Marcos Douglas
to Hans-Peter Diettrich
Hans-Peter Diettrich wrote 34 lines
Am 27.12.2014 um 22:49 schrieb Marco van de Voort: > > On Fri, Dec 19, 2014 at 06:49:07PM -0300, Mario R. Carro wrote: >> >> There's a LogLog variable that is destroyed in the finalization section >> (and fails). The problem is that the TLogODSAppender created in the ctor >> ends up destroyed inside the AddAppender call. This is the AddAppender >> code: >> >> procedure TLogLogger.AddAppender(const Appender: ILogAppender); [...] > > Yes, this is a known kind of buggy code. (buildin assumptions of Delphi > code that only "happens" to work on Delphi). Delphi /usually/ destroys > automatically created temps at the end of a function, while FPC can also do > it after a block. Best is to explicitely keep a local variable reference, > local variable references are kept to the end at the moment, as you already > found out. >> >> Who is at fault here? Log4D? The compiler? Me? > > Hard to say, there is some opinion involved there. > > Like many cases, Delphi does not specify this behaviour, and in general > for > Delphi not much is known what of the implementation is intended, or what > is > coincidence. This is aggrevated by the fact that too much Delphi code > messing with interface refcounting seems to be created with trial and > error. Also strings had (have?) problems with refcounting of *const* parameters in Delphi. As a first try I remove the "const", in case of unexpected trouble.

Delphi and FPC have problems with refcounting -- but I'm referring only interfaces.
Circular references is a big problem too.

This link could help:
http://blog.synopse.info/post/2012/06/18/Circular-reference-and-zeroing-weak-pointers

Regards,
Marcos Douglas

Reply by email

Marco van de Voort
to Hans-Peter Diettrich: [...] Also strings had (have?) problems with refcounting of *const* parameters in Delphi.…
Hans-Peter Diettrich wrote 7 lines
> Like many cases, Delphi does not specify this behaviour, and in general for > Delphi not much is known what of the implementation is intended, or what is > coincidence. This is aggrevated by the fact that too much Delphi code > messing with interface refcounting seems to be created with trial and error. Also strings had (have?) problems with refcounting of *const* parameters in Delphi. As a first try I remove the "const", in case of unexpected trouble.

Afaik the problem is that "const" as used in classic delphi is not very portable. In theory it is specified as immutable (can't be modified), but widely abused to mean "by reference". This happens to be the case for x86 ABIs, but this is not portable (which why FPC introduced constref)

Reply by email

Marco van de Voort
to Marcos Douglas: On Sun, Dec 28, 2014 at 1:03 AM, Hans-Peter Diettrich Delphi and FPC have problems with…
Marcos Douglas wrote 5 lines
> Delphi. As a first try I remove the "const", in case of unexpected trouble. Delphi and FPC have problems with refcounting -- but I'm referring only interfaces. Circular references is a big problem too.

This is totally new for me. I have used interfaces a lot, and while I had trouble from time to time, I haven't really encountered these.

IMHO wrong. GC also has problems, e.g. when two roots circularly reference eachother. The major implementations are somewhat hardened against it, but more involved reference counting implementations (like Python) are too.

Delphi/FPC reference counting is simple and cheap. The problem is IMHO not the implementation, but the fact that people try to abuse it for things it wasn't meant for (a holistic automatic memory management solution).

ARC is IMHO no solution but only damage control (by explicitely naming pure references). IIRC Python has similar tricks to keep circular detections cheap (cycle checking can stop at a weak reference)

Reply by email

Marcos Douglas
to Marco van de Voort
Marco van de Voort wrote 9 lines
On Sun, Dec 28, 2014 at 07:51:25PM -0200, Marcos Douglas wrote: > > Delphi. As a first try I remove the "const", in case of unexpected trouble. > > Delphi and FPC have problems with refcounting -- but I'm referring > only interfaces. > Circular references is a big problem too. This is totally new for me. I have used interfaces a lot, and while I had trouble from time to time, I haven't really encountered these.

New? What do you mean?
This problem was talked in FPC list and here before. I use "weak reference". Somethings are not possible to implement without this "technique".
For example:
https://github.com/mdbs99/AWS/blob/master/src/aws_s3.pas#L166 https://github.com/mdbs99/AWS/blob/master/src/aws_s3.pas#L304

These objects have circular reference and they need weak reference to keep memory safe.

Show quoted text 6 lines
> This link could help: > http://blog.synopse.info/post/2012/06/18/Circular-reference-and-zeroing-weak-pointers IMHO wrong. GC also has problems, e.g. when two roots circularly reference eachother. The major implementations are somewhat hardened against it, but more involved reference counting implementations (like Python) are too.

I know some others approaches have problems, but Delphi/FPC is not perfect too.

Delphi/FPC reference counting is simple and cheap. The problem is IMHO not the implementation, but the fact that people try to abuse it for things it wasn't meant for (a holistic automatic memory management solution).

I agree.

ARC is IMHO no solution but only damage control (by explicitely naming pure references). IIRC Python has similar tricks to keep circular detections cheap (cycle checking can stop at a weak reference)

Happy New year!

Regards,
Marcos Douglas

Reply by email

Marco van de Voort
to Marcos Douglas
Marcos Douglas wrote 14 lines
> > This is totally new for me. I have used interfaces a lot, and while I had > trouble from time to time, I haven't really encountered these. New? What do you mean? This problem was talked in FPC list and here before. I use "weak reference". Somethings are not possible to implement without this "technique". For example: https://github.com/mdbs99/AWS/blob/master/src/aws_s3.pas#L166 https://github.com/mdbs99/AWS/blob/master/src/aws_s3.pas#L304 These objects have circular reference and they need weak reference to keep memory safe.

Ah, you mean that. I didn't consider that an interface problem, since the ref is in an object. (whose aren't refcounted). IOW by design :-)

Show quoted text 5 lines
> eachother. The major implementations are somewhat hardened against it, but > more involved reference counting implementations (like Python) are too. I know some others approaches have problems, but Delphi/FPC is not perfect too.

Certainly. It doesn't even try, and as soon as you try to overuse it, it breaks down. It is IMHO a designchoice and not a flaw though.

Show quoted text 5 lines
> Delphi/FPC reference counting is simple and cheap. The problem is IMHO not > the implementation, but the fact that people try to abuse it for things it > wasn't meant for (a holistic automatic memory management solution). I agree.

Then why do you write such code ? :-)

Show quoted text 5 lines
> ARC is IMHO no solution but only damage control (by explicitely naming pure > references). IIRC Python has similar tricks to keep circular detections > cheap (cycle checking can stop at a weak reference) Happy New year!

Happy New Year. May 3.0 be all that we hope it will be.

Reply by email

Marcos Douglas
to Marco van de Voort
Marco van de Voort wrote 18 lines
On Thu, Jan 01, 2015 at 01:04:22AM -0200, Marcos Douglas wrote: > > > > This is totally new for me. I have used interfaces a lot, and while I had > > trouble from time to time, I haven't really encountered these. > > New? What do you mean? > This problem was talked in FPC list and here before. > I use "weak reference". Somethings are not possible to implement > without this "technique". > For example: > https://github.com/mdbs99/AWS/blob/master/src/aws_s3.pas#L166 > https://github.com/mdbs99/AWS/blob/master/src/aws_s3.pas#L304 > > These objects have circular reference and they need weak reference to > keep memory safe. Ah, you mean that. I didn't consider that an interface problem, since the ref is in an object. (whose aren't refcounted). IOW by design :-)

If I understood right, you mean it only uses interfaces but not refcount?

Show quoted text 16 lines
> > eachother. The major implementations are somewhat hardened against it, but > > more involved reference counting implementations (like Python) are too. > > I know some others approaches have problems, but Delphi/FPC is not perfect > too. Certainly. It doesn't even try, and as soon as you try to overuse it, it breaks down. It is IMHO a designchoice and not a flaw though. > > Delphi/FPC reference counting is simple and cheap. The problem is IMHO not > > the implementation, but the fact that people try to abuse it for things it > > wasn't meant for (a holistic automatic memory management solution). > > I agree. Then why do you write such code ? :-)

I didn't understand. What code, AWS Lib? If yes, well I wrote such code because I know the workarounds to limit such problems with interfaces, refcount and circular reference. But is not easy, memleaks could happen. Unit tests and -gh argument have helping a lot.

Show quoted text 7 lines
> > ARC is IMHO no solution but only damage control (by explicitely naming pure > > references). IIRC Python has similar tricks to keep circular detections > > cheap (cycle checking can stop at a weak reference) > > Happy New year! Happy New Year. May 3.0 be all that we hope it will be.

WOW! I can not wait for it!

Marcos Douglas

Reply by email

Hans-Peter Diettrich
to Marco van de Voort: Afaik the problem is that "const" as used in classic delphi is not very portable. In…
Marco van de Voort wrote
Afaik the problem is that "const" as used in classic delphi is not very portable. In theory it is specified as immutable (can't be modified), but widely abused to mean "by reference".

Const parameters exist for optimization, the compiler is allowed to pass them by reference instead of a copy.

This happens to be the case for x86 ABIs, but this is not portable (which why FPC introduced constref)

I don't see how this is not portable - as long as the procedure is OPL. External procedures of other languages don't leave room for such an optimization, even in x86 ABIs, that's why constref was introduced for them.

DoDi

Reply by email