Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2021-06

Anyone an idea were/how to look for the missing merge in 3.0.2

15 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Martin Frb

I have the following issue.

A testcase (LazDebuggerFp/test) works fine. Except for 3.2.0 and 3.2.2 (win 64) with -O3 or -O2

In detail:
- It works in 32 bit   3.2.2     --  O3
- It works in 64 bit  3.0.4 / 3.3.1  -- O1 O2 and O3   (My 3.3.1 is 2 weeks old.)
- It works in 64 bit  LINUX  3.2.2     --  O3
- It works in 64 bit   3.2.0 / 3.2.2  -- O1
- It works in 64 bit   3.2.2  -- O3  with {$implicitexceptions off} in several units

* It does not work in 64bit  3.2.0 / 3.2.2  -O2 or O3  with default
{$implicitexceptions ON}

Yes, it could be a bug in my code, but at this point, my primary suspicion points to a different origin for the trouble.

I could not find the faulty code yet. I suspect that it is a random memory write (dangling pointer) that causes an error long after. And  given that I can't use valgrind....

So anyone can thing of any bug fixed (as it works with 3.3.1), that may not have been merged?
I have not checked 3.2.1 yet. / Will be a while, I am away a couple of days.

Reply by email

Pierre Muller
to Martin Frb

Le 09/06/2021 à 00:31, Martin Frb via fpc-devel a écrit :

Show quoted text 20 lines
I have the following issue. A testcase (LazDebuggerFp/test) works fine. Except for 3.2.0 and 3.2.2 (win 64) with -O3 or -O2 In detail: - It works in 32 bit   3.2.2     --  O3 - It works in 64 bit  3.0.4 / 3.3.1  -- O1 O2 and O3   (My 3.3.1 is 2 weeks old.) - It works in 64 bit  LINUX  3.2.2     --  O3 - It works in 64 bit   3.2.0 / 3.2.2  -- O1 - It works in 64 bit   3.2.2  -- O3  with {$implicitexceptions off} in several units * It does not work in 64bit  3.2.0 / 3.2.2  -O2 or O3  with default {$implicitexceptions ON} Yes, it could be a bug in my code, but at this point, my primary suspicion points to a different origin for the trouble. I could not find the faulty code yet. I suspect that it is a random memory write (dangling pointer) that causes an error long after. And  given that I can't use valgrind.... So anyone can thing of any bug fixed (as it works with 3.3.1), that may not have been merged? I have not checked 3.2.1 yet. / Will be a while, I am away a couple of days.

Martin,

could you check if this relates to bug #38973?

https://bugs.freepascal.org/view.php?id=38973

This bug also does not show in current trunk.

If you get the same pattern of failures, it is most probably related to this wrong optimization.

Pierre

Reply by email

Martin Frb
to Pierre Muller
Pierre Muller wrote 11 lines
could you check if this relates to bug #38973? https://bugs.freepascal.org/view.php?id=38973  This bug also does not show in current trunk.  If you get the same pattern of failures, it is most probably related to this wrong optimization.

Unlikely.

I don't know exactly where the error is caused. I usually end up in some internal write_ansi_string. I checked asm and register values. And at that time the string is corrupted. But I do not know when it got corrupted. Could have been anywhere else.

I have not yet tried -OoNOPEEPHOLE => I am away a few days, will try next week.

However the fact that not compiling in "implicit finalization" does fix it points to something else.

Reply by email

Martin Frb
to Martin Frb
Martin Frb wrote 26 lines
On 09/06/2021 09:29, Pierre Muller via fpc-devel wrote: > > > > could you check if this relates to bug #38973? > > https://bugs.freepascal.org/view.php?id=38973 > >  This bug also does not show in current trunk. > >  If you get the same pattern of failures, it is most probably related > to this wrong optimization. Unlikely. I don't know exactly where the error is caused. I usually end up in some internal write_ansi_string. I checked asm and register values. And at that time the string is corrupted. But I do not know when it got corrupted. Could have been anywhere else. I have not yet tried -OoNOPEEPHOLE => I am away a few days, will try next week. However the fact that not compiling in "implicit finalization" does fix it points to something else.

ok, just taken the 5 minutes. -OoNOPEEPHOLE  does also seem to fix it.

Reply by email

Martin Frb
to Pierre Muller: Le 09/06/2021 à 00:31, Martin Frb via fpc-devel a écrit : could you check if this…
Pierre Muller wrote 11 lines
  Martin, could you check if this relates to bug #38973? https://bugs.freepascal.org/view.php?id=38973  This bug also does not show in current trunk.  If you get the same pattern of failures, it is most probably related to this wrong optimization.

Unfortunately I had no success in narrowing it down.

I had hoped that generating -al
and moving the {$implicitexceptions off} around to narrow it to a single function,
I would get a small enough asm diff to tell....

But even narrowing it down to a single "end;" statement that needs to have {$implicitexceptions off} does not help.
This line affects the entire specialization of a base class for one of the classes in that unit. (I did -Si- too)
So the diff is still to big, and many of the changes are outside of what I can judge to be ok or not.

Reply by email

J. Gareth Moreton
to Martin Frb

Ah curses.  I fear it was something I did.  I don't like the idea of stepping through each revision until we find one that works.

https://bugs.freepascal.org/view.php?id=38294
https://bugs.freepascal.org/view.php?id=38334
https://bugs.freepascal.org/view.php?id=38339
https://bugs.freepascal.org/view.php?id=38343

Those ones relate to buggy optimisations in some way, but they might be a bit too late on to affect 3.2.0.

There are quite a few others too - hmmm.

Gareth aka. Kit

Martin Frb wrote 31 lines
On 09/06/2021 09:29, Pierre Muller via fpc-devel wrote: > >   Martin, > > could you check if this relates to bug #38973? > > https://bugs.freepascal.org/view.php?id=38973 > >  This bug also does not show in current trunk. > >  If you get the same pattern of failures, it is most probably related > to this wrong optimization. Unfortunately I had no success in narrowing it down. I had hoped that generating -al and moving the {$implicitexceptions off} around to narrow it to a single function, I would get a small enough asm diff to tell.... But even narrowing it down to a single "end;" statement that needs to have {$implicitexceptions off} does not help. This line affects the entire specialization of a base class for one of the classes in that unit. (I did -Si- too) So the diff is still to big, and many of the changes are outside of what I can judge to be ok or not. _______________________________________________ fpc-devel maillist  -  https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Reply by email

J. Gareth Moreton
to J. Gareth Moreton

Actually, remind me... what revision number is 3.2.0 based off?

Gareth aka. Kit

J. Gareth Moreton wrote 48 lines
Ah curses.  I fear it was something I did.  I don't like the idea of stepping through each revision until we find one that works. https://bugs.freepascal.org/view.php?id=38294 https://bugs.freepascal.org/view.php?id=38334 https://bugs.freepascal.org/view.php?id=38339 https://bugs.freepascal.org/view.php?id=38343 Those ones relate to buggy optimisations in some way, but they might be a bit too late on to affect 3.2.0. There are quite a few others too - hmmm. Gareth aka. Kit On 13/06/2021 20:04, Martin Frb via fpc-devel wrote: > On 09/06/2021 09:29, Pierre Muller via fpc-devel wrote: >> >>   Martin, >> >> could you check if this relates to bug #38973? >> >> https://bugs.freepascal.org/view.php?id=38973 >> >>  This bug also does not show in current trunk. >> >>  If you get the same pattern of failures, it is most probably >> related to this wrong optimization. > > Unfortunately I had no success in narrowing it down. > > I had hoped that generating -al > and moving the {$implicitexceptions off} around to narrow it to a > single function, > I would get a small enough asm diff to tell.... > > But even narrowing it down to a single "end;" statement that needs to > have {$implicitexceptions off} does not help. > This line affects the entire specialization of a base class for one > of the classes in that unit. (I did -Si- too) > So the diff is still to big, and many of the changes are outside of > what I can judge to be ok or not. > > _______________________________________________ > fpc-devel maillist  -  > https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel >
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Reply by email

Sven Barth
to J. Gareth Moreton: Ah curses.  I fear it was something I did.  I don't like the idea of stepping through…

J. Gareth Moreton via fpc-devel schrieb am
Do., 1. Juli 2021, 01:29:

Ah curses. I fear it was something I did. I don't like the idea of stepping through each revision until we find one that works.

That's where bisecting comes into play: find a revision where it is working (or not working, depending on the context) and then half the interval to the top most revision. If if still works in that revision (or not works) then you do the same with the upper interval, otherwise you continue with the lower one. This continues until only one revision is left which is the culprit.

This is so common that e.g. Git contains a command for simplifying that (cause there are no linear revision numbers after all).

Regards,
Sven

Reply by email

Martin Frb
to Martin Frb: I have the following issue. A testcase (LazDebuggerFp/test) works fine. Except for 3.2.0…

Update to my original mail:
https://lists.freepascal.org/pipermail/fpc-devel/2021-June/043884.html I found the needle in the haystack.

It appears fixed in current (2 day old) fixes 3.3.1. / I have not tested trunk.
-------------------------

The code is in: components\lazutils\lazcollections.pas

procedure TLazFifoQueue.Grow(ADelta: integer);
var
   NewList: array of T;
   c: Integer;
   i: QWord;
begin
   c:=Max(FQueueSize + ADelta, Integer(FTotalItemsPushed -
FTotalItemsPopped));
   setlength(NewList{%H-}, c);
   i:=FTotalItemsPopped;
   while i < FTotalItemsPushed do begin
     NewList[i mod c] := FList[i mod FQueueSize];
     inc(i);
   end;
   FList := NewList;
   FQueueSize:=c;
end;

Without peephole (the "max" is inlined, below is the compare code) # Var c located in register ebx

    cmpq    %rax,%rcx
    jl    .Lj37
    jmp    .Lj38
.Lj37:
    jmp    .Lj39
.Lj38:
    movq    %rcx,%rax
.Lj39:
    movl    %eax,%ebx
    movslq    %ebx,%rax

The 2 moves at the end do 2 things
1) extend 32 to 64 bit (afaik)
2) move the result to ebx

Compiled with peephole the code looks like this
    cmpq    %rax,%rcx
    cmovnlq    %rcx,%rax
    movslq    %eax,%rax

The 2 moves have been combined => ebx is no longer loaded.

But ebx is used for c. And hence c has the wrong value.

<little rant>Because for some mystery SetLength does not use the register value, but loads from the stackframe, the array is correctly sized. Therefore the code above never crashes. But it does not move all data to the new array. Other code then accesses random memory. So my crashes always happened elsewhere. Really not helpful</little rant>

In 3.3.1 the result is
    cmpq    %rax,%rcx
# Peephole Optimization: JccMov2CMov
    cmovnlq    %rcx,%rax
    movl    %eax,%ebx
# Peephole Optimization: %ebx = %eax; changed to minimise pipeline stall (MovXXX2MovXXX)
.Ll23:
    movslq    %eax,%rax

Reply by email

Yuriy Sydorov
to Martin Frb
Martin Frb wrote 66 lines
Update to my original mail: https://lists.freepascal.org/pipermail/fpc-devel/2021-June/043884.html I found the needle in the haystack. It appears fixed in current (2 day old) fixes 3.3.1. / I have not tested trunk. ------------------------- The code is in: components\lazutils\lazcollections.pas procedure TLazFifoQueue.Grow(ADelta: integer); var   NewList: array of T;   c: Integer;   i: QWord; begin   c:=Max(FQueueSize + ADelta, Integer(FTotalItemsPushed - FTotalItemsPopped));   setlength(NewList{%H-}, c);   i:=FTotalItemsPopped;   while i < FTotalItemsPushed do begin     NewList[i mod c] := FList[i mod FQueueSize];     inc(i);   end;   FList := NewList;   FQueueSize:=c; end; Without peephole (the "max" is inlined, below is the compare code) # Var c located in register ebx     cmpq    %rax,%rcx     jl    .Lj37     jmp    .Lj38 .Lj37:     jmp    .Lj39 .Lj38:     movq    %rcx,%rax .Lj39:     movl    %eax,%ebx     movslq    %ebx,%rax The 2 moves at the end do 2 things 1) extend 32 to 64 bit (afaik) 2) move the result to ebx Compiled with peephole the code looks like this     cmpq    %rax,%rcx     cmovnlq    %rcx,%rax     movslq    %eax,%rax The 2 moves have been combined => ebx is no longer loaded. But ebx is used for c. And hence c has the wrong value. <little rant>Because for some mystery SetLength does not use the register value, but loads from the stackframe, the array is correctly sized. Therefore the code above never crashes. But it does not move all data to the new array. Other code then accesses random memory. So my crashes always happened elsewhere. Really not helpful</little rant> In 3.3.1 the result is     cmpq    %rax,%rcx # Peephole Optimization: JccMov2CMov     cmovnlq    %rcx,%rax     movl    %eax,%ebx # Peephole Optimization: %ebx = %eax; changed to minimise pipeline stall (MovXXX2MovXXX) .Ll23:     movslq    %eax,%rax

Nice catch!
Indeed this code reproduces the bug consistently. I'll figure out what is needed to merge to the 3.2 branch in order to fix it.

Yuriy.

Reply by email

J. Gareth Moreton
to Yuriy Sydorov

I have a suspicion as to what it might be.  Can you produce the faulty assembly language with DEBUG_AOPTCPU so it shows the comments?  Does it say "Mov2Nop 3" where the missing instruction lies?

Gareth aka. Kit

P.S. Good job in spotting the fault.

Yuriy Sydorov wrote 83 lines
On 04.10.2021 4:44, Martin Frb via fpc-devel wrote: > Update to my original mail: > https://lists.freepascal.org/pipermail/fpc-devel/2021-June/043884.html > I found the needle in the haystack. > > It appears fixed in current (2 day old) fixes 3.3.1. / I have not > tested trunk. > ------------------------- > > The code is in: components\lazutils\lazcollections.pas > > procedure TLazFifoQueue.Grow(ADelta: integer); > var >    NewList: array of T; >    c: Integer; >    i: QWord; > begin >    c:=Max(FQueueSize + ADelta, Integer(FTotalItemsPushed - > FTotalItemsPopped)); >    setlength(NewList{%H-}, c); >    i:=FTotalItemsPopped; >    while i < FTotalItemsPushed do begin >      NewList[i mod c] := FList[i mod FQueueSize]; >      inc(i); >    end; > >    FList := NewList; >    FQueueSize:=c; > end; > > Without peephole (the "max" is inlined, below is the compare code) > # Var c located in register ebx > >      cmpq    %rax,%rcx >      jl    .Lj37 >      jmp    .Lj38 > .Lj37: >      jmp    .Lj39 > .Lj38: >      movq    %rcx,%rax > .Lj39: >      movl    %eax,%ebx >      movslq    %ebx,%rax > > The 2 moves at the end do 2 things > 1) extend 32 to 64 bit (afaik) > 2) move the result to ebx > > Compiled with peephole the code looks like this >      cmpq    %rax,%rcx >      cmovnlq    %rcx,%rax >      movslq    %eax,%rax > > The 2 moves have been combined => ebx is no longer loaded. > > But ebx is used for c. And hence c has the wrong value. > > <little rant>Because for some mystery SetLength does not use the > register value, but loads from the stackframe, the array is correctly > sized. Therefore the code above never crashes. But it does not move > all data to the new array. Other code then accesses random memory. So > my crashes always happened elsewhere. Really not helpful</little rant> > > In 3.3.1 the result is >      cmpq    %rax,%rcx > # Peephole Optimization: JccMov2CMov >      cmovnlq    %rcx,%rax >      movl    %eax,%ebx > # Peephole Optimization: %ebx = %eax; changed to minimise pipeline > stall (MovXXX2MovXXX) > .Ll23: >      movslq    %eax,%rax > Nice catch! Indeed this code reproduces the bug consistently. I'll figure out what is needed to merge to the 3.2 branch in order to fix it. Yuriy. _______________________________________________ fpc-devel maillist  -  https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Reply by email

Yuriy Sydorov
to J. Gareth Moreton
J. Gareth Moreton wrote
I have a suspicion as to what it might be.  Can you produce the faulty assembly language with DEBUG_AOPTCPU so it shows the comments?  Does it say "Mov2Nop 3" where the missing instruction lies?

I've already fixed this bug in trunk and included the provided test code: https://gitlab.com/freepascal.org/fpc/source/-/commit/e9d318e7e2f772bf455a92461cd5c229e69858d8

The fix will be ported to the 3.2.x branch somehow.

Yuriy.

Reply by email

J. Gareth Moreton
to Yuriy Sydorov

Ah, fair enough.  I don't recall touching much of the CMOV source code, if any, but I'm glad it got fixed.

Gareth aka. Kit

Yuriy Sydorov wrote 16 lines
On 04.10.2021 20:24, J. Gareth Moreton via fpc-devel wrote: > I have a suspicion as to what it might be.  Can you produce the > faulty assembly language with DEBUG_AOPTCPU so it shows the > comments?  Does it say "Mov2Nop 3" where the missing instruction lies? I've already fixed this bug in trunk and included the provided test code: https://gitlab.com/freepascal.org/fpc/source/-/commit/e9d318e7e2f772bf455a92461cd5c229e69858d8 The fix will be ported to the 3.2.x branch somehow. Yuriy. _______________________________________________ fpc-devel maillist  -  https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

Reply by email

Martin Frb
to Yuriy Sydorov: I've already fixed this bug in trunk and included the provided test code:…
Yuriy Sydorov wrote 10 lines
On 04.10.2021 20:24, J. Gareth Moreton via fpc-devel wrote: > I have a suspicion as to what it might be.  Can you produce the > faulty assembly language with DEBUG_AOPTCPU so it shows the > comments?  Does it say "Mov2Nop 3" where the missing instruction lies? I've already fixed this bug in trunk and included the provided test code: https://gitlab.com/freepascal.org/fpc/source/-/commit/e9d318e7e2f772bf455a92461cd5c229e69858d8 The fix will be ported to the 3.2.x branch somehow.

Great thanks...

Reply by email