I haven't been able to build trunk i386 fpc for a couple of weeks. Is this supposed to be working, or have I misconfigured something? (I don't think I have changed anything.) The output is below. I don't think it is significant, but I am building on a 64 bit system, but with a 32 bit starting compiler. (The 64 bit build with the 64 bit starting compiler works fine.)

Colin

make distclean install PREFIX=/home/me/usr FPC=/home/me/fpc/bootstrap/ppc386 (....)
make[7]: Entering directory '/home/me/fpc/trunk/fpcsrc/rtl' make -C linux all
make[8]: Entering directory '/home/me/fpc/trunk/fpcsrc/rtl/linux' /home/me/fpc/trunk/fpcsrc/compiler/ppc1 -Ur -Ur -Xs -O2 -n -Fi../inc -Fi../i386 -Fi../unix -Fii386 -FE.
-FU/home/me/fpc/trunk/fpcsrc/rtl/units/i386-linux -di386 -dRELEASE -Us -Sg system.pp
make[8]: *** [Makefile:3912: system.ppu] Segmentation fault (core dumped) make[8]: Leaving directory '/home/me/fpc/trunk/fpcsrc/rtl/linux' make[7]: *** [Makefile:2057: linux_all] Error 2
make[7]: Leaving directory '/home/me/fpc/trunk/fpcsrc/rtl' make[6]: *** [Makefile:4552: rtl] Error 2
make[6]: Leaving directory '/home/me/fpc/trunk/fpcsrc/compiler' make[5]: *** [Makefile:4403: next] Error 2
make[5]: Leaving directory '/home/me/fpc/trunk/fpcsrc/compiler' make[4]: *** [Makefile:4413: ppc2] Error 2
make[4]: Leaving directory '/home/me/fpc/trunk/fpcsrc/compiler' make[3]: *** [Makefile:4422: cycle] Error 2
make[3]: Leaving directory '/home/me/fpc/trunk/fpcsrc/compiler' make[2]: *** [Makefile:2834: compiler_cycle] Error 2 make[2]: Leaving directory '/home/me/fpc/trunk/fpcsrc' make[1]: *** [Makefile:2866: build-stamp.i386-linux] Error 2 make[1]: Leaving directory '/home/me/fpc/trunk/fpcsrc' make: *** [Makefile:2306: fpcsrc/build-stamp.i386-linux] Error 2

Am Fr., 23. Nov. 2018, 10:26 hat C Western geschrieben:

We have i386 trunk building on at least one x86_64-linux machine, so it can't be a principal problem. :/
Could you check whether it's FPC itself that segfaults or something else and if the former try to retrieve a stacktrace, please?

Regards,
Sven

Hi,

you should always try to start with a release compiler.

If you have the latest i386 compiler installed start by:

cd fpcsrc/compiler
make distclean cycle OPT="-n" FPC=/path/to/release/ppc386

save the resulting ppc386 on success to /home/me/fpc/bootstrap/ppc386

and then start

make distclean install PREFIX=/home/me/usr FPC=/home/me/fpc/bootstrap/ppc386

If you have no i386 compiler installed,
use your x86_64 release with

cd fpcsrc/compiler
make distclean cycle OS_TARGET=linux CPU_TARGET=i386 OPT="-n" FPC=/path/to/release/ppcx64

Please test this out and report if it solves your problem or not.

Pierre Muller

Le 23/11/2018 à 10:21, C Western a écrit :

The compiler I was starting with was obtained from a 3.0.4 release from the fpc website. Trying:

make distclean cycle OPT="-n" OS_TARGET=linux CPU_TARGET=i386 FPC=~/fpc/bootstrap/ppcx64

goes further (or is it just different as it is a cross-compile?) but gives:

make[1]: Entering directory '/home/me/fpc/trunk/fpcsrc/compiler' /usr/bin/mkdir -p i386/units/i386-linux
/usr/bin/mkdir -p i386/bin/i386-linux
/home/ctcmw/fpc/trunk/fpcsrc/compiler/ppcross386 -Pi386 -XPi386-linux- -Xr -Fui386 -Fusystems -Fu../rtl/units/i386-linux -Fii386 -FEi386/bin/i386-linux -FUi386/units/i386-linux -n -di386 -dGDB -dBROWSERLOG -Fux86 -Sew    -di386 -dGDB -dBROWSERLOG -Fux86 -Sew pp.pas pp.pas(277,1) Error: Util i386-linux-ld not found, switching to external linking
pp.pas(277,1) Fatal: There were 1 errors compiling module, stopping Fatal: Compilation aborted
make[1]: *** [Makefile:4355: ppc386] Error 1
make[1]: Leaving directory '/home/me/fpc/trunk/fpcsrc/compiler' make: *** [Makefile:4461: cycle] Error 2

The OS is Fedora 29, and I am reasonably certain a complete set of Gnu i386 tools are present.

I will try and generate a backtrace as suggested by Sven later.

Colin

No need to. The compiler does not crash. It cannot find i386-linux-ld.

You need the binutils for cross-compilation. See the buildfaq.

Or, create them yourself. Add the following bash-scripts somewhere in your path and make them executable: (Works on Fedora, might be different on other systems)

i386-linux-as:
--------------------------
#!/bin/bash
# name this file /usr/bin/i386-linux-as
as --32 $@
--------------------------

i386-linux-ld:

--------------------------
#!/bin/bash
# name this file /usr/bin/i386-linux-ld
ld -A elf32-i386 $@
--------------------------

Regards,

Joost.

Looks like the ppc1 compiler crashes during setup:

gdb) run
Starting program: /home/me/fpc/trunk/fpcsrc/compiler/ppc1

Program received signal SIGSEGV, Segmentation fault. 0x08068760 in INITTLS () at system.pp:543
543            case phdr^.p_type of
(gdb) bt
#0  0x08068760 in INITTLS () at system.pp:543
#1  0x082a63fb in _FPC_PROC_START () at ./i386/si_prc.inc:105 (gdb) p phdr
$1 = (PPHDR) 0x8048034
(gdb) p phdr^
Cannot access memory at address 0x8048034

Creating i386-linux-ld allows the make distclean cycle to complete. However the resulting compiler (ppc386) gives a segmentation fault on startup, just as I had with the previous approach. See my other message for the stack trace.

Colin

Am Sa., 24. Nov. 2018, 10:09 hat C Western geschrieben:

Looks like something related to the new threadvar handling. Though by default that shouldn't be enabled... 🤔

Regards,
Sven

Some more information:

It seems to work fine on Ubuntu (32 bit built on 64 bit OS, same machine). Comparing the binaries with objdump, it looks as though the Fedora one has an offset of 1000, which the Ununtu one does not, and the offset of 1000 needs to be applied to the program header address, but is not. On Fedora:

ppc1:     file format elf32-i386
ppc1
architecture: i386, flags 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
start address 0x082a63d0

Program Header:
    LOAD off    0x00001000 vaddr 0x08049000 paddr 0x08049000 align 2**12          filesz 0x0025d450 memsz 0x0025d450 flags r-x     LOAD off    0x0025f000 vaddr 0x082a7000 paddr 0x082a7000 align 2**12          filesz 0x00020f04 memsz 0x00020f04 flags r--     LOAD off    0x00280000 vaddr 0x082c8000 paddr 0x082c8000 align 2**12          filesz 0x000f7280 memsz 0x00102d84 flags rw-    STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**4          filesz 0x00000000 memsz 0x00000000 flags rw-

On Ubuntu:

./ppc1:     file format elf32-i386
./ppc1
architecture: i386, flags 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
start address 0x082a5470

Program Header:
    LOAD off    0x00000000 vaddr 0x08048000 paddr 0x08048000 align 2**12          filesz 0x0027e3f4 memsz 0x0027e3f4 flags r-x     LOAD off    0x0027f000 vaddr 0x082c7000 paddr 0x082c7000 align 2**12          filesz 0x000f7280 memsz 0x00102d84 flags rw-    STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**4          filesz 0x00000000 memsz 0x00000000 flags rw-

gdb indicates both versions try to access 0x8048034, but this is not mapped on the Fedora version.

Colin

And going through the versions indicates the problem is introduced by:

r40272 | florian | 2018-11-07 22:03:02 +0000 (Wed, 07 Nov 2018) | 1 line Changed paths:
   M /trunk/compiler/cgbase.pas
   M /trunk/compiler/x86/agx86att.pas
   M /trunk/compiler/x86/cgx86.pas
   M /trunk/compiler/x86/nx86ld.pas
   M /trunk/rtl/linux/i386/si_prc.inc
   M /trunk/rtl/linux/system.pp

+ i386-linux support for tls-based threadvars

Can you please check what value phdr has at the crash location after applying the following patch?

diff --git a/rtl/linux/system.pp b/rtl/linux/system.pp
index 0d7ed2b152..74b4592ace 100644
--- a/rtl/linux/system.pp
+++ b/rtl/linux/system.pp
@@ -525,6 +525,8 @@ procedure InitTLS; [public,alias:'FPC_INITTLS'];
     while assigned(auxp^) do
       inc(auxp);
     inc(auxp);
+    phdr:=nil;
+    phnum:=0;
     { now we are at the auxillary vector }
     while assigned(auxp^) do
       begin

That gives the same value as shown above, even after applying the patch. Single stepping the code indicates that both variables are assigned: (Some lines deleted).

Breakpoint 1, INITTLS () at system.pp:523
523        auxp:=ppointer(envp);
(gdb) n
525        while assigned(auxp^) do
(gdb) n
533            case plongint(auxp)^ of
(gdb) n
535                phdr:=pphdr(ppointer(auxp+1)^); (gdb) n
539            inc(auxp,2);
(gdb) p phdr
$3 = (PPHDR) 0x8048034
(gdb) p phdr^
Cannot access memory at address 0x8048034
(gdb) n
533            case plongint(auxp)^ of
(gdb) n
539            inc(auxp,2);
(gdb) n
533            case plongint(auxp)^ of
(gdb) n
537                phnum:=pdword(auxp+1)^; (gdb) n
539            inc(auxp,2);
(gdb) p phnum
$4 = 4
(gdb)
543        for i:=1 to phnum do
(gdb) n
545            case phdr^.p_type of
(gdb) n

Program received signal SIGSEGV, Segmentation fault. 0x0806ef01 in INITTLS () at system.pp:545
545            case phdr^.p_type of
(gdb) p phdr
$5 = (PPHDR) 0x8048034
(gdb)

Colin

This is pretty strange then. This code is pretty simple and in general, tls initialization should be done regardless if FPC uses it for threadvars or not. What version of Fedora are you using?

Fedora 29 (64 bit). Did you see my other message in the thread about a different offset in program header when compiled under Fedora and Ubuntu? (It works on Ubuntu). It is though an offset needs to be applied to find the program header, but I can't see where to find it.

Colin

I am not aware that somewhere a fixed offset is applied. All info is read from the auxiliary data and the header.

You mean the offset field in the header? I cannot imaging how the offset in the header could influence the header itself?

Florian Klämpfl wrote

I am not aware that somewhere a fixed offset is applied. All info is read from the auxiliary data and the header.

You can have a load address offset in case there is a PT_PHDR. From http://www.gabriel.urdhr.fr/2015/01/22/elf-linking/ :

// Simplified code from the GNU dynamic linker source code:
for (ph = phdr; ph < &phdr[phnum]; ++ph)
   if (ph->p_type == PT_PHDR)
     main_map->l_addr = (ElfW(Addr)) phdr - ph->p_vaddr;

There doesn't seem to be one in Colin's program though.

Jonas

Jonas Maebe wrote

On 25/11/18 13:51, Florian Klämpfl wrote: > I am not aware that somewhere a fixed offset is applied. All info is read from the auxiliary data and the header. You can have a load address offset in case there is a PT_PHDR.

Yes.

Indeed. The program header itself seems to be wrong/interpreted wrong.

What generates the header? Is it fpc, an external linker or the loader?

Colin
Sent from my mobile.

Colin Western wrote

What generates the header? Is it fpc, an external linker or the loader?

The linker.

Jonas

Colin Western wrote

What generates the header? Is it fpc, an external linker or the loader?

Can you please run the following C program:

#include <sys/auxv.h>
#include <stdio.h>

int main()
{

  printf("AT_PHDR: %lx\n",getauxval(AT_PHDR));
  printf("AT_PHNUM: %lx\n",getauxval(AT_PHNUM));

}

and include also the relevant part of an objdump?

gcc -m32 -g florian.c

./a.out
AT_PHDR: 8048034
AT_PHNUM: b

objdump -x a.out | more

a.out:     file format elf32-i386
a.out
architecture: i386, flags 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
start address 0x08049070

Program Header:
    PHDR off    0x00000034 vaddr 0x08048034 paddr 0x08048034 align 2**2          filesz 0x00000160 memsz 0x00000160 flags r--   INTERP off    0x00000194 vaddr 0x08048194 paddr 0x08048194 align 2**0          filesz 0x00000013 memsz 0x00000013 flags r--     LOAD off    0x00000000 vaddr 0x08048000 paddr 0x08048000 align 2**12          filesz 0x0000032c memsz 0x0000032c flags r--     LOAD off    0x00001000 vaddr 0x08049000 paddr 0x08049000 align 2**12          filesz 0x00000260 memsz 0x00000260 flags r-x     LOAD off    0x00002000 vaddr 0x0804a000 paddr 0x0804a000 align 2**12          filesz 0x0000015c memsz 0x0000015c flags r--     LOAD off    0x00002f0c vaddr 0x0804bf0c paddr 0x0804bf0c align 2**12          filesz 0x00000110 memsz 0x00000114 flags rw-  DYNAMIC off    0x00002f14 vaddr 0x0804bf14 paddr 0x0804bf14 align 2**2          filesz 0x000000e8 memsz 0x000000e8 flags rw-     NOTE off    0x000001a8 vaddr 0x080481a8 paddr 0x080481a8 align 2**2          filesz 0x00000044 memsz 0x00000044 flags r-- EH_FRAME off    0x0000202c vaddr 0x0804a02c paddr 0x0804a02c align 2**2          filesz 0x0000003c memsz 0x0000003c flags r--    STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**4          filesz 0x00000000 memsz 0x00000000 flags rw-    RELRO off    0x00002f0c vaddr 0x0804bf0c paddr 0x0804bf0c align 2**0          filesz 0x000000f4 memsz 0x000000f4 flags r--

Dynamic Section:
  NEEDED               libc.so.6
  INIT                 0x08049000   FINI                 0x08049248   INIT_ARRAY           0x0804bf0c
  INIT_ARRAYSZ         0x00000004
  FINI_ARRAY           0x0804bf10
  FINI_ARRAYSZ         0x00000004
  GNU_HASH             0x080481ec
  STRTAB               0x0804826c
  SYMTAB               0x0804820c
  STRSZ                0x00000061
  SYMENT               0x00000010
  DEBUG                0x00000000
  PLTGOT               0x0804c000
  PLTRELSZ             0x00000018
  PLTREL               0x00000011
  JMPREL               0x08048314
  REL                  0x0804830c   RELSZ                0x00000008
  RELENT               0x00000008
  VERNEED              0x080482dc
  VERNEEDNUM           0x00000001
  VERSYM               0x080482ce

Version References:
  required from libc.so.6:
    0x06969196 0x00 03 GLIBC_2.16
    0x0d696910 0x00 02 GLIBC_2.0

Sections:
Idx Name          Size      VMA       LMA       File off  Algn   0 .interp       00000013  08048194  08048194  00000194  2**0                   CONTENTS, ALLOC, LOAD, READONLY, DATA   1 .note.ABI-tag 00000020  080481a8  080481a8  000001a8  2**2                   CONTENTS, ALLOC, LOAD, READONLY, DATA   2 .note.gnu.build-id 00000024  080481c8  080481c8  000001c8 2**2                   CONTENTS, ALLOC, LOAD, READONLY, DATA   3 .gnu.hash     00000020  080481ec  080481ec  000001ec  2**2                   CONTENTS, ALLOC, LOAD, READONLY, DATA   4 .dynsym       00000060  0804820c  0804820c  0000020c  2**2                   CONTENTS, ALLOC, LOAD, READONLY, DATA   5 .dynstr       00000061  0804826c  0804826c  0000026c  2**0                   CONTENTS, ALLOC, LOAD, READONLY, DATA   6 .gnu.version  0000000c  080482ce  080482ce  000002ce  2**1                   CONTENTS, ALLOC, LOAD, READONLY, DATA   7 .gnu.version_r 00000030  080482dc  080482dc  000002dc  2**2                   CONTENTS, ALLOC, LOAD, READONLY, DATA   8 .rel.dyn      00000008  0804830c  0804830c  0000030c  2**2                   CONTENTS, ALLOC, LOAD, READONLY, DATA   9 .rel.plt      00000018  08048314  08048314  00000314  2**2                   CONTENTS, ALLOC, LOAD, READONLY, DATA  10 .init         00000024  08049000  08049000  00001000  2**2                   CONTENTS, ALLOC, LOAD, READONLY, CODE  11 .plt          00000040  08049030  08049030  00001030  2**4                   CONTENTS, ALLOC, LOAD, READONLY, CODE  12 .text         000001d5  08049070  08049070  00001070  2**4                   CONTENTS, ALLOC, LOAD, READONLY, CODE  13 .fini         00000018  08049248  08049248  00001248  2**2                   CONTENTS, ALLOC, LOAD, READONLY, CODE  14 .rodata       00000029  0804a000  0804a000  00002000  2**2                   CONTENTS, ALLOC, LOAD, READONLY, DATA  15 .eh_frame_hdr 0000003c  0804a02c  0804a02c  0000202c  2**2                   CONTENTS, ALLOC, LOAD, READONLY, DATA  16 .eh_frame     000000f4  0804a068  0804a068  00002068  2**2                   CONTENTS, ALLOC, LOAD, READONLY, DATA  17 .init_array   00000004  0804bf0c  0804bf0c  00002f0c  2**2                   CONTENTS, ALLOC, LOAD, DATA  18 .fini_array   00000004  0804bf10  0804bf10  00002f10  2**2                   CONTENTS, ALLOC, LOAD, DATA  19 .dynamic      000000e8  0804bf14  0804bf14  00002f14  2**2                   CONTENTS, ALLOC, LOAD, DATA  20 .got          00000004  0804bffc  0804bffc  00002ffc  2**2                   CONTENTS, ALLOC, LOAD, DATA  21 .got.plt      00000018  0804c000  0804c000  00003000  2**2                   CONTENTS, ALLOC, LOAD, DATA  22 .data         00000004  0804c018  0804c018  00003018  2**0                   CONTENTS, ALLOC, LOAD, DATA  23 .bss          00000004  0804c01c  0804c01c  0000301c  2**0                   ALLOC
 24 .comment      0000002c  00000000  00000000  0000301c  2**0                   CONTENTS, READONLY  25 .gnu.build.attributes 000004e4  0804c020  0804c020  00003048 2**2                   CONTENTS, READONLY  26 .debug_aranges 00000020  00000000  00000000  0000352c  2**0                   CONTENTS, READONLY, DEBUGGING  27 .debug_info   0000030b  00000000  00000000  0000354c  2**0                   CONTENTS, READONLY, DEBUGGING  28 .debug_abbrev 000000cc  00000000  00000000  00003857  2**0                   CONTENTS, READONLY, DEBUGGING  29 .debug_line   000000f4  00000000  00000000  00003923  2**0                   CONTENTS, READONLY, DEBUGGING  30 .debug_str    00000265  00000000  00000000  00003a17  2**0                   CONTENTS, READONLY, DEBUGGING SYMBOL TABLE:
08048194 l    d  .interp    00000000              .interp 080481a8 l    d  .note.ABI-tag    00000000 .note.ABI-tag 080481c8 l    d  .note.gnu.build-id    00000000 .note.gnu.build-id 080481ec l    d  .gnu.hash    00000000              .gnu.hash 0804820c l    d  .dynsym    00000000              .dynsym 0804826c l    d  .dynstr    00000000              .dynstr 080482ce l    d  .gnu.version    00000000 .gnu.version 080482dc l    d  .gnu.version_r    00000000 .gnu.version_r 0804830c l    d  .rel.dyn    00000000              .rel.dyn 08048314 l    d  .rel.plt    00000000              .rel.plt 08049000 l    d  .init    00000000              .init 08049030 l    d  .plt    00000000              .plt 08049070 l    d  .text    00000000              .text 08049248 l    d  .fini    00000000              .fini

Somehow this thread reminds my one open bug :
https://bugs.freepascal.org/view.php?id=34475

I have found out that you need to have either linked against libc or have small binary stub generated from assembler source (like 64-bit pascal programs/libraries have that doesn't link against libc) linked against your 32-bit program so it can access PHDR structure without crashing on access violation exception.

An additional note - it may well be a ld version issue. On Ubuntu the ld version is 2.30, but on Fedora ld reports 2.31.1-13.fc29. (Maybe I should have held off upgrading to Fedora 29...)

Colin

Looking at this a little more I think the headers are simply not available to the program. Simply commenting out the INITTLS call allows the compiler to be built successfully. I note that multi-threaded programs don't call it (at least those using cthreads) and if the program isn't multi-threaded the call to INITTLS is most likely not needed so I suspect this would fix most use cases. My main non-gui program (multi-threaded) runs fine. However on Fedora 29 32 bit builds on 64 bit systems are broken for lazarus because the 32 bit -devel packages are broken:

https://bugzilla.redhat.com/show_bug.cgi?id=1651231

https://bugzilla.redhat.com/show_bug.cgi?id=1645328

so maybe I should just conclude that 32 bit apps are broken on Fedora.

Colin

Can you test if patch from my bug report
<url:https://bugs.freepascal.org/view.php?id=34475> fixes your problem? You can omit changes for x86_64 target from that patch.

This is loader-abitag.diff? With that applied, both 64 and 32 bit fpc trunk build, and basic testing indicates that they are working fine. I note however that compiling a basic "Hello World" program now produces a dynamically linked, rather than a statically linked executable, and INITTLS is not called.

Colin

Strange. All my (32/64 bits) binaries are static ones.

What I can see at seeing the source code is that INITTLS is only defined for i386 target.

Oh, I forgot to mention that I use Arch Linux. I run the i686 and x86_64 versions of Arch as an LXC containers.

Sorry - my statement about "Hello World" producing a dynamically linked executable was wrong - I must have mixed up the source file. It now behaves as I would expect, and can recommend your patch. I will add a note in the bug tracker

Colin