This is a follow-up on a thread on the macpascal mailing list, where the problem at hand couldn't be solved.

Regards,

Adriaan van Os

-------

While compiling a large Macintosh Pascal project with FPC, I am running into a number of problems, which I suspect are related:

1. After a clean, the full build succeeds
2. Ater a successive build (without a clean or source changes) several units are recompiled, until the compiler crashes
3. After another build, the compiler (sometimes) endlessly keeps compiling a selected number of units.

This is with the latest FPC svn compiler, under Xcode 3.0 using a Makefile target. I am compiling on an Intel MacBookPro under Mac OS X 10.5.4.

A b o u t t h e c r a s h

error: Compilation aborted
An unhandled exception occurred at $00128505 :
EAccessViolation : Access violation
$00128505
$0010722E
$00108507
$00108868
$00108B92
$00104CFD
$00107F0C
$000E3D21
$000F4835
$000F0CC9
$000E3D21
$000F4835
$00100414
$00101138
$000E3D21
$000F4835
$00100414

make: *** [compile_tool_compile] Error 217

When running ppc386 under GDB, I get

Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_PROTECTION_FAILURE at address: 0x00000000 0x00000000 in ?? ()

or

	Program received signal EXC_BAD_ACCESS, Could not access memory.
	Reason: KERN_PROTECTION_FAILURE at address: 0x00000030
	GEN_LOAD_RETURN_VALUE (LIST=0x59b9a60) at ncgutil.pas:1295
	1295	        if (ressym.refs>0) or (ressym.vardef.needs_inittable) then

A b o u t r e c o m p i l i n g

When I add -va or -vv to the FPC argument list, I notice lines like

[28.068] Recompiling MMyutils0, checksum changed for V7Stuff

upon a rebuild for units that haven't , I did the following:

a. clean and build, followed by a ppudump *.ppu > ppudump_clean.txt b. build without a clean, followed by a ppudump *.ppu > ppudump_rebuild.txt c. compare the ppudump files with diff -u ppudump_clean.txt ppudump_rebuild.txt > ppudump_diff.txt

Now, the ppudump_diff.txt file has lines like

@@ -29,7 +29,7 @@
Uses unit: Sglobals (Crc: 207BA0FC, IntfcCrc: 207BA0FC) Uses unit: MProutil (Crc: 2A51D9C7, IntfcCrc: 875453BF) Uses unit: InterruptCode (Crc: C36B6157, IntfcCrc: 6D45DAC1) -Uses unit: V7Stuff (Crc: 942FBF47, IntfcCrc: D85A6EB1) +Uses unit: V7Stuff (Crc: 42F16682, IntfcCrc: 18B40446) Uses unit: MMyutils (Crc: DD59B0C2, IntfcCrc: AFB920A0) Uses unit: MMyutils2 (Crc: 181A1B43, IntfcCrc: ED58FC40) Uses unit: MMyutils0 (Crc: BF00BD89, IntfcCrc: D4DEB0DA)

and even (for some global variables)

@@ -389309,7 +389309,7 @@
SymOptions : Public
Spez : Value
Regable : None
- Addr Taken : TRUE
+ Addr Taken : FALSE
Var Type : (1761) Unit 0, DefId 19
Options : TypedConst
DefaultConst : (1770) Nil

M y g u e s s

So far the facts, the rest is what I am concluding and guessing, based on the above:

- there seems to be a unit symbol checksum calculation problem, which is causing unneeded recompilations.
- the checksum problem may also produce the crash, or at least trigger some unusual state in the compiler that wasn't foreseen in the code and then leads to the crash
- I get the impression that the checksum problem is related to the order of used units in unit interface and/or implementation USES clauses
- this may (or may not) be related to mixing macpascal and delphi modes (the project is a mix from an old CodeWarrior version and the PC Delphi version)
- this may (or may not) be related to using (or even redefining) types that are different in macpascal and delphi modes, like INTEGER or STRING (any advice I give on coding habits is ignored)
- this may (or may not) be related to using delphi mode declared objects from macpascal mode and the other way round.

I don't have a simple reproducible case. I welcome advice on how to get there, based on above observations.

Regards,

Adriaan van Os

-------

Adriaan van Os wrote

[28.068] Recompiling MMyutils0, checksum changed for V7Stuff upon a rebuild for units that haven't , I did the following:

The two main reasons I know of that currently can change the interface crc are a) calls to inline functions of which only the interface was parsed when the function was called b) calls to functions which are declared as regular functions in the interface of a unit, but as external functions in the implementation

The reason for a) is that the code for inline functions is stored in the unit interface (as it must be exported to other units), and for b) is that the assembler symbol name of the function changes (the compiler also inserts a code to redirect calls to the old name to the new name, so the first compilation will also work).

The addr_taken field is not used in the interface crc calculation, so this shouldn't trigger any recompilations or crc changes. I'm not entirely sure how it can change from true to false though (except in case of inlining).

- the checksum problem may also produce the crash, or at least trigger some unusual state in the

compiler that wasn't foreseen in the code and then leads to the crash

Probably some kind of circular dependency, but I'm not sure how.

- I get the impression that the checksum problem is related to the order of used units in unit

interface and/or implementation USES clauses

That's certainly possible, as it is possible that depending on the order more or fewer calls occur to inline functions whose body is not yet parsed.

- this may (or may not) be related to mixing macpascal and delphi modes (the project is a mix

from an old CodeWarrior version and the PC Delphi version)

That should not have any effect.

- this may (or may not) be related to using (or even redefining) types that are different in

macpascal and delphi modes, like INTEGER or STRING (any advice I give on coding habits is ignored)

That should not make any difference. Using type with the same name from different units can sometimes cause confusing compiler messages ("expected typename, but got typename", although it should prefix it with the unit name nowadays), but should not have any effects on interface crc's.

- this may (or may not) be related to using delphi mode declared objects from macpascal mode and

the other way round.

Every macpas program that uses objects uses objects declared in delphi/objfpc mode, because the basic class (= macpas object) support is implemented using objfpc mode.

I don't have a simple reproducible case. I welcome advice on how to get there, based on above

observations.

If you have inline functions, that would be a good place to start.

Jonas

-------

external functions in the implementation

The reason for a) is that the code for inline functions is stored in the unit interface (as it

must be exported to other units), and for b) is that the assembler symbol name of the function changes (the compiler also inserts a code to redirect calls to the old name to the new name, so the first compilation will also work).

Thanks for your reply.

I went so far as to remove all inline directives, as well as all external directives from implementation parts. Unfortunately, the compiler problems remain. So, I suspect it's a checksum calculation bug.

> I don't have a simple reproducible case. I welcome advice on how to get there, based on above

observations.

If you have inline functions, that would be a good place to start.

Any more ideas ?

Regards,

Adriaan van Os

-------

as external functions in the implementation

> The reason for a) is that the code for inline functions is stored in the unit interface (as it

must be exported to other units), and for b) is that the assembler symbol name of the function changes (the compiler also inserts a code to redirect calls to the old name to the new name, so the first compilation will also work).

Thanks for your reply. I went so far as to remove all inline directives, as well as all external directives from

implementation parts. Unfortunately, the compiler problems remain. So, I suspect it's a checksum calculation bug.

More likely is that there is at least yet one more way to change the interface checksum in the implementation. Unfortunately, I've never heard of any. I've also never before debugged this kind of problem (Peter Vreman has quite some experience with it, but he's not on this list; maybe Florian also?), so I can't give any specific pointers on how to tackle this either.

Sorry.

Jonas

This is a follow-up on a thread on the macpascal mailing list, where the problem at hand couldn't be solved.

Regards,

Adriaan van Os

-------

While compiling a large Macintosh Pascal project with FPC, I am running into a number of problems, which I suspect are related:

1. After a clean, the full build succeeds
2. Ater a successive build (without a clean or source changes) several units are recompiled, until the compiler crashes
3. After another build, the compiler (sometimes) endlessly keeps compiling a selected number of units.

This is with the latest FPC svn compiler, under Xcode 3.0 using a Makefile target. I am compiling on an Intel MacBookPro under Mac OS X 10.5.4.

A b o u t t h e c r a s h

error: Compilation aborted
An unhandled exception occurred at $00128505 :
EAccessViolation : Access violation
$00128505
$0010722E
$00108507
$00108868
$00108B92
$00104CFD
$00107F0C
$000E3D21
$000F4835
$000F0CC9
$000E3D21
$000F4835
$00100414
$00101138
$000E3D21
$000F4835
$00100414

make: *** [compile_tool_compile] Error 217

When running ppc386 under GDB, I get

Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_PROTECTION_FAILURE at address: 0x00000000 0x00000000 in ?? ()

or

	Program received signal EXC_BAD_ACCESS, Could not access memory.
	Reason: KERN_PROTECTION_FAILURE at address: 0x00000030
	GEN_LOAD_RETURN_VALUE (LIST=0x59b9a60) at ncgutil.pas:1295
	1295	        if (ressym.refs>0) or (ressym.vardef.needs_inittable) then

A b o u t r e c o m p i l i n g

When I add -va or -vv to the FPC argument list, I notice lines like

[28.068] Recompiling MMyutils0, checksum changed for V7Stuff

upon a rebuild for units that haven't , I did the following:

a. clean and build, followed by a ppudump *.ppu > ppudump_clean.txt b. build without a clean, followed by a ppudump *.ppu > ppudump_rebuild.txt c. compare the ppudump files with diff -u ppudump_clean.txt ppudump_rebuild.txt > ppudump_diff.txt

Now, the ppudump_diff.txt file has lines like

@@ -29,7 +29,7 @@
Uses unit: Sglobals (Crc: 207BA0FC, IntfcCrc: 207BA0FC) Uses unit: MProutil (Crc: 2A51D9C7, IntfcCrc: 875453BF) Uses unit: InterruptCode (Crc: C36B6157, IntfcCrc: 6D45DAC1) -Uses unit: V7Stuff (Crc: 942FBF47, IntfcCrc: D85A6EB1) +Uses unit: V7Stuff (Crc: 42F16682, IntfcCrc: 18B40446) Uses unit: MMyutils (Crc: DD59B0C2, IntfcCrc: AFB920A0) Uses unit: MMyutils2 (Crc: 181A1B43, IntfcCrc: ED58FC40) Uses unit: MMyutils0 (Crc: BF00BD89, IntfcCrc: D4DEB0DA)

and even (for some global variables)

@@ -389309,7 +389309,7 @@
SymOptions : Public
Spez : Value
Regable : None
- Addr Taken : TRUE
+ Addr Taken : FALSE
Var Type : (1761) Unit 0, DefId 19
Options : TypedConst
DefaultConst : (1770) Nil

M y g u e s s

So far the facts, the rest is what I am concluding and guessing, based on the above:

- there seems to be a unit symbol checksum calculation problem, which is causing unneeded recompilations.
- the checksum problem may also produce the crash, or at least trigger some unusual state in the compiler that wasn't foreseen in the code and then leads to the crash
- I get the impression that the checksum problem is related to the order of used units in unit interface and/or implementation USES clauses
- this may (or may not) be related to mixing macpascal and delphi modes (the project is a mix from an old CodeWarrior version and the PC Delphi version)
- this may (or may not) be related to using (or even redefining) types that are different in macpascal and delphi modes, like INTEGER or STRING (any advice I give on coding habits is ignored)
- this may (or may not) be related to using delphi mode declared objects from macpascal mode and the other way round.

I don't have a simple reproducible case. I welcome advice on how to get there, based on above observations.

Regards,

Adriaan van Os

-------

Adriaan van Os wrote

[28.068] Recompiling MMyutils0, checksum changed for V7Stuff upon a rebuild for units that haven't , I did the following:

The two main reasons I know of that currently can change the interface crc are a) calls to inline functions of which only the interface was parsed when the function was called b) calls to functions which are declared as regular functions in the interface of a unit, but as external functions in the implementation

The reason for a) is that the code for inline functions is stored in the unit interface (as it must be exported to other units), and for b) is that the assembler symbol name of the function changes (the compiler also inserts a code to redirect calls to the old name to the new name, so the first compilation will also work).

The addr_taken field is not used in the interface crc calculation, so this shouldn't trigger any recompilations or crc changes. I'm not entirely sure how it can change from true to false though (except in case of inlining).

- the checksum problem may also produce the crash, or at least trigger some unusual state in the

compiler that wasn't foreseen in the code and then leads to the crash

Probably some kind of circular dependency, but I'm not sure how.

- I get the impression that the checksum problem is related to the order of used units in unit

interface and/or implementation USES clauses

That's certainly possible, as it is possible that depending on the order more or fewer calls occur to inline functions whose body is not yet parsed.

- this may (or may not) be related to mixing macpascal and delphi modes (the project is a mix

from an old CodeWarrior version and the PC Delphi version)

That should not have any effect.

- this may (or may not) be related to using (or even redefining) types that are different in

macpascal and delphi modes, like INTEGER or STRING (any advice I give on coding habits is ignored)

That should not make any difference. Using type with the same name from different units can sometimes cause confusing compiler messages ("expected typename, but got typename", although it should prefix it with the unit name nowadays), but should not have any effects on interface crc's.

- this may (or may not) be related to using delphi mode declared objects from macpascal mode and

the other way round.

Every macpas program that uses objects uses objects declared in delphi/objfpc mode, because the basic class (= macpas object) support is implemented using objfpc mode.

I don't have a simple reproducible case. I welcome advice on how to get there, based on above

observations.

If you have inline functions, that would be a good place to start.

Jonas

-------

external functions in the implementation

The reason for a) is that the code for inline functions is stored in the unit interface (as it

must be exported to other units), and for b) is that the assembler symbol name of the function changes (the compiler also inserts a code to redirect calls to the old name to the new name, so the first compilation will also work).

Thanks for your reply.

I went so far as to remove all inline directives, as well as all external directives from implementation parts. Unfortunately, the compiler problems remain. So, I suspect it's a checksum calculation bug.

> I don't have a simple reproducible case. I welcome advice on how to get there, based on above

observations.

If you have inline functions, that would be a good place to start.

Any more ideas ?

Regards,

Adriaan van Os

-------

as external functions in the implementation

> The reason for a) is that the code for inline functions is stored in the unit interface (as it

must be exported to other units), and for b) is that the assembler symbol name of the function changes (the compiler also inserts a code to redirect calls to the old name to the new name, so the first compilation will also work).

Thanks for your reply. I went so far as to remove all inline directives, as well as all external directives from

implementation parts. Unfortunately, the compiler problems remain. So, I suspect it's a checksum calculation bug.

More likely is that there is at least yet one more way to change the interface checksum in the implementation. Unfortunately, I've never heard of any. I've also never before debugged this kind of problem (Peter Vreman has quite some experience with it, but he's not on this list; maybe Florian also?), so I can't give any specific pointers on how to tackle this either.

Sorry.

Jonas

These kind of recompile failures are a known issue. Years ago it is already identifed and an idea how to rewrite the unit handling. But it is a huge project and time is very limited.

You can try to compile the compile with -dINTFPPU this will generate a .ppu.intf after the interface seciton has been parsed. Diffing the ppudump between the .ppu and .ppu.intf will give a better indication what is changed in the ppu between the interface parsing and implementation parsing.

For the rest we can't help without having complete sources to reproduce. A single gdb backtrace doesn't help in this case. It shows only a result of a bug and not the real location where it goes wrong.

Peter

Thanks for the hint. I managed to rebuild the compiler with -dINTFPPU and this does produce .ppu.intf files when compiling with it. The diff between the ppudump of the the .ppu files and the .ppu.intf files does show checksum and other differences. Not sure which ones to look at.

For example:

@@ -1,20 +1,21 @@
-Analyzing Objects/DrumStyles.ppu.intf (v91)
+Analyzing Objects/DrumStyles.ppu (v91)

Header
-------
Compiler version : 2.3.1
Target processor : i386
Target operating system : Darwin-i386
-Unit flags : little_endian
-FileSize (w/o header) : 31358
-Checksum : F4F82F2E
-Interface Checksum : 7FE18E88
-Definitions stored : 0
-Symbols stored : 0
+Unit flags : static_linked, little_endian, local_threadvars, local_symtable +FileSize (w/o header) : 81958
+Checksum : BFD83432
+Interface Checksum : 86AA546F
+Definitions stored : 1096
+Symbols stored : 1428

Interface section
------------------
Module Name: DrumStyles
+Source file 1 : DrumStyles.pas 2008/08/29 23:22:46 Uses unit: System (Crc: 286B12B3, IntfcCrc: E9DCB7B9) Uses unit: MacPas (Crc: F374769E, IntfcCrc: 72C6383A) Uses unit: MacOSAll (Crc: A4E5FD79, IntfcCrc: A2FF6F65) @@ -64,11 +65,16 @@
Uses unit: Hints (Crc: DB360E5E, IntfcCrc: 55A8384F) Uses unit: MBFiles (Crc: AC9F8454, IntfcCrc: 72616F2F) Uses unit: Classes (Crc: 42E7F73E, IntfcCrc: E050CFF2) -DerefMapsize: 3
+Link unit object file: DrumStyles.o (static )
+!! Skipping unsupported PPU Entry in General Part: 100 +!! Skipping unsupported PPU Entry in General Part: 82 +DerefMapsize: 5
DerefMap[0] = SYSTEM
DerefMap[1] = MACOSALL
DerefMap[2] = CLASSES
-Derefdata length: 4785
+DerefMap[3] = MGLOBALS
+DerefMap[4] = SYSUTILS
+Derefdata length: 28828

Interface definitions
----------------------
@@ -86,9 +92,10 @@
Level : 2
Class : (24) Nil
Procsym : (26) SymId 57
- File Pos : 1 (94,10)
+ File Pos : 1 (3411,10)
SymOptions : Public
Import Nr : 0
+ Alias names : _DRUMSTYLES_READDRUMSTYLE$SHORTSTRING$$BOOLEAN ------ parast ------
------ parast ------
Symtable datasize : 1
@@ -98,18 +105,18 @@
File Pos : 1 (94,24)
SymOptions : Public
Spez : Value
- Regable : IntReg
- Addr Taken : FALSE
+ Regable : Addr
+ Addr Taken : TRUE
Var Type : (13) Unit 0, DefId 19
Options : HasLocalCopy
DefaultConst : (22) Nil
ParaNr : 10
- VarState : 2
+ VarState : 3
** Definition Id 1 **
Record definition
Type symbol : (32) SymId 59
DefOptions :
- DefStates :
+ DefStates : InitTable Used, RTTITable Used, InitTable Written, RTTITable Written FieldAlign : 0
RecordAlign : 4
PadAlign : 4
@@ -204,7 +211,7 @@
Record definition
Type symbol : (134) SymId 73
DefOptions :
- DefStates :
+ DefStates : InitTable Used, RTTITable Used, InitTable Written, RTTITable Written FieldAlign : 0
RecordAlign : 4
PadAlign : 4

etcetera.

For the rest we can't help without having complete sources to reproduce. A single gdb backtrace doesn't help in this case. It shows only a result of a bug and not the real location where it goes wrong.

I can put the ppudump files somewhere, or try something else at your advice or send you the complete source code by private email. Whatever you prefer.

Regards,

Adriaan van Os

For the record, File Pos changes and is crc'ed.

Are you sure ?

For example, in the ppudump of the .ppu.intf file, we find the FilePos of the procedure declaration in the interface section

Interface definitions
----------------------
** Definition Id 0 **
Procedure definition
Type symbol : (0) Nil
DefOptions :
DefStates :
Return type : (4) Unit 0, DefId 3
Fpu used : 0
TypeOption : Procedure
CallOption : MWPascal
Options : Global
Number : 65535
Level : 2
Class : (13) Nil
Procsym : (15) SymId 33
File Pos : 1 (17,11)
SymOptions : Public
Import Nr : 0
------ parast ------
------ parast ------
Symtable datasize : 0
Symtable alignment: 0

whereas in the ppudump of the .ppu file, we read the FilePos of the same procedure in the implementation section

Interface definitions
----------------------
** Definition Id 0 **
Procedure definition
Type symbol : (0) Nil
DefOptions :
DefStates :
Return type : (4) Unit 0, DefId 3
Fpu used : 0
TypeOption : Procedure
CallOption : MWPascal
Options : Global
Number : 65535
Level : 2
Class : (13) Nil
Procsym : (15) SymId 33
File Pos : 1 (20,11)
SymOptions : Public
Import Nr : 0
Alias names : _AZK_ALLASSIGNZMELEZ
------ parast ------
------ parast ------
Symtable datasize : 0
Symtable alignment: 0

....
procedure AllAssignZMELEZ;
implementation
procedure AllAssignZMELEZ;
.....

For this unit, the checksums do match.

Header
-------
Compiler version : 2.3.1
Target processor : i386
Target operating system : Darwin-i386
Unit flags : little_endian
FileSize (w/o header) : 1809
Checksum : FE3E2A08
Interface Checksum : 24A9FEDB
Definitions stored : 0
Symbols stored : 0

Header
-------
Compiler version : 2.3.1
Target processor : i386
Target operating system : Darwin-i386
Unit flags : static_linked, little_endian, local_threadvars, local_symtable FileSize (w/o header) : 2310
Checksum : FE3E2A08
Interface Checksum : 24A9FEDB
Definitions stored : 3
Symbols stored : 52

Regards,

Adriaan van Os

Indeed, I were wrong.

I think, I solved the checksum mystery. It is related to what Jonas wrote on the macpascal mailing list:

The two main reasons I know of that currently can change the interface crc are a) calls to inline functions of which only the interface was parsed when the function was called b) calls to functions which are declared as regular functions in the interface of a unit, but as external functions in the implementation

The tricky part is that fpc in macpas mode implicitely assumes that (b) happens if a procedure or function is declared in the interface part but not implemented in the implementation part ! Neither is - as far as I know - a warning issued by fpc in this situation, as CodeWarrior does !

For example, for a test program checksumwoes.pas:

{$mode macpas}
unit checksumwoes;
interface
procedure Forgotten;
implementation
end.

the ppudump of checksumwoes.ppu.intf reads:

PPU-Analyser Version 2.3.1
Copyright (c) 1998-2007 by the Free Pascal Development Team

Analyzing checksumwoes.ppu.intf (v91)

Header
-------
Compiler version : 2.3.1
Target processor : i386
Target operating system : Darwin-i386
Unit flags : little_endian
FileSize (w/o header) : 519
Checksum : EE8A133F
Interface Checksum : 2BCCD624
Definitions stored : 0
Symbols stored : 0

whereas the ppudump of checksumwoes.ppu reads:

PPU-Analyser Version 2.3.1
Copyright (c) 1998-2007 by the Free Pascal Development Team

Analyzing checksumwoes.ppu (v91)

Header
-------
Compiler version : 2.3.1
Target processor : i386
Target operating system : Darwin-i386
Unit flags : static_linked, little_endian, local_threadvars, local_symtable FileSize (w/o header) : 1002
Checksum : 33531017
Interface Checksum : 39ACE269
Definitions stored : 2
Symbols stored : 12

I haven't checked if this solves (or works around) the compiler crash. A "not implemented" warning in macpas mode would be helpful !

Regards,