Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2021-03

Smart-linking versus Whole Program Optimization

13 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

LacaK

Hi,

my goal is DO NOT include unused procedures, methods in final program/library on Win32 target exe/dll.

A.) I compile my (units) program/library with -CX -XX (set in Project Options)

B.) I did also experiments with WPO: -OWall -FWprogram.wpo -Xs- (plus -Owall -Fwprogram.wpo)

Questions:

1. I did not notice difference between A and B (it seems like WPO did not reduce size of EXE (may be that I do something wrong)) 1.1 is there way how I can check which procedures were striped out or vice-versa included?

2. Is smart-linking/WPO capable to remove unused:
2.1  regular procedures/functions ?
2.2  methods of classes (non virtual, virtual) ?
2.3  class methods of classes ?

Thank you

-Laco.

Reply by email

Jonas Maebe
to LacaK
LacaK wrote
1. I did not notice difference between A and B (it seems like WPO did not reduce size of EXE (may be that I do something wrong)) 1.1 is there way how I can check which procedures were striped out or vice-versa included?

You can generate link maps (-Xm) and compare those.

2. Is smart-linking/WPO capable to remove unused: 2.1  regular procedures/functions ? 2.2  methods of classes (non virtual, virtual) ? 2.3  class methods of classes ?

Smartlinking can remove any non-referenced non-virtual methods, global variables, and regular procedures/functions.

Regarding what WPO can do, see
https://wiki.freepascal.org/Whole_Program_Optimization

Jonas

Reply by email

Sven Barth
to LacaK: my goal is DO NOT include unused procedures, methods in final program/library on Win32…

LacaK via fpc-pascal schrieb am Mi., 10.
März 2021, 10:34:

Show quoted text 16 lines
Hi, my goal is DO NOT include unused procedures, methods in final program/library on Win32 target exe/dll. A.) I compile my (units) program/library with -CX -XX (set in Project Options) B.) I did also experiments with WPO: -OWall -FWprogram.wpo -Xs- (plus -Owall -Fwprogram.wpo) Questions: 1. I did not notice difference between A and B (it seems like WPO did not reduce size of EXE (may be that I do something wrong))

It might simply be that WPO did not have anything to do.

1.1 is there way how I can check which procedures were striped out or

vice-versa included?

You can create a map file using -Xm. This will list all sections that were included, though it's quite a read. ;)

2. Is smart-linking/WPO capable to remove unused: 2.1 regular procedures/functions ?

Done by smartlinking.

2.2 methods of classes (non virtual, virtual) ?

The former is done by smartlinking (if they aren't required for e.g. published properties). The later might be done by WPO, though I think that the case of a virtual method not being used at all is not handled.

2.3 class methods of classes ?

Done by smartlinking.

In essence you gain the most by smartlinking. WPO only works in certain situations (e.g. the compiler has a class hierarchy that works very well for devirtualisation) and some things that could in theory be done with WPO are simply not implemented.

Regards,
Sven

Reply by email

Jonas Maebe
to Yuriy Sydorov
Yuriy Sydorov wrote 10 lines
On 10.03.2021 11:34, LacaK via fpc-pascal wrote: > > A.) I compile my (units) program/library with -CX -XX (set in Project > Options) > Just a small note: -CX -XX are not needed for the top-tier targets nowadays. The section based smart linking is a default option in recent FPC versions.

At least -XX is still necessary, otherwise we don't enable dead code/data stripping in the linker (at least not on Linux). And on macOS, -CX is also needed, otherwise the compiler does not emit a directive in the object files that dead code/data stripping should be enabled based on whether or not there are references to the global symbols preceding them.

Neither parameter may be necessary on Windows, but that's probably the only one.

Jonas

Reply by email

LacaK
to Jonas Maebe

It will be nice have this in documentation. For example in https://www.freepascal.org/docs-html/prog/progse30.html#x195-2000007.3

At least explain:
- relation of -CX and -XX
- what is to be removed: non-referenced non-virtual methods, global variables, and regular procedures/functions
- (may be some infos about behavior on various targets)

-Laco.

Show quoted text 23 lines
On 10/03/2021 13:17, Yuriy Sydorov via fpc-pascal wrote: > On 10.03.2021 11:34, LacaK via fpc-pascal wrote: >> A.) I compile my (units) program/library with -CX -XX (set in Project >> Options) >> > Just a small note: > > -CX -XX are not needed for the top-tier targets nowadays. The section > based smart linking is a default option in recent FPC versions. At least -XX is still necessary, otherwise we don't enable dead code/data stripping in the linker (at least not on Linux). And on macOS, -CX is also needed, otherwise the compiler does not emit a directive in the object files that dead code/data stripping should be enabled based on whether or not there are references to the global symbols preceding them. Neither parameter may be necessary on Windows, but that's probably the only one. Jonas _______________________________________________ fpc-pascal maillist - https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Reply by email

Jonas Maebe
to LacaK
LacaK wrote 5 lines
It will be nice have this in documentation. For example in https://www.freepascal.org/docs-html/prog/progse30.html#x195-2000007.3 At least explain: - relation of -CX and -XX

Indeed.

- what is to be removed: non-referenced non-virtual methods, global variables, and regular procedures/functions

Disagreed. Those are implementation details, may vary from target to target, may depend on the assembler/linker rather than on FPC, etc. What I wrote is just the high level idea. When you have e.g. published properties and their RTTI, you can easily start having a lot of hidden references.

Jonas

Reply by email

Michael Van Canneyt
to Jonas Maebe
Jonas Maebe wrote 8 lines
On 10/03/2021 14:33, LacaK via fpc-pascal wrote: > It will be nice have this in documentation. For example in > https://www.freepascal.org/docs-html/prog/progse30.html#x195-2000007.3 > > At least explain: > - relation of -CX and -XX Indeed.

I am all for it.

But please explain them in plain language so I can update the documentation, because from the help pages, I don't know what exactly to explain.

Michael.

Reply by email

LacaK
to Jonas Maebe: Indeed. Disagreed. Those are implementation details, may vary from target to target, may…
Show quoted text 14 lines
> It will be nice have this in documentation. For example in > https://www.freepascal.org/docs-html/prog/progse30.html#x195-2000007.3 > > At least explain: > - relation of -CX and -XX Indeed. > - what is to be removed: non-referenced non-virtual methods, global > variables, and regular procedures/functions Disagreed. Those are implementation details, may vary from target to target, may depend on the assembler/linker rather than on FPC, etc. What I wrote is just the high level idea. When you have e.g. published properties and their RTTI, you can easily start having a lot of hidden references.

I do not insist that it must be written in this way, but something should be there explained.

Goal is that: reader should understand what can expect from smart-linking.

-Laco.

Reply by email

Jonas Maebe
to Michael Van Canneyt: I am all for it. But please explain them in plain language so I can update the…
Michael Van Canneyt wrote 16 lines
On Wed, 10 Mar 2021, Jonas Maebe via fpc-pascal wrote: > On 10/03/2021 14:33, LacaK via fpc-pascal wrote: >> It will be nice have this in documentation. For example in >> https://www.freepascal.org/docs-html/prog/progse30.html#x195-2000007.3 >> >> At least explain: >> - relation of -CX and -XX > > Indeed. I am all for it. But please explain them in plain language so I can update the documentation, because from the help pages, I don't know what exactly to explain.

The general explanation is:
-CX: create object files that can be smartlinked
-XX: perform smartlinking during the linking phase (this requires that the units were compiled with -CX)

On some targets, -CX is a no-op and -XX works even when -CX was not specified (e.g. Linux), or both -CX and -XX are no-ops and smart linking is always enabled (Windows).

Jonas

Reply by email

Yuriy Sydorov
to Jonas Maebe: At least -XX is still necessary, otherwise we don't enable dead code/data stripping in…
Jonas Maebe wrote 17 lines
On 10/03/2021 13:17, Yuriy Sydorov via fpc-pascal wrote: > On 10.03.2021 11:34, LacaK via fpc-pascal wrote: >> >> A.) I compile my (units) program/library with -CX -XX (set in Project >> Options) >> > > Just a small note: > > -CX -XX are not needed for the top-tier targets nowadays. The section > based smart linking is a default option in recent FPC versions. At least -XX is still necessary, otherwise we don't enable dead code/data stripping in the linker (at least not on Linux). And on macOS, -CX is also needed, otherwise the compiler does not emit a directive in the object files that dead code/data stripping should be enabled based on whether or not there are references to the global symbols preceding them.

Yes, you are right. On Windows, since introduction of the internal linker long time ago, both -CX -XX are ignored and smart linking is always ON. I've supposed that this is also true for any target which supports tf_smartlink_sections and these options are only needed for older smart linking method which involves generation of static libraries containing tons of object files.

Neither parameter may be necessary on Windows, but that's probably the only one.

Android and MS-DOS targets also ignore -CX -XX

Yuriy.

Reply by email

Karoly Balogh
to Yuriy Sydorov

Hi,

Yuriy Sydorov wrote
> Neither parameter may be necessary on Windows, but that's probably the > only one. Android and MS-DOS targets also ignore -CX -XX

* when using the internal linker... :) External linkers may be supported for these targets, just as for Windows. In this case smartlinking might not be, or only supported in the old fashioned way.

Other targets: m68k doesn't have any internal assembler or linker, but at least any target using the vasm/vlink assembler-liker combo also has smartlinking enabled, using named sections in the assembler file, and vlink's section garbage collection feature. (Similar to what recent GNU LD has for ELF.)

This includes, Amiga, Atari, and the Sinclair PowerPC targets like AmigaOS 4, and MorphOS also supports it.

Charlie

Reply by email

Yuriy Sydorov
to Karoly Balogh
Karoly Balogh wrote 12 lines
Hi, On Wed, 10 Mar 2021, Yuriy Sydorov via fpc-pascal wrote: >> Neither parameter may be necessary on Windows, but that's probably the >> only one. > > Android and MS-DOS targets also ignore -CX -XX * when using the internal linker... :) External linkers may be supported for these targets, just as for Windows. In this case smartlinking might not be, or only supported in the old fashioned way.

Here I've meant ignoring of the -CX -XX options, not the actual implementation of smartlinking. For Android the external linker is always used and the section based smartlinking is the only option. For ms-dos by default the wlink linker is used with the library based smartlinking and -CX -XX are also ignored.

Show quoted text 8 lines
Other targets: m68k doesn't have any internal assembler or linker, but at least any target using the vasm/vlink assembler-liker combo also has smartlinking enabled, using named sections in the assembler file, and vlink's section garbage collection feature. (Similar to what recent GNU LD has for ELF.) This includes, Amiga, Atari, and the Sinclair PowerPC targets like AmigaOS 4, and MorphOS also supports it.

Sure, building for retro targets can't work at all without smartlinking enabled by default.

Yuriy.

Reply by email