The first version of the OO rewrite branch is ready for alpha testing :-) Now I need assistance in testing and profiling it. Can somebody do that, or give me instructions how to do that myself?

The current version still uses global variables, which shall be moved into objects in the next round. Since this difference may affect the runtime behaviour, a tag for this version would be nice, so that different models can be derived from the same base.

Another idea is a cache for frequently created objects, like procinfo and the register allocators.

DoDi

Hans-Peter Diettrich wrote

The first version of the OO rewrite branch is ready for alpha testing :-) Now I need assistance in testing

Try building fpc:
make all

Run the testsuite:
cd fpc/tests
make clean all digest TEST_FPC=../compiler/ppc386
compare the resulting faillog from trunk and from your branch

and profiling it.

As a first step, I consider the result of
cd compiler
time make cycle
with a hot disk cache before and after a change as good enough to see if a change hurts performance.

When changing things like unit loading or file handling, I time lazarus compilation as well.

If the time changes, I usually use callgrind to find out what has changed.

Can somebody do that, or give me instructions how to do that myself? The current version still uses global variables,

This is imo no problem so far, better do small steps and let's do it properly.

Basically, I'am against this, time should be spent better in optimizations where all program benefit. Especially fillchar is probably a problem.

Just a another remark:

uses clause of pbase must be cleaned up: it causes a lot of cyclic and strange dependencies. Especially the dependency on parser_opl is a no go. IMO there needs to go a lot of code into parser_opl.

That's just the reason for my proposition. I suspect that fillchar is slow only on fresh memory, due to RAM cache misses and copy-on-write.

DoDi

Right, some references could be removed just now. But I left them there, in preparation of moving global variables into the parser base class, as references or method arguments. The uses in the implementation cannot cause cyclic unit references.

The parser_opl reference is required just now. It can be replaced later, by a common front-end factory and registered parser classes.

DoDi

> The first version of the OO rewrite branch is ready for alpha testing :-)

Well, the alpha test revealed some problems, that have been fixed (except one).

Now the differences between branch and updated trunk already result in problems[1], that make testing the OO compiler near impossible. This raises the question, how to proceed with the integration of both version?

The longer we wait the more it will be improbable that both versions ever can be merged, by SVN/patch features. That's why IMO a quick decision is required, whether the refactored scanner/parser classes shall become part of the trunk, so that further refactoring (of classes and globals) can be tested on top of the new trunk. I'm willing to update the branch to a form, that is acceptable for integration - please specify what has to be done therefore.

> Now I need assistance in testing Try building fpc: make all

There remains one bug, of unknown reason. How can this be debugged?

How is the compiler/ppc386 created?

What's the Windows equivalent of "time"?

[1] Lazarus (code tools) also have problems with the recent update. Will the parsers be updated soon? Who is doing that?

DoDi

That would be easier to answer if you would at least mention a) which difference
causes
b) which problem

In whichever way you are used to debug: with a debugger, with writeln's, ... The compiler is no different than any other program in that respect.

By "make all"

[1] Lazarus (code tools) also have problems with the recent update. Will the parsers be updated soon? Who is doing that?

From this remark I guess that you are talking about the change to rtl/objpas/fgl.pp? Please don't make people guess when asking for information.

To avoid problems with your branch simply use an older revision of the RTL to test (r15646 or earlier). In the future you may want to branch the entire FPC repository rather than only the compiler if you want to avoid such problems.

Jonas

Problem is: breaking trunk too often scares even more people to use trunk. But we need people using trunk to get it tested as well as possible so we must do our best to keep trunk also in a good shape, see below.

I'm willing to update the branch to a form, that is acceptable for integration - please specify what has to be done therefore.

I didn't have time to look at the last commits yet, so I cannot comment on this in detail of the restructuring.

But requirements are always: no regressions for fpc itself, the test suite and if possible lazarus.

[...] [1] Lazarus (code tools) also have problems with the recent update. Will the parsers be updated soon? Who is doing that?

What problem?

Mattias

In our previous episode, Hans-Peter Diettrich said:

(I wouldn't do performance testing on Windows, the buildtime is significantly longer there, which skews results)

Sorry, I cannot find out why I had to change part of the statement procedure. SVN[1] doesn't show trunk changes since the creation of the branch, for the units I had in mind. Possibly I've only been fooled by Lazarus code tool problems, not sure right now :-(

The problem is a Nil reference, most probably resulting from changed management of the global variables. I just found out that parser.compile() is used in different context, once for the command line file argument, and multiple times for the compilation of used units. In the latter case a ppumodule object already exists, so that both cases must be handled slightly differently. Most probably I missed that difference in my initial changes to the parser setup for a new module :-(

[1] How can I find out the differences between trunk and branch versions, using [Tortoise]SVN? Or the differences between two revisions in general? Kdesvn seems to offer an option "compare revisions"...

While I can use Lazarus to debug the compilation of a single source file, no such feature exists for Makefiles. I even don't know how to translate an error address to a source code position. In Delphi I had a "jump to address..." menu item...

I don't know what has changed where and when. At least does Lazarus (trunk version) fail to determine the selected branch in {$IF }, and also disabled "Result" in code completion inside a function, in at least one case. That's why I think that something's broken now.

To avoid problems with your branch simply use an older revision of the RTL to test (r15646 or earlier). In the future you may want to branch the entire FPC repository rather than only the compiler if you want to avoid such problems.

One of my problems seems to be the fpc.cfg, that should refer to the trunk compiler and source files first, in the compilation of the new compiler, and to the new compiler (and an
older source revision) later. Where should I place such different config files, or how do I select one for Make?

DoDi

Problem is: breaking trunk too often scares even more people to use trunk. But we need people using trunk to get it tested as well as possible so we must do our best to keep trunk also in a good shape, see below.

Full ACK.

Most probably my assumption was wrong, that the compiler code has been touched since I created the OO branch. Now I have to revert parts of my modifications, that seem to have introduced the bugs.

DoDi

I use fpc/tests/utils/fptime.pp

(I wouldn't do performance testing on Windows, the buildtime is significantly longer there, which skews results)

For make cycle it doesn't matter much especially since the windows rtl make file uses a build unit.

Use make all as "Execute before" step in Lazarus, check all 3 checks and uncheck the compiler "Call on" checks in the last page of the project options (Compiler options -> compiler).

Marc

After these changes I get an error:

Unable to run the tool "Executing command before": Program make not found

But make.exe is correctly entered in the environment options (the one that came with Lazarus 0.9.28).

DoDi

Zitat von Hans-Peter Diettrich :

Under windows the program is called 'make.exe'.
If you do not specify a path the IDE searches it in the PATH.

But make.exe is correctly entered in the environment options (the one that came with Lazarus 0.9.28).

If you want to use that make, use the macro $(make).

Mattias

Aha!

If you want to use that make, use the macro $(make).

Thanks :-)

DoDi

What is the OO rewrite?

What is the OO rewrite?

It's an SVN branch, aimed at making the compiler more object-oriented. It should simplify the maintenance of the compiler code, the test of different algorithms and procedures for specific parts, and finally the use of threads and installable front- and back-ends.

Unfortunately I killed this second approach (after the parser_rewrite branch), by too complex changes without detailed knowlege about the currently used global variables and other inobvious dependencies. Even if the refactored parser is still working in general, the framework fails in certain situations, for yet unknown reasons :-(

So I have to explore the dependencies amongst the many modules first, and the unexpected changes of variables from code in arbitrary places. Even if I can understand - to some degree - that related subroutines reside in different units, to avoid cyclic unit dependencies, such problems must be solved now in a better way (by encapsulation).

Since most compiler internals are undocumented, I appreciate any assistance in figuring out that puzzle. Some questions can be answered without detailed knowledge of the compiler, by simply collecting all places in the code, where certain critical variables are changed. This cannot be achieved by a static analysis of the code, instead chains of calls have to be found out, that can change variables (i.e. the state of the application or of its components) in an unexpected way. For that purpose the suspect variables can be changed into properties, of a unit or class, so that their use can be debugged. It would be very helpful when somebody could help in instrumenting the compiler code this way, and in finding out what's going on under the surface.

DoDi