Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-devel / 2019-02

I'll be straight

10 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

J. Gareth Moreton

I'll be straight here.  Exactly what should I even be looking at and working on?  Being self-guided is evidently not cutting it because recently, everything I see and think has potential or could use improvement has been rejected or dismissed out of hand because it damages maintainability, is deemed too risky to even experiment with or touches a unit that was derived from a third party source and one doesn't want to make changes to it in the name of being able to compare it line-for-line (even though the third party source is in C)

If I'm the wrong kind of person for this project, I understand, and you can boot me out.  What am I but naïve 'new blood' who is overly optimistic and doesn't understand risk?

Gareth

Reply by email

J. Gareth Moreton

Thanks Michael,
In the last patch that Jonas almost immediately closed, the speed savings were inconclusive because the number of cycles saved is probably only a few dozen, but I would argue it makes the code a bit more reasonable too because it replaces things like loc[low(loc)] with loc[0] and fillchar with a for-loop that initialises each element of an array to zero (it's slightly faster because the element size is a multiple of 16, while fillchar is general-purpose and spends a lot of time jumping around and even performing a multiplication before it starts filling things up).  I guess seeing it marked as "won't fix" within 20 minutes was a moment of horror. I just want to avoid working on something, building passion for it which is an unfortunate character flaw of mine, and then seeing it rejected or ignored when it's finished.

I guess I can get a bit impatient, because some of the things I can improve in, say, the peephole optimiser, are dependent on other patches (in this case, the x86-64 optimiser overhaul) being accepted or rejected rather than being left in limbo (trying to make the changes now, before the patch is accepted, will just cause merge conflicts later).  Mind you, I understand the need to do careful testing before committing something (and I've made bug fixes to the overhaul already based on problems people have found).  The node dumping feature is also slightly essential for a couple of things I need to research that Florian's approved of (some node-level optimisations for things like the floor() function).

I've just hit a couple of impasses since I'm waiting on the node dump feature to be approved and the status of the optimiser overhaul (I tend to be quite good at doing peephole optimisations... at least a peephole optimisation was the very first piece of code I submitted to FPC).

Other things that I'm experimenting with or researching, like attempting to allow pure assembler routines to be inlined, partly as a way to support intrinsics and also to provide a speed  boost for some internal functions like Trunc (which is only a single command under x86_64), is something that Florian is afraid to begin contemplating, whereas I like to jump right in and try it out, and see if it blows up in my face or not.  Mind you, as a volunteer with a relatively unknown track record outside of Free Pascal, I can be a bit of a pariah sometimes.  The way I see it though, being a programmer who likes to play around with gaming and mathematics, I seek to help make Free Pascal a tool that is more than suitable for the job and something I can tell others "yes, this is a good language to use".  Now, I accept that for certain time critical things or things that take weeks to complete, where even the slightest saving can multiply into a whole day (e.g. folding proteins or determining the primality of a Mersenne number), nothing beats raw assembly language - I just have to remember that most people don't like to mess with it!

Gareth aka. Kit

On Sat 09/02/19 07:52 , Michael Van Canneyt sent:

J. Gareth Moreton wrote
I'll be straight here.  Exactly what should I even be looking at and working on?  Being self-guided is evidently not cutting it because recently, everything I see and think has potential or could use

improvement

has been rejected or dismissed out of hand because it damages maintainability, is deemed too risky to even experiment with or touches

a

unit that was derived from a third party source and one doesn't want to make changes to it in the name of being able to compare it line-for-line (even though the third party source is in C)

No-one stops you from doing this anyway, but you will need to prove that the
replacement code is equally correct or better if you want to get it included.
The more sensitivethe code, the better the proof must be...

If I'm the wrong kind of person for this project, I understand, and you can boot me out.  What am I but naïve 'new blood' who is overly optimistic and doesn't understand risk?

No-one is a priori the wrong kind of person for this (or any) project. Any software has parts that are more sensitive and less sensitive. Diving in and starting to tinker with the sensitive parts is of course more prone to be met with criticism or reluctance...

I am not familiar with the compiler internals, but executable speed surely

can still be optimized a lot. I'm sure Florian, Sven or Jonas can point to

things that can use some work.

Michael.

Reply by email

Michael Van Canneyt
to J. Gareth Moreton: I'll be straight here.  Exactly what should I even be looking at and working on?  Being…
J. Gareth Moreton wrote 8 lines
I'll be straight here.  Exactly what should I even be looking at and working on?  Being self-guided is evidently not cutting it because recently, everything I see and think has potential or could use improvement has been rejected or dismissed out of hand because it damages maintainability, is deemed too risky to even experiment with or touches a unit that was derived from a third party source and one doesn't want to make changes to it in the name of being able to compare it line-for-line (even though the third party source is in C)

No-one stops you from doing this anyway, but you will need to prove that the replacement code is equally correct or better if you want to get it included. The more sensitivethe code, the better the proof must be...

If I'm the wrong kind of person for this project, I understand, and you can boot me out.  What am I but naïve 'new blood' who is overly optimistic and doesn't understand risk?

No-one is a priori the wrong kind of person for this (or any) project. Any software has parts that are more sensitive and less sensitive. Diving in and starting to tinker with the sensitive parts is of course more prone to be met with criticism or reluctance...

I am not familiar with the compiler internals, but executable speed surely can still be optimized a lot. I'm sure Florian, Sven or Jonas can point to things that can use some work.

Michael.

Reply by email

J. Gareth Moreton

Well, at least we're having a good civil conversation about it!  As long as we can talk and discuss things, then there's no harm, and hopefully someone like me who is over-enthusiastic at times can still be useful!

I suppose loc[low(loc)] isn't necessarily bad and is guaranteed to be correct - just feels a little clumsy, that's all, plus with the almost universal standard of starting arrays at 0, low(loc) seems a bit superfluous (although looks neat when paired with "High", e.g. "for x := low(loc) to high(loc)" etc.)
All I can promise is that the code I write tends to be completely original.  I was exploring cpupara.pas with no knowledge of libffi, so looking for optimisations with fresh eyes.  Admittedly I was looking there while researching something else and just saw something and thought "that looks a bit inefficient" and also was a little bemused at the mixed standards (i.e. loc[0] and loc[low(loc)], and also using "fillchar" and a for-loop to initialise an array of the same record type).

I try to avoid hacks and produce clean code, or if I have to use something a little strange, like calling Create as a standard method rather than as a constructor (I did this in my patch for #34679), I make sure the comments explain clearly what's going on.
If I had to describe my own standard, it would be "iterative improvement".  I start with something rough (although hopefully still relatively clean according to a design plan) and then refactor it over time after leaving it for a bit, or in the case of FPC, coming to the code with no previous knowledge of it and studying how it works.  I also look for small performance gains that, by themselves, don't amount to much, but when several of them stack together, start to have a noticeable effect.

My biggest submission to date, but which is still pending analysis and approval by Florian and others, is the x86-64 optimiser overhaul (although it also affects i386 now as well, since separating out the 32-bit code for testing reasons just made everything incredibly untidy with $ifdef's).  That one is a case of "if it ain't broke, don't fix it", although given my preliminary tests show a 15% speed increase, it's a little bit deeper than just fixing it!  The main objective with the overhaul was to reduce the number of passes over individual assembler blocks - currently it's a minimum of 5 (pre-peephole, pass 1, pass 1 again, pass 2 and post-peephole), while the overhaul attempts to reduce this to 2 (it merges pre-peephole, the two pass 1 runs, and pass 2, while adding some extra code in the individual procedures to ensure the optimiser doesn't perform worse than before under -O1).  A few other developers spotted problems, mostly on Linux, but I believe those have been resolved now. Admittedly, the bounty that was offered a few months ago is a minor incentive too, although I play around with FPC because I love it!

I hope despite us being a little bit different in philosophies, we can find common ground and both improve FPC as a team.  Thanks for your understanding response, and I hope I wasn't too challenging or critical over #34679.

Gareth aka. Kit

On Sat 09/02/19 11:32 , Jonas Maebe sent:

J. Gareth Moreton wrote 12 lines
In the last patch that Jonas almost immediately closed, the speed savings were inconclusive because the number of cycles saved is probably only a few dozen, but I would argue it makes the code a bit more reasonable too because it replaces things like loc[low(loc)] with loc[0] and fillchar with a for-loop that initialises each element of an array to zero (it's slightly faster because the element size is a multiple of 16, while fillchar is general-purpose and spends a lot of time jumping around and even performing a multiplication before it starts filling things up).  I guess seeing it marked as "won't fix" within 20 minutes was a moment of horror.

When it comes to the compiler, I'm indeed usually inclined to the "if it ain't broken, don't fix it" school of thought. There are already plenty of things that break when necessary changes are performed, so why risk more breakage by minor touch-ups? As an aside, I also disagree that loc[low(loc)] is unreasonable. I consider it defensive programming. That said, since the convention is far from followed throughout all of that code, removing it doesn't make things really worse either.

That said, it is true that the parameter classification code for x86-64 is quite complex and probably takes a lot of time. Additionally, sometimes we even calculate the parameter layout for the same type two times in a row (first in push_addr_param, and then again if that one returned false).

I think a better way to tackle this would be to add a small cache (possibly just an array of 10 elements that is searched linearly; a hash table risks spending more time and memory/cache flushing on maintaining the table than gaining from it) that keeps track of recently requested parameter layouts. This would have a potentially much bigger impact, and at the same time would not require fiddling with the internals of the existing code.

Show quoted text 5 lines
I just want to avoid working on something, building passion for it which is an unfortunate character flaw of mine, and then seeing it rejected or ignored when it's finished.

I understand this is annoying. I realise that I am rather conservative in this respect. While I think I have good reasons for this stance (having spent a lot of time on fixing bugs added by other people while I wanted to work on my own stuff; not that other people have never been blocked by bugs I added, of course), it also bothers me that I discourage new contributors with this attitude.

I've just hit a couple of impasses since I'm waiting on the node dump feature to be approved and the status of the optimiser overhaul (I tend to be quite good at doing peephole optimisations... at least a peephole optimisation was the very first piece of code I submitted to FPC).

That's also how I started on FPC in 1997 :) But yes, it definitely was much easier back then to jump right in, because there were way fewer barriers to getting your code included. At the same time, we paid a hefty price for that for many years after that (from copyright issues to cleaning out hacks).

Jonas

Reply by email

Jonas Maebe
to J. Gareth Moreton: In the last patch that Jonas almost immediately closed, the speed savings were…
J. Gareth Moreton wrote 10 lines
In the last patch that Jonas almost immediately closed, the speed savings were inconclusive because the number of cycles saved is probably only a few dozen, but I would argue it makes the code a bit more reasonable too because it replaces things like loc[low(loc)] with loc[0] and fillchar with a for-loop that initialises each element of an array to zero (it's slightly faster because the element size is a multiple of 16, while fillchar is general-purpose and spends a lot of time jumping around and even performing a multiplication before it starts filling things up).  I guess seeing it marked as "won't fix" within 20 minutes was a moment of horror.

When it comes to the compiler, I'm indeed usually inclined to the "if it ain't broken, don't fix it" school of thought. There are already plenty of things that break when necessary changes are performed, so why risk more breakage by minor touch-ups? As an aside, I also disagree that loc[low(loc)] is unreasonable. I consider it defensive programming. That said, since the convention is far from followed throughout all of that code, removing it doesn't make things really worse either.

That said, it is true that the parameter classification code for x86-64 is quite complex and probably takes a lot of time. Additionally, sometimes we even calculate the parameter layout for the same type two times in a row (first in push_addr_param, and then again if that one returned false).

I think a better way to tackle this would be to add a small cache (possibly just an array of 10 elements that is searched linearly; a hash table risks spending more time and memory/cache flushing on maintaining the table than gaining from it) that keeps track of recently requested parameter layouts. This would have a potentially much bigger impact, and at the same time would not require fiddling with the internals of the existing code.

I just want to avoid working on something, building passion for it which is an unfortunate character flaw of mine, and then seeing it rejected or ignored when it's finished.

I understand this is annoying. I realise that I am rather conservative in this respect. While I think I have good reasons for this stance (having spent a lot of time on fixing bugs added by other people while I wanted to work on my own stuff; not that other people have never been blocked by bugs I added, of course), it also bothers me that I discourage new contributors with this attitude.

I've just hit a couple of impasses since I'm waiting on the node dump feature to be approved and the status of the optimiser overhaul (I tend to be quite good at doing peephole optimisations... at least a peephole optimisation was the very first piece of code I submitted to FPC).

That's also how I started on FPC in 1997 :) But yes, it definitely was much easier back then to jump right in, because there were way fewer barriers to getting your code included. At the same time, we paid a hefty price for that for many years after that (from copyright issues to cleaning out hacks).

Jonas

Reply by email

Marģers .
to J. Gareth Moreton: In the last patch that Jonas almost immediately closed, the speed savings were…

 
fillchar is one useful function, but should be
avoided from use in time critical code by any
cost. Only reasonable way to improve fillchar is
to make it internal function, where fpc can
decide, depending on parameters, what will be best solution for filling mem with specified value. But it will also slow down compilation time. Not sure fpc core team will accept that kind improvement.

Optimized code for speed most likely became less
readable and less maintainable. It's misfortune of optimization.

----- Reply to message -----
Subject: Re: [fpc-devel] I'll be straight
Date: 2019. gada 9. februāris 10:35:51
From: J. Gareth Moreton
To: FPC developers' list

Thanks Michael, In the last patch that Jonas almost immediately

closed, the speed savings were inconclusive
because the number of cycles saved is probably
only a few dozen, but I would argue it makes the
code a bit more reasonable too because it replaces things like loc[low(loc)] with loc[0] and fillchar with a for-loop that initialises each element of
an array to zero (it's slightly faster because the element size is a multiple of 16, while fillchar
is general-purpose and spends a lot of time
jumping around and even performing a
multiplication before it starts filling things
up).  I guess seeing it marked as "won't fix"
within 20 minutes was a moment of horror.

Reply by email

Bart
to J. Gareth Moreton: In the last patch that Jonas almost immediately closed, the speed savings were…
AM J. Gareth Moreton wrote
I guess I can get a bit impatient,

Reaction time on ptaches in the fpc part of the bugtracker is rather long (especially when compared to the lazarus part of the bugtracker), I find that discouraging as well.
I have had patches (simple ones for RTL or packages) waiting there for months without feedback, even after asking for it on this ML.

I can see that it must be even more frustrating when you supply lot's of ideas and possible patches (I only did a few, most likely less than 10).

Also the bugtracker could do well with a little more active management (moving issues to another project, confirming a bug, setting relationships: in my experience it is lacking behind a bit).

Anyhow, in my experience with this project so far, if keep you supplying good patches, you wil get feedback (and help if needed), so don't give up..

Bart

Reply by email

Michael Van Canneyt
to Bart
Bart wrote 10 lines
On Sat, Feb 9, 2019 at 9:35 AM J. Gareth Moreton wrote: > I guess I can get a bit impatient, Reaction time on ptaches in the fpc part of the bugtracker is rather long (especially when compared to the lazarus part of the bugtracker), I find that discouraging as well. I have had patches (simple ones for RTL or packages) waiting there for months without feedback, even after asking for it on this ML.

This is not intentional. It can happen we don't see bugs.

And I recall distinctly saying several times that you can mail directly if you see no-one is reacting.

Michael.

Reply by email

Bart
to Michael Van Canneyt
PM Michael Van Canneyt wrote
This is not intentional. It can happen we don't see bugs.

Of course this can happen, it happens on the lazarus side as well.

And I recall distinctly saying several times that you can mail directly if you see no-one is reacting.

You very may have.
That however feels a little bit to intrusive to me (I can't blame you for that now, can I?).

Bart

Reply by email

Florian Klämpfl
to J. Gareth Moreton: I'll be straight here.  Exactly what should I even be looking at and working on?  Being…
J. Gareth Moreton wrote 13 lines
I'll be straight here.  Exactly what should I even be looking at and working on?  Being self-guided is evidently not cutting it because recently, everything I see and think has potential or could use improvement has been rejected or dismissed out of hand because it damages maintainability, is deemed too risky to even experiment with or touches a unit that was derived from a third party source and one doesn't want to make changes to it in the name of being able to compare it line-for-line (even though the third party source is in C) If I'm the wrong kind of person for this project, I understand, and you can boot me out.  What am I but naïve 'new blood' who is overly optimistic and doesn't understand risk?

Well, when changing something you have always to think about the gains. Making some difficult code more complex for little speed gain is simply not worth the effort. As I guess you are talking about 35045: the parameter managers are a very complex piece of software as they:
- have to take care of the abi to implement
- the different abis of a CPU on the same architecture
- different abis on different OSes
- getting all the information right so FPC can process it further Furthermore, bugs in the parameter managers are typically hard to track down as they might appear only during very rare circumstances. So nobody really wants to touch this code. Even if you do so, the person who commits the patch will be later on responsible to get things fixed the patch breaks it.

To get patches accepted the following per-requisites are very helpful:
- the advantage of the patch is clearly visible:
* better code generation on as much as possible platforms
* measurable speed ups of the compiler
* code clean ups: their use can be easily measured in the number of lines remove
* rewrite of some ugly code: this is hard to judge, so before doing such a rewrite, discuss first on the mailing list if the code is really considered ugly
- it is regression tested by the submitter on as much as possible affected targets, if it is a x86 patch, at least on i386 and x86-64
- it contains no "noise", i.e. unrelated changes
- the patch is finished

Reply by email