Hi,

I've spoken to Tom and he agreed. We moved the EpikTimer repository to Github. The Lazarus-CCR version will be deleted by Tom in due time.

I applied some of my changes to code in Github (more changes recently discussed in this mailing list will follow soon), fixed the "etdemo" project to be compilable again, updated the demo to use FPC Resources instead of the old *.lrs files.

I've also bumped the version number and created a new release. Hopefully this will resolve the issue about people downloading the 2006 old archive version.

I've also updated the Free Pascal Wiki page with the latest information.

Wiki Page:
http://wiki.freepascal.org/EpikTimer

Source code:
git clone https://github.com/graemeg/epiktimer.git

Release downloads:
https://github.com/graemeg/epiktimer/releases

Regards,
- Graeme -

Graeme Geldenhuys wrote

I've spoken to Tom and he agreed. We moved the EpikTimer repository to Github. The Lazarus-CCR version will be deleted by Tom in due time.

While I am happy about your move to do an "official release of EpikTimer. I'd like to add that IMHO it would be appropriate to have the file "epiktimer.pas" that is independent of Lazarus and makes sense to be use in not Lazarus based projects in the fpc RTL svn.

I did not check the latest release, yet, but I suppose there are some more files (that allow for "visible" placing of a TEpikTimer Instance on a form). Same should come as a Lazarus package.

Thanks for your work,
-Michael

Michael Schnell wrote

While I am happy about your move to do an "official release of EpikTimer. I'd like to add that IMHO it would be appropriate to have the file "epiktimer.pas" that is independent of Lazarus and makes sense to be use in not Lazarus based projects in the fpc RTL svn.

I don't understand exactly what you are trying to say. Is this:
- a request to Graeme
- a promise that you are going to provide a patch to the FPC RTL
- a remark
- something else

Thanks,
Reinier

Sorry for being unclear.

I meant this as a general request to the "powers" of the fpc (rtl) project with svn write access to allow Greame, Tom (or maybe myself) to include the TEpikTimer class in the next release of the official RTL. (And maybe help with doing so)

In the Lazarus Forum I am trying to discuss some improvements in EpikTimer, I would like to introduce before that move. (I already did and tested this with the pre-release file that had been provided on "Lararus CCR").

Thanks for listening,
-Michael

Michael Schnell wrote

On 05/26/2014 11:55 AM, Reinier Olislagers wrote: I meant this as a general request to the "powers" of the fpc (rtl) project with svn write access to allow Greame, Tom (or maybe myself) to include the TEpikTimer class in the next release of the official RTL.

Well, the normal procedure for doing this is to submit a patch if you want something to be added to FPC.

If you need help with that, you could put up your modified code on a public repository.

An example is somebody who modified Lazarus to use/create other widgetsets: https://github.com/x2nie/LiteZarus

Reinier Olislagers wrote

Well, the normal procedure for doing this is to submit a patch if you want something to be added to FPC.

Thanks for your hints.

As I locally do use svn, I suppose I will be able to do a patch for a not yet existing file :)

I can't do a patch that will have the file integrated in the make process, though.

Anyway before doing this I'd like to do some more improvements and tests. (including trying to use )

A big

Reinier Olislagers wrote

Well, the normal procedure for doing this is to submit a patch if you want something to be added to FPC.

Thanks for your hints.

As I locally do use svn, I suppose I will be able to do a patch for a not yet existing file :)

I can't do a patch that will have the file integrated in the make process, though.

Anyway before doing this I'd like to do some more improvements and tests (including trying to use vDSO).

A big problem with EpikTimer is that is _should_ be decently useful with all archs and OSes.

This might be a nightmare to test.

-Michael

Michael Schnell wrote

file "epiktimer.pas" that is independent of Lazarus and makes sense to be use in not Lazarus based projects in the fpc RTL svn.

As I mentioned in the Lazarus mailing list. I don't believe everything should always be dumped in the FPC repository. It adds the extra burden for the FPC team to maintain it, makes it harder for the general public to make/commit changes, others to experiment with new ideas etc. Plus EpikTimer (as a project) contains more than just a single .pas unit. It has Lazarus package files, demos etc - those don't belong in the FPC repository, and will only fragment tho EpikTimer codebase.

Of course you are welcome to submit a patch to the FPC team (I can't stop you), but please change the name so as not to conflict with the official EpikTimer component.

Regards,
Graeme

On Mon, May 26, 2014 13:12, Michael Schnell wrote: .
.

A big problem with EpikTimer is that is _should_ be decently useful with all archs and OSes. This might be a nightmare to test.

You don't need to do that - the new addition may be enabled selectively for the CPU+OS combinations for which it has been already tested successfully (and this approach is preferred over enabling it for everything after testing just a small subset of all targets supported by FPC). Additional targets would then get enabled later by the individual target maintainers as appropriate.

Tomas

project with svn write access to allow Greame, Tom (or maybe myself) to include the TEpikTimer class in the next release of the official RTL. (And maybe help with doing so)

I don't see a need for this in the FCL yet. At least not until the dust that you are whirling up has settled down as FPC has a quite slow release cycle.

Regards,
Sven

In fact I would like to use EpikTimer in my upcoming "ActiveNoGUI" Lazarus WidgetType. "ActiveNoGUI" is especially useful with small Gadgets that typically don't have X32 or X64 archs.

That is why I am after multi-arch support.

And that is why I feel that it makes sense to use EpikTimer in "ActiveNoGUI" instead of directly including the timer related code. With EpicTimer the "ActiveNoGUI" implementation itself would be automatically arch and OS independent.

-Michael

I absolutely agree.

Nonetheless I feel that (a decently multi-arch enabled version) of TEpikTimer does make sense in the FCL as a courtesy class (in fact TList is a "courtesy class" as well, as it does not support syntax candy but just is provided directly for the user to use). Especially as the multi-arch support of EpikTimer is really critical and it would be very helpful to know that this issue is managed by the fpc team.

Hence including it in the svn seems nice to me, especially regarding that I'd like to use it in a Lazarus enhancement.

-Michael

TList and TEpikTimer are two completely different things. The former is already required out of Delphi compatibility while TEpikTimer is not. Additionally container structures can be considered essential for basically every software while a timing component is not...

Especially as the multi-arch support of EpikTimer is really critical and it would be very helpful to know that this issue is managed by the fpc team.

And you don't think we FPC developers have enough to do with the remaining RTL and FCL on all those platforms we support?

Hence including it in the svn seems nice to me, especially regarding that I'd like to use it in a Lazarus enhancement.

Then let it be put in Lazarus in components/epiktimer ... Lazarus releases more often than FPC does.

Regards,
Sven

Hi Sven,

You raised all the points which I already mentioned to Michael Schnell - why I think EpikTimer should NOT be in the FCL.

* FPC has too a slow release cycle
* No need to burden the FPC team with supporting yet another component
* It will reduce the possibility for experimentation and easy contributions from others.
* EpikTimer contains more than just a single unit. The other files don't belong in FPC. So why fragment the project's source code.

I'm perfectly happy where EpikTimer is now. If anybody wants to use it in a project, download and use it. If anybody wants to contribute, then send a pull request or patches.

I also told Michael S. that if he does submit a patch to the FPC team (I can't stop him), make sure it has a different name, so as not to conflict with the official EpikTimer component.

Just my 2c worth.

Regards,
Graeme

Sven Barth wrote

The former is already required out of Delphi compatibility while TEpikTimer is not.

(For the record: I do hate this argument. Especially regarding the latest moves of Delphi getting more and more incompatible to itself. fpc can easily be both more "stringent", more versatile and more platform independent.)

Additionally container structures can be considered essential for basically every software while a timing component is not...

(For the record: I do hate this argument. fpc (and even Delphi) can very decently be used for embedded software and much more non-Desktop or "special Desktop" (e.g. Gaming) purposes.)

And you don't think we FPC developers have enough to do with the remaining RTL and FCL on all those platforms we support?

Of course you are right. I did not want to push anybody. I just see that here the best possible multi-platform support is provided.

Then let it be put in Lazarus in components/epiktimer ... Lazarus releases more often than FPC does.

Hmm.
While I personally in fact would like to _use_ it for a Lazarus enhancement, I would be happy to see it usable without Lazarus.

-Michael

Graeme Geldenhuys wrote

I also told Michael S. that if he does submit a patch to the FPC team (I can't stop him), make sure it has a different name, so as not to conflict with the official EpikTimer component.

In fact I do want the best possible stuff and not a fork. I am just trying to help (as I would like to use it in the said current project). I of course would discuss any enhancements with the original creators and am not at all inclined to break anything. .

-Michael

Graeme Geldenhuys wrote

Plus EpikTimer (as a project) contains more than just a single .pas unit. It has Lazarus package files, demos etc - those don't belong in the FPC repository, and will only fragment tho EpikTimer codebase.

Which is really great !

But how to decently provide a fully released version of EpikTimer to not Lazarus enabled users of fpc ?

-Michael

It is not about environments for running programs. The classes are on a completely different level.

Lists (and containers) are general purpose, almost language-level constructs. Timers simply are not. They may be important to you, no argument there.

It's just a source file. You cownload and it in FPC means it puts an additional burden on FPC devs, however you turn it. This is a hobby project, still, so we put in what we feel is useful and can be maintained. None seems willing to take it on, but you can stil download and .

But how to decently provide a fully released version of EpikTimer to not Lazarus enabled users of fpc ?

https://github.com/graemeg/epiktimer/releases

Download. Extract. Copy needed file(s). Profit.

Where is the f****** problem?

R.

In fact I do want the best possible stuff and not a fork. I am just trying to help (as I would like to use it in the said current project).

In that case some attention points:
- help implementing and testing fine grained timings on *nix. Now it only has a special case for linux.
- Seems high precision is not used on anything but x86.
- Is rdtsc safe for CPUs that can vary clock of cores independently like Core Mono? What if the process changed CPU to a different clocked core?

Reimar Grabowski wrote

Where is the f****** problem?

Supposedly none (as I already stated in the Lazarus list).

Thanks !
-Michael

(What I'd like to prevent is that here again the award for the best way to avoid ubiquitous usage of community based code is close at hand:
- someone asks "what can I use to ...."
- (s)he gets the name of a perfectly working thingy
- (s)he searches for it in the internet
- what (s)he finds is an outdated release and documentation that does not easily give away it the thingy is usable for exactly this purpose
- (s)he takes pains to install it
- does not work
- frustration, even though the current release would have been a perfectly fit.
)

While I of course see you point, besides high resolution, another benefit of EpikTimer (at least on X86) is low overhead access to a time source. And this is what makes it useful for my needs.

Already the current version on X86 (at least on Linux 32 bit) it uses the appropriate ASM instruction: Perfect (but supposedly some testing regarding the safety issue you state).

I suppose on any arch Linux vDSO could be used to provide both lowest possible overhead and highest possible resolution. So this is what I will try to do some day soon.

-Michael

- The rdtsc instruction needs to be protected from out of order execution. Some people use cpuid, which is expensive. It looks like the linux kernel uses mfence or lfence/mfence depending on CPU type.

... meaning the current version of incorrect ?!?!?

What would the effect of ignoring this = ?

Things like this is why I'd rather use dVSO.

-Michael

Marco van de Voort wrote

- Is rdtsc safe for CPUs that can vary clock of cores independently like Core Mono? What if the process changed CPU to a different clocked core?

I've read that on recent CPUs, the TSC is unaffected by the actual clock rate of the CPU. On linux, The TSC gets calibrated and the synchronisation is tested, which may result in the TSC clock source being marked as unstable and disabled. In this case, it will fall back to using other clock sources (HPET is next in line on my computer).

Henry

In our previous episode, Henry Vermaak said:

Yes, on Nehalem and newer afaik the clock is on the uncore. But that is still to new a requirement to assume for general purpose code like RTL and FCL, if you agree to clip real old stuff.

Both the Core2 generations (Conroe and Wolfdale) are still too common.

On linux, The TSC gets calibrated and the synchronisation is tested, which may result in the TSC clock source being marked as unstable and disabled. In this case, it will fall back to using other clock sources (HPET is next in line on my computer).

I assume the same system underlies queryperformancecounter and family on Windows. But that means you need to use OS timing functions, and not ASM.

I assumed that, too. All the clocksource calibration and selection happens at startup, so by the time you call
clock_gettime()/QueryPerformanceCounter() it knows whether to query the TSC/HPET/whatever.

Blindly making assumptions about TSC stability can get you into trouble. Microsoft advises against this, too:

http://msdn.microsoft.com/en-gb/library/windows/desktop/ee417693%28v=vs.85%29.aspx

Henry

Indeed. What's worse is that it _always_ uses the TSC on x86, without knowing whether it's actually a reliable clock source for the particular hardware configuration.

I can only recommend to never use this component, just use an ifdef to call QueryPerformanceCounter/()clock_gettime() based on OS.

Things like this is why I'd rather use dVSO.

Calling the vDSO will certainly make things faster. I don't know what the overhead of QueryPerformanceCounter() is.

Henry

In our previous episode, Marco van de Voort said:

still to new a requirement to assume for general purpose code like RTL and FCL, if you agree to clip real old stuff.

_EVEN_ if you agree to clip real old stuff (read: anything Pre core2 intel like Pentium D and Core duo)

Michael Schnell wrote

In fact I do want the best possible stuff and not a fork. I am just trying to help (as I would like to use it in the said current project).

Then fork it on Github and start publishing your changes. I'll gladly review the suggestions and merge it what works.

Regards,
Graeme

Henry Vermaak wrote

an ifdef to call QueryPerformanceCounter/()clock_gettime() based on OS. > Things like this is why I'd rather use dVSO. Calling the vDSO will certainly make things faster.

For me, the point is, that with vDSO, the Linux infrastructure will handle the dirty stuff that is involved with such low level greatly arch depending things. It is supposed to work "out of the box", even if for certain archs no support from the hardware is available and provide the best possible support by the Kernel software.

For C programmers, this is automatically in place, as libC and the Kernel are done appropriately. fpc (supposedly for good reasons) is determined to reduce libC dependance as much as possible. Hence the rtl needs to provide arch dependent low level stuff internally.

As it is provided directly by the Linux Kernel, vDSO is a way to take advantage of Kernel provided arch-independence without using libC.

-Michael

Marco van de Voort wrote

. But that means you need to use OS timing functions, and not ASM.

Meaning either syscalls or vDSO.

As in Linux syscalls do a usermode->Kernelmode->usermode switch, they introduce a huger overhead.

In Windows I suppose syscalls usually are not done directly by the rtl, but functions calls to a Kernel dll are done, so that that Windows.provide dll might decide to stay in usermode if possible.

-Michael

Henry Vermaak wrote

Blindly making assumptions about TSC stability can get you into trouble. Microsoft advises against this, too: http://msdn.microsoft.com/en-gb/library/windows/desktop/ee417693%28v=vs.85%29.aspx

As my Target is Linux, this does not help with the implementation. But the warning not to use TSC holds, anyway !

-Michael

Graeme Geldenhuys wrote

Then fork it on Github and start publishing your changes. I'll gladly review the suggestions and merge it what works.

OK. (Of course only after I did as much testing as possible - in fact I can't do by far enough.)

This discussion shows that the arch depending stuff in the library/packet really should be part of the rtl. There is not much chance to finally provide a decent / manageable / stable arch independent functionality to the users.

-Michael

"Michael Schnell" wrote

In Windows I suppose syscalls usually are not done directly by the rtl,

but functions calls to a Kernel dll are done, so that that Windows.provide dll might decide to stay in usermode if possible.

On Windows the QueryPerfomanceCounter/Frequency calls always do a syscall, because the corresponding code is implemented in the HAL which is only accessible from kernel mode.

Regards,
Sven

"Michael Schnell" wrote

In Windows I suppose syscalls usually are not done directly by the rtl,

but functions calls to a Kernel dll are done, so that that Windows.provide dll might decide to stay in usermode if possible.

Addendum: yes, the RTL calls the core DLLs of the Win32 subsystem like kernel32.dll, but they are just that: the core DLLs of the Win32 subsystem. They don't implement any core OS functionality like hardware/device management, because that is done by the NT kernel below it which is called by the Win32 DLLs if needed using the ntdll.dll which acts as a gateway to the kernel (some calls will do a Syscall then, for example hardware related functions, while others like functions to handle list structures will stay in the current mode). That's basically a remnant of the microkernel nature of the NT OS.

Regards,
Sven

This is rather similar to what I assumed.

Hence, if (as your other post suggests) that Kernel dll switches to Kernel mode to access TSC in an X86 archs while this is not necessary, that is a fault of Windows' and not a fault of fpc's and hence we can't help it. As we found, "blindly" reading is not a good idea, so a really fast rtl function would need to detect the CPU sub-arch and act appropriately. I understand that this is close to impossible.

-Michael

In our previous episode, Michael Schnell said:

Maybe, but is that relevant? We were talking about precision, not speed.

In Windows I suppose syscalls usually are not done directly by the rtl,

No. Windows calls kernel32/user32, which then mostly calls nt.dll functions to do the actual syscalls afaik. And a lot more is userland on Windows.

Marco van de Voort wrote

Maybe, but is that relevant? We were talking about precision, not speed.

I have been talking about overhead (speed) all the time. This is my intention to discuss the issue. Bus regarding time measurement of course speed and precision is highly related, so IMHO it's really decent to handle both aspects together.

-Michael

Marco van de Voort wrote

> In Windows I suppose syscalls usually are not done directly by the rtl, No. Windows calls kernel32/user32, which then mostly calls nt.dll functions

I feel free to translate this to: "In Windows, the fpc rtl calls kernel32/user32, ..."

Which is _exactly_ what I said.

-Michael