Lazarus 1.3 r44997M FPC 2.6.4 i386-linux-gtk 2 - Ubuntu 14.04 --------------------------------------------------------------------------------------

I need to develop a game for all possible Operating Systems.

It will be based on time. I will use hundreds of time.

I wanted some tips with low-level routines to improve performance.

I'll be using multimedia features (sound and images) and access to the server using any internet protocol.

I plan on using compiler directives for each operating system to use API to improve performance.

Thanks

Junior wrote

It will be based on time. I will use hundreds of time.

Take a uses hardware timers where available, with an easy to use API for the developer.

Regards,
- Graeme -

I agree. But the original question could also be read as a plea for good practice tips when using multimedia, network comms and so on.

yes

I will investigate this component.

thanks

Graeme Geldenhuys wrote

Take a uses hardware timers where available, with an easy to use API for the developer.

IO took a look.

Seemingly this is only available for X86 and X86_64.

In Windows, "QueryPerformanceCounter" is called. I supposed (I don't have a Lazarus on Windows at hand) this library call in fact uses a hardware feature of the CPU instead of doing a time cosuming system call.

But on Linux the EpikTimer (exactly like "Now") just does "fpgettimeofday()" (exactly like "Now") and this finally does a system call.

Are there no better options ?

Thanks,
-Michael

How did you get that idea? The wiki page even explicitly mentions ARM.

In Windows, "QueryPerformanceCounter" is called. I supposed (I don't have a Lazarus on Windows at hand) this library call in fact uses a hardware feature of the CPU instead of doing a time cosuming system call.

Do you mean direct access to the hardware bypassing the OS? I thought that wasn't possible in Windows?

Unless something's changed, I do not believe that to be the case. I've previously compiled and tested a program that uses it on SPARC/PPC/ARM Linux.

I've got reservations about excessive use of floating point, but at present I believe it's still the best option available.

Reinier Olislagers wrote

The wiki page even explicitly mentions ARM.

Right. As it does a system call in Linux, this should be independent of the CPU arch

Do you mean direct access to the hardware bypassing the OS? I thought that wasn't possible in Windows?

I don't know how the dynamic library call in Windows works.

AFAIR, all modern X86 CPUs feature a CPU register and an ASM instruction for that purpose. As this is just a read, there is no reason for this ASM instruction to be protected in any way.

I'll recheck ASAP.

-Michael

Yes, it WORKS on arm.

But on all systems except i386, you can just as well use Now() and Sleep(), because that is what epiktimer uses:

(sources quoted from the lazarus-ccr repository)

function SystemTicks: TickType;
{$IFDEF Windows}
begin
   QueryPerformanceCounter(Result);
{$ELSE}
var t : timeval;
begin
   fpgettimeofday(@t,nil);
   Result := (TickType(t.tv_sec) * 1000000) + t.tv_usec;
{$ENDIF}

and

function TEpikTimer.SystemSleep(Milliseconds: Integer):Integer;
{$IFDEF Windows}
begin
   Sleep(Milliseconds);
   Result := 0;
end;
{$ELSE}
   {$IFDEF CPUX86_64}
begin
   Sleep(Milliseconds);
   Result := 0;
end;
   {$ELSE}
var
   timerequested, timeremaining: timespec;
begin
   timerequested.tv_sec:=Milliseconds div 1000;
   timerequested.tv_nsec:=(Milliseconds mod 1000) * 1000000;
   Result := fpnanosleep(@timerequested, @timeremaining) // returns 0 if ok
end;
{$ENDIF}
{$ENDIF}

Why you would not use fpnanosleep on CPUX86_64 as well is a mystery to me...

Epiktimer is probably the most overrated component on lazarus-ccr. No idea why people still recommend it, unless I missed something :(

Michael.

If in doubt, take a look at ReactOS ;)

QueryPerformanceCounter calls NtQueryPerformanceCounter (which is a system call), which in turn calls KeQueryPerformanceCounter (which is implemented by the Hardware Abstraction Layer) which is defined here at around line 250:
http://svn.reactos.org/svn/reactos/trunk/reactos/hal/halx86/generic/timer.c?revision=58489&view=markup

Regards,
Sven

On linux, glibc (and others) will route gettimeofday() (and clock_gettime() for certain clock IDs) via vDSO and no syscall will be called, so it's very fast. I don't think the fpc rtl does this, though?

More info in the vdso man page for which functions are supported for which architectures.

Henry

Henry Vermaak wrote

On linux, glibc (and others) will route gettimeofday() (and clock_gettime() for certain clock IDs) via vDSO and no syscall will be called, so it's very fast. I don't think the fpc rtl does this, though?

I checked with X86 32 Bit Linux: you are right.

e.g. http://x86.renejeschke.de/html/file_module_x86_id_277.html

talks about the ASM instruction "RDPMC":

Hence, I don't know if accessing the " performance-monitoring counter" this is possible / viable in the fpc RTL.

-Michael

That's hardly surprising, since the various processor ID etc. registers were available in real mode.

There's an obvious pitfall there: a poor implementation might /think/ that it's directly accessing the counters when in actual fact it's being virtualised or fixed up by a signal handler. There's debate elsewhere about inadvertently relying on the OS to fix up e.g. alignment errors, with a vast performance hit being suggested for e.g. Solaris if poor coding practice forces the OS to straighten things out.

Mark Morgan Lloyd wrote

There's an obvious pitfall there: a poor implementation might /think/ that it's directly accessing the counters when in actual fact it's being virtualised or fixed up by a signal handler.

Regarding just reading the "performance-monitoring counter" Register (not creating a timer event e.g. by means of a signal) it seemingly just depends on the allowance bit that is (or is no) set by the os if or if not this can be done in user mode. Of course we would need a guaranteed behavior of an Arch/OS to do a viable implementation

-Michael

Plus confidence that it wasn't being emulated slowly in a virtualisation layer.

This doesn't happen on x86 for me (with Debian stable, at least). I can't find the implementation in glibc, either.

If you really want to use the performance counters, you'll have to use the perf subsystem.

This doesn't have anything to do with vDSO, though. The standard library doesn't use any assembler to read these registers. It calls into a page that the kernel makes available to userspace that contains certain functions where the overhead of a syscall is deemed excessive/undesirable. This kernel code will then read the time (using HPET/TSC to get nanoseconds):

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/x86/vdso/vclock_gettime.c

Henry

This is a good idea, as the same stuff might be available for other CPU archs, potentially doing system calls that just do some software based things instead of using CPU hardware.

It might be viable to do the implementation of this in the RTL or LCL to provide arch and OS independent calls to the users. (i.e. integrating an advanced version of the Epik Timer package in the library)

-Michael

Michael Van Canneyt wrote

Why you would not use fpnanosleep on CPUX86_64 as well is a mystery to me...

My copy does on x86_64, plus a few other improvements. It seems the lazarus-ccr version is a bit out of date - at least against my version. I'll make a plan in sharing my improvements.

Regards,
Graeme

Hopefully you got rid of the fpgettimeofday, or else the component is no better than using Now() :-)

Michael.

Michael Van Canneyt wrote

Hopefully you got rid of the fpgettimeofday, or else the component is no better than using Now() :-)

Michael,

Even though I of course was aware of this, I did used fpgettimeofday to implement TTimer with the "ActiveNoGui" WidgetType, as I did not find something more appropriate in the RTL.

Of course I could try to directly use the X86's "Performance counter" register, but there were discussions if same would by reliable usable in CPU / OS combinations. I also could try to use of course I do need vDSO in Linux arcs (is that always available ?) Of course in the end I need something that works for any CPU and OS.

At best, fpgettimeofday itself should be improved (maybe using "Performance counter" register if the Arch allows for or vDSO if the OS allows for).

-Michael

--- sorry, previous mail seems to be digested by some scrambler ---

Michael Van Canneyt wrote

Hopefully you got rid of the fpgettimeofday, or else the component is no better than using Now() :-)

Michael,

Even though I of course was aware of this, I did used fpgettimeofday to implement TTimer with the "ActiveNoGui" WidgetType, as I did not find something more appropriate in the RTL.

Of course I could try to directly read the X86's "Performance counter" register, but there were discussions if same would by reliable usable in CPU / OS combinations. I also could try to use vDSO in Linux archs (is that always available ?)

Of course in the end I need something that works for any CPU and OS.

At best, fpgettimeofday itself should be improved (maybe using "Performance counter" register if the Arch allows for or vDSO if the OS allows for).

-Michael

There is nothing cross platform. GetTickCount64 comes closest.

Only in recent versions, and I am not sure it is always enabled, however I have not studied the subject in depth.

Of course in the end I need something that works for any CPU and OS.

There is nothing. EpikTimer pretends to fill this gap, but it does not.

Don't get me wrong: I have nothing against the epiktimer, but it is presented as some super cross-platform solution (or so I perceive it).

Which is simply not correct. Unless you use the i386 version of the compiler, it is not better than using now().

At best, fpgettimeofday itself should be improved (maybe using "Performance counter" register if the Arch allows for or vDSO if the OS allows for).

FYI: fpgettimeofday is a direct kernel call, there is nothing to improve.

get_clock() and friends could be used, but we need to add this to the (base)unix units first. Now it is only available in the linux unit. I seem to recall that Marco Van De Voort was going to look into this, but memory is somewhat fuzzy on this subject...

Michael.

Yep.

If necessary I could try to do something myself (i.e. an improved version of fpgettimeofday), with alternative {$if -ed implementations for different settings of the arch and OS settings. But thoroughly testing this seems like a nightmare.

In fact in Linux it does call fpgettimeofday() (or did I install an old version ? )

I do know (I did ASM stepping), but it does not need to stay that way. (Or maybe there is a platform independent function in the RTL that is used to delegate to fpgettimeofday)

If the arch and OS allows for it could work differently (e.g. use vDSO or use a performance counter register, adding value calculated an initializing action.

-Michael

It needs to stay that way, that is what the function is for: expose the POSIX GetTimeOfDay call. It is an official API and need not be changed.

If the arch and OS allows for it could work differently (e.g. use vDSO or use a performance counter register, adding value calculated an initializing action.

fpGetTimeOfDay is used to implement Now(). This should not be changed.

For timers and whatnot, the fpgettimeofday call is simply not suitable. It is an incredibly slow call, and needs to be replaced with something else. get_clock() seems to be the way to go.
vSDO is a very specific linux mechanism; I would recommend against it.

Michael.

Michael Van Canneyt wrote

fpGetTimeOfDay is used to implement Now(). This should not be changed.

Why not, If there are better ways ?

I did a quick search in the RTL sources and found that there is "GetTickCount64" that maybe could be used as a central cross-platform function to implement certain timing stuff (including NOW).

Right now this is not the case. fpGetTimeOfDay is called in several other locations. I suppose, here GetTickCount64 could be used.

Depending on "HAVECLOCKGETTIME" GetTickCount64 just calls fpGetTimeOfDay.

In - more advanced ? - archs. GetTickCount64 calls "clock_gettime".

In my X86 32 Bit Linux it indeed does call clock_gettime.

But this again is just a system call and so I suppose it does not help to improve performance.

-Michael

Michael Schnell wrote

At best, fpgettimeofday itself should be improved (maybe using "Performance counter" register if the Arch allows for or vDSO if the OS allows for).

Only the speed can be improved by calling into the vDSO, not the precision. You need to use clock_gettime() for nanosecond precision (and to be unaffected by discontinuous clock changes).

Henry

I will stop the discussion, you clearly have no idea what you are talking about :)

Michael.

Henry Vermaak wrote

Only the speed can be improved by calling into the vDSO, not the precision. You need to use clock_gettime() for nanosecond precision

I see.

Is clock_gettime not available via vDSO ? To me this seems even more viable than for fpGetTimeOfDay.

(and to be unaffected by discontinuous clock changes).

Thanks for pointing out this

-Michael

Yes, as I said earlier. It's in the man page, too. And in the kernel source I pointed to.

Henry

"Better" needs to be defined here. If you mean "faster", then you need to call through the vDSO, so just link to the C library and use their implementation. Chances are that you are linking to the C library anyway (we're on the lazarus list after all). There isn't much reason for fpc to implement this, but perhaps it warrants a comment in the documentation (e.g. "glibc may have a much faster implementation that doesn't use a syscall, if you're worried about performance").

gettimeofday() is the correct way to get the real wall clock time, there is no "better" way for fpc to implement it (except for using clock_gettime() with CLOCK_REALTIME, since technically gettimeofday() is obsolete).

gettimeofday() is not what you want to use for a timer, though, since it will change when someone sets the time on the system. In that case you want to use clock_gettime() with CLOCK_MONOTONIC, or even CLOCK_MONOTONIC_RAW (linux only). GetTickCount64 will use this on linux, but it reduces the precision to milliseconds.

You seem a bit confused about some of these things, and people are talking at cross purposes, so I'll end my part in the discussion here.

Henry

I note that IBM mainframes have a counter which is defined as incrementing one particular bit at a 1mSec rate. The behaviour of bits to its right varies depending on the machine, and bits to its left count as expected. I can't remember the fixed-rate bit position, but it strikes me as an elegant intermediate format where an absolute date etc. isn't needed.

Henry, thank you for explaining it very clearly.

I no longer had the desire to do so.

Michael.

Henry Vermaak wrote

You seem a bit confused about some of these things, and people are talking at cross purposes,

Right you are.

I e.g. was inappropriately ignoring the fact that the time of day can be unaffectedly modified under the hood.

Now I know better. (Disregarding resolution that in fact could be handled in the RTL) seemingly we need as well a truly continuous time base plus a "time of day" time base.

Thanks,
-Michael

Henry Vermaak wrote

Yes, as I said earlier. It's in the man page, too. And in the kernel source I pointed to.

Henry,
Thanks a lot for you patience !

So this (using vDSO to call "clock_gettime" in fact is what the OP was searching for, and what would needed tp be implemented in EpikTimer for make same acceptable for Michael vC (and myself), or even better might be provided by the standard RTL.

-Michael

"Michael Van Canneyt" wrote

get_clock() seems to be the way to go. vSDO is a very specific linux

mechanism; I would recommend against it.

Nevertheless we could add vDSO support to the Linux unit so that users can make use of it without the need to link to libc or implement it themselves. :)

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140515/95e8d78c/attachment.html>

No argument there.

Michael.

Hence fpGetTimeOfDay would need multiple implementations.

clock_gettime seems to be Linux-only anyway.

-Michael

So this (using vDSO to call "clock_gettime" in fact is what the OP was searching for

May I quote the OP for you:

"I need to develop a game for all possible Operating Systems."

Any mismatch between his statement and your conclusion?

R.

The conversation was good, but I wonder if the epiktimer component would be the correct option. I need precision, I'll be working with many timers in milliseconds.

(Windows 32 bits/64bits; Linux 32 bits/64bits; Android; MacOS; among others)

Thanks

The conversation was good, but I wonder if the epiktimer component would be the correct option. I need precision, I'll be working with many timers in milliseconds.

In general it does it's job good enough for realtime purposes but take a look at the castle game engine they have their own cross plattform time keeping solution. For simple stuff even now() is good enough. I am not sure why you need precision because games normally don't need that precise time keeping. I hope you know what you are doing because basing your game on timers doesn't sound like a reasonable design decision to me.

R.

You know the game Travian? The game will be similar to it, but with other themes.

Travian is an online game using a browser, in my project, the game will be a desktop with the same features of Travian program, but with internet access.
The multimedia part will benefit by seeking the images, sounds and videos on your local computer.

No normal (not dedicatedtly "realtime") OS provides "decent" millisecond precision. The OS might stall any program at any time for seconds.

But a game is not supposed to be "hard realtime" but "very soft realtime". Hence it should not matter if such glitches once and again.

I supposed, an enhanced version of "EpicTimer" might be useful. Decent arch/OS independent sub-millisecond timer support (e.g. using QueryPerformanceCounter and "clock_gettime" via vDSO) would be even nicer.

-Michael

Reimar Grabowski wrote

OP: "I need to develop a game for all possible Operating Systems."

"EpicTimer" uses "QueryPerformanceCounter" for Windows. I did not check into this, but I suppose this already is what he needs.

But it uses "fpgettimeofday" in Linux. This is what Michael vC states to be not appropriate.

Hence using the vDSO to call "clock_gettime"sing in Linux should make the OP happy with Linux and Windows.

Sorry, but no Idea about other OSes.

-Michael

Dear Mr. Michael Van Canneyt,

I'm adding some clarification and context to your ongoing comments about EpikTimer in this and other threads.

<< Snip EpikTimer code>>

Why you would not use fpnanosleep on CPUX86_64 as well is a mystery to

me...

There's no mystery here. I wrote EpikTimer in 2003 with a few months of experience on a 32 bit version of Lazarus. The code still works, but a lot has changed around it over the last 11 years.

Epiktimer is probably the most overrated component on lazarus-ccr. No

idea why people still recommend it, unless I missed something :(

No mystery here either. You never miss an opportunity to criticize EpikTimer for reasons I've never been able to understand.

If there's another component or technique that does this better, can you please recommend it rather then relentlessly criticizing the one I contributed?

> Of course in the end I need something that works for any CPU and OS. There is nothing. EpikTimer pretends to fill this gap, but it does not. Don't get me wrong: I have nothing against the epiktimer, but it is

presented as some super cross-platform solution (or so I perceive it).

To clarify, EpikTimer was created in 2003 to help simplify the measurement of elapsed time. It used the x86 TSC hardware, if available and reverted to the system clock if it wasn't. I needed this capability and thought others might as well. My original version was documented to only work on 32 bit Linux and was never presented as a "super cross-platform solution". Over the years other contributors modified it for Windows and 64 bit operation.

Epiktimer was written during the early days of the Lazarus project when cross platform capability was an evolving dream and the IDE was just starting to be stable on Linux. The code is 11 years old, but despite it's age, EpikTimer still works well, is downloaded many times per week and actively used across a variety of projects. It could use an update and re-release that I was starting to work on but at this point I've lost my enthusiasm to complete it.

-Tom

On Mon, May 12, 2014 at 7:57 AM, Michael Van Canneyt <

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140517/2da39670/attachment-0001.html>

Nevertheless, I have named the reasons explicitly every time.

It is simply frustrating to see that people ask for something cross-platform, *better than Now()* for timing, and then epiktimer is recommended. Never fails.

Your component is probably perfectly suited for Intel 32-bit. On other platforms, it simply is not: there, one might as well use Now() directly, since that is what epiktimer uses.

I realize that my repeated critique sounds harsh; But, please, do not take the critique personal:

Your part (Intel 32-bit) undoubtedly is good for the job. I have never judged that. It is the repeatedly acclaimed support for other platforms that I am having serious difficulties with.

The day that part is rewritten, you will no longer hear a peep from me. Before writing, I always check in the hope that someone finally took up the challenge and made it better than now() on non-i386 platforms. Till now, I was always disappointed.

Several useful suggestions have been made already in the course of this thread. Maybe someone will finally take up the challenge.

Michael.

Henry Vermaak wrote

gettimeofday() is not what you want to use for a timer, though, since it will change when someone sets the time on the system. In that case you want to use clock_gettime() with CLOCK_MONOTONIC, or even CLOCK_MONOTONIC_RAW (linux only).

And that is exactly what my local copy of EpikTimer does for over a year alread - just one of many improvements I've made to my copy of EpikTimer, but sadly never got around to sharing the code (which I'll do shortly).

----8<-------------8<-------------8<-------------8<-------------8<----

function SystemTicks: TickType;
{$IFDEF Windows}
begin
  QueryPerformanceCounter(Result);
{$ELSE}
const
  CLOCK_MONOTONIC = 1;
        { Experimental, no idea if this works or is implemented correctly }
        function newGetTickCount: Cardinal;
        var
          ts: TTimeSpec;
          i: TickType;
          t: timeval;
        begin
          // use the Posix clock_gettime() call
//          if
do_syscall(syscall_nr_clock_gettime,TSysParam(CLOCK_MONOTONIC),TSysParam(@ts))
//  0 then //kernels 2.4.* does not support
          if clock_gettime(CLOCK_MONOTONIC, @ts)=0 then
          begin
            // Use the FPC fallback
            fpgettimeofday(@t,nil);
            // Build a 64 bit microsecond tick from the seconds and
microsecond longints
            Result := (TickType(t.tv_sec) * NanoPerMilli) + t.tv_usec;
            Exit;
          end;
          i := ts.tv_sec;
          i := (i*MilliPerSec) + ts.tv_nsec div NanoPerMilli;
          Result := i;
        end;
begin
  Result := newGetTickCount;
{$ENDIF}
end;
----8<-------------8<-------------8<-------------8<-------------8<----

Regards,
- Graeme -

Michael Schnell wrote

clock_gettime seems to be Linux-only anyway.

No, that is incorrect. It is simply FPC's outdated FreeBSD support, which I believe I have raised a bug report for.

On my FreeBSD 9.1 and 10 systems....

----8<-------------8<-------------8<-------------8<-------------8<---- [~]$ man clock_gettime
CLOCK_GETTIME(2) FreeBSD System Calls Manual CLOCK_GETTIME(2)

NAME
clock_gettime, clock_settime, clock_getres ? get/set/calibrate date and time

LIBRARY
Standard C Library (libc, -lc)

SYNOPSIS
     #include <time.h>
     int
     clock_gettime(clockid_t clock_id, struct timespec *tp);
     int
     clock_settime(clockid_t clock_id, const struct timespec *tp);
     int
     clock_getres(clockid_t clock_id, struct timespec *tp);

DESCRIPTION
The clock_gettime() and clock_settime() system calls allow the calling process to retrieve or set the value used by a clock which is specified ----8<-------------8<-------------8<-------------8<-------------8<----

Regards,
- Graeme -

Tom Lisjac wrote

I'm adding some clarification and context to your ongoing comments about EpikTimer in this and other threads.

Tom, as I have mentioned in this thread, I have some improvements in my local copy of EpikTimer, which oddly I haven't shared with the community. I'll create a patch set and pass it on to whomever maintains EpikTimer (in Lazarus-CCR I believe?).

Regards,
- Graeme -

Tom Lisjac wrote

I'm adding some clarification and context to your ongoing comments about EpikTimer in this and other threads.

What Michael also keeps forgetting, is that EpikTimer gives a unified timing interface, which is very handy in cross-platform applications. Yes we could manually use QueryPerformanceCounter under Windows, clock_gettime() under Linux and FreeBSD, and fpgettimeofday() everywhere else.... but EpikTimer gives us a clean API to use, and takes care of the platform differences for us. Makes for much cleaner code in our applications, which I personally think is nice!

Regards,
- Graeme -

On Sun, May 18, 2014 at 12:17 PM, Graeme Geldenhuys <

Hi Graeme,

I remember you mentioned making some improvements and it would be great to include them in a new release. The demo program is compiling again thanks to some much appreciated help from the list last week. Also installed Lazarus on the Raspberry Pi to see if EpikTimer has any issues on ARM. If so, I'd like to get them resolved before we post an updated zip on the Lazarus-CCR.

Thanks,

-Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140518/0b4726e2/attachment.html>

I have never tried to suggest that a component as epiktimer does not fill a gap.

Once more, the only issue I am having is the continued suggestion that it is better and offers more fine-grained/accurate timing than simply using now() when going cross-platform.

If the result of my continued hammering is that now someone will actually contribute improvements, I can only appreciate the effort and you won't hear me about it ever again.

Michael.

Thanks a lot !

If the code is decently tested and documented, and available for all supported platforms and OSes, I vote for adding it to the RTL

-Michael

Graeme Geldenhuys wrote

{$IFDEF Windows} begin QueryPerformanceCounter(Result);

Did you check that this is a low overhead function ?

{$ELSE} do_syscall(syscall_nr_clock_gettime,TSysParam(CLOCK_MONOTONIC),TSysParam(@ts))

It seems that a syscall is not necessary with some / many environments.

vDSO seems to provide a user-land implementation in (many) current Kernel versions.

Maybe directly accessing the appropriate CPU registers is possible / viable in some architectures

-Michael

Michael Van Canneyt wrote

Your component is probably perfectly suited for Intel 32-bit.

(At least) For Linux I doubt this. vDSO or even direct access to the CPU register (if really possible) seem more appropriate than doing a syscall..

I suppose a version of EpicTimer that does what is the best possible way for all Archs and OSes supported by fpc is a lot of work, hard to be tested and a moving target. But a viable project anyway.

-Michael

The register is already used for tick counts.

But I am eagerly awaiting your vDSO implementation.

Michael.

Michael Van Canneyt wrote

If the result of my continued hammering is that now someone will actually contribute improvements,

Similar as with ActiveNoGui :-) :-) :-) .

I obviously would like to use an improved version of EpricTimer there as a very handy in cross-platform unified
timing interface for TTimer.

-Michael

Michael Van Canneyt wrote

But I am eagerly awaiting your vDSO implementation.

Still hoping for Tom ...

But if EpicTimer is not improved I might be willing to test this for ActiveNoGui.

-Michael

I suspect because epiktimer has better PR/visibility than the native FPC alternatives.
Why not update the EpikTimer page with information regarding the alternatives?

Michael Van Canneyt wrote

The register is already used for tick counts.

Please let me know where.

For TTimer in AvtiveNoGu I now use TThread.GetTickCount64 (or SysUtils.GetTickCount64)-

I did ASM stepping into same and it does a syscall. (Linux X86/32)

-Michael

I'm hoping that the epiktimer implementation will be improved.

Meanwhile, enough suggestions on how to improve the implementation have been made.

Michael.

Line 362 of epiktimer or thereabouts.

// Execute the Pentium's RDTSC instruction to access the counter value.
function HardwareTicks: TickType; assembler; asm DW 0310FH end;

Obviously only used if you get hardware ticks, not system ticks. (for the latter obviously a system call is done)

Michael.

Michael Van Canneyt wrote

. function HardwareTicks: TickType; assembler; asm DW 0310FH end;

OK. Found it.

Lets see how to do something arch independent....

-Michael

Michael Schnell wrote

I obviously would like to use an improved version of EpricTimer there ...

Obviously I can't _use_ EpricTimer there, as it uses (IMHO inappropriately) (graphics-) stuff that is not implemented in a non-GUI project. So I will need to extract some of the source code.

-Michael

Michael Schnell wrote

Obviously I can't _use_ EpricTimer there, as it uses (IMHO inappropriately) (graphics-) stuff that is not implemented in a non-GUI project.

You are clearly using a very outdated version. That was fixed 2+ years ago. Get the latest code from SubVersion instead of the ZIP download.

Regards,
- Graeme -

On Mon, May 19, 2014 at 11:54 AM, Graeme Geldenhuys <

There's no question that the component needs an update as the current zip on the Lazarus-ccr was packaged in 2006. SVN is more current and Graeme and I will get together and roll any additional improvements he's made into a new release.

If we need to make improvements to the core measurement routines for better cross platform support, some additional input would be helpful. For hardware timing, I see the following support across processor types:

http://en.wikipedia.org/wiki/Hardware_performance_counter

There is a Performance API (PAPI) standard for cross platform access to those counters, but probably not a candidate as it requires OS support. Regarding OS based timing, my experience is limited to Linux and Intel/AMD, so if there's a more universal approach, I'd be very interested in learning more about it.

Thanks,

-Tom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140519/b44e75d1/attachment.html>

You are welcome to test yourself. Google Search will show you that that is the recommended high-precision timing option under Windows.

It seems that a syscall is not necessary with some / many environments. vDSO seems to provide a user-land implementation in (many) current Kernel versions.

Tom and I would welcome patches for EpikTimer.

Maybe EpikTimer should move to the Git repo of Lazarus-CCR so others could easily clone and share their feature branches (say via Github).

Regards,
- Graeme -

Correction, it was fixed 4 years ago. :)

Regards,
- Graeme -

Fully agree... great idea!

-Tom

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140519/b36bfc36/attachment.html>

I'll let you take care of that then.... I don't have access to create a new bare git repo in Lazarus-CCR.

In the mean time, I created a Git clone from the SubVersion Lazarus-CCR which contains the full history for EpikTimer (as of this morning). I published that at my Github account as the "master" branch.

I then created another branch called "gg-changes" which contains some of my changes. I have multiple copies of EpikTimer on my various systems - I'll go through them all to collect all other changes too.

Here is a pull request summary:

=================================
The following changes since commit 9024600e (master):

Patch from Denis Golovan to fix epiktimer compilation in 32bit unixes (2010-09-15 06:40:58 +0000)

are available in the git repository at:

https://github.com/graemeg/epiktimer.git gg-changes

for you to fetch changes up to 79a9a8dd:

Adds git helper files to keep the repo clean (2014-05-20 06:29:28 +0100)

---------------------------------------------------------------- Graeme Geldenhuys (8):
Replaced magic numbers with constants
Removed redundant x86_64 SystemSleep() code.
minor code formatting - making it easier to read.
using a high resolution system kernel timer.
Replaces low-level do_syscall with clock_gettime instead Fixes compilation under FreeBSD
Updates the unit copyright notice.
Adds git helper files to keep the repo clean

.gitattributes | 27 +++++++++++++++++++++
.gitignore | 9 +++++++
epiktimer.pas | 102 +++++++++++++++++++++++++------------------------ 3 files changed, 92 insertions(+), 46 deletions(-) create mode 100644 .gitattributes
create mode 100644 .gitignore

=================================

Once you have setup a bare git repo on Lazarus-CCR you should be able to clone my Github one, and simply push it to Lazarus-CCR to retain the full history (at least the "master" branch). My "gg-changes" branch is obviously up to you.

Regards,
- Graeme -

Michael Van Canneyt wrote

Your component is probably perfectly suited for Intel 32-bit. On other platforms, it simply is not: there, one might as well use Now() directly, since that is what epiktimer uses.

Yes sure, but x86 is the most popular desktop platform, and a lot of people will compile for x86 even on a 64 bits desktop. And most people using Lazarus are using it for desktop apps.

Felipe

And your point is ?

Michael.

Graeme Geldenhuys wrote

Tom and I would welcome patches for EpikTimer.

I am not interested in EpicTimer in it's current stat, as it needs a GUI based LCL Widget Type to be compiled, and my interest regarding timing is the creation of the "ActiveNoGUI" widget type.

I suppose I some day soon will try to use vDSO directly in the appropriate "interfaces" unit I am working on. Of course I will provide the code I will be using there.

Maybe EpikTimer should move to the Git repo of Lazarus-CCR so others could easily clone and share their feature branches (say via Github).

IMHO, EpicTimer (or maybe only the "technical" part of same that does not support visibly placing the component) should reside in the FPC RTL (and it's svn), while in the LCL (and it's svn) the "visible" part (e.g. a sibling of the "technical" component) can be done.

Generally I am convinced that it is essential for a "quality" unit to _use_ as few stuff as possible, to restrict cross dependencies as much as possible.

-Michael

Felipe Monteiro de Carvalho wrote

And most people using Lazarus are using it for desktop apps.

While this of course is true, one of the the strategical advantages of fpc/Lazarus above Delphi is that it is inherently cross-platform: multiple CPU archs, multiple OSes, and multiple "WidgetTypes" allow for using it in multiple purposes, including Desktop, embedded, CGI, ...

Unfortunately many contributors ignore the non-Desktop uses and just assume a GUI enabled Widget type is available in all projects, even thought the functionality they are working on does not require this.

-Michael

Graeme Geldenhuys wrote

You are clearly using a very outdated version. That was fixed 2+ years ago. Get the latest code from SubVersion instead of the ZIP download.

Right you are,

The files contained in the ZIP are dated at max at 10/2006.

I never came across any other source. Can you point me to one ?

Thanks,
-Michael

Great !

Interesting read, indeed.

vDSO seems like the way to go in Linux. Here the Kernel is supposed to provide a fully user space function that accesses the hardware provided by the arch or emulates it in the best possible way.

-Michael

Graeme Geldenhuys wrote

Correction, it was fixed 4 years ago. :)

Seems more like 8 years :-) :-) :-)

-Michael

Graeme is right with four years. The dependency on LCL units was removed on 18th May 2010.

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140521/48797bf2/attachment.html>

based LCL Widget Type to be compiled, and my interest regarding timing is the creation of the "ActiveNoGUI" widget type.

Did you even take a look at the current state? The EpikTimer unit only requires units from the FCL and the RTL. Additionally Graeme uses it in his fpGui projects which doesn't provide a "GUI based LCL widget type" either...

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140521/550ab442/attachment.html>

Sounds great.

Trying to find the sources...

-Michael

I was just joking, nagging that a ZIP distribution had not been done since that :-[ .

-Michael

As a staring point:

In /usr/src/linux/Documentation/vDSO there are two C files that show how vDSO can be used without the need to attach to glibc.so.

(The old Kernel sources I have at hand right now (Suse 2013) only provide this for X86-64, though, and I did not find more suitable stuff with quick googeling.)

-Michael

requires units from the FCL and the RTL. Additionally Graeme uses it in his fpGui projects which doesn't provide a "GUI based LCL widget type" either...

Search for Lazarus-CCR on SourceForge, select SVN at the top and navigate to components/epiktimer.

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20140521/9dfd0986/attachment.html>

Works.

Thanks,
-Michael

Michael Schnell wrote

I am not interested in EpicTimer in it's current stat, as it needs a GUI based LCL Widget Type to be compiled,

Please READ the answers/replies before spreading FUD. As I said, EpicTimer doesn't have a GUI dependency since 4 years ago when I patched it. Please get an update from Lazarus-CCR's SubVersion repo, or get even later code from my Github repo (details posted in another message).

Regards,
- Graeme -

On 05/21/2014 07:46 PM, Graeme Geldenhuys wrote:

...
(This was no FUD, as the _officially_released_ (ZIP) distribution really suffers from the restriction I described. Don't be angry on me pointing this out. No pun to the good work done.)

As I said, EpicTimer doesn't have a GUI dependency since 4 years ago when I patched it. Please get an update from Lazarus-CCR's SubVersion repo, or get even later code from my Github repo (details posted in another message).

After Sven pointed me to the version of the code you talked about, I of course immediately tested it and found that you are right: it does not have such dependencies and I can easily _use_ "epictimer.pas" in the interfaces.pas unit that I am doing for ActiveNoGUI.

Now I really would like to see the file epictimer.pas in the fpc rtl svn !

I will continue the work on ActiveNoGUI depending on epictimer.pas instead of any internal implementation of access to a timebase.

Here I'd like to consider some suggestions (I of course can implement this locally and we can later discuss a release for the fpc RTL or whatever.

- function GetHardwareTicks could be a class function so I wold not need to create an instance to use it
- FHWTickSupportAvailable could be calculated in the initialization section so that it is not necessary to re-do this for any instance.
- property HWTickSupportAvailable could be a class property, as a consequence.

- I'd like to use a "GetTicks" class-function that provides raw ticks and automatically uses GetHardwareTicks if FHWTickSupportAvailable and software ticks if not.

This done we might consider using vDSO in Linux to provide straight arch independent support for all Linux systems.

-Michael

I could do this in a very straight forward way:

I moved several variables from TEpikTimer private to Implementation var:
var
   FSystemTicks:TimeBaseData; // The system timebase
   FHWCapabilityDataAvailable:Boolean; // True if hardware tick support
is available
   FHWTickSupportAvailable:Boolean; // True if hardware tick support is
available
   FHWTicks:TimeBaseData;     // The hardware timebase
   StartupCorrelationSample:TimebaseCorrelationData; // Starting ticks
correlation snapshot
   FMicrosecondSystemClockAvailable:Boolean; // true if system has
microsecond clock
   UpdatedCorrelationSample:TimebaseCorrelationData; // Snapshot of last
correlation sample

(For this, some properties needed to be modified to use set procedures and get functions that access the variables.)

I created an initialization section:
Initialization
begin
   InitTimebases;
end;

(To do so I moved several functions out of the TEpikTimer class so there could referenced

I redefind a function to be a class function:

       class function GetHardwareTicks:TickType; // return raw tick

value from hardware source
I defined a new class function:

       class function GetTicksFrequency:TickType;

I tested this and it works nice for me (Linux X86 32 bit): After the start of the application, I can simply do TEpikTimer.GetTicksFrequency and TEpiktimer.GetHardwareTicks.

In fact This is all I need.

I feel that it does make sense not to call InitTimebases with any TEpiktimer.Create (Of course this also can be achieved by just checking if it already had been called. But even here the implementation independent variables are necessary.)

I don't know if the modified version of EpikTimer.pas in fact is viable as a base for future development.

-Michael

(should we start a new mailing list thread ? )

Graeme,

While doing the said modifications I could not continue to have
       function SystemSleep(Milliseconds: Integer):integer; Virtual;
       function CalibrateCallOverheads(Var TimeBase:TimebaseData) :
Integer; Virtual;
       function CalibrateTickFrequency(Var TimeBase:TimebaseData):
Integer; Virtual;
as an overidable functions.

I understand that these functions are used in the TEpikTimer.Create. Hence when virtual you could create a sibling class of TEpikTimer that provides alternate implementations of these function.

Who would want to do this ? Is this necessary ?

If yes it might make sense to implement an alternator creator that is called to allow for doing a re-calibration with alternate implementations of these functions.

Would it be necessary to support multiple instances of different siblings of TEpikTimer which use individual calibration processes ?

-Michael