Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2013-04

using mtprocs

4 messages · 2 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

John Lee

I have a program that captures xml data every 2-6 secs from a power sensor, win , xp. Would like to use freearc ebvery minute or so to compress the previous data in parallel w/o stopping the capture. Running arc from cmd line using exec eg exec('/c arc.exe outfile infile'); The compression may take 30 secs or so.

Is using mtprocs as in
http://wiki.lazarus.freepascal.org/Parallel_procedures#Getting_MTProcs

the best way to do this? Or threads. Any other ideas?

John

Reply by email

Mattias Gaertner
to John Lee
Show quoted text 10 lines
I have a program that captures xml data every 2-6 secs from a power sensor, win , xp. Would like to use freearc ebvery minute or so to compress the previous data in parallel w/o stopping the capture. Running arc from cmd line using exec eg exec('/c arc.exe outfile infile'); The compression may take 30 secs or so. Is using mtprocs as in http://wiki.lazarus.freepascal.org/Parallel_procedures#Getting_MTProcs the best way to do this? Or threads. Any other ideas?

mtprocs uses TThreads.
It creates them on demand and only once and destroys them at the end of the program. If there is nothing to do the threads are waiting using no CPU cycles and can wake up very quickly. So they are like light weight threads.
The nice thing is that mtprocs tries to guess the number of cores and automatically creates this amount of threads. You can limit the number of threads or create more if you want. You can free the threads. I use mtprocs for several xml servers. One server for example reads files with 12 cores in parallel, scaling nicely. Another program of mine starts 100 pings in parallel. mtprocs supports recursion and exceptions.
And it is written entirely in FPC, so it works X-platform.

Mattias

Reply by email

John Lee
to Mattias Gaertner

Thanks - however, my attempt to modify the test program by adding the exec as below so it works in 'parallel' with the main program doesn't work, so I must have missed something.

program Test;
{$mode objfpc}{$H+}
uses windows,classes, dos,  crt, sysutils,MTProcs;
var j:longint;
// a simple parallel procedure
procedure DoSomethingParallel(Index: PtrInt; Data: Pointer; Item:
TMultiThreadP♦
var
  i: Integer;
begin
  writeln(Index);
  if index=1 then
  begin
  //this compression, using arc (but it could be 7z takes 30 secs)
   exec('c:\windows\system32\cmd','/c c:\utils\arc a -mx tmp
c:\live_sol\ans.dat');
   exit;
  end;
  for i:=1 to Index*1000000 do ; // do some work
end;
begin
  ProcThreadPool.DoParallel(@DoSomethingParallel,1,5,nil); // address,
startind♦
 //main prog eg capturing the xml
  for j:=1 to 10 do
   begin write(j,' '); sleep(10000); end;
end.

John

Show quoted text 31 lines
> I have a program that captures xml data every 2-6 secs from a power sensor, > win , xp. Would like to use freearc ebvery minute or so to compress the > previous data in parallel w/o stopping the capture. Running arc from cmd > line using exec eg exec('/c arc.exe outfile infile'); The compression may > take 30 secs or so. > > Is using mtprocs as in > http://wiki.lazarus.freepascal.org/Parallel_procedures#Getting_MTProcs > > the best way to do this? Or threads. Any other ideas? mtprocs uses TThreads. It creates them on demand and only once and destroys them at the end of the program. If there is nothing to do the threads are waiting using no CPU cycles and can wake up very quickly. So they are like light weight threads. The nice thing is that mtprocs tries to guess the number of cores and automatically creates this amount of threads. You can limit the number of threads or create more if you want. You can free the threads. I use mtprocs for several xml servers. One server for example reads files with 12 cores in parallel, scaling nicely. Another program of mine starts 100 pings in parallel. mtprocs supports recursion and exceptions. And it is written entirely in FPC, so it works X-platform. Mattias _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Reply by email

Mattias Gaertner
to John Lee
Thanks - however, my attempt to modify the test program by adding the exec as below so it works in 'parallel' with the main program doesn't work, so I must have missed something.

What does not work?

Show quoted text 14 lines
program Test; {$mode objfpc}{$H+} uses windows,classes, dos, crt, sysutils,MTProcs; var j:longint; // a simple parallel procedure procedure DoSomethingParallel(Index: PtrInt; Data: Pointer; Item: TMultiThreadP♦ var i: Integer; begin writeln(Index); if index=1 then begin

Index runs from 1..5. This is executed only in the first step.

Show quoted text 14 lines
//this compression, using arc (but it could be 7z takes 30 secs) exec('c:\windows\system32\cmd','/c c:\utils\arc a -mx tmp c:\live_sol\ans.dat'); exit; end; for i:=1 to Index*1000000 do ; // do some work end; begin ProcThreadPool.DoParallel(@DoSomethingParallel,1,5,nil); // address, startind♦ //main prog eg capturing the xml for j:=1 to 10 do begin write(j,' '); sleep(10000); end;

Do you see this output?

end.

Mattias

Reply by email