Hello.

Some developers treat me as dinosaur because I use threads in place of doing parallelism.

Huh, ok, but why parallelism is better and how to do it with fpc ?

Thanks.

Fre;D

-----
Many thanks ;-)

Hi,

a nice article I've found long ago when I was researching the same topic "Threading/Concurrency vs. Parallelism" is the following :

http://tutorials.jenkov.com/java-concurrency/concurrency-vs-parallelism.html

and I tried the multithreadprocslaz package in Lazarus :

http://wiki.freepascal.org/Parallel_procedures

Happy coding ....

Regards,

Showing nicely that parallelism in a single process implies threads. I would not pay too much attention to what these developers are saying.

Michael.

Hi,

On 29/3/2017 11:15 μμ, Michael Van Canneyt wrote:

< snip >

Showing nicely that parallelism in a single process implies threads. I would not pay too much attention to what these developers are saying.

besides hardware parallelism, how software parallelism ( in the sense of true simultaneous execution ) in a single process without using ( OS dependent ) threads can be achieved ?

regards,

I would not pay too much attention to what these developers are saying.

Excellent idea, I will do the same ;-)

how software parallelism ( in the sense of true simultaneous execution ) in a single process without using ( OS dependent ) threads can be achieved ?

Huh, it is exactly was I do not understand...

Thanks.

Fre;D

-----
Many thanks ;-)

besides hardware parallelism,

Is it possible, with fpc, to assign one processor (if multi) for a thread and say to the system to use this one only for the thread ? But maybe it will gives more problems than solutions.

Fre;D

-----
Many thanks ;-)

Hi,

Yes, it's called thread/process affinity, and most operating systems have a specific API for it. But one should treat such things extremely carefully, otherwise with badly designed data access concurrency you end up with potentially hard to identify structural problems like false sharing (see Wikipedia on this) and others. This can be especially bad for for example on NUMA architectures, or others, where data storage locality to a given core is not obvious. (Just think about clusters, etc.)

In other words, as the OS knows best your hardware's layout and how it can best provide performance, energy, bandwith, etc, it's usually very wise to rely on the OS to solve the thread to CPU core assignments for you.

Bottom line: parallelism and threading is a very complex problem and very diverse area, and anyone laughing at any approach is just shows how little they know about true depth of the whole topic, IMO. Thinking about any technical solution, or way of implementation as a silver bullet for everything is a very "expert beginner" approach.

My 2 cents.

Charlie

@Karoly Balogh (Charlie/SGR)

Perfect, I have now all the arguments to defend the "Dinosaur Threading" choice.

Thanks.

Fre;D

-----
Many thanks ;-)

Parallelism is what hipster Node programmers do, threads are everything else.

/jk

Look at an OpenCL tutorial for more information. If you select a CPU device (instead of a GPU device with hundreds of compute units) then the API will automatically allocate the amount of threads available on the all cores available (which may be only 2). It’s still threaded but scales easily depending on the task. I don’t know why but I got the OpenCL CPU target to process multiple times faster than just using the CPU directly.

Parallelism is only helpful if you have a problem involving massive amounts of discrete tasks which are part of larger task and can be designed as such. If your problem is designed like this then you can offload the processing to the GPU and get some serious performance benefits. The catch is that reading/writing memory to the GPU is slow so unless you’re processing on large batches of data you’ll spend more time accessing memory than processing.

Regards,
Ryan Joseph

fredvs wrote

@Karoly Balogh (Charlie/SGR) Perfect, I have now all the arguments to defend the "Dinosaur Threading"choice. Thanks.

I'd second Charlie's point, and add that a very small change to a system's layout, e.g. a DIMM on a NUMA node going dodgy and being excluded at boot, can have a drastic effect if process or interrupt affinity has been locked down inadvisedly.

Another thing to take into account is that when many people mention parallelisation they're thinking of something like OpenMP, which in practice is built on top of threads and processes and obviously introduces substantial overhead. It's good for large jobs carefully designed, particularly on very large datasets.

Some languages were designed with at least some measure of parallelisation in mind. I'd highlight in particular APL, which tried to minimise control flow variation so that in principle at least a function could be applied to all elements of an array simultaneously. Granted that in those days CPUs were a scarce resource, but even then there were understood to be hazards which could break the programming model.

Ryan mentions OpenCL etc., which is about as close as we've got to "a vector processor on every desktop".

Finally, I suggest that you look at least briefly at https://en.wikipedia.org/wiki/Vector_Pascal which appears to have some quite good stuff in it.

fredvs wrote

Huh, ok, but why parallelism is better and how to do it with fpc ?

Parallelism within a process always is based on threads.

AFAIK, fpc does not (yet) provide a more convenient abstraction for parallelism (such as parallel loops) than TThread.

-Michael

Mark Morgan Lloyd wrote

Finally, I suggest that you look at least briefly at https://en.wikipedia.org/wiki/Vector_Pascal which appears to have some quite good stuff in it.

Quoting from manual section 5.4.

Future machines like the Larrabee will have considerably wider SIMD registers, increasing the benefits of SIMD code. But newer chips also have multiple cores. For these, the recent versions of the Glasgow Pascal Compiler will parallelise across multiple cores if the arrays being worked on are of rank 2. The Pascal source code of the program remains the same independently of whether it is being targeted at a simple sequential machine, a SIMD machine or a multi-core SIMD machine. Targeting is done by flags passed to the compiler:

[...]

Two threads are dispatched to process the work using a fork - rejoin paradigm. The run time library is built on top of pthreads. For a two core machine, two server threads are initiated at program start up. These wait on a semaphore until post_job passes them the address of a procedure and a stack frame context within which the procedure is to be executed.

Either I'm dense, which is entirely possible, or the author of that article didn't do a good job of making his point, even assuming he had one.

Thinking about this a bit more I would say that "threads" are a means (vehicle) to achieve parallelism. I think that in the majority of programming contexts they are synonymous. But I can think of other contexts where they may not be: Say animating a film. You have 1000 computers generating frames to be assembled into a final sequence. The process of dispatching each frame render is most likely not going to be done with the OSes thread call, but through a networked work queue of some sort (ala Torque or something). So you have the "task" (rendering the movie) being worked on in "parallel" via a networked job dispatch mechanism.

Or the case of using "make -j ..." which "forks" multiple jobs (compile, link, ...) in parallel based on a dependency tree.

I say threading is parallelism, even if just one form of it. The other methods of parallelism I mentioned here could also be done in FPC with the appropriate code. But that is a rather nebulous topic to discuss. Concrete answers for which are only available when you know how a specific job needs to be broken up.

I'm still looking for a good easy to use job queue myself...

-Jon

It’s my understanding that for parallelism to make sense you need to have at least more than 1 separate compute unit, be that a CPU core or a GPU.

If you had a GPU with 250 compute units or a CPU with 250 cores you would need to design your task in a way so that it could be broken down into as many discrete portions as possible so that you could take advantage of the multiple cores running in parallel. Even if you didn’t have a single thread and the execution blocked until finished you wouldn’t see any performance increases unless you designed your program to scale for parallelism. Running 250 threads on a single core isn’t going to be 250x faster but running 250 threads on 250 cores may be.

Regards,
Ryan Joseph

The problem I have with this thread (no pun intended) is that it is not comparing like with like. As demonstrated by many of the replies, Parallelism and Threads are not the same thing.

I would offer the following definitions:

- Parallelism is a (design) concept for expressing collateral actions in which the processing order of the actions is unspecified. They may take place serially or contemporaneously in real time, or a mixture of the two.

- Threads are an implementation mechanism for realising collateral actions within a single processing environment.

Neither of the above implies multiple CPUs or processing units.

Jon Foster wrote

I say threading is parallelism, even if just one form of it. The other methods of parallelism I mentioned here could also be done in FPC with the appropriate code.

Threading is the way parallelism can be achieved using a "standard" programming language and running the result as a "standard" program on a "standard" OS with "standard" computer hardware.

Of course there are lots of other options such as Multiple Data CPU instructions, starting multiple programs. and even using FPGAs (to be programmed in languages like "VHDL" (where parallelism is the "simple" syntax, while sequential execution needs do be dedicatedly coded).

The only thing that seems to be a viable option for a future "normal" PC compiler could be stuff like "parallel loops" (appropriate Pascal syntax is already done in Oxygene), as a syntax candy on a library allowing for parallelism by using a thread pool.

-Michael

I would agree wholeheartedly with most of that. Parallelism is purely a concept of multiple tasks running at the same time

Threads or processes are just implementations of that concept. Threads tend to be used for related tasks in a single process. Separate processes tend to be used for unrelated or independent tasks. Those are not hard and fast rules.

However, multiple independent compute units must be required for *true* parallelism. On a single processor any tasks running at the same time is just an illusion, normally created by the OS in time slicing between tasks based on certain criteria (priority, I/O, cpu usage etc.). That applies equally to threads or processes

Various languages assist, or purely exist, to make creating multi-tasking easier, but it ultimately all boils down to the same thing.

Regards,
Gary

On Mar 31, 2017, at 3:55 PM, Gary Doades wrote: However, multiple independent compute units must be required for *true* parallelism. On a single processor any tasks running at the same time is just an illusion, normally created by the OS in time slicing between tasks based on certain criteria (priority, I/O, cpu usage etc.). That applies equally to threads or processes

Yeah exactly. Even if those are nodes on a network you need more than one. Unless you're making a bot-net or software designed for specific hardward which is known to have multiple cores, parallelism probably means using the GPU via an API like OpenCL, which is far cry from threading some tasks to run async.

Regards,
Ryan Joseph

Gary Doades wrote

However, multiple independent compute units must be required for*true* parallelism. On a single processor any tasks running at the same time is just an illusion, normally created by the OS in time slicing between tasks based on certain criteria (priority, I/O, cpu usage etc.). That applies equally to threads or processes

I think that what are referring to here is not so much *true* parallelism but that when parallelism is designed into an application, it enables real time parallel computing when the application is deployed.

For example, this distinction is very important in matrix algorithms. When operating on two matrices to produce another, the operations on each cell can be identified as n x m parallel actions at design time. At deployment time, it is often desirable to have a scalable implementation that can use anything from 1 to n x m processors to do the job. Thus you can have a design that identifies parallelism leading to an implementation that can non-parallel, partly or wholly parallel (in real time) depending on the size of the matrices and the number of processors available.

At what point does this become *true* parallelism?

On Mar 31, 2017, at 4:38 PM, Tony Whyman wrote: For example, this distinction is very important in matrix algorithms. When operating on two matrices to produce another, the operations on each cell can be identified as n x m parallel actions at design time. At deployment time, it is often desirable to have a scalable implementation that can use anything from 1 to n x m processors to do the job. Thus you can have a design that identifies parallelism leading to an implementation that can non-parallel, partly or wholly parallel (in real time) depending on the size of the matrices and the number of processors available.

That’s a good candidate for parallelism but you need an API like OpenCL to implement it properly so you can access the actual hardware required. From the little time I spent with OpenCL you really don’t want to (or shouldn’t) be intentionally designing your programs like this unless you have a real need for “true” parallelism with multiple compute units.

Regards,
Ryan Joseph

Tony Whyman wrote

Neither of the above implies multiple CPUs or processing units.

Regarding the view of the application (disregarding execution speed) or of the application programmer, there is no difference between real ("Hardware") and virtual (e.g. threads) parallelism. These dirty basics need to be handled by the software and hardware infrastructure.

The use of real (e.g. multi CPU) parallelism that the application allows for being divided into multiple parallel "Threads". his fact given Hardware parallelism can speed up the execution, while even virtual parallelism allows for improving the latency of definable parts the application.

-Michael

I’m not understanding how parallelism could apply to anything besides breaking down a task so that it can run on multiple hardware compute units.

Why would you ever break a task into 100 threads when you could just run it one thread?

Regards,
Ryan Joseph

"Events".

One gets into the grey area of threads and "processors". As an example you could divide a program into two threads, one reading and one writing. Immediately after issuing a write request, you could start reading the next item in a separate thread before the write is complete. This works because the I/O subsystem is mostly independent so that the OS can schedule another thread (or process) which is not waiting for the I/O subsystem to reply. Using only a single thread, the whole program has to wait for the I/O write to finish before starting the next read.

In this way a single process on a single "processor" (at least CPU) can interleave tasks to speed up the overall performance of the application. This could be extended to a myriad of cases of course.

Hence the recent upsurge in "async" routines, which only works if used properly of course.

Cheers,
Gary.

Methinks that programs should be designed or programmed in such an abstract way that you do not care or know whether multiple processes are being used, IPV mechanisms, or threads... I.e. you write your program in such a way that it could take advantage of multi cpus or gpus or whatever, using anything from threads to actual proceses, and your program is so high level that you dont care ... Then if you have a performance issue because the underlying low level details didnt work, there may be some setting to tweak as a last resort. Kind of like how theoretically you could use sqldb at a high level without knowing which database implementation you are using underneath

I think Erlang, and errrrr, golang are trying to look at the problem this way too. You may also check Rob Pike youtube talk about concurrency. This is an open area of computing science still being worked on. Also, how would quantum computers deal with threading and would you even need threads, or would a single quantum cpu be so powerful that there would be little need to thread things off separately...

It would be interesting to see if TProcess could be used in fpc, or assignstream, to mimmick thread programming but instead of threads a process is used , and you still can program in a thread like way or some high level way. SimpleIPC comes close but it's more like sending a message, not calling a procedure as normal. Each pascal procedure or C function is like a mini program itself, so why couldn't each procedure be a separate EXE program if you wanted it to be...

In OOP its harder to think this way as an object is many procedures attached to a struct/record

Another idea is that each unit file could be a separate exe or thread , or each class/object could be a separate exe and you design the program so that each object does its own work in a separate exe then communicates with some other main exe

Likely in order to understand any of these creative ideas, which could be bad ideas, one will need some THC or good weed

What about Erlang.. Errrrrr
But I have not so much interest in dynamic typed languages so have done little research on it.
Been meaning to experiment with APL or learn more about it too.

Those languages that have special symbols or require a special keyboard I am sure turn lots of people off.

This email could go into fpc other i suppose

Or processes? If unix could just make processes even lighter weight or faster loading, I might avoid threads and just use processes... Or use some abstract tool that does not require I even know what is being used under the hood...

Il 30/03/2017 00:26, fredvs ha scritto:

Perfect, I have now all the arguments to defend the "Dinosaur Threading" choice.

When I'm charged of not using the most cool and new technology, my favourite argument are the tombstones. They're made out of stone, which is a stone-age technology. But, as nothing more suitable has been found for that purpose, the stone-age technology survives. Up to now it turns out to be the most appropriate technology for that application.

The best course is always to pick up the most *appropriate* technology, as opposed to the one most *modern* or *fashionable*. There are applications where parallel computing is the best solution, and applications where parallel computing is useless.

My main line is on industrial process control. You must react in real-time to "events". A multi-threading approach, with threads activated by events, gives a performance which you can't achieve with any other technique. But if you attempt to apply our event-driven approach to optimize a matrix calculation you'll miserably fail.

Just my 2c

Giuliano

Or modify the film format...

Example: create a 10 hour video that is composed of 100 files.

You have 100 computers, or 100 threads, or, 100 processes..Each computer creates 1 file.
The video is composed of 100 files but is really just a single video.

You open up VLC to play the video, and it opens 100 files, but the end user just opens a single file that links to each of these 100 files.

This could be the stupidest ridiculous parallel video creation technique ever. I'm not sure...
Or it could be an idea...

What about the difference between parallelism vs concurrency?

Rob Pike has some videos on it and meetup events discussing it on youtube.

I think it depends on what you're trying to program: some applications (like HPC/scientific, networking, rendering) need time to complete their calculations (even if it 'only' needs uploading data to hw accelerator as in rendering recently) and if you need also interactivity on them, you have no choice but do some kind of multitasking. And then there is the problem of (certain) GUI functions not being thread-safe, or, only working properly if called from the same context. Not limited to GUI too, some 3rd party libraries come with this kind of warnings too.

In principle it would probably let you do as you want. Only a lot of 'grassroots' required for a 'single procedure program' to be multitasked that way and communicate through IPC. Signal to noise (amount of user code to amount of plumbing code) ratio here would be really awful... even if the IPC was really basic and self contained in a unit, or library (maybe only better if communication happens over shared-library* or the ubiquitous netlink of sorts)

* of course, if through 'shared library' you have also to ensure all the elements use the same library ALWAYS.

The clue is in 'commumnicates' really here: what you need communicated and what communication framework you want to use for that.

Likely in order to understand any of these creative ideas, which could be bad ideas, one will need some THC or good weed

Nah, but I need to go replenish my caffeine about now ;)

-L.

Lars wrote

If unix could just make processes even lighter weight or faster loading, I might avoid threads and just use processes...

in Unix/Linux processes are not less "light" then threads. You can create a process by "fork". no "Loading" involved. it just creates the process. If you want to have the new process execute any code that is not shared with the you need to do another system call to replace the code with the new one. Moreover even if "loading" new code - in case another process already runs this file, no actual loading takes place, either, as the memory management just uses the code page already in RAM.

This definitively is as light as it gets.

-Michael

a process by "fork". no "Loading" involved. it just creates the process. If you want to have the new process execute any code that is not shared with the you need to do another system call to replace the code with the new one. Moreover even if "loading" new code - in case another process already runs this file, no actual loading takes place, either, as the memory management just uses the code page already in RAM.

This definitively is as light as it gets.

A process definitely is less "light" than threads even on Unix systems: a process has its own address space (even if it shares all pages with its parent process) and also structures keeping track of the used resources (e.g. open file descriptors). A thread does not need all this as it always shares the same address space and the same resources. Why do you think the concept of threads was introduced in Unix? Early Unix systems only had processes.

Regards,
Sven

Not really true (see below).

Why do you think the concept of threads was introduced in Unix? Early Unix systems only had processes.

Because if you do something that is usually called "Thread" you _want_ shared memory in the same address space, not because of that being "light" but because the way you want to handle your memory based objects in your user code needs this.

In fact the "real" difference between Threads and processes (i.e. the Kernel knows that a processing entity is supposed to be a "Thread" associated with other threads of a process) in Linux only had been introduced with the "Native POSIX Thread Library" ("NPTL") -> https://en.wikipedia.org/wiki/Native_POSIX_Thread_Library. Before NPTL, you just used processes for your threads telling the system that for the new thread you wanted a shared address space.

Again (AFAIK) NPTL had not been introduce to make threads "lighter" but to allow for threads behaving in a decently POSIX compatible way (e.g. the threads of a process getting only a common share of time slices).

-Michael

Michael Schnell wrote

Again (AFAIK) NPTL had not been introduce to make threads "lighter" but to allow for threads behaving in a decently POSIX compatible way (e.g. the threads of a process getting only a common share of time slices).

In any event, processes on unix are *defined* as owning resources- memory, handles and so on- while threads only manage control flow. I believe that MS also have "fibers" which are non-preemptive threads.

If somebody wants to define a new kind of thread, something even lighter-weight, they're going to need a new name: inverting well-accepted precedence is not an option.

In our previous episode, Mark Morgan Lloyd said:

In any event, processes on unix are *defined* as owning resources- memory, handles and so on- while threads only manage control flow. I believe that MS also have "fibers" which are non-preemptive threads.

They are not really threads. They must be scheduled within threads.

https://msdn.microsoft.com/en-us/library/windows/desktop/ms682661(v=vs.85).aspx

from the article they don't seem to have that many advantages, except to convert existing manual m:n scheduling apps.

Marco van de Voort wrote

In our previous episode, Mark Morgan Lloyd said:> > In any event, processes on unix are *defined* as owning resources- > memory, handles and so on- while threads only manage control flow. I > believe that MS also have "fibers" which are non-preemptive threads. They are not really threads. They must be scheduled within threads. https://msdn.microsoft.com/en-us/library/windows/desktop/ms682661(v=vs.85).aspx from the article they don't seem to have that many advantages, except toconvert existing manual m:n scheduling apps.

Which effectively makes them coroutines, but since they're MS-only they're nonportable. Which is more or less where Ryan started off the other thread.

In our previous episode, Mark Morgan Lloyd said:

What exactly makes them coroutines? Maybe that they implement a local stack concept. But you still would have to integrate that with FPC (exception handling, stack checking), so at the basic level it only saves some asm switching contexts.

But maybe the OS support for stack switching makes things like keeping SEH working easier. (not that that is enabled by default for 32-bit windows x86)

Which is more or less where Ryan started off the other thread.

It suddenly occured to me that that piece of code is compatible to nothing in FPC, since the only likely target in FPC (win32 x86) doesn't use SEH. There is a define to enable it though ( -dTEST_WIN32_SEH )

Marco van de Voort wrote

In our previous episode, Mark Morgan Lloyd said:> > They are not really threads. They must be scheduled within threads.> > https://msdn.microsoft.com/en-us/library/windows/desktop/ms682661(v=vs.85).aspx> > from the article they don't seem to have that many advantages, except toconvert existing manual m:n scheduling apps.> > Which effectively makes them coroutines, but since they're MS-only > they're nonportable. What exactly makes them coroutines? Maybe that they implement a local stackconcept.

Yes, exactly. Which I think is an abstraction that FPC doesn't have. Even Turbo Pascal interrupt handlers worked within the same stack, as distinct from TopSpeed's IOTRANSFER() which switched.

But you still would have to integrate that with FPC (exceptionhandling, stack checking), so at the basic level it only saves some asmswitching contexts.

It's interesting that coroutines (Modula-2, perhaps others) precede exceptions etc. (C++?). It might turn out that in practice they /have/ to be implemented first, otherwise the amount of doctrine that grows up around exceptions makes the job implausible.

But maybe the OS support for stack switching makes things like keeping SEHworking easier. (not that that is enabled by default for 32-bit windows x86) > Which is more or less where Ryan started off the > other thread. It suddenly occured to me that that piece of code is compatible to nothingin FPC, since the only likely target in FPC (win32 x86) doesn't use SEH.There is a define to enable it though ( -dTEST_WIN32_SEH )

I think I said at the time that he should be looking at (an equivalent of) fibers.

Affinity

If a thread is dedicated to say , polling a serial or Ethernet port which has a high input data rate , then dedicating one CPU to that task/thread is useful.

(Hard Affinity) with multi-core CPU's is to dedicate a specific core to a task (thread) and just poll the input memory mapped source. This works very well.

http://www.ibm.com/developerworks/linux/library/l-affinity/index.html#download

These Linux API functions set the affinity for processes :

sched_set_affinity() (for altering the bitmask)
sched_get_affinity() (for viewing the current bitmask)

These functions set the affinity for threads :

     pthread_setaffinity_np (pthread_t thread, size_t cpusetsize,  const
cpu_set_t *cpuset);
     pthread_getaffinity_np (pthread_t thread, size_t cpusetsize,  cpu_set_t
*cpuset);

They are very simple to translate into Free Pascal

Brian

In our previous episode, Brian said:

Well, the driver actually does the hardware, so it is already buffered for userland ?

They are very simple to translate into Free Pascal

Please provide patches in the bugtracker then. Linux calls go into the "linux" unit, pthreads into the pthreads unit, which is general *nix, but with an include per OS

The information about using affinity with Linux was only posted so anyone who was interested could use it , and was not intended as an addition to Free Pascal.

Regrading the previous comment about the hardware buffering the data , one still must do something with the data in user space. The choice there is to let the OS / CPU decide which core to use or dedicate a CPU core to the thread. Your choice.

Here is the code and the Linux library you must link to in order to use the functions.

{$LINKLIB libc.so}

...

function sched_getaffinity(pid : Ptruint; cpusetsize : longint; cpuset :
pointer) : longint; cdecl; external;
function sched_setaffinity(pid : Ptruint; cpusetsize : longint; cpuset :
pointer) : longint; cdecl; external;
function pthread_setaffinity_np(pid : Ptruint; cpusetsize : longint; cpuset
: pointer) : longint; cdecl; external;
function pthread_getaffinity_np(pid : Ptruint; cpusetsize : longint; cpuset
: pointer) : longint; cdecl; external;

Brian

Isn't the speed of serial (so slow) that it would be overkill to dedicate a Cpu to it?

Maybe usb 2 or 3, or possibly Gigabyte ethernet 1000 speed ;-)

I should have said "streaming" rather than "serial". True Parallelism is a long way off in the future , but a form of parallelism can be had using multiple CPU cores and hard affinity as described in the excellent article.

http://www.ibm.com/developerworks/linux/library/l-affinity/index.html#download

Consider the case of a special purpose custom design PCIe card and Linux. The classical way to interface with the card is to write an interrupt handler and accept the fact that doing so opens all your proprietary code due to the GPL. It also must wrestle with the black magic mess of BIOS PCI IRQ assignment and conflicts with other hardware.

Another approach is to memory map the PCIe card in user space, use hard affinity and dedicate one CPU core to poll the card memory in user space. In this case the GPL does not apply and the issue of IRQ assignment is not an issue.

Another approach is to abandon the add-in (PCIe) card approach and use gigabit Ethernet with a dedicated CPU core as the I/O. This what was done on the current project.

Brian