I have a device comm simulation program created in Delphi 7 (or maybe 2007), which uses TurboPower Async Pro components for the serial and TCPIP communications. It is used for production testing and also for development purposes. On Windows of course....

Now I would like to port this to the Raspberry Pi environment so the simulator could be placed in a small package and replicated easily. Since it is Delphi I thought about FPC and Lazarus as the obvious destination.
I have ported some other simple GUI programs using the tools built into Lazarus with no real big problems, but those used only standard Delphi components.

Now I am not sure if it is even possible to port the AsyncPro serial components to fpc...

I use the TApdWinsockPort which implements an *event driven* serial comm port, which can switch between serial and sockets communications. The incoming data are dealt with in the OnTriggerAvail event. So these are the only specific items used by me, but obviously in the background there are a whole lot of things going on... I tried to trace the tree of used units starting from the TApdWinsockPort and it just grows and grows. Seems like a project I could not do on my own.

Has anyone here tried/succeeded?
If so is there some web resource that describes how it is done?

Or is there some other way to accomplish the task?

You would be my (and many others') hero if you port Async Pro to Linux and provide the result publicly. This has been requests already multiple times.

IMHO this is one of the greatest shortcomings with Lazarus/fpc (even though of course there are alternatives).

Let me know if I can be helpful on that behalf.

-Michael

I suspected as much...

AsyncPro in itself is MASSIVE with loads of component classes and functions geared towards all kinds of modem communication. And it is all very convoluted.
A few years back before I retired I wanted to see what the dependencies looked like starting at the TApdComPort, so I could minimize the number of APro units I needed to put into version control.
But after some time I gave up. At that time I had traversed between 10 and 20 units, which just added more dependencies all the time. So I put the complete AsyncPro into our version control and stopped thinking about it.

Now it jumps back when I try to port to Linux on ARM...

Unfortunately I am not capable of doing such a full port.

I know I have asked here before about serial/tcpip communications using event driven sockets and serial port components, but the solutions seem to be far between....

In fact for the simulator I want to port I could probably rip out the APro component and replace it with a TCPIP socket one instead since the RS232 is not going to be used anymore.
But agin the code is geared towards event driven comm and most of what I see in FPC for Linux TCPIP is blocking and that would mean more surgery in the existing code.

I started testing LNet a few months ago but did not get so far then and had to drop it for other tasks.
(See also thread about packages in fpc where LNet is not automatically installed, yet it is an fpc stock package.)

An LNet implementation which is tweaked to use events for the receive would be a possible starting point..

Bo Berglund wrote

I started testing LNet a few months ago but did not get so far then and had to drop it for other tasks.

I can also recommend Indy 10 (source code direct from their SVN repository). Indy 10 works very well these days with FPC 2.6.4 and FPC 3.0 - personally tested under Linux, Windows and FreeBSD. I'm told from Remy that OSX is also supported (but I haven't personally tested that yet).

Just yesterday I submitted a patch that fixes the indylaz.lpk package - just some minor unit paths were wrong, and then it installed into Lazarus IDE without error. That should hopefully be committed soon. But I normally use Indy without the use of Lazarus Packages.

Work is also being done to bring the Indy Unit Test suite up to date, so that should make it much easier to verify functionality with different compilers and platforms.

Regards,
- Graeme -

Implementing Event triggered behavior (such as AsyncPro) no great problem. I did several test on that (in a *portable OS independent* way).

IMHO the best way to go is:

- Do the receiving in a Worker Thread based on a blocking (serial or socket) receive function (i.e. the thread always hangs in the receive and only does some work when (at least) a character has come in) .This is exactly how AsyncPro works.

- The Thread collects the appropriate amount of data (based on dynamical settings provided to the class. This is exactly how AsyncPro works.

- Once the appropriate data is collected you *create* a "transfer object" (an instance of a transfer class that allocates and includes buffer for the data (not just a pointer if the buffer could be reused by the thread ! - You also could use newly allocated managed buffers for data collection, avoiding doing a copy -). This object provides a "APdataavailable" procedure that is to be called to extract the data and (of course) a destructor that frees the buffer.

- You do TThread.Queue providing the transfer object's APdataavailable procedure. After that you don't need to store the pointer to that transfer object, as it will free itself (including the data buffer) later in the main thread.

- The APdataavailable procedure will be executed as an event in the main thread. It will call the appropriate event handler defined as an AsynPro property.

- The last statement of the APdataavailable procedure (after calling the user event handler) will be just "free".

have fun,
-Michael

Michael Schnell wrote

- Once the appropriate data is collected you *create* a "transfer object" (an instance of a transfer class that allocates and includes buffer for the data (not just a pointer if the buffer could be reused by the thread !

That of course is a significant issue. I've written "bare metal" stuff which moved data around like that with lots of consistency checking which meant it was robust. OTOH "real" OSes tend to move pointers around with minimal hardware protection against overrun etc. when a pointer somehow gets moved outside its (ring) buffer.

I wonder whether using something reference counted would be a useful compromise here.

Bo Berglund wrote

I could probably rip out the APro component and replace it with a TCPIP socket one instead since the RS232 is not going to be used anymore.

The beauty of AsyncPro is that it provides the same user interface for serial and socket component. Hence a serial port could be local or done with a "ComServer" device without a problem

-Michael.

Mark Morgan Lloyd wrote

That of course is a significant issue.

I don't see what could prevent this from working.

The simple method would be to create a new buffer for the data to be transferred to the main thread and copy the data already received in a "static buffer" there.

If you want to avoid copying you could transfer the data in the receive buffer (which later would be freed) and allocate a new receive buffer for the next receive.

I don't understand the "OS" issue, that you speak of .

-Michael

I didn't say anything wouldn't work.

Copying data tends to introduce a significant performance hit, which is why many OSes try to avoid it by passing pointers. The typical result of passing pointers is to introduce the possibility of buffer overflows, because the lower-level (and more privileged) code doesn't (or can't) apply application-specific tests.

Mark Morgan Lloyd wrote

I didn't say anything wouldn't work.

Sorry. I took "issue" for a potential problem

Right you are. That i why the second method - if carefully done - would create less overhead.

Nonetheless with any transfer to the main thread, a new buffer needs to be allocated.

-Michael

Michael Schnell wrote

The beauty of AsyncPro is that it provides the same user interface for serial and socket component. Hence a serial port could be local or done with a "ComServer" device without a problem

That is why the AsyncPro component wound up in my software. This was in a different piece of code for industrial automation where the machine tool was available with both RS232 and TCP/IP comm. But it used the exact same protocol! So the same software was able to switch between the two ways just by setting a single property on the component.

Then I used that later on in my own projects too....

I tried to find the url for svn Indy10 but the closest I got was this: https://svn.atozed.com:444/svn/Indy10/

It requires a user name: Indy-Public-RO but no password.

When drilling down into the tags dir I only see tags for Embarcadero RAD Studio XE versions. And trunk is obviously a moving target.

What svn URL do you recommend?
And which tag?

Bo Berglund wrote

What svn URL do you recommend? And which tag?

I use the URL: https://svn.atozed.com:444/svn/Indy10/trunk

ie: I use Trunk (v10.6.2) directly. The repository is not that fast moving, and Remy seems to be on the ball quite quickly if there are any noted issues.

Attached you will find a drop-in replacement "indylaz.lpk" package file. This is the fixed package and should hopefully appear in the repository soon.

I've also attached a unit (small test suite for IPv4 TCP Client & Server) which shows the setup of a TCP Server and TCP Client instance and how the client connects to the server.

Regards,
- Graeme -

Thanks for the information!
Question:
My earlier experience is that the Indy download is VERY big and when I analyzed what needed to be actually used I found that the only files needed (for Delphi) were those in subdirectories Core, Protocols and System. Everything else is not needed for the actual use in Delphi. I assume the same goes for Lazarus?
If so do you know how to formulate the svn command to check out these subfolders only (to save some download time and disk space)? I am a CVS person myself so I have only very limited svn exposure.

Attached you will find a drop-in replacement "indylaz.lpk" package file. This is the fixed package and should hopefully appear in the repository soon.

I will use that.

I've also attached a unit (small test suite for IPv4 TCP Client & Server) which shows the setup of a TCP Server and TCP Client instance and how the client connects to the server.

Good, I will have a look.

Bo Berglund wrote

My earlier experience is that the Indy download is VERY big and when I analyzed what needed to be actually used I found that the only files needed (for Delphi) were those in subdirectories Core, Protocols and

I don't know the exact dependencies per Indy component. There are loads of components. ;-) But I would guess your assumption for Delphi applies to Lazarus too.

Indy isn't too big though. I took a look at my copy. The source code zipped is 7.5MB. Uncompressed that equates to 16MB.

I used Git to clone the whole SVN Trunk history, and that equated to 44MB. Lazarus and FPC repositories are *much* larger. :)

Any further repository updates will be very small (normally only patch deltas are copied) - compared to a whole new download. So I highly recommend you do a 'svn co' or 'git svn clone' and it will save you lots of bandwidth for future updates.

Regards,
- Graeme -

I made a test checkout as follows:

pi at rpi2-jessie2:~/dev $ svn co
https://svn.atozed.com:444/svn/Indy10/trunk/ indy10

Then I had a look inside the indy10 dir and found a ton of stuff... But what I referred to is located in subdir Lib.

Sizewise this is what I got:
pi at rpi2-jessie2:~/dev $ du -sh ./indy10/
77M ./indy10/

So the total download is 77 MB
But the dirs I earlier found were needed are like this:

pi at rpi2-jessie2:~/dev $ du -sh ./indy10/Lib/
25M ./indy10/Lib/

pi at rpi2-jessie2:~/dev $ du -sh ./indy10/Lib/Core/ 8.8M ./indy10/Lib/Core/

pi at rpi2-jessie2:~/dev $ du -sh ./indy10/Lib/Protocols/ 11M ./indy10/Lib/Protocols/

pi at rpi2-jessie2:~/dev $ du -sh ./indy10/Lib/System/ 2.3M ./indy10/Lib/System/

So a total of 22 MB active files. And Lib itself is just 25 MB

So I will make an attempt to copy only the Lib dir to Lazarus components dir (renaming to indy10) and run the install from there. But now is dinnertime so it will be later...

Yes, and that is only one revision. SVN is terrible!

As far as I remember from SVN, it keeps a full (uncompressed) copy of everything you checked out side the hidden ".svn" directory. That's how SVN can do a diff comparison.

Git doesn't work like stores history as compressed files, and only deltas of each commit - never full copies of managed files. The complete history of Indy (in Git) is the size of a single revision checkout in SVN.

So I will make an attempt to copy only the Lib dir to Lazarus components dir (renaming to indy10) and run the install from there.

You don't need to do that. Simply open the indylaz.lpk package and click Compile. After that, Lazarus IDE will automatically be able to find the Indy source code, no matter the location on your hard drive.

Regards,
- Graeme -

Well,
I did as I said I would before I saw your reply.... So I copied the entire Lib directory over and then renamed to indy10.

And I copied in indylaz.lpk file from your previous message on top of the one from svn.

Opened it in Lazarus Packages/Open... and hit Use/Install.

Unfortunately it did not build correctly, here is the final error message:

IdStream.pas(29,3) Fatal: Cannot find IdStreamVCL used by IdStream, incompatible
ppu=/home/pi/dev/lazarus/1.6RC2/components/indy10/lib/arm-linux/IdStreamVCL.ppu, package indylaz

Do you have any idea why this happened?

The offending line IdStream.pas(29,3) looks like this: uses
{$IFDEF DOTNET}
IdStreamNET
{$ELSE}
IdStreamVCL <== Error here!
{$ENDIF};

This time I went directly ahead to Install and skipped the Compile button. Could that be the problem?

---- Later:

Well, I did two things:
1) I removed a Lazbuild compile option I had added earlier today to make Lazarus strip the symbols off of itself when rebuilding. It was something like
-g- -Xs
It reduces Lazarus size from about 125 Mb to 25 Mb or thereabouts.

2) I hit the Compile button again and it compiled sucessfully.

3) Hit Use/Install and after a while Lazarus closed and reappeared with the Indy tabs on the palette.

But why are there duplicate tabs like "Indy Clients Protocols (am)" and "Indy Clients Protocols (nz)"??
American and New Zealand versions or what?

Lazarus Packages (the files it lists) are dependent to a set directory structure. Because you copied the Lib directory to a new location and copied the indylaz.lpk in there, now the paths don't match up - so that explains why it does work.

Simply delete your components/indy/ directory. Open the indylaz.lpk (my copy) from inside the Indy SVN checkout directory. Click Compile, then Install. It should work just fine then.

But why are there duplicate tabs like "Indy Clients Protocols (am)" and "Indy Clients Protocols (nz)"?? American and New Zealand versions or what?

hehehe... As I said, I don't normally install packages into Lazarus, so I never noticed that. I just tested, and see those in my copy too - no idea what the 'am' and 'nz' is meant to be. I'll take a look at the source code later to see if that gives any hints. I don't think it is country codes though. ;-)

Regards,
- Graeme -

No, I don't think so.
Paths are relative to the lpk file and there are no path going up (../) there.
I actually looked in your lpk file and found that all listed units in the Files node are in either Core, Protocols or System.

But I also found an interesting setting:
<OtherUnitFiles
Value=".;Core;Design;FCL;Protocols;Security;SuperCore;System"/> This shows that in addition to the dirs I mentioned earlier also FCL, Security and SuperCore are included in some path setting.

...

Well, nz looked quite like one to me. OTOH the tabs contain different components so the duplication is really only the names.

I will try to use the TCP client and server components to implement my simulator. It is more familiar to me than any of the FPC networking objects since I have used Indy quite a lot over the years. In Delphi of course...

Bo Berglund wrote

Well, nz looked quite like one to me. OTOH the tabs contain different components so the duplication is really only the names.

First tab is for components with names starting with letters A to N, and second tab for components starting with letters M to Z ;)

Best regards,
Zilvinas

I have an edition of Control Terminal using Synapse for Lazarus here: http://ctrlterm.com/ports.htm

I'm retired (and visiting Sarasota, Florida for several weeks), but from what I've read in this thread I think I'd like to add an Indy edition (and also an LNet edition). Help and suggestions welcome!

Regards,
Paul
www.ControlPascal.com

I would like to check out an earlier version of Indy10 but I don't understand how to find the correct tag name to use in the checkout...

If I browse to the server using https://svn.atozed.com:444/svn/Indy10 then I can enter the tags dir but the names there are seemingly incompatible with the svn co command since they seem to contain loads of spaces, for example:
Indy 10.5.8 - XE2 RTM

How could I retrieve the 10.5.8 version (last before 10.6.x)? On my Delphi I use the 10.5.7 version so I thought (in hindsight) taht it would be better to use that also in Lazarus...

Never mind, I realized I could put %20 as an override for all spaces. Then it worked.

Bo Berglund wrote

On my Delphi I use the 10.5.7 version so I thought (in hindsight) taht it would be better to use that also in Lazarus...

Glad you got it sorted out in the end. I would still recommend using the latest code in SVN Trunk though. Bug fixes are always a good thing. :)

Regards,
- Graeme -

My public PGP key: http://tinyurl.com/graeme-pgp