I asked about serial communications in my thread about embedded development and I was told to look at the serial unit, which I have done now.

I also looked at the Lazarus wiki about hardware interfaces where there is a section about serial communications, which actually points to the SynaSer system.
(http://wiki.lazarus.freepascal.org/Hardware_Access#Serial_Communication) There were also replies with brief discussions about AsyncPro clone development etc.

Now after having a deeper look at the two suggseted comm systems serial and syanaser I am not very comfortable....

It looks like the serial.pp unit implements a blocking system even though it is not clearly written as such. But the synaser unit is definitely blocking.

Since I am used to having event driven serial communication my question is really twofold:

1) Is serial communication with events not possible in FPC on Linux?

2) How can one use blocking reads when there is no way to know exactly when data will arrive? Everything else will stop when waiting for data, right?

Additionally I have another nonrelated question:

3) How can one enumerate the existing com ports on Linux? In Windows I know how to do this and thus I can get a list of valid ports for the user to select from. Is there some way also in Linux using FPC?

Best regards,

Bo Berglund

Bo Berglund wrote

1) Is serial communication with events not possible in FPC on Linux?

Of course it is possible. You just have to write your own library to do it. One possibility would be adapting lNet, like Synaser adapts Synapse.

2) How can one use blocking reads when there is no way to know exactly when data will arrive? Everything else will stop when waiting for data, right?

By using a timeout. Timeout = 0 returns immediately, so you can use this to check periodicaly if data comes.

Synapse works very well for Tcp and serial comms. Use threads to wait for the data. It also supplies an event on data rx'd which you can use to call yourself when necessary

Quickest way I have found is to enumerate the device files in /dev extracting all 'ttyxx' (comports) and 'ttyU' (usb ports). This gives a list of 'probables'. Then try to connect (using SynaSer for example) to verify actual usable ports in the list

I had hoped there would be ready-made code for such common things as using serial communications....

Where is the timeout? Here is the serial.SerRead function:

function SerRead(Handle: TSerialHandle; var Buffer; Count: LongInt):
LongInt;

AFAICS there is only a Count parameter specifying the number of bytes to read.

Bo Berglund wrote

Where is the timeout? Here is the serial.SerRead function:

Try RecvByte

OK, I have to dive into the thread handling on Linux then I assume.... I have next to no exposure to Linux so far.

On my Ubuntu 10 virtual machine I get this result from ls /dev/t*:

/dev/tty /dev/tty19 /dev/tty3 /dev/tty40 /dev/tty51 /dev/tty62 /dev/tty0 /dev/tty2 /dev/tty30 /dev/tty41 /dev/tty52 /dev/tty63 /dev/tty1 /dev/tty20 /dev/tty31 /dev/tty42 /dev/tty53 /dev/tty7 /dev/tty10 /dev/tty21 /dev/tty32 /dev/tty43 /dev/tty54 /dev/tty8 /dev/tty11 /dev/tty22 /dev/tty33 /dev/tty44 /dev/tty55 /dev/tty9 /dev/tty12 /dev/tty23 /dev/tty34 /dev/tty45 /dev/tty56 /dev/ttyS0 /dev/tty13 /dev/tty24 /dev/tty35 /dev/tty46 /dev/tty57 /dev/ttyS1 /dev/tty14 /dev/tty25 /dev/tty36 /dev/tty47 /dev/tty58 /dev/ttyS2 /dev/tty15 /dev/tty26 /dev/tty37 /dev/tty48 /dev/tty59 /dev/ttyS3 /dev/tty16 /dev/tty27 /dev/tty38 /dev/tty49 /dev/tty6 /dev/tty17 /dev/tty28 /dev/tty39 /dev/tty5 /dev/tty60 /dev/tty18 /dev/tty29 /dev/tty4 /dev/tty50 /dev/tty61

I am absolutely certain there there is a maximum of 2 com ports avilable to my virtual machine so where did all of the others come from?

Looks a bit hard to go around opening all these ports to check if they exist....

I cannot see that in the serial.pp file that is part of the Lazarus install I did in Ubuntu.
I just used the Synapse Package Manager and searched for Lazarus and the let it install.
Did I get a bad version?

Bo Berglund wrote

I cannot see that in the serial.pp file that is part of the

It's in Synaser. Should be in the example in the wiki.

About threading one can use TThread which is cross-platform

I think you can narrow that down to only the following:

Serial (COM) ports: /dev/ttyS*
USB ports: /dev/ttyU*

Bo Berglund wrote

I had hoped there would be ready-made code for such common things as using serial communications....

Take a look in the MSEgui code. There is a serial components already created. Martin (author of MSEide & MSEgui) will can give you more details on it. Either way, it should be less effort to port that to plain FPC neutral code (that's if there are any MSEgui dependencies - I don't know), than starting from scratch.

More details on MSEide&MSEgui is here:
http://wiki.freepascal.org/MSEide_&_MSEgui

The source code repositories are on SourceForge (primary site) and Berlios(mirror site).

absolutely correct - forgot to add the 'S' in my note :) sorry about that

Fine, so now I have to find code to do a listing of files... I guess that FindFirst and FindNext from Windows API does not work...
How is it done in FPC/Lazarus to list files in a directory? Especially on Linux.

2010/9/28 Bo Berglund :

Hi,

I use threaded and event-driven serial port communication all day with FPC and I did not have to write anything.
Have a look at the great CPortLaz library

http://sourceforge.net/projects/cportlaz/

It also has a few lazarus components, but I easily removed them from the source code to make it a FPC-only library.

Cheers,

Roberto

Of course FindFirst/FindNext/FindClose works. :)

http://www.freepascal.org/docs-html/rtl/sysutils/findfirst.html

A tip from me: If you have a Delphi function/procedure from which you don't know whether it's also implemented in FPC or in Lazarus you might try a search from this function on the FPC site.

Example for Google:

findfirst site:freepascal.org

This lists the documentation page as one of the first entries. :)

(You might also try to press F1 on a function/method identifier in Lazarus to see whether it finds a help entry)

Regards,
Sven

Thanks for the link!
But I was a bit disappointed by the text on the download page:

"This project is the conversion of the Comport Package of dejancrn and dybdahl, for lazarus pascal project. It just work on windows but in the future, I traduce it to
work under lazarus for linux."

Since my big concern is to run the serial comm under Linux, it looks like this solution is also not going to work for me....

I would also recommend you install DocView and the INF help files. This is a very small download. This gives you offline, indexed and searchable help in Lazarus IDE. Setup for context sensitive help via F1 or Ctrl+F1 is very easy too - just a few seconds to setup, no need to recompile Lazarus IDE.

For more details see:

http:/msg21437.html

FindFirst/Findnext in the sysutils unit works on all platforms.

Michael.

Felipe Monteiro de Carvalho wrote

By using a timeout. Timeout = 0 returns immediately, so you can use this to check periodicaly if data comes.

Polling never is nice: unnecessarily consuming CPU cycles and introducing additional latency.

long timeout -> large latency, short timeout - large CPU overhead.

So IMHO using bocking communication in threads (such as AsyncPro provides) is the way to go.

-Michael

Bo Berglund wrote

Where is the timeout? ..

The "Linux Way" to deal with blocking "sockets" is not using Threads and thread communication e.g. by Messages (as one would do in Windows and thus in Delphi and thus in Lazarus), but using the appropriate "multiplex" driver interface by functions like "select()" or (mor "modern" "epoll()" (misleading name: it does not do polling at all).

-Michael

Felipe Monteiro de Carvalho wrote

About threading one can use TThread which is cross-platform

Doing the communication completely in a TThread is doable just with same. But in the end you need the data in the main thread. Here you need thread synchronization which is not (perfectly) cross-platform.

AsyncPro fires all it's events in the main thread, so the user programmer does not need to bother with threads and thread communication. The threads are just internal in AsyncPro. Thus (provided same does works correctly) the user pert is inherently thread safe, The perfect "Object-Pascal" way to handle serial (and TCP/IP) communication.

-Michael

Bo Berglund wrote

OK, I have to dive into the thread handling on Linux then I assume.... I have next to no exposure to Linux so far.

IMHO if you are at it, doing a portable (or appropriately ifdef'ed) and (partly) AsyncPro work-alike unit , keeping the thread handling completely inside would be the best way to go for you as a knowledgeable AsyncPro user. I'll be happy to be involved in that project.

-Michael

Bo Berglund wrote

It looks like the serial.pp unit implements a blocking system even though it is not clearly written as such. But the synaser unit is definitely blocking.

But the serial does allow you access to the handle, so it's trivial to use select() (actually fpSelect()) to implement the timeout.

1) Is serial communication with events not possible in FPC on Linux?

I'd comment at this point that some Windows programmers are excessively oriented towards events and messages (in my opinion) since that's about all that was available pre-NT.

2) How can one use blocking reads when there is no way to know exactly when data will arrive? Everything else will stop when waiting for data, right?

I agree with what other people have said about threads etc. but on occasion they're needless complexity. For example if all you're doing is handling a simple state machine polling (which looks at the OS buffer rather than the hardware itself) is frequently adequate.

3) How can one enumerate the existing com ports on Linux? In Windows I know how to do this and thus I can get a list of valid ports for the user to select from. Is there some way also in Linux using FPC?

As other people have said, this is messy. I find myself looking for ttyS, ttyU and ttyI, making an arbitrary decision as to how many of each could reasonably be installed, and then tentatively opening each to see whether I've got access. It gets worse if you want to lock them (i.e. one program at a time gets access).

Mark Morgan Lloyd wrote

But the serial does allow you access to the handle, so it's trivial to use select() (actually fpSelect()) to implement the timeout.

Is spSelect() portable / Cross-Platform ?

If not I would not recommend using it other than encapsulated in a portable unit that uses ifdefs to handle cross-platform issues.

I'd comment at this point that some Windows programmers are excessively oriented towards events and messages (in my opinion) since that's about all that was available pre-NT.

IMHO: Object Pascal Events are not at all the same as Windows messages (even though they often are implemented using Windows Messages in Delphi)

The beauty of Object Pascal is that it's based on an event driven paradigm and thus in most cases prevents the user programmer from complex stuff like thread communication, mutex, and select(). So IMHO decent libraries should internally use whatever they want (threads, Windows messages, select(), Mutex, Futex, signals, pipes, ...) but should be compilable for all available platforms and provide a user interface based on functions and Object Pascal Events that (if not especially stated in the docs) are fired in the context of the main thread. (I do like thread Events, but this is another issue and should be dedicated to advanced programming.)

-Michael

It certainly is in the context of socket programming, I've not checked- yet- the situation with serial ports and general files.

However if I recall correctly low-level Windows read and write syscalls have their own timeout parameters, so the correct thing would be to wrap (for example) serial.pp and TComPort in a mid-level object and then implement things like event notification at that level.

interface based on functions and Object Pascal Events that (if not especially stated in the docs) are fired in the context of the main thread.

I agree. But on the other hand I think it's a mistake to write low-level support with too many embedded assumptions, using serial comms for illustration you might want events to go to the main thread if writing a terminal emulator but not if writing a multithreaded store-and-forward server.

Since I am used to having event driven serial communication my question is really twofold: 1) Is serial communication with events not possible in FPC on Linux?

We have here:

http://wiki.lazarus.freepascal.org/5dpo

a component that does that (I think!)

Is there anything that makes it unsuitable for your problem?

Paulo Costa

Bo Berglund wrote:
...

The "Simple Serial" zips use SynaSer and there is a SynaSer routine used by my programs to list the serial ports.

http://www.turbocontrol.com/simpleserial.htm

Mark Morgan Lloyd wrote

... but not if writing a multithreaded store-and-forward server.

You are absolutely right. Doing too much ini the main thread might slow down server-type applications a lot due to the necessary serialization of the events. But doing such application is "advanced programming" and who wants to dive into this will need to bother with the well known dangerous dirty details of thread programming. There just should be a package that allows for using serial communication also to "basic" (non multi-threaded) programming. If the same package also allows for "advanced" usage by additionally supplying "thread-events": the better !

-Michael

This does look nice !

-Michael

Michael Schnell wrote

> http://wiki.lazarus.freepascal.org/5dpo This does look nice !

Although it's unclear how much complexity it drags into a program (or, for that matter, into the programmer's development environment). As you've said elsewhere, what we need is a simple, general-purpose library which can then be built on for either single- or multi-threaded programs, using either polling or event handling.

Mark Morgan Lloyd wrote

what we need is a simple, general-purpose library which can then be built on for either single- or multi-threaded programs, using either polling or event handling.

Maybe 5dpo can be enhanced to meet this requirement. Maybe this is easier than porting parts of AsyncPro.

I did only take a very small might be done to provide an internally multi-threaded unit that features a main-thread-event interface. This is by far the most challenging issue, as it needs an implementation of a fast thread-safe FIFO for the serial (receive) data and a portable implementation for firing asynchronous queued main thread events from threads.

Calling synchronous events right within the (receive) thread for "advanced" server-type applications is a nice to have and easy to do add-on.

-Michael

En/na Michael Schnell ha escrit:

Doing the communication completely in a TThread is doable just with same. But in the end you need the data in the main thread. Here you need thread synchronization which is not (perfectly) cross-platform.

It is if you use SyncObjs

Bye

Luca Olivetti wrote

It is if you use SyncObjs

(How) Can SyncObjs be used to from a thread fire asynchronous events in the main thread that are scheduled "between" the GUI events that are fired asynchronously anyway ?

-Michael

En/na Michael Schnell ha escrit:

Oh, I thought you were talking about protecting access to variables/structures, for that you can use cross-platform the classes defined in syncobjs.
To fire events in the main thread there's synchronize, but you're right, that's not asynchronous.

Bye

I (the OP) simply want to have a serial comm function that can enable me to communicate with the hardware data acquisition system we use. For this I need to send commands (binary packets) to the equipment and then wait for results.
Some interchanges are command driven, such that my program sends a command and then waits for the response. But the size of the response cannot be determined in advance, it depends on the data. However for these transfers the first 6 bytes of the response contains information on the total length of the packet to expect.
A transfer can take a rather long time; if we are dealing with the max size (1 Mbytes) and the speed 19200 it will be about 9 minutes, which requires a non-blocking approach. Otherwise the application will die in the eyes of the user.

Some interchanges are initiated from the equipment itself once an initial command has been sent. These messages can come at any time; within a few seconds or after several hours.

When I implemented the interface on Windows I could use events from the serial component. I have tried different componets like TSerial (by P Crowther) and AsyncPro and they all have an event signalling the arrival of new data. So in the event procedure I can receive and briefly analyze the data to see if I have a complete packet or not. If I have then I can set a flag to this effect and the main code can look at this and act when it gets set. Works fine and does not lock up the application.

So now I am after a serial comm implementation for Lazarus/Fpc to work in a Linux embedded environment which gives me these possibilities.

So far most of what has been shown builds on Synaser or similar, which are blocking systems....

Bo Berglund

Bo Berglund wrote

So now I am after a serial comm implementation for Lazarus/Fpc to work in a Linux embedded environment which gives me these possibilities.

Yes, and as I've said before I'm using the standard serial unit plus fpSelect() without problems. Horses for courses, as we say over here.

Irrespective of what solution you choose one thing that you might need to consider is how accurately you can time things. In particular, the type of protocol that I work with needs timeouts applied from the point where the computer has actually sent the final character of a packet, so whatever code is handling your state machine (or whatever) needs to be able to see the comms subsystem at that level- not just at an abstracted event level.

function SerReadTimeout(Handle: TSerialHandle; var Buffer; mSec:
LongInt): LongInt;
(* This is similar to Serial.SerRead() but adds a mSec timeout. Note
that this  *)
(* variant returns as soon as a single byte is available, or as dictated
by the *)
(* timeout.
         *)
VAR     readSet: TFDSet;
         selectTimeout: TTimeVal;
begin
   fpFD_ZERO(readSet);
   fpFD_SET(Handle, readSet);
   selectTimeout.tv_sec:= mSec DIV 1000;
   selectTimeout.tv_usec:= mSec * 1000;
   RESULT:= 0;
   IF fpSelect(Handle + 1, @readSet, NIL, NIL, @selectTimeout) > 0 THEN
     RESULT:= fpRead(Handle, Buffer, 1)
end { SerReadTimeout } ;
function SerReadTimeout(Handle: TSerialHandle; var Buffer: ARRAY OF
BYTE; count, mSec: LongInt): LongInt;
(* This is similar to Serial.SerRead() but adds a mSec timeout. Note
that this  *)
(* variant attempts to accumulate as many bytes as are available, but
does not  *)
(* exceed the timeout.
         *)

On top of those you put whatever state machines etc. that you need.

On 10/01/2010 11:10 AM, Bo Berglund wrote:

So far most of what has been shown builds on Synaser or similar, which are blocking systems....

To do a program that does serial communication and at the same time can do anything else (e.g. handle a GUI, handle other serial ports or TCP/IP ports), you either need to do the serial communication completely in a thread (and thus deal with the dirty implications of multi-threaded programming in the main program) or create/use a toolbox (such as AsyncPro) that internally does the thread based stuff and provides mainthread events to the user.

-Michael

En/na Bo Berglund ha escrit:

So far most of what has been shown builds on Synaser or similar, which are blocking systems....

Yes, so you put it in a TThread and, when the data is ready, use synchronize to signal the main thread.

Bye

That is how it is done on the 5dpo component.
I has been working very nicely for us.

Paulo Costa

When you say "component" do you mean a component in the Delphi way where you drop a component on to a form etc?

I don't like that way because I want to have better control and I also want to use the functions in form-less application (console) so I usually try to use classes that implement stuff by instantiating objects when needed for this purpose.
Is that possible with 5dpo?

Bo Berglund

Most if not all 'components' can be used that way - Create the instance of its object (make sure you Free it upon shutdown), assign appropriate procedures to On[] events and set properties by hand/in code.

(GLScene can be made running that way, i don't see why others would not)

Bo Berglund

L.

Good to know, I will try to use it.
I downloaded the sources and extracted the files and read the INSTALL file. Now I have a problem because this is what it says:

"move the Sdpo/ directory to the components/ directory of your Lazarus installation"

The problem is that I have no clue as to *where* Lazarus is installed. I used the Synaptic Package Manager and let it find and install Lazarus for me (on Ubuntu 10.04-2). I was not asked where it should go, it just installed....
And being a Linux newbie (I installed this Ubuntu machine last Sunday) I don't know where programs go in the confusing file system. Probably a silly and stupid question, but how can I find the install directory of any program in Linux?

Reopen the Synaptic package manager and highlight Lazarus. Then click on "Properties" and you will find a list of the files that were installed and their location.

If only you could do that with a Windows installation file ;-)

Malcolm

I found lazarus in this location:
/usr/lib/lazarus/0.9.28.2
I have downloaded the component zip file into my Downloads directory. Then I have extracted the contents, which created a new directory: Downloads/Sdpo

But when I try to do as directed (move Sdpo folder into lazarus components folder) I get a "permission denied" error. :-(

Why is that? And how do you go about installing the components if you are not allowed to put the files where they belong???

just like on windows (finally!), you have to be root to do things in "protected" areas... this means that you have to log in as root or use sudo to gain root privileges... once you have root privileges, then you can copy those files and directories over there as you need to do ;)

No! Wrong approach!

Don't write into locations where you shouldn't write to!

You should do it the following way:

Create a "components" or "3rdparty" directory somewhere where you have write access to (at the side of directory containing your projects perhaps). Then copy the "Sdpo" directory to this new directory and proceed as mentioned in the INSTALL file, but using that directory instead of the "/usr/lib/lazarus/0.9.28.2/components" one.

This is a much cleaner way of doing this in my opinion.

Regards,
Sven

very very true! i was, however, answering his question for the problem that the documentation presented...

agreed... except, how are multiple users to be able to use the component like those built into Laz? it appears to me that the instructions were thinking in this manner and so the component is added to those that come with Laz so that it is available to all users...

In our company we have the 3rd party components (and our own ones) included in the svn repo as well. And everyone installs them in the IDE when it's first configured.

I'm following this approach at home as well: I have three computers that I'm using (one small Windows PC when traveling, one i386-linux laptop as main computer and a x86_64-linux desktop as "server") and they are all getting the custom Lazarus packages (e.g. lnet) from my Git repository.

Regards,
Sven

that's an interesting idea... however, we need to also look at the *nix skill level of the OP... we should also be considering that not everyone has the ways and means to do this... it is good to point it out, though! ;)

i've never used a repository for any of my coding in the 30+ years that i've been coding... i have been considering installing one and seeing how it goes... one of my concerns is the space it consumes... whether that is database space or drive space or both... speed and efficiency are also of concern... especially considering the equipment i have at hand that might be pressed into this type of service ;)

)\/(ark

Instead of installing a repository yourself, one solution is to pay for a company which provides a repository for you.

I use assembla and it works very well and is quite cheap: https://www.assembla.com/

that is an option, for sure... but not for me ;)

I use assembla and it works very well and is quite cheap: https://www.assembla.com/

i'll add it to the notes and keep it in mind :)

thanks for the response!

I'm back.... Was away over the week-end and did not have time to force the directory into the wrong place. Lucky! :-)
I will put it in my own home dir then and not worry because I am for sure the ONLY user of this computer (it is a virtual machine on my Windows box).

If by "using a repository" you mean "using a source versioning system", then I strongly advice to set up a system so you can safeguard your code into the future. I have used CVS for 10 years and I am still today wondering how in the world we managed to get structure into our work before I brought the CVS server into the company. Simply a must have!
Of course these days it is not CVS one would install if starting afresh, probably SVN or GIT, but no matter which system you choose running with one is infinitely better than running without one.

Thanks for the suggestion! I have now installed the serial component of the suite on my Ubuntu Lazarus.
As indicated there is an event for receiving the incoming data even though that event does not tell how many characters are actually available. So how do I know how many I can read in from the port?

But there is one strange thing about the way the component works: I can call the Open method for ttyS0 and there is no error even though the ttyS0 does not exist...

I run my Ubuntu in a VMWare virtual machine and I had forgotten to add a serial port to its hardware so there is definitely no such port and yet there is no indication from the component that it has failed to open the port.
How come?

Should there not be an exception raised if the wanted port does not exist?

You don't need to know. When you are in the OnRxData event if you call ReadData():

procedure TForm1.SdpoSerial1RxData(Sender: TObject);
var mess: string;
begin
   mess := SdpoSerial1.ReadData();
   ...
end;

It will return a string with all available bytes in the input buffer.

Yes. There is an exception that is raised when the open() call fails.

Paulo Costa

Thanks, that explains it enough that I can move on! .-)

> Should there not be an exception raised if the wanted port does not > exist? Yes. There is an exception that is raised when the open() call fails.

It looks like my virtual Ubuntu 10 thinks it has a serial port even though none has been created by VMWare Workstation. Here is no exception when I try to open it....

Bo Berglund

Bo Berglund wrote

It looks like my virtual Ubuntu 10 thinks it has a serial port even though none has been created by VMWare Workstation. Here is no exception when I try to open it....

Try writing a nul (or a syn, or whatever is comparatively harmless in your application area) and see if that raises an exception.

As I've already said, you need to get the underlying device access sorted out- including control line handling etc.- before you can start thinking about higher-level events etc.

As the "drop on Form" mechanism is just a shortcut for the instantiation of a class you always can do this manually, as well.

But regarding a "form-less application": if you don't use either of Windows, KDE, Gnome, Mac or fpgui widget set, some thread-based components will not work, as they depend on messages to trigger main thread events.

-Michael