Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2015-10

Lazarus implementation of TListView etc?

63 messages · 11 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Bo Berglund

Note: Posted this by accident on the FPC list....
-------------------------------------------------- I want to know why comctrls is not found by Lazarus and hence the definitions of a lot of objects in my Delphi code ported to FPC generate errors?
I have googled extensively on "lazarus TListView" "lazarus comctrls" etc and I wind up on lazarus wiki pages explaining the properties of the objects. But as soon as I add the unit defining these the compile stops way up in the top of my files with the not found fatal error.

Again there must be something I have not gotten right...

My program is a pretty simple command line program but it uses some object files from Delphi where the objects are able to manipulate TListItems if that is called. This will not be used here but it is a PITA to take these out of the Delphi code..
At a later step I might well be doing a GUI program, using the same object units.

The easier solution seems to be to keep the code in place but then I need the definition units to come on-line...

Bo Berglund Developer in Sweden

Reply by email

Mattias Gaertner
to Bo Berglund
Show quoted text 5 lines
Note: Posted this by accident on the FPC list.... -------------------------------------------------- I want to know why comctrls is not found by Lazarus and hence the definitions of a lot of objects in my Delphi code ported to FPC generate errors?

comctrls is in package LCL.
Check "Project / Project Inspector / Required Packages" has package "LCL".

Mattias

Reply by email

Bo Berglund
to Mattias Gaertner
Mattias Gaertner wrote
comctrls is in package LCL. Check "Project / Project Inspector / Required Packages" has package "LCL".

Thanks! Worked OK.

Follow-up:
Is there a package with a TCP client socket as well? I need to replace a serial comm component with a TCPIP socket.

Bo Berglund Developer in Sweden

Reply by email

Michael Van Canneyt
to Bo Berglund
Bo Berglund wrote 12 lines
On Tue, 20 Oct 2015 19:16:13 +0200, Mattias Gaertner wrote: > comctrls is in package LCL. > Check "Project / Project Inspector / Required Packages" has package > "LCL". > Thanks! Worked OK. Follow-up: Is there a package with a TCP client socket as well? I need to replace a serial comm component with a TCPIP socket.

Use laz_synapse (blcksock unit) or unit ssockets (standard in fpc).

Michael.

Reply by email

Bo Berglund
to Michael Schnell
Michael Schnell wrote 7 lines
On 10/20/2015 07:06 PM, Bo Berglund wrote: > My program is a pretty simple command line program Description of TListview: " Use TListView to manage and display a list of items in a form" As a command line application does not have a form, this can't work.

I know that, but the existing rather big Delphi class unit I am converting to FPC does have some methods that are directed towards a GUI use and population of listviews.
I just want the unit to compile, but I will not use these methods in the command line application...
It is easier to leave the extra baggage in place rather than trying to remove it because of side effects and later GUI use. With LCL added it does work!

Bo Berglund Developer in Sweden

Reply by email

Michael Schnell
to Bo Berglund
Bo Berglund wrote
I just want the unit to compile, but I will not use these methods in the command line application...

A very bad idea, indeed. In many cases just using a unit will automatically call procedures in certain (here: GUI) libraries. Moreover using such a unit will pull multiple more (here: GUI) units

It is easier to leave the extra baggage in place rather than trying to remove it because of side effects and later GUI use.

If you want the project to "just compile" you need to implement the unsatisfied externals (classes, methods, properties) in your own "dummy" unit.

With LCL added it does work!

But now it is not a command line program any more.

-Michael

Reply by email

Mattias Gaertner
to Michael Schnell
[...]. > With LCL added it does work! But now it is not a command line program any more.

That is wrong in so many ways.

As long as you don't add the "interfaces" unit, no external GUI lib is used.
And even if you add the interfaces unit you can choose nogui widgetset to avoid using external libs.
And even if you use the Win32 widgetset, you can still have a command line program.
Under Linux/BSD every program is a command line program.

Mattias

Reply by email

Zeljko
to Mattias Gaertner
Mattias Gaertner wrote
Under Linux/BSD every program is a command line program.

It is, but with exception that qtlcl and gtk2lcl cannot run on headless linux machine. I'm giving this objection since one can create cmd line program on windows and then expect that it should work on headless linux machine (eg cross compiled for gtk2/qt).

zeljko

Reply by email

Mark Morgan Lloyd
to Zeljko
Zeljko wrote 8 lines
On 10/21/2015 02:18 PM, Mattias Gaertner wrote: > Under Linux/BSD every program is a command line program. It is, but with exception that qtlcl and gtk2lcl cannot run on headless linux machine. I'm giving this objection since one can create cmd line program on windows and then expect that it should work on headless linux machine (eg cross compiled for gtk2/qt).

A machine running Lazarus programs can be /physically/ headless, i.e. no physical video controller, but provided it's got X11 libraries and the appropriate widget set they can be run over a network connection.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Zeljko
to Mark Morgan Lloyd: A machine running Lazarus programs can be /physically/ headless, i.e. no physical video…
Mark Morgan Lloyd wrote 13 lines
Zeljko wrote: > On 10/21/2015 02:18 PM, Mattias Gaertner wrote: > >> Under Linux/BSD every program is a command line program. > > It is, but with exception that qtlcl and gtk2lcl cannot run on > headless linux machine. I'm giving this objection since one can create > cmd line program on windows and then expect that it should work on > headless linux machine (eg cross compiled for gtk2/qt). A machine running Lazarus programs can be /physically/ headless, i.e. no physical video controller, but provided it's got X11 libraries and the appropriate widget set they can be run over a network connection.

Yes, I meant no X11 running.

zeljko

Reply by email

Graeme Geldenhuys
to Bo Berglund: On Wed, 21 Oct 2015 10:08:40 +0200, Michael Schnell I know that, but the existing rather…
Bo Berglund wrote
With LCL added it does work!

I'm with Michael Schnell on this one - that is a terrible idea. So too is the LCL-NoGUI hack.

I would instead implement a Mock TListView class that implements the most basic interface your unit requires, but doesn't really do anything
- and don't include any LCL dependency. That would be my first step to "I just want it to compile".

Then refactor / clean-up the code to remove such "gui" dependencies if you want a true console application. Windows always has a GUI - so there you can get away with including GUI units, widgets or clipboard in your "console" application. But for Unix-based systems that have true headless environments, you *must* create a true console application, otherwise the application will not run.

Regards,
- Graeme -

fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp

Reply by email

Sven Barth
to Michael Schnell: Of course any executable can be started from the command line. But I don't suppose that…
"Michael Schnell" wrote
On 10/21/2015 02:18 PM, Mattias Gaertner wrote: > > And even if you use the Win32 widgetset, you can still have a command

line program.

Of course any executable can be started from the command line. But I

don't suppose that this is what Bo meant when he said he wants do do a "command line program".

But usage of the GUI on Windows does not preclude the usage of the console. That's only defined by the AppType and even an application of apptype GUI can allocate a console. If running in cmd.exe that would even result in that console being used (doesn't work in PowerShell however :/ ).

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20151021/e9b7d3ec/attachment.html>

Reply by email

Michael Schnell
to Graeme Geldenhuys: I'm with Michael Schnell on this one - that is a terrible idea. So too is the LCL-NoGUI…
Graeme Geldenhuys wrote
So too is the LCL-NoGUI hack.

That is why I am doing the "Active NoGUI" LCL Widget Type that is supposed to be a real application

Then refactor / clean-up the code to remove such "gui" dependencies

As I pointed out in the beginning of the discussion with Bo, a (IMHO the only) decent way to port a "RAD" Delphi application to a headless system is to first modify the application (without changing the environment and the functionality to a "non RAD" design by separating the business logic and the GUI in a different set of units. Now even the original project has improved.

In the next step the GUI units can be modified to be dummy or provide a minimal local or remote monitoring/debugging user interface.

-Michael

Reply by email

Bo Berglund
to Michael Schnell: Of course any executable can be started from the command line. But I don't suppose that…
Michael Schnell wrote 6 lines
On 10/21/2015 02:18 PM, Mattias Gaertner wrote: > And even if you use the Win32 widgetset, you can still have a command > line program. Of course any executable can be started from the command line. But I don't suppose that this is what Bo meant when he said he wants do do a "command line program".

Right, my target is a command line program started by cron and with no user interaction whatever. Could be thought of as a service.

But when programming it I need to use existing classes from Delphi in which I will use a subset of the functionality only. No functions dealing with user interface will be used at all, but in the same class is embedded the bulk of the communications protocol and data interpretation and translation and these I do need.

In any case adding LCL solved the compile problem for that unit.

My development machine is now also the target, a Raspberry Pi2, with FPC and Lazarus installed but it is headless. So my interface is PuTTY for command line work in Raspbian Jessie and for GUI (= Lazarus) I use tightvncserver on the Pi and TightVNCclient on Windows 7.

It works fine except for some small glitches like the inability to type { } [ ] $ @ \ | into the Lazarus code editor. I have to open the system text editor and type it there then copy paste it into the Lazarus editor....
It seems to be some kind of problem with the way keystrokes are sent in VNC and interpreted in the Lazarus code editor. I can enter these chars into the Lazarus property editors of GUI components though as well as in the Raspbian Text editor, all via VNC.

There is a separate thread about this here...

Bo Berglund Developer in Sweden

Reply by email

Michael Schnell
to Bo Berglund
Bo Berglund wrote
Could be thought of as a service. But when programming it I need to use existing classes from Delphi

Poor chap.

I have been investigating exactly this for years.

I posted the results in many answers to your forum messages.

-Michael

Reply by email

Mark Morgan Lloyd
to Bo Berglund: On Wed, 21 Oct 2015 14:38:54 +0200, Michael Schnell Right, my target is a command line…
Bo Berglund wrote 13 lines
My development machine is now also the target, a Raspberry Pi2, with FPC and Lazarus installed but it is headless. So my interface is PuTTY for command line work in Raspbian Jessie and for GUI (= Lazarus) I use tightvncserver on the Pi and TightVNCclient on Windows 7. It works fine except for some small glitches like the inability to type { } [ ] $ @ \ | into the Lazarus code editor. I have to open the system text editor and type it there then copy paste it into the Lazarus editor.... It seems to be some kind of problem with the way keystrokes are sent in VNC and interpreted in the Lazarus code editor. I can enter these chars into the Lazarus property editors of GUI components though as well as in the Raspbian Text editor, all via VNC.

As I've already said, that's a VNC problem. And again as I've already said, in most cases you'll do better running Lazarus over the SSH link.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Graeme Geldenhuys
to Bo Berglund: On Wed, 21 Oct 2015 14:38:54 +0200, Michael Schnell Right, my target is a command line…
Bo Berglund wrote 5 lines
But when programming it I need to use existing classes from Delphi in which I will use a subset of the functionality only. No functions dealing with user interface will be used at all, but in the same class is embedded the bulk of the communications protocol and data interpretation and translation and these I do need.

That still doesn't explain why the original Delphi code required a TListView instance. TListView is purely a GUI component - so I'm very puzzled.

If you needed a list of objects, and maybe an associated text string or something, then use TObjectList instead. I honestly don't see why TListView is need in any Console application.

Regards,
- Graeme -

fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp

Reply by email

Bo Berglund
to Graeme Geldenhuys
Graeme Geldenhuys wrote 7 lines
That still doesn't explain why the original Delphi code required a TListView instance. TListView is purely a GUI component - so I'm very puzzled. If you needed a list of objects, and maybe an associated text string or something, then use TObjectList instead. I honestly don't see why TListView is need in any Console application.

Is it so hard to understand that the class units were developed (a long time ago) in Delphi7 with a target of a GUI program? Tens of thousands of lines of code...
So it had helpers to show the recorded data in listviews and other such GUI related stuff (like handling progressbars etc). All of which does exactly nothing unless they have such components assigned to them.

In this case I do not want to re-invent all of the communications and data parsing code that is the main part of these units and which I need to use in the lean and mean communications program I have to build. So I do not want to rip the units apart and rewrite them if I can avoid it. This is the only reason.
And I did get a working way to handle this by including LCL. No more compile errors caused by these classes.

And my post at a late time in this thread was just to clarify this situation, not to ask any more questions so there was no need for more commenting.

Bo Berglund Developer in Sweden

Reply by email

Sven Barth
to Graeme Geldenhuys: I'm with Michael Schnell on this one - that is a terrible idea. So too is the LCL-NoGUI…

Am 23.10.2015 07:39 schrieb "Graeme Geldenhuys" <

Show quoted text 7 lines
: Then refactor / clean-up the code to remove such "gui" dependencies if you want a true console application. Windows always has a GUI - so there you can get away with including GUI units, widgets or clipboard in your "console" application. But for Unix-based systems that have true headless environments, you *must* create a true console application, otherwise the application will not run.

While I agree that Bo should refactor his code three is also the point that including just the units does no harm as Matthias has said. Only the "Interfaces" unit does that, cause that is the one that binds the components to the widgetset and is only used in the main program. This is especially true if the code related to the GUI components isn't even used (yes, I knew such stuffed units as well -.- ).

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20151023/4e4e4013/attachment.html>

Reply by email

Michael Schnell
to Graeme Geldenhuys: That still doesn't explain why the original Delphi code required a TListView instance.…
Graeme Geldenhuys wrote
That still doesn't explain why the original Delphi code required a TListView instance. TListView is purely a GUI component - so I'm very puzzled.

I had been at this point already years ago. As you might know, I myself do not have any Pascal "user" project to do, but what I wanted was to provide a "toolbox" for my colleagues, who manage a huge "embedded" Delphi project and would like to run it on Linux. As the project in fact is a Windows Service, the Linux machine is supposed to run "headless".

This is exactly the same task as Bo has in mind.

Now my colleagues (like Bo) decline to modify the Delphi code greatly, as the project still needs to be able to compile and run in Delphi for a long time in the future. (A friend of mine, as he faced the same issue, in fact rewrote his complete former Delphi project in Python and finally was very happy with this.)

So what they want is workalikes for the visual Delphi components (as they are provided by the LCL) but in a variant (i.e. "Widget type") that does not need a GUI binding. That is why I hat been working on an "ActiveNoGUI" Widget type (a better name might be "General Service Application", as its supposed to be independent of the OS and the ARCH).

In a first step it just should provide handling the fpc-RTL's TThread queue events, Application.ProcessMessages(),
Application.QueuAsyncCall(), and TTimer.

In a second step as many dummies for visual components as possible (or necessary) should be provided, maybe with some basic display/keyboard functions via pipe/telnet

In a third step "ifi" functionality should be added for providing an as complete as possible (or necessary) remote GUI.

But right now, these plans are of less interest in the company, as WIN 10 IOT provides running Delphi Service applications on a not bloated cost-free OS and cheap hardware such as the MinnowBoard MAX.. (Maybe not a bad idea for Bo, as well.)

As with any newer version of Windows, a Service is not allowed any more to (optionally) show a GUI, we (company) are pursuing the "un-RAD"ing of the software (separating the business-logic and the GUI in distinct units) anyway. So the result will be a compiled Delphi project and run on Win IOT. A remote user interface can be done by TCP/IP - streaming the completely propriety interface between these two brands of units.

Regarding Bo's project: This "un-RAD"ing would eliminate TListView and friend and allow for making it a reals Service (even still in Delphi) and then port it to headless Linux using Lazarus. (As I don't think he wold be inclined to follow me on the long (endless ?) path: General Service Application" Widget Type -> enhancing same to the visual components he uses -> "ifi").

-Michael

Reply by email

Michael Schnell
to Bo Berglund: On Thu, 22 Oct 2015 17:32:30 +0100, Graeme Geldenhuys Is it so hard to understand that…
Bo Berglund wrote
So it had helpers to show the recorded data in listviews and other such GUI related stuff

This is called "RAD" ("Rapid Application Development") and a big marketing argument of all companies who sold Delphi.

But it is very bad for portability. Investing some effort, it can be overcome, though (see my message to Graeme.)

Lazarus does a great job regarding portability of RAD projects, but it mainly has desktop usage in mind and not embedded gadgets.

In this case I do not want to re-invent all of the communications and data parsing code...

You obviously are out of luck (As already discussed, additionally to the "visual-component" issues, the Event-queue issue will hit you as well. To solve this, non-RAD will not be enough.).

There is no shortcut on this way. That is why I recommended to do a standard GUI application with Lazarus.

- Michael

Reply by email

Martin Schreiber
to Michael Schnell
Michael Schnell wrote 10 lines
On 10/22/2015 11:13 PM, Bo Berglund wrote: > So it had helpers to show the recorded data in listviews and other > such GUI related stuff This is called "RAD" ("Rapid Application Development") and a big marketing argument of all companies who sold Delphi. But it is very bad for portability. Investing some effort, it can be overcome, though (see my message to Graeme.)

In my understanding "RAD" is visually placing components in forms and datamodules, to set component properties in objectinspector and to use event properties in order to react on events.
Separating GUI and business logic is perfectly possible with RAD. Implement the business logic by components, properties and code in datamodules and additional units, for the GUI use forms witout or only GUI specific code. And before Graeme shows his image with the thousand components in a single datamodule here how it could look:
http://mseide-msegui.sourceforge.net/pics/mseguirad.png ;-)
In order to connect the GUI with the business model use TDatasource and TAction for example or use glue code. MSEgui additionally has the MSEifi components from component palette tab "ifi".
Applications without GUI use tnoguiapplication instead of tguiapplication as application instance for the main-eventqueue. tnoguiapplication has no dependency on GUI-libraries and GUI-services and provides a platform independent API. I assume Lazarus has a similar approach.

Martin

Reply by email

Michael Schnell
to Martin Schreiber
Martin Schreiber wrote
In my understanding "RAD" is visually placing components in forms and datamodules, ...

OK, "RAD" is just a name for whatever.

I don't disagree to calling "not using a separate set of units for business logic and GUI stuff" with another name. But I don't know such.

-Michael

Reply by email

Marcos Douglas
to Martin Schreiber: In my understanding "RAD" is visually placing components in forms and datamodules, to set…
Martin Schreiber wrote
Separating GUI and business logic is perfectly possible with RAD. Implement the business logic by components, properties and code in datamodules and additional units, for the GUI use forms witout or only GUI specific code.

Possible, yes, but is more difficult to understand, maintenance is compromised and hard to make tests.

Marcos Douglas

Reply by email

Sven Barth
to Michael Schnell: This is called "RAD" ("Rapid Application Development") and a big marketing argument of…
"Michael Schnell" wrote
> In this case I do not want to re-invent all of the communications and

data parsing code...

You obviously are out of luck (As already discussed, additionally to the

"visual-component" issues, the Event-queue issue will hit you as well. To solve this, non-RAD will not be enough.).

There is no shortcut on this way. That is why I recommended to do a

standard GUI application with Lazarus.

I somehow have the feeling that you and Graeme as well don't understand what Bo's precise problem is. He has a unit which is a collection of various different methods, classes, whatever. Some of these are related to GUI components (e.g. TTreeView) while others are not. He wants to use the non-GUI related code in his project without restructering the unit (for now). So this has /nothing/ to do with RAD or hom writing a console application or whatever. He simple needs to get the unit to compile (which he did by adding the LCL dependency) and stay clear from using any of the code that might use GUI. In that case there will be no dependency on a specific GUI, because the Interfaces unit is missing and the compiler might even smartlink away the GUI-related code that isn't used anyway.

TL;DR: Yes, Bo should restructure his code, but he does not /need/ to do so right now.

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20151023/49bf2955/attachment.html>

Reply by email

Michael Schnell
to Sven Barth
Sven Barth wrote 8 lines
I somehow have the feeling that you and Graeme as well don't understand what Bo's precise problem is. ... ... Yes, Bo should restructure his code, but he does not /need/ to do so right now.

Bo's precise problem is that he does not want to restructure his code.

-Michael

Reply by email

Michael Schnell
to Sven Barth: data parsing code... "visual-component" issues, the Event-queue issue will hit you as…
Sven Barth wrote 5 lines
He simple needs to get the unit to compile (which he did by adding the LCL dependency) and stay clear from using any of the code that might use GUI. In that case there will be no dependency on a specific GUI, because the Interfaces unit is missing and the compiler might even smartlink away the GUI-related code that isn't used anyway.

What about code in the "initialization" section of the multiple units that will get pulled even if using a single GUI related unit from the LCL ?

-Michael

Reply by email

Bo Berglund
to Sven Barth: Am 23.10.2015 07:39 schrieb "Graeme Geldenhuys" < While I agree that Bo should refactor…
Sven Barth wrote
This is especially true if the code related to the GUI components isn't even used (yes, I knew such stuffed units as well -.- ).

I just looked in our CVS server for the history of the most problematic unit. It was added to version control back in February 2001 and to date there are 99 revisions checked in. It does not show the best OO design one has seen...

Another pice of info:
About 2-3 years ago I had to make available a Windows DLL to support a university which needed to create its own software for research and it was rellay not possible for them to handle the communications side of it.
So I made (in Delphi) a Windows DLL which used this unit and basically also the other units I am now porting to Lazarus/FPC. Inside that DLL are between 22 functions related to the instrumentation handling, but none that refers to the GUI components.
But the problem unit is used in any case.
This DLL works just fine to control the instrumentation in Windows used in a Windows console program written in Delphi. It could also be used from Visual Basic 6 without any issues (I tested both).

So I am a bit surprised that just using this unit would make the FPC command program not work...

Bo Berglund Developer in Sweden

Reply by email

Graeme Geldenhuys
to Marcos Douglas: LOL! One or two times on year I see this image do you talked hehehe PS: Graeme, I agree…
Marcos Douglas wrote
LOL! One or two times on year I see this image do you talked hehehe

:-)

Two additional points:
- Martin's example is not nearly as big or complex as the example I showed in the past.
- The project I worked on was started with Delphi 2 if I recall, and progressed though the various Delphi's as they got released. I also vaguely remember up until Delphi 6 or 7 you could only have one DataModule per project. With later Delphi versions, Lazarus and MSEide that is not an issue any more.

The bottom line is, Delphi-style RAD promotes bad habits (this is fact with the Delphi projects I've seen over the last 20+ years), and we all know bad habits die hard.

Regards,
- Graeme -

fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp

Reply by email

Graeme Geldenhuys
to Sven Barth: data parsing code... "visual-component" issues, the Event-queue issue will hit you as…
Sven Barth wrote
I somehow have the feeling that you and Graeme as well don't understand what Bo's precise problem is.

Indeed - not at the start when Bo opened this thread. But as he explained later what he wanted, I understood.

Regards,
- Graeme -

fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp

Reply by email

Marcos Douglas
to Graeme Geldenhuys: :-) Two additional points: - Martin's example is not nearly as big or complex as the…
Graeme Geldenhuys wrote 6 lines
[...] - The project I worked on was started with Delphi 2 if I recall, and progressed though the various Delphi's as they got released. I also vaguely remember up until Delphi 6 or 7 you could only have one DataModule per project. With later Delphi versions, Lazarus and MSEide that is not an issue any more.

Maybe Delphi 4 because in version 5 you could have more than one DataModule per project.

The bottom line is, Delphi-style RAD promotes bad habits (this is fact with the Delphi projects I've seen over the last 20+ years), and we all know bad habits die hard.

I agree.

Marcos Douglas

Reply by email

Sven Barth
to Michael Schnell: What about code in the "initialization" section of the multiple units that will get…
"Michael Schnell" wrote
On 10/23/2015 02:12 PM, Sven Barth wrote: > > He simple needs to get the unit to compile (which he did by adding the

LCL dependency) and stay clear from using any of the code that might use GUI. In that case there will be no dependency on a specific GUI, because the Interfaces unit is missing and the compiler might even smartlink away the GUI-related code that isn't used anyway.

What about code in the "initialization" section of the multiple units

that will get pulled even if using a single GUI related unit from the LCL ?

Those and the code they link to would remain. It will depend on the specific initialization whether it can do any harm (especially with the widgetset not initialized) if left in...

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20151023/a4118d1a/attachment.html>

Reply by email

Sven Barth
to Bo Berglund: On Fri, 23 Oct 2015 08:01:20 +0200, Sven Barth I just looked in our CVS server for the…
"Bo Berglund" wrote
So I am a bit surprised that just using this unit would make the FPC command program not work...

It's not exactly FPC's fault, but more that one must not take one's environment for granted. E.g. on Windows you basically always have a GUI which is provided by the core OS (yes, there are things like headless servers or the new Windows 10 for IoT). On Linux systems on the other hand the GUI is entirely optional and provided by the distribution (I simplified here). In addition there can be multiple GUI toolkits (GTK and Qt to name two). If you now have for some reason a dependency on a Qt library but try to run it on a system without Qt your applications won't work. This part is solved by not using the Interfaces unit which is required for LCL GUI applications and which pulls in the widgetset (WinAPI, GTK, Qt, etc.) dependency (you are already doing this).
However there /might/ be unit initialization sections in the LCL units that rely on the fact that the widgetset had been initialized (because the Interfaces unit is usually one of the first used in the program file) and thus will result in a crash during the initialization of the unit. Again this does not happen with Delphi's VCL as there is no such widgetset abstraction like the LCL has, because it directly accesses the WinAPI (which is always there).

So I at least suggest you to split the GUI and non-GUI parts of the unit to be on the really safe side.

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20151023/e5606a9b/attachment-0001.html>

Reply by email

Mattias Gaertner
to Sven Barth: LCL dependency) and stay clear from using any of the code that might use GUI. In that…
[...] Those and the code they link to would remain. It will depend on the specific initialization whether it can do any harm (especially with the widgetset not initialized) if left in...

The most harmful are probably:
- forms.pp installs an exception handler
- lazutf8 under Windows tweaks some WideString manager functions optimized for UTF8 and sets DefaultSystemCodePage to CP_UTF8.

The other sections create objects and register classes, which don't do anything on their own.

Mattias

Reply by email

Sven Barth
to Michael Schnell
"Michael Schnell" wrote 5 lines
On 10/23/2015 05:01 PM, Sven Barth wrote: > > > It will depend on the specific initialization whether it can do any

harm (especially with the widgetset not initialized) if left in...

> So you suggest that they would not do any API calls to any external

Widget Set that in the target environment might not exist ?

Yes and Matthias did indeed confirm that.

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20151026/416ef1da/attachment.html>

Reply by email

Bo Berglund
to Michael Schnell: Bo's precise problem is that he does not want to restructure his code. -Michael
Michael Schnell wrote 10 lines
On 10/23/2015 02:12 PM, Sven Barth wrote: > > I somehow have the feeling that you and Graeme as well don't > understand what Bo's precise problem is. ... > > ... Yes, Bo should restructure his code, but he does not /need/ to do > so right now. > > Bo's precise problem is that he does not want to restructure his code.

Well, I might at one point or other decide to make a clone of the unit with the GUI dependency by cutting out the methods that are involved with these components. I have already done a similar restructuring by making a couple of unit copies like CommonFuncs => CommonFuncs_Laz where only the functions used by the new project remain. But these units are comparatively easy since they are just collections of "good to have" functions for various purposes and over the years they have collected garbage functions from all the users of these... The one I asked about is a dinosaur, which will take a while to fix...

But following the LCL addition I was no longer hindered by code that did not compile and could move on to other parts of the problem.

The biggest hurdle I have now is getting a TCP socket component working so I can plug it into the problem unit. I have tried several different approaches but am unfortunately not done yet. And I hesitate to ask more questions here because I feel I am getting flamed for not getting how it should be done. :(

And the last week I have been diverted to other pressing things too.

Bo Berglund Developer in Sweden

Reply by email

Bo Berglund
to Sven Barth: It's not exactly FPC's fault, but more that one must not take one's environment for…
Sven Barth wrote 26 lines
Am 23.10.2015 14:29 schrieb "Bo Berglund" : > So I am a bit surprised that just using this unit would make the FPC > command program not work... It's not exactly FPC's fault, but more that one must not take one's environment for granted. E.g. on Windows you basically always have a GUI which is provided by the core OS (yes, there are things like headless servers or the new Windows 10 for IoT). On Linux systems on the other hand the GUI is entirely optional and provided by the distribution (I simplified here). In addition there can be multiple GUI toolkits (GTK and Qt to name two). If you now have for some reason a dependency on a Qt library but try to run it on a system without Qt your applications won't work. This part is solved by not using the Interfaces unit which is required for LCL GUI applications and which pulls in the widgetset (WinAPI, GTK, Qt, etc.) dependency (you are already doing this). However there /might/ be unit initialization sections in the LCL units that rely on the fact that the widgetset had been initialized (because the Interfaces unit is usually one of the first used in the program file) and thus will result in a crash during the initialization of the unit. Again this does not happen with Delphi's VCL as there is no such widgetset abstraction like the LCL has, because it directly accesses the WinAPI (which is always there). So I at least suggest you to split the GUI and non-GUI parts of the unit to be on the really safe side.

OK, thanks for the explanation!
I guess that I will have a run-through of the methods in that unit and exclude all GUI related ones from the unit copy, thus minimizing code and getting rid of LCL altogether.

My most recent investigation is about how to create a TCPIP socket object that has event driven receive functions. I have no use for the problem unit in that quest so this thread is not really valid for those problems. Last I tested Indy10 with the help of Remy Lebeau but in the end I got caught by my inability to use a blocking socket with threads and fire off events into the main code running as the command line program.
I cannot use Synchronize because then the events fire off only when the main program code ends and if I don't use Synchronize then I get a SIGSEG exception somewhere.
Seems like I am crashing into the problem hinted at by several commentators here that it is near impossible unless the program is a GUI one....

I have taken a break for a few days now.

Bo Berglund Developer in Sweden

Reply by email

Sven Barth
to Bo Berglund: On Fri, 23 Oct 2015 14:15:59 +0200, Michael Schnell Well, I might at one point or other…
"Bo Berglund" wrote 5 lines
The biggest hurdle I have now is getting a TCP socket component working so I can plug it into the problem unit. I have tried several different approaches but am unfortunately not done yet. And I hesitate to ask more questions here because I feel I am getting flamed for not getting how it should be done. :(

Simply ignore those flaming answers.

Regards,
Sven
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20151031/7322e1ad/attachment.html>

Reply by email

Mark Morgan Lloyd
to Bo Berglund: On Fri, 23 Oct 2015 17:13:55 +0200, Sven Barth OK, thanks for the explanation! I guess…
Bo Berglund wrote 7 lines
My most recent investigation is about how to create a TCPIP socket object that has event driven receive functions. I have no use for the problem unit in that quest so this thread is not really valid for those problems. Last I tested Indy10 with the help of Remy Lebeau but in the end I got caught by my inability to use a blocking socket with threads and fire off events into the main code running as the command line program.

What does the main thread do? If you can contrive to write that as a loop, then as you've already been told you can call CheckSynchronize() which hands control to whatever's been queued by threads using Synchronize().

Depending on exactly what the program does, you might need to do something like fudging keyboard access using fpSelect(), but basically it works... I've got a mainframe emulator here which responds to keyboard commands and also interleaves output from the various backend processes, I've used Lazarus as the IDE but it's entirely a console app.

I cannot use Synchronize because then the events fire off only when the main program code ends and if I don't use Synchronize then I get a SIGSEG exception somewhere.

Assume that none of the LCL is thread-safe, i.e. background threads have no safe UI access. Assume that the RTL and FCL are not specifically thread safe except where documented, but that you can use it in conjunction with conventional "cautious" programming techniques.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Bo Berglund
to Mark Morgan Lloyd
Mark Morgan Lloyd wrote 6 lines
Bo Berglund wrote: What does the main thread do? If you can contrive to write that as a loop, then as you've already been told you can call CheckSynchronize() which hands control to whatever's been queued by threads using Synchronize().

Right now the main thread (you mean the program code itself in the console program DoRun event, right) does this (mostly debug messages into the console):

procedure TIndyTCPtest.DoRun;
var
  ErrorMsg: String;
  FComm: TSockReadThread;
begin
  FTerminated := false;
  { add your program here }
  try
    T1 := GetTickCount64;
    Writeln('Creating TCP client');
    FComm := TSockReadThread.Create('www.sunet.se', 80);
    FComm.OnConnected := @OnConnect;
    FComm.OnDisconnected := @OnDisConnect;
    Fcomm.OnData := @OnRxData;
    FComm.OnError := @OnError;
    FComm.OnTerminate := @OnTerminate;
    T2 := GetTickCount64;
    Writeln('Start communications at time ' + IntToStr(T2-T1));
    FComm.Start;
    Writeln('Connecting to www.sunet.se on port 80');
    T2 := GetTickCount64;
    Writeln('Going to sleep 500 at time ' + IntToStr(T2-T1));
    Sleep(500);
    T2 := GetTickCount64;
    Writeln('Waking up from sleep 500, at time ' + IntToStr(T2-T1));
    T2 := GetTickCount64;
    Writeln('Going to sleep 300 at time ' + IntToStr(T2-T1));
    Sleep(300);
    T2 := GetTickCount64;
    Writeln('Waking up from sleep 300 at time ' + IntToStr(T2-T1));
    Writeln('Terminating thread');
    FComm.Terminate;
    T2 := GetTickCount64;
    Writeln('Freeing FComm at time ' + IntToStr(T2-T1));
  finally
    FComm.Free;
  end;
  // stop program loop
  Terminate;
end;

And the output into the console is this:

Creating TCP client
Start communications at time 27
Connecting to www.sunet.se on port 80
Going to sleep 500 at time 27
Event: Socket connected! Time = 71 ms
Waking up from sleep 500, at time 527
Going to sleep 300 at time 527
Waking up from sleep 300 at time 827
Terminating thread
Freeing FComm at time 827

When I run it in Lazarus there is a pop-up window saying:

Project IndyTCPtest raised exception class 'External: SIGSEGV'.

At address 3132

Depending on exactly what the program does, you might need to do

I just wanted to make a test with the TThread child I created, which also contains the TIdTcpClient object, to see if the events fire off. But I only get a single one into my code, then the SIGSEGV error.

something like fudging keyboard access using fpSelect(), but basically it works... I've got a mainframe emulator here which responds to keyboard commands and also interleaves output from the various backend processes, I've used Lazarus as the IDE but it's entirely a console app.

I will have to figure out a way to make the test code do that loop. I had hoped that while executing sleep() statements the main thread would yield to the TCP thread but it seems like it does not...

Show quoted text 8 lines
> I cannot use Synchronize because then the events fire off only when > the main program code ends and if I don't use Synchronize then I get a > SIGSEG exception somewhere. Assume that none of the LCL is thread-safe, i.e. background threads have no safe UI access. Assume that the RTL and FCL are not specifically thread safe except where documented, but that you can use it in conjunction with conventional "cautious" programming techniques.

I have assumed as much so there is no side access to any data as far as I can tell.

Again, this is just a very simple console test program to check the operations of the threaded TCPIP communications, nothing near the final code because there I will have exit conditions and such.

Bo Berglund Developer in Sweden

Reply by email

Mark Morgan Lloyd
to Bo Berglund
Bo Berglund wrote 13 lines
On Sat, 31 Oct 2015 10:22:08 +0000, Mark Morgan Lloyd wrote: > Bo Berglund wrote: > What does the main thread do? If you can contrive to write that as a > loop, then as you've already been told you can call CheckSynchronize() > which hands control to whatever's been queued by threads using > Synchronize(). Right now the main thread (you mean the program code itself in the console program DoRun event, right) does this (mostly debug messages into the console):

I don't use Indy, but If I'm reading that lot correctly the answer is "outputs a couple of messages and terminates, within a second of startup".

Everybody's usage of threads etc. differs, depending on what they're doing and what their background is, so I hope somebody else is checking what I'm saying here (there's a big difference between using something and being competent to explain it- I consider myself in the former category). What I'd do is change the main code to be a loop to output your test messages without starting a background thread, and to loop for say a minute with Sleep() and CheckSynchronize(): if this crashes then you've got bigger problems.

Once this runs enable your comms thread and get it to output a couple of messages via Synchronize(), again this should not crash while the main program is running.

After that concentrate on- in particular- shutting down in good order, i.e. making sure that background threads are terminated before pulling the process out from under them.

Show quoted text 20 lines
Writeln('Freeing FComm at time ' + IntToStr(T2-T1)); finally FComm.Free; end; // stop program loop Terminate; end; And the output into the console is this: Creating TCP client Start communications at time 27 Connecting to www.sunet.se on port 80 Going to sleep 500 at time 27 Event: Socket connected! Time = 71 ms Waking up from sleep 500, at time 527 Going to sleep 300 at time 527 Waking up from sleep 300 at time 827 Terminating thread Freeing FComm at time 827

I hope somebody else will comment about how much housekeeping FPC does here, but I do believe you've got a race condition: what happens if the main thread (i.e. the overall process) terminates faster than the background thread?

Show quoted text 5 lines
When I run it in Lazarus there is a pop-up window saying: Project IndyTCPtest raised exception class 'External: SIGSEGV'. At address 3132

Make sure you're compiling with debugging enabled.

Show quoted text 5 lines
> Depending on exactly what the program does, you might need to do I just wanted to make a test with the TThread child I created, which also contains the TIdTcpClient object, to see if the events fire off. But I only get a single one into my code, then the SIGSEGV error.

You need to loop dequeuing events.

Show quoted text 8 lines
> something like fudging keyboard access using fpSelect(), but basically > it works... I've got a mainframe emulator here which responds to > keyboard commands and also interleaves output from the various backend > processes, I've used Lazarus as the IDE but it's entirely a console app. I will have to figure out a way to make the test code do that loop. I had hoped that while executing sleep() statements the main thread would yield to the TCP thread but it seems like it does not...

Definitely not. Think about it for a moment: Sleep() is basically a low-level OS call, it really doesn't want to get involved with the complexity of deciding whether it's already in an event handler or is in a background thread.

A good starting point would be having the main loop run for a fixed time- a minute or something like that. In my case I've fudged my own keyboard handler etc. in a way that (hopefully) will allow me to use a GUI frontend, in other words it has to either poll the keyboard (the simpler case) or use fpSelect(). Potentially, for a non-GUI program keyboard handling etc. could simply be moved into its own thread so when it's accumulated a command (e.g. 'quit') it simply enqueues it using Synchronize() and the main thread decides what to do with it.

Show quoted text 14 lines
>> I cannot use Synchronize because then the events fire off only when >> the main program code ends and if I don't use Synchronize then I get a >> SIGSEG exception somewhere. > Assume that none of the LCL is thread-safe, i.e. background threads have > no safe UI access. Assume that the RTL and FCL are not specifically > thread safe except where documented, but that you can use it in > conjunction with conventional "cautious" programming techniques. I have assumed as much so there is no side access to any data as far as I can tell. Again, this is just a very simple console test program to check the operations of the threaded TCPIP communications, nothing near the final code because there I will have exit conditions and such.

Start off with a very simple program to /only/ test a background thread. I'd suggest leaving Indy etc. out of it until you're confident that it works and never throws an exception, /particularly/ during startup and shutdown.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Bo Berglund
to Mark Morgan Lloyd
Mark Morgan Lloyd wrote 28 lines
What I'd do is change the main code to be a loop to output your test messages without starting a background thread, and to loop for say a minute with Sleep() and CheckSynchronize(): if this crashes then you've got bigger problems. Once this runs enable your comms thread and get it to output a couple of messages via Synchronize(), again this should not crash while the main program is running. I hope somebody else will comment about how much housekeeping FPC does here, but I do believe you've got a race condition: what happens if the main thread (i.e. the overall process) terminates faster than the background thread? > When I run it in Lazarus there is a pop-up window saying: > > Project IndyTCPtest raised exception class 'External: SIGSEGV'. > > At address 3132 Make sure you're compiling with debugging enabled. Start off with a very simple program to /only/ test a background thread. I'd suggest leaving Indy etc. out of it until you're confident that it works and never throws an exception, /particularly/ during startup and shutdown.

Mark,
thank you very much for your excellent descriptions and tips!

I did not know of CheckSynchronize before but I immediately threw in a number of such calls in the existing code that throws the SIGSEGV exception and Lo-and-behold! I am now getting the event output I was expecting and no SIGSEGV exception! :)

New console output:

Creating TCP client
Start communications at time 26
Connecting to www.sunet.se on port 80
Going to sleep 500 at time 26
Event: Socket connected! Time = 101 ms <== only previous event Waking up from sleep 500, at time 526
Going to sleep 300 at time 526
Waking up from sleep 300 at time 826
Terminating thread
Event: Socket disconnected! Time = 903 ms <== new event Event: Thread terminated! Time = 927 ms <== new event Freeing FComm at time 927

Now I will return to the test code and do a better main loop as well as trying to send something to the server.

The sending part is a bit of a mystery to me as well because the socket is owned by the reading thread and I don't currently know the best way to actually tell it to transmit some data. I got stuck by the non-appearance of the event execution before...

As you have seen I have used a website as the target just in order to verify the connect and disconnect events. To do real work I will need to get a test server going on my laptop which responds to actual commands sent by the client.

Here is the current code for the reading thread. The FClient object is created within the thread in the constructor and freed in the destructor.
I need to add a Write method that accepts input data in a byte array or similar and safeguard that against manipulation from other threads...

type
  TRxDataEvent = procedure(Sender: TObject; const AData: TIdBytes) of
object;
  TRxErrorEvent = procedure(Sender: TObject; AException: Exception) of
object;
  { TSockReadThread }
  TSockReadThread = class (TThread)
  private
    FClient: TIdTCPClient;
    FData: TIdBytes;
    FException: Exception;
    FOnConnected: TNotifyEvent;
    FOnDisconnected: TNotifyEvent;
    FOnData: TRxDataEvent;
    FOnError: TRxErrorEvent;
    procedure DoConnected;
    procedure DoDisconnected;
    procedure DoData;
    procedure DoError;
    function GetConnected: boolean;
  protected
    procedure Execute; override;
  public
    constructor Create(AHost: String; APort: TIdPort); reintroduce;
    destructor Destroy; override;
    property Connected: boolean read GetConnected;
    property OnConnected: TNotifyEvent read FOnConnected write
FOnConnected;
    property OnDisconnected: TNotifyEvent read FOnDisconnected write
FOnDisconnected;
    property OnData: TRxDataEvent read FOnData write FOnData;
    property OnError: TRxErrorEvent read FOnError write FOnError;
  end;

Tested towards the webserver before posting above message: -----------------------------------------------------------------
- Added public method Write(AData: TIdBytes); to TSockReadThread

- Implemented it as follows:
procedure TSockReadThread.Write(AData: TIdBytes);
begin
  if FClient.Connected then
     FClient.IOHandler.Write(AData);
end;
- Added extra lines in the main DoRun code:
    Writeln('Sending HELO to server');
    if FComm.Connected then
    begin
       FSendString := 'HELO'#13#10;
       SetLength(FSendData, Length(FSendString));
       Move(FSendString[1],FSendData[0],Length(FSendString));
       FComm.Write(FSendData);
    end;

Result in the console in Lazarus:

Creating TCP client
Start communications at time 25
Connecting to www.sunet.se on port 80
Going to sleep 500 at time 26
Event: Socket connected! Time = 99 ms
Waking up from sleep 500, at time 526
Sending HELO to server
Going to sleep 300 at time 526
Event: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="https:///">here</a>.</p> <hr>
<address>Apache/2.4.7 (Ubuntu) Server at localhost Port 80</address> </body></html>

Event: Socket disconnected! Time = 711 ms
Event: Connection Closed Gracefully. Time= 711ms
Event: Socket connected! Time = 716 ms
Waking up from sleep 300 at time 826
Terminating thread
Event: Socket disconnected! Time = 917 ms
Event: Thread terminated! Time = 926 ms
Freeing FComm at time 926
End of program at time 1127

So I *am* actually receiving data too! HOORAY!
But something is not timed correctly, for instance connection is reported at T=716 after the client has been disconnected... So there is still some work to be done.

Finally
My communications object does not use Synchronize() for the events. Should I add this back now that CheckSynchronize seems to have cured a few "shortcomings"?

Bo Berglund Developer in Sweden

Reply by email

Mark Morgan Lloyd
to Bo Berglund
Bo Berglund wrote 6 lines
thank you very much for your excellent descriptions and tips! I did not know of CheckSynchronize before but I immediately threw in a number of such calls in the existing code that throws the SIGSEGV exception and Lo-and-behold! I am now getting the event output I was expecting and no SIGSEGV exception! :)

I'm pretty sure it was mentioned a couple of days ago, since I had to ask what it was called (despite using it myself).

The sending part is a bit of a mystery to me as well because the socket is owned by the reading thread and I don't currently know the best way to actually tell it to transmit some data. I got stuck by the non-appearance of the event execution before...

Threads don't own things, things are owned by processes. I'm not entirely sure about this, but I think that you could probably have one thread reading and another writing.

HOWEVER...

Finally My communications object does not use Synchronize() for the events. Should I add this back now that CheckSynchronize seems to have cured a few "shortcomings"?

A lot will depend on whether you decide to use an off-the-shelf library or component suite, which would probably be appropriate if you're implementing a standard protocol. My understanding is that CheckSynchronize() does a significant amount of what the LCL main loop would do, which accounts for much of the difference between program types. Alternatively look at the lnet code that comes with FPC, I'm using the Telnet client from that to talk to things like simulated IBM mainframes.

As a general point, if you're implementing a protocol from scratch you might find it appropriate to use UDP rather than TCP. Since UDP is unreliable you'd obviously need to implement your own sequence numbers etc. to detect missed messages, but it has the advantage that messages won't need reassembly if they get fragmented due to e.g. a router having a smaller-than-expected MTU.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Michael Schnell
to Bo Berglund: On Fri, 23 Oct 2015 14:15:59 +0200, Michael Schnell Well, I might at one point or other…
Bo Berglund wrote 6 lines
> Bo's precise problem is that he does not want to restructure his code. . And I hesitate to ask more questions here because I feel I am getting flamed for not getting how it should be done. :(

I never intended to flame you. On the contrary I am fully with you, as I since years intended to provide a toolbox that allows to easily (i.e. without changing the user code) convert Delphi projects to headless Linux gadgets.

Unfortunately this is not as easy as it might be desirable :)

-Michael

Reply by email

Michael Schnell
to Bo Berglund: On Sat, 31 Oct 2015 10:22:08 +0000, Mark Morgan Lloyd Right now the main thread (you mean…
Bo Berglund wrote
I had hoped that while executing sleep() statements the main thread would yield to the TCP thread but it seems like it does not...

Sleep() indeed puts the running (main) thread to sleep and allows for other threads to run in the mean time.

The problem with sleep is not that it would not work as expected, but that it increases the latency. It your Thread fires main thread events (that later are called by "CheckSynchronize()"), with sleep they are not executed immediately (as one would desire) but only with a latency of (up to) the sleeping time.

-Michael

Reply by email

Michael Schnell
to Bo Berglund: On Sat, 31 Oct 2015 13:28:40 +0000, Mark Morgan Lloyd thank you very much for your…
Bo Berglund wrote
Should I add this back now that CheckSynchronize seems to have cured a few "shortcomings"?

Only with "CheckSynchronize" your thread can use TThread.Synchronize, TThread.Queue or PostMessage (with a Mainthread Object featuring "Procedure .... message" )

I don't know if Indy uses some of those. But if it does "CheckSynchronize" is a must.

-Michael

Reply by email

Bo Berglund
to Michael Schnell: IMHO using lnet or Synapse would be a lot "clearer" than using the overly complex Indy…
Michael Schnell wrote 5 lines
On 10/31/2015 07:28 PM, Mark Morgan Lloyd wrote: > Alternatively look at the lnet code that comes with FPC, ... IMHO using lnet or Synapse would be a lot "clearer" than using the overly complex Indy Library.

I had already looked at Synapse, but it is blocking. I did not know about lnet (or INet as it is shown when one searches) before but now I have had a look and it seems to fit the bill for me since it is event oriented, like I need for this particular task.

Thanks for pointing me in this direction!

I am now examining all the files and figuring out what to put into the RPi2 file system. THen I will make some tests with this before I go on threading Indy.

BTW: this thread has totally changed content from when it started, the subject is not really valid anymore. So if I need some more input I will create a more appropriate new thread (hmmm, a thread about threads)...

Bo Berglund Developer in Sweden

Reply by email

José Mejuto
to Bo Berglund

El 02/11/2015 a las 20:32, Bo Berglund escribi?:

Show quoted text 7 lines
> IMHO using lnet or Synapse would be a lot "clearer" than using the > overly complex Indy Library. I had already looked at Synapse, but it is blocking. I did not know about lnet (or INet as it is shown when one searches) before but now I have had a look and it seems to fit the bill for me since it is event oriented, like I need for this particular task.

Hello,

I was following your thread from the beginning and or I do not understand absolutely nothing about your needs or you are focusing the problem in the opposite way. Let me explain me, I think you had inherited a GUI app which you need to transform in a console one and this GUI app uses events to coordinate data flows, so in example the app connects to a server, gets a web page and some event handler process this data and outputs something to a file. The problem was that the event handler also updates information on screen (AKA TListView, ...) which you already removed.

That's fine, but my question is why you need event sockets ? I had converted some small apps with events in console ones and using only blocking sockets because I do not need to update screen information in GUI, only in console and as the console (in my case) do not need events from the user like response to keypress I do not need real events, so I wrote your "thread" in the main thread and manually "evented" each reception, something like this in pseudocode:

begin
   Socket.Connect...
   MyClass.Connect(Socket);
   Socket.http('xxxxx');
   while Socket.Connected do begin
     If Timedout() then break;
     ReceivedBytes:=Socket.Read(Buffer,0);
     if ReceivedBytes>=0 then begin
        ReceivedBytes:=Socket.Read(Buffer,ReceivedBytes);
        //Now I call the event reception
        //not a real event, but that's not important.
        MyClass.Received(Socket,Buffer,ReceivedBytes);
     end;
   end;
   Socket.Close;
   MyClass.Disconnect(Socket,REASON_GREACEFUL_CLOSE);
   Socket.Free;
end;

Of course the same can be done with multiple sockets at a time and call different handlers based in the socket that connects, receive data, etc...

Even if you use http transfers its more easy with blocking, something like:

Document:=http.get('http://xxxxxxx);
MyClass.Connect();
MyClass.Receive(Buffer,Length(Buffer));
MyClass.Closed();

Your class do not know if the data comes from an event or from a blocking socket. A different thing is if you need some kind of timing between operations.

Reply by email

Bo Berglund
to Bo Berglund: On Mon, 02 Nov 2015 10:24:54 +0100, Michael Schnell I had already looked at Synapse, but…
Bo Berglund wrote 11 lines
On Mon, 02 Nov 2015 10:24:54 +0100, Michael Schnell wrote: >On 10/31/2015 07:28 PM, Mark Morgan Lloyd wrote: >> Alternatively look at the lnet code that comes with FPC, ... I did not know about lnet before but now I have had a look and it seems to fit the bill for me since it is event oriented, like I need for this particular task. I will make some tests with this before I continue threading Indy.

UPDATE:
I successfully made a test console application using the lnet TLTcp class and it works like my previous test using Indy and a thread, except I don't have to go through the hassle of threading myself to fire off the receive data event.

So again:
Mark and Michael:
Thank you very much for your input and making me look at LNet! It seems to be an easier way forward for my project!

Bo Berglund Developer in Sweden

Reply by email

Bo Berglund
to José Mejuto: El 02/11/2015 a las 20:32, Bo Berglund escribi?: I was following your thread from the…
Jos? Mejuto wrote 14 lines
El 02/11/2015 a las 20:32, Bo Berglund escribi?: >> IMHO using lnet or Synapse would be a lot "clearer" than using the >> overly complex Indy Library. > I had already looked at Synapse, but it is blocking. > I did not know about lnet (or INet as it is shown when one searches) > before but now I have had a look and it seems to fit the bill for me > since it is event oriented, like I need for this particular task. > I was following your thread from the beginning and or I do not understand absolutely nothing about your needs or you are focusing the problem in the opposite way. Let me explain me, I think you had inherited a GUI app

Not really...
I had created a GUI program for talking to a data collection instrument via RS232 a long time ago. And to make things cleaner I had packaged all the specific communications functions in a class so the program only needed to call high level methods.
The communications protocol is proprietary and binary and rather complex.

which you need to transform in a console one and

The task I have to complete is to port the communications classes over to FPC/Lazarus so I can run a cron job for unattended data collection operations. THe controller is a Raspberry Pi2 with ARM7 CPU. And meanwhile the communications channel has switched from RS232 to TCPIP.

this GUI app uses events to coordinate data flows, so in example the app connects to a server, gets a web page and some event handler process this data and outputs something to a file.

No, there is no web server or file involved at all! The instrument itself is a WiFi access point and it also exposes the internal serial communications line (which still exists internally) on a TCP port (like 2001 or similar). So I need to use a TCP client component for the connection instead of using a serial comm component. But the serial component is embedded in the communications class and it is built on incoming data being made available via an OnReceive event. So in order to make things work I need to replace that serial comm component with a TCP component that receives data in the same way as the serial one does.

event handler also updates information on screen (AKA TListView, ...)

Now, this is the original topic of this thread since I discovered that the comm class also has helper methods to display received data on GUI components like TListView etc and so did not compile in Lazarus. THe helpers will not be called at all in the console case but by just being there and referencing TListView etc they stopped the compile until I was told to add LCL to the project dependencies.

And when I did so the compilation worked and this thread should have been closed, but unfortunately it continued but changed topic to event driven network TCP communications...

That's fine, but my question is why you need event sockets ?

Because I want to change the serial comm to TCP comm in the existing code that is entirely event based for data reception. See above.

Code example follows...

Well, if I had the opportunity and time to go ahead and rewrite the whole thing I might do it differently and maybe use your suggestions. But that is not an option given the size of the involved units...

Right now I am just happy that Mark pointed me towards LNet, which is a TCP comm suite using the event model. It seems like this will be a solution for me. Initial testing

Bo Berglund Developer in Sweden

Reply by email

Michael Schnell
to Bo Berglund: On Mon, 02 Nov 2015 10:24:54 +0100, Michael Schnell I had already looked at Synapse, but…
Bo Berglund wrote
I had already looked at Synapse, but it is blocking.

IMHO this is exactly what makes it "clearer". You can run it in a thread and do the communication with the main thread in any way you like.

A non-blocking thingy will either internally use multiple Threads (e.g. AsyncPro works in that way) or do stuff behind the screens that might hit you unexpectedly and prevent you intended cooperation with cron.

I did not know about lnet

AFAIK, it does not provide non-blocking mechanisms, either.

-Michael

Reply by email

Michael Schnell
to Michael Schnell: Maybe I was wrong here this mechanism might use a different mechanism (which supposedly…

For the record:

I verified that with the gtk2 (Linux) Widget type, "PostMessage()" uses a different Event Queue ("TGTKMessageQueue") than "TThread.Queue()" and TThread.Synchronize()".

So for this, calling "CheckSynchronize() will not help. You need to use the appropriate GUI-based Widget type.

Application.QueuAsyncCall() uses yet an additional Event Queue. Here using a WidgetType is not strictly necessary, but calling "CheckSynchronize() will not help, either. Here, TApplication provides "RemoveAsyncCalls()" which in a GUI Bases Widget Type is called in "Forms". As a non-GUI Project does not have Forms, RemoveAsyncCalls() needs to be called by a different functionality.

-Michael

Reply by email

Mark Morgan Lloyd
to Michael Schnell
Michael Schnell wrote 8 lines
For the record: I verified that with the gtk2 (Linux) Widget type, "PostMessage()" uses a different Event Queue ("TGTKMessageQueue") than "TThread.Queue()" and TThread.Synchronize()". So for this, calling "CheckSynchronize() will not help. You need to use the appropriate GUI-based Widget type.

Thanks for that, useful. That's going to impact anybody trying to convert code that uses custom Windows messages to pass events around.

Show quoted text 6 lines
Application.QueuAsyncCall() uses yet an additional Event Queue. Here using a WidgetType is not strictly necessary, but calling "CheckSynchronize() will not help, either. Here, TApplication provides "RemoveAsyncCalls()" which in a GUI Bases Widget Type is called in "Forms". As a non-GUI Project does not have Forms, RemoveAsyncCalls() needs to be called by a different functionality.

I don't think I've used QueueAsyncCall() in the context of a non-GUI program. Where I do find it useful is when converting an interactive non-GUI (i.e. console) program into a Lazarus GUI app: enqueue an async call in the main form's FormCreate() with the payload being e.g. a looping interactive command interpreter which calls Application.ProcessMessages when it's got nothing to do.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email