Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2017-09

FCL-WEB and PostgreSQL Support: Current Status

18 messages · 8 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

African Wild Dog

Hello,

I am about to start a new project for a very small web application.

I have some questions:

1 - Are the fcl-web and postgres packages stable?
2 - Are they recommended for use in production environments? 3 - Do they have any serious bug (for example, memory leaks)?

Regards

Reply by email

Michael Van Canneyt
to African Wild Dog
African Wild Dog wrote 7 lines
Hello, I am about to start a new project for a very small web application. I have some questions: 1 - Are the fcl-web and postgres packages stable?

Since 8 years.

2 - Are they recommended for use in production environments?

Absolutely.
I use them in very high-load (and we're talking VERY high load) environments, with 24/7 availability.

3 - Do they have any serious bug (for example, memory leaks)?

None that I know of.

Michael.

Reply by email

Karoly Balogh (Charlie/SGR
to Michael Van Canneyt

Hi,

Michael Van Canneyt wrote 5 lines
> 2 - Are they recommended for use in production environments? Absolutely. I use them in very high-load (and we're talking VERY high load) environments, with 24/7 availability.

Can you ellaborate a bit more on this Michael? Like how and where it is used? (Not mentioning company names, if it's a problem.) I was thinking about using it in various places, but as I don't know how widely they're tested and used and under what conditions, I didn't had the balls to offer this as viable alternative against competing technology X.

We definitely need more use case studies...

Charlie

Reply by email

African Wild Dog
to Michael Van Canneyt: Since 8 years. Absolutely. I use them in very high-load (and we're talking VERY high load)

2017-09-27 12:29 GMT-03:00 Michael Van Canneyt :

Show quoted text 8 lines
2 - Are they recommended for use in production environments? > Absolutely. I use them in very high-load (and we're talking VERY high load) environments, with 24/7 availability.

Could you tell which web server you are using in this environment? Are you using FastCGI?

Regards

Reply by email

Michael Van Canneyt
to African Wild Dog
African Wild Dog wrote 12 lines
2017-09-27 12:29 GMT-03:00 Michael Van Canneyt : > > 2 - Are they recommended for use in production environments? >> > > Absolutely. I use them in very high-load (and we're talking VERY high load) > environments, with 24/7 availability. > > Could you tell which web server you are using in this environment? Are you using FastCGI?

Apache, indeed using FastCGI.
Used both mod_fastcgi as mod_fcgid. mod_proxy should work as well.

Michael.

Reply by email

Michael Van Canneyt
to Karoly Balogh (Charlie/SGR: Can you ellaborate a bit more on this Michael? Like how and where it is used? (Not…
Karoly Balogh wrote 15 lines
Hi, On Wed, 27 Sep 2017, Michael Van Canneyt wrote: > > 2 - Are they recommended for use in production environments? > > Absolutely. > I use them in very high-load (and we're talking VERY high load) > environments, with 24/7 availability. Can you ellaborate a bit more on this Michael? Like how and where it is used? (Not mentioning company names, if it's a problem.) I was thinking about using it in various places, but as I don't know how widely they're tested and used and under what conditions, I didn't had the balls to offer this as viable alternative against competing technology X.

It is extremely viable.

I have used it in 2 companies, first production dating back to 2010.

Mainly it's used to offer read/write REST apis, with access to Firebird or Postgres databases. You can add some Cassandra, MemCached and Redis into the mix, and even (yeeeeeecch) mysql.

It's also used as a middle layer on top Microsoft/Google APIS to offer a single, unified API to access google docs or onedrive to the REST client. Thus, the oauth2, json apis of FPC are well tested, as well as the webclient units.

It also offers a variety of export data (CSV/XML/HTML/Excel).

All 24/7.

The new fpreport is used to provide reporting services - outputting PDF and sending that directly to the browser.

All this has been tested heavily on Linux and Windows.

So yes, I can say it is safe to use in production.

And it will only get better once pas2js is released (expected somewhere in December). FPC will offer a full web stack that sends NodeJS reeling...

Michael.

Reply by email

Mark Morgan Lloyd
to Karoly Balogh (Charlie/SGR: Can you ellaborate a bit more on this Michael? Like how and where it is used? (Not…
Karoly Balogh wrote 5 lines
Hi, On Wed, 27 Sep 2017, Michael Van Canneyt wrote: >> 2 - Are they recommended for use in production environments?>> Absolutely.> I use them in very high-load (and we're talking VERY high load)> environments, with 24/7 availability. Can you ellaborate a bit more on this Michael? Like how and where it isused? (Not mentioning company names, if it's a problem.) I was thinkingabout using it in various places, but as I don't know how widely they'retested and used and under what conditions, I didn't had the balls to offerthis as viable alternative against competing technology X. We definitely need more use case studies...

The only problems I've experienced with FPC's support for PostgreSQL were related to handling SQL queries (e.g. spurious parameter substitution) rather than to dataflow. As such they were predictable and could be worked round.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Marco van de Voort
to Mark Morgan Lloyd

In our previous episode, Mark Morgan Lloyd said:

Show quoted text 7 lines
> Can you ellaborate a bit more on this Michael? Like how and where it isused? (Not mentioning company names, if it's a problem.) I was thinkingabout using it in various places, but as I don't know how widely they'retested and used and under what conditions, I didn't had the balls to offerthis as viable alternative against competing technology X. > We definitely need more use case studies... The only problems I've experienced with FPC's support for PostgreSQL were related to handling SQL queries (e.g. spurious parameter substitution) rather than to dataflow. As such they were predictable and could be worked round.

Zeljan once explained an interesting practical problem he had.

He said the connection per transaction model could be dangerous, specially in a mobile era where connections are often broken, and thus must wait on timeout. This means during rush times, a server can run out of a fairly limited ephemeral port inventory before abandoned connections start to time out.

That is a postgres problem though, not the FPC components.

Reply by email

Michael Van Canneyt
to Marco van de Voort
Marco van de Voort wrote 18 lines
In our previous episode, Mark Morgan Lloyd said: > > Can you ellaborate a bit more on this Michael? Like how and where it isused? (Not mentioning company names, if it's a problem.) I was thinkingabout using it in various places, but as I don't know how widely they'retested and used and under what conditions, I didn't had the balls to offerthis as viable alternative against competing technology X. > > We definitely need more use case studies... > > The only problems I've experienced with FPC's support for PostgreSQL > were related to handling SQL queries (e.g. spurious parameter > substitution) rather than to dataflow. As such they were predictable and > could be worked round. Zeljan once explained an interesting practical problem he had. He said the connection per transaction model could be dangerous, specially in a mobile era where connections are often broken, and thus must wait on timeout. This means during rush times, a server can run out of a fairly limited ephemeral port inventory before abandoned connections start to time out. That is a postgres problem though, not the FPC components.

1. In a REST setting, if you use FastCGI and connnection pooling, this should not be a problem.
The webserver normally has a reliable connection with the DB server:)

2. On Postgres specifically, you can use PGBouncer. It mitigates the number of connections to the server.

Michael.

Reply by email

Mark Morgan Lloyd
to Marco van de Voort: In our previous episode, Mark Morgan Lloyd said: Zeljan once explained an interesting…
Marco van de Voort wrote
In our previous episode, Mark Morgan Lloyd said:> > Can you ellaborate a bit more on this Michael? Like how and where it isused? (Not mentioning company names, if it's a problem.) I was thinkingabout using it in various places, but as I don't know how widely they'retested and used and under what conditions, I didn't had the balls to offerthis as viable alternative against competing technology X.> > We definitely need more use case studies...> > The only problems I've experienced with FPC's support for PostgreSQL > were related to handling SQL queries (e.g. spurious parameter > substitution) rather than to dataflow. As such they were predictable and > could be worked round. Zeljan once explained an interesting practical problem he had. He said the connection per transaction model could be dangerous, speciallyin a mobile era where connections are often broken, and thus must wait ontimeout. This means during rush times, a server can run out of a fairlylimited ephemeral port inventory before abandoned connections start to timeout. That is a postgres problem though, not the FPC components

Which I suppose really starts getting to be an issue when somebody tries to "put something in the cloud" without careful consideration of that type of scenario.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]

Reply by email

Marcos Douglas B. Santos
to African Wild Dog: I am about to start a new project for a very small web application. I have some questions:
African Wild Dog wrote 9 lines
Hello, I am about to start a new project for a very small web application. I have some questions: 1 - Are the fcl-web and postgres packages stable? 2 - Are they recommended for use in production environments? 3 - Do they have any serious bug (for example, memory leaks)?

I have been using fcl-web in production about 2 years. However, my environment is Windows and Microsoft IIS, running FastCGI. Most of these apps are used to provide REST API's. I don't have any problems. These apps are fast and I can use shared code between Desktop and Web. Very good.

Best regards,
Marcos Douglas

Reply by email

Daniel Gaspary
to Michael Van Canneyt: Apache, indeed using FastCGI. Used both mod_fastcgi as mod_fcgid. mod_proxy should work…
Michael Van Canneyt wrote
Apache, indeed using FastCGI. Used both mod_fastcgi as mod_fcgid. mod_proxy should work as well.

What do you use to debug this environment, just log messages / log server..?

Sometime ago I have seen someone mentioned developing with a separated application which communicated to the cgi / fastcgi part. This Cgi / fastcgi part was used just to receive / response requests from / to the web server. This way the debugging became much easier.

My question: How this application communicate with the Cgi / FastCgi part?

TSimpleIpc seens to only communicate one way. Anyone using two TSimpeIpc's components to simulate a duplex channel?

Any chance of extend the component / create a new one, with support to receive and send data ?

Are you using Sockets(Ip not unix sockets, like SimpleIpc)? I would like to avoid that, because of firewall restrictions to open a server socket

Reply by email

Michael Van Canneyt
to Daniel Gaspary
Daniel Gaspary wrote 7 lines
On Wed, Sep 27, 2017 at 12:53 PM, Michael Van Canneyt wrote: > Apache, indeed using FastCGI. Used both mod_fastcgi as mod_fcgid. mod_proxy > should work as well. What do you use to debug this environment, just log messages / log server..?

See below.

Show quoted text 7 lines
Sometime ago I have seen someone mentioned developing with a separated application which communicated to the cgi / fastcgi part. This Cgi / fastcgi part was used just to receive / response requests from / to the web server. This way the debugging became much easier. My question: How this application communicate with the Cgi / FastCgi part?

You don't need that at all, that is way too complicated.

You have 3 options to debug:

1. You can perfectly start the fastcgi in the debugger if you use proxy server or if you use mod_fastcgi confgured with fastcgiexternalserver configuration setting. I believe nginx only uses the proxy mechanism.

2. If that is not an option, there is also the following: FPC contains a cgi to fastcgi request converter, you can use that to debug the fastcgi by running it inside the debugger, and let apache forward the request using the cgi bridge.

3. Third option:
FCL-Web is transparant in the sense that the code for CGI/FastCGI/Apache module or standalone HTTP server is 100% the same. Only the used TCustomApplication descendent differs for these 4 environments.

That means you can debug your code using the HTTP server, and switch to using FastCGI in production. I use an {$IFDEF } in the project file to switch the hosting environment. For the business code this is transparant.

Show quoted text 5 lines
TSimpleIpc seens to only communicate one way. Anyone using two TSimpeIpc's components to simulate a duplex channel? Any chance of extend the component / create a new one, with support to receive and send data ?

No. I don't see the need.
See above, there are plenty of mechanisms to debug a web environment.

Michael.

Reply by email

Daniel Gaspary
to Michael Van Canneyt
Michael Van Canneyt wrote
1. You can perfectly start the fastcgi in the debugger if you use proxy server or if you use mod_fastcgi confgured with fastcgiexternalserver configuration setting. I believe nginx only uses the proxy mechanism.

Nice to know about this configuration. It seens similar to what I was imagining, using unix sockets(optionally)

2. If that is not an option, there is also the following: FPC contains a cgi to fastcgi request converter, you can use that to debug the fastcgi by running it inside the debugger, and let apache forward the request using the cgi bridge.

Could you elaborate about this bridge? Is some apache/nginx configuration, an external tool, using some apache mod.. ??

Show quoted text 8 lines
3. Third option: FCL-Web is transparant in the sense that the code for CGI/FastCGI/Apache module or standalone HTTP server is 100% the same. Only the used TCustomApplication descendent differs for these 4 environments. That means you can debug your code using the HTTP server, and switch to using FastCGI in production. I use an {$IFDEF } in the project file to switch the hosting environment. For the business code this is transparant.

This and the first appear to be the simpler options.

Show quoted text 5 lines
> Any chance of extend the component / create a new one, with support to > receive and send data ? No. I don't see the need. See above, there are plenty of mechanisms to debug a web environment.

I mean .. Not just for this case, but to have a more capable IPC.

Reply by email

Michael Van Canneyt
to Daniel Gaspary: On Thu, Sep 28, 2017 at 6:11 PM, Michael Van Canneyt Nice to know about this…
Daniel Gaspary wrote 17 lines
On Thu, Sep 28, 2017 at 6:11 PM, Michael Van Canneyt wrote: > 1. You can perfectly start the fastcgi in the debugger if you use proxy > server > or if you use mod_fastcgi confgured with fastcgiexternalserver configuration > setting. I believe nginx only uses the proxy mechanism. Nice to know about this configuration. It seens similar to what I was imagining, using unix sockets(optionally) > 2. If that is not an option, there is also the following: > FPC contains a cgi to fastcgi request converter, you can use that to debug > the fastcgi by running it inside the debugger, and let apache forward the > request using the cgi bridge. Could you elaborate about this bridge? Is some apache/nginx configuration, an external tool, using some apache mod.. ??

See unit fcgigate.pp. It has some comments explaining usage.

To the outside world and the webserver, your application is a CGI app. The cgi application simply repacks the request, sends it to the fastcgi server, retrieves the response and sends it back to the webserver.

You just need to set the port your FastCGI app listens on. It uses a default of 2015, so if you use that, no need to configure anything. Just compile the gateway, drop it wherever you want/need it, and you're ready to go.

Show quoted text 11 lines
> 3. Third option: > FCL-Web is transparant in the sense that the code for CGI/FastCGI/Apache > module or standalone HTTP server is 100% the same. Only the used > TCustomApplication descendent differs for these 4 environments. > > That means you can debug your code using the HTTP server, and switch to > using FastCGI in production. I use an {$IFDEF } in the project file to > switch the hosting environment. For the business code this is transparant. This and the first appear to be the simpler options.

2 is also quite simple. It doesn't even need a recompile, and no need to configure anything in the webserver.

Show quoted text 8 lines
>> Any chance of extend the component / create a new one, with support to >> receive and send data ? > > No. I don't see the need. See above, there are plenty of mechanisms to debug > a web environment. I mean .. Not just for this case, but to have a more capable IPC.

Well, it's called SimpleIPC for a reason. The current implementation is already far too complicated to my taste...

But there is nothing preventing us to add a 2-way communication mechanism. but I suggest using WST or somesuch instead of SimpleIPC. WST has lots of communication channels available,
and it's very easy to set up a simple general-purpose messaging system.

FPC really has all you need, even in 2017 ;)

Michael.

Reply by email