Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2020-06

fpWeb and server-sent events

18 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Luca Olivetti

Hello,

I need to embed a simple web server in my application that

1) should server a couple of static files
2) send server-sent events

https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events

I can see that 1) is easy, but 2) I cannot see how (fpWeb model doesn't seem to allow to take over the socket and keep sending data on it). I saw an old message regarding websockets and bauglir-websocket but it seems it implements a different server and I'd like to run everything from the same socket.

Bye

Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007

Reply by email

Michael Van Canneyt
to Luca Olivetti
Luca Olivetti wrote 15 lines
Hello, I need to embed a simple web server in my application that 1) should server a couple of static files 2) send server-sent events https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events I can see that 1) is easy, but 2) I cannot see how (fpWeb model doesn't seem to allow to take over the socket and keep sending data on it). I saw an old message regarding websockets and bauglir-websocket but it seems it implements a different server and I'd like to run everything from the same socket.

Simply said: You can't in FPC.

The only way to do this with FPC is using websockets, indeed using bauglir-websocket. I use it myself like that.

Michael.

Reply by email

Luca Olivetti
to Michael Van Canneyt

El 3/6/20 a les 15:54, Michael Van Canneyt via lazarus ha escrit:

Show quoted text 26 lines
On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote: > Hello, > > I need to embed a simple web server in my application that > > 1) should server a couple of static files > 2) send server-sent events > > https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events > > > > I can see that 1) is easy, but 2) I cannot see how (fpWeb model > doesn't seem to allow to take over the socket and keep sending data on > it). > I saw an old message regarding websockets and bauglir-websocket but it > seems it implements a different server and I'd like to run everything > from the same socket. Simply said: You can't in FPC. The only way to do this with FPC is using websockets, indeed using bauglir-websocket. I use it myself like that.

I didn't look into it too much, but does it also serve "normal" http? If so I could adapt it to manage events (after all it's much simpler than websockets).

Bye

Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007

Reply by email

Luca Olivetti
to Luca Olivetti

El 3/6/20 a les 16:32, Luca Olivetti via lazarus ha escrit:

Show quoted text 31 lines
El 3/6/20 a les 15:54, Michael Van Canneyt via lazarus ha escrit: > > > On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote: > >> Hello, >> >> I need to embed a simple web server in my application that >> >> 1) should server a couple of static files >> 2) send server-sent events >> >> https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events >> >> >> >> I can see that 1) is easy, but 2) I cannot see how (fpWeb model >> doesn't seem to allow to take over the socket and keep sending data >> on it). >> I saw an old message regarding websockets and bauglir-websocket but >> it seems it implements a different server and I'd like to run >> everything from the same socket. > > Simply said: You can't in FPC. > > The only way to do this with FPC is using websockets, indeed using > bauglir-websocket. I use it myself like that. I didn't look into it too much, but does it also serve "normal" http? If so I could adapt it to manage events (after all it's much simpler than websockets).

I see that I can connect the EventSource to a different socket, so I can use fpWeb to serve static content and a simple socket to send events.

Bye

Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007

Reply by email

Michael Van Canneyt
to Luca Olivetti: El 3/6/20 a les 15:54, Michael Van Canneyt via lazarus ha escrit: I didn't look into it…
Luca Olivetti wrote 32 lines
El 3/6/20 a les 15:54, Michael Van Canneyt via lazarus ha escrit: > > > On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote: > >> Hello, >> >> I need to embed a simple web server in my application that >> >> 1) should server a couple of static files >> 2) send server-sent events >> >> https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events >> >> >> >> I can see that 1) is easy, but 2) I cannot see how (fpWeb model >> doesn't seem to allow to take over the socket and keep sending data on >> it). >> I saw an old message regarding websockets and bauglir-websocket but it >> seems it implements a different server and I'd like to run everything >> from the same socket. > > Simply said: You can't in FPC. > > The only way to do this with FPC is using websockets, indeed using > bauglir-websocket. I use it myself like that. I didn't look into it too much, but does it also serve "normal" http? If so I could adapt it to manage events (after all it's much simpler than websockets).

No, it does not serve normal http.

Michael.

Reply by email

silvioprog
to Luca Olivetti: I need to embed a simple web server in my application that 1) should server a couple of…

Hi Luca.

I hope you are fine in the current world situation.

Some answers below to your questions 1 and 2.

On Wed, Jun 3, 2020 at 10:30 AM Luca Olivetti via lazarus <

Show quoted text 8 lines
wrote: Hello, I need to embed a simple web server in my application that 1) should server a couple of static files 2) send server-sent events

1) So, you can use any Pascal structure to serve files, even in a single routine. BTW, if you want to resolve media types, you can use fpmimetypes or BrookMediaTypes
<https://github.com/risoflora/brookframework/blob/expr/Source/BrookMediaTypes.pas> .

2) As Michael explained, unfortunately, it is not supported in FCL-Web (yet) . However, if you want a SSE structure out-of-the-box, you can provide it using the Brook streaming, e.g.:

Minimal console example:
https://github.com/risoflora/brookframework/blob/expr/Examples/Console/FPC/httpsrvsse.lpr

Minimal LCL example:
https://github.com/risoflora/brookframework/blob/expr/Examples/LCL/HTTPServerSSE_frMain.pas

Both examples above declare a small HTML client showing how to consume it via event source (EventSource). Optionally, you can use some async structure (e.g.: TSpinWait.SpinUntil) to create a small polling to push/pop the messages provided to the clients in a single long lived event.

HTH

Silvio Clécio -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20200603/b852f2d4/attachment-0001.html>

Reply by email

Luca Olivetti
to silvioprog

El 3/6/20 a les 18:53, silvioprog via lazarus ha escrit:

Show quoted text 9 lines
2) As Michael explained, unfortunately, it is not supported in FCL-Web (yet) . However, if you want a SSE structure out-of-the-box, you can provide it using the Brook streaming, e.g.: Minimal console example: https://github.com/risoflora/brookframework/blob/expr/Examples/Console/FPC/httpsrvsse.lpr Minimal LCL example: https://github.com/risoflora/brookframework/blob/expr/Examples/LCL/HTTPServerSSE_frMain.pas

Thank you.
I tested the 2 ports approach (fpweb on one port and a custom socket for SSE on a different port) and it works, but this is nicer. Is brookframework stable enough to run 24/7?

Bye

Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007

Reply by email

silvioprog
to Luca Olivetti

On Wed, Jun 3, 2020 at 2:04 PM Luca Olivetti via lazarus <

Show quoted text 7 lines
wrote: El 3/6/20 a les 18:53, silvioprog via lazarus ha escrit: Thank you. I tested the 2 ports approach (fpweb on one port and a custom socket for SSE on a different port) and it works, but this is nicer.

Pretty good! Maybe providing a small example would be useful for the community. ☺

Is brookframework stable enough to run 24/7?

Yes, it is. We have been using it as Apache/Nginx replacement running in production in an application as service (daemon) 24/7 in ~47 customers providing data for several clients (web, mobile and desktop).

Silvio Clécio -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20200603/e10a2727/attachment.html>

Reply by email

Luca Olivetti
to silvioprog

El 3/6/20 a les 19:41, silvioprog via lazarus ha escrit:

Show quoted text 13 lines
On Wed, Jun 3, 2020 at 2:04 PM Luca Olivetti via lazarus < <mailto:>> wrote: El 3/6/20 a les 18:53, silvioprog via lazarus ha escrit: Thank you. I tested the 2 ports approach (fpweb on one port and a custom socket for SSE on a different port) and it works, but this is nicer. Pretty good! Maybe providing a small example would be useful for the community. ☺

Too quick and dirty to be an example ;-) Maybe tomorrow I'll clean it up a bit, or maybe I'll go with brookframework directly :-D

Show quoted text 7 lines
Is brookframework stable enough to run 24/7? Yes, it is. We have been using it as Apache/Nginx replacement running in production in an application as service (daemon) 24/7 in ~47 customers providing data for several clients (web, mobile and desktop).

Good, then I guess the question is if firexox and/or chromium are stable enough to run 24/7 (which I doubt...)

Bye

Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007

Reply by email

Luca Olivetti
to Michael Van Canneyt: Simply said: You can't in FPC. The only way to do this with FPC is using websockets,…

El 3/6/20 a les 15:54, Michael Van Canneyt via lazarus ha escrit:

Show quoted text 23 lines
On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote: > Hello, > > I need to embed a simple web server in my application that > > 1) should server a couple of static files > 2) send server-sent events > > https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events > > > > I can see that 1) is easy, but 2) I cannot see how (fpWeb model > doesn't seem to allow to take over the socket and keep sending data on > it). > I saw an old message regarding websockets and bauglir-websocket but it > seems it implements a different server and I'd like to run everything > from the same socket. Simply said: You can't in FPC.

It turns out I can with some careful, or reckless ;-), overriding.

https://paste.ubuntu.com/p/CdHfc7BTHB/

Bye

Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007

Reply by email

Michael Van Canneyt
to Luca Olivetti
Luca Olivetti wrote 29 lines
El 3/6/20 a les 15:54, Michael Van Canneyt via lazarus ha escrit: > > > On Wed, 3 Jun 2020, Luca Olivetti via lazarus wrote: > >> Hello, >> >> I need to embed a simple web server in my application that >> >> 1) should server a couple of static files >> 2) send server-sent events >> >> https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events >> >> >> >> I can see that 1) is easy, but 2) I cannot see how (fpWeb model >> doesn't seem to allow to take over the socket and keep sending data on >> it). >> I saw an old message regarding websockets and bauglir-websocket but it >> seems it implements a different server and I'd like to run everything >> from the same socket. > > Simply said: You can't in FPC. It turns out I can with some careful, or reckless ;-), overriding. https://paste.ubuntu.com/p/CdHfc7BTHB/

Haha, very nice job :-)

I'll see if I can integrate this in the HTTPServer request/response.

It won't work in fastCGI/CGI of course, but if we can make it available in a http application, why not...

Michael.

Reply by email

Luca Olivetti
to Michael Van Canneyt

El 4/6/20 a les 11:48, Michael Van Canneyt ha escrit:

Show quoted text 11 lines
> It turns out I can with some careful, or reckless ;-),  overriding. > > https://paste.ubuntu.com/p/CdHfc7BTHB/ Haha, very nice job :-) I'll see if I can integrate this in the HTTPServer request/response. It won't work in fastCGI/CGI of course, but if we can make it available in a http application, why not...

I think that approach (take over the socket from the response) could be useful in more cases, e.g. streaming content, websockets, etc. As you say it has to be a stand-alone/embedded server and probably it only works when it's threaded (due to the connection being kept active for an indeterminate time), but that's better than nothing.

Now, I have another problem: if the server cannot bind to the port it throws an exception, but then I cannot try again, I have to destroy and recreate the server, e.g. this doesn't work (the second call to FServer.Run return immediately):

   FServer:=TSseServerHandler.Create(nil);
   FServer.Port:=8080;
   FServer.Threaded:=true;
   while not terminated do
   try
     FServer.Run;
   except
     on E:Exception do
     begin
       writeln(E.Message);
       sleep(1000);
       FServer.Terminate;
     end;
   end;

this does (but I don't like it)

   while not terminated do
   try
     FServer:=TSseServerHandler.Create(nil);
     FServer.Port:=8080;
     FServer.Threaded:=true;
     FServer.Run;
   except
     on E:Exception do
     begin
       writeln(E.Message);
       sleep(1000);
       FServer.Free;
     end;
   end;

Bye

Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007

Reply by email

Luca Olivetti
to Luca Olivetti

El 4/6/20 a les 12:04, Luca Olivetti via lazarus ha escrit:

Show quoted text 59 lines
El 4/6/20 a les 11:48, Michael Van Canneyt ha escrit: >> It turns out I can with some careful, or reckless ;-),  overriding. >> >> https://paste.ubuntu.com/p/CdHfc7BTHB/ > > Haha, very nice job :-) > > I'll see if I can integrate this in the HTTPServer request/response. > > It won't work in fastCGI/CGI of course, but if we can make it > available in a > http application, why not... I think that approach (take over the socket from the response) could be useful in more cases, e.g. streaming content, websockets, etc. As you say it has to be a stand-alone/embedded server and probably it only works when it's threaded (due to the connection being kept active for an indeterminate time), but that's better than nothing. Now, I have another problem: if the server cannot bind to the port it throws an exception, but then I cannot try again, I have to destroy and recreate the server, e.g. this doesn't work (the second call to FServer.Run return immediately):   FServer:=TSseServerHandler.Create(nil);   FServer.Port:=8080;   FServer.Threaded:=true;   while not terminated do   try     FServer.Run;   except     on E:Exception do     begin       writeln(E.Message);       sleep(1000);       FServer.Terminate;     end;   end; this does (but I don't like it)   while not terminated do   try     FServer:=TSseServerHandler.Create(nil);     FServer.Port:=8080;     FServer.Threaded:=true;     FServer.Run;   except     on E:Exception do     begin       writeln(E.Message);       sleep(1000);       FServer.Free;     end;   end;

Oh, and I had to do some more overriding so that the response could have a reference to the calling thread (needed if I want to use synchronize).

Bye

Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007

Reply by email

Michael Van Canneyt
to Luca Olivetti
Luca Olivetti wrote 14 lines
>     FServer.Port:=8080; >     FServer.Threaded:=true; >     FServer.Run; >   except >     on E:Exception do >     begin >       writeln(E.Message); >       sleep(1000); >       FServer.Free; >     end; >   end; Oh, and I had to do some more overriding so that the response could have a reference to the calling thread (needed if I want to use synchronize).

That seems a bit strange, since you can do synchronize without needing access to the thread object using a class method of TThread ?

Can you send me the changes you had to do ?

Michael.

Reply by email

Luca Olivetti
to Michael Van Canneyt

El 4/6/20 a les 15:37, Michael Van Canneyt via lazarus ha escrit:

Show quoted text 22 lines
On Thu, 4 Jun 2020, Luca Olivetti via lazarus wrote: >>      FServer.Port:=8080; >>      FServer.Threaded:=true; >>      FServer.Run; >>    except >>      on E:Exception do >>      begin >>        writeln(E.Message); >>        sleep(1000); >>        FServer.Free; >>      end; >>    end; > > Oh, and I had to do some more overriding so that the response could > have a reference to the calling thread (needed if I want to use > synchronize). That seems a bit strange, since you can do synchronize without needing access to the thread object using a class method of TThread ?

TThread.Synchronize needs a TThread as the first parameter. This is my work in progress

https://paste.ubuntu.com/p/cKdQmCQw7p/

The main difference from the previous pastebin is that I subclass TFPHTTPConnection to add a TThread field and override the CreateConnectionThread to add the thread reference to the connection. And I added more parameters to the server thread to pass on events from the main thread.

Tlogger is a simple class defined in utils.pas (not included) that logs messages.

The idea is that every time a new sse connection comes in, I notify the main thread (not yet implemented), then in DoSendConnect I enter a loop where I get the data from the main thread (with synchronize) then wait on an event (that the main thread signals every time the data changes). If there is no new data, every 10 seconds I send an empty message as a keepalive.

So, I need synchronize:

- to notify a new connection
- to get data for that connection
- to notify that the connection terminated

Bye

Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007

Reply by email

Sven Barth
to Luca Olivetti

Luca Olivetti via lazarus schrieb am Do.,
4. Juni 2020, 15:51:

Show quoted text 27 lines
El 4/6/20 a les 15:37, Michael Van Canneyt via lazarus ha escrit: > > > On Thu, 4 Jun 2020, Luca Olivetti via lazarus wrote: > >>> FServer.Port:=8080; >>> FServer.Threaded:=true; >>> FServer.Run; >>> except >>> on E:Exception do >>> begin >>> writeln(E.Message); >>> sleep(1000); >>> FServer.Free; >>> end; >>> end; >> >> Oh, and I had to do some more overriding so that the response could >> have a reference to the calling thread (needed if I want to use >> synchronize). > > That seems a bit strange, since you can do synchronize without needing > access to the thread object using a class method of TThread ? TThread.Synchronize needs a TThread as the first parameter. This is my work in progress

The thread parameter can be Nil. TThread will then use the CurrentThread threadvar and failing that it can also work without any thread object.

In fact it is safer not to pass a thread.

Regards,
Sven

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20200604/c19b0412/attachment-0001.html>

Reply by email

Luca Olivetti
to Sven Barth

El 4/6/20 a les 17:07, Sven Barth ha escrit:

Show quoted text 9 lines
TThread.Synchronize needs a TThread as the first parameter. This is my work in progress The thread parameter can be Nil. TThread will then use the CurrentThread threadvar and failing that it can also work without any thread object. In fact it is safer not to pass a thread.

Oh, I didn't know that. Thank you.

Bye

Luca Olivetti Wetron Automation Technology http://www.wetron.es/ Tel. +34 93 5883004 (Ext.3010) Fax +34 93 5883007

Reply by email

silvioprog
to Luca Olivetti: El 3/6/20 a les 19:41, silvioprog via lazarus ha escrit: Too quick and dirty to be an…

On Wed, Jun 3, 2020 at 3:46 PM Luca Olivetti via lazarus <

Show quoted text 5 lines
wrote: El 3/6/20 a les 19:41, silvioprog via lazarus ha escrit: Good, then I guess the question is if firexox and/or chromium are stable enough to run 24/7 (which I doubt...)

😂

Silvio Clécio -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.lazarus-ide.org/pipermail/lazarus/attachments/20200605/3479ebf4/attachment.html>

Reply by email