Hi,

I am trying to test the websocket client in fcl-web by making a simple echo client. I have been unable to make it work using fpc trunk on linux64. I also crosscompiled to win64 and ran it on wine with identical results. The project is attached here if anyone can give some hints.

I am testing it with a server I found here
https://github.com/vi/wsmirror (written in rust) but also I tried it with some public echo servers. There is a html/javascript websocket example also for testing it which is working.

I have no real guesses as to the problem. I modified the server that the client connects to enough to know when it receives messages and what kind. The server never identifies a proper message after the connection handshake is completed. It gets something but it doesn't identify it as a text message or ping or close etc. The server is dropping the connection after a few seconds with ss -t dport = 8080 giving this:

State             Recv-Q Send-Q                         Local Address:Port

CLOSE-WAIT            5 0                                127.0.0.1:47828

I believe those are several ping packets the server is sending that the client doesn't read.

One thing the websocket is doing is disabling MSG_SIGNAL so that when the pipe is broken it doesn't show an exception. However the component doesn't check for this and doesn't send the disconnect event.

I have made my own websocket component in the past which is working but I wanted to try the one included with fpc.

I'm really baffled, I spent a long time to try and find what is wrong, even putting debug code in the underlying socket components. And verifying that the websocket frames are correct.

An oddity I noted is that TInetSocket.CanRead seems to always return false if it's used as a client. I may not have understood that fully.

Regards,

Andrew Haines

The FPC websocket protocol has been tested on the
'unofficial' testsuite for websocket

https://github.com/crossbario/autobahn-testsuite

and it passes all tests, and I have used it in commercial programs. so your story is quite surprising to me.

Does the sample chat client/program work for you ?

When I have a moment, I'll look at your test program.

Michael.

Michael Van Canneyt wrote

Does the sample chat client/program work for you ?

See below

When I have a moment, I'll look at your test program.

Thank you

Michael.

I'm not sure, maybe. It seems like the clients are not receiving messages. Also, using strace I realized the server ignores the port and uses 6060. I tried with and without threading.

./server/wsserver -p 8080 -t pool
Received message: { "from" : "client1", "msg" : "Hello, this is a friendly greeting message from the client", "to" : "client1" } Received message: { "from" : "client2", "msg" : "Hello, this is a friendly greeting message from the client", "to" : "client2" } Received message: { "from" : "client2", "msg" : "Hello to 1", "to" : "client1" }

Received message: { "from" : "client1", "msg" : "I didn't hear anything. Are you there?", "to" : "client2" }

Connection 1 disappeared
Connection 1 disappeared
Connection 2 disappeared
Connection 2 disappeared

./client/wsclient -u ws://localhost:6060/  -a client2 Enter message or command (/stop /help), empty message will just check for incoming messages
client2> Hello to 1
Recipient> client1
client2>

client2> /quit

./client/wsclient -u ws://localhost:6060/ -p -a client1 Enter message or command (/stop /help), empty message will just check for incoming messages
client1>
client1> I didn't hear anything. Are you there?
Recipient> client2
client1>
client1> /quit

State               Recv-Q Send-Q                             Local Address:Port                                Peer Address:Port Process ESTAB               176 0 127.0.0.1:36486 127.0.0.1:6060 users:(("wsclient",pid=1026336,fd=3))
ESTAB               204 0 127.0.0.1:36502 127.0.0.1:6060 users:(("wsclient",pid=1026342,fd=3))

Andrew

PS Here is the strace from a client without sending messages, but pressing the 'enter' key once..

socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 3
getsockopt(3, SOL_SOCKET, SO_RCVTIMEO_OLD,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", [16]) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(6060), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
sendto(3, "GET / HTTP/1.1\r\n", 16, MSG_NOSIGNAL, NULL, 0) = 16 sendto(3, "Host: localhost:6060\r\n", 22, MSG_NOSIGNAL, NULL, 0) = 22 sendto(3, "Upgrade: websocket\r\n", 20, MSG_NOSIGNAL, NULL, 0) = 20 sendto(3, "Connection: Upgrade\r\n", 21, MSG_NOSIGNAL, NULL, 0) = 21 sendto(3, "Origin: localhost\r\n", 19, MSG_NOSIGNAL, NULL, 0) = 19 sendto(3, "Sec-WebSocket-Key: 3pqrw1xgwqSel"..., 45, MSG_NOSIGNAL, NULL,
0) = 45
sendto(3, "Sec-WebSocket-Version: 13\r\n", 27, MSG_NOSIGNAL, NULL, 0) = 27 sendto(3, "\r\n", 2, MSG_NOSIGNAL, NULL, 0) = 2
recvfrom(3, "H", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "T", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "T", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "P", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "/", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "1", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, ".", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "1", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, " ", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "1", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "0", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "1", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, " ", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "S", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "w", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "i", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "t", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "c", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "h", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "i", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "n", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "g", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, " ", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "P", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "r", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "o", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "t", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "o", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "c", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "o", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "l", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "s", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "\r", 1, MSG_NOSIGNAL, NULL, NULL) = 1 recvfrom(3, "\n", 1, MSG_NOSIGNAL, NULL, NULL) = 1 recvfrom(3, "U", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "p", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "g", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "r", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "a", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "d", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, ":", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, " ", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "w", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "b", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "s", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "o", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "c", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "k", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "t", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "\r", 1, MSG_NOSIGNAL, NULL, NULL) = 1 recvfrom(3, "\n", 1, MSG_NOSIGNAL, NULL, NULL) = 1 recvfrom(3, "C", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "o", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "n", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "n", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "c", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "t", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "i", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "o", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "n", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, ":", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, " ", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "U", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "p", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "g", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "r", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "a", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "d", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "\r", 1, MSG_NOSIGNAL, NULL, NULL) = 1 recvfrom(3, "\n", 1, MSG_NOSIGNAL, NULL, NULL) = 1 recvfrom(3, "S", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "c", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "-", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "W", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "b", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "S", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "o", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "c", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "k", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "t", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "-", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "V", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "r", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "s", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "i", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "o", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "n", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, ":", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, " ", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "1", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "3", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "\r", 1, MSG_NOSIGNAL, NULL, NULL) = 1 recvfrom(3, "\n", 1, MSG_NOSIGNAL, NULL, NULL) = 1 recvfrom(3, "S", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "c", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "-", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "W", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "b", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "S", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "o", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "c", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "k", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "t", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "-", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "A", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "c", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "c", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "p", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "t", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, ":", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, " ", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "Y", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "d", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "C", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "Y", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "e", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "Y", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "M", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "S", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "G", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "8", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "E", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "q", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "i", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "f", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "7", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "E", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "J", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "R", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "K", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "B", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "/", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "C", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "N", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "g", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "C", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "7", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "A", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "=", 1, MSG_NOSIGNAL, NULL, NULL) = 1
recvfrom(3, "\r", 1, MSG_NOSIGNAL, NULL, NULL) = 1 recvfrom(3, "\n", 1, MSG_NOSIGNAL, NULL, NULL) = 1 recvfrom(3, "\r", 1, MSG_NOSIGNAL, NULL, NULL) = 1 recvfrom(3, "\n", 1, MSG_NOSIGNAL, NULL, NULL) = 1 write(1, "Enter message or command (/stop "..., 92Enter message or command (/stop /help), empty message will just check for incoming messages ) = 92
mmap(NULL, 32768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7f86d8b81000
mmap(NULL, 32768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = 0x7f86d8b79000
sendto(3, "\201n{ \"from\" : \"client2\", \"msg\" : "..., 112, MSG_NOSIGNAL, NULL, 0) = 112
select(4, [3], NULL, NULL, {tv_sec=0, tv_usec=0}) = 0 (Timeout) write(1, "client2> ", 9client2> )                = 9 read(0, [--ENTER KEY--]

"\n", 256)                      = 1 select(4, [3], NULL, NULL, {tv_sec=0, tv_usec=0}) = 1 (in [3], left {tv_sec=0, tv_usec=0})
write(1, "client2> ", 9client2> )                = 9 read(0,

I will check this.

From your strace it looks like recent changes in ssockets broke something:

the 1 character sized reads are not 'll report back.

Michael.

I fixed the -p/--port option.

I could reproduce the broken behaviour in the fcl-web example.

As I thought, a change in the fcl-net ssockets unit is the cause of the behaviour. I fixed it.

Please check if your example now also works ?

Michael.

I tested the client/server example and they are working now. My echo test program still sends but does not receive.

Andrew

Please make a version of your program that does not use the LCL. Then I'll test that too.

2 reasons for this request:
- I don't have a working version of Lazarus with FPC trunk.
- I want to exclude the problems of dealing with the main application message loop.

Michael.

I attached a console version. It uses threads. If this is useful at all please feel free to add it to the examples and modify it however you want.

I found the problem. First was the problem of no data being read, which you fixed by changing the inheritance. But the second problem is that the OutgoingFrameMask value was not being used. In the rfc section 5.2 it states in the Mask bit explanation that all frames sent from a client must be masked. Unfortunately the echo server was not giving a close frame with a protocol error which would have been useful.

In the attached patch there are 3 hunks that include the OutgoingFrameMask with the payload. The rest of the patch is about handling a connection that encounters an error sending data. I added an exception and also a flag that ignores the exception and tries to close gracefully.

You can compile the console client I sent without applying the patch to see how it was failing before. There are default servers to choose from.

Maybe in the client, the constructor should pick a mask at random by default since all frames from the client are supposed to be masked anyway. Section 5.3 has some strong language about the random mask having a strong entropy source. It actually says each frame should have a new unique mask so I'm not sure the OutgoingFrameMask property really makes sense since the mask is given to the connection only once when the connection is created. I doubt most server implementations care about this.

Regards,

Andrew

Hm.

I can say with certainty then that not all server implementations check this :)

Indeed, I can confirm the problem, and that your patch fixes it.

Maybe in the client, the constructor should pick a mask at random by default since all frames from the client are supposed to be masked anyway.

Good suggestion. I will add this.

Indeed, see above.

The whole mask idea is shaky anyway.
The mask is sent in the frame, so that's pretty useless for security, which begs the question why it was needed in the first place. If one wants secure connections, one should use the wss protocol. Probably the reason why most servers don't care too much about it.

Anyway: I have applied the patch, and added your example with some minor modifications. Thank you for both !

Michael.

Awesome thanks very much!

Andrew