I'm working on a client/server project which involves sending SQL commands from server to client and vice versa.

I've been using the LTCP component for both the client and the server, which have been working perfectly during testing on the same machine and between different virtual machines, sending and receiving packets of data several thousand bytes in length.

However, when connecting over ethernet or the internet there appears to be a packet limit of 1036 bytes and any packets of data larger than this are truncated. I assume this is to do with maximum permissible transport units in the routers forming the connections and I was mistaken in believing that the lNet TCP implementation would take care of breaking the data down into transmissible packets and reassembling them at the destination.

Before I set out to implement the procedures to do this myself, can someone reassure me that this needs to be done and that I am not re-inventing the wheel?

Malcolm

2012/2/1 Malcolm Poole :

Before I set out to implement the procedures to do this myself, can someone reassure me that this needs to be done and that I am not re-inventing the wheel?

The TCP protocol will transparently split the data into the packets of needed size without user intervention, so the sending site is probably already ok.

But on the receiving side (at least with lNet) you will get a separate event for each received packet and have to combine them again, so you need to implement something on top of lNet to buffer and combine the received chunks of data and only process it when one of your protocol messages is complete. You need a robust way to determine how long your protocol message is, at which byte exactly it ends and the next message begins.

2012/2/4 Bernd :

You need a robust way to determine how long your protocol message is, at which byte exactly it ends and the next message begins.

The simplest way would probably be if you prepend all your messages with a length field and a checksum field to be sent as the very first bytes of each of your custom protocol messages (also take care of endianness and exactly specify this in your protocol description). Then you can either read as many bytes are available on the stream (if it is less than still needed) or read only as many bytes as are still missing (if equal or more are available), leave the rest of the data in the stream, process the message and trigger the read event again to read what can only be the beginning of the next message.

You can also use lNet in blocking mode (which can sometimes make things easier) simply by specifying a timeout value for the socket. Then you can just have a thread looping and trying to read indefinitely, counting received bytes and let it generate your own events on completion of a message instead of using the lNet events which fire on every TCP packet.

Many thanks Bernd, for your very helpful reply. The messages already start with a header record, which includes the length, so I'll add a checksum field to that and work along the lines you suggest.