Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2020-10

netdb, DNS and TCP

5 messages · 2 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Noel Duffy

Hi all,

To keep track of the work I've been doing on DNS in netdb, I created bug #37906 in Mantis.

https://bugs.freepascal.org/view.php?id=37906

I've attached to a note on that bug report a proof-of-concept version of netdb.pp and a small program that uses it to make text queries over TCP. This version of netdb has different record types for TCP and UDP (technically, it has three, one for UDP queries and replies, one for TCP queries, one for TCP replies). The wire protocol for DNS over TCP has a mandatory two-octet length field at the start for both queries and replies, and I didn't want to have to use a full-length 64k buffer for both when a smaller 512 byte buffer suffices.

Overall I'm not really enthused with this implementation. Having separate record types for the different payload types means having separate versions of a number of helper functions as well. Functions like BuildPayload, NextRR, and SkipAnsQueries.

To avoid that duplication, the code would need to use a dynamic buffer and fill it just before socket writing. The TCP variant would include its length field in the buffer. Additionally, after reading the response from the socket, the buffer contents would have to be copied into a record data structure. It's a tradeoff between execution efficiency and simplicity.

On a related point, does anyone have any tips for testing low-level network functions? I'm not sure if there's an easy way, perhaps with add-on test libraries, to simulate things like network errors, packet loss, and the like. I'm currently thinking the only way to do it would be to create mocked versions of the fpsend and fprecv calls. I know about tc netem on linux, but something that can be used inside unit tests would be ideal.

Reply by email

Michael Van Canneyt
to Noel Duffy
Noel Duffy wrote 26 lines
Hi all, To keep track of the work I've been doing on DNS in netdb, I created bug #37906 in Mantis. https://bugs.freepascal.org/view.php?id=37906 I've attached to a note on that bug report a proof-of-concept version of netdb.pp and a small program that uses it to make text queries over TCP. This version of netdb has different record types for TCP and UDP (technically, it has three, one for UDP queries and replies, one for TCP queries, one for TCP replies). The wire protocol for DNS over TCP has a mandatory two-octet length field at the start for both queries and replies, and I didn't want to have to use a full-length 64k buffer for both when a smaller 512 byte buffer suffices. Overall I'm not really enthused with this implementation. Having separate record types for the different payload types means having separate versions of a number of helper functions as well. Functions like BuildPayload, NextRR, and SkipAnsQueries. To avoid that duplication, the code would need to use a dynamic buffer and fill it just before socket writing. The TCP variant would include its length field in the buffer. Additionally, after reading the response from the socket, the buffer contents would have to be copied into a record data structure. It's a tradeoff between execution efficiency and simplicity.

It always is.

But I am inclined to think that DNS queries benefit more from execution efficiency, hence the preference for avoiding the copying of buffers as much as possible.

One could of course argue that with TCP, the speed suffers in each case.

If you have suggestions for a better implementation, I am all ears...

Michael.

Reply by email

Noel Duffy
to Michael Van Canneyt
Michael Van Canneyt wrote 12 lines
On Sun, 11 Oct 2020, Noel Duffy via fpc-pascal wrote: > To avoid that duplication, the code would need to use a dynamic buffer and fill it just before socket writing. The TCP variant would include its length field in the buffer. Additionally, after reading the response from the socket, the buffer contents would have to be copied into a record data structure. It's a tradeoff between execution efficiency and simplicity. It always is. But I am inclined to think that DNS queries benefit more from execution efficiency, hence the preference for avoiding the copying of buffers as much as possible. One could of course argue that with TCP, the speed suffers in each case.

Therein lies the rub. Once your code reads or writes to the network, all bets are off with regard to speed.

If you have suggestions for a better implementation, I am all ears...

Well, better depends on what you're optimizing for, of course. In many places the DRY principle is considered a Holy Commandment graven on stone tablets and breaking it tantamount to heresy.

If the community thinks breaking DRY is acceptable to preserve speed in this case then I'm willing to accept that and run with it.

Reply by email

Noel Duffy
to Michael Van Canneyt: It always is. But I am inclined to think that DNS queries benefit more from execution…
Michael Van Canneyt wrote 12 lines
On Sun, 11 Oct 2020, Noel Duffy via fpc-pascal wrote: > To avoid that duplication, the code would need to use a dynamic buffer and fill it just before socket writing. The TCP variant would include its length field in the buffer. Additionally, after reading the response from the socket, the buffer contents would have to be copied into a record data structure. It's a tradeoff between execution efficiency and simplicity. It always is. But I am inclined to think that DNS queries benefit more from execution efficiency, hence the preference for avoiding the copying of buffers as much as possible. One could of course argue that with TCP, the speed suffers in each case.

Therein lies the rub. Once your code reads or writes to the network, all bets are off with regard to speed.

If you have suggestions for a better implementation, I am all ears...

Well, better depends on what you're optimizing for, of course. In many places the DRY principle is considered a Holy Commandment graven on stone tablets and breaking it tantamount to heresy.

If the community thinks breaking DRY is acceptable to preserve speed in this case then I'm willing to accept that and run with it.

Reply by email

Michael Van Canneyt
to Noel Duffy
Noel Duffy wrote 28 lines
On 11/10/20 9:42 pm, Michael Van Canneyt via fpc-pascal wrote: > > On Sun, 11 Oct 2020, Noel Duffy via fpc-pascal wrote: >> To avoid that duplication, the code would need to use a dynamic buffer and fill it just before socket writing. The TCP variant would include its length field in the buffer. Additionally, after reading the response from the socket, the buffer contents would have to be copied into a record data structure. It's a tradeoff between execution efficiency and simplicity. > > It always is. > > But I am inclined to think that DNS queries benefit more from execution > efficiency, hence the preference for avoiding the copying of buffers as much > as possible. > > One could of course argue that with TCP, the speed suffers in each case. Therein lies the rub. Once your code reads or writes to the network, all bets are off with regard to speed. > > If you have suggestions for a better implementation, I am all ears... Well, better depends on what you're optimizing for, of course. In many places the DRY principle is considered a Holy Commandment graven on stone tablets and breaking it tantamount to heresy.

Then I am a heretic :-)

I have a pragmatic approach to such things. DRY is a valuable principle. But normally I will prefer clarity of code to DRY. I still want to understand today's code 15 years from now at a glance. Or someone else's code today.

Michael.

Reply by email