Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2016-01

Writing floating points to steams

18 messages · 9 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Torsten Bonde Christiansen

Hi all.

What is the best/preferred way to write floatingpoint to a steam?

Since TStream doesn't have any native WriteFloat/Double and the likes, some conversion is needed - but what is a good strategy for this?

A simple conversion like Stream.WriteDWord( DWord(<some float variable>)) is not possible due to illegal type conversion.

Any help appriciated.

Regards,
Torsten.

Torsten Bonde Christiansen
to Graeme Geldenhuys
Graeme Geldenhuys wrote 10 lines
On 2016-01-12 10:45, Torsten Bonde Christiansen wrote: > Since TStream doesn't have any native WriteFloat/Double and the likes, > some conversion is needed - but what is a good strategy for this? I use FloatStr() to do the conversion, then Write() to actually write it to a stream. Regards, - Graeme -

In my case I am trying to write a datetime value, and in order not to worry about the format, separators, etc. i would much rather just write the actual floating point value...

Regards,
Torsten.

Michael Van Canneyt
to Torsten Bonde Christiansen
Torsten Bonde Christiansen wrote 14 lines
On 2016-01-12 12:04, Graeme Geldenhuys wrote: > On 2016-01-12 10:45, Torsten Bonde Christiansen wrote: >> Since TStream doesn't have any native WriteFloat/Double and the likes, >> some conversion is needed - but what is a good strategy for this? > I use FloatStr() to do the conversion, then Write() to actually write it > to a stream. > > > Regards, > - Graeme - > In my case I am trying to write a datetime value, and in order not to worry about the format, separators, etc. i would much rather just write the actual floating point value...

Format the date/time as RFC3339 or ISO 8601 (they differe in details only) and you should not have a problem. ScanDateTime from strutils will happily perform the opposite operation.

Michael.

Torsten Bonde Christiansen
to Michael Van Canneyt
Michael Van Canneyt wrote 24 lines
On Tue, 12 Jan 2016, Torsten Bonde Christiansen wrote: > On 2016-01-12 12:04, Graeme Geldenhuys wrote: >> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote: >>> Since TStream doesn't have any native WriteFloat/Double and the likes, >>> some conversion is needed - but what is a good strategy for this? >> I use FloatStr() to do the conversion, then Write() to actually >> write it >> to a stream. >> >> >> Regards, >> - Graeme - >> > In my case I am trying to write a datetime value, and in order not to > worry about the format, separators, etc. i would much rather just > write the actual floating point value... Format the date/time as RFC3339 or ISO 8601 (they differe in details only) and you should not have a problem. ScanDateTime from strutils will happily perform the opposite operation.

I will give it a try.

But out of curiosity is there a reason why there are not floating point Read/Write methods for TStream (and decendants)?

Regards,
Torsten.

Michael Schnell
to Torsten Bonde Christiansen
Torsten Bonde Christiansen wrote
But out of curiosity is there a reason why there are not floating point Read/Write methods for TStream (and decendants)?

As

   procedure TStream.WriteQWord(q: QWord);
     begin
       WriteBuffer(q,8);
     end;

it's rather easy to do your own TStream.WriteDateTime procedure and friends.

-Michael

Serguei TARASSOV
On 2016-01-12 10:45, Torsten Bonde Christiansen wrote: > Since TStream doesn't have any native WriteFloat/Double and the likes, > some conversion is needed - but what is a good strategy for this?

Depends on size constraints.
In the simple case you should convert float to locale-independent string value (no spaces, no commas separators) then write it. If the size does matter, you may convert float to corresponding byte array then write it.

Regards,
Serguei

Mark Morgan Lloyd
to Serguei TARASSOV
Serguei TARASSOV wrote 9 lines
On 13/01/2016 08:47, wrote: > On 2016-01-12 10:45, Torsten Bonde Christiansen wrote: >> Since TStream doesn't have any native WriteFloat/Double and the likes, >> some conversion is needed - but what is a good strategy for this? Depends on size constraints. In the simple case you should convert float to locale-independent string value (no spaces, no commas separators) then write it. If the size does matter, you may convert float to corresponding byte array then write it.

Can I ask a naive question here please: does a binary stream store endianness anywhere? In other words, is there any indication that something running on ARM is about to get into trouble by reading something written by (big-endian) MIPS?

Should writing binary floating point to a stream note that it's IEEE format, just in case anybody ever tries to process it on a platform that supports alternatives?

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]
Torsten Bonde Christiansen
to Mark Morgan Lloyd
Mark Morgan Lloyd wrote 15 lines
Serguei TARASSOV wrote: > On 13/01/2016 08:47, wrote: >> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote: >>> Since TStream doesn't have any native WriteFloat/Double and the likes, >>> some conversion is needed - but what is a good strategy for this? > Depends on size constraints. > In the simple case you should convert float to locale-independent > string value (no spaces, no commas separators) then write it. > If the size does matter, you may convert float to corresponding byte > array then write it. Can I ask a naive question here please: does a binary stream store endianness anywhere? In other words, is there any indication that something running on ARM is about to get into trouble by reading something written by (big-endian) MIPS?

Not that I'm aware of - i would think of it as a stream of bytes. Endianess is defined by the CPU not by the file.

Show quoted text 5 lines
Should writing binary floating point to a stream note that it's IEEE format, just in case anybody ever tries to process it on a platform that supports alternatives?

It would be great if storing floating point could be in IEEE, to have a standard as reference.

Regards,
Torsten.

Michael Schnell
to Mark Morgan Lloyd: Can I ask a naive question here please: does a binary stream store endianness anywhere?…
Mark Morgan Lloyd wrote
Can I ask a naive question here please: does a binary stream store endianness anywhere?

AFAIK, generally, such binary (file-, Network transfer, ....) formats are not supposed to be "natively" portable. To be portable they should be either propriety (defined by the *single* application that uses them) or Text-based (such as XML or "INITFILE").

-Michael

Mark Morgan Lloyd
to Torsten Bonde Christiansen: Not that I'm aware of - i would think of it as a stream of bytes. Endianess is defined by…
Torsten Bonde Christiansen wrote 19 lines
On 2016-01-13 10:06, Mark Morgan Lloyd wrote: > Serguei TARASSOV wrote: >> On 13/01/2016 08:47, wrote: >>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote: >>>> Since TStream doesn't have any native WriteFloat/Double and the likes, >>>> some conversion is needed - but what is a good strategy for this? >> Depends on size constraints. >> In the simple case you should convert float to locale-independent >> string value (no spaces, no commas separators) then write it. >> If the size does matter, you may convert float to corresponding byte >> array then write it. > > Can I ask a naive question here please: does a binary stream store > endianness anywhere? In other words, is there any indication that > something running on ARM is about to get into trouble by reading > something written by (big-endian) MIPS? Not that I'm aware of - i would think of it as a stream of bytes. Endianess is defined by the CPU not by the file.

Thanks, I assumed that was the case but thought it worth checking.

I suppose that one could stream variants, i.e. the enumeration saying what one was followed by a value.

Show quoted text 6 lines
> Should writing binary floating point to a stream note that it's IEEE > format, just in case anybody ever tries to process it on a platform > that supports alternatives? > It would be great if storing floating point could be in IEEE, to have a standard as reference.

I'm not at all sure about this, but I think I've seen something that suggested that byte ordering in external representations was covered by the standard.

Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues]
Marco van de Voort
to Torsten Bonde Christiansen: Hi all. What is the best/preferred way to write floatingpoint to a steam? Since TStream…

In our previous episode, Torsten Bonde Christiansen said:

What is the best/preferred way to write floatingpoint to a steam?
var d : double; // or single, avoid extended
d:=<some expression>;
stream.write(d,sizeof(d));
stream.read(d,sizeof(d));

All the stream helpers are just that.

One could add writedouble/single to the stream typehelpers in streamex.

Torsten Bonde Christiansen
to Mark Morgan Lloyd: Thanks, I assumed that was the case but thought it worth checking. I suppose that one…
Mark Morgan Lloyd wrote 37 lines
Torsten Bonde Christiansen wrote: > On 2016-01-13 10:06, Mark Morgan Lloyd wrote: >> Serguei TARASSOV wrote: >>> On 13/01/2016 08:47, wrote: >>>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote: >>>>> Since TStream doesn't have any native WriteFloat/Double and the >>>>> likes, >>>>> some conversion is needed - but what is a good strategy for this? >>> Depends on size constraints. >>> In the simple case you should convert float to locale-independent >>> string value (no spaces, no commas separators) then write it. >>> If the size does matter, you may convert float to corresponding >>> byte array then write it. >> >> Can I ask a naive question here please: does a binary stream store >> endianness anywhere? In other words, is there any indication that >> something running on ARM is about to get into trouble by reading >> something written by (big-endian) MIPS? > Not that I'm aware of - i would think of it as a stream of bytes. > Endianess is defined by the CPU not by the file. Thanks, I assumed that was the case but thought it worth checking. I suppose that one could stream variants, i.e. the enumeration saying what one was followed by a value. >> Should writing binary floating point to a stream note that it's IEEE >> format, just in case anybody ever tries to process it on a platform >> that supports alternatives? >> > It would be great if storing floating point could be in IEEE, to have > a standard as reference. I'm not at all sure about this, but I think I've seen something that suggested that byte ordering in external representations was covered by the standard.

I'm not even sure how (or even if) TStream decendants handles endianness for integers.... I'm guessing they are not.

Regards,
Torsten.

Sven Barth
to Mark Morgan Lloyd: Thanks, I assumed that was the case but thought it worth checking. I suppose that one…

Am 13.01.2016 10:40 schrieb "Mark Morgan Lloyd" <

Show quoted text 13 lines
: Torsten Bonde Christiansen wrote: > > On 2016-01-13 10:06, Mark Morgan Lloyd wrote: >> >> Serguei TARASSOV wrote: >>> >>> On 13/01/2016 08:47, wrote: >>>> >>>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote: >>>>> >>>>> Since TStream doesn't have any native WriteFloat/Double and the

likes,

>>>>> some conversion is needed - but what is a good strategy for this? >>> >>> Depends on size constraints. >>> In the simple case you should convert float to locale-independent

string value (no spaces, no commas separators) then write it.

>>> If the size does matter, you may convert float to corresponding byte

array then write it.

>> >> >> Can I ask a naive question here please: does a binary stream store

endianness anywhere? In other words, is there any indication that something running on ARM is about to get into trouble by reading something written by (big-endian) MIPS?

> Not that I'm aware of - i would think of it as a stream of bytes.

Endianess is defined by the CPU not by the file.

Show quoted text 5 lines
Thanks, I assumed that was the case but thought it worth checking. I suppose that one could stream variants, i.e. the enumeration saying

what one was followed by a value.

That's not the task of TStream however. For that you should use a custom class that uses TStream under the hood.

>> Should writing binary floating point to a stream note that it's IEEE

format, just in case anybody ever tries to process it on a platform that supports alternatives?

>> > It would be great if storing floating point could be in IEEE, to have a

standard as reference.

I'm not at all sure about this, but I think I've seen something that

suggested that byte ordering in external representations was covered by the standard.

I've checked recently due to big endian problems in the Castle game engine and found out that endianess isn't really covered for floating points...

Regards,
Sven

Sven Barth
to Torsten Bonde Christiansen: I'm not even sure how (or even if) TStream decendants handles endianness for integers....…
"Torsten Bonde Christiansen" wrote 14 lines
On 2016-01-13 10:40, Mark Morgan Lloyd wrote: > > Torsten Bonde Christiansen wrote: >> >> On 2016-01-13 10:06, Mark Morgan Lloyd wrote: >>> >>> Serguei TARASSOV wrote: >>>> >>>> On 13/01/2016 08:47, wrote: >>>>> >>>>> On 2016-01-12 10:45, Torsten Bonde Christiansen wrote: >>>>>> >>>>>> Since TStream doesn't have any native WriteFloat/Double and the

likes,

>>>>>> some conversion is needed - but what is a good strategy for this? >>>> >>>> Depends on size constraints. >>>> In the simple case you should convert float to locale-independent

string value (no spaces, no commas separators) then write it.

>>>> If the size does matter, you may convert float to corresponding byte

array then write it.

>>> >>> >>> Can I ask a naive question here please: does a binary stream store

endianness anywhere? In other words, is there any indication that something running on ARM is about to get into trouble by reading something written by (big-endian) MIPS?

> > >> Not that I'm aware of - i would think of it as a stream of bytes.

Endianess is defined by the CPU not by the file.

Show quoted text 5 lines
> > > Thanks, I assumed that was the case but thought it worth checking. > > I suppose that one could stream variants, i.e. the enumeration saying

what one was followed by a value.

> >>> Should writing binary floating point to a stream note that it's IEEE

format, just in case anybody ever tries to process it on a platform that supports alternatives?

>>> >> It would be great if storing floating point could be in IEEE, to have a

standard as reference.

> > > I'm not at all sure about this, but I think I've seen something that

suggested that byte ordering in external representations was covered by the standard.

I'm not even sure how (or even if) TStream decendants handles endianness

for integers.... I'm guessing they are not.

TStream always read in its native order. If you know that the data is of the opposite endianess you need to convert this yourself as TStream can't know that.

Regards,
Sven

Marco van de Voort
to Torsten Bonde Christiansen: I'm not even sure how (or even if) TStream decendants handles endianness for integers....…

In our previous episode, Torsten Bonde Christiansen said:

I'm not even sure how (or even if) TStream decendants handles endianness for integers.... I'm guessing they are not.

TStream is basically an abstraction for stream of bytes. The read<type> methods are just convenience, not some protocol.

As far as endianess go, see the streamex unit's class helper for TStream, and its use (here and there) in package chm.

Dennis
to Sven Barth: Am 13.01.2016 10:40 schrieb "Mark Morgan Lloyd" < string value (no spaces, no commas…
Sven Barth wrote 21 lines
>>> Should writing binary floating point to a stream note that it's IEEE format, just in case anybody ever tries to process it on a platform that supports alternatives? >>> >> It would be great if storing floating point could be in IEEE, to have a standard as reference. > > > I'm not at all sure about this, but I think I've seen something that suggested that byte ordering in external representations was covered by the standard. I've checked recently due to big endian problems in the Castle game engine and found out that endianess isn't really covered for floating points... Regards, Sven

Do you imply that, unlike floating point, endianess for integers are handled automatically by TStream?

Dennis

Sven Barth
to Dennis
"Dennis" wrote 6 lines
Sven Barth wrote: > > >>> Should writing binary floating point to a stream note that it's IEEE

format, just in case anybody ever tries to process it on a platform that supports alternatives?

> > >>> > >> It would be great if storing floating point could be in IEEE, to have

a standard as reference.

> > > > > > I'm not at all sure about this, but I think I've seen something that

suggested that byte ordering in external representations was covered by the standard.

> > I've checked recently due to big endian problems in the Castle game

engine and found out that endianess isn't really covered for floating points...

Show quoted text 6 lines
> > Regards, > Sven > > Do you imply that, unlike floating point, endianess for integers are

handled automatically by TStream?

Not at all, but it could have been that IEEE floating points are defined to be always little endian for example.

Regards,
Sven