Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2017-11

Float packing

6 messages · 3 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Ryan Joseph

I’m trying to pass a value between 1-1000 to an GLSL shader program which only supports floats with a precision of 1 decimal place so the plan is to pack the value into a vec4 (RGBA components between 0-1) which I could decode in the shader program.

For example if I want to encode the value 50 I do: 50/1000 = 0.050 and get a value with 3 decimal places which could fit into an RGB value,

but how can I extract the decimal places? I think it has something to do bit shifting but I wasn’t able to figure it out. The final result should be:

r = 0
g = 5
b = 0

Regards,
Ryan Joseph

Ryan Joseph
to Ryan Joseph

I actually got the problem wrong and still haven’t figured it out so here’s another go.

I have a number which I need to encode as an RGB value each component being 4 bytes (12 bytes total) so I need to extract each digit as a byte (and then each byte back to a digit). It’s basic low level computer science stuff I never learned well. The old version of GLSL I’m using (1.2) doesn’t support bit shifting (which I think I could use) so I need to do this using arithmetic.

On Google I found a good method to have the compiler convert a word to an array of bytes using variant records such as:

type
	TBytes = record
		case byte of
			0: (i : word);
			1: (u : array[0..3] of byte);
			2: (r, g, b, a: byte);
		end;

so for the number 1234:

var
 c: TBytes;
FillChar(c, sizeof(c), 0);
c.i := 300;
writeln(c.r, ' ', c.g, ' ', c.b, ' ', c.a);

I get: 44 1 0 0

How do I convert this back to 300 using arithmetic? I see you can add 255 to first value plus the next value (1) to get 300 but what’s the correct method to use I could make into a function?

Show quoted text 19 lines
On Nov 5, 2017, at 4:10 PM, Ryan Joseph wrote: I’m trying to pass a value between 1-1000 to an GLSL shader program which only supports floats with a precision of 1 decimal place so the plan is to pack the value into a vec4 (RGBA components between 0-1) which I could decode in the shader program. For example if I want to encode the value 50 I do: 50/1000 = 0.050 and get a value with 3 decimal places which could fit into an RGB value, but how can I extract the decimal places? I think it has something to do bit shifting but I wasn’t able to figure it out. The final result should be: r = 0 g = 5 b = 0 Regards, Ryan Joseph _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Regards,
Ryan Joseph

Ryan Joseph
to Ryan Joseph
On Nov 6, 2017, at 10:43 AM, Ryan Joseph wrote: I have a number which I need to encode as an RGB value each component being 4 bytes (12 bytes total)

I don’t know why I typed this, each component is 1 byte obviously so each digit can be up to 255 for a total value of 255,255,255? Not sure if i’m getting this though.

Regards,
Ryan Joseph

Karoly Balogh (Charlie/SGR
to Ryan Joseph: I actually got the problem wrong and still haven’t figured it out so here’s another…

Hi,

Ryan Joseph wrote 12 lines
I actually got the problem wrong and still haven’t figured it out so here’s another go. I have a number which I need to encode as an RGB value each component being 4 bytes (12 bytes total) so I need to extract each digit as a byte (and then each byte back to a digit). It’s basic low level computer science stuff I never learned well. The old version of GLSL I’m using (1.2) doesn’t support bit shifting (which I think I could use) so I need to do this using arithmetic. On Google I found a good method to have the compiler convert a word to an array of bytes using variant records such as:

I'm not particularly fond of this method, because this is not endian safe. It won't work on bigendian CPUs (which are marginal, and a thing of the past, sort of, but their heritage is still with us thanks to the network byte order, which is bigendian, and some languages like Java, which use bigendian internally).

Show quoted text 18 lines
type TBytes = record case byte of 0: (i : word); 1: (u : array[0..3] of byte); 2: (r, g, b, a: byte); end; so for the number 1234: var c: TBytes; FillChar(c, sizeof(c), 0); c.i := 300; writeln(c.r, ' ', c.g, ' ', c.b, ' ', c.a); I get: 44 1 0 0

A Word in Pascal is always two bytes. So I don't understand what you want here.

How do I convert this back to 300 using arithmetic? I see you can add 255 to first value plus the next value (1) to get 300 but what’s the correct method to use I could make into a function?

You're seeing the bytes of a two byte "word" value in little endian, which is the endianness of x86 and ARM CPUs by default. If you need to process the data byte by byte, you need to swap the order of bytes around.

In your example:

writeln((c.g shl 8) + c.r);
should print "300";

If you change "i" to dword, For a 32bit value this would be:

writeln((c.a shl 24) + (c.b shl 16) + (c.g shl 8) + c.a;

But you cannot apply this method to floats, such byte-magic only works on integers (well, you need to also swap around bytes of a float if you process them as bytes, but that's beside the point here). You also cannot extract the 10-base digits of an integer value like this (unless we're talking about BCD numbers, which is again, an entirely different topic).

I still not entirely understand what you want to achieve, but to extract the digits of a fractional number, you could simply do the following:

var
  a: single;
begin
  a:=1234.5678;
  // integer part
  writeln(trunc(a) mod 10000 div 1000);
  writeln(trunc(a) mod 1000 div 100);
  writeln(trunc(a) mod 100 div 10);
  writeln(trunc(a) mod 10 div 1);
  // fractional part
  writeln(trunc(a*10) mod 10);
  writeln(trunc(a*100) mod 10);
  writeln(trunc(a*1000) mod 10);
  writeln(trunc(a*10000) mod 10);
end.

But anyway, I guess this is not so useful for you any more. So if you can tell what *actually* you want to do, like convert a vec4 of GLSL float to Pascal, or convert a 32bit RGBA value to a GLSL vec4, maybe that would be more easy to help you with.

Also, are you trying to generate GLSL source with this, or you're processing binary data returned by your shader code in a buffer?

Charlie

Marco van de Voort
to Karoly Balogh (Charlie/SGR

In our previous episode, Karoly Balogh (Charlie/SGR) said:

Show quoted text 9 lines
> > On Google I found a good method to have the compiler convert a word to > an array of bytes using variant records such as: I'm not particularly fond of this method, because this is not endian safe. It won't work on bigendian CPUs (which are marginal, and a thing of the past, sort of, but their heritage is still with us thanks to the network byte order, which is bigendian, and some languages like Java, which use bigendian internally).

I thought that OpenGL had a normalize paramter to glvertexattribpointer that allowed to convert color values to vec4 (of singles, maybe vec3 too) ?

Or is it an uniform?

Ryan Joseph
to Karoly Balogh (Charlie/SGR: I'm not particularly fond of this method, because this is not endian safe. It won't work…
Show quoted text 6 lines
On Nov 6, 2017, at 7:36 PM, Karoly Balogh (Charlie/SGR) wrote: But anyway, I guess this is not so useful for you any more. So if you can tell what *actually* you want to do, like convert a vec4 of GLSL float to Pascal, or convert a 32bit RGBA value to a GLSL vec4, maybe that would be more easy to help you with.

I need to encode an integer (2 byte word is actually enough) into the pixels of a 2D texture where each component is a byte (0-255). In the shader I then decode that value from the pixel (RGB value) of the texture. It’s for sorting value in a 2D game

I guess I need to pay attention to byte swapping (not a big problem right?) but your example writeln((c.a shl 24) + (c.b shl 16) + (c.g shl 8) + c.a; is basically what I needed, however I’m using an older version of GLSL which doesn’t have bit shift operators.

Another user contacted me privately and suggested this method using just arithmetic. From the vec3 I translate the fraction back to 0-255 then basically do what you did with bit shifting. I can kind of see how this is shifting by 256 each time which is the size of a byte and therefore bit shifting.

float unpack (vec3 c) {
	c.r = c.r*255;
	c.g = c.g*255;
	c.b = c.b*255;
	return floor(c.r + c.g * 256 + c.b * 256 * 256);
}

I think I got this solved but i need to implement it fully.

Regards,
Ryan Joseph