Hi everyone,

So to start the story, I'm planning to make use of the nodes that were introduced for the SSE and AVX intrinstics as part of some vectorisation code, since they can help manage the code generation and contain some sanity checks.  I noticed though that some intrinsics are missing; for example, the VMASKMOV instructions, which were introduced with AVX and don't have a direct SSE equivalent (MASKMOVDQU and MASKMOVQ can only write to memory and work on integers rather than floating-point values, and mixing MM integer and floating-point instructions incur a CPU state switch penalty).

Instructions like VMASKMOV are very useful because it would allow vectorisation of 3-component arrays (e.g. Cartesean coordinates), so I plan to look at introducing nodes for these instructions. Would this be okay to do?

On another note, I'm wondering if the RTL could benefit from some initializers for the MM types for ease of use.  For example, one of the registers used in VMASKMOV is a mask, and for a programmer using intrinstics, being able to do something like "mmval := x86_vmaskmovps(Coord, [True, True, True, False]);" - granted I have to think about performance since all those Boolean consrants should ideally be merged into single 128-bit memory block (as
FFFFFFFFFFFFFFFFFFFFFFFF00000000) that is loaded into an XMM register with VMOVPS.

What would you suggest? I'm just speaking a bit from experience in that using C++ intrinsics can get a little cumbersome sometimes and easy to get wrong (at least as far as performance and alignment are concerned, for example), and having the FPC ones be friendlier would make a world of difference.

I'm still working out quite a few things and experimenting a lot.  I'll be sure to be doing a lot of documentation.  However, any help or insight into the current design practices for intrinstics and their respective nodes will be greatly appreciated.

Gareth aka. Kit

P.S. Regarding vectorisation challenges, I'm looking at sequences like "V.X*V.X + V.Y*V.Y + V.Z*V.Z" (scalar length of a 3-dimensional vector), which I would love to be able to naturally compile into:

VMOVPS XMM1, Mask_1110
VMASKMOVPS XMM0, XMM1, V
VMULPS XMM0, XMM0, XMM0
VHADDPS XMM0, XMM0, XMM0
VHADDPS XMM0, XMM0, XMM0

And then maybe take it further to produce:

VMOVPS XMM1, Mask_1110
VMASKMOVPS XMM0, XMM1, V
VDPPS XMM0, XMM0, $71 { 01110001b }

(This could be an optimisation at the node level rather than a peephole optimisation, although if it doesn't know exactly what VMASKMOVPS is doing, then the immediate in (V)DPPS will be forced to be $FF)

Hi

I think another related idea is adding some "attribute" to dynamic arrays types to force them to be aligned to 32/64 bytes. Precisely for this purpose, I am forced to maintain my own FPC fork.

-- Regards,
Denis Golovan

I did suggest it once - https://bugs.freepascal.org/view.php?id=32780 - it would make things useful and allow for cleaner typecasting with the internal __m128 type etc.  It was assigned, but appears to have stalled. There might be some portability issues with it, although I think the feature already exists in the form of pre-processor directives, but those are a little cumbersome sometimes.

Gareth aka Kit.

I did suggest it once - https://bugs.freepascal.org/view.php?id=32780 - it would make things useful and allow for cleaner typecasting with the internal __m128 type etc. It was assigned, but appears to have stalled.

Yeap. That's right it's a long awaited feature.

There might be some portability issues with it, although I think the feature already exists in the form of pre-processor directives, but those are a little cumbersome sometimes.

Could you point me to this solution?

-- Regards,
Denis Golovan

From tests/test/cg/tvectorcall1.pp

{$PUSH}
{$CODEALIGN RECORDMIN=16}
{$PACKRECORDS C}
type
   TM128 = record
     case Byte of
       0: (M128_F32: array[0..3] of Single);
       1: (M128_F64: array[0..1] of Double);
   end;
{$POP}

(This declaration mimics how __m128 is defined in Microsoft Visual C++)

Gareth aka. Kit

And even this solution does not support dynamic arrays with scalars, right?

{$PUSH}
{$CODEALIGN RECORDMIN=32}
{$PACKRECORDS C}
type
   TIntArray = array of Integer;
   {$POP}
var v:TIntArray;
begin
   SetLength(v,100);
   Writeln(NativeInt(Pointer(v)) and 31); // writes 16 instead of 0
end.

-- Regards,
Denis Golovan

In that instance, I think it's because TIntArray is not a record type (it's essentially a smart pointer) and so RECORDMIN doesn't apply to it.  I do agree though that having an 'align' attribute of some kind will make life a lot easier since you don't have to play around with multiple preprocessor directives that are very hard to get correct.

Gareth aka. Kit

One thing to be slightly aware of with
dynamic pointers and alignment. For V: atray
of Integer etc, make sure you check the
alignment of V[0], not V. V points to an
internal structure that is largely opaque,
while V[0] points to the actual data, which I
imagine is what you want to pass into VMOVAPS
etc.

Gareth aka. Kit

On Tue 21/04/20 11:17 , "denisgolovan"
sent:

TIntArray

is not a record type > (it's essentially a smart pointer) and so RECORDMIN doesn't apply to > it. I do agree though that having an

'align'

attribute of some kind > will make life a lot easier since you

don't have

to play around with > multiple preprocessor directives that are

very

not enough for practical

Exactly.
That's why I think current functionality is not enough for practical purposes.

What do other developers think about that?

-- Regards,
Denis Golovan

I believe your are wrong here.
For dynamic arrays, service data is located at negative offset and pointer itself _is_ zero element address.

type
   TIntArray = array of Integer;
var v:TIntArray;
begin
   SetLength(v,100);
   Writeln(NativeInt(Pointer(v)) = NativeInt(Pointer(@v[0]))); // always gives true
end.

-- Regards,
Denis Golovan

The value stored at V is the pointer to V[0],
not the zeroth element's data, hence why your
comparison always returns true. In assembly
language, you'd have to effectively do:

MOV RDX, [RSP] { or wherever V's service data
is stored }
VMOVAPS YMM0, [RDX]

Passing RSP (V) into VMOVAPS directly will
load garbage data and possibly cause a buffer
overrun.

The reason why it is a pointer to a pointer
is because the variable has to have a
constant address, but the data may have to be
reallocated to a new location if the dynamic
array is lengthened.

Sorry for getting off topic a bit. I do a
agree that there should be a way to enforce
the alignment of the data pointer. It may
require versions of ReallocMem etc. where you
can enforce the alignment without having to
do any dangerous pointer arithmetic.

Gareth aka. Kit

On Tue 21/04/20 13:58 , "denisgolovan"
sent:

> I think Gareth meant the address of V

instead

pointer to pointer :)

BTW, Sven, what's your opinion on dynamic

arrays alignment problem?

I don't have any viable solution besides

maintaining my own compiler fork

so far. LLVM backend could also help with

autovectorization, but it would require

denisgolovan via fpc-devel schrieb am Di.,
21. Apr. 2020, 13:32:

I think Gareth meant the address of V instead (Pointer(@V)).

Regards,
Sven

I think Gareth meant the address of V instead (Pointer(@V)). Regards, Sven

Hm. I have no idea why anyone would take pointer to pointer :)

BTW, Sven, what's your opinion on dynamic arrays alignment problem?

I don't have any viable solution besides maintaining my own compiler fork so far. LLVM backend could also help with autovectorization, but it would require proper alignment as well.

-- Regards,
Denis Golovan

denisgolovan schrieb am Di., 21. Apr. 2020, 14:58:

Because they might have forgotten that dynamic arrays behave different from static ones?

BTW, Sven, what's your opinion on dynamic arrays alignment problem?

I have none, cause I'm not dealing with such problems.

Regards,
Sven

I made a bit of a big post with "Future development plans" where I mentioned 'aligned allocation', especially since C11 has the feature and Microsoft have their own non-portable set of functions.  Hopefully it will spawn some discussion.

Sorry to keep going back to the dynamic array issue.  The comparison with static arrays is a good one, because this is where bugs often appear.

Take the common Move function for some array V (assume it's an array of Integers for the sake of this example) and a pointer to a block of memory:

Move(V, DestPtr^, Length(V) * SizeOf(Integer)); { V and DestPtr can be swapped around if you like }

If V is a static array, this will work as expected, since V points to the first element in the array.  However, if V is a dynamic array, then this call will likely raise an access violation, or if it doesn't, DestPtr^ will contain garbage - if you're writing to V, you'll end up giving it an invalid pointer to its data, while the real pointer will end up dangling.  You have to specify V[0] rather than V for it to work as expected (which works for static arrays too).

Gareth aka. Kit

Hi Florian,

As I mentioned in my future development post,
do you think FPC could benefit from
GetMemAligned and ReallocMemAligned routines?
The internal functions could be used to help
control such custom alignment in dynamic
arrays.

We're still waiting on a good language
feature to better control alignment - e.g.
https://bugs.freepascal.org/view.php?id=32780

Gareth aka. Kit

On Tue 21/04/20 19:59 , Florian Klämpfl
sent:

Am 21.04.20 um 14:58 schrieb denisgolovan

via fpc-devel:

pointer

to pointer :) > > BTW, Sven, what's your opinion on dynamic

arrays

alignment problem? It should be fixed/there should be a

possibility to get them aligned

properly. To be honest though, I did not

look into it yet.

It should be fixed/there should be a possibility to get them aligned properly. To be honest though, I did not look into it yet.

just as point for consideration, I'm not sure if data alignment will improve speed on future processors:
https://lemire.me/blog/2012/05/31/data-alignment-for-speed-myth-or-reality/

Food for thought: imagine if we had single (32 bits floating point) values dynamic arrays with 1 million values each: a b and c. I would love to have something like this:
a := b + c;

I have the following code in a unit (I need to provide a memset function to be able to link to freertos):

function memset(pxBuffer:pointer; value : byte; count : Tsize):pointer;
cdecl;
begin
   FillChar(pxBuffer,count,value);
   Result := pxBuffer;
end;

When I look at the assembler code generated by latest Trunk fpc something is very odd @08000970, I'd expect to see

LDR           R0, [R11, #-0x30]

but in fact I see:

 08000970   SUB.W          R0, R11, #0x30

which makes no sense to me and later the code crashes inside of the FillChar routine.

Compiler Bug? Or me overlooking something? I tried to compile with -O- and -O1, with -O2 the generated code is a little different but there the SP is loaded to R0 which I do not understand, I'd expect R0 to be set to address of pxBuffer. (this is for armv7em, for armv6 also looks odd.

Thank you for your help,

Michael

FREERTOS_$$_MEMSET$POINTER$BYTE$LONGWORD$$POINTER
DEBUGSTART_$FREERTOS
memset
$Thumb
begin
 08000958   MOV            R12, SP
 0800095A   PUSH.W         {R11, LR}
 0800095E   SUB.W          R11, R12, #4  08000962   SUB            SP, SP, #0x48  08000964   STR            R0, [R11, #-0x30]  08000968   STRB           R1, [R11, #-0x34]  0800096C   STR            R2, [R11, #-0x38] FillChar(pxBuffer,count,value);
 08000970   SUB.W          R0, R11, #0x30  08000974   LDRB           R2, [R11, #-0x34]  08000978   LDR            R1, [R11, #-0x38]  0800097C   BL             SYSTEM_$$_FILLCHAR$formal$LONGINT$BYTE ; 0x080002
Result := pxBuffer;
 08000980   LDR            R0, [R11, #-0x30]  08000984   STR            R0, [R11, #-0x3C]

Can you run gdb on there? When you break inside memset does the pxBuffer pointer make sense (not 0x0)? Do the other two variables come in correctly?

Do you call memset or does an api in the OS call it? Perhaps it's a pointer to a pointer on accident?

If you are required to provide memset are you also providing other memory functions, for example to allocate? Is it passing a valid pointer back (one that eventually gets to memset)?

Just a bit of a brainstorm.. sorry if I've oversimplified or missed the mark entirely :)

The information you see is coming straight out of the debugger (Segger Ozone),

and perhaps I should have said this in the first post,

the pointer that is fed in the Fillchar procedure is complete nonsense, it points into the stack area which means im my case that fillchar overwrites the stack and at some point writes into non-RAM which crashes the cpu.

Value of R0 inside of Fillchar: 20017ef4 (Stack Area at the end of RAM)

Value of R0 on entry: 200000f8 which looks OK as it is close to the start of RAM where it got statically allocated:

implementation
var
   xIdleTaskTCB : TStaticTask;
   uxIdleTaskStack : array[0..configMINIMAL_STACK_SIZE-1] of TStackType;
   xTimerTaskTCB : TStaticTask;
   uxTimerTaskStack : array[0..configTIMER_TASK_STACK_DEPTH-1] of
TStackType;

The only other thing than a compiler bug could be that my Implementation of the memset wrapper is wrong, the prototype is:

void * memset ( void * ptr, int value, size_t num );

and both h2pas and me came to the same implementation:

function memset(pxBuffer:pointer; value : uint32; count : Tsize):pointer; cdecl;

You can ignore the fact that I get an uint32 passed in and Fillchar uses a byte, only the byte part is relevant:

 From C Documentation of memset:
      void * memset ( void * ptr, int value, size_t num );

        Fill block of memory
        Sets the first /num/ bytes of the block of memory pointed by /ptr/ to the specified /value/ (interpreted as an unsigned char).

I have also hacked the code of my memset function in a way that I think that it should work:

begin
  asm
    LDR     R0, [R11, #-0x30]
    LDRB    R2, [R11, #-0x34]
    LDR     R1, [R11, #-0x38]
    bl fillchar
  end;
  Result := pxBuffer;
end;

Implemented this way the function works as expected.

Michael

You need pxBuffer^ instead of pxBuffer (you want to fill the block to which pxBuffer points, not the 4 bytes of pxBuffer itself).

Jonas

On 2020-04-21 20:47, J. Gareth Moreton wrote:

Hi,

Wouldn't it be better to use either some kind of global property or an optional additional argument than completely new routines requiring explicit handling not only from the end-user programs, but also from all units which may in turn be involved in memory allocation (including various containers etc.)?

Tomas

Possibly, but it can become wasteful, especially on systems that might be strapped for memory.

Generally, you only want a coarser alignment for specialised applications, such as for certain instruction sets (e.g. AVX) and memory paging.  The mechanics behind the regular GetMem is more than adequate for most applications.  True, it might be useful to have a global alignment parameter so you don't have to develop new classes, but I think most programmers would be willing to develop their own frameworks if a coarse memory alignment is required.

It's also following on from C11 and C++17 that introduced an aligned allocation into their standard libraries.

However, I will say that having an additional optional parameter to the existing GetMem and ReallocMem routines is definitely a nicer idea than brand new functions.

Gareth aka. it

I think so too.

What's more, I thought the memory allocation routines already guarantee some form of alignment, simply because of the internal structures used by the memory manager ?

Michael.

AAAAArrrrrrrrrggggghhhhhhhh.......

I feel completely stupid!!!!!

Thank you for the Hint Jonas, using pxBuffer^ did the trick!

Michael

same.. lol

Whether they be new functions or just an optional parameter, I do agree that they should be compatible with FreeMem rather than requiring their own function - Microsoft's non-portable _aligned_malloc requires calling a specialised 'free' function, but the C11 and C++17 standards, despite having new functions, can use 'free' without any problems.

My impression is that the alignment of GetMem is implementation-specific and is fine for most applications.  On x86_64 at least, it seems to be at least 16 bytes, but there are some cases where you want 32 bytes (256-bit AVX), 64 bytes (AVX512) or 4,096 bytes (memory paging).

Gareth aka. Kit

I'm not totally certain what the design intentions are with new language features - back in the 90s, a friend told me Pascal (actually Delphi 2.0) has the simplicity of Visual Basic but the power of Visual C++.  I'm not sure how true that is, but it's what's inspired me.  Going by that, I would like new or adapted features to be relatively easy to use while still being efficient.

A global option, like a preprocessor directive or command line option, feels a bit dangerous and can easily introduce unexpected bugs if it's left out, so an explicit optional parameter (e.g. GetMem(AlignedVector, 4096, 64);) would feel like the way to go.

Gareth aka. Kit

P.S. The only exception to an 'easy to use' feature in my eyes is inline assembly, which I've always seen as "it's there if you want to use it".

On 2020-04-22 15:45, J. Gareth Moreton wrote:
.
.

A global option, like a preprocessor directive or command line option, feels a bit dangerous and can easily introduce unexpected bugs if it's left out, so an explicit optional parameter (e.g. GetMem(AlignedVector, 4096, 64);) would feel like the way to go.

.
.

I didn't mean a preprocessor directive or a command line option (that wouldn't work for previously compiled units anyway). I rather meant a property inside unit system (probably within the memory manager structure) that would allow setting the required behaviour. But I don't know if it's a better option than a new optional parameter for the respective routines, it was just an idea towards reducing the impact to existing code and possibly also the amount of knowledge required for using this feature.

Tomas

A global property inside the System unit I don't think would be wise because anything could set it and it wouldn't be threadsafe (for a single-thread example, two different units might set it to what they require, but one may get its value overwritten and have a smaller alignment than asked - you could have a procedure that sets it to the largest alignment requested, but if you have one unit that asks for 32 bytes and another that asks for 4,096 bytes, you're going to get a lot of wasted memory).  At least for C11 and C++17, you get an error if you specify a size that is not a multiple of the alignment (C11 originally specified 'undefined behaviour', but this was later corrected), but personally I think it would be better to silently round the size up (assuming it is impossible, non-portable or prohibitively slow to reserve the exact size asked).

I am personally leaning towards an optional parameter for GetMem and ReallocMem because that would show up in the code completer, thus informing users that it exists, and a little easier to document too.  Also, because you're specifying the alignment at the moment you are allocating a block of memory, there won't be any problems with global variables being overwritten.  I do have some unanswered questions though, like what happens if you reserve aligned memory and then you pass it through ReallocMem without the alignment setting, and vice versa (reserve unaligned memory, then reallocate it with an alignment, especially a smaller size which doesn't usually require relocation) - should the compiler attempt to honour the original or new alignment, or throw an exception for mixing aligned and unaligned allocations this way (that would likely require storing extra information in the memory manager)?

Most objects don't need specialised alignment, although Denis Golovan earlier in this e-mail thread does want aligned dynamic arrays (e.g. "VectorArray: array of Double align 32;"), where the variable VectorArray itself might not be aligned (it might be on the stack), but VectorArray[0] is aligned to the required alignment - such a feature could reuse, and would greatly benefit from, an aligned GetMem's internal functionality.  For one thing, it means you can use standard Pascal constructs for such specialised tasks instead of having to play around with pointers.

Gareth aka. Kit