Hi,

I'm have some tough times with arrays (I use them very little). With trial and error I found that to use a pointer to an array of int32 values returned by a C API (Xlib to be exact), I have to define it as follows in Object Pascal.

type
  TAtomArray = array[0..0] of TAtom;
  PAtomArray = ^TAtomArray;
var
  xdndtypes: PAtomArray;
  a: TAtom;  // just a culong type
begin
   ...
    XGetWindowProperty(Display, FSrcWinHandle,
        XdndTypeList, 0, 16000,
        TBool(False),
        AnyPropertyType,
        @actualtype, @actualformat, @count, @remaining,
        @xdndtypes);
    // 'count' tells me the amount of items in the returned xdndtypes array
    // so I can access the array items as follows
    a := xdndtypes^[i]
   ...
   XFree(xdndtypes);
end;

Now if I change TAtomArray to the follow, then my code doesn't work. :-)

  TAtomArray = array of TAtom;   // a dynamic array

So what exactly is the difference between these two?

    TAtomArray = array[0..0] of TAtom;
 vs
    TAtomArray = array of TAtom;

Is array[0..0] not actually a dynamic array, but just a static array with a single item? If so, then why does the returned value from the C API call, which returns a pointer to an array of culong's (normally more that one item) work? Somewhere I'm getting confused with all this, but would love to understand it correctly, and why it works. :)

From the XGetWindowProperty() man page, it says the the following

about the return type which get stored in xdndtypes value (the last parameter in the declaration below).

int XGetWindowProperty(Display *display, Window w, Atom property, long long_offset, long
long_length, Bool delete, Atom req_type, Atom
*actual_type_return, int *actual_for‐
mat_return, unsigned long *nitems_return, unsigned long *bytes_after_return,

              unsigned char **prop_return);

"If the returned format is 32, the property data will be stored as an array of longs (which in a 64-bit application will be 64-bit values that are padded in the upper 4 bytes)."

2010/9/22 Graeme Geldenhuys :

In brief AFAIK:

type
  T1 = array[l..h] of T3;
  T2 = array of T3;

says that T1 is a value type and T2 is a pointer type (ptr casted deref of a T2 typed variable may(? depends on implementation) safely alias a T1 typed entity, but a pointer to a first element of a T1 value can't be safely aliased to a T2 typed entity). Additionally dynamic arrays are a compiler handled structures, the differences are similar to those between shortstrings and AnsiStrings types.

unsigned char **prop_return

you can actually define it as:

type
  PAtom = ^TAtom;
  PPAtom = ^PAtom;

and pass a PPAtom variable instead.

Hi Graeme!

Yes.

If so, then why does the returned value from the C API call, which returns a pointer to an array of culong's (normally more that one item) work? Somewhere I'm getting confused with all this, but would love to understand it correctly, and why it works. :)

I assume you are running into an often made error when switching from static array to dynamic array that is used as a buffer and used with commands that expect a pointer to the buffer.

For a static array the compiler knows the size of the array since it can't be changed during runtime so it doesn't need to store the array length in an internal value. So a pointer to the static array points to the first element of the array. But a dynamic array need to remember its actual size. So the internal data structure of a dynamic array will look like TDynIntegerArray = record

  ArraySize: Integer;
  Array: ^Integer;

end;
The real array buffer can be at a complete different place in the memory. So if you now send a pointer of TDynIntegerArray to a function that simply returns an array it will set the ArraySize to the value of the first array element. But the much more problematic part will be that if the array is bigger than SizeOf(TDynIntegerArray) it will overwrite memory that follows TDynIntegerArray and can lead to a memory access violation.

So what you need to do is to send the address of the first array element, in your case @xdndtypes[0].

But I don't know if this works well with an empty dynamic array. Especially since you can only set the array size by SetLength(). If in your case you do SetLength(xdndtypes, count) after the call I would expect that it will allocate memory for the array since the new size is greater than the old size (which is 0 for an unused dyn. array) and will overwrite the pointer of your function call array. Another problem is often to free memory in Pascal that was allocated by C.

The only case when I have used a dynamic array (or in the follow example a string which is a kind of dyn. char array) for a function call was when I set the array to a maximum size before the call, specify the maximum size in the call and reduced the array to the actual size after the call like:
Filename: String;
UsedLength: Integer;
SetLength(Filename, MAXPATH);
GetWhateverFilenameFromTheSystem(@Filename[1], MAXPATH, @UsedLength); // String start at index 1;
SetLength(Filename, UsedLength); // or UsedLength - 1 when UsedLength includes a #0 as the char. array end indicator

So when the array memory is allocated by the calling function and freed by another function and you only what to be able to access the elements using the array braces you should use the static array approach. But I would define the static array type as follows TAtomArray = array[0..MaxInt] of TAtom; // or whatever is the allowed maximum index to avoid that Pascal will raise a range checking error when you activate range checking.

To prove my explanation you can simply print the memory addresses of @xdndtypes and @xdndtypes[0] for a static array and a (initialized?) dynamic array and you'll get the same addresses for the static array and different addresses for the dynamic.

Regards

Michael

Another problem is often to free memory in Pascal that was allocated by C.

Having done some interaction with C libs IMHO the best option is not to free the memory in pascal at all but doing it with the provided C functions (if the lib does not do the freeing automatic, that is).

In cases like Graemes I always used leledumbos approach which never failed me.

type
  PAtom = ^TAtom;
  PPAtom = ^PAtom;

This gives you access to the array but the memory management is done entirely by the lib/C-code.

hih
R.

Hi Michael,

Ha, I think a light bulb just came on! :-)

So let me see if I got this right. I defined TAtomArray as a static array with a single element. In static arrays, all elements follow each other in sequence in memory. I only know the length of the static array after the C API call, but because I know the memory address of the first element in the array, I can access the other elements too. And because I currently have range-checking disabled, Pascal doesn't complain that I am going over the bounds of my TAtomArray definition.

If I got that right, I should then also know how to pass such an array to the C API (which was going to be my next question). The easiest would be to create a static array with a largish size in Pascal (for DND mime types, 30-50 elements should be plenty), then pass the address of element 0 to the C API, and another parameter that specifies the length of the array (only elements with actual data).

Once again, thank you very much for taking the time to explain things in such detail. I really helps a lot. As for the differences between static and dynamic arrays, and a string array - i think I'll annotate my FPC Language Reference docs with what you mentioned, or see if Michael van Canneyt could add that info into the official docs so others can benefit too.

Out of interest. Do you know if dynamic array elements are in sequence (storage area in memory) too - like static arrays? So could I do the same as above, but instead of using a static array, use a dynamic array, and pass the address of the first element to the C API?

As for freeing the array that the C API returned to me... I don't do the freeing, I simply notify the C library when I am done using the array, and the library does the freeing for me.

To prove my explanation you can simply print the memory addresses of @xdndtypes and @xdndtypes[0] for a static array and a (initialized?) dynamic array and you'll get the same addresses for the static array and different addresses for the dynamic.

Indeed, this is what I got. :-)

What do you want added ? I haven't seen anything which isn't already mentioned somehow in the docs ?

Yes.

No. This you cannot do, since the pascal memory manager allocated the array, it must also free it. The C memory manager has no idea about this 'memory'. You must manually do a SetLength(myarray,0);

Michael.

2010/9/23 Graeme Geldenhuys :

Out of interest. Do you know if dynamic array elements are in sequence (storage area in memory) too - like static arrays? So could I do the same as above, but instead of using a static array, use a dynamic array, and pass the address of the first element to the C API?

AFAIK yes as long as you don't break (it's possible using some wild casting) the ref counting mechanism and handle the zero length case.

var
  V1: array [L..H] of T;
  V2: array of T;
magic_c_function(@V1[L]);
if Length(V2)  0 then
  magic_c_function(@V2[0])
else
  magic_c_function(nil);

I'll re-read the docs covering array and dynamic arrays just to make sure. What I think is missing, is the "gotcha" usages of arrays and pointers to various array types. eg:

* Pointer to first array element, using static array can be: @myarray or @myarray[0]
(if your low bound array elements start at 0)

* Pointer to first array element, using dynamic array can be @mydynarray[0] (if your low bound array elements start at 0)
... *but* it cannot be: @mydynarray

* A pointer to a string (a string is actually a byte array) should be: @mystring[1]
... and *not* the following: @mystring or @mystring[0]

* array elements (in static or dynamic arrays) are in sequence in memory. Handy to
know if your code interacts with other languages like C.

Like I said, I read the FPC lang ref docs regarding arrays, but the above information was not immediately obvious to me. I'll re-read the docs now to see if I maybe just missed that information (which could be possible too).

Side note:
Even if the above is in the docs (maybe worded differently), this gives me a reason to complete the DocView annotations support anyway, so I can add extra info for when my brain is not working so well. :-)

> Out of interest. Do you know if dynamic array elements are in sequence > (storage area in memory) too - like static arrays? So could I do the Yes.

Perfect, using dynamic arrays will be easier (and cleaner implementation) when I need to send an array to the C library.

Maybe you understood me wrong, or indeed there is something I need to free myself. What I was referring to in the quoted text above, is when the C API allocated the array, and I simply retrieved a pointer to that array. In such cases I believe I don't need to do the freeing, I simply tell the C API to do the freeing. In the case of Xlib, that is done with the XFree() call.

Now during the opposite action, when I send an array to the C API, then yes I allocated the memory, so I need to free that memory when I have no need for the array any more. This is clear to me. I always develop with heaptrc active, so I can immediately stop when I allocated memory, but didn't free it (in the case of a external library doing the allocation one has to be more careful - I know).

Honza wrote

AFAIK yes as long as you don't break (it's possible using some wild casting) the ref counting mechanism and handle the zero length case.

In my case [working with the Xlib library], I will never have the case of a zero length array. So that I don't need to worry about. I will only need an array if I need to send more than three items (DND mime types in this case) to the Xlib library.

[ Snip: I'll see about what I can add to make it more clear ]

Take care that in this case you should not use a dynamic array to refer to the memory allocated by the C API:

Dynamic arrays are finalized at the end of a procedure, and since the Pascal memory manager has no knowledge of the memory allocated by the C lib, things will go horribly wrong when the dynamic array goes out of scope.

Michael.

Michael Van Canneyt wrote

[ Snip: I'll see about what I can add to make it more clear ]

Thanks. I just want the docs to be all that they can be. :-)

Take care that in this case you should not use a dynamic array to refer to the memory allocated by the C API:

I don't. For all arrays returned by the C API, I use static arrays as defined in my first message of this thread.