As I understand the for..in loop GetEnumerator method is expected to create a new object each time it’s called and FPC destroys it later when the loos is finished. Can I retain the enumerator and just reset it in-between calls? I’d like to remove all these alloc/deallocs so I can use for..in more efficiently in tight loops.

Regards,
Ryan Joseph

Ryan Joseph wrote

As I understand the for..in loop GetEnumerator method is expected to create a new object each time it’s called and FPC destroys it later when the loos is finished. Can I retain the enumerator and just reset it in-between calls? I’d like to remove all these alloc/deallocs so I can use for..in more efficiently in tight loops.

You can do so by overriding the newinstance and it's sister method of your enumerator class.

As an alternative you can create an object enumeator. It's simply allocated on the stack, and you can reset it in the enumerator operator.

See the example in http://wiki.freepascal.org/for-in_loop section: Using any identifiers instead of builtin MoveNext and Current

Michael.

In our previous episode, Michael Van Canneyt said:

As an alternative you can create an object enumeator. It's simply allocated on the stack, and you can reset it in the enumerator operator.

Yup, or a record. See e.g. http://www.stack.nl/~marcov/lightcontainers.zip

Ryan Joseph wrote

I’d like to remove all these alloc/deallocs so I can use for..in more efficiently in tight loops.

I've had the same requirement, and also needed that functionality before the for..in syntax existed in FPC. Take a look at my Iterator interface and implementation. It allows you to move forward, backwards, reset, filter data etc. You can hold on to the instance reference as long as you like.

This code lives in the tiOPF project, but can be used outside of the tiOPF project too (I do that often) - simply delete the iterator implementations for TtiObjectList.

https://github.com/graemeg/tiopf/blob/tiopf2/Options/tiIteratorIntf.pas

https://github.com/graemeg/tiopf/blob/tiopf2/Options/tiIteratorImpl.pas

Some years ago I wrote a article for a magazine about this, and that is when I implemented the code. You can still find the "Iterator" article in the link below - and the accompanied source code too (though the tiOPF code is newer).

http://geldenhuys.co.uk/articles/

Regards,
Graeme

Michael Van Canneyt wrote

As an alternative you can create an object enumeator. It's simply allocated on the stack, and you can reset it in the enumerator operator.

That is by far the easiest solution (records need $modeswitch advancedrecords, but are otherwise equivalent).

As an example, here's how I iterate over at TNodeSet returned by fcl-xml's XPath engine: <https://pastebin.com/Zj2CLRbX>

The object is allocated on the stack and simply cleared/reset every time the operator is executed.

On Oct 4, 2017, at 5:19 PM, Marco van de Voort wrote: Yup, or a record. See e.g. http://www.stack.nl/~marcov/lightcontainers.zip

This seems like the simplest more efficient method. Does FPC just know it’s a record internally and not try to dealloc it? GetEnumerator seems like a magic method so I’m not sure how the memory is being managed but I assume the stack space for the method return value is reserved and I just fill in the values of the record to that location.

Regards,
Ryan Joseph

On Oct 4, 2017, at 4:26 PM, Michael Van Canneyt wrote: You can do so by overriding the newinstance and it's sister method of your enumerator class.

Can you explain how this works or give an example? Not sure how these gets around the problem of alloc/dealloc for each iterator.

Regards,
Ryan Joseph

Careful, the iterator is instantiated only once per loop ?

Newinstance allocates the memory for a new instance of the class. By default this is GetMem(instanceSize).

What you can do is allocate somewhere a block of the correct size, once, and always return this single block.

Of course, you need to know for sure that only 1 instance of your enumerator will be active at any given point in your program. If there are multiple, you could allocate a static array and allocate from that.

The FreeInstance procedure does a freemem, by default, but you can change that to do nothing. (in case of an array, you mark the element unused)

Michael.

On Oct 4, 2017, at 10:03 PM, Michael Van Canneyt wrote: Newinstance allocates the memory for a new instance of the class. By default this is GetMem(instanceSize).

So you override the class method Newinstance in the enumerator class and return the same block of memory? Then when I override FreeInstance and do nothing I need to manually free the memory in the calling class I guess.

That’s an interesting solution and I didn’t know TObject did that even but using records still seems like a better solution and about as efficient (perhaps). I’ll try this tomorrow also to make sure I understand it. Thanks.

Regards,
Ryan Joseph

As I understand the for..in loop GetEnumerator method is expected to create a new object each time it’s called and FPC destroys it later when the loos is finished. Can I retain the enumerator and just reset it in-between calls? I’d like to remove all these alloc/deallocs so I can use for..in more efficiently in tight loops.

Here is an example how to use a global enumerator object:

type
  TMyEnumerator = class
  private
    FOwner: TComponent;
    FCurrent: TComponent;
  public
    procedure Init(Owner: TComponent);
    function MoveNext: boolean;
    property Current: TComponent read FCurrent;
    function GetEnumerator: TMyEnumerator;
  end;
  TForm1 = class(TForm)
  public
    function GetEnumerator: TMyEnumerator;
  end;
var
  Form2: TForm2;
  MyEnumerator: TMyEnumerator;
implementation
procedure TMyEnumerator.Init(Owner: TComponent);
begin
  FOwner:=Owner;
  FCurrent:=nil;
end;
function TMyEnumerator.MoveNext: boolean;
var
  i: Integer;
begin
  if FCurrent=nil then begin
    if FOwner.ComponentCount=0 then exit(false);
    FCurrent:=FOwner.Components[0];
  end else begin
    i:=FCurrent.ComponentIndex+1;
    if i>=FCurrent.Owner.ComponentCount then exit(false);
    FCurrent:=FCurrent.Owner.Components[i];
  end;
  Result:=true;
end;
function TMyEnumerator.GetEnumerator: TMyEnumerator;
begin
  Result:=Self;
end;
function TForm1.GetEnumerator: TMyEnumerator;
begin
  if MyEnumerator=nil then
    MyEnumerator:=TMyEnumerator.Create;
  MyEnumerator.Init(Self);
end;
finalization
  MyEnumerator.Free;
end.