While waiting for a 64 bits Delphi, I am doing experimentation with fpc-Lazarus 64 bits. And this is my problem:

In most of my programs I use my own library for parsing and evaluating floating point functions like

'x sin(y z)'

passed by the user as strings. The basic organization is as follows:

{---------------simplified code------------}

   TFunctionEvaluator= class(TObject)
       ...
       code: AnsiString;
       ...
     public
          {call the parser}
       function ParseAFunctText(t, vars: AnsiString): AnsiString;
          {call the evaluator}
       V: function(X: array of Double): Double;
   end;

{----------end code------------------------}

Suppose that "theF" is an already created instance of TFunctionEvaluator and suppose that a program calls

   msg:= theF.ParseAFunctText('pi', 'xyz');

Here, 'pi' is the text of the constant funcion that must give 3.14... as result, 'xyz' contains the chars in the function text that the parser should take as independent variables if they appear in t. The result will be an error message "msg" if something fails. Then ParseAFunctText builds "code" and puts

    @V:= PChar(code);

Suppose that a program calls for example

    r:= theF.V([1, 2, 3]);

Then Lazarus (fpc 2.2.4, lazarus 0.9.28.2 beta, target windows 64 bits, mode Delphi) compiles, but it raises an

'External:SIGSEGV' error.

In the debugger the exception is raised before the effective call of the function code. If then I press F7, then the debugger jumps to the following assemblies

push %rbp
mov %esp,%ebp
fldpi {loads pi to the fpu}
ftspl -0x18(%rbp) {copy pi to stack}
movsd -0x18(%rbp),%xmm0 {copies pi to XMM0, so caller {may find also there the result}
leaveq
retq

I have tried all possible combinations like V:= PChar(code), V:= @code[1], etc. to no avail. Thus I think something is wrong in my understanding of the whole business in 64 bits or in fpc-Lazarus, because this is not a problem in Delphi nor it was in an old version of fpc-Lazarus 32 bits. Please give me a tip.
Many thanks in advance.

You probably have to mark the memory in which you write the code as "executable" first (this has to be done using an OS-specific function; I know next to nothing about Windows programming, but you should be able to find the necessary information in MSDN).

In fact, you should also do this on 32 bit platforms, since more and more systems mark memory by default as not executable for security purposes.

Jonas

PS: you should mention that you are not subscribed to the mailing list in all messages you send, because otherwise people won't know that they have to CC you when replying to your messages (I know only because I'm the list admin and let your message through when it was stuck in the moderation queue).

In our previous episode, Jonas Maebe said:

This is called DEP (Data Execution Protection, which is an amalgam of resp AMD and Intel hardware protection and some software forms). IIRC it is always on for 64-bit, and for 32-bit server versions that use a PAE kernel. (client OSes don't have PAE version on windows afaik)

Afaik it can only be turned on before starting a binary. An executable can't disable it while running (since then it would be useless, as a virus could simply disable it)

See e.g.
http://techblissonline.com/enable-disable-dep-in-windows-xp-vista/

Searching for "disable dep" or so will probably yield more results.

Of course, this is not healthy and supportable application behaviour, but I assume this is already known.

This is not entirely correct. Quote from Wikipedia regarding PAE: ===quote begin===
Windows XP SP2 and later, by default, on processors with the no-execute (NX) or execute-disable (XD) feature, runs in PAE mode in order to allow NX. ===quote end===

I personally would say that mapping a page with the Execute flag set and storing the to-be-executed content there should be enough. After all JIT compilers must do that as well. ;)

Regards,
Sven

Also according to here:
http://en.wikipedia.org/wiki/Data_Execution_Prevention#Configuration DEP is not enabled by default for all applications.

Regards,
Sven

In our previous episode, Sven Barth said:

See the exception of "OptIn". Both are before W7 released, so I assume it goes for W7/64 too.

memory at runtime then calling that code? If so then maybe the following can help you.

I made some trampoline procedures where I wrote some executable code at runtime and I had to, both on Window and Linux, allocate some memory that was marked as executable. Here's a snippet of how I allocated the memory. The code worked on Windows 7 x64/32 and Gentoo Linux x64/32

 PTrampolineBlock = ^TTrampolineBlock;
  TTrampolineBlock = record
    NextBlock: Pointer;
    Cursor: DWord;
    Size: DWord;
    Code: PtrUint; // this is actually the start of the remainder of the
allocated size
  end;
procedure TTrampolineManager.AllocateBlock;
const
  AllocSize = $1000;
var
  NewBlock: PTrampolineBlock;
begin
  {$IFDEF UNIX}
  NewBlock := fpmmap(nil, AllocSize, PROT_EXEC or PROT_READ or
PROT_WRITE, MAP_ANONYMOUS or MAP_PRIVATE, -1, 0);
  {$ENDIF}
  {$IFDEF MSWINDOWS}
  NewBlock := VirtualAlloc(nil,AllocSize,MEM_COMMIT,PAGE_EXECUTE_READWRITE);
  {$ENDIF}
NewBlock^.Cursor:=PtrUint(@NewBlock^.Code)-PtrUint(@NewBlock^.NextBlock);
  NewBlock^.Size:=AllocSize;
  NewBlock^.NextBlock:=Block;
  Block := NewBlock;
end;
procedure TTrampolineManager.FreeBlock(var ABlock: PTrampolineBlock);
begin
  if ABlock = nil then
    Exit;
  FreeBlock(ABlock^.NextBlock);
  {$IFDEF UNIX}
  Fpmunmap(ABlock, ABlock^.Size);
  {$ENDIF}
  {$IFDEF MSWINDOWS}
  VirtualFree(ABlock, 0, MEM_RELEASE);
  {$ENDIF}
  ABlock := nil;
end;
procedure TTrampolineManager.WriteData(const AData; ASize: Byte);
begin
  if Block = nil then AllocateBlock;
  //WriteLn('Writing: ', hexStr(PtrUint(AData), ASize*2));
  if Block^.Cursor + ASize >= Block^.Size then
    raise TrampolineBlockFullException.Create('');
  Move(AData, PByte(@Block^.Code)[Block^.Cursor], ASize);
  Inc(Block^.Cursor, ASize);
end;
so the usage would be like so
function TTrampolineManager.GenerateCode(args: ....): Pointer;
begin
try
  Result := CurrentBlock.Position; // = @Block + Block.Cursor
  repeat
    WriteData(your_data, size_of_data);
  until done;
except
    on e: TrampolineBlockFullException do
      begin
        TrampolineManager.AllocateBlock;
        Result := GenerateCode(args);
      end;
  end;
end;

Hope this helps :)

Regards,

Andrew Haines

Sven Barth wrote

I personally would say that mapping a page with the Execute flag set and storing the to-be-executed content there should be enough. After all JIT compilers must do that as well. ;)

Thanks. I shall study this issue.

Andrew Haines wrote

> From the other comments it seems like you are writing some assembly to memory at runtime then calling that code?

That is right.

A lot! I understand that for purposes other than trampolining, the essence of this is the the way of building an almost contiguous list, say

  Block: array[0..lastCodePiece] of Pointer;

of DEP executable codes inside an authorized chunk of memory, and that the call for assigning one of them, Block[i], to some function declared as

     otherObject.thisFunction(other args): Double;

would be

   @otherObjectInstance.thisFunction:= Block[i];

And also the way to obtain another block when one of them has been exhausted, and the way of disposing of them, right?

Many thanks, Andrew. I intend to apply this to my old programs in 32 bit Windows. I'll tell you about the outcome.

Many thanks, Andrew. I intend to apply this to my old programs in 32 bit Windows. I'll tell you about the outcome.

Below you will see a console application for testing your suggestions or better my understanding of them. The results are (in Windows XP 64 bits Pro):

1. When Data Execution Prevention (DEP) is enforced, the program fails.

2. When DEP is enforced only for essential Windows programs, the program functions (that is, it writes the value of pi). But if one uncomments the commented line of code, that is makes

    codeFunction:= '',

the program fails.

What may be happening here?

Many thanks

Forgot to tell that the program was written, compiled and tested with Delphi6 PE for Windows 32 bits
Sorry.

This code is wrong:

functionCode : AnsiString; {the function opCode sequence} ...

   Move(functionCode, PChar(FBlock^.code), len);

An ansistring is a pointer. Move takes formal const/var parameters. So the move() moves the pointer along with whatever data comes after it over the "FBlock^.code" pointer and whatever data comes after that. What you want, is to move the data to which functionCode points into the memory block to which FBlock^.code points. The correct statement is therefore:

   Move(functionCode[1], FBlock^.code^, len);

It would also be cleaner to use a dynamic array instead of a string to store arbitrary binary data (in that case, you'd have to use functionCode[0] above though).

Jonas

I second the motion of using a Dynamic array too instead of the manual creation/destruction of a memory block.

I use this for a virtual machine I created a few years back and it works under XP/Vista for me :)

cheers,
Paul

I have tried yours and other combinations. The following one works at last in Delphi6 PE. This will keep me occupied some time updating my 32 bits programs, so that they will be DEP safe. Thanks to you all. Note that I have dropped the use of Move. As for fpc-Lazarus 64 bits, it does not work yet: more about this in another thread.

{--------------code snippet begins--------------}

   len:= Length(cod);   { cod  is the ansiString containing
                          the opcodes to be executed}
   SetLength(ba, len);   { ba   is an auxiliary array of bytes}
   for i:= 0 to len-1 do
     ba[i]:= ord(cod[i+1]);
   { exeBlock   has been declared as PChar }
   exeBlock:= VirtualAlloc(nil, len + SizeOf(exeBlock), MEM_COMMIT,
           PAGE_EXECUTE_READWRITE);
   for i:= 0 to len-1 do
     exeBlock[i]:= chr(ba[i]);
   @F:= @exeBlock[0];  {  F  is the function whose code is
                        here assigned to  cod}
   ba:= nil;
   cod:= '';

{----------code ended---------------}

For people interested in details: the use of an AnsiString for storing the opcodes, that is for building "cod", eases this by the use of the special procedures to treat strings (adding them, etc.). The use of exeBlock as PChar serves to guarantee that a contiguous chunk of memory starting in exeBlock[0] becomes reserved for the exclusive use of exeBlock and cannot overlap another variable: this is my interpretation of the failure that did happen when, after assigning the code to F, I did put cod = ''.