Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2004-12

Customizing tply

12 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Agustin Barto

After some work I was able to translate my parser's grammar from Coco/R to Lex/Yacc. It wasn't easy and I hate to loose the more "functional" aspects of Coco/R, but at least my project can compile both on Windows (with Delphi) and Linux (with fpc) (at least the non-UI parts).

When I started to modify lexlib to work with streams instead of just files (I have to parse stuff that could be on memory) I dicovered some new issues:

* How can I use standard input/ouput with streams?
* Is there an easy way to do a ReadLn (or similar) with streams? The idea is instead of rewriting the lexlib io functions, I replace the standard io functions like Eof, ReadLn, Write and WriteLn. My initial idea was to read the whole stream on a TStrings, but that could waste a lot of memory.

Thanks in advance,
Agustin

Reply by email

Steve Williams
to Agustin Barto
Show quoted text 10 lines
When I started to modify lexlib to work with streams instead of just files (I have to parse stuff that could be on memory) I dicovered some new issues: * How can I use standard input/ouput with streams? * Is there an easy way to do a ReadLn (or similar) with streams? The idea is instead of rewriting the lexlib io functions, I replace the standard io functions like Eof, ReadLn, Write and WriteLn. My initial idea was to read the whole stream on a TStrings, but that could waste a lot of memory.

Here is a unit that allows you to use ReadLn with streams. I know it works in Delphi. I have not tried it in FPC.

Sly

{== Unit StreamIO =====================================================}
{: Implements a text-file device driver that allows textfile-style I/O
    on streams.
@author Dr. Peter Below
@desc   Version 1.0 created 4 Januar 2001<BR>
         Current revision 1.0<BR>
         Last modified       4 Januar 2001<P>                           }
{======================================================================}
{: Example of use
  Var
    stream: TFileStream;
    F: Textfile;
  Begin
    stream:= TFilestream.Create( fsSource, fmOpenRead or fmShareDenyNone );
    try
      AssignStream( F, stream );
      Reset(F);
      try
        ...read stuff from F
      finally
        CloseFile(F);
      end;
    finally
      stream.Free
    end;
  End;
}
{======================================================================}
Unit StreamIO;
Interface
Uses classes;

{-- AssignStream ------------------------------------------------------} {: Attach a stream to a Textfile to allow I/O via WriteLn/ReadLn @Param F is the textfile to attach the stream to
@Param S is the stream
@Precondition S nil
@Desc The passed streams position will be set to 0 by Reset and Rewrite and to the streams end by Append. The stream is not freed when the textfile is closed via CloseFile and it has to stay in existence as long as the textfile is open.
}{ Created 4.1.2001 by P. Below
-----------------------------------------------------------------------} Procedure AssignStream( Var F: Textfile; S: TStream );

Implementation
Uses sysutils;

{-- GetDevStream ------------------------------------------------------} {: Get the stream reference stored in the textrec userdata area @Param F is the textfile record
@Returns the stream reference
@Postcondition result nil
}{ Created 4.1.2001 by P. Below
-----------------------------------------------------------------------} Function GetDevStream( Var F: TTextRec ): TStream; Begin { GetDevStream }

     Move( F.Userdata, Result, Sizeof( Result ));
     Assert( Assigned( Result ));
   End; { GetDevStream }

{-- DevIn -------------------------------------------------------------} {: Called by Read, ReadLn etc. to fill the textfiles buffer from the device.
@Param F is the textfile to operate on
@Returns 0 (no error)
}{ Created 4.1.2001 by P. Below
-----------------------------------------------------------------------} Function DevIn( Var F: TTextRec ): Integer;
Begin { DevIn }

     Result := 0;
     With F Do Begin
       BufEnd := GetDevStream(F).Read( BufPtr^, BufSize );
       BufPos := 0;
     End; { With }
   End; { DevIn }

{-- DevFlushIn --------------------------------------------------------} {: A dummy method, flush on input does nothing.
@Param F is the textfile to operate on
@Returns 0 (no error)
}{ Created 4.1.2001 by P. Below
-----------------------------------------------------------------------} Function DevFlushIn( Var F: TTextRec ): Integer;
Begin { DevFlushIn }

     Result := 0;
   End; { DevFlushIn }

{-- DevOut ------------------------------------------------------------} {: Write the textfile buffers content to the stream. Called by Write, WriteLn when the buffer becomes full. Also called by Flush. @Param F is the textfile to operate on
@Returns 0 (no error)
@Raises EStreamError if the write failed for some reason. }{ Created 4.1.2001 by P. Below
-----------------------------------------------------------------------} Function DevOut( Var F: TTextRec ): Integer;
Begin { DevOut }

     Result := 0;
     With F Do
       If BufPos > 0 Then Begin
         GetDevStream(F).WriteBuffer( BufPtr^, BufPos );
         BufPos := 0;
       End; { If }
   End; { DevOut }

{-- DevClose ----------------------------------------------------------} {: Called by Closefile. Does nothing here.
@Param F is the textfile to operate on
@Returns 0 (no error)
}{ Created 4.1.2001 by P. Below
-----------------------------------------------------------------------} Function DevClose( Var F: TTextRec ): Integer;
Begin { DevClose }

     Result := 0;
   End; { DevClose }
{-- DevOpen -----------------------------------------------------------}
{: Called by Reset, Rewrite, or Append to prepare the textfile for I/O
@Param F is the textfile to operate on
@Returns 0 (no error)
}{ Created 4.1.2001 by P. Below
-----------------------------------------------------------------------}
Function DevOpen( Var F: TTextRec ): Integer;
   Begin { DevOpen }
     Result := 0;
     With F Do Begin
       Case Mode Of
         fmInput: Begin { Reset }
             InOutFunc := @DevIn;
             FlushFunc := @DevFlushIn;
             BufPos := 0;
             BufEnd := 0;
             GetDevStream( F ).Position := 0;
           End; { Case fmInput }
         fmOutput: Begin { Rewrite }
             InOutFunc := @DevOut;
             FlushFunc := @DevOut;
             BufPos := 0;
             BufEnd := 0;
             GetDevStream( F ).Position := 0;
           End; { Case fmOutput }
         fmInOut: Begin { Append }
             Mode := fmOutput;
             DevOpen( F );
             GetDevStream(F).Seek( 0, soFromEnd );
           End; { Case fmInOut }
       End; { Case }
     End; { With }
   End; { DevOpen }
Procedure AssignStream( Var F: Textfile; S: TStream );
   Begin { AssignStream }
     Assert( Assigned( S ));
     With TTextRec(F) Do Begin
       Mode := fmClosed;
       BufSize := SizeOf(Buffer);
       BufPtr := @Buffer;
       OpenFunc := @DevOpen;
       CloseFunc := @DevClose;
       Name[0] := #0;
       { Store stream reference into Userdata area }
       Move( S, Userdata, Sizeof(S));
     End; { With }
   End; { AssignStream }

End { StreamIO }.

Reply by email

Michalis Kamburelis
to Agustin Barto: After some work I was able to translate my parser's grammar from Coco/R to Lex/Yacc. It…

Hi,

I'm attaching my unit TextReader, it

- implements TTextReader class (that is able to read from any TStream class using simple Readln and Eof methods) and

- initializes/finalizes some variables to treat standard input/output like a TStream classes (or, only in case of standard input, like a TTextReader class).

Works on Win32, Linux, FreeBSD (should work on any UNIX). I'm also attaching some simple demo program.

This code is part of my GNU GPLed KambiClassUtils unit on my WWW pages, I just extracted it for you to send something small.

Michalis Agustin Barto wrote: > After some work I was able to translate my parser's grammar from > Coco/R to Lex/Yacc. It wasn't easy and I hate to loose the more > "functional" aspects of Coco/R, but at least my project can compile > both on Windows (with Delphi) and Linux (with fpc) (at least the > non-UI parts). > > When I started to modify lexlib to work with streams instead of just > files (I have to parse stuff that could be on memory) I dicovered some > new issues: > > * How can I use standard input/ouput with streams? > * Is there an easy way to do a ReadLn (or similar) with streams? The > idea is instead of rewriting the lexlib io functions, I replace the > standard io functions like Eof, ReadLn, Write and WriteLn. My initial > idea was to read the whole stream on a TStrings, but that could waste > a lot of memory. > > Thanks in advance, > Agustin > > _______________________________________________ > fpc-pascal maillist - > http://lists.freepascal.org/mailman/listinfo/fpc-pascal >

Reply by email

Michael Van Canneyt
to Michalis Kamburelis
Michalis Kamburelis wrote 10 lines
Hi, I'm attaching my unit TextReader, it - implements TTextReader class (that is able to read from any TStream class using simple Readln and Eof methods) and - initializes/finalizes some variables to treat standard input/output like a TStream classes (or, only in case of standard input, like a TTextReader class).

Actually, do you think you could extend this and
create a bridge between streams and text files ?

I mean 2 calls like

First: Text->Stream.

Var
  f : Text;
  S : TSTream;
begin
  AssignTextStream(F,S);
  Rewrite(F);
  Writeln(F,'Something'); // Writes actually to stream S.
  Close(F);
end.

Second: Stream->Text.

Var
  f : Text;
  S : TSTream;
  L : TStringList;
begin
  Assign(F,'File.txt');
  Rewrite(F);
  Writeln(F,'Following lines written to stream:');
  S:=TSomeStream.Create(F);
  L.SaveToStream(S);
  S.Free;
  Close(F);
end;

The second will be probably less used, but the first one may be quite useful.

Michael.

Reply by email

Steve Williams
to Michael Van Canneyt
Show quoted text 17 lines
Actually, do you think you could extend this and create a bridge between streams and text files ? I mean 2 calls like First: Text->Stream. Var f : Text; S : TSTream; begin AssignTextStream(F,S); Rewrite(F); Writeln(F,'Something'); // Writes actually to stream S. Close(F); end.

That's exactly how the unit I posted works. As for the second method you mentioned, I do not see much use for that.

Sly

Reply by email

Michael Van Canneyt
to Steve Williams
Steve Williams wrote 20 lines
> Actually, do you think you could extend this and > create a bridge between streams and text files ? > > I mean 2 calls like > > First: Text->Stream. > > Var > f : Text; > S : TSTream; > > begin > AssignTextStream(F,S); > Rewrite(F); > Writeln(F,'Something'); // Writes actually to stream S. > Close(F); > end. That's exactly how the unit I posted works. As for the second method you mentioned, I do not see much use for that.

Owowow...
I should have looked more careful to your message. That is of course exactly what I asked for.
Can I rework the code you sent and include it in the FCL ?

The second part is mainly to interface with older legacy code that still uses text files, without having to rewrite it. I have several such units for my day-time job.

Michael.

Reply by email

Agustin Barto
to Michael Van Canneyt

I integrated the generated parser and libs with the old code and now I get a lot of run-time errors. I traced some of them to lexlib's assigns of the default input and ouput so I commented out those (It's a GUI app anyway). But then I started to get access violations. What puzzles me is that my console based test-application worked just fine.

Right now I'm still using Delphi, so it's ok if you decide not to answer this.

I traced the error to this piece of the generated code:

if (yyd[yystate]=0) and (yychar=-1) then
    (* get next symbol *)
    begin
      ==> yychar := yylex; if yychar<0 then yychar := 0; <==
    end;

btw, the problem is not StreamIO, I did some extensive testing of it and it worked fine. Besides, it doesn't seem to be an I/O related problem, cause the parser gets to execute the action of the first token.

Reply by email

Steve Williams
to Michael Van Canneyt: Owowow... I should have looked more careful to your message. That is of course exactly…

I cannot remember where I found that code. From the unit header, it was written by a Peter Below (a member of TeamB) and posted on UseNet a while back. So I guess it is up to you to include it or not, or if you want to try to track down Peter Below and ask his permission.

Sly

Michael Van Canneyt wrote 11 lines
Owowow... I should have looked more careful to your message. That is of course exactly what I asked for. Can I rework the code you sent and include it in the FCL ? The second part is mainly to interface with older legacy code that still uses text files, without having to rewrite it. I have several such units for my day-time job. Michael.

Reply by email

Agustin Barto
to Agustin Barto: I did a little more testing, and the console based apps fail too, with the same access…

Success!!!

My problem was that I forgot to use $H- for the parser unit. Now everything works on Linux with fpc and on Windows with Delphi with the same code. in windows I have to compile tply from the latest 1.9.5 with Delphi, but it works just fine.

Thanks for all the help.

Agustin

Agustin Barto wrote
I did a little more testing, and the console based apps fail too, with the same access violation exception.

Reply by email

Bartek
to Agustin Barto

hi,
i'm trying to get this piece of code to work. but it fails on execution and returns a SIGSEV signal. i think this is caused by the weird code, fpc creates. fpc tryed to access the wide sse registers with multiple short 32bit registers, and at some point, protected memory is accessed.

i'm using win32 with fpc 1.9.5 .

program test;
type
        vector4 = array[0..3] of single;
var
        a, b, c: vector4;

operator + (v1,v2 :vector4) v_r:vector4;assembler;inline; asm
movups xmm0, v1
addps xmm0, v2
movups v_r, xmm0
end;

begin
c:=a+b;
end.

--------------------------------------------------------------------------------------------------------

# [15] c:=a+b;
leal _B,%edi
movl %edi,-4(%ebp)
leal _A,%edi
movl %edi,-8(%ebp)
leal -24(%ebp),%edi
movl %edi,-56(%ebp)
.L640:
.Ll9:
.stabn 68,0,9,.Ll9 - program_init
# [9] vups xmm0, v1
movups -8(%ebp),%xmm0
.Ll10:
.stabn 68,0,10,.Ll10 - program_init
# [10] dps xmm0, v2
addps -4(%ebp),%xmm0
.Ll11:
.stabn 68,0,11,.Ll11 - program_init
# [11] vups v_r, xmm0
movups %xmm0,-12(%ebp)
.L641:
.Ll12:
.stabn 68,0,15,.Ll12 - program_init
leal _C,%edi
leal -24(%ebp),%esi
cld
movl $4,%ecx
rep
movsl

Reply by email

Florian Klaempfl
to Bartek
Bartek wrote 5 lines
hi, i'm trying to get this piece of code to work. but it fails on execution and returns a SIGSEV signal. i think this is caused by the weird code, fpc creates. fpc tryed to access the wide sse registers with multiple short 32bit registers, and at some point, protected memory is accessed.

FPC doesn't align the stack properly to sixteen byte boundaries currently.

Show quoted text 58 lines
i'm using win32 with fpc 1.9.5 . program test; type vector4 = array[0..3] of single; var a, b, c: vector4; operator + (v1,v2 :vector4) v_r:vector4;assembler;inline; asm movups xmm0, v1 addps xmm0, v2 movups v_r, xmm0 end; begin c:=a+b; end. -------------------------------------------------------------------------------------------------------- # [15] c:=a+b; leal _B,%edi movl %edi,-4(%ebp) leal _A,%edi movl %edi,-8(%ebp) leal -24(%ebp),%edi movl %edi,-56(%ebp) .L640: .Ll9: .stabn 68,0,9,.Ll9 - program_init # [9] vups xmm0, v1 movups -8(%ebp),%xmm0 .Ll10: .stabn 68,0,10,.Ll10 - program_init # [10] dps xmm0, v2 addps -4(%ebp),%xmm0 .Ll11: .stabn 68,0,11,.Ll11 - program_init # [11] vups v_r, xmm0 movups %xmm0,-12(%ebp) .L641: .Ll12: .stabn 68,0,15,.Ll12 - program_init leal _C,%edi leal -24(%ebp),%esi cld movl $4,%ecx rep movsl _______________________________________________ fpc-pascal maillist - http://lists.freepascal.org/mailman/listinfo/fpc-pascal

Reply by email