Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2017-02

How to use pipes ?

10 messages · 4 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

fredvs

Hello.

I have done lot of search on internet about pipes without success. Even for Delphi. no demo nor explanation.
The same in fpc wki page.

For example, how to use bytesavailable with pipes ?

In:
CreatePipeHandles(InHandle, OutHandle, APipeBufferSize);
InPipe.Read(Bufferout[0],BPipeBufferSize);

What is the the ratio with APipeBufferSize vs BPipeBufferSize ?

And length of Bufferout[0] vs APipeBufferSize vs BPipeBufferSize ?

In short, a demo will be welcome.

Fre;D

-----
Many thanks ;-)

View this message in context: http://free-pascal-general.1045716.n5.nabble.com/How-to-use-pipes-tp5727538.html Sent from the Free Pascal - General mailing list .
Mattias Gaertner
to fredvs
Show quoted text 7 lines
Hello. I have done lot of search on internet about pipes without success. Even for Delphi. no demo nor explanation. The same in fpc wki page. For example, how to use bytesavailable with pipes ?

http://wiki.freepascal.org/Executing_External_Programs#How_to_redirect_output_with_TProcess

Show quoted text 6 lines
In: CreatePipeHandles(InHandle, OutHandle, APipeBufferSize); InPipe.Read(Bufferout[0],BPipeBufferSize); What is the the ratio with APipeBufferSize vs BPipeBufferSize ?

APipeBufferSize is how much the other process can produce before congestion.
BPipeBufferSize is how much you want to process/eat each iteration.
Usually APipeBufferSize >= BPipeBufferSize.

And length of Bufferout[0] vs APipeBufferSize vs BPipeBufferSize ?

Bufferout[0] is just the first value. The whole Bufferout can be written by InPipe.Read. Therefore Bufferout must be >= BPipeBufferSize.

In short, a demo will be welcome.

Mattias

fredvs
to Mattias Gaertner

Many thanks Mattias.

Huh, and about bytesavailable ?

See here:

 CreatePipeHandles(InHandle, OutHandle, PipeBufferSize);
InPipe := TInputPipeStream.Create(InHandle);
OutPipe := TOutputPipeStream.Create(OutHandle);
httpget := TThreadHttpGetter.Create(url, OutPipe);
// is length ok ?
 setlength(BufferURL, PipeBufferSize);
setlength(buffertemp, PipeBufferSize);
// Is it the right way to fill BufferURL ?
len := 1 ;
len2 := 0 ;
  while (len2 < PipeBufferSize) and (len > 0) do
  begin
 len := InPipe.Read(buffertemp[0],PipeBufferSize-len2);
  if len > 0 then  for i := 0 to len -1 do
    BufferURL[i+len2] := buffertemp[i] ;
      len2 := len2 + len;
   end;

Thanks.

-----
Many thanks ;-)

View this message in context: http://free-pascal-general.1045716.n5.nabble.com/How-to-use-pipes-tp5727538p5727543.html Sent from the Free Pascal - General mailing list .
fredvs
to fredvs

Hello.

Some more explanation.

With that code, only +- 10 loops are working, after, no more Ouframes... why ?

const
FramesWanted : 640;
PipeBufferSize : 10240;
BufferURL, BufferTemp : tbytes;
BufferFloat : array of float;
...
setlength(BufferURL, PipeBufferSize);
setlength(BufferTemp,  FramesWanted * 2);
setlength(BufferFloat, FramesWanted * 2);
CreatePipeHandles(InHandle, OutHandle, PipeBufferSize);
InPipe := TInputPipeStream.Create(InHandle);
OutPipe := TOutputPipeStream.Create(OutHandle);
HandleOP :=  op_test_memory(BufferURL[0],PipeBufferSize, Err);
Outframes := 1;
outst := 1;
outst2 := 0;
// The loop, is it the correct way ?
while Outframes > 0 do
begin
   while (outst2 < FramesWanted)  and (outst > 0)  do
  begin
     outst := InPipe.Read(BufferTemp[0],FramesWanted-outst2);
    if outst > 0 then  for i := 0 to outst -1 do
   BufferURL[i+outst2] := BufferTemp[i] ;
     outst2 := outst2 + outst;
  end;
// Outframes > 0 only +- 10 loops, why ?
Outframes := op_read_float(HandleOP,  @Buffer[0], FramesWanted , nil));
...
end;

Thanks.

Fre;D

-----
Many thanks ;-)

View this message in context: http://free-pascal-general.1045716.n5.nabble.com/How-to-use-pipes-tp5727538p5727544.html Sent from the Free Pascal - General mailing list .
Andrew Haines
to fredvs
fredvs wrote 6 lines
Hello. Some more explanation. With that code, only +- 10 loops are working, after, no more Ouframes... why ?

Where is it stopping? It may be that you are asking for more bytes than are available and it is blocking waiting for more data. Or the opposite could be true. You are blocking processing data but the buffer from the kernel is full. On many Linux pc's the kernel buffer size is 65536 bytes.

Regards,

Andrew

fredvs
to Andrew Haines

Ho Andrew, nice to read you. ;-)

I battle hard to use your Thttpgetter for Opus, like you did for mpg123.

The difference with mpg123 is that Opus needs a buffer (mpg123 a named pipe).

I am nearly sure that the problem comes from my code to get the buffer from the pipe:

  while (outst2 < FramesWanted)  and (outst > 0)  do
  begin
     outst := InPipe.Read(BufferTemp[0],FramesWanted-outst2);
    if outst > 0 then  for i := 0 to outst -1 do
   BufferURL[i+outst2] := BufferTemp[i] ;
     outst2 := outst2 + outst;
  end;
Where is it stopping?

In the main loop, it is stopping a legal way, after +- 10 loops, because Outframes = 0:

Outframes := op_read_float(HandleOP,  @Buffer[0], FramesWanted , nil);
if Outframes = 0 then exit;
You are blocking processing data but the buffer from the kernel is full. On many Linux pc's the kernel buffer size is 65536 bytes.

Yes, it seems that this is the problem (but how to solve that?)

Thanks.

Fre;D

-----
Many thanks ;-)

View this message in context: http://free-pascal-general.1045716.n5.nabble.com/How-to-use-pipes-tp5727538p5727546.html Sent from the Free Pascal - General mailing list .
José Mejuto
to fredvs
fredvs wrote
Outframes := op_read_float(HandleOP, @Buffer[0], FramesWanted , nil); if Outframes = 0 then exit;

Hello,

if OutFrames < 0 then exit;

The fact that op_read_float is zero is not an error, simply the amount of samples read by opus engine is zero, maybe because internal buffer is full and do not need more data by now.

Reading from internet is usually faster than playing bitrate. Maybe you should sleep for a bit and retry.

From docs:

Returns:
The number of samples read per channel on success, or a negative value on failure.

fredvs
to José Mejuto

Hello Jose.

The fact that op_read_float is zero is not an error, simply the amount of

samples read by opus engine is zero

Exactly. I know that isnot a error otherwise OutFrames < 0.

simply the amount of samples read by opus engine is zero.

Exactly too and this because my conversion pipe into buffer is wrong. I never worked with pipes and, to be honest, I do not get everything... How to translate it to a "classical" buffer of a memory stream...

IMO, my code did read all the file from internet after +- 10 loops, the main buffer of the pipe is full.
I am in the dark, the mechanism of pipes escape to me.

Otherwise, many thanks to make me discover Opus.
It seems really the universal audio format.
And open source.
One of the developers is the one who created SoundFile library (a must).

Fre;D

-----
Many thanks ;-)

View this message in context: http://free-pascal-general.1045716.n5.nabble.com/How-to-use-pipes-tp5727538p5727552.html Sent from the Free Pascal - General mailing list .
fredvs
to fredvs

Re-hello.

Some more news:

OK, I make work a other method:

op_open_callbacks(InPipe, op_callbacks, BufferURL[0], PipeBufferSize, err);

This method use a named pipe and can be use of the box with Andrew's THttpgetter.
No need to do complicated conversion of pipe into buffer.

op_callbacks does it for you.

Nice, it works, but just like with op_open_memory, only 2 seconds of music then it stops.

The same reason: op_read_float => result after 2 seconds : Outframes = 0.

Houlala, I feel that we are near the goal.

Fre;D

-----
Many thanks ;-)

View this message in context: http://free-pascal-general.1045716.n5.nabble.com/How-to-use-pipes-tp5727538p5727553.html Sent from the Free Pascal - General mailing list .
fredvs
to fredvs: Hello. I have done lot of search on internet about pipes without success. Even for…

Huh, sorry to monopolize but...

I get it. ;-)

Changing the built-in read-callback method with this:

function OpusReadCB(stream: Pointer; var buffer; nbytes: cint): cint; cdecl;
begin
  if nbytes0
  then
 result := TInputPipeStream(stream^).read(Buffer, nbytes)
   else
    result := 0;
end;

Does the tick.

This is real web-streaming, for hours in super-high quality ! Many thanks to everybody for helping in this (hard) battle. I will clean the code and commit it asap.

fpc always win.

Fre;D

-----
Many thanks ;-)

View this message in context: http://free-pascal-general.1045716.n5.nabble.com/How-to-use-pipes-tp5727538p5727554.html Sent from the Free Pascal - General mailing list .