Hello

In a C method:

OP_WARN_UNUSED_RESULT OggOpusFile *op_test_memory(const unsigned char *_data,
size_t _size,int *_error);

translated in Pascal with this:

var
 op_test_memory: function(const _data; const size: cuint; out error: cint):
TOggOpusFile;

How to use a pipe for _data ?

The pipe is a TInputPipeStream.

Using this does not work:
pipein was created and linked to a url.

var
pipein :TInputPipeStream; /
PipeBufferSize := $4000;
...
op_test_memory(pointer(InPipe),PipeBufferSize, Err);

The error is -129 which is a null pointer or other generic internal error.

Thanks.

Fre;D

-----
Many thanks ;-)

That will never work.

OP_WARN_UNUSED_RESULT OggOpusFile *op_test_memory(const unsigned char *_data, size_t _size,int *_error);

expects a memory buffer; you cannot pass it a stream.

Michael.

Ha, ok.

And how to convert a pipe into a memory buffer ?

Thanks.

Fre;D

-----
Many thanks ;-)

Read from it and append the content to the buffer. This probably means some polling mechanism:
fpSelect on unix, WaitForObject on windows.
Or maybe using a thread. But that will require some synchronisation.

Michael.

Thanks Michael.

OK, I know the way, I will try it (but code example is very welcome).

Fre;D

-----
Many thanks ;-)

Hello Fred,

I didn't understand the software structure / problem behind the C-> Pascal translation, but for me named pipes work very well for many kinds of interprocess communication.

Markus

didn't understand the software structure / problem behind the C-> Pascal translation

Huh, IMO, the translation is ok.
The problem is that the method ask for a memory buffer and I give a pipe.

but for me named pipes work very well for many kinds of interprocess communication.

Yes, for example, mpg123 library is very happy with this:

PipeBufferSize := $4000 ;
CreatePipeHandles(InHandle, OutHandle, PipeBufferSize);
InPipe := TInputPipeStream.Create(InHandle);
OutPipe := TOutputPipeStream.Create(OutHandle);
httpget := TThreadHttpGetter.Create(opus_url,OutPipe);
mpg123_open_handle(AHandle, Pointer(InPipe));

But for Opus library, there is error while using:

op_test_memory(pointer(InPipe),PipeBufferSize, Err);

Like Michael explained, the buffer of the pipe must be entered.

And I do not know how to do this... ;-(

Fre;D

-----
Many thanks ;-)

Hello.

Some news from the front:

var
BufferURL : array of float;

.....

PipeBufferSize := $4000 ;
CreatePipeHandles(InHandle, OutHandle, PipeBufferSize);
InPipe := TInputPipeStream.Create(InHandle);
OutPipe := TOutputPipeStream.Create(OutHandle);
httpget := TThreadHttpGetter.Create(opus_url,OutPipe);
setlength(BufferURL, PipeBufferSize);
// This to have the buffer as parameter ???
InPipe.Read(BufferURL,PipeBufferSize);
op_test_memory(BufferURL,PipeBufferSize, Err);

=> Gives as error: -133 =>
A required header packet was not properly formatted, contained illegal values, or was missing altogether.

So it seems that the buffer was accepted, only miss the header ?

Difficult to explore when there is no documentation how to do nor examples.

Any idea is highly welcome.

Fe;D

-----
Many thanks ;-)

I had never tried pipes before, but it seems FPC has a unit to handle that: `iostream`, so today I played a very simple test:

program project1;
uses sysutils, iostream;
const buf_sz = 1024;
var
  buf: tbytes;
  sz: longint;
  stdin: tiostream;
begin
  stdin := tiostream.create(iosinput);
  try
    setlength(buf, buf_sz);
    // it is just a test, so I limited it to 1024 bytes, please read the
entire content by yourself! :-)
    sz := stdin.read(buf[0], buf_sz);
    if sz < buf_sz then
      setlength(buf, sz);
    writeln(tencoding.utf8.getstring(buf));
  finally
    stdin.free;
  end;
end.

in my terminal:

$ echo "Yes" | ./project1
Yes

However, I don't know if you are searching exactly that, anyway you can check the iosinput sources. :-)

Any idea is highly welcome.

Thanks Silvio, I will try it tonight.
Write you later.

Fre;D

-----
Many thanks ;-)

Hello Silvio:

Yes, we are on the good way.
Following your advice, here from https-url-opus the result of :

var
  BufferURL: tbytes;
  setlength(BufferURL, PipeBufferSize);
  CreatePipeHandles(InHandle, OutHandle, PipeBufferSize);
  InPipe := TInputPipeStream.Create(InHandle);
  OutPipe := TOutputPipeStream.Create(OutHandle);
  httpget := TThreadHttpGetter.Create(url, OutPipe);
  InPipe.Read(BufferURL[0],PipeBufferSize);
  writeln(tencoding.utf8.getstring(BufferURL));

======>

Lavf57.57.100encoder=Lavc57.65.100 libopustitle=Be Thankful%artist=For What You GotWilliam De Vau$album=Blaxploitation Vol.3 The Payba date=1997TRACKNUMBER=11OggS��d,
@*�m��|��G���"��>_4Zѡ�/kv��e"��bvZ6�j�

�z2��PY�*��]luN��H�t�A5QR�s:3�0���:�^����͋�¢f��8�$5{w>)YZ���K>�𠑡T�`�+������z�g "F^�������a h��«�o?"t�C�d]J=��kdw�Z3|�L��B���a���r�b�Ĉ}��u� �+r���{<�`�_��O&�t� G�g�܄~��oᐄQ|�~7�E��~!��<�$3� E��I����4����h;��
,�rk��&������=����V��+�I3~�~j��^��g�x�a֣�,l'�_ =����(��aԦ���u�ZҬ��
�|N+9զ~e͓���2���r�a֥R�E����� �Fޡj��\��QA��s<6�Cal������ۨ��Ä�zE����!Z�&G�"���a֥��7�rɝnt�Q̊4�N��{�w[ ��zt��p��%�ƌ�ZYN����}䒀���7������

You may note that some opus infos are there and seems to be the header. Sadly, op_test_memory(BufferURL,PipeBufferSize, Err) gives this error:

error -132: A purported Ogg Opus stream did not begin with an Ogg page, a purported header packet did not start with one of the required strings, "OpusHead" or "OpusTags", or a link in a chained file was encountered that did not contain any logical Opus streams.

But there is some light, we will win.

Fre;D

-----
Many thanks ;-)

fredvs wrote

Hello Silvio: Yes, we are on the good way.

Awesome. :-)

[...]

Some streams requires you set its cursor to 0 before writing/reading buffer, so you need to check it:

  CreatePipeHandles(InHandle, OutHandle, PipeBufferSize);
  InPipe := TInputPipeStream.Create(InHandle);
  OutPipe := TOutputPipeStream.Create(OutHandle);
  httpget := TThreadHttpGetter.Create(url, OutPipe);
  OutPipe.Seek(0, soBeginning);
  InPipe.Seek(0, soBeginning);
  InPipe.Read(BufferURL[0],PipeBufferSize);

Notice tencoding was just for testing. :-)

Hello,

The first thing to debug this problems is to determine the expected data for opus_test_memory. Usually streams (audio, video, etc...) are composed by two pieces the raw data and the transport envelope. In this case opus usually uses Ogg as transport envelope but it could be streamed in another container like MP4, MKV, etc.

Opus library should provide a play engine that decodes raw streams, extracted from its transport layer and maybe another series of APIs for the default transport layer, Ogg in this case. This last job seems to be done by the OpusFile API.

So to resume, you file is a raw opus audio or an Ogg file ?

Before using pipes with http try pipes with a local file. If your pipes on local file works but http does not maybe you are receiving the http headers, or any other unexpected situation, so dump the piped data to a local file and match it against the remote one.

Said that, I had never touched opus in my life :) so take my comments with care.

Hello.

=====>

"Exception at 000000000043A720: EPipeSeek:
Cannot seek on pipes."

So to resume, you file is a raw opus audio or an Ogg file ?

Huh, I do not know. It was converted from a mp3.
Localy, from a hard-disk:

Using op_test_file("test.opus") + op_open_test() is working perfectly.

IMO, the problem comes from abuffer in op_test_memory(abuffer,lengthb).

I am not sure that a array of byte is what op_test_memory want. It is not a pipe (like lot of other audio lib use), if it is not a array of byte, what does he want...?

Said that, I had never touched opus in my life :)

So I am not the only one. ;-)

PS: On Opus forum, there is only one topic this year. Without answer. (And this topic is from me !).

Fre;D

-----
Many thanks ;-)

Hello.

Some other news from the front:

var
  BufferURL, buffadd: tbytes;
...
  setlength(BufferURL, PipeBufferSize);
  setlength(buffadd, PipeBufferSize);
  CreatePipeHandles(InHandle, OutHandle, PipeBufferSize);
  InPipe := TInputPipeStream.Create(InHandle);
  OutPipe := TOutputPipeStream.Create(OutHandle);
  httpget := TThreadHttpGetter.Create(url, OutPipe);
PipeBufferSize := 1024;
len2 := 0;
len := 1;
while (len2 < PipeBufferSize) and (len > 0) do
  begin
 len := InPipe.Read(buffadd[0],PipeBufferSize);
  if len > 0 then  for i := 0 to len -1 do
  BufferURL[i+len2] := buffadd[i] ;
     len2 := len2 + len;
   end;
 HandleOP := op_test_memory(BufferURL[0],PipeBufferSize, Err);

if Err=0
then begin

         Err := op_test_open(HandleOP);

....
==============================>
...
Try to connect to
https://sites.google.com/site/fredvsbinaries/guit_kungs.opus error: op_test_open: 0
OK open
OpusTag.comments = 6
ALBUM=Summer 2016
ARTIST=Kungs
DATE=2016
DESCRIPTION=Loop Guitar.
GENRE=Dance
TITLE=guit_kungs.opus
op_bitrate = 2147483647
Data.Channels 2

===========>

Yep, yep, yep, connected and retrieving the tag + infos ok.

The trick was to do a loop with InPipe.Read and add buffadd to BufferURL.

So, good, there is one step more.

Sadly, op_read_float(HandleOP,@Buffer[0], Wantframesl) does not work. I suspect that some read callback must be define for reading.

There is a method for opening steams with custom callback:

OP_WARN_UNUSED_RESULT OggOpusFile *op_test_callbacks(void *_source, const OpusFileCallbacks *_cb,const unsigned char *_initial_data, size_t _initial_bytes,int *_error) OP_ARG_NONNULL(2);

Translated with this:

op_test_callbacks: function(const source; const cb: TOpusFileCallbacks;
const initial_data; initial_bytes: cuint;
  out error: cint): TOpusFile;

But sadly I do not understand what is source and not sure what is initial_data.
Also I do not understand how to deal with cb and assign read (to InPipe.Read ?)

TOpusFileCallbacks = record
    read: op_read_func;
    seek: op_seek_func;
    tell: op_tell_func;
    close: op_close_func;
  end;

This does not work:

HandleOP := op_test_callbacks(pointer(InPipe),op_callbacks, BufferURL[0],
PipeBufferSize, err);

Huh, once again, your advices will be extremely appreciated.

Thanks.

Fre;D

-----
Many thanks ;-)

C doesn't even really have arrays since it is so low level (well this is disputed). Often an API function that accepts a buffer could allow multiple types of buffers, that you allocate yourself... Make sure the function doesn't modify the buffer size (bad api practice) and make you allocate the memory

It just seems like this is similar to windows API programming where the windows C api would allow a fixed array of byte (or similar), or your own allocated heap array.... i.e. whatever you want.

Advantage of heap array is you can change it; fixed static stack array is fixed and not modifiable on heap... But make sure the api doesn't muck with the buffer on it's end or try to allocate memory for it, as that should be your job. But you never know with some api's. This should all be in their docs? They don't have proper docs?

But I know nothing of Opus, sorry. (and can you look into the actual source code of opus or is this closed source?)

Hello Lars.

Thanks for your brilliant light.

(and can you look into the actual source code of opus or is this closed source?)

==> https://github.com/xiph/opus

And translation into fpc-Pascal:

==> https://github.com/fredvs/uos/blob/master/src/uos_opusfile.pas

Fre;D

-----
Many thanks ;-)

Hello,

As you need to use records you should specify {$PACKRECORDS C} or you could get a different record layout than C expects.

you should specify {$PACKRECORDS C}

Nice tip !

Added in each unit that deal with c libraries.

Many hanks.

Fre;D

PS: But this did not help ;-(

-----
Many thanks ;-)

@ Lars:

The library used for op_memory_open and op_seek is libopusfile.so: https://github.com/xiph/opusfile

Translation into fpc-Pascal of the header:
https://github.com/fredvs/uos/blob/master/src/uos_opusfile.pas

And Opusfile.so depends on libopus.so :
https://github.com/xiph/opus

Translation into fpc-Pascal of the header:
https://github.com/fredvs/uos/blob/master/src/uos_opus.pas

Fre;D

-----
Many thanks ;-)

Doesn't some C compilers pack records differently, so packrecords is only compatiable with GNU c compiler but some other compiler could do it differently? Sorry I don't know

FPC adheres to the system's ABI. So packrecords c might result in different records on e.g. Linux or Windows or even the same OS on different platforms. So as long as the used C compiler adheres to the ABI as well, there shouldn't be any problem.

Regards,
Sven

Well the function you were using with a buffer is here:

https://github.com/xiph/opusfile/search?utf8=%E2%9C%93&q=op_test_memory

in src/opusfile.c

OggOpusFile *op_test_memory(const unsigned char *_data,size_t _size, int *_error){

    OpusFileCallbacks cb;
    return

op_test_close_on_failure(op_mem_stream_create(&cb,_data,_size),&cb,

   _error);

}

So it calls op_mem_stream_create which is:
https://github.com/xiph/opusfile/search?utf8=%E2%9C%93&q=op_mem_stream_create&type=Code

in src/stream.c

void *op_mem_stream_create(OpusFileCallbacks *_cb,
 const unsigned char *_data,size_t _size){
  OpusMemStream *stream;
  if(_size>OP_MEM_SIZE_MAX)return NULL;
  stream=(OpusMemStream *)_ogg_malloc(sizeof(*stream));
  if(stream!=NULL){
    *_cb=*&OP_MEM_CALLBACKS;
    stream->data=_data;
    stream->size=_size;
    stream->pos=0;
  }
  return stream;
}

Thanks to github's cool web gui you can find code pretty fast and share where code is with developers, compared to if it was on your hard drive you could not "share your hard drive" unless you were on emule or vnc ;-)

Hello Lars.

Many thanks to spent your time for us.

But after lot of test and re-test, my conclusion is:

op_open_memory() is buggy.

It looses the pointer assigned to the input-buffer of bytes.

And when using op_read() it points to nil in place of buffer-in.

Voila.

(But maybe I m wrong...).

Fre;D

-----
Many thanks ;-)

op_open_memory() is buggy. (But maybe I m wrong...).

Yes I was wrong.

Opus needs a bigger buffer than mp3, increasing the size fixed it.

It works, sound is perfect from a https too.
Only remain a problem: after 10 seconds of playing, op_read_float() does not give output.

I must study deeper how pipes work.

But this is a detail, the solution will follow fast.

Many thanks to everybody for your help.

Fre;D

-----
Many thanks ;-)