Hi,

I'm drawing a bitmap to the canvas by using LoadFromRawImage in combination with a TImage. The OnPaint code below works just fine. When I move the scrollbar, the images are drawn to the canvas, however there is still a bit of flicker.

procedure TForm1.Image1Paint(Sender:
TObject);
begin
Image1.Picture.Bitmap.LoadFromRawImage(bitmaplist[ScrollBar1.Position].RawImage,
False);
 Application.ProcessMessages;
end;

When I setup a timer,
for automatic looping of the images, the flicker gets so bad, that the canvas turns blank and nothing is shown anymore.

procedure
TForm1.Timer1Timer(Sender: TObject);
var
 p: integer;
begin
 p :=
ScrollBar1.Position + 1;
 if p > ScrollBar1.Max then
ScrollBar1.Position := 0
 else
 ScrollBar1.Position := p;
end;

I
understand I must be doing something wrong here. So my question remains, Given a list of bitmaps, what would be the fastest way to draw them to the screen consequtively?

Regards, Darius
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20120208/e710c9f5/attachment.html>

2012/2/8 :

Given a list of bitmaps, what would be the fastest way to draw them to the screen consequtively?

I recently had a similar thing to do and I ended up simply writing the data directly into the RawImage of the TImage.Bitmap. This is a code snippet from my program which will copy the bitmap from a camera (the image is in memory already as an array of record with red, green blue bytes, I could copy the longword directly because they colors are ordered differently):

var
    LinePointer, PixelPointer: PByte;
    Display: TImage;
    ....
begin
  with Display.Picture.Bitmap do begin
    Width := CamDimensions.Width;
    Height := CamDimensions.Height;
    Clear;
    with RawImage do begin;
      BytesPerLine := Description.BytesPerLine;
      BytesPerPixel := Description.BitsPerPixel div 8;
      RedShift := Description.RedShift;
      GreenShift := Description.GreenShift;
      BlueShift := Description.BlueShift;
    end;
  end;
    ....
    LinePointer := Display.Picture.Bitmap.RawImage.Data;
    LineOffset := 0;
    for y:=0 to CamDimensions.Height-1 do begin
      PixelPointer := LinePointer;
      for x:=0 to CamDimensions.Width-1 do begin
        with CamImage[LineOffset + x] do begin
          PInteger(PixelPointer)^ := Red << RedShift
                                  or Green << GreenShift
                                  or Blue << BlueShift;
          PixelPointer += BytesPerPixel;
        end;
      end;
      LineOffset += CamDimensions.Width;
      LinePointer += BytesPerLine;
    end;

And very important: before you call this method call BeginUpdate and after you are done EndUpdate. The last one should trigger the paint method. And don't do this from within the paint method, leave the paint method alone!

This example will most likely not directly be suitable for your bitmaps, they might not be organized the same way as my camera images but its the same principle:

Image.BeginUpdate;
Image.Picture.Bitmap.Clear;
write the data directly to Image.Picture.Bitmap.RawImage.Data^
Image.EndUpdate

The last one will then trigger the painting on the screen and *don't* use the OnPaint event yourself, leave this completely alone.

2012/2/8 Bernd :

Image.BeginUpdate; Image.Picture.Bitmap.Clear; write the data directly to Image.Picture.Bitmap.RawImage.Data^ Image.EndUpdate

Oh, I forgot: set the size, this is actually the whole point of the clear Method: force it to create this raw image of the needed size if it does not yet exist.

Image.BeginUpdate;
Image.Picture.Bitmap.Width := xxxx;
Image.Picture.Bitmap.Height := yyyy;
Image.Picture.Bitmap.Clear;
write the data directly to Image.Picture.Bitmap.RawImage.Data^
Image.EndUpdate

2012/2/8 :

I'm drawing a bitmap to the canvas by using LoadFromRawImage in combination with a TImage. The OnPaint code below works just fine. When I move the scrollbar, the images are drawn to the canvas, however there is still a bit of flicker.

I don't recommend using TImage for fast changing graphics. The really fast and reliable way to draw motion graphics is to implement a TCustomControl descendent like explained in the wiki:

http://wiki.lazarus.freepascal.org/Developing_with_Graphics#Create_a_custom_control_which_draws_itself

2012/2/8 <dhkblaszyk at zee....>:

with a TImage. The OnPaint code below works just fine. When I move the scrollbar, the images are drawn to the canvas, however there is still a bit of flicker.

I don't know the graphics system of LCL, but I thought I would ask the obvious question first. Do you have Double Buffering enabled?

Another problem might be that fcl-image (what LCL graphics is based on), is apparently the slowest graphics routines out there! I didn't do the research, but it was brought to my attention by somebody else. Henk compared various image loading and painting frameworks. He put the results on his website. He also sent me a copy of his test program (if anybody else was interested in taking a look).

http://members.upc.nl/h.speksnijder4/software/fpGUI/images.html

On Wed, Feb 8, 2012 at 9:37 AM, Graeme Geldenhuys

Another problem might be that fcl-image (what LCL graphics is based on), is apparently the slowest graphics routines out there!

fcl-image routines are utilized in the LCL mostly only for load/saving to/from disk. That table is also for disk operations. And I don't see any save/load operation in the description of the problem.

Felipe Monteiro de Carvalho wrote

fcl-image routines are utilized in the LCL mostly only for load/saving to/from disk.

As I mentioned, I don't know the graphics stack of the LCL.

BTW:
You didn't touch on my other point. Is double buffering enabled by default for all painting routines in the LCL? Or must the end-user / developers specifically enable it somewhere?

Graeme Geldenhuys wrote

You didn't touch on my other point. Is double buffering enabled by default for all painting routines in the LCL? Or must the end-user / developers specifically enable it somewhere?

He is using TImage, which is not built for fast changing graphics, it is built for displaying a static image or images which change sometimes only, so he is already using the wrong architecture.

But the answer would be: depends. There is a DoubleBuffer property which affects only Windows. I think that all other widgetsets always do Double-buffering without an option to disable it. But TImage probably already defines DoubleBuffer. His flickering could be due to partial updates or background erasing. But summing up, his problem is using the wrong component for the job, not the DoubleBuffer propery.

Why are you calling "Application.ProcessMessages" here?

Regards,
Sven

i've learned before to do the double buffer manually when using timage. i think it's the more secure way, you don't need to guess if the widgetset do itself....

but, obviously, depends on the context...
________________________________
William de Oliveira Ferreira
Bacharel em Sistemas de Informa??o

2012/2/8 Sven Barth

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20120208/8ae9faf4/attachment.html>

Well afaik that's exactly what I'm doing with

Image1.Picture.Bitmap.LoadFromRawImage(bitmaplist[ScrollBar1.Position].RawImage, False);

The flicker went away by placing the code outside the OnPaint handler. I can now draw with about 11fps. The images are streamed continuously from disk! So performance is not really bad, but I would like it to be better.

Regards, Darius

I've did some tests with a tcustomcontrol, but that hardly gave me any speed benefit. Both methods are drawing at 11fps

Regards, Darius

Thanks Graeme. Very interesting indeed. I could not find the imlib2 binding code. Do you have it somewhere? Although I didn't ask for loading, it could actually help. Now I stream pngs from dist to a TPortableNetworkGraphic object. Perhaps this is the bottleneck? I will profile my code later. But it might be a solution.

Regards, Darius

As said, hardly any speed improvement noticed.

Regards, Darius

Because otherwise the screen just does not update.

Darius

Good catch. I guess the OP has used this previously to repaint controls/images during a long-running procedure... (and even there it's often not the best solution)
@dhlblaszyk: when you want to force the repaint of the whole control, call Image1.Repaint. If you want to just repaint some invalidated part of it, call Image1.Update (not sure TGraphicControl has Update, so you may need to use Parent.Update instead). Of course you should do this *outside* the OnPaint. Also, is Image1.Transparent set to false? BTW what do you mean by "The images are streamed continuously from disk"?

Regards,
Fl?vio

You can download Henk Speksnijder's test application from the URL below. 150K download - 4th link from the top. It contains the imlib2 bindings too.

http://members.upc.nl/h.speksnijder4/software/softwareindex.html

If you wanted to contact Henk directly, you can contact him at:

I have played with Image.Update, .Invalidate and .Repaint, but none of them seem to work for me. Only when I put Application.ProcessMessages the painted images show on screen. See below for the testloop code. The images are streamed from a file cache. For 24 frames this seems overkill, but running at 24fps the memeory requirements quickly get very large!

Image.Transparent is also set to false btw, which is default. Otherwise the drawing on the canvas is even messed up.

Regards, Darius

procedure TForm1.Button1Click(Sender: TObject);
var
 s: TDateTime;
 i:
Integer;
begin
 s := now;
 for i := 1 to 24 do
 begin
 if
filecache_get_from_cache(i, png) then
 begin
Image1.Picture.Bitmap.LoadFromRawImage(png.RawImage, False);
Application.ProcessMessages;
 end;
 end;

ShowMessage(FloatToStr(24 /
((now - s) * 24 * 3600)))
end;

Fl?vio Etrusco wrote

Sven Barth wrote

> Am 08.02.2012

01:07, schrieb : [1]

> >> Hi, I'm drawing a

bitmap to the canvas by using LoadFromRawImage in combination with a TImage. The OnPaint code below works just fine. When I move the scrollbar, the images are drawn to the canvas, however there is still a bit of flicker. procedure TForm1.Image1Paint(Sender: TObject); begin Image1.Picture.Bitmap.LoadFromRawImage(bitmaplist[ScrollBar1.Position].RawImage, False); Application.ProcessMessages; end;

> Why are you calling

"Application.ProcessMessages" here? Regards, Sven

Good catch. I

guess the OP has used this previously to repaint

controls/images

during a long-running procedure... (and even there

it's often not the

best solution)

@dhlblaszyk: when you want to force the repaint of the

whole control,

call Image1.Repaint. If you want to just repaint some

invalidated part

of it, call Image1.Update (not sure TGraphicControl

has Update, so you

may need to use Parent.Update instead). Of course

you should do this

*outside* the OnPaint. Also, is Image1.Transparent

set to false?

BTW what do you mean by "The images are streamed

continuously from disk"?

Here it is clear that you need to use Application.ProcessMessages, because "LoadFromRawImage" will only tell the control that it needs to repaint itself, but does not execute this repainting (this is done in Application.ProcessMessages). In the example you posted in the beginning the call to Application.ProcessMessages is located inside the OnPaint handler of the Image which is - in my opinion - not good. Maybe also the loading of the image inside the OnPaint handler is not good...

Maybe you should try - like Felipe suggested - something else than TImage. Try for example TPaintBox in combination with the Application.OnIdle event:

procedure TForm1.ApplicationOnIdle(aSender: TObject; var aDone: Boolean);
begin
   LoadNextImageFromCache; // this will load the next image into some
private variable of the form (let's call it fImage). [this is based on
your filecache_get_from_cache any you'll need to write it of course ;) ]
   PaintBox1.Invalidate;
   aDone := False; // important!
end;
procedure TForm1.PaintBox1Paint(aSender: TObject);
begin
   PaintBox1.Canvas.Draw(fImage); // or however you'll get the content
of the fImage onto the canvas of the PaintBox
end;

Note: Don't forget to assign your ApplicationOnIdle to Application.OnIdle or use the ApplicationProperties component located in the Additional tab.

Regards,
Sven

Despite all the advices, none of the suggestions of drawing on a TPaintBox or creating a TCustomControl actually speed up the painting significantly. Therefore I did some profiling. It appears that loading a 40kb png file (50% HD) from a stream takes about 60msec another 30msec is used to paint the bitmap to screen. It all results in approx. 11fps which is actually pretty poor when you imagine a full HD loop will make the rate drop 2.5fps.

As I see it now, I will need to lower the access time to the png's from disk. I will do a test with storing only raw pixel data instead. This means that there will be no conversion (or decompression in the case of PNG) when streaming from disk. However, although the minor part of the total loop, I was hoping to improve the painting speed significantly as well. Hopefully someone else has an idea that works. Is there a way to access even lower level functions from LCL? Like GDI and GTK2 functions to copy the pixel data as fast as possible?

Appreciate the help from everybody so far.

Regards, Darius

Darius Blaszyk wrote

Despite all the advices, none of the suggestions of drawing on a TPaintBox or creating a TCustomControl actually speed up the painting significantly. Therefore I did some profiling. It appears that loading a 40kb png file (50% HD) from a stream takes about 60msec another 30msec is used to paint the bitmap to screen. It all results in approx. 11fps which is actually pretty poor when you imagine a full HD loop will make the rate drop 2.5fps.

the first thing that comes to my mind is hardware related... what hardware are you using? even with "fast" hardware, it is possible that FPS is very slow... as an example, i have numerous tools that draw to the screen on my laptop much slower than if they are run on a dedicated desktop machine with graphic intensive hardware and then there is IF graphic hardware routines are used or not...

As I see it now, I will need to lower the access time to the png's from disk.

handling this bottleneck is always a GoodThing<tm> ;)

I will do a test with storing only raw pixel data instead. This means that there will be no conversion (or decompression in the case of PNG) when streaming from disk. However, although the minor part of the total loop, I was hoping to improve the painting speed significantly as well.

you can't go any faster than the data is made available, FWTW...

Hopefully someone else has an idea that works. Is there a way to access even lower level functions from LCL? Like GDI and GTK2 functions to copy the pixel data as fast as possible?

how far are you in to multiplatform stuffs? is your program also tuned to the winwhatever world? the reason i ask is because it is possible that some things may be slower in the other "world" than that you are developing in...

I seem to remember some code that was in the NativeJPG examples that loaded the file into a TMemoryStream first, then did a LoadfromSteam instead of a LoadfromFile. It might be that the loadfromfile (or in your case LoadFromRawImage) is slower than a
TMemorysteam.loadfromfile+Image.LoadfromStream. I admit I have never tried as I have never needed it, but it might help. Please let us know if there is a speed improvement.
Ciao
brett

My 2 cents.

Try to load in memory some png first and descompress them, when the buffer is full, launch other thread that only get data from the buffer and paint it while the first thread fills the buffer with more files, your bottleneck is always loading and decoding png files from disk.

Plase let us know your improvements,
Alex

El 09/02/2012 3:33, Brett Hunter escribi?:

Darius Blaszyk wrote

Despite all the advices, none of the suggestions of drawing on a TPaintBox or creating a TCustomControl actually speed up the painting significantly. Therefore I did some profiling. It appears that loading a 40kb png file (50% HD) from a stream takes about 60msec another 30msec is used to paint the bitmap to screen. It all results in approx. 11fps which is actually pretty poor when you imagine a full HD loop will make the rate drop 2.5fps.

Darius. If you could send me a sample of 25 images in private, I'll try something here on my side. I'll also plug them into a fpGUI application so we can see if there is a difference between a LCL app and fpGUI app - trying to determine where the bottlenecks are... loading, graphics stack, toolkit etc. I also have 3 different speeds hard drives (and various file systems) in my system. That could also be used to profile the loading times from each hard drive.

If you have a sample LCL app that you use for testing, that would save me some trouble too. Though the Button1Click() code you posted earlier is pretty straight forward. :)

Hi Alex,

I will add memory buffering as well of course, but on longer sequences, this will help little. The memory buffer will run empty before a second thread will fill it with new images.

Darius

On

Alejandro P?rez wrote

My 2 cents. Try to load in

memory some png first and descompress them, when the

buffer is full,

launch other thread that only get data from the buffer

and paint it

while the first thread fills the buffer with more files,

your

bottleneck is always loading and decoding png files from disk.

Plase let us know your improvements,

Alex El 09/02/2012 3:33,

Brett Hunter escribi?:

> I seem to remember some code that was in

the NativeJPG examples that loaded the file into a TMemoryStream first, then did a LoadfromSteam instead of a LoadfromFile. It might be that the loadfromfile (or in your case LoadFromRawImage) is slower than a TMemorysteam.loadfromfile+Image.LoadfromStream. I admit I have never tried as I have never needed it, but it might help. Please let us know if there is a speed improvement. Ciao brett -----Original Message----- From: Darius Blaszyk [mailto: [3]] Sent:
Thursday, 9 February 2012 11:52 AM To: Lazarus mailing list Subject: Re: [Lazarus] Fast drawing to canvas Despite all the advices, none of the suggestions of drawing on a TPaintBox or creating a TCustomControl actually speed up the painting significantly. Therefore I did some profiling. It appears that loading a 40kb png file (50% HD) from a stream takes about 60msec another 30msec is used to paint the bitmap to screen. It all results in approx. 11fps which is actually pretty poor when you imagine a full HD loop will make the rate drop 2.5fps. As I see it now, I will need to lower the access time to the png's from disk. I will do a test with storing only raw pixel data instead. This means that there will be no conversion (or decompression in the case of PNG) when streaming from disk. However, although the minor part of the total loop, I was hoping to improve the painting speed significantly as well. Hopefully someone else has an idea that works. Is there a way to access even lower level functions from LCL? Like GDI and GTK2 functions to copy the pixel data as fast as possible? Appreciate the help from everybody

none of them seem to work for me. Only when I put
Application.ProcessMessages the painted images show on screen. See below for the testloop code. The images are streamed from a file cache. For 24 frames this seems overkill, but running at 24fps the memeory requirements quickly get very large! Image.Transparent is also set to false btw, which is default. Otherwise the drawing on the canvas is even messed up. Regards, Darius procedure TForm1.Button1Click(Sender: TObject); var s: TDateTime; i: Integer; begin s := now; for i := 1 to 24 do begin if filecache_get_from_cache(i, png) then begin Image1.Picture.Bitmap.LoadFromRawImage(png.RawImage, False); Application.ProcessMessages; end; end; ShowMessage(FloatToStr(24 / ((now
- s) * 24 * 3600))) end;

>> Here it is clear that you need to use

Application.ProcessMessages, because

> "LoadFromRawImage" will only

tell the control that it needs to repaint itself, but does not execute this repainting (this is done in Application.ProcessMessages). In the example you posted in the beginning the call to
Application.ProcessMessages is located inside the OnPaint handler of the Image which is - in my opinion - not good. Maybe also the loading of the image inside the OnPaint handler is not good...

> >> Maybe you

should try - like Felipe suggested - something else than TImage.

> Try

for example TPaintBox in combination with the Application.OnIdle event:
procedure TForm1.ApplicationOnIdle(aSender: TObject; var aDone:
Boolean); begin LoadNextImageFromCache; // this will load the next image
into

> >> y you'll need to write it of course ;) ]

PaintBox1.Invalidate; aDone := False; // important! end; procedure
TForm1.PaintBox1Paint(aSender: TObject); begin
PaintBox1.Canvas.Draw(fImage); // or however you'll

> nt of the fImage

onto the canvas of the PaintBox end; Note: Don't forget to assign your ApplicationOnIdle to Application

> >> type="cite"

style="padding-left:5px; border-left:#1010ff 2px solid; margin-left:5px;
width:100%">Regards, Sven --

I'm working on a test app that I will put on the CCR on sourceforge for everyone to test. Will put it up later this afternoon. I will challenge everyone to come up with speed improvements! :)

Regards,
Darius

Graeme Geldenhuys wrote

On 9 February 2012

Darius Blaszyk wrote

> Despite all the advices, none of the

suggestions of drawing on a TPaintBox or creating a TCustomControl actually speed up the painting significantly. Therefore I did some profiling. It appears that loading a 40kb png file (50% HD) from a stream takes about 60msec another 30msec is used to paint the bitmap to screen. It all results in approx. 11fps which is actually pretty poor when you imagine a full HD loop will make the rate drop 2.5fps.

Darius. If you could send me a sample of 25 images in private, I'll

try something here on my side. I'll also plug them into a fpGUI

application so we can see if there is a difference between a LCL app

and fpGUI app - trying to determine where the bottlenecks are...

loading, graphics stack, toolkit etc. I also have 3 different speeds

hard drives (and various file systems) in my system. That could also

be used to profile the loading times from each hard drive.

If you

have a sample LCL app that you use for testing, that would save

me

some trouble too. Though the Button1Click() code you posted earlier

is

pretty straight forward. :)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20120209/8e52f5a4/attachment-0001.html>

Darius Blaszyk wrote

Despite all the advices, none of the suggestions of drawing on a TPaintBox or creating a TCustomControl actually speed up the painting significantly. Therefore I did some profiling. It appears that loading a 40kb png file (50% HD) from a stream takes about 60msec another 30msec is used to paint the bitmap to screen. It all results in approx. 11fps which is actually pretty poor when you imagine a full HD loop will make the rate drop 2.5fps.

A quick test on my system, using my own "animation" images.

* using the fpGUI toolkit
* Each BMP image is 140kb (fpGUI doesn't have PNG support yet)
* each image is loaded one-by-one. No images are cached.
* just before a new image is loaded, the old image is freed.
* Partition has the ReiserFS filesystem under Linux 64-bit
* hard drive is an avg Seagate 250GB sata drive.

Initial framerate was 1500 fps. I couldn't actually see any animation at that speed, and had to add writeln(i) statements to make sure I am loading all images - which it did. :-)

I then added a sleep (10) between each image load/paint job. Animation was now visible to the eye and a sleep(1)
gives me 577fps.

I don't know what resolution your images are, so I simply used the 40kb file size you mentioned as the bare minimum. I took a screenshot of my desktop and resized the image and saved it to PNG until I got +- a 40kb file. I then created a 26 frame animation using that image an the background. I then converted those images to BMP, so fpGUI could read them. PNG to BMP made the images go from 40kb to 140kb in size.

I'll wait for your sample images before I take another look. But from my initial testing, there is something seriously screwy in your case to get only 11fps. I can't imagine loading a 40kb PNG vs a 140kb BMP can make up that a huge difference in framerate.

This may be the problem: you should force the display of every loaded image. Only then it's guaranteed that all conversions are performed, required to show the images on screen.

I'll wait for your sample images before I take another look. But from my initial testing, there is something seriously screwy in your case to get only 11fps. I can't imagine loading a 40kb PNG vs a 140kb BMP can make up that a huge difference in framerate.

Hardware support may be a bottleneck. Can you disable graphics hardware acceleration on your system?

Many graphics adapters require proprietary drivers for hardware acceleration, which are not installed by default on Linux systems.

DoDi

Hi Graeme,

1500fps is massively impressive. Unfortunately I'm behind a firewall atm, but as soon as I can I will upload my test project. It shows the different methods proposed by everyone and actually times them as well. The size I have been testing on is 50%HD ie 960x540 and HD. The footage will also be uploaded to SVN.

The 11fps is
indeed too low and there must be something wrong. I would have expected (hoped for) something like 30fps for full , Darius

Graeme Geldenhuys wrote

On 9 February 2012 02:52,

Darius Blaszyk wrote

> Despite all the advices, none of the

suggestions of drawing on a TPaintBox or creating a TCustomControl actually speed up the painting significantly. Therefore I did some profiling. It appears that loading a 40kb png file (50% HD) from a stream takes about 60msec another 30msec is used to paint the bitmap to screen. It all results in approx. 11fps which is actually pretty poor when you imagine a full HD loop will make the rate drop 2.5fps.

A

quick test on my system, using my own "animation" images.

* using

the fpGUI toolkit

* Each BMP image is 140kb (fpGUI doesn't have PNG

support yet)

* each image is loaded one-by-one. No images are

cached.

* just before a new image is loaded, the old image is freed.

* Partition has the ReiserFS filesystem under Linux 64-bit

* hard

drive is an avg Seagate 250GB sata drive.

Initial framerate was

1500 fps. I couldn't actually see any animation

at that speed, and had

to add writeln(i) statements to make sure I am

loading all images -

which it did. :-)

I then added a sleep (10) between each image

load/paint job. Animation

was now visible to the eye and running at

92fps. Having a sleep(1)

gives me 577fps. I don't know what

resolution your images are, so I simply used the

40kb file size you

mentioned as the bare minimum. I took a screenshot

of my desktop and

resized the image and saved it to PNG until I got +-

a 40kb file. I

then created a 26 frame animation using that image an

the background.

I then converted those images to BMP, so fpGUI could

read them. PNG to

BMP made the images go from 40kb to 140kb in size.

I'll wait for

your sample images before I take another look. But from

my initial

testing, there is something seriously screwy in your case

to get only

11fps. I can't imagine loading a 40kb PNG vs a 140kb BMP

can make up

that a huge difference in framerate.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20120209/8e7157f8/attachment-0001.html>

The situation has improved in the recent years. At least the open source ATI drivers have a rather good hardware acceleration support (though maybe not on the newest cards). E.g. on my Mobility Radeon 9700 they outperform the last binary release for that card from around 2009.

Regards,
Sven

Hans-Peter Diettrich wrote

This may be the problem: you should force the display of every loaded image. Only then it's guaranteed that all conversions are performed, required to show the images on screen.

Yes, that is why I have a call to fpgApplication.ProcessMessages between every frame. This give the windowing system chance to draw the frame before the next iteration. Even with that call (and without a Sleep() call), I got 1500fps.

Hardware support may be a bottleneck. Can you disable graphics hardware acceleration on your system?

fpGUI doesn't use hardware also don't have it
enable on my Linux desktop (no 3D desktop effects). I am using the open source ATI driver in X11.

2012/2/9 <dhkblaszyk at ....>:

1500fps is massively impressive. Unfortunately I'm behind a firewall atm, but as soon as I can I will upload my test project.

No worries.

I also did timing around the loading of a 140kb BMP, which took 1ms per image. I then did timing on painting each image to the screen. That too, took 1ms. I used EpikTimer for my timing results.

have been testing on is 50%HD ie 960x540 and HD.

Those must be pretty boring pictures then! :-) By that I mean, to reduce a 960x540 image to a 40kb PNG. A screenshot of my desktop at that resolution gives me a 600kb PNG image. :)

2012/2/9 <dhkblaszyk at ze*****>:

methods proposed by everyone and actually times them as well. The size I have been testing on is 50%HD ie 960x540 and HD. The footage will also be uploaded to SVN.

OK, recreated my test images at 960x540 resolution. They are now 1.5 Mbyte BMP images. Loading time of each image is between 7-9ms (never higher than 9ms). Painting is still constant at 1ms per image.

Running the 26 frame animation, I get 104fps without any Sleep() call. So your target of 30fps is definitely achievable.

I've put up the test app on the CCR. All three methods draw at roughly 10fps on windows and 6fps on mac. I haven't tried linux yet. I challenge everyone to find methods to speed up the the drawing using the current implementation. May the best man win!

Regards, Darius

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20120209/57f23bbc/attachment.html>

Darius Blaszyk wrote

I've put up the test app on the CCR. All three methods draw at roughly 10fps on windows and 6fps on mac. I haven't tried linux yet. I challenge everyone to find methods to speed up the the drawing using the current implementation. May the best man win!

On Linux I have roughly 5 FPS with all three methods... :(

I then tried to display the pictures using the OpenGLContext. For this I modified your code quite a bit:
* the images are converted to a "simple image" (a record consisting of width, height and the image data as an array of a record of 3 byte values (red, green, blue))
* the images are kept in memory (no writing to a cache on disk the use of which I don't really understand...)
* draw the images to the OpenGLContext

With this I achieve roughly 25 FPS using the 50% HD images, but the startup time (the conversion from the PNG image to the simple image) is around 5 to 10 minutes... (maybe .Canvas.Pixel[...] access isn't that good for that purpose ^^)

Regards,
Sven

Sven Barth wrote

(maybe .Canvas.Pixel[...] access isn't that good for that purpose ^^)

One should basically almost never use Canvas.Pixels in LCL-Gtk2, unless you want to draw some points, because the image is located in the X server.

Draw first to a TLazIntfImage and then convert it to a TBitmap.

Qt in native mode under X11 (not graphicssystem raster) also fits here.

zeljko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20120210/0aa217f6/attachment.html>

Right... didn't thought of that... I'll try that today evening when I'm at home again (where I have the code).

But why does the canvas of a TGraphic descendant (TBitmap, TPortableNetworkGraphic, etc.) need to store the data on the X-Server? Wouldn't it be enough to transport that only when it's displayed?

Regards,
Sven

2012/2/9 Darius Blaszyk :

I've put up the test app on the CCR. All three methods draw at roughly 10fps on windows and 6fps on mac.

Using your project as-is, I get 9fps under Linux. Also I have a 10-20 seconds delay just for the program to start up - I didn't investigate this.

I then went and modified your TImage code to look as shown below... Basically this is so I can do a direct comparison to my fpGUI test project. It now loads 1.5Mbyte BMP versions of your images. This gives me a 7fps animation. Yuck! Oh, and I had to enable double buffering, otherwise I would have gone into shock. ;-)

----------------------------
procedure TForm1.mnuTImageClick(Sender: TObject);
var
  s: TDateTime;
  i: integer;
  j: integer;
  f: boolean;
begin
  PageControl1.ActivePage := ImageTabSheet;
  s := Now;
    for i := 1 to 23 do
    begin
        Image1.Picture.LoadFromFile(Format('./images_50/%.4d.bmp', [i]));
        Application.ProcessMessages;
    end;
  AddTestToListBox('TImage', 23 / ((Now - s) * 24 * 3600));
end;
----------------------------

So this got me thinking. We already know that fcl-image really really sucks at loading images from disk. fpGUI proves with it's own JPG and BMP loading code, that it could be done in a fraction of the time it takes fcl-image.

So to prove this point further, I quickly implement PNG support in fpGUI, by using the fcl-image loading code, and populating a TfpgImage instead. Now my fpGUI based animation test can use BMP, JPG and PNG images.

The result....using 50% HD images.

BMP (native) in fpGUI.... 100fps
JPG (native) in fpGUI... 37fps
PNG (via fcl-image) in fpGUI.... 8fps

So this clearly shows one huge bottleneck... Loading via fcl-image is very bad! The FPC team really needs to investigate this further.

Unfortunately I haven't implemented native PNG loading in fpGUI yet, but you are welcome to port the BMP or JPG code to your LCL application. This shouldn't be hard, because the code doesn't have much to do with GUI toolkits. Though that means your images need to be converted too - which I gather you would rather not want to do.

My fpGUI animation test project and the image samples can be downloaded from the following URL. If your ISP bans DynamicDNS sites, email me in private and I'll send it to you directly.

http://opensoft.homeip.net:8080/~graemeg/animation_test.tar.gz (2Mb)

Sven Barth wrote

But why does the canvas of a TGraphic descendant (TBitmap, TPortableNetworkGraphic, etc.) need to store the data on the X-Server? Wouldn't it be enough to transport that only when it's displayed?

But then it would not a Gtk native bitmap and would not be a native X image. Remember, existing widgetsets try to be as native as possible and TBitmap has a Handle property to access the native Handle.

In LCL-CustomDrawn I went on the opposite direction. There TCanvas.Pixels[] is just a front end to TLazIntfImage.Colors[] so Pixels[] is fast and executes no X11 operations. On each OnPaint event only 1 single X11 operation is executed: upload the final form image. But of course it still has a huge way to go before getting usable in desktop applications.

Hi Sven,

I just comitted a second version, which is also based on a "simple image" as you put it. At 50% HD I get 88fps on windows and 19fps on Mac. Couldn't test on Linux as that machine refuses to work properly. I have not used OpenGL but TImage instead. In other tests I have never seen any actual speed improvements of OpenGL over a native canvas, unless you create memory textures and can utilize the hardware acceleration.

The reason to stream directly form disk is that if I would like to display a longer list of images I will not be able to store them in memory. Say I have a movie clip @ 24fps for 60sec. This would require (50%HD) 24 * 60 * 1.5 = 2GB of memory. What if I would like to display 10min? No chance... Please note that I really need the full resolution images here, so converting to jpeg to reduce memory footprint is out of the question. Implementing a memory cache is also on my list, but when playing the entire loop from start to end this will not help as each frame is displayed only once of course.

In the "fast"
version I stream data from a png and this is actually pretty fast now. However I will implement native loading of the images to optimize the process. But this is only optionally as my issue is not with the setup of the file cache but more with the drawing speed. As said, on windows I get 50%HD @ 88fps and full HD @35fps. Now Mac is worrying as it draws full HD @ only 12fps. Maybe opengl can draw more efficienctly in this case? I would also like to test SDL.

Regards, Darius

Sven Barth wrote

Am 09.02.2012 19:41, schrieb Darius Blaszyk: >

I've put up the test app on the CCR. All three methods draw at roughly 10fps on windows and 6fps on mac. I haven't tried linux yet. I challenge everyone to find methods to speed up the the drawing using the current implementation. May the best man win!

On Linux I have roughly 5 FPS

with all three methods... :(

I then tried to display the pictures

using the OpenGLContext. For this I

modified your code quite a bit:

* the images are converted to a "simple image" (a record consisting of

width, height and the image data as an array of a record of 3 byte

values (red, green, blue))

* the images are kept in memory (no writing

to a cache on disk the use

of which I don't really understand...) *

draw the images to the OpenGLContext

With this I achieve roughly 25

FPS using the 50% HD images, but the

startup time (the conversion

from the PNG image to the simple image) is

around 5 to 10 minutes...

(maybe .Canvas.Pixel[...] access isn't that

good for that purpose

^^)

Right...

Hmm... maybe I should try that with LCL-CustomDrawn ^^

Do the following components work already?

* TPaintBox
* TOpenGLContext
* TImage
* TCustomControl (and descendants)

Regards,
Sven

Hi Graeme,

I will test the app you made on Mac as well. On
Windows I think the performance will be similar now.

As for the delay
in startup, don't worry that is the file cache being created. It's not more than a stream of raw pixel data for each frame and a simple list of stream references for each frame. In my "new" version which is also in CCR this loading is actually pretty fast now. It takes a sec or 2 to setup. I'm sure this could be improved, but that is only a minor problem for now.

Regards, Darius

Graeme Geldenhuys wrote

2012/2/9 Darius Blaszyk :

> I've put up the test app on the CCR. All

three methods draw at roughly 10fps on windows and 6fps on mac.

Using your project as-is, I get 9fps under Linux. Also I have a 10-20

seconds delay just for the program to start up - I didn't investigate

this.

I then went and modified your TImage code to look as shown

below...

Basically this is so I can do a direct comparison to my fpGUI

test

project. It now loads 1.5Mbyte BMP versions of your images. This

gives

me a 7fps animation. Yuck! Oh, and I had to enable double

buffering,

otherwise I would have gone into shock. ;-)

----------------------------

procedure TForm1.mnuTImageClick(Sender:

TObject);

boolean;

begin PageControl1.ActivePage := ImageTabSheet; s :=

Now;

for i := 1 to 23 do begin

Image1.Picture.LoadFromFile(Format('./images_50/%.4d.bmp', [i]));

Application.ProcessMessages;

end; AddTestToListBox('TImage', 23 /

((Now - s) * 24 * 3600));

end; ---------------------------- So

this got me thinking. We already know that fcl-image really really

sucks at loading images from disk. fpGUI proves with it's own JPG and

BMP loading code, that it could be done in a fraction of the time it

takes fcl-image.

So to prove this point further, I quickly

implement PNG support in

fpGUI, by using the fcl-image loading code,

and populating a TfpgImage

instead. Now my fpGUI based animation test

can use BMP, JPG and PNG

images. The result....using 50% HD

images.

BMP (native) in fpGUI.... 100fps JPG (native) in fpGUI...

37fps

PNG (via fcl-image) in fpGUI.... 8fps So this clearly shows

one huge bottleneck... Loading via fcl-image is

very bad! The FPC team

really needs to investigate this further.

Unfortunately I haven't

implemented native PNG loading in fpGUI yet,

but you are welcome to

port the BMP or JPG code to your LCL

application. This shouldn't be

hard, because the code doesn't have

much to do with GUI toolkits.

Though that means your images need to be

converted too - which I

gather you would rather not want to do.

My fpGUI animation test

project and the image samples can be

downloaded from the following

URL. If your ISP bans DynamicDNS sites,

email me in private and I'll

send it to you directly.

http://opensoft.homeip.net:8080/~graemeg/animation_test.tar.gz [1] (2Mb)

Links:
------
[1]
http://opensoft.homeip.net:8080/~graemeg/animation_test.tar.gz -------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20120210/5a9ca8b9/attachment.html>

At least on Windows that sounds like a nice improvement (though not necessarily perfect ^^).

That's a valid reason, but I removed the disk cache to test pure drawing performance.

I don't know whether OpenGL is really more efficiently, I just wanted to throw in an alternative variant.

Regards,
Sven

sdl might help: http://www.libsdl.org/

-Michael

Sven Barth wrote

Do the following components work already? * TPaintBox

Never tested, probably yes.

* TOpenGLContext

No.

* TImage * TCustomControl (and descendants)

Yes for both

But note that text rendering was implemented only for Android and Win32 at the moment. (my most important initial target is Android) But we already have part of the work done for cross-platform text rendering via LazFreeType.

Would be nice if that even gets supported for Android :)

As we're currently only talking about image rendering I see no problem here with missing font support ;)

Regards,
Sven

Man, that's slow for just drawing an image every frame. I would really be interested how the 'draw the images to the OpenGLContext' part looks like.

R.

That's a valid reason, but I removed the disk cache to test pure drawing performance.

In that case 25FPS at 50% HD is very poor. I would expect at least 3 times as much (see also my test app). How do you draw the pixels? Do you use glBitmap or do you use GL_POINTS? Could you share the test application you've made with me?

Of course, evaluate all possibilities. I'm hoping that OpenGL will be faster over all platforms, as I've seen substantial performance loss when using the Mac canvas.

Regards, Darius

properly. I have not used OpenGL but TImage instead. In other tests I have never seen any actual speed improvements of OpenGL over a native canvas, unless you create memory textures and can utilize the hardware acceleration.

What is the difference between a texture and a memory texture? I never heard the second term.

Why are you displaying single images at all?
It sounds a lot like you are playing movie clips, so why do you not use a suitable codec/container combination like xvid/avi, h.264/mpeg-4 or some other video format that fits your needs?

R.

Sven Barth wrote

Would be nice if that even gets supported for Android :)

I started the Android backend with OpenGL ES, but then it prooved to be way more work and zero advantage over directly rendering on an Android View. Nope, it is not faster, at least not for 2D bitmap drawing, that seams to be a myth. In X11 it is also not faster, because it just uses the same image putting methods which any library would use to talk to X11. And to make things worse OpenGL ES 2 removed all the good functions to make 2D projections and changed everything. So I gave up on it and went with the standard Android View which works nicely. But I suppose one could put another View inside my main view and render OpenGL there.

But remember the major catch: Android does not have OpenGL, it has OpenGL ES, which from what I saw was immensely different from normal OpenGL and way harder to use. This fact will also probably kill the changes of porting TOpenGLContext to Android because this control was made for desktop OpenGL. We would need a TOpenGLES2Context.

This salad of incompatible OpenGL libraries is really a terrible thing.

I'll share it when I'm at home tonight. I just made this yesterday before going to bed, so I didn't want to send anything anymore ;)

If I remember correctly I use glBitmap, but you'll see later.

Regards,
Sven

I mostly want it for normal OpenGL usage (games, etc) not for custom drawing myself...

Too many things to do and so less time *sigh*

Regards,
Sven

I was talking about glDrawPixels (CPU) and glBindTexture (GPU). The latter being much faster of course.

Yes, I'm playing movie clips. But playing back images from a video codec is besides the point. I have implemented my own "simple" raw image codec (filecache.pas) that reads pixels from disk. My problems lie with the playback of video. It has proven to be very difficult to playback video (on Mac) with an acceptable speed. Full HD images are played back @ 11fps wich is just too slow. Creating memory caches will not help in this situation because they will have to be so large (in the order of 1GB just to playback 60sec) that the system will grind to a halt.

Regards, Darius

My problems lie with the playback of video. It has proven to be very difficult to playback video (on Mac) with an acceptable speed.

Yes, the Virtual Magnifying Glass has the same problem. I currently have no idea why LCL-Carbon drawing is so slow. Probably OpenGL will help here.

> On Fri, 10 Feb 2012 11:34:20 +0100 > wrote:

[...]

If I recall correctly, programs like MPlayer (or was it FFMpeg?) had an 'input' that could read static images and output video stream to stdout (but I might not remember correctly, pls check docs)
Then the video stream could be piped into the player program,... so FFMpeg reads series of images - pipes into MPlayer that displays it- or MPlayer can be told to read series of images as it were frames - is that what you're trying to achieve ?

L.

I was talking about glDrawPixels (CPU) and glBindTexture (GPU). The latter being much faster of course.

glDrawPixels should be avoided as much as possible. Btw. glBindTexture is not really comparable to glDrawPixels, glTex(Sub)ImageXD is the comparable function with the sub one mostly being faster even if you update a complete texture.

Yes, I'm playing movie clips. But playing back images from a video codec is besides the point.

If you say so, but that is exactly what you are doing.

The fastest way should be using OpenGL. But not some naive implementation like it seems to be the case with the OpenGL tests performed by followers of this thread. The magic words are PBO and texture streaming. This approach reads the data directly into a buffer on the graphics card which you use as texture to display your images. You can even read the data in an other thread than the OpenGL one. Only the reading of the data is performed on the CPU and the data will go from disk straight to VRAM.

hih
R.

I think your referring to a frameserver.

Then the video stream could be piped into the player program,... so FFMpeg reads series of images - pipes into MPlayer that displays it- or MPlayer can be told to read series of images as it were frames - is that what you're trying to achieve ?

No, I'm working on an NLA editor and compositor actually. In this editor images can be loaded from a movie file (already got the ffmpeg header file from MiSchi) , but other "devices" are also supported such as photos and generated images.

The editor needs to be able to playback the results and this was the reason for my original post. The playback was simply too slow.

Regards, Darius

Nope, it is not faster, at least not for 2D bitmap drawing, that seams to be a myth.

Sorry to tell you, but OpenGL is only fast if you know what you are doing (no offense). If you try to use it as you would use your standard software graphics (TPaintBox and friends for example) it will be slower most of the time.

And to make things worse OpenGL ES 2 removed all the good functions to make 2D projections and changed everything.

2D projection functions have never been part of OpenGL AFAIK, mind telling me which functions you talk about? OpenGL ES 2.0 did not change everything. It just removed the fixed function pipeline (as OpenGL 3 did) in favor of the programmable one, which is a Good Thing(tm). And nobody forces you to use ES 2 just use 1.x if you like it more.

But remember the major catch: Android does not have OpenGL, it has OpenGL ES, which from what I saw was immensely different from normal OpenGL and way harder to use.

It is not, the two are very, very similar.

This salad of incompatible OpenGL libraries is really a terrible thing.

This is what it may look like for a beginner. The problem is just that you don't learn OpenGL in a day or a week. It is a very powerful graphics library but as with most powerful stuff it is not as easy to use as many people wish.

Your frustration is understandable, I just want to put you "rant" into perspective.

greetings
R.

P.S.: Btw, thanks for your work on Android, so I did not had to do it.

> Yes, I'm playing movie clips. But playing back images from a video codec > is besides the point. If you say so, but that is exactly what you are doing.

I know :) The streaming is actually pretty fast. Tests have shown that I'm able to get almost 90fps (for the complete drawing cycle) on 50%HD on windows which is perfect. However the same loop drops down to 11fps on mac. So the problem lies not in the streaming but in the drawing bit. After I solve the drawing problems (perhaps by switching to OpenGL as you also mention) I will focus on the filecache. There's a tradeoff between compression/decompression and disc access times. I just don't know where the turning point is, but tests will show that later. For now I'll stick to raw pixel data, later I will use a movie file probably (with optional lossless compression).

Your comments certainly help! I think you will probably be right for Mac. I expect that OpenGL will be more stable over the different platforms as the TCanvas implementation seems to be. However on Windows and Ubuntu OpenGL PBO does not win from TCanvas as demonstrated here:

http://wiki.lazarus.freepascal.org/Fast_direct_pixel_access#Speed_comparison

I was shocked to see this, but I confirmed this results at least on windows.

Regards, Darius

Good to know...

Well, I admit that I'm not a OpenGL pro. I "just" reused some code which was taught us at university and worked "fine" so far for the use case we had (render the image of a web camera to a OpenGL context for some augmented reality stuff). Also it was yesterday night after 10 o'clock when I implemented that, so I didn't bother to read whether there is anything faster or whether glDrawPixels is a bad idea <TM>.

The magic words are PBO and texture streaming. This approach reads the data directly into a buffer on the graphics card which you use as texture to display your images. You can even read the data in an other thread than the OpenGL one. Only the reading of the data is performed on the CPU and the data will go from disk straight to VRAM.

I definitely need to find the time somewhen to learn OpenGL better than just simple drawing of polygons (*cough* using glBegin and glEnd *cough*)...

Regards,
Sven

I've attached a zip containing the important files (e.g. not the image files and project resources). As Felipe remarked my bitmap conversion is not the best and the rendering method is not that good either according to Reimar.

Regards,
Sven

-------------- next part --------------
A non-text attachment was scrubbed...
Name: drawtestgl.zip
Type: application/zip
Size: 5714 bytes
Desc: not available
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20120210/2b634d32/attachment-0001.zip>

The problem may be multiple files. I'd try to merge multiple frames, if not all, into one big file, then test again on mac.

DoDi

Well I'm already doing that in filecache.pas. It's just one big file-stream with raw pixel data in it. See also my previous mail where I mention the possibility to compress data. It will be a tradeoff between (de)compression and file access.

Regards, Darius

http://wiki.lazarus.freepascal.org/Fast_direct_pixel_access#Speed_comparison I was shocked to see this, but I confirmed this results at least on windows.

The same on Linux, but the methods that were faster than OpenGL (with or without PBO) don't produce any visuell output for me so I was not really that shocked. :) Either I don't understand the code (just took a brief look at the drawing methods) and there is some bug or they are not even meant to produce any output.

R.

Has to be a bug then. On Windows I see output. I could not get the app to run on Mac yet.

Darius

Reimar Grabowski wrote

On Fri, 10 Feb 2012 20:13:51 +0100 Darius Blaszyk wrote: >

http://wiki.lazarus.freepascal.org/Fast_direct_pixel_access#Speed_comparison [1] I was shocked to see this, but I confirmed this results at least on windows.

The same on Linux, but the methods that were faster than

OpenGL (with or without PBO) don't produce any visuell output for me so I was not really that shocked. :)

Either I don't understand the code

(just took a brief look at the drawing methods) and there is some bug or they are not even meant to produce any output.

Reimar Grabowski wrote

2D projection functions have never been part of OpenGL AFAIK, mind telling me which functions you talk about?

  glMatrixMode(GL_PROJECTION);
  glLoadIdentity;
  glOrtho(0, OpenGLControl1.Width, OpenGLControl1.Height, 0, 0, 1);
  glMatrixMode(GL_MODELVIEW);
  glLoadIdentity();
  glDisable(GL_DEPTH_TEST);
  glViewport(0, 0, OpenGLControl1.Width, OpenGLControl1.Height);
  glGenTextures(1, @TextureId);
  glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);

OpenGL ES 2.0 did not change everything. It just removed the fixed function pipeline (as OpenGL 3 did) in favor of the programmable one, which is a Good Thing(tm).

Not good for people drawing 2D bitmaps

And nobody forces you to use ES 2 just use 1.x if you like it more.

Since it is version 2 I suppose people will start dropping support for version 1 in the future.

It is not, the two are very, very similar.

lol! But just read any OpenGL tutorial which you can imagine. It always starts with glBegin, glEnd and boom ... doesnt work in OpenGL ES. And, read the code here:

http://svn.freepascal.org/cgi-bin/viewvc.cgi/trunk/components/fpvectorial/examples/fpv3d_mainform.pas?root=lazarus&revision=35186&content-type=text%2Fplain

Which part of this code works in OpenGL ES 2? And it isnt even 2D, it is 3D triangles. Anything from it works at all?

But my real question is: how can you have 1 code for all those OpenGL versions to draw 2D bitmaps? Seriously, Id like to know. If it is impossible then how can one say they are very similar?

This is what it may look like for a beginner.

For me, not beginner friendly is a very strong indicator of a bad design.

You are right, I forgot this one.

glMatrixMode(GL_MODELVIEW); glLoadIdentity();

These are all deprecated but still available since OpenGL 3 (2008) and removed in 3.1 (if you create an OpenGL 3 and up only context - 2009), as they belong to the fixed function pipeline. They were run on the CPU anyway so if you really need this kind of matrix manipulations you can just implement them yourself (as most people using OpenGL have done anyway since you need to know your matrices for many tasks and OpenGL had no fast way to retrieve them).

glDisable(GL_DEPTH_TEST); glViewport(0, 0, OpenGLControl1.Width, OpenGLControl1.Height); glGenTextures(1, @TextureId);

Still available.

glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);

Not needed anymore due to GLSL.

> And nobody forces you to use ES 2 just use 1.x if you like it more. Since it is version 2 I suppose people will start dropping support for version 1 in the future.

That's not how it normally works in OpenGL. You could use OpenGL without the fixed function pipeline since 2.0 (2004). Support was removed in 2009 and only if you ask for a special context. If you use a 'standard' context OpenGL 1.0 code (1992) works on current gen graphics hardware unchanged (unlike old DirectX code). Perhaps on phones they are a little bit faster to deprecate old stuff but I don't think vendors will drop support soon.

lol! But just read any OpenGL tutorial which you can imagine. It always starts with glBegin, glEnd and boom ... doesnt work in OpenGL ES. And, read the code here:

glBegin and friends (called immediate mode) was unefficient since 1997 and a bad idea to be used in production code. Normal mesh sizes start around 1000 vertices nowadays most are much larger. You do not want to specify such structures with glBegin/glEnd. You must layout your data in special ways to get the most from your hardware. This is just not possible with glBegin/glEnd. Yes, it was used a lot in tutorials but that doesn't change the above.

http://svn.freepascal.org/cgi-bin/viewvc.cgi/trunk/components/fpvectorial/examples/fpv3d_mainform.pas?root=lazarus&revision=35186&content-type=text%2Fplain Which part of this code works in OpenGL ES 2? And it isnt even 2D, it is 3D triangles. Anything from it works at all?

This still works:

glClearColor(1.0, 1.0, 1.0, 1.0);
glClear(GL_COLOR_BUFFER_BIT or GL_DEPTH_BUFFER_BIT);
glEnable(GL_DEPTH_TEST);

:)

The rest no, as it is either immediate mode or fixed function matrix stuff. You need to learn GLSL (a C like language that runs on the GPU) to do anything in modern OpenGL or OpenGL ES 2.x.

But my real question is: how can you have 1 code for all those OpenGL versions to draw 2D bitmaps? Seriously, Id like to know.

Of course you can't. On modern hardware (later than 2004) you should use the programmable pipeline for best performance (not available on old cards) and even if you use the old fixed function pipeline it is emulated with GLSL programs and slower than doing it yourself since too much legacy stuff is supported. OpenGL is a low level API and it's design reflects the rapid advances in graphics hardware. OpenGL is suitable for doing 2D stuff but it's main purpose was always 3D rendering.

If it is impossible then how can one say they are very similar?

To quote wikipedia:
OpenGL ES 1.0 is drawn up against the OpenGL 1.3 specification, OpenGL ES 1.1 is defined relative to the OpenGL 1.5 specification and OpenGL ES 2.0 is defined relative to the OpenGL 2.0 specification.

So OpenGL ES is very similar to OpenGL.

For me, not beginner friendly is a very strong indicator of a bad design.

OpenGL is a low level graphics lib, as bare to the metal as you can get with regard to graphics cards, so it reflects the state of the hardware. Ease of use is only a second thought, performance is the important metric. It is meant for graphic programmers that are familiar with the concepts. If you need something simpler you should not use it directly but a graphics lib that builds on top of OpenGL optimized for your use case (there are some GUI libs out there). Graphics programming is a complicated topic. Your use case is simple in your eyes (because you never had to worry about how the gfx card actually does its job, coming from software rendering/drawing), but it is not so common for interactive 3D graphics, where some simple quads are good enough for a GUI (like in any current game) and most of your time is spent pushing millions of vertices to the graphics card animating them and doing advanced lighting and other effects. Naturally OpenGL is more optimized for the later tasks than the former one. Please don't bash it based on insufficient knowledge about the topic. The code you posted was the way to use OpenGL before 1997 (when VertexArrays were introduced) or 2004 (when the programmable pieline was introduced) and you complain that it does't work like that anymore in 2012. Well, times have changed and hardware a lot, too. Some simple stuff got more complicated but many complicated stuff became a lot simpler and you now have a lot more options.

R.

Reimar Grabowski wrote

> glMatrixMode(GL_MODELVIEW); > glLoadIdentity(); These are all deprecated but still available since OpenGL 3 (2008) and removed in 3.1 (if you create an OpenGL 3 and up only context - 2009), as they belong to the fixed function pipeline. They were run on the CPU anyway so if you really need this kind of matrix manipulations you can just implement them yourself (as most people using OpenGL have done anyway since you need to know your matrices for many tasks and OpenGL had no fast way to retrieve them).

How would one set matrices then? (<= this is a question out of curiosity)

The deprecation of glBegin/glEnd I knew already :)

Regards,
Sven

How would one set matrices then? (<= this is a question out of curiosity)

Passing them as uniforms to the shader program for example.

R.

But as you see everywhere in this test, Canvas.Pixels is the slowest always;
Even slower with update locking;

Also seems to me like in Test2 they have the Vertical Scan lock enabled and this actually may be dependent on driver settings (tip: look at the test app's CPU usage during test).

And actually.
What's the point of creating frames friggilion times per second, if your graphic card can only display 60fps on most /normal/ nowadays monitors ? (the crt's that were able to do >=75FPS are gone in most places, there is now TFT's everywhere) Would it not be a Better Idea (IMHO) to have the frame ready in time for display and have the rest of the CPU time available for something else ?

L.

Lukasz Sokol wrote

Would it not be a Better Idea (IMHO) to have the frame ready in time for display and have the rest of the CPU time available for something else ?

Yep. Burt you would need to take advantage of the screen refresh interrupt to know the time you can start to get the next frame ready.

-Michael

OpenGL provides that, where the appropriate extension is available, and e.g. GLScene can use it. And as to /when/ to prepare next frame - well as soon as the previous has displayed (within what handles the vert-refresh interrupt) So, invoke your rendering engine from there, and tell it that the display step is 1/f, so it can prepare next frame appropriate to media timing.

Alternatively you can run a timer every 1/f ms with the screen refresh procedure (some way of automation to read screen refresh rate from wherever it can be read, would come handy) and it will at some point sync with screen refresh but you won't even know that :)

L.

Lukasz Sokol wrote

Alternatively you can run a timer every 1/f ms with the screen refresh ..

A separate time base unavoidably will lead to jitter.

-Michael

Yeah, that's why I ':)'d that part.

L.

The limit in OpenGL is going to be uploading speed. IIRC speed also depends on your opengl implementation match your videocard (iow, do not create opengl3 context on e.g. gf8x00 series, but do for new cards)

I still have some code that does this. (OpenGL viewer widget + image type that supports a few basic kinds of bmp and png).

But it is Delphi XE generics code, and I need some bugs fixed first :-)

If your imagetype supports topdown and bottomup, bmp reading and writing is limited by I/O, since they are essentially binary dumps of pixeldata with a header before it.

Somehow I have the feeling that the last statement was meant for me... ^^

Regards,
Sven

Well, then it is still not in release versions, so that was more or less fun. Also you are not the only one, I also have an handfull of bugs in the embarcadero tracker :-)

It is possible to manually expand the generics though. I didn't, because i wanted to preserve inlining functionality here and there.

The GL code was originally written by Ales, and I've expanded it since. The GL code is not generics dependent, the base type however is.

The limit in OpenGL is going to be uploading speed.

True, the absolut limit is PCI-E transfer speed (for 2.0 it is 5GT/s with a data rate of 500MB/s so a x32 connector can do about 16GB/s). Looks more like HD speed is a bigger issue than PCI-E transfer speed.

IIRC speed also depends on your opengl implementation match your videocard (iow, do not create opengl3 context on e.g. gf8x00 series, but do for new cards)

That's news to me. Do you have any links where I can look this up? AFAIK this are the general upload bottlenecks:
- the driver (CPU overhead)
- conversion or reformatting of data
- the CPU bandwidth while copying the data from user space
- the PCI-E bus while transferring the data

For OpenGL it is hard to predict anything. Most of the time there isn't even one optimal way as much depends on hardware/driver combination.

R.

I did some tests with that, including with a x16 card in a x1 slot, and didn't notice any difference. (P35 mobo)

The speed (with a non integrated GPU) is always about 1mb/ms so about 1gbyte/s. But that is for a single uploaded frame (see below). If I increase the framerate (generate a new frame more often), I see that the times get significantly (several times) longer, so the opengl call blocking time is not entirely the same as the time the card is busy.

> IIRC speed also depends > on your opengl implementation match your videocard (iow, do not create > opengl3 context on e.g. gf8x00 series, but do for new cards) That's news to me. Do you have any links where I can look this up?

Benchmarking with own code. I had enabled gl3 contexts and PBOs, and I saw that that was faster on new machines, but slower on older ones. Old and new was mainly videocard. (HD5400 series from ATI vs older GF8500's)

AFAIK this are the general upload bottlenecks: - the driver (CPU overhead) - conversion or reformatting of data

While the code can do other types, I've most experience with 8-bit grayscale. (transformed to RGBA or ABGR while uploading)

- the CPU bandwidth while copying the data from user space - the PCI-E bus while transferring the data

I simply measure my frame upload method times, which are zero copy in my own code, thus entirely bound by opengl.

This consists out of one texture upload, and some linedrawing which is done with ARB buffers.

For OpenGL it is hard to predict anything. Most of the time there isn't even one optimal way as much depends on hardware/driver combination.

True.

Benchmarking with own code. I had enabled gl3 contexts and PBOs, and I saw that that was faster on new machines, but slower on older ones. Old and new was mainly videocard. (HD5400 series from ATI vs older GF8500's)

This is not only old vs new it is also ATI vs NVidia and it is only two cards. Nothing wrong with that but I would not draw general conclusions from that. Enabling PBOs sounds like there is a specific way to use them but there isn't. Using one, two or a pool of PBOs are the options you have for using them for this case and they all should differ in speed (and implementation complexity). I have not tested this but reliable sources (forum on OpenGL.org) say that how and when you map/unmap the buffer makes a difference, too. Before mapping the buffer calling glBufferData with null as data may speed up the mapping time.

I simply measure my frame upload method times, which are zero copy in my own code, thus entirely bound by opengl.

It sounds like you measure OpenGL call blocking time which may or may not be equal to the real upload time. If you use PBOs you most likely map/unmap the buffer in that method, too, which I don't consider upload time. I don't want to question your method just saying that measuring OpenGL times is not as straight forward as it seems to be. That's why NVidia provides a complete SDK (PerfSDK) to measure your OpenGL performance. In the OpenGL world nothing is as easy as it seems.

so the opengl call blocking time is not entirely the same as the time the card is busy

Of course not. That's how OpenGL works and what makes timing single OpenGL operations difficult.

Nonetheless some interesting info you gave there.

R.

It is sufficient indication that one and the same binary must be able to do both. Regardless if that is between vendors or not.

Enabling PBOs sounds like there is a specific way to use them but there isn't. Using one, two or a pool of PBOs are the options you have for using them for this case and they all should differ in speed (and implementation complexity).

Of course not. You can test OpenGL version and extensions and conditionally execute code accordingly.

I do. But for me that is the interesting benchmark. And though I readily admit that I only started to scratch the surface of testing the various performance aspects, I do present realistic benchmarks of about 2 days efforts spent on the issue.

If you use PBOs you most likely map/unmap the buffer in that method, too, which I don't consider upload time.

It depends on your purpose. I'm in computer vision, and I want to show my frame as soon as possible. If you know a way to fire an event after upload is finished, don't hesitate to mention it. As long as I don't have that, I have no choice than to block on it.

This is not a game where you just fire all textures on level load, and just proceed and block at the end, hoping that every paralellizes as much as possible.

I don't want to question your method just saying that measuring OpenGL times is not as straight forward as it seems to be.

No doubt about that.

That's why NVidia provides a complete SDK (PerfSDK) to measure your OpenGL performance. In the OpenGL world nothing is as easy as it seems.

Nvidia's border conditions are typically not my own. And in practice only real-life (read: average) performance is usuable. Peak performance under idealized circumstances is a mere footnote.

> so the opengl call blocking time is not entirely the same as the time the card is busy Of course not. That's how OpenGL works and what makes timing single OpenGL operations difficult. Nonetheless some interesting info you gave there.

(*) average uptime of our machines. Reasons for reboot are often external to
the application (read: powerdown the entire productionline for machine maintenance)

It is sufficient indication that one and the same binary must be able to do both. Regardless if that is between vendors or not.

If you have to support a range of cards, you are absolutely right. Choosing which technique to use is the problem most of the time. Either an invisible mini-benchmark at programm startup (render-to-texture but never show it to the user, which gives you at least an idea about the real performance) or some heuristic based on OpenGL version/vendor/driver version/card name/star alignment,

Of course not. You can test OpenGL version and extensions and conditionally execute code accordingly.

That's a given.
I think you misunderstood me. I was talking about different ways of using the PBO. All of the three mentioned methods will work if PBOs are supported.

I do. But for me that is the interesting benchmark. And though I readily admit that I only started to scratch the surface of testing the various performance aspects, I do present realistic benchmarks of about 2 days efforts spent on the issue.

I am doing OpenGL programming for over 10 years (nowadays mostly as a hobby, so I am not up to date on the current bleeding edge stuff) and I have seen a lot in this time. If you tested your stuff on 2 cards and 2 driver versions that's all your results are really valid for. It is not unreasonable to draw some conlusions from that but they may be wrong nonetheless (not likely in your case, but still possible). That does not mean that you are doing anything wrong only that I take even my own tests with a grain of salt. OpenGL can be a b*tch and from one driver version to the next (one vendor to the other or an X more in the name of the card) performance of a give code path may change dramatically (at least that's what I have seen). Perhaps the situation got better, but I doubt it.

It depends on your purpose. I'm in computer vision, and I want to show my frame as soon as possible.

I thought especially in computer vision you would try to create the frame on the GPU perhaps using CUDA or OpenCL. At least there is quite some info about computer vision algorithms and GPGPU on the net, but you are the expert. Just uploading one textue, showing an image and doing not much more on the card is not a purpose OpenGL is heavily optimized for.

If you know a way to fire an event after upload is finished, don't hesitate to mention it. As long as I don't have that, I have no choice than to block on it.

No events in OpenGL, sorry. Only the driver knows what the driver does. That's why I mentioned the NVidia SDK. ATI can give you similar information but I currently don't know what they call it.

This is not a game where you just fire all textures on level load, and just proceed and block at the end, hoping that every paralellizes as much as possible.

Your knowledge about game technology seems to be a little outdated.

Nvidia's border conditions are typically not my own. And in practice only real-life (read: average) performance is usuable. Peak performance under idealized circumstances is a mere footnote.

It is an SDK. It gives you information from the card/driver that is not accessible in any other way. At least I am thankfull that now I can get at least some info and am not working with a black box as it used to be. You as developer decide what you use it for and most of the time you use it to benchmark your own code so I fail to see what you are talking about. It is for example used by gDebugger which I can recommend.

R.