Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / fpc-pascal / 2013-02

Performance problems with Image Conversions

11 messages · 5 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Andrew Brunner

Hi guys,

I just finished (a beta) of a collage app for Aurawin. I'm seeing a huge load on my servers when scaling images using the Image factory classes for JPEG, and PNG.

Prefix:
6 Core AMD 64 32GB RAM Ubuntu 12.10 x64
8 Core AMD 64 32GB RAM Ubuntu 12.10 x64
FPC/Lazarus daily svn/trunk

Overview:
1.) end-user grabs references to files (stored on my server) 2.) back-end core object scales camera photos to something more net friendly 512px max to scale.
3.) server streams photos back to client over the web and 4.) server assembles XML stream of Mime encoded file data for things like https://aurawin.com/cb?3

Specifics

The pics in this sample take about 30 seconds to go from their originals to 512px. The app waits for the collage to save to the cloud. The problem gets worse for larger collages with more images.

iX,iY are no larger than 512

class function Image.Transform(Stream:TStream; var
sContentType,srcKind,dstKind:string; var iX,iY:Integer):boolean;
var
   FReader      : TFPCustomImageReader;
   FWriter      : TFPCustomImageWriter;
   cReader      : TFPCustomImageReaderClass;
   cWriter      : TFPCustomImageWriterClass;
   Factor       : Double;
   FTransparent : boolean;
   dstImg       : TFPMemoryImage;
   srcImg       : TFPMemoryImage;
   srcCanvas    : TFPImageCanvas;
   dstCanvas    : TFPImageCanvas;
   procedure FreeImageData;
   begin
     if (srcCanvasnil) then
       srcCanvas.Free();
     if (dstCanvasnil) then
       dstCanvas.Free();
     if (FReadernil) then
       FReader.Free();
     if FWriternil then
       FWriter.Free();
     if srcImgnil then
       srcImg.Free();
     if dstImgnil then
       dstImg.Free();
   end;
begin
   Result:=false;
   if SameText(srcKind,Image.Kind.GIF.Name) then begin
     sContentType:=hHTTP.ctPNG;
     dstKind:=Image.Kind.PNG.Name;
     FTransparent:=true;
   end else if SameText(srcKind,Image.Kind.PNG.Name) then begin
     dstKind:=Image.Kind.PNG.Name;
     FTransparent:=true;
   end else begin
     sContentType:=hHTTP.ctJPG;
     dstKind:=Image.Kind.JPG.Name;
     FTransparent:=false;
   end;
   cReader:=FPImage.ImageHandlers.ImageReader[srcKind];
   cWriter:=FPImage.ImageHandlers.ImageWriter[dstKind];
   dstCanvas:=nil; srcCanvas:=nil; FReader:=nil; FWriter:=nil;
srcImg:=nil; dstImg:=nil;
   Result:=(cReadernil) and (cWriternil);
   if Result then begin
     Try
       FReader:=cReader.Create();
       FWriter:=cWriter.Create();
       if SameText(dstKind,Image.Kind.PNG.Name) then begin
         TFPWriterPNG(FWriter).Indexed:=false;
         TFPWriterPNG(FWriter).UseAlpha:=true;
       end;
       srcImg:=TFPMemoryImage.Create(0,0);
       srcImg.UsePalette:=false;
       Stream.Position:=0;
       Try
         srcImg.LoadFromStream(Stream,FReader);
         Stream.Position:=0;
         if (srcImg.Width>=srcImg.Height) then begin
           Factor:=iX/srcImg.Width;
         end else begin
           Factor:=iY/srcImg.Height;
         end;
         if (srcImg.Width>iX) or (srcImg.Height>iY) then begin
           if FTransparent then begin
             if SameText(srcKind,Image.Kind.GIF.Name) then begin
               FTransparent:=TFPReaderGIF(FReader).Transparent;
               TFPWriterPNG(FWriter).UseAlpha:=true;
             end;
           end;
           iX:= Round(srcImg.Width * Factor);
           iY:= Round(srcImg.Height * Factor);
           dstImg:=TFPMemoryImage.Create(iX,iY);
           dstImg.UsePalette:=false;
           dstCanvas:=TFPImageCanvas.create(dstImg);
           dstCanvas.StretchDraw(0,0,iX,iY,srcImg);
           Stream.Size:=0;
           dstImg.SaveToStream(Stream,FWriter);
           Stream.Position:=0;
           Result:=true;
         end;
       Except
         Stream.Size:=0;
         Stream.Position:=0;
         Result:=false;
       end;
     Finally
       FreeImageData();
     end;
   end;
end;

Further, when a user views images via my preview app, I don't send the originals, I scale those too using the same methods. The speed is killing my server's performance.

Does anyone have any idea how to speed up the underlining code?

Andrew Brunner Aurawin LLC 15843 Garrison Circle Austin, TX 78717 https://aurawin.com Aurawin is a great new way to store, share, and explore all your content featuring our innovative cloud social computing platform.

Reply by email

Michael Van Canneyt
to Andrew Brunner
Andrew Brunner wrote 18 lines
Hi guys, I just finished (a beta) of a collage app for Aurawin. I'm seeing a huge load on my servers when scaling images using the Image factory classes for JPEG, and PNG. Prefix: 6 Core AMD 64 32GB RAM Ubuntu 12.10 x64 8 Core AMD 64 32GB RAM Ubuntu 12.10 x64 FPC/Lazarus daily svn/trunk Overview: 1.) end-user grabs references to files (stored on my server) 2.) back-end core object scales camera photos to something more net friendly 512px max to scale. 3.) server streams photos back to client over the web and 4.) server assembles XML stream of Mime encoded file data for things like https://aurawin.com/cb?3

Try to get rid of the XML, it is always going to be slow. Using XML can slow your applications with a factor 6. If you are mime-encoding an image, it'll be even more.

Show quoted text 27 lines
Specifics The pics in this sample take about 30 seconds to go from their originals to 512px. The app waits for the collage to save to the cloud. The problem gets worse for larger collages with more images. iX,iY are no larger than 512 class function Image.Transform(Stream:TStream; var sContentType,srcKind,dstKind:string; var iX,iY:Integer):boolean; var FReader : TFPCustomImageReader; FWriter : TFPCustomImageWriter; cReader : TFPCustomImageReaderClass; cWriter : TFPCustomImageWriterClass; Factor : Double; FTransparent : boolean; dstImg : TFPMemoryImage; srcImg : TFPMemoryImage; srcCanvas : TFPImageCanvas; dstCanvas : TFPImageCanvas; procedure FreeImageData; begin if (srcCanvasnil) then srcCanvas.Free();

Do a FreeAndNil(srcCanvas) here.
1. Free already checks if the pointer is not nil.
You do not need to check again.
2. FreeAndNil will make sure the pointer is Nil

srcImg:=TFPMemoryImage.Create(0,0);

Do not use TFPMemoryImage.
It is a catch-all memory format, not optimized at all, using 64 bits for the images. Instead, use e.g. TFPCompactImgRGBA8Bit if you need alpha or TFPCompactImgRGB8Bit if you do not need Alpha.

dstImg:=TFPMemoryImage.Create(iX,iY); dstImg.UsePalette:=false; dstCanvas:=TFPImageCanvas.create(dstImg); dstCanvas.StretchDraw(0,0,iX,iY,srcImg);

Phew, this is also a real killer.

Do not draw on a canvas with stretchdraw. That will be slow as well; Rather, attempt to manipulate the image in memory by doing a stretch directly. Choose an algorithm which is fast. The default interpolation will result in quite a lot of calculations.

Michael.

Reply by email

Sven Barth
to Michael Van Canneyt
Michael Van Canneyt wrote 7 lines
> srcImg:=TFPMemoryImage.Create(0,0); Do not use TFPMemoryImage. It is a catch-all memory format, not optimized at all, using 64 bits for the images. Instead, use e.g. TFPCompactImgRGBA8Bit if you need alpha or TFPCompactImgRGB8Bit if you do not need Alpha.

Nice, I did not know we have more compact image types available. O.o

Show quoted text 12 lines
> dstImg:=TFPMemoryImage.Create(iX,iY); > dstImg.UsePalette:=false; > dstCanvas:=TFPImageCanvas.create(dstImg); > dstCanvas.StretchDraw(0,0,iX,iY,srcImg); Phew, this is also a real killer. Do not draw on a canvas with stretchdraw. That will be slow as well; Rather, attempt to manipulate the image in memory by doing a stretch directly. Choose an algorithm which is fast. The default interpolation will result in quite a lot of calculations.

Maybe a collection of algorithms for in memory rescaling would be nice...

Regards,
Sven

Reply by email

Michael Van Canneyt
to Sven Barth
Sven Barth wrote 10 lines
On 24.02.2013 11:44, Michael Van Canneyt wrote: >> srcImg:=TFPMemoryImage.Create(0,0); > > Do not use TFPMemoryImage. It is a catch-all memory format, not > optimized at all, using 64 bits for the images. > Instead, use e.g. TFPCompactImgRGBA8Bit if you need alpha or > TFPCompactImgRGB8Bit if you do not need Alpha. > Nice, I did not know we have more compact image types available. O.o

If memory serves well, Mattias Gaertner made them.

Show quoted text 15 lines
>> dstImg:=TFPMemoryImage.Create(iX,iY); >> dstImg.UsePalette:=false; >> dstCanvas:=TFPImageCanvas.create(dstImg); >> dstCanvas.StretchDraw(0,0,iX,iY,srcImg); > > Phew, this is also a real killer. > > Do not draw on a canvas with stretchdraw. That will be slow as well; > Rather, attempt to manipulate the image in memory by doing a stretch > directly. > Choose an algorithm which is fast. The default interpolation will result > in quite a lot of calculations. Maybe a collection of algorithms for in memory rescaling would be nice...

There are a lot of them, that is what the interpolations are for, but they work with a canvas. The case of rescaling an image and simply send the resulting image as a file was not foreseen at the time of writing.

Michael.

Reply by email

Sven Barth
to Michael Van Canneyt
Michael Van Canneyt wrote 17 lines
On Sun, 24 Feb 2013, Sven Barth wrote: > On 24.02.2013 11:44, Michael Van Canneyt wrote: >>> srcImg:=TFPMemoryImage.Create(0,0); >> >> Do not use TFPMemoryImage. It is a catch-all memory format, not >> optimized at all, using 64 bits for the images. >> Instead, use e.g. TFPCompactImgRGBA8Bit if you need alpha or >> TFPCompactImgRGB8Bit if you do not need Alpha. >> > > Nice, I did not know we have more compact image types available. O.o If memory serves well, Mattias Gaertner made them.

According to the SVN log your memory is correct. :)

Show quoted text 19 lines
> >>> dstImg:=TFPMemoryImage.Create(iX,iY); >>> dstImg.UsePalette:=false; >>> dstCanvas:=TFPImageCanvas.create(dstImg); >>> dstCanvas.StretchDraw(0,0,iX,iY,srcImg); >> >> Phew, this is also a real killer. >> >> Do not draw on a canvas with stretchdraw. That will be slow as well; >> Rather, attempt to manipulate the image in memory by doing a stretch >> directly. >> Choose an algorithm which is fast. The default interpolation will result >> in quite a lot of calculations. > > Maybe a collection of algorithms for in memory rescaling would be nice... There are a lot of them, that is what the interpolations are for, but they work with a canvas. The case of rescaling an image and simply send the resulting image as a file was not foreseen at the time of writing.
It seems that it would be good if we'd refactor the interpolations then
so that they can be used with either a canvas or an image as destination...

Regards,
Sven

Reply by email

Michael Van Canneyt
to Sven Barth
Sven Barth wrote 8 lines
>> Maybe a collection of algorithms for in memory rescaling would be nice... > > There are a lot of them, that is what the interpolations are for, but > they work with a canvas. The case of rescaling an image and simply send > the resulting image as a file was not foreseen at the time of writing. It seems that it would be good if we'd refactor the interpolations then so that they can be used with either a canvas or an image as destination...

Yes. I'll put it on my todo list.

Michael.

Reply by email

Andrew Brunner
to Michael Van Canneyt: Try to get rid of the XML, it is always going to be slow. Using XML can slow your…
Michael Van Canneyt wrote 5 lines
Try to get rid of the XML, it is always going to be slow. Using XML can slow your applications with a factor 6. If you are mime-encoding an image, it'll be even more.

I was just thinking about that. I think JSON would be ok, especially if I can use the RTL to get the data into the fields. XML code takes about 33% of my dbModules for each implementation. That would take a lot of work for each DBModule and CoreObject, both client (js) and server(pas).

Show quoted text 5 lines
Do a FreeAndNil(srcCanvas) here. 1. Free already checks if the pointer is not nil. You do not need to check again. 2. FreeAndNil will make sure the pointer is Nil

Done.

Show quoted text 5 lines
Do not use TFPMemoryImage. It is a catch-all memory format, not optimized at all, using 64 bits for the images. Instead, use e.g. TFPCompactImgRGBA8Bit if you need alpha or TFPCompactImgRGB8Bit if you do not need Alpha.

Done. I'm preparing to check for performance gains...

Show quoted text 12 lines
> dstImg:=TFPMemoryImage.Create(iX,iY); > dstImg.UsePalette:=false; > dstCanvas:=TFPImageCanvas.create(dstImg); > dstCanvas.StretchDraw(0,0,iX,iY,srcImg); Phew, this is also a real killer. Do not draw on a canvas with stretchdraw. That will be slow as well; Rather, attempt to manipulate the image in memory by doing a stretch directly. Choose an algorithm which is fast. The default interpolation will result in quite a lot of calculations.

Ok, I noticed some units surrounding the graphic code. Is this something that is illustrated on the wiki? Where do I get the methods? Anyone else is welcome to comment on scaling methods here... I have a feeling this would yield the most performance gains.

Digital Photos scaled go from 3264x2448 down to 512x384 so the scaling is somewhat slow. The streaming is slow too...

If not, does anyone have lightning fast scaling methods in C++ or Java that I can port/contribute?

Reply by email

Reinier Olislagers
to Andrew Brunner
Andrew Brunner wrote
If not, does anyone have lightning fast scaling methods in C++ or Java that I can port/contribute?

You might want to look if e.g. ImageMagick is faster for your needs.

There's bindings for it in Pascal:
http://wiki.lazarus.freepascal.org/PascalMagick

Apparently, exactimage was designed to be faster than imagemagick: http://www.exactcode.de/site/open_source/exactimage/ It's apparently GPL. No idea about bindings for that.

Some more info about the built in FPC image support: http://wiki.lazarus.freepascal.org/fcl-image#Known_Issues_and_limitations

Reply by email

Graeme Geldenhuys
to Andrew Brunner: I was just thinking about that. I think JSON would be ok, especially if I can use the…
Andrew Brunner wrote
If not, does anyone have lightning fast scaling methods in C++ or Java that I can port/contribute?

A fpGUI user did a speed comparison of a lot of various image libraries and pure Object Pascal imaging code. I can find the link to his work and results if you want. FCL-Image was by a large margin the slowest of the bunch. I can't remember off the top of my head what was the fastest, but ImageMagick will probably be a good choice.

Regards,
- Graeme -

fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/

Reply by email

Andrew Brunner
to Andrew Brunner: I just finished (a beta) of a collage app for Aurawin. I'm seeing a huge load on my…

Ok, This matter has been solved.

The updated code is so simple and many factors faster!

Thanks to all!

class function Image.Transform(Stream:TMemoryStream; var
iX,iY:Integer):boolean;
var
   status             : MagickBooleanType;
   wand               : PMagickWand;
   srcWidth,srcHeight : Cardinal;
   dstWidth,dstHeight : Cardinal;
   Factor             : Double;
   img                : Pimage;
   blob               : PByte;
   bSize              : QWord;
begin
   Result:=false;
   wand := NewMagickWand;
   Try
     Stream.Position := 0;
     status := MagickReadImageBlob(wand, Stream.Memory, Stream.Size);
     if (status = MagickTrue) then begin
       img := GetImageFromMagickWand(wand);
       srcHeight := MagickGetImageHeight(wand);
       srcWidth := MagickGetImageWidth(wand);
       if (srcWidth>=srcHeight) then begin
         Factor:=iX/srcWidth;
       end else begin
         Factor:=iY/srcHeight;
       end;
       dstWidth:=Trunc(srcWidth*Factor);
       dstHeight:=Trunc(srcHeight*Factor);
       if (srcWidth>iX) or (srcHeight>iY) then begin
         //MagickResizeImage(wand, 106, 80, LanczosFilter, 1.0);
         MagickScaleImage(wand, dstWidth, dstHeight);
         Stream.Size:=0;
         blob:=MagickGetImageBlob(wand,@bSize);
         Stream.Write(blob^,bSize);
         Result:=true;
       end;
     end;
   finally
     DestroyMagickWand(wand);
   end;
end;
Andrew Brunner Aurawin LLC 15843 Garrison Circle Austin, TX 78717 https://aurawin.com Aurawin is a great new way to store, share, and explore all your content featuring our innovative cloud social computing platform.

Reply by email