Hi, I would like to know how can I let a CGI app return a JPeg image from a TChart.

Here's what I'm doing:

procedure TFPWebModule1.getchartRequest(Sender: TObject; ARequest:
TRequest;
  AResponse: TResponse; var Handled: Boolean);
var
  lRect: TRect;
  lChart: TChart;
  lJPegImage: TFPMemoryImage;
  lJPegWriter: TFPWriterJPEG;
  lCanvas: TFPCustomCanvas;
begin
  lChart := TChart.Create(nil);
  lJPegImage := TFPMemoryImage.Create(100, 100);
  lJPegImage.UsePalette := False;
  lCanvas := TFPImageCanvas.create(lJPegImage);
  lJPegWriter := TFPWriterJPEG.Create;
  try
    lChart.Width := 100;
    lChart.Height := 100;
    DoSomeStuffWithTheChart(lChart);
    lRect := Rect(0,0, lChart.Width, lChart.Height);
    lCanvas.CopyRect(0,0,TFPCustomCanvas(lChart.Canvas),lRect);
    lJPegWriter.ImageWrite(AResponse.ContentStream, lJPegImage);
  finally
    lChart.Free;
    lJPegWriter.Free;
    lCanvas.Free;
    lJPegImage.Free;
  end;
  Handled := True;
end;

After executing the method I'm getting "Control '' has no parent window' at the CopyRect line. I'm sure there is another method to copy a TCanvas to a TFPCustomCanvas.

Any hint?

Thanks in advance,

Use Chart.PaintOnCanvas instead. See savedemo for an example. Note that you must give it full TCanvas -- just TFPCanvas is not enough.

Yes, but, how can I use a TCanvas in a CGI app?

If I write:

Chart.PaintOnCanvas(TCanvas(lCanvas), Rect(0, 0, 100, 100));

I get an Access Violation.

Simply said: you can't.

The current TAChart is not suitable for a CGI application. It expects a GUI.

Michael.

2011/3/21 Michael Van Canneyt :

Of course, since you are converting an object to the type it does not belong to. You should create a TBitmap or another similar object. Did you look at the savedemo as I proposed in a previous mail?

Simply said: you can't. The current TAChart is not suitable for a CGI application. It expects a GUI.

Why do you think so?
TAChart should be prefectly capable of rendering on bitmap-based canvas.

Yes, if I try to PaintOnCanvas of a TBitmap, for example. I get "Canvas does not allow drawing".

> Simply said: you can't. > The current TAChart is not suitable for a CGI application. It expects a GUI.

I'm afraid Michael is right.

Because both TCanvas and TBitmap suppose a GUI. TFPCustomCanvas does not.

Yes, I know about the 'nogui' widgetset, and no, I don't think that's a proper solution.

Michael.

That is very strange. I have created a nogui demo in r29942, which does not have any gui, and creates a bitmap file with the chart. Does it work for you?

Hm. For me GUI means "graphical user interface" -- that is, windows (aka forms), interacting with mouse, etc.
I have just committed a demo which does not have any of this, and yet successfully draws TChart to a file -- see previous mail. Maybe for you GUI means a different thing?

It works when compiled with Gtk2 widgetset. If you set the nogui widgetset you'll get the "Canvas does not allow drawing error".

That is true.
You have to link you CGI app with some graphic widgetset, which will unfortunately bloat an executable, but should otherwise work.

Well, I tested that, but I get "Internal Server Error" from my Apache.

What does the log show?

[Sun Mar 20 16:30:09 2011] [error] [client ::1] (cgicharts:16246): Gtk-WARNING **: cannot open display:
[Sun Mar 20 16:30:09 2011] [error] [client ::1] Premature end of script headers: cgicharts

Unfortunately, I am not an expert in GTK -- it seems that it tries to initialize display even if there is nothing to draw. I am not sure if it is fixable -- the demo I posted does work as a CGI under Apache on Windows, so it may be GTK problem, Lazarus problem, or X-server/GTK/Apache configuration problem.
You might want to try fpgui widgetset -- it depends only on X, which may work even if GTK does not.

Of course, it would be ideal to create a version of TAChart that does not require a widgetset, but the task is complicated by compatibility requirements. I have already made some progress in this direction -- see IChartDrawer infrastructure.
TAChart is mostly independent of TCanvas -- it only needs one for events and interactive tools.
Few drawing back-ends exist -- AggPas, OpenGL, and more are planned, in particular SVG, which will probably be most suitable for web apps. Unfortunately, all back-ends have limitations compared to TCanvas (for example OpenGL has very limited text rendering capabilities, and "pure" TFPCanvas does not even support TColor).

Thanks for the explanation Alexander.

Looking at the TChart.PaintOnCanvas method, I see you use a TCanvasDrawer class to draw over a TCanvas. I think it shouldn't be too difficult to use a TFPImageCanvas when using NoGui widgetset. What do you think?

Try using xvfb. This uses a virtual framebuffer that doesn't need a display. I've had to use it with a webserver before.

Henry

There are two problems here:
1) I suspect that the error you see is generated at the application startup, not when TChart tries to paint something. So to work around it, it is not enough to remove TCanvas in one place -- EVERY instance of TCanvas should be removed
(so that Graphics unit is not used any more).
Unfortunately, this is poses a serious compatibility hazard.
2) Even is the previous is possible (I believe it is, although difficult -- TChart should be split into several classes in different units, with only the descendants depending on Graphics),
TFPCanvas version it will still have serious limitations -- no TColor, no font orientation, etc.

Can it be that hard as TCanvas inherits from TFPCustomCanvas? (putting point 2 aside)

Regards,
Sven

Thanks Henry, it seems to almost work, at least I don't get the Cannot Open Display error, but...

When I do this:

    lChart.Width := 100;
    lChart.Height := 100;
    with lChart.SaveToImage(TJPEGImage) do
    begin
      SaveToStream(AResponse.ContentStream);
      AResponse.ContentType := 'image/jpeg';
      AResponse.ContentLength := AResponse.ContentStream.Size;
      AResponse.SendContent;
    end;

I get a black square of 100 x 100 pixels.

If I try this:

    lChart.SaveToBitmapFile('test.bmp');

I get the error "Image palette is too big or absent" from my CGI.

Sorry Henry, the correct command to run Xvfb is this:

Xvfb :12 -screen 0 1024x768x16 &

Now it works!

Yes. The GUI of the screen.
By accessing the LCL canvas you access the widgetset (gtk2) which accesses the current screen. For example the gtk2 asks for theme, fonts, screen color depths and sub pixel rendering parameters. A cgi program has no screen.
The LCL canvas is an implementation of the abstract TFPCustomCanvas, and uses a widgetset to render. The nogui widgetset does not implement rendering, so you can not draw with it.
There are other implementations of TFPCustomCanvas without the need of a screen. The fcl provides one. Probably TAggFPCanvas can work without screen too. The last time I checked the main problem is the font support. It is far less comfortable as the LCL. It is a good idea to add a render function to TAChart for TFPCustomCanvas.

Mattias

It is ok to ask for all these things since thay are necessary to render the chart.
However, the fact they are unconditionally taken from the screen is IMO a deficiency of either GTK or LCL-GTK.
As you can see in this thread, as soon as this deficiency is worked around, (in this case, by using Xvfb utility), the problem is solved.

There are other implementations of TFPCustomCanvas without the need of a screen. The fcl provides one. Probably TAggFPCanvas can work without screen too. The last time I checked the main problem is the font support. It is far less comfortable as the LCL.

Also, AggPas supports neither Pen.Style not Brush.Style.

It is a good idea to add a render function to TAChart for TFPCustomCanvas.

Generally, yes.
But as I stated in the previous mail, simply adding a function may not help. Still, I am working on this ;)

The display error comes from the gtk.

As you can see in this thread, as soon as this deficiency is worked around, (in this case, by using Xvfb utility), the problem is solved.

I wonder how much overhead it is for a cgi program to start gtk on xvfb compared to a fpcanvas program.

Mattias

A lot. A connection to the X server must be established, and the whole of GTK must be set up. This is an expensive operation.

I can't believe people even consider this approach. It's like PHP devels would suddenly say that "well, from now on you must run your web apps with an X server". They would be the laughing stock of the whole PHP programmer's community.

Yet, in lazarus, no-one seems to mind this kind of statements. It's even encouraged.

None of "theme, screen color depths and sub pixel rendering parameters" are necessary to create a bitmap. A bitmap is a matrix of pixels with a certain color. No more, no less. Some of these parameters are needed when the bitmap is being displayed, but definitely not when it is created.

When making a picture with a digital camera, the camera also has no knowledge of the screen you'll be displaying the picture on. Nevertheless, it makes pictures.

Michael.

True, although something like FastCGI may alleviate startup costs.

Using widgetset is perhaps not the ideal solution, but the practical one, given the contraints of existing tools.
Surely they may be improved, but that is not fast not easy to do.

None of "theme, ?screen color depths and sub pixel rendering parameters" are necessary to create a bitmap. A bitmap is a matrix of pixels with a certain color. No more, no less. Some of these parameters are needed when the bitmap is being displayed, but definitely not when it is created.

I am not sure how you could create a bitmap without (at least implicitly) specifying color depth. And while other paremeters (especially fonts) are not required for bitmap, they are definitely required to draw a chart on it. There are many ways to specify drawing parameters and primitives, and using widgetset is quite valid way.
I even suspect that in many cases widgetset drawing is faster than pure FPCanvas.
Now, the fact that a widgetset tries to access *screen* to create a bitmap is a problem, I would even call it a bug, although I do not know if it is in GTK or LCL.

When making a picture with a digital camera, the camera also has no knowledge of the screen you'll be displaying the picture on. Nevertheless, it makes pictures.

When used at web-server, TChart also has no knowledge of the screen the chart will de displayed on.
And to not tell me that digital cameras are not aware of the parameters of their own
screens ;-)

Still, as I have already said, I argee that FPCanvas-based TChart would be a good idea
(actually, even FPCCanvas might not be needed in case of SVG back-end), and I am already slowly working on it for the last few months. Some of the problems I face:
1) TColor vs TFPColor -- I would very much prefer if TColor concept together with all
Color properties would be moved to the FPImage.
It is the primary source of incompatibility and confusion. I think I know how to handle it, but it will not be pretty.
2) TPen/TBrush/TFont -- I can not simply replace these with the FP versions, since it will be a serious loss of functionality. Fonts are most problematic.
Solutions I imagine now involve rather messy set of interfaces and/or generic classes,
which are hard to get right and will not be easy to maintaing.
3) Events -- this is trivial compared to above, but still -- existing TChart events often pass
TCanvas as a parameter. This can be solved easily by breaking backwards compatibility,
but I am reluctant to do this.

Correct. Unfortunately, no-one seems to be willing to take on the task :(

There is no need for 'color depth'. A color is 4 words; that's it. Color depth is only needed when displaying on screen, because the screen has a 'color depth'; namely the number of colors it can display in a pixel.

Green is green. It means RB=0, G=max value; No matter what the color depth is...

And while other paremeters (especially fonts) are not required for bitmap, they are definitely required to draw a chart on it. There are many ways to specify drawing parameters and primitives, and using widgetset is quite valid way.

Here I differ in opinion.
If what you are saying is true, things like PostScript could never exist. And PHP makes very nice charts without screen.

I suppose the LCL, because most likely it asks some system metrics (DPI etc). The widgetset can only get them from the display...

Rightly so, it does not need it.

Well, I suppose a 'simplified' color handling could be introduced in fpImage.

2) TPen/TBrush/TFont -- I can not simply replace these with the FP versions, since it will be a serious loss of functionality.

Why ? As far as I know, all operations in LCL are present in fpImage. If some are missing, we can implement them.

Fonts are most problematic.

Again, why ? the FreeType engine can be used to render any text in any available font on a bitmap.

(Although I admit that the interface currently is rather clumsy, and could use some improvements - some kind of font factory is needed.)

I understand it is not easy.

You suffer the same problem as the reporting engines. They are designed on top of a GUI, when in fact there is no need for this. They need an abstract canvas, representing a sheet of paper on which to Draw. PostScript can do it, and so can Pascal.

Again, I don't claim it is easy, but I regret that everyone starting a set of components time and again takes the same road...

Well.
Very likely I'll need charting in a web project of mine. At that time, I'll probably have to re-implement TAchart on top of TFPImage :(

One thing should be clear: none of what I say implies that I think TAChart is a bad component. It's probably well written for a GUI, but unfortunately not suitable for a Web environment in my opinion.

Michael.

Huh?
AFAIK you can do quite a lot with fpcanvas. You
can use fonts. But using fonts is not very comfortable because you have to search and load it yourself.

48bit for each pixel costs a lot of mem and speed. But it is easy with fpimage to define a RGB 8bit image or RGBA or grayscale.

[...] > I even suspect that in many cases widgetset drawing is faster than > pure FPCanvas.

Why?
AFAIK in cgi you can not use the hardware acceleration.

Since you are the second to blame the LCL, let's try the most simple gtk program:

program test1;
uses gtk2;
begin
  gtk_init(nil, nil);
end.
$ ./test1

(test1:3683): Gtk-WARNING **: cannot open display:

The gtk is made for the screen, not for cgi. Maybe you can use gtk's drawing backend cairo without a display.

TColor does not support alpha.
And TColor has some VCL/LCL specific special colors. I don't see the point integrating that into fpimage.

[...]

Mattias

2011/3/21 :

> Using widgetset is perhaps not the ideal solution, but the practical one, > given the contraints of existing tools. > Surely they may be improved, but that is not fast not easy to do. Correct. Unfortunately, no-one seems to be willing to take on the task :(

But I am willing, and doing it right now ;)

Please do not conflate "screen" with "widgetset".

That is a deficiency that could be fixed.

Why ? As far as I know, all operations in LCL are present in fpImage. If some are missing, we can implement them.

For starters, the absence of following is most problematic:

Canvas.FillRect
Canvas.RadialPie
Font.Orientation

Again, I don't claim it is easy, but I regret that everyone starting a set of components time and again takes the same road...

I did not start TAChart, but I do try to improve it's design.

Well. Very likely I'll need charting in a web project of mine. At that time, I'll probably have to re-implement TAchart on top of TFPImage :(

If you implement the missing parts of FPCanvas,
there is a good chance you will not have to re-implement TAChart ;)

Because color manupulation in 16-bit depth may be slower than in 8-bit, and because even withouh acceleration, OS drawing functions may be much better optimized then FPPCanvas ones.

Since you are the second to blame the LCL, let's try the most simple gtk program:

...

?gtk_init(nil, nil);

Well, of course. The question is -- should LCL call gtk_init even if no gtk functions are called?
Maybe initializing gtk_pixbuf would be enough for TBitmap?

It would avoid ugly workarounds like the one I am writing for TAChart now. Anyway, this is not a huge problem -- just some code duplication.

You don't need it, either.

When generating a bitmap, I don't see how hardware accelleration of the graphics card comes into play ?
When generating a bitmap 'on screen', the double buffering mechanism means that
a) you somehow create it in memory
b) it is then moved 'accelerated' to screen.
and in 95% of cases, even without double buffering, this is the mechanism used.

Since CGI only needs step a), the 'acceleration' is not needed.

Obviously, but AFAIK it is the LCL that calls gtk_init ? Simply linking to GTK doesn't open a display, I presume.

Michael.

Glad to hear it :)

In my opinion "No screen => no widgetset." ?

FillRect and RadialPie can be solved easily. The Font.Orientation can be added as a property, I would have to look at FreeType to see how it can be implemented. Consider it on the todo list. I can add the missing properties/methods quickly, their implementations may take slightly longer :-)

Great =-)

I will try to do so ASAP.

Michael.

Maybe, maybe not.
In case of gtk I doubt that *many* functions are much faster. In my measurements the difference between 16 vs 8bit was not big. It made a big difference if your bitmap is 16bit. I recommend to not use the default fp memory image. Writing a 8bit is easy. It would be nice if fpimage would provide some other memory image implementations. For example 8bit RGB, RGBA and
Grayscale.

Yes, because it is has to parse the command line arguments. It is the way how a gtk program works. If you don't want a gtk program, then you must not use the LCL gtk interface.

Maybe initializing gtk_pixbuf would be enough for TBitmap?

Maybe fpimage is enough for TBitmap.

Mattias

So, create a GUI app (always on) with HTTP support. The CGI request charts to this app.

Marcos Douglas

Good idea, I can easily add these.

Michael.

Hi Marcos. In my case I cannot do that, because the app will run on a headless, no X, server.

I have implemented initial FPCanvas backend.
Since r29962, nogui demo uses nogui widgetset -- thus, it should not need GTK anymore.
Please test.

Yes, please at least add dummy Orientation property. Unfortunately, I have to keep compatibility with FPC 2.4.2 until at least the next Lazarus release, but at least there will be hope ;-)

I have implemented FPCanvas back-end, (see nogui demo since r29966), and found the following additional problems:
1) TextSize/TextOut do not work at all (raise ENotImplemented).
2) Clipping is broken (try uncommenting a line after FIXME and see the result)
3) FPCanvasHelper.Assign not implemented -- there exist DoCopyProps procedure which does the same, but is protected for some reason.
4) Ploygon/Polyline interface is weaker than in TCanvas, and so may require extra array copying
5) TFPImageCanvas has abstract methods(!).
Even standard drawing.pp demo from the fcl-image/examples gives warnings.

I have implemented FillRect. RadialPie is added but not yet functional. I also added PolyBezier, but it is not yet functional.

I will look at this.

4) Ploygon/Polyline interface is weaker than in TCanvas, and so may require extra array copying 5) TFPImageCanvas has abstract methods(!). Even standard drawing.pp demo from the fcl-image/examples gives warnings.

I will look at this as well :-)

Michael.

It seems to work, at least from command line, no fonts of course, but graphics and colors works ok.

I'll test the CGI app from home when I'll go back from work.

(If people know how to fix/abstract that, I'd be interested due to a possible application in an OpenGL gui)

As you can see further in the thread, it is already half-fixed ;-) TAChart still can not work without any widgetset,
but already works with nogui.
BTW, TAChart can draw to OpenGL.

You have to link you CGI app with some graphic widgetset, which will unfortunately bloat an executable, but should otherwise work.

Not 100% true. If you run that CGI app under a Linux server that doesn't have X11 installed, it WILL fail.

It's just a wild idea, and maybe TAChart is not feasable. But we to display a whole bunch of trend meters in our app.

Fast, and predictable update times, scaling possibilities (the nearly free pan and zoom are the main reason for opengl).

I'm planning to review our opengl routines this summer, and a HUD based trendmeter is high on the agenda.

I tested it in a CGI sample last night, but no luck. I keep getting the "Canvas does not allow drawing error.".

That is very strange -- please make sure you are using the right version using nogui widgetset (you can check it in Project Options -> Build modes -> LCLWidgetType).
Otherwise, it sounds like nogui widgetset actually requires a GUI -- pretty serious bug, I'd say.

Can anybody else tell what's going on here?

Currently, TAChart back-ends are very simple and are only concerned with actual drawing (in fact, they are called "drawers" in code). So, zoom and panning in OpenGL mode are still performed by CPU. My overall vision does include higher-level back-ends (I currently plan to call them "renderers"), which will handle view transformation
themselves -- but that requires yet another extensive refactoring, so I do not think I will be able to do it soon.

I'm planning to review our opengl routines this summer, and a HUD based trendmeter is high on the agenda.

Please send a message when you will start this --
maybe it will motivate me enough to push through with renderers ;-)

Nevermind, it worked wonderfully.

I attached your nogui example compiled as a CGI.

2011/3/23 Leonardo M. Ram? :

Nevermind, it worked wonderfully. I attached your nogui example compiled as a CGI.

Excellent.
Now I just have to figure out how to draw text on FPCanvas...

You need the ftfont, freetype, freetypeh units.
Then create a TFreetypeFont instance, assign to canvas.font, and call TextRect() or whatever you would call normally.

in the SVN of FPC, check directory tests/utils/testsuite, unit utests.

It is a CGI webprogram that e.g. draws a pie chart and puts a legend on it using FPimagecanvas.

Michael.

2011/3/23 Michael Van Canneyt :

Thanks for the pointer. I have added TFreetypeFont to the FPCanvas drawer. See nogui demo since r29999.

However, I have encountered many problems:
1) FPCanvas drawer no requires freetype library even if no text is actually drawn.
I can work around it by creating separate descendant drawer, but it is frustrating.
2) FreeType library from gtk-windows is called 'freetype6.dll', but freetypeh unit wants ''freetype-6.dll'.
I have no idea which is correct, but no the dll must be copied twice (or symlinked, of course, but Windows symlink support is poor).
3) Kerning is totally broken. Maybe it is windows-specific problem, but look at the attached code and screenshot to see what I mean.

2011/3/23 Alexander Klenin :

Does this mean that TAChart now always depends on TFreeTypeFont? And that therefore Lazarus now requires the freefont libraries to be built (and run) on *nix and to be run on windows?

Vincent

2011/4/4 Vincent Snijders :

The reason I ask, on powerpc-darwin (Mac OS X), I get the following errors: Linking ../lazarus
/usr/bin/ld: Undefined symbols:
_FT_Done_FreeType
_FT_Done_Glyph
_FT_Get_Char_Index
_FT_Get_Glyph
_FT_Get_Kerning
_FT_Glyph_Copy
_FT_Glyph_To_Bitmap
_FT_Glyph_Transform
_FT_Init_FreeType
_FT_Load_Glyph
_FT_New_Face
_FT_Set_Char_Size
_FT_Set_Pixel_Sizes
An error occurred while linking
Error: Error while linking
Fatal: There were 1 errors compiling module, stopping Fatal: Compilation aborted

Vincent

If so, then it is a bug in FTFont.
BTW, I have found similar bug in GLUT -- see r29994. I can remove font support for MacOS as I have done for Win64 -- just tell me the correct define to check.

Alexander Klenin wrote

I can remove font support for MacOS as I have done for Win64 -- just tell me the correct define to check.

Alternatively, I can create a separate packages for FPCanvas and OpenGL, as I have already done for AggPas and BGRABitmap.
I hoped to aviod that, since it would bring the TAChart to a total of five packages.

Alexander Klenin wrote

3) Kerning is totally broken. Maybe it is windows-specific problem, but look at the attached code and screenshot to see what I mean.

This has been already fixed in trunk. I even think it has been merged in 2.4.3.

Darius