Sign in Join

Live chat for Free Pascal, Lazarus, Delphi and every Pascal programmer.

Mailing lists / lazarus / 2016-05

Flexible exporter for grid-like components

12 messages · 6 people · –

Reply by email Subscribe to this list Your subscription

Read-only archive. Use j and k to move between messages.

Werner Pamler

TStringGrid provides the methods SaveToCSVFile/Stream for saving the grid content to a csv file or stream. This is very convenient for users, but it is not very flexible: The export works only for the csv data format, no html, no possibility to plug in other formats. It requires always a modification of the /grids /unit to extend these methods by additional options. And it works only with TStringGrid - there are other grid-like controls, such as custom "virtual" grids derived from TCustomDrawGrid, TListview in report mode, TKGrid and other third-party grids which are not supported: if they do not have such a method on their own the export code has to be re-written (often: duplicated).

The attached file contains code for a flexibile export system for 2D-data classes which is open and can easily be extended to other formats and other controls. It essentially consists of two parts:

* *TLazExporter *represents the file format. The class is abstract and provides the methods to write cells and rows to stream and file. I implemented an exporter for *csv *and *html *files for the lclbase package. But in addition there is also a dedicated SpreadsheetExporter which takes care of *Excel *and/or *Opendocument *file formats (using fpspreadsheet).
* *TLazExporterLink *provides the data to be exported. It is an abstract class between the control to be exported and the writer. It exposes methods to navigate from cell to cell and row to row, and to define the strings assigned to each cell. I implemented a TGridExporterLink (accessing *TCustomStringGrid*, in the long run: *TCustomDrawGrid*) and *TCustomListView*. The same principle works for any other classes with 2D data arrays (matrix).

The unit /GridExporter /exposes functions to write grid content to file and to stream. The exporter instance is passed as a parameter, i.e. it is very easy to switch from csv to hml or any other format provided the corresponding exporter is available.

          procedure ExportGridToFile(AGrid: TCustomStringGrid;
AExporter: TLazExporter; const AFileName: String; AOptions:
TGridExportOptions); overload;
          procedure ExportGridToStream(AGrid: TCustomStringGrid;
AExporter: TLazExporter; AStream: TStream; AOptions:
TGridExportOptions); overload;

Similarly, there are also ExportListviewToFile/Stream procedures in the unit /ListViewExporter /for the export from a TListview.

Properties of the exporter can be used to fine-tune the export. In case of the html exporter, for example, a set of css statements can be specified to format the exported html table:

          htmlexporter := THTMLExporter.Create;
          with htmlexporter do
          begin
            CSS.Add('table { border: 1px solid #DDDDFF; }');
            CSS.Add('th { background-color: #DDDDFF; }');
            CSS.Add('h1 { font-family:Helvetica, Arial, sans-serif;
font-size:16pt; color:blue; background-color: #EEEEFF}; }');
         end;
         ExportGridToFile(StringGrid1, exporter, AFileNameForGrid,
[geoFixedRows, geoFixedCols, geoVisibleColsOnly]);
         ExportListViewToFile(Listview1, exporter, AFileNameForListView,
[leoTitles, leoVisibleColsOnly]);

In addition to these procedures there are overloaded versions taking the exporter class (instead of an exporter instance) as a parameter. In this way the default exporter parameters are applied during the export. Of course, these procedures can be added as methods to the classes referred to by the first parameter.

The attached demo shows the exporter requires no
modification of any LCL packages or units. In the long run, however, I'd propose to add unit /LazExporter /to package /LazUtils /and /GridExporter /and /ListviewExporter /to package /LclBase /(/SpreadsheetExporter /would go to /laz_//fpspreadsheet/), and to make these modifications to the /grids /unit:

* Extend the ExportGridToFile/Stream procedures to accept also TCustomDrawGrid descendants. This would be possible by introducing a virtual method GetCellText(Col,Row) to TCustomDrawGrid (or maybe even TCustomGrid) which would catch the cell text from a special event OnGetCellText; this would help to create "virtual" grids using TCustom(Draw)Grids. In case of the TStringGrid, of course, GetCellText would simply return the cell strings Cells[Col, Row].
* Replace the code in SaveToCSVFile/Stream by the csv exporter. I'd also vote to deprecate these methods because they are special cases of the csv exporter.
* Replace the copy-to-clipboard code (CopyCellRectToClipboard) by the csv exporter. In addition, a html format could be written to the clipboard with almost no extra code.

I would greatly appreciate any comments.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20160501/50995bce/attachment-0001.html> -------------- next part --------------
A non-text attachment was scrubbed...
Name: exporters.zip
Type: application/x-zip-compressed
Size: 14039 bytes
Desc: not available
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20160501/50995bce/attachment-0001.bin>

Reply by email

Bart
to Werner Pamler
Werner Pamler wrote
I would greatly appreciate any comments.

Nice piece of work.

I would have the name of the base class (LazExporter) reflect that it is intended for use with "2D-Data", and that it exports text.

The HTML exporter should not (or not always) write a complete HTML page, most likely I would use the exporter to create the tables only and insert them in an existing HTML document.
(The default setup IMO should be to just export it to a plain table (only table, tr, th, td (and closing) tags.)
But this is nitpicking about implementation details.

Bart

Reply by email

Werner Pamler
to Bart
I would have the name of the base class (LazExporter) reflect that it is intended for use with "2D-Data", and that it exports text.

TLaz2DTextExporter, or TCustom2DTextExporter? Or anything else?

Show quoted text 5 lines
The HTML exporter should not (or not always) write a complete HTML page, most likely I would use the exporter to create the tables only and insert them in an existing HTML document. (The default setup IMO should be to just export it to a plain table (only table, tr, th, td (and closing) tags.)

I could give the HTMLExporter a corresponding property. But I see the main use case of html export in stand-alone html documents, therefore I disagree on the default setup (i.e. I'd name the property "WriteFragmentOnly" instead of "WriteFullHTML"). And how do you want to insert the table in your document? By copy to clipboard? You must remember that we had constructed the "Clipboard.SetAsHtml" method such as to always write a complete html stream because some Office applications want that (http://bugs.freepascal.org/view.php?id=29146, see also comment in front of TClipboard.SetAsHtml).

Reply by email

Luiz Americo Pereira Camara
to Werner Pamler
"Werner Pamler" wrote 5 lines
> > I would have the name of the base class (LazExporter) reflect that it > is intended for use with "2D-Data", and that it exports text. TLaz2DTextExporter, or TCustom2DTextExporter? Or anything else?

Does not grid implies a 2D format?

TLazGridTextExporter?

TLazTextGridExporter?

Is necessary Laz prefix?

Luiz

Show quoted text 9 lines
> The HTML exporter should not (or not always) write a complete HTML > page, most likely I would use the exporter to create the tables only > and insert them in an existing HTML document. > (The default setup IMO should be to just export it to a plain table > (only table, tr, th, td (and closing) tags.) I could give the HTMLExporter a corresponding property. But I see the

main use case of html export in stand-alone html documents, therefore I disagree on the default setup (i.e. I'd name the property "WriteFragmentOnly" instead of "WriteFullHTML"). And how do you want to insert the table in your document? By copy to clipboard? You must remember that we had constructed the "Clipboard.SetAsHtml" method such as to always write a complete html stream because some Office applications want that ( http://bugs.freepascal.org/view.php?id=29146, see also comment in front of TClipboard.SetAsHtml).

Show quoted text 8 lines
-- _______________________________________________ Lazarus mailing list http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20160502/0a7532cb/attachment.html>

Reply by email

Werner Pamler
to Luiz Americo Pereira Camara
Does not grid implies a 2D format? TLazGridTextExporter? TLazTextGridExporter? Is necessary Laz prefix?

Bart was talking of the ancestor of TGridExporter and TListviewExporter. No, the Laz prefix is not necessary; TExporter, on the other hand, is a very simple name which could exist in some other package already.

Reply by email

zeljko
to Luiz Americo Pereira Camara: Em 2 de mai de 2016 07:55, "Werner Pamler" Does not grid implies a 2D format?…
Luiz Americo Pereira Camara wrote 16 lines
Em 2 de mai de 2016 07:55, "Werner Pamler" < <mailto:>> escreveu: >> >> I would have the name of the base class (LazExporter) reflect that it >> is intended for use with "2D-Data", and that it exports text. > > TLaz2DTextExporter, or TCustom2DTextExporter? Or anything else? Does not grid implies a 2D format? TLazGridTextExporter? TLazTextGridExporter? Is necessary Laz prefix?

IMO, no, but TXXXXXExporter would be ok where XXXX = class name eg. TStringGridExporter, TListViewExporter, TTreeViewExporter

zeljko

Reply by email

Bart
to Werner Pamler: TLaz2DTextExporter, or TCustom2DTextExporter? Or anything else? I could give the…
Werner Pamler wrote
TLaz2DTextExporter, or TCustom2DTextExporter? Or anything else?

I'm rubbish when it comes to coming up with good names ;-)

Show quoted text 5 lines
I could give the HTMLExporter a corresponding property. But I see the main use case of html export in stand-alone html documents, therefore I disagree on the default setup (i.e. I'd name the property "WriteFragmentOnly" instead of "WriteFullHTML"). And how do you want to insert the table in your document?

I'm very old school. I'ld write it to a file, open the file then do copy-paste/

Show quoted text 5 lines
By copy to clipboard? You must remember that we had constructed the "Clipboard.SetAsHtml" method such as to always write a complete html stream because some Office applications want that (http://bugs.freepascal.org/view.php?id=29146, see also comment in front of TClipboard.SetAsHtml).

That in itself is a better argument for defaulting to a full html document. Option like WriteFragmentOnly would be welcome though. Note: for my own HTML editor i would need to copy the HTML code as text to the clipboard....

Bart

Reply by email

Werner Pamler
to Werner Pamler: TStringGrid provides the methods SaveToCSVFile/Stream for saving the grid content to a…
Werner Pamler wrote 92 lines
TStringGrid provides the methods SaveToCSVFile/Stream for saving the grid content to a csv file or stream. This is very convenient for users, but it is not very flexible: The export works only for the csv data format, no html, no possibility to plug in other formats. It requires always a modification of the /grids /unit to extend these methods by additional options. And it works only with TStringGrid - there are other grid-like controls, such as custom "virtual" grids derived from TCustomDrawGrid, TListview in report mode, TKGrid and other third-party grids which are not supported: if they do not have such a method on their own the export code has to be re-written (often: duplicated). The attached file contains code for a flexibile export system for 2D-data classes which is open and can easily be extended to other formats and other controls. It essentially consists of two parts: * *TLazExporter *represents the file format. The class is abstract and provides the methods to write cells and rows to stream and file. I implemented an exporter for *csv *and *html *files for the lclbase package. But in addition there is also a dedicated SpreadsheetExporter which takes care of *Excel *and/or *Opendocument *file formats (using fpspreadsheet). * *TLazExporterLink *provides the data to be exported. It is an abstract class between the control to be exported and the writer. It exposes methods to navigate from cell to cell and row to row, and to define the strings assigned to each cell. I implemented a TGridExporterLink (accessing *TCustomStringGrid*, in the long run: *TCustomDrawGrid*) and *TCustomListView*. The same principle works for any other classes with 2D data arrays (matrix). The unit /GridExporter /exposes functions to write grid content to file and to stream. The exporter instance is passed as a parameter, i.e. it is very easy to switch from csv to hml or any other format provided the corresponding exporter is available. procedure ExportGridToFile(AGrid: TCustomStringGrid; AExporter: TLazExporter; const AFileName: String; AOptions: TGridExportOptions); overload; procedure ExportGridToStream(AGrid: TCustomStringGrid; AExporter: TLazExporter; AStream: TStream; AOptions: TGridExportOptions); overload; Similarly, there are also ExportListviewToFile/Stream procedures in the unit /ListViewExporter /for the export from a TListview. Properties of the exporter can be used to fine-tune the export. In case of the html exporter, for example, a set of css statements can be specified to format the exported html table: htmlexporter := THTMLExporter.Create; with htmlexporter do begin CSS.Add('table { border: 1px solid #DDDDFF; }'); CSS.Add('th { background-color: #DDDDFF; }'); CSS.Add('h1 { font-family:Helvetica, Arial, sans-serif; font-size:16pt; color:blue; background-color: #EEEEFF}; }'); end; ExportGridToFile(StringGrid1, exporter, AFileNameForGrid, [geoFixedRows, geoFixedCols, geoVisibleColsOnly]); ExportListViewToFile(Listview1, exporter, AFileNameForListView, [leoTitles, leoVisibleColsOnly]); In addition to these procedures there are overloaded versions taking the exporter class (instead of an exporter instance) as a parameter. In this way the default exporter parameters are applied during the export. Of course, these procedures can be added as methods to the classes referred to by the first parameter. The attached demo shows the exporter requires no modification of any LCL packages or units. In the long run, however, I'd propose to add unit /LazExporter /to package /LazUtils /and /GridExporter /and /ListviewExporter /to package /LclBase /(/SpreadsheetExporter /would go to /laz_//fpspreadsheet/), and to make these modifications to the /grids /unit: * Extend the ExportGridToFile/Stream procedures to accept also TCustomDrawGrid descendants. This would be possible by introducing a virtual method GetCellText(Col,Row) to TCustomDrawGrid (or maybe even TCustomGrid) which would catch the cell text from a special event OnGetCellText; this would help to create "virtual" grids using TCustom(Draw)Grids. In case of the TStringGrid, of course, GetCellText would simply return the cell strings Cells[Col, Row]. * Replace the code in SaveToCSVFile/Stream by the csv exporter. I'd also vote to deprecate these methods because they are special cases of the csv exporter. * Replace the copy-to-clipboard code (CopyCellRectToClipboard) by the csv exporter. In addition, a html format could be written to the clipboard with almost no extra code. I would greatly appreciate any comments.

In preparing a patch for a Mantis feature request I came across this issue: fpc does already contain a series of exporters (packages/fcl-db/export, component palette "Data Export"), among them a TCSVExporter, TSimpleXMLExporter etc. Since they focus on database export there is no overlap with the new exporters primarily. But they make things a lot more complicated...

The easiest way would be to stay with the "two exporter" solution, one set for database, the other one for non-database export. The new exporters could be renamed to contain a prefix "Laz" in their name, or "Text", or "NonDB", or similar.

On the other hand, it should be possible to generalize the fcl exporters such that they support also non-database sources. This would avoid a lot of code duplication, and it would present a clearer class hierarchy for data export to the user. BUT: A lot of refactoring of the corresponding fcl code is involved in this solution. Therefore, since Lazarus will support the last pre-3.0 version of fpc for a long time, this solution will not be available in a Lazaraus released version for the next couple of years...

What's your opinion out there?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lazarus.freepascal.org/pipermail/lazarus/attachments/20160506/ad800416/attachment.html>

Reply by email

Bart
to Werner Pamler
Werner Pamler wrote
In preparing a patch for a Mantis feature request I came across this issue: fpc does already contain a series of exporters (packages/fcl-db/export, component palette "Data Export"), among them a TCSVExporter, TSimpleXMLExporter etc. ....

I would create our own set.
You could try to implement it in such a way that rewriting it in the future to use fpc features, may become less cumbersome. The fpc development cycle is far slower than ours, and it would mean > 1 year waiting for this feature.

Keep the Laz prefix in the base class.

My 2 cents.

About the name TLazExporter:
The base class should probably start with "TCustom" ?

About writing a patch:
Don't you have svn commit (to LCL)?
If so, just commit it, since nobody seems to object to the basic philosophy of your idea and implementation.
Once committed, others will participate.

Since it is only in trunk, it does not really matter if (even radical) changes are made to the system in the road to the next major release.

Bart

Reply by email

Graeme Geldenhuys
to Werner Pamler: TStringGrid provides the methods SaveToCSVFile/Stream for saving the grid content to a…
Werner Pamler wrote
TStringGrid provides the methods SaveToCSVFile/Stream for saving the grid content to a csv file or stream. This is very convenient for users, but it is not very flexible: The export works only for the csv data format, no html, no possibility to plug in other formats.

And this is why I think it is crazy that the Lazarus developers went that route. Plus, why is the Grid now burdened with file and exporting/importing functionality. They should have kept those functionalities separate. The same applies for populating a grid from a data file (csv, xml, json etc). And that?s exactly what I did in fpGUI - separate functionality, thus separate classes and units.

I think your idea is definitely a step in the right direction. The TStringGrid?s SaveToCSVxxx and LoadFromCSVxxx methods should also be marked deprecated and after one release be removed - replaced by something like what you created.

Regards,
Graeme

fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal http://fpgui.sourceforge.net/ My public PGP key: http://tinyurl.com/graeme-pgp

Reply by email

Jesus Reyes A.
to Bart: I would create our own set. You could try to implement it in such a way that rewriting it…

En Fri, 06 May 2016 15:51:41 -0500, Bart escribi?:

Show quoted text 8 lines
On 5/6/16, Werner Pamler wrote: ... About writing a patch: Don't you have svn commit (to LCL)? If so, just commit it, since nobody seems to object to the basic philosophy of your idea and implementation. Once committed, others will participate.

+1

I would only suggest to keep the new units to a minimum, or if two units are needed name the grid related code as gridutils.

Jesus Reyes A.

Reply by email