CsvDocument: what changed

Revision 0 to 1. Removed lines have a minus sign; added lines have a plus sign.

Title: CsvDocument

New		Line
1	+	### About
2	+
3	+	CsvDocument library is a unit contaning set of classes for [CSV files](http://en.wikipedia.org/wiki/Comma-separated_values) handling. The library was created to exchange data with OpenOffice Calc / MS Office Excel using CSV as intermediate format.
4	+
5	+	Its main characteristics are:
6	+
7	+	- Random read/write access to CSV fields based on object model (TCSVDocument class). This requires preloading CSV file content into memory but allows to edit any field at any moment.
8	+	- Serial access (field-by-field) parsing of CSV files using TCSVParser class. It helps to read CSV file content directly without preloading the whole file into memory.
9	+	- Field-by-field building of CSV files using TCSVBuilder class. This helps to write CSV content directly to file and thus avoid using in-memory buffer. Implemented in version 0.4 and above.
10	+	- CSV implementation is compatible with that of OpenOffice Calc / MS Office Excel. This means CSV files saved from Calc/Excel can be edited using CsvDocument library and vice versa.
11	+	- Both UTF-8 encoding and Windows-xxx codepages can be used with the CsvDocument library. The library uses string type for all string operations and will not do any encoding conversion for you. Keep in mind though that Excel does not support CSV files in UTF-8.
12	+	- Support for line breaks embedded into CSV fields. It was one of the reasons to reinvent the wheel. OO Calc supports this feature as well, but MS Excel does not.
13	+	- StringGrid-like field access with TCsvDocument.Cells\[ACol, ARow\]. Field access is safe: i.e. when you try to access non-existing field you do not get "Index out of bounds" exception, but get an empty string result. To distingush between empty and non-existing fields there are methods and properties like HasRow/HasCell and RowCount/ColCount.
14	+	- Support for search in single row/column using IndexOfRow(AString, ACol) / IndexOfCol(AString, ARow). Implemented in version 0.3 and above.
15	+	- No additional limits on field length, number of rows/fields, etc. other than performance, memory usage, string type limit of 2 Gb and signed 32 bit integer value limit.
16	+	- Compliance with RFC 4180 (http://tools.ietf.org/html/rfc4180#section-2) using default settings, as of version 0.4 and above.
17	+	- Compliance with unofficial CSV specification (http://www.creativyst.com/Doc/Articles/CSV/CSV01.htm#FileFormat), as of version 0.3 and above. Requires setting IgnoreOuterWhitespace to True in version 0.4 and above.
18	+
19	+	### Author
20	+
21	+	Vladimir Zhirov
22	+
23	+	### Contributors
24	+
25	+	Luiz Americo Pereira Camara
26	+
27	+	Mattias Gaertner
28	+
29	+	Reinier Olislagers
30	+
31	+	### License
32	+
33	+	[Modified](http://svn.freepascal.org/svn/lazarus/trunk/COPYING.modifiedLGPL.txt)[LGPL](http://svn.freepascal.org/svn/lazarus/trunk/COPYING.LGPL.txt) (same as FPC RTL and Lazarus LCL).
34	+
35	+	### Dependencies
36	+
37	+	The library is a single Pascal source file that only depends on FPC RTL and FCL.
38	+
39	+	### Availability/Download
40	+
41	+	Since FPC 3.0, CSVDocument belongs to the FCL (folder fcl-base/src, units csvreadwrite.pp and csvdocument.pp) where it is actively maintained.
42	+
43	+	An older version is available at Lazarus-CCR and kept for compatibility, but not maintained any more: <http://sourceforge.net/projects/lazarus-ccr/files/CsvDocument/CsvDocument%200.5/csvdocument-0.5.zip/download>
44	+
45	+	### Change Log
46	+
47	+	#### Version 0.1 (2010-01-22)
48	+
49	+	- initial release.
50	+
51	+	#### Version 0.2 (2010-05-31)
52	+
53	+	- fixed bug in line ending conversion.
54	+	- fixed compilation with range checking on.
55	+	- exposed QuoteCSVString function to simplify CSV generation without TCsvDocument class.
56	+	- minor code cleanup.
57	+
58	+	#### Version 0.3 (2011-01-14)
59	+
60	+	- CsvParser API changed: callback methods were replaced by ResetParser/ParseNextCell methods to allow using the library in pure functional programs. See example of new API usage in TCSVDocument.LoadFromStream.
61	+	- CsvDocument API changed: TrimEmptyCells were renamed to RemoveTrailingEmptyCells.
62	+	- support for trimming leading and trailing whitespace in fields. New TrimWhitespace property added. Warning: TrimWhitespace is enabled by default.
63	+	- support for search in single row/column using IndexOfRow(AString, ACol) / IndexOfCol(AString, ARow).
64	+	- performance improvements.
65	+	- code cleanup.
66	+	- added simple test suite.
67	+
68	+	#### Version 0.4 (2011-05-10)
69	+
70	+	- replaced QuoteCSVString function with TCSVBuilder class. See example of its usage in TCSVDocument.SaveToStream.
71	+	- renamed TrimWhitespace to IgnoreOuterWhitespace (to be consistent with newly introduced QuoteOuterWhitespace property).
72	+	- support for equal column count throughout the document (required by RFC 4180). See new EqualColCountPerRow property.
73	+	- changed default settings to RFC 4180 compliant ones (comma as Delimiter, double quote as QuoteChar, CRLF as line ending, ignoring outer whitespace off, equal column count on).
74	+	- implemented conversion of line endings embedded into fields when parsing and building a CSV file. It helps to prevent mixed line endings in output file when using OS-specific line endings and working with the same file on multiple OSes.
75	+	- moved CSV format settings to TCSVHandler class (an ancestor of TCSVParser, TCSVBuilder and TCSVDocument).
76	+	- updated test suite for RFC 4180 compliance.
77	+	- fixed minor annoyances in demo application.
78	+	- included optimized ChangeLineEndings function by Mattias Gaertner.
79	+	- speed optimizations, demo improvements and Lazarus package by Luiz Americo Pereira Camara.
80	+
81	+	#### Version 0.5 (2014-10-25)
82	+
83	+	- fixed bug in TrimEmptyCells method, it was removing empty cells in the middle of a row (patch by Reinier Olislagers)
84	+	- implemented MaxColCount property: the maximum number of columns found parsing a CSV file (patch by Reinier Olislagers)
85	+	- commented procedures/functions/properties (based on patch by Reinier Olislagers)
86	+	- fixes in demo application (updating view after loading CSV document, separate project files for Lazarus 0.9.30)
87	+
88	+	#### Since move to FCL (2015-04-04)
89	+
90	+	- Please see the GIT commit notes in the FPC repository
91	+
92	+	### Installation
93	+
94	+	No "real" installation needed. Simply include unit `CsvDocument` and/or `csvreadwrite` in the `uses` section.
95	+
96	+	### Usage examples
97	+
98	+	#### Extracting a specific cell from a CSV document
99	+
100	+	``` pascal
101	+	uses
102	+	csvdocument;
103	+
104	+	function ReadCSVCell(AFileName: String; ARow, ACol: Integer): String;
105	+	var
106	+	CSVDoc:TCSVDocument;
107	+	begin
108	+	CSVDoc := TCSVDocument.Create;
109	+	try
110	+	CSVDoc.Delimiter := ';';
111	+	CSVDoc.LoadFromFile(AFileName);
112	+	Result := CSVDoc.Cells[ACol, ARow];
113	+	finally
114	+	CSVDoc.Free;
115	+	end;
116	+	end;
117	+	```
118	+
119	+	#### Loading a csv document into a StringGrid
120	+
121	+	This sample code uses `TCSVParser` from the `csvreadwrite` unit to load a CSV/TSV/semicolon-separated file into a stringgrid, similar to StringGrid.LoadFromCSVFile but it supports all file format variations that `CSVDocument` supports.
122	+
123	+	The advantage of using `TCSVParser` versus `TCSVDocument` is that you can e.g. parse only the first 20 rows and load them into the grid, which saves a lot of memory if dealing with large CSV files.
124	+
125	+	``` pascal
126	+	unit stringgridutil;
127	+	{ Use csvreadwrite to import csv data into stringgrid. Offers more robust support for CSV file variations than TStringGrid.LoadFromCSVFile }
128	+
129	+	{$mode objfpc}{$H+}
130	+	interface
131	+
132	+	uses
133	+	Classes, SysUtils, Grids, csvreadwrite, LazFileUtils;
134	+	// replace "csvreadwrite" by "csvdocument" if you use the out-dated version from CCR.
135	+
136	+	procedure LoadGridFromCSVFile(Grid: TStringGrid;AFilename: string;
137	+	ADelimiter:Char=','; WithHeader:boolean=true; AddRows:boolean=true);
138	+	// Loads (quasi)CSV document in AFilename into Grid, using ADelimiter as
139	+	// delimiter. If WithHeader is true, it won't import the first row.
140	+	// If AddRows is true, it will add rows if the existing grid is too short.
141	+
142	+	implementation
143	+
144	+	procedure LoadGridFromCSVFile(Grid: TStringGrid;AFilename: string;
145	+	ADelimiter:Char=','; WithHeader:boolean=true;AddRows:boolean=true);
146	+	const
147	+	DefaultRowCount=10; //Number of rows to show by default
148	+	var
149	+	FileStream: TFileStream;
150	+	Parser: TCSVParser;
151	+	RowOffset: integer;
152	+	begin
153	+	Grid.BeginUpdate;
154	+	// Reset the grid:
155	+	Grid.Clear;
156	+	Grid.RowCount:=DefaultRowCount;
157	+	Grid.ColCount:=6; //Vaguely sensible
158	+	if not(FileExistsUTF8(AFileName)) then exit;
159	+
160	+	Parser:=TCSVParser.Create;
161	+	FileStream := TFileStream.Create(AFilename, fmOpenRead+fmShareDenyWrite);
162	+	try
163	+	Parser.Delimiter:=ADelimiter;
164	+	Parser.SetSource(FileStream);
165	+
166	+	// If the grid has fixed rows, those will not receive data, so we need to
167	+	// calculate the offset
168	+	RowOffset:=Grid.FixedRows;
169	+	// However, if we have a header row in our CSV data, we need to
170	+	// discount that
171	+	if WithHeader then RowOffset:=RowOffset-1;
172	+
173	+	while Parser.ParseNextCell do
174	+	begin
175	+	// Stop if we've filled all existing rows. Todo: check for fixed grids etc, but not relevant for our case
176	+	if AddRows=false then
177	+	if Parser.CurrentRow+1>Grid.RowCount then break; //VisibleRowCount doesn't seem to work.
178	+
179	+	// Widen grid if necessary. Slimming the grid will come after import done.
180	+	if Grid.Columns.Enabled then
181	+	begin
182	+	if Grid.Columns.VisibleCount<Parser.CurrentCol+1 then Grid.Columns.Add;
183	+	end
184	+	else
185	+	begin
186	+	if Grid.ColCount<Parser.CurrentCol+1 then Grid.ColCount:=Parser.CurrentCol+1;
187	+	end;
188	+
189	+	// If header data found, and a fixed row is available, set the caption
190	+	if (WithHeader) and
191	+	(Parser.CurrentRow=0) and
192	+	(Parser.CurrentRow<Grid.FixedRows-1) then
193	+	begin
194	+	// Assign header data to the first fixed row in the grid:
195	+	Grid.Columns[Parser.CurrentCol].Title.Caption:=Parser.CurrentCellText;
196	+	end;
197	+
198	+	// Actual data import into grid cell, minding fixed rows and header
199	+	if Grid.RowCount<Parser.CurrentRow+1 then
200	+	Grid.RowCount:=Parser.CurrentRow+1;
201	+	Grid.Cells[Parser.CurrentCol,Parser.CurrentRow+RowOffset]:=Parser.CurrentCellText;
202	+	end;
203	+
204	+	// Now we know the widest row in the import, we can snip the grid's
205	+	// columns if necessary.
206	+	if Grid.Columns.Enabled then
207	+	begin
208	+	while Grid.Columns.VisibleCount>Parser.MaxColCount do
209	+	begin
210	+	Grid.Columns.Delete(Grid.Columns.Count-1);
211	+	end;
212	+	end
213	+	else
214	+	begin
215	+	if Grid.ColCount>Parser.MaxColCount then
216	+	Grid.ColCount:=Parser.MaxColCount;
217	+	end;
218	+
219	+	finally
220	+	Parser.Free;
221	+	FileStream.Free;
222	+	Grid.EndUpdate;
223	+	end;
224	+	end;
225	+	end.
226	+	```
227	+
228	+	### Contact
229	+
230	+	Please send bug reports and patches to the FPC gitlab bugtracker (https://gitlab.com/freepascal.org/fpc/source/-/issues).
231	+
232	+	### See Also
233	+
234	+	- [[CSV]]
235	+	- [[FPSpreadsheet]]
236	+	- [[JCSV (Jans CSV Components)]]
237	+
238	+	---
239	+	From the Free Pascal wiki, page "CsvDocument", as it stood on 2022-06-29 after a change by Wp, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).
240	+	Category: Components
241	+	Category: Data import and export
242	+	Category: Lazarus-CCR

Back to history