Codetools: what changed

Revision 3 to 4. Removed lines have a minus sign; added lines have a plus sign.

Old	New		Line
1	1		# What are the codetools?
2	2
3	3		Codetools is a Lazarus package providing tools to parse, explore, edit and refactor Pascal sources. The codetools have been packaged in their own module and are licensed under GPL. Many examples showing how to use the codetools in your own programs can be found under components/codetools/examples.
4	4
5	5		svn:
6	6
7	7		- Lazarus: <http://svn.freepascal.org/svn/lazarus/trunk>
8	8		- Only CodeTools: <http://svn.freepascal.org/svn/lazarus/trunk/components/codetools>
9	9
10	10		# Using the codetools without the IDE
11	11
12	12		You can use the codetools without the IDE. This can be used to test a new tool. An easy example is
13	13
14		-	~~` <lazarusdir>/components/codetools/examples/methodjumping.lpi`~~
	14	+	```
	15	+	<lazarusdir>/components/codetools/examples/methodjumping.lpi
	16	+	```
15	17
16	18		To test find declaration, the codetools need to parse sources. Especially the RTL and FCL sources. The examples use the following environment variables:
17	19
18	20		- FPCDIR: path to the FPC sources, the default is ~/freepascal/fpc.
19	21		- PP: path to the compiler executable (/usr/bin/fpc or /usr/bin/ppc386 or C:\lazarus\ppc386.exe). The codetools need to ask the compiler for the settings. The default is to search for 'fpc' via the PATH variable.
20	22		- FPCTARGETOS: tell the codetools to scan for another operating system (cross compiling). For example: linux, freebsd, darwin, win32, win64, wince
21	23		- FPCTARGETCPU: when scanning for another CPU. For example: i386, powerpc, x86_64, arm, sparc
22	24		- LAZARUSDIR: path of the lazarus sources. Only needed if you want to scan them.
23	25
24	26		FPC is a very complex project with lots of search paths, include files and macros. The codetools need to know all these paths and macros in order to parse this jungle. To setup all this easily the codetools contain predefined templates for FPC, Lazarus, Delphi and Kylix source directories. See for a find declaration example
25	27
26		-	~~` <lazarusdir>/components/codetools/examples/finddeclaration.lpi`~~
	28	+	```
	29	+	<lazarusdir>/components/codetools/examples/finddeclaration.lpi
	30	+	```
27	31
28	32		Because the FPC sources contain multiple versions of some units, and the FPC sources change often, the codetools do not use a fixed table of paths, but rather scan the whole FPC directory structure on first use, applying a set of rules specifying what is the correct source for the current TargetOS and TargetCPU. This scan may take a while depending on your disk speed. All examples save the result in codetools.config, so that the next time Lazarus is started this possibly lengthy scan is skipped.
29	33
30	34		Whenever the FPC sources are changed or a unit is renamed, just delete the codetools.config file. The Lazarus IDE has its own config file and does a rescan whenever the compiler executable changes (or when the user forces a 'Tools \> Rescan FPC source directory').
31	35
32	36		## Defining search paths and macros
33	37
34	38		Important: Codetools assumes that you have multiple directories with different compiler settings, e.g. the FPC sources have different search paths than your sources. You can have FPC, Delphi and pas2js directories - all at the same time.
35	39
36	40		The codetools uses "define templates" to generate search paths and macros. Define templates are trees of rules.
37	41
38	42		### Example: Expand IncPath for a directory and/or its sub directories
39	43
40	44		The following demo demonstrate various ways to expand the IncPath for a directory and/or its sub directories:
41	45
42	46		``` pascal
43	47		program SetIncludePath;
44	48
45	49		{$mode objfpc}{$H+}
46	50
47	51		uses
48	52		SysUtils, CodeToolManager, DefineTemplates, LazFileUtils;
49	53
50	54		var
51	55		Directory: String;
52	56		DirectoryTemplate: TDefineTemplate;
53	57		IncPathTemplate: TDefineTemplate;
54	58		SubDirectory: String;
55	59		begin
56	60		// create a template for the current directory
57	61		// all child nodes of this template are only valid for this directory.
58	62		Directory:=ExpandFileNameUTF8(GetCurrentDirUTF8);
59	63		DirectoryTemplate:=TDefineTemplate.Create('Current working directory',
60	64		'Example template for current working directory','',Directory,da_Directory);
61	65
62	66		// Example 1:
63	67		// Add a sub template to extend the include search path #IncPath
64	68		// only for the current directory, not its sub directories.
65	69		// Note: 'myincludes' is a relative path, so it won't work for sub directories.
66	70		IncPathTemplate:=TDefineTemplate.Create('Add myincludes to the IncPath',
67	71		'Add myincludes to the include search path',
68	72		IncludePathMacroName, // variable name: #IncPath
69	73		IncludePathMacro+';myincludes' // new value: $(#IncPath);myincludes
70	74		,da_Define // da_Define extends the IncPath only for this directory
71	75		);
72	76		DirectoryTemplate.AddChild(IncPathTemplate);
73	77
74	78		// Example 2:
75	79		// Add a sub template to extend the include search path #IncPath
76	80		// with an absolute path for the current directory and all its sub directories.
77	81		// Note: '/tmp/myincludes' is an absolute path, so works for sub directories.
78	82		IncPathTemplate:=TDefineTemplate.Create('Add /tmp/myincludes to the IncPath',
79	83		'Add /tmp/myincludes to the include search path',
80	84		IncludePathMacroName, // variable name: #IncPath
81	85		IncludePathMacro+';'+SetDirSeparators('/tmp/myincludes') // new value: $(#IncPath);/tmp/myincludes
82	86		,da_DefineRecurse // da_DefineRecuse extends the IncPath for this directory and all sub directories
83	87		);
84	88		DirectoryTemplate.AddChild(IncPathTemplate);
85	89
86	90		// Example 3:
87	91		// Using the #DefinePath macro you can use the current directory to create absolute paths.
88	92		IncPathTemplate:=TDefineTemplate.Create('Add ./myincludes to the IncPath',
89	93		'Add ./myincludes to the include search path',
90	94		IncludePathMacroName, // variable name: #IncPath
91	95		IncludePathMacro+';'+DefinePathMacro+'/myincludes' // new value: $(#IncPath);$(#DefinePath)/myincludes
92	96		,da_DefineRecurse // da_DefineRecuse extends the IncPath for this directory and all sub directories
93	97		);
94	98		DirectoryTemplate.AddChild(IncPathTemplate);
95	99
96	100		// add the directory template to the tree
97	101		CodeToolBoss.DefineTree.Add(DirectoryTemplate);
98	102
99	103		writeln('Directory="',Directory,'"',
100	104		' IncPath="',CodeToolBoss.GetIncludePathForDirectory(Directory),'"');
101	105		SubDirectory:=AppendPathDelim(Directory)+'sub';
102	106		writeln('SubDirectory="',SubDirectory,'"',
103	107		' IncPath="',CodeToolBoss.GetIncludePathForDirectory(SubDirectory),'"');
104	108		end.
105	109		```
106	110
107	111		### Example: Expand global UnitPath
108	112
109	113		Here is an example how to extend the global (all directories) unit search path and query the unit path of one directory. This might be useful for single minded command line tools.
110	114
111	115		``` pascal
112	116		uses
113	117		Classes, SysUtils, CodeToolManager, DefineTemplates, FileProcs;
114	118
115	119		var
116	120		Directory: String;
117	121		UnitPathTemplate: TDefineTemplate;
118	122		begin
119	123		// add a sub template to extend the include search path #UnitPath.
120	124		UnitPathTemplate:=TDefineTemplate.Create(
121	125		'Add myunits to the UnitPath', // optional: an arbitrary name of the template, useful for finding it later
122	126		'Add /tmp/myunits to the unit search path', // optional: a description
123	127		UnitPathMacroName, // variable name: #UnitPath
124	128		UnitPathMacro+';/tmp/myunits' // new value: $(#UnitPath);/tmp/myunits
125	129		,da_DefineRecurse
126	130		);
127	131		// add the unit path template to the tree
128	132		CodeToolBoss.DefineTree.Add(UnitPathTemplate);
129	133
130	134		Directory:=ExpandFileNameUTF8(GetCurrentDirUTF8);
131	135		writeln('Directory="',Directory,'"',
132	136		' UnitPath="',CodeToolBoss.GetUnitPathForDirectory(Directory),'"');
133	137		end.
134	138		```
135	139
136	140		## Define Template Macros
137	141
138	142		Macros defined by define templates are used by the Pascal parser. The Codetools have functions to query the Free Pascal Compiler for its macros and convert them to define templates. This is done by CodeToolBoss.Init(Options); which you can find in many examples.
139	143
140	144		Additionally there some predefined macros, which start with a hash (#). See unit definetemplates for the full list. Here are some important ones:
141	145
142	146		``` pascal
143	147		ExternalMacroStart = '#';
144	148		// Standard macros
145	149		DefinePathMacroName = ExternalMacroStart+'DefinePath'; // the current directory
146	150		UnitPathMacroName = ExternalMacroStart+'UnitPath'; // unit search path separated by semicolon (same as given to FPC)
147	151		IncludePathMacroName = ExternalMacroStart+'IncPath'; // include file search path separated by semicolon (same as given to FPC)
148	152		SrcPathMacroName = ExternalMacroStart+'SrcPath'; // unit source search path separated by semicolon (not given to FPC)
149	153		```
150	154
151	155		For example the IncludePathMacroName is \#IncPath and is used to define the include file search path. Keep in mind that macro values depend on the directory.
152	156
153	157		Define templates values can contain macros. The macro has to be enclosed in dollar and brackets: \$(macro).
154	158
155	159		In the above example the value was '\$('+IncludePathMacroName+');/tmp/myincludes', which is '\$(#IncPath);/tmp/myincludes', which executes to the old include search path plus ';/tmp/myincludes', which means a path is appended. For readability you can use the constant IncludePathMacro instead of '\$('+IncludePathMacroName+')':
156	160
157	161		``` pascal
158	162		DefinePathMacro = '$('+DefinePathMacroName+')'; // the path of the define template
159	163		UnitPathMacro = '$('+UnitPathMacroName+')';
160	164		IncludePathMacro = '$('+IncludePathMacroName+')';
161	165		SrcPathMacro = '$('+SrcPathMacroName+')';
162	166		```
163	167
164	168		## Define Template Actions
165	169
166	170		A define template has a name, an optional description, a variable, a value and an action. The name and description are optional. The meaning of variable and value depends on the action. You can add define templates as children of define templates - creating a tree of define templates. You can see many examples for define templates in the Lazarus' dialog Tools / CodeTools Defines Editor.
167	171
168	172		- da_Block - Used for grouping templates. When the block is executed all children are executed.
169	173		- da_Directory - Use this to define all rules of a directory. If this is a root directory set Value to the full expanded directory path (use function CleanAndExpandDirectory). If this is a sub directory (parent template is a directory) Value is the sub path. Can contain macros. Children are only executed if directory fits.
170	174		- da_Define - sets a macro (Variable) value (Value) of the current directory. The value can contain macros. Note that when a macro value is set to empty string it is still defined, that means{\$IFDEF variable}will still result intrue''. Children are not executed.
171	175		- da_Undefine - clears a macro (Variable) of the current directory. {\$IFDEF macro} results in false. Children are not executed.
172	176		- da_DefineRecurse - as da_Define, but for current directory and sub directories.
173	177		- da_UndefineRecurse - as da_Undefine, but for current directory and sub directories.
174	178		- da_UndefineAll - clear all macro values.
175	179		- da_IfDef - if macro Variable is defined then execute children and skip following da_Else and da_ElseIf.
176	180		- da_IfNDef - if macro Variable is not defined then execute children and skip following da_Else and da_ElseIf.
177	181		- da_If,daElseIf - if the boolean expression Value executes to true then execute children and skip following da_Else and da_ElseIf. Value can contain macros.
178	182		- da_Else - When this template is executed then execute all children.
179	183
180	184		## How to extend the include path of a directory
181	185
182	186		See the example lazarus/components/codetools/examples/setincludepath.lpr. It demonstrates the use of relative paths, absolute paths, how to use the DefinePathMacro and the difference between da_Define and da_DefineRecurse.
183	187
184	188		# Using the codetools in the IDE with the IDEIntf
185	189
186	190		See \<lazarusdir\>/examples/idequickfix/quickfixexample.lpk package. It demonstrates:
187	191
188	192		- How to write an IDE package. When you install this package it will register a Quick Fix item in the IDE.
189	193		- How to write a Quick Fix item for the compiler message: 'Parameter "Sender" not used'
190	194		- How to use the codetools to:
191	195		- parse a unit
192	196		- convert Filename, Line, and Column data into a codetools source position
193	197		- find a codetools node at a cursor position
194	198		- find a procedure node, and the begin..end node
195	199		- create a nice insertion position for a statement at the beginning of the begin..end block
196	200		- obtain line indentation information so that a new line will work in a sub-procedure as well
197	201		- insert code using the codetools
198	202
199	203		# Codetools rules for FPC sources
200	204
201	205		When the codetools searches the source of a fpc ppu it uses a set of rules. You can write your own rules, but normally you will use the standard rules, which are defined in the include file components/codetools/fpcsrcrules.inc. You can test the rules with the command line utility: components/codetools/examples/testfpcsrcunitrules.lpi.
202	206
203	207		## Usage of testfpcsrcunitrules
204	208
205	209		Usage: lazarus/components/codetools/examples/testfpcsrcunitrules -h
206	210
207	211		-c <compiler file name>, --compiler=<compiler file name>
208	212		Default is to use environment variable PP.
209	213		If this is not set, search for fpc
210	214
211	215		-T <target OS>, --targetos=<target OS>
212	216		Default is to use environment variable FPCTARGET.
213	217		If this is not set, use the default of the compiler.
214	218
215	219		-P <target CPU>, --targetcpu=<target CPU>
216	220		Default is to use environment variable FPCTARGETCPU.
217	221		If this is not set, use the default of the compiler.
218	222
219	223		-F <FPC source directory>, --fpcsrcdir=<FPC source directory>
220	224		Default is to use environment variable FPCDIR.
221	225		There is no default.
222	226
223	227		-u <unit name>, --checkunit=<unit name>
224	228		Write a detailed report about this unit.
225	229
226	230		## Example for testfpcsrcunitrules
227	231
228	232		Open the testfpcsrcunitrules.lpi in the IDE and compile it. Then run the utility in a terminal/console:
229	233
230		-	~~`./testfpcsrcunitrules -F ~/fpc/sources/2.5.1/fpc/`~~
	234	+	```
	235	+	./testfpcsrcunitrules -F ~/fpc/sources/2.5.1/fpc/
	236	+	```
231	237
232	238		This will tell you which compiler is used, which compiler executes, which config files were tested and parsed, and it warns you about duplicate units in the FPC search path and about duplicated unit source files. Note: This example caches results in the file codetools.config. You should delete codetools.config when you update the compiler or your fpc.cfg file.
233	239
234	240		## Duplicate source files
235	241
236	242		You find out that the codetools opens for target wince/arm the wrong source of the unit mmsystem. Run the tool with the -u parameter:
237	243
238	244		./testfpcsrcunitrules -F ~/fpc/2.5.1/fpc/ -T wince -P arm -u mmsystem
239	245
240	246		This will give you a detailed report where this unit was found and what score each source file got. For example:
241	247
242		-	~~`Unit report for mmsystem`\~~
243		-	~~` WARNING: mmsystem is not in PPU search path`\~~
244		-	~~`GatherUnitsInFPCSources UnitName=mmsystem File=packages/winunits-base/src/mmsystem.pp Score=11`\~~
245		-	~~`GatherUnitsInFPCSources UnitName=mmsystem File=packages/winceunits/src/mmsystem.pp Score=11 => duplicate`~~
	248	+	``` pascal
	249	+	Unit report for mmsystem
	250	+	WARNING: mmsystem is not in PPU search path
	251	+	GatherUnitsInFPCSources UnitName=mmsystem File=packages/winunits-base/src/mmsystem.pp Score=11
	252	+	GatherUnitsInFPCSources UnitName=mmsystem File=packages/winceunits/src/mmsystem.pp Score=11 => duplicate
	253	+	```
246	254
247	255		This means there are two source files with the same score, so the codetools took the first. The last one in winceunits is for target wince and the first one is for win32 and win64.
248	256
249	257		Now open the rules file fpcsrcrules.inc.
250	258
251	259		Rules work like this:
252	260
253	261		``` pascal
254	262		Score:=10;
255	263		Targets:='wince';
256	264		Add('packages/winceunits');
257	265		```
258	266
259	267		The Add adds a rule for all files beginning with 'packages/winceunits' that adds a score of 10 to all these files. The Targets is a comma separated list of target operating systems and/or target processors. For example Targets='wince,linux,i386' means: apply this rules to TargetOS wince or linux and to all TargetCPU i386.
260	268
261	269		# How the codetools parses sources differently from the compiler
262	270
263	271		A compiler is optimized to parse code linearly and to load needed units and include files as soon as it parses a uses section or a directive. The codetools are optimized to parse only certain code sections. For example jumping from the method declaration to the method body only needs the unit and its include files. When a codetool searches a declaration it searches backwards. That means it starts searching in the local variables, then upwards from the implementation. When it finds a uses section it searches the identifiers in the interface section of the units. When the identifier is found it stops. The result and some intervening steps are cached. Because it often only needs to parse a few interface sections it finds an individual identifier very quickly.
264	272
265	273		The codetools do not parse a source in a single step (as the compiler does) but in several steps, which depend on what the current codetool needs:
266	274
267	275		- First a source file is loaded in a TCodeBuffer. The IDE uses this step to change the encoding to UTF8. The files are kept in memory and only reloaded if the modification date changes or if a file is manually reverted. There are several tools and functions which work directly on the buffer.
268	276		- The next step is to parse the unit (or include file). A unit must be parsed from the beginning, so the codetools tries to find the main file, the first file of a unit. It does that by looking for a directive in the first line like {%MainUnit ../lclintf.pp}. If that does not exist, it searches in the includelink cache. The IDE saves this cache to disk, so the IDE learns over time.
269	277		- After finding the main file a TCodeTool is created and a TLinkScanner parses the source. It handles compiler directives, such as include directives and conditional directives. The scanner can be given a range, so it might, for instance, parse only a unit's interface. The scanner creates the clean source. The clean source is put together from all include files, having been stripped of code in the else part of conditional directives, which is skipped. It also creates a list of links which map the clean source to the real source files. The clean source is now Pascal containing no else code. Note: there are also tools designed to scan a single source for all directives. These tools create a tree of directives.
270	278		- After creating the clean source a TCodeTool parses it and creates a tree of TCodeTreeNode. It can also be given a range. This parser skips a few parts, for example class members, begin..end blocks and parameter lists. Many tools don't need them. These sub nodes are created on demand. A TCodeTreeNode has a range StartPos..EndPos which are clean positions, that means positions in the clean source. There are only nodes for the important parts. Creating nodes for every detail would need more memory than the source itself and is seldom needed. There are plenty of functions to find out the details. For example if a function has the 'cdecl' calling convention.
271	279		- When searching for an identifier the search stores the base types it finds and creates caches for all identifiers in the interface section.
272	280
273	281		Every level has its own caches, which need to be checked and updated before calling a function. Many high level functions accessible via the CodeToolBoss do that automatically. For others it is the responsibility of the caller.
274	282
275	283		Example for:
276	284
277	285		unit1.pas:
278	286
279	287		``` pascal
280	288		unit Unit1;
281	289		{$I settings.inc}
282	290		interface
283	291		uses
284	292		{$IFDEF Flag}
285	293		unix,
286	294		{$ELSE}
287	295		windows,
288	296		{$ENDIF}
289	297		Classes;
290	298		```
291	299
292	300		settings.inc:
293	301
294	302		``` pascal
295	303		{%MainUnit unit1.pas}
296	304		{$DEFINE Flag}
297	305		```
298	306
299	307		clean source:
300	308
301	309		``` pascal
302	310		unit Unit1;
303	311		{$I settings.inc}{%MainUnit unit1.pas}
304	312		{$DEFINE Flag}
305	313		interface
306	314		uses
307	315		{$IFDEF Flag}
308	316		unix,
309	317		{$ELSE}{$ENDIF}
310	318		Classes;
311	319		```
312	320
313	321		Hint: To easily parse a unit and build the nodes, use CodeToolBoss.Explore.
314	322
315	323		# Tool, Node, LinkScanner
316	324
317	325		## Tool
318	326
319	327		A tool is a TCodeTool. For Pascal files every unit, program and package gets its own TCodeTool. These are automatically created by the CodetoolBoss when parsing sources. A tool has a Scanner (TLinkScanner) and a Root (TCodeTreeNode), which is the root node of the tree of Pascal nodes of the module (e.g. unit).
320	328
321	329		## LinkScanner
322	330
323	331		Every TCodeTool creates a TLinkScanner to scan the files and creates the CleanedSrc, which is the parsed unit source, include files included and code in skipped \$IF directives removed. When only the unit interface was parsed, the CleanedSrc may not be complete. TLinkScanner is not shared between TCodeTools. There is a 1:1 mapping between TLinkScanner and TCodeTool.
324	332
325	333		## Node
326	334
327	335		- A TCodeTreeNode is part of a tree of such nodes. The root node is stored in a TCodeTool.Root. Nodes are not shared between tool. Every node is associated with exactly one TCodeTool.
328	336		- When a function returns a Node, it always returns the associated Tool as well.
329	337		- Node.StartPos/EndPos are positions 1-based in the Scanner.CleanedSrc.
330	338		- To get from the cleaned postion to the file, line, column use Tool.CleanPosToCaret(Node.StartPos,xyp).
331	339
332	340		# CleanPos and CursorPos
333	341
334	342		There are several methods to define a position in the codetools.
335	343
336	344		The codetools Absolute position is related to the source as a continuous string starting at character 1. For example a TCodeBuffer holds the file contents as a single string in its Source property. Caret or cursor positions are given as X,Y where X is the column number and Y the line number. Each value (X and Y) starts at 1. A TCodeBuffer provides the member functions LineColToPosition and AbsoluteToLineCol'' to convert between (X,Y) values and the Absolute codetools position. When working with multiple source files (such as a unit which may contain several include files), the clean position relates to the absolute position in the stripped codeSrc. Here Src is a string whose clean positions start at 1. Cursor positions are specified as TCodeXYPosition (Code,X,Y). A TCodeTool provides the functionsCaretToCleanPosandCleanPosToCaret''' to convert between them.
337	345
338	346		# Inserting, deleting, replacing - the TSourceChangeCache
339	347
340	348		When making changes to the source code of a unit (or its include files) you should use the CodetoolBoss.SourceChangeCache instead of altering the source directly.
341	349
342	350		- Simple usage: Connect, Replace, Replace, ... Apply. See below.
343	351		- You can use cleanpos as given by the node tree OR you can use direct position in a file.
344	352		- You can use Replace to insert and delete, which automatically calls events, so connected editors are notified of changes.
345	353		- It can automatically insert needed spaces, line breaks or empty lines in front or behind each Replace. For example you define that there should be an empty line in front. The SourceChangeCache checks what is inserted and how much space there is already and will insert needed space.
346	354		- It checks if the replaced/deleted span is writable.
347	355		- You can do multiple Replaces and you control when they are applied. Keep in mind that inserting code means that the parsed tree becomes invalid and needs rebuilding.
348	356		- Multiple replaces are checked for intersection. For example an insert in the middle of deleted code gives an error.
349	357		- Mutiple insertions at the same place are added FIFO - first at the top.
350	358		- You can combine several functions altering code to one bigger function. See below.
351	359
352	360		## Usage
353	361
354	362		The SourceChangeCache works on a unit, so you need to get a TCodeTool and scan a unit/include file. For example:
355	363
356	364		``` pascal
357	365		// Step 1: load the file and parse it
358	366		Code:=CodeToolBoss.LoadFile(Filename,false,false);
359	367		if Code=nil then
360	368		raise Exception.Create('loading failed '+Filename);
361	369		if not CodeToolBoss.Explore(Code,Tool,false) then
362	370		...;// parse error ...
363	371
364	372		// Step 2: connect the SourceChangeCache
365	373		CodeToolBoss.SourceChangeCache.MainScanner:=Tool.Scanner;
366	374
367	375		// Step 3: use Replace to insert and/or delete code
368	376		// The first two parameters are the needed spaces in front and behind the insertion
369	377		// The FromPos,ToPos defines the deleted/replaced range in CleanPos positions.
370	378		// The NewCode is the string of new code. Use '' for a delete.
371	379		if not CodeToolBoss.SourceChangeCache.Replace(gtNone,gtNone,FromPos,ToPos,NewCode) then
372	380		exit; // e.g. source read only or a former Replace has deleted the place
373	381		...do some more Replace...
374	382
375	383		// Step 4: Apply the changes
376	384		if not CodeToolBoss.SourceChangeCache.Apply then
377	385		exit; // apply was aborted
378	386		```
379	387
380	388		## BeginUpdate/EndUpdate
381	389
382	390		BeginUpdate/EndUpdate delays the Apply. This is useful when combining several code changing functions. For example:
383	391
384	392		You want to scan the unit, add a unit to the interface uses section, and remove the unit from the implementation uses section. The two functions AddUnitToMainUsesSection and RemoveUnitFromUsesSection use Apply, altering the source, so the second function would rescan the unit a second time. But since the two functions are independent of each other (they change different parts of the source) you can combine them and do it with one scan:
385	393
386	394		``` pascal
387	395		// Step 1: parse unit and connect SourceChangeCache
388	396		if not CodeToolBoss.Explore(Code,Tool,false) then
389	397		...;// parse error ...
390	398		CodeToolBoss.SourceChangeCache.MainScanner:=Tool.Scanner;
391	399
392	400		// Step 2: delay Apply
393	401		CodeToolBoss.SourceChangeCache.BeginUpdate;
394	402
395	403		// Step 3: add unit to interface section
396	404		// AddUnitToMainUsesSection would apply and change the code
397	405		// Because of the BeginUpdate the change is not yet done, but stored in the SourceChangeCache
398	406		if not Tool.AddUnitToMainUsesSection('Classes','',CodeToolBoss.SourceChangeCache) then exit;
399	407
400	408		// Step 4: remove unit from implementation section
401	409		// Without the BeginUpdate the RemoveUnitFromUsesSection would rescan the unit
402	410		if Tool.FindImplementationUsesSection<>nil then
403	411		if not Tool.RemoveUnitFromUsesSection(Tool.FindImplementationUsesSection,'Classes',CodeToolBoss.SourceChangeCache) then exit;
404	412
405	413		// Step 5: apply all changes
406	414		if not CodeToolBoss.SourceChangeCache.EndUpdate then
407	415		exit; // apply was aborted
408	416		```
409	417
410	418		BeginUpdate/EndUpdate work with a counter, so if you call BeginUpdate twice you need to call EndUpdate twice. This means you can put the above example in a function and combine that with another function.
411	419
412	420		## Saving changes to disk
413	421
414	422		The above changes are made to the code buffers and the buffers are marked modified. To save the changes to disk, you have to call Save for each modified buffer.
415	423
416	424		- The buffers that will be modified in the next Apply/EndUpdate are in SourceChangeCache.BuffersToModify and BuffersToModifyCount.
417	425		- The events SourceChangeCache.OnBeforeApplyChanges/OnAfterApplyChanges are used by the CodeToolBoss, which connects it to its own OnBeforeApplyChanges/OnAfterApplyChanges. The Lazarus IDE sets these events and automatically opens modified files in the source editor, so all changes go into the undo list of synedit.
418	426
419	427		# Hints/Tips/Guide Lines
420	428
421	429		- BuildTree checks the current state and will only parse if needed. If a former call has parsed the interface and you need the interface again, BuildTree will not do anything. If you need the full unit, only the implementation will be parsed. Nodes are only freed if some files have changed on disk or some settings have changed (initial macros). BuildTree does not check all files on every call. Instead it uses the directory cache of the codetools. So, if nothing has changed it will return very quickly. You should call it before any search operation.
422	430		- Do not call BuildTree after an operation. That is a waste of CPU.
423	431		- BuildTree raises an exception when there are missing include files or syntax errors. You should enclose your code into
424	432
425	433		``` pascal
426	434		try
427	435		Tool.BuildTree(lsrEnd);
428	436		... search ... replace
429	437		except
430	438		on E: Exception do
431	439		CodeToolBoss.HandleException(E);
432	440		end;
433	441		```
434	442
435	443		- After calling SourceChangeCache.Replace (multiple times) the sources (TCodeBuffers) have not changed immediately. You must call SourceChangeCache.Apply to change the sources. This will not save the changes to file.
436	444
437	445		# Test suite
438	446
439	447		The codetools test suite can be found here:
440	448
441		-	~~`lazarus/components/codetools/tests/runtestscodetools`~~
	449	+	```
	450	+	lazarus/components/codetools/tests/runtestscodetools
	451	+	```
442	452
443	453		Compile it and run it:
444	454
445		-	~~`cd lazarus/components/codetools/tests`\~~
446		-	~~`./runtestscodetools`~~
	455	+	```
	456	+	cd lazarus/components/codetools/tests
	457	+	./runtestscodetools
	458	+	```
447	459
448	460		# Links
449	461
450	462		- [[Lazarus IDE Tools]] - A tutorial about the built-in tools of the standard IDE
451	463		- [[Cody]] - IDE package adding advanced code tools to the IDE
452	464		- [[Extending the IDE]] - How to write your own codetools plugins for the IDE.
453	465
454	466		---
455	467		From the Free Pascal wiki, page "Codetools", as it stood on 2020-12-31 after a change by Mattias2, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).
456	468		Category: Components

Back to history