WebAssembly/Internals: what changed

Revision 1 to 2. Removed lines have a minus sign; added lines have a plus sign.

Old	New		Line
1	1		## Outdated page
2	2
3	3		This page is outdated. It is currently used as a content staging area during refactoring of the various pages.
4	4
5	5		## Assemblers
6	6
7	7		There are different assemblers available, from Wabt, emscripten.org and LLVM. The expected format is a slightly different between those two:
8	8
9	9		### wat2wasm (Wabt)
10	10
11	11		Example:
12	12
13	13		``` lisp
14	14		(module
15	15		(func $add (param $lhs i32) (param $rhs i32) (result i32)
16	16		local.get $lhs
17	17		local.get $rhs
18	18		i32.add
19	19		)
20	20		(export "add" (func $add))
21	21		)
22	22		```
23	23
24	24		According to the official site "Wabt" is using it's own format of the Wasm. It's slightly different from the official documentation. The most current version of Wabt matches the specs, as well as supports the old syntax.
25	25
26	26		Online studio, that's using the older version of wabt syntax. <https://webassembly.studio/>
27	27
28	28		For example. instead of
29	29
30		-	~~`local.get`~~
	30	+	```text
	31	+	local.get
	32	+	```
31	33
32	34		it's using
33	35
34		-	~~`get_local`~~
	36	+	```text
	37	+	get_local
	38	+	```
35	39
36	40		### wasm-as (emscripten)
37	41
38	42		The assembler is recommended for the use in a compiler by the WebAssembly.org
39	43
40	44		``` lisp
41	45		(module
42	46		(func $add (param $lhs i32) (param $rhs i32) (result i32)
43	47		(
44	48		local.get $lhs
45	49		local.get $rhs
46	50		i32.add
47	51		)
48	52		)
49	53		(export "add" (func $add))
50	54		)
51	55		```
52	56
53	57		### llvm-mc
54	58
55	59		This is the assembler from the LLVM project. It uses a GAS-like syntax.
56	60
57	61		## Use on the Wiki
58	62
59	63		WebAssembly is using s-expressions as its textual format (for either Wabt or Emscript) . it's handy to use syntax highlighter for the code and use "lisp" language to set colors.
60	64
61	65		<source lang="lisp">
62	66		;; web assembly goes here
63	67		</source>
64	68
65	69		## Limitations
66	70
67	71		- WebAssembly doesn't have a stand-alone executable format. It's always a "plugin" to some environment. Thus the compiler currently only supports producing a library and program should not be used. Using library requires exporting a certain functions from the library. Those functions will be the "export" part of Wasm module as well.
68	72		- WebAssembly format doesn't provide any special information for static linking (for the purpose of combining multiple .wasm files into one). No support is provided at this time, thus the entire Pascal code should be put in a single library file.
69	73		- Importing functions into Pascal is supported partially; see External.
70	74		- A memory object must always be provided externally, using the module env with the name memory. This is hard-coded in the Wasm assembler. (Todo: needs to be configurable.)
71	75
72	76		## Validation
73	77
74	78		There's a certain requirements to the binary code of WebAssembly.
75	79
76	80		The validation is described at the specification <http://webassembly.github.io/spec/core/valid/index.html>
77	81
78	82		A code that satisfies the validation rules, is considered to be well-formed
79	83
80	84		It's possible to generate a binary file with the code that doesn't satisfy the validation steps. However, the browsers would not instantiate such binary file. Neither any other host should.
81	85
82	86		In some cases, there might be an overhead to create well-formed code.
83	87
84	88		## Global Symbols
85	89
86	90		Global symbols are stored in global variables. Each variable is of i32 type and stores an offset in memory, where the object is usually stored.
87	91
88	92		In order to get the address of the variable, one should invoke get_global for the desired symbols. The memory offset if predefined and cannot be changed during the runtime. The contents of the memory can be changed.
89	93
90	94		## Code Branching/Flow Control
91	95
92	96		todo: rewrite to the better explanation
93	97
94	98		WebAssembly doesn't allow direct jumps to (address/label)/or jump by offset. It only allows to jump "out-of" a code block, where the code block would be identified by a label. I.e. beginning of the loop label (for continue) or end of the loop label (for break). Loop itself is a single code block, known as "loop" in WebAssembly ).
95	99
96	100		FPC basic implementation is based of the ability of conditional jump instructions to a specified label. For that reason, the default implementation of TCGIfNode doesn't apply. Also, not every conditional jump instructions can have a label assign to it. Only instructions that jump out of the block are should use labels. Other, simply rely on the fact, that the next instruction IS the point they needs to be.
97	101
98	102		### IF-block
99	103
100	104		Wasm IF do return a value (one of the basic WebAsm types). Pascal IF-statement does not return a value.
101	105
102	106		Thus the feature of value returning is not used (todo: but should be).
103	107
104	108		A dummy 0 value is pushed on the stack. IF is hard-coded to return i32 at all times.
105	109
106	110		According to the official documentation, IF always comes with "ELSE" block. However, some earlier versions do allow IF without an else statement. (which is also allowed in Binary specification of WASM). Such IF comes without any resulting type, and the execution of IF should leave stack unmodified in the end.
107	111
108	112		At this time "ELSE" branch is always generated for any IF statement. However, if there's no actual ELSE code (in pascal), a single "nop" instruction and a dummy result value produced.
109	113
110	114		``` pascal
111	115		function cmp(a,b: integer): Integer;
112	116		begin
113	117		if a > b then
114	118		cmp := 1
115	119		else
116	120		cmp := 0;
117	121		end;
118	122		```
119	123
120	124		turns into:
121	125
122	126		``` lisp
123	127		(func $cmp
124	128		(param i32)
125	129		(param i32)
126	130		(result i32)
127	131		(local i32) ;; Temp 2,1 allocated
128	132		(local i32) ;; Temp 3,1 allocated
129	133		;; [3] begin
130	134		;; [4] if a > b then
131	135		get_local 0
132	136		set_local 3
133	137		get_local 3
134	138		get_local 1
135	139		i32.gt_s
136	140		if (result i32) ;; hard-coded i32 type of IF result
137	141		;; [5] cmp := 1
138	142		i32.const 1
139	143		set_local 2
140	144		i32.const 0 ;; mandatory wasm-IF result
141	145		else
142	146		;; [7] cmp := 0;
143	147		i32.const 0
144	148		set_local 2
145	149		i32.const 0 ;; mandatory wasm-IF result
146	150		end
147	151		drop ;; dropping IF-result from the stack, as unused
148	152		;; [8] end;
149	153		get_local 2
150	154		return
151	155		)
152	156		```
153	157
154	158		The code generation of cifnode is overridden.
155	159
156	160		### Loop-block
157	161
158	162		A loop is a block of instructions that has the label set to the beginning of the code.
159	163
160	164		Here is an example of a loop. The function tries to sum up a number X amount of times
161	165
162	166		``` pascal
163	167		function mulbyadd(Num, Cnt: integer): integer;
164	168		Result := 0;
165	169		while Cnt>0 do begin
166	170		dec(Cnt);
167	171		Result := Result + Num;
168	172		end;
169	173		end;
170	174		```
171	175
172	176		\<source lang="lisp\>
173	177
174	178		` (func $mulbyadd (param $Num i32) (param $Cnt i32) (result i32)`\
175	179		` (local i32) ;; local variable (that gets index #2) this is $Result`\
176	180		` i32.const 0`\
177	181		` set_local 2`
178	182
179	183		` block`\
180	184		` loop `\
181	185		` get_local $Cnt ;; comparing Cnt to 0`\
182	186		` i32.const 0 ;; `\
183	187		` i32.le_s `\
184	188		` br_if 1 ;; if Cnt is less or equal to 0, then we should exit the loop`
185	189
186	190		` get_local $Cnt ;; decreasing Cnt`\
187	191		` i32.const 1 `\
188	192		` i32.sub `\
189	193		` set_local $Cnt `
190	194
191	195		` get_local $Num ;;`\
192	196		` get_local 2 ;; `\
193	197		` i32.add ;;`\
194	198		` set_local 2 ;; Result := Result + Num`
195	199
196	200		` br 0 ;; loop back to the beginning `\
197	201		` end `\
198	202		` end`\
199	203		` get_local 2 ;; pushing the result to the stack (for the return value)`\
200	204		` )`
201	205
202	206		### FPC Labels
203	207
204	208		Since branching is relative in Wasm (the absolute goto-like is only a proposed feature for Wasm), the use of TAsmLabel is different than other targets. The field labelnr is changed to indicate the relative jump. Typically 0 - to return to the loop or 1 - to jump out of the loop - or N to exit the function.
205	209
206	210		## External
207	211
208	212		One can declare an external function. For a WebAssembly module the function must be provided during a module instantiation. Any external function must have a library name specified (Static linking is not supported!). The library name and the "exteral" name will be used during instantiation phase.
209	213
210	214		Only basic types can be used as parameters (integer, floats, memory pointers) and function result.
211	215
212	216		Here's an example:
213	217
214	218		The code tries to use an external function that accepts only 1 integer parameter. (The function should write out the parameter)
215	219
216	220		``` pascal
217	221		library testext;
218	222
219	223		procedure logint(i: integer); external 'env' name 'logint'; // name of external module and function name
220	224
221	225		procedure run;
222	226		begin
223	227		logint(304);
224	228		end;
225	229
226	230		exports
227	231		run name 'run';
228	232		end.
229	233		```
230	234
231	235		In the browser, during instantiation the function must be provided:
232	236
233	237		``` js
234	238		const importObject = {
235	239		// note "env" should match the name of the function's library used in the pascal
236	240		// thus it's possible to import functions from different sources.
237	241		"env": {
238	242
239	243		"memory": new WebAssembly.Memory({initial:32, maximum:32})
240	244		// this is the implementation of "logint" function
241	245		// it accepts exactly 1 parameter, and puts it into a the browser's console
242	246		,"logint": function(k) { console.log(k) }
243	247		}
244	248
245	249		}
246	250		fetch('../out/main.wasm').then(response =>
247	251		response.arrayBuffer()
248	252		).then(bytes => WebAssembly.instantiate(bytes, importObject)).then(results => {
249	253		instance = results.instance;
250	254
251	255		document.getElementById("container").textContent =
252	256		// running the export run() function
253	257		instance.exports.run();
254	258		}).catch(console.error);
255	259		```
256	260
257	261		### PChar Strings
258	262
259	263		Strings would require a more work from the client side. There's no "string" type in Wasm, thus strings are treated as chunks in memory. From Javascript perspective that means, that a reference to a memory is received from a Wasm code and should be converted to JS string (that typically involves a text decoder)
260	264
261	265		``` pascal
262	266		library testext;
263	267
264	268		type
265	269		pchar = ^char;
266	270
267	271		procedure logstr(p: pchar); external 'env' name 'logstr';
268	272		procedure writedom(p: pchar); external 'dom' name 'writedom';
269	273
270	274		procedure run;
271	275		begin
272	276		logstr('hello');
273	277		logstr('world');
274	278		writedom('<b>Hello World</b> and <input type="button" value="FPC">');
275	279		end;
276	280
277	281		exports
278	282		run name 'run';
279	283
280	284		end.
281	285		```
282	286
283	287		On the host side the function needs to read the memory and prepare the string for the output:
284	288
285	289		``` javascript
286	290		const memory = new WebAssembly.Memory({initial:32, maximum:32});
287	291
288	292		// the function takes, strofs that indicates the start of string
289	293		// and then tries to get the string out of the memory, by searching for null-terminating character
290	294		function getStr(strofs)
291	295		{
292	296		var arr = new Uint8Array(memory.buffer,strofs);
293	297		for (var i=0; i < arr.length; i++)
294	298		if (arr[i]==0) // null terminating char is found
295	299		arr = arr.subarray(0, i);
296	300		var str = new TextDecoder("utf-8").decode(arr);
297	301		return str;
298	302		}
299	303
300	304		const importObject = {
301	305		"env": {
302	306		"memory": memory
303	307		,"logstr": function(k)
304	308		{ console.log(getStr(k)); }
305	309		},
306	310		"dom": {
307	311		"writedom": function(k) { document.write(getStr(k)) }
308	312		}
309	313		}
310	314		fetch('../out/main.wasm').then(response =>
311	315		response.arrayBuffer()
312	316		).then(bytes => WebAssembly.instantiate(bytes, importObject)).then(results => {
313	317		instance = results.instance;
314	318		document.getElementById("container").textContent = instance.exports.run();
315	319		}).catch(console.error);
316	320		```
317	321
318	322		Obviously pascal Ansi or WideStrings can work in the similar manner, and can be more efficient (as the size of the string is known ahead of time). However, strings can only be passed as constant strings, rather than a string variable for change (as implementing a reference count between Wasm and JS can be a bit more complicated)
319	323
320	324		## Object File Contents
321	325
322	326		There's wasm-objdump that's useful for inspection of a generated object file.
323	327
324	328		## Static Linking
325	329
326	330		See LLD linker: <https://lld.llvm.org/WebAssembly.html> <https://github.com/WebAssembly/tool-conventions/blob/master/Linking.md>
327	331
328	332		So far, there's only 1 wasm linker exists: lld (aka wasm-ld). LLVM linked ported for WASM.
329	333
330	334		There's no official specification on object linking in WebAssembly. Tools are typically a common convention with a special "linking" section.
331	335
332	336		One should keep in mind, that "linking" section is binary format specific. There are no way to explicitly define it via textual format. (which imposes some risk, when the implementation of a particular assembler changes)
333	337
334	338		### Linking section
335	339
336	340		For each linkable part of the code (i.e. function) an entry must be created in "linking" section.
337	341
338	342		### Linking Symbols
339	343
340	344		In order for wasm-ld to successfully connect two symbols, the following approach must be taken:
341	345
342	346		- a symbol declared in implementation section
343	347
344		-	~~`WASM_SYM_VISIBILITY_HIDDEN `\~~
345		-	~~`WASM_SYM_BINDING_LOCAL `~~
	348	+	```text
	349	+	WASM_SYM_VISIBILITY_HIDDEN
	350	+	WASM_SYM_BINDING_LOCAL
	351	+	```
346	352
347	353		This indicates that the symbol should not be linked anywhere, except for within the object file itself.
348	354
349	355		- the unit declaring a symbol in interface section (linkable by other units) should declare the unit with flags set to:
350	356
351		-	~~`WASM_SYM_VISIBILITY_HIDDEN `~~
	357	+	```text
	358	+	WASM_SYM_VISIBILITY_HIDDEN
	359	+	```
352	360
353	361		This indicates that while the symbol can be linked to other object files, but should not be part of the linked library (executable)
354	362
355	363		- a same symbol that's being used by a different unit, should have a stub function (with unreachable in its body) and have the following flags set:
356	364
357		-	~~`WASM_SYM_BINDING_WEAK`\~~
358		-	~~`WASM_SYM_VISIBILITY_HIDDEN`~~
	365	+	```text
	366	+	WASM_SYM_BINDING_WEAK
	367	+	WASM_SYM_VISIBILITY_HIDDEN
	368	+	```
359	369
360	370		This indicates to wasm-ld linker to get rid of the unreachable stub and use the actual implementation.
361	371
362	372		- a symbols that's explicitly declared in exports section of a library should not have a specific flags. However, the following flag can be also be used:
363	373
364		-	~~`WASM_SYM_EXPORTED`~~
	374	+	```text
	375	+	WASM_SYM_EXPORTED
	376	+	```
365	377
366	378		- an external symbol (that's imported from the host) should be marked by
367	379
368		-	~~`WASM_SYM_UNDEFINED `~~
	380	+	```text
	381	+	WASM_SYM_UNDEFINED
	382	+	```
369	383
370	384		The flag is automatically set by wat2wasm for every import function.
371	385
372	386		### Tools
373	387
374	388		wat2wasm - specifics:
375	389
376	390		- a function must be either called in the unit OR it should be declared as a part of a (function pointer)table (via element) in order to be added to "linking" section;
377	391		- in order to mark a function as undefined it should be either "exported" or "imported" function. Note "importing" a function requires declaring a unit name, which is divided by a "dot" (.) character from the function import name.
378	392		- functions declared as "export" functions, would NOT be put into linking section and will be removed by the wasm-ld linker
379	393
380	394		wasm-ld (LLD 9.0.0) - specifics:
381	395
382	396		- when linking two or more modules for the function with the same name, linker keeps the code that's marked as defined (a special flag in linking section). The code that's marked as weak or undefined would be discarded.
383	397		- wasm exported functions are ignored by default, if they don't have a symbol assigned in linking section.
384	398		- the linker would use a symbol name for an exported function, instead of declared export name
385	399
386	400		wasmtool <https://github.com/skalogryz/wasmbin>
387	401
388	402		- the utility allows to change export names
389	403
390	404		## See Also
391	405
392	406		- [[WebAssembly]]
393	407		- [[WebAssembly/Instructions]] - Instructions of the WASM code
394	408		- <https://webassembly.org/> - the official site
395	409		- <https://webassembly.github.io/spec/core/text/index.html> - text format (S-Expression) specs
396	410		- <https://webassembly.github.io/spec/core/binary/index.html> - binary format specs
397	411		- <https://developer.mozilla.org/en-US/docs/WebAssembly/Understanding_the_text_format>
398	412		- <https://rsms.me/wasm-intro> - introduction to webassembly
399	413		- <https://blog.scottlogic.com/2018/04/26/webassembly-by-hand.html>
400	414
401	415		---
402	416		From the Free Pascal wiki, page "WebAssembly/Internals", as it stood on 2022-04-05 after a change by PierceNg, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).
403	417		Category: WebAssembly

Back to history