Cocoa Internals/Input: what changed

Revision 3 to 4. Removed lines have a minus sign; added lines have a plus sign.

Old	New		Line
1	1		## LCL Event Handling
2	2
3	3		For general "event loop" handling you might want to refer to [[Cocoa Internals/Application\|application event loop]] article.
4	4
5	5		## Mouse
6	6
7	7		The biggest difference between Apple and LCL ideology is the way of handling MouseMove events.
8	8
9	9		According to Apple - mouse events can easily flood the event queue and thus are disabled by default, unless a user presses and holds the button.
10	10
11	11		Apple provides a special NSWindow mode to allow MouseMove events to come through even with buttons unreleased. However, such MouseMove events would be reported to the focused controls (aka FirstResponder), rather than the control immediately under the cursor.
12	12
13	13		(MouseDown, MouseUp events are reported directly to the control under the cursor, similar to the LCL).
14	14
15	15		Because of that, the Widgetset mouse event processing routine first checks if cursor is actually above the focused control, and if it's not, it would let Cocoa propagate the mouse event, down (from child to parent) through the controls hierarchy.
16	16
17	17		Such propagation happens, when the NSView inherited mouseMove event is called.
18	18
19	19		If the actual control is found, the propagation of the event is stopped.
20	20
21	21		### Implementation
22	22
23	23		The whole set of methods should be overridden for each sub-classed Objective-C control:
24	24
25	25		``` pascal
26	26		function acceptsFirstMouse(event: NSEvent): Boolean; override;
27	27		procedure mouseDown(event: NSEvent); override;
28	28		procedure mouseUp(event: NSEvent); override;
29	29		procedure rightMouseDown(event: NSEvent); override;
30	30		procedure rightMouseUp(event: NSEvent); override;
31	31		procedure rightMouseDragged(event: NSEvent); override;
32	32		procedure otherMouseDown(event: NSEvent); override;
33	33		procedure otherMouseUp(event: NSEvent); override;
34	34		procedure otherMouseDragged(event: NSEvent); override;
35	35		procedure mouseDragged(event: NSEvent); override;
36	36		procedure mouseMoved(event: NSEvent); override;
37	37		procedure scrollWheel(event: NSEvent); override;
38	38		```
39	39
40	40		Even if a target LCL control doesn't publish the events (eg TScrollBar doesn't have any mouse events published), all those methods should still be overriden. The purpose is to give LCL control over that. For example, if a control is used at design time, ALL of the Cocoa default handling should be blocked.
41	41
42	42		The typical implementation for mouseDown/mouseUp events (including "right" and "other" versions) should look like this:
43	43
44	44		``` pascal
45	45		procedure TCocoaControl.mouseDown(event: NSEvent);
46	46		begin
47	47		if not Assigned(callback) or not callback.MouseUpDownEvent(event) then
48	48		inherited mouseDown(event);
49	49		end;
50	50
51	51		procedure TCocoaControl.mouseUp(event: NSEvent);
52	52		begin
53	53		if not Assigned(callback) or not callback.MouseUpDownEvent(event) then
54	54		inherited mouseUp(event);
55	55		end;
56	56		```
57	57
58	58		Such an implementation notifies the LCL first (via callback) about a mouse event. If the LCL doesn't block it (by returning "true" from MouseUpdownEvent) the event is passed further to Cocoa. Actually there's no harm in non-blocking mouseUp and always inheriting it.
59	59
60	60		However, for some controls, when either implementing drag and drop functionality or some sort of dragging action (eg ScrollBar) mouseDown must be implemented in the following way:
61	61
62	62		``` pascal
63	63		procedure TCocoaScrollBar.mouseDown(event: NSEvent);
64	64		begin
65	65		if not Assigned(callback) or not callback.MouseUpDownEvent(event) then
66	66		begin
67	67		inherited mouseDown(event);
68	68
69	69		callback.MouseUpDownEvent(event, true); // forced mouse-up event
70	70		end;
71	71		end;
72	72		```
73	73
74	74		The reason for that is due to the Cocoa (and Apple in general) implementation of mouse-hold button loops. Once inherited mouseDown is called, the procedure would not return until the mouse button is released or the action is cancelled. It's quite typical to do an event processing loop, until mouse-up event received.
75	75
76	76		The issue is that the actual mouseUp() selector would never be called, and must be emulated via a callback.MouseUpDownEvent(, true) call.
77	77
78	78		The implementation of the mouseUp method should still remain.
79	79
80	80		Also, this approach doesn't apply to rightMouseDown and otherMouseDown. Apple doesn't make any event loops for those.
81	81
82	82		### Ctrl+Left Click / Right Click
83	83
84	84		Apple's classic mouse is a single button mouse. The additional actions are performed by holding the mouse button down. (One might observe the similar behavior for smartphone devices, where holding a finger on the screen for a certain period of time opens up an additional action).
85	85
86	86		However, context menus are also part of the user interface. To open a context menu a user would need to hold the "Control" button down and click the mouse button.
87	87
88	88		With the modern mouse device (where having 3 buttons+scroll wheel is the norm) it ended up with `Ctrl+Left` being recognised as a Right click.
89	89
90	90		The Cocoa WidgetSet recognizes the `Ctrl+Left` click as a Context Menu request only and forwards an appropriate message (LM_CONTEXTMENU) to the control which ends in a call to the OnContextPopup event. Yet, Cocoa doesn't alternate the information about left-click in any manner. (The Carbon WidgetSet reports the Button as TRightButton, but the shift state indicates the that Left mouse button is actually pressed).
91	91
92	92		### Capture
93	93
94	94		Only 1 handle (control) can have the mouse captured. This is typically happening on MouseDown and the capture is released on MouseUp.
95	95
96	96		There are corresponding WidgetSet methods available:
97	97
98		-	~~```~~
	98	+	```text
99	99		GetCapture
100	100		SetCapture
101	101		ReleaseCapture
102	102		```
103	103
104	104		There are a few tricks.
105	105
106	106		- most of the Cocoa controls are doing "mouse loop" capture themselves for the MouseDown key. (Mouse events are still propagated to the LCL as if the loop is not happening)
107	107		- if a control is disabled by an EnableWindow() function call, the Capture MUST BE released. This is not documented in WinAPI, yet this is how it's working. (see \#36491)
108	108		- if a control is destroyed with DestoryHandle, the Capture MUST BE released as well.
109	109
110	110		The captured control Handle is stored in the CaptureControl (FCaptureControl) property of TCocoaWidgetSet.
111	111
112	112		The code at TLCLCommonCallback.MouseUpDownEvent checks for the CaptureControl to be assigned. And if it's assigned then it would pass the mouse event to it and would prevent the Cocoa control from processing the event.
113	113
114	114		### Scrolling Wheel
115	115
116	116		Scrolling is different between LCL (WinAPI) approach and macOS (Cocoa or Carbon)
117	117
118	118		- LCL - reports the HARDWARE mouse wheel scroll events (in 120-based deltas). The values ARE NOT a subject to the system configuration of "Number of Lines per scroll" (SPI_GETWHEELSCROLLLINES).
119	119		- Cocoa - reports wheel scrolls as adjustment to lines. (how many lines should be scrolled... and it doesn't matter if there are any kind of lines in the control or what's the size of lines. Just logical lines). The amounts of lines is adjusted by "Scrolling Speed" setting (and thus can be different depending on the setting for the same physical scroll of a line).
120	120
121	121
122	122		scrolling speed can be acquired from either Preferences dialog or via command line:
123	123
124		-	~~```~~
	124	+	```text
125	125		defaults read .GlobalPreferences com.apple.scrollwheel.scaling
126	126		```
127	127
128	128
129	129		in runtime:
130	130
131		-	~~```~~
	131	+	```text
132	132		NSUserDefaults.standardUserDefaults.doubleForKey(NSSTR('com.apple.scrollwheel.scaling'));
133	133		```
134	134
135	135		Cocoa would increase "wheel" delta if the wheel is scrolled fast enough. WinAPI would continue to report 120.
136	136
137	137		Both WinAPI and Cocoa would "accumulate" the delta, if main thread is working slow. Thus the next wheel event might bring a larger delta.
138	138
139	139		If the hardware allows, WinAPI delta would be different than 120. Cocoa might also start report different amounts.
140	140
141	141		## Keyboard
142	142
143	143		The KeyDown event is handled through the NSWindow sendEvent method. There's explicit code written to pass an event to the focused control (firstResponder).
144	144
145	145		### Virtual Key code
146	146
147	147		The LCL uses the concept of Virtual Key codes.
148	148
149	149		macOS APIs don't follow such a concept. Instead Cocoa is using "characters" for this purpose. For example: Menu Short Cuts or special Key Combinations are typically specified by characters, rather than some special codes. For each key pressed, macOS does provide a hardware key code (available through event.keyCode). The key represents a hardware code. Meaning that on a different keyboard layout the keyCode would remain the same, while character might be different. (For example for QWERTY vs Dvorak vs AZERTY layout, the same "q" key pressed, would produce the same keyCode, yet different character).
150	150
151	151		For this particular reason, Virtual Keycodes are determined with the following approach:
152	152
153	153		- the character (event.characterWithoutModifiers) is examined. If it's available and it can be mapped to a known VirtualKey code - then character to VK would be used and non-numpad key (i.e. "/" on the keyboard is different than "/" on number keypad)
154	154		- otherwise the Virtual Key code is determined based on keyCode.
155	155
156	156		It covers the most common cases. A problem remains for national characters (i.e. German [ß](https://en.wikipedia.org/wiki/%C3%9F)). The character itself is not within the range of VK_(a-z) codes. It's encoded as VK_OEM_xx code. However such a VK code would be different on Cocoa and Windows.
157	157
158	158		### Special Keys
159	159
160	160		Some controls wants special keys, such as Tabs and Arrow (Cursor) keys. For example TMemo with WantsTab set to true, or TTrackBar to change thumb position.
161	161
162	162		Both tabs and arrow keys are used for switching focus within the application itself.
163	163
164	164		Thus, it's critical for a control to let the LCL know that it will handle those keys so that focus is not switched.
165	165
166	166		For this purpose an additional method has been introduced:
167	167
168		-	~~```~~
	168	+	```text
169	169		lclExpectedKeys:::
170	170		```
171	171
172	172		This method accepts variable 3 parameters:
173	173
174	174		- wantTabs (default value false), should be set to true, if the control expects to process tab key
175	175		- wantArrows (default value false), should be set to true, if the control expects to process arrow keys (all of them!)
176	176		- wantAllKeys (default value true?)... not sure how it's being used.
177	177
178	178		### Major Patch
179	179
180	180		See <https://bugs.freepascal.org/view.php?id=35449> by Zoë Peterson
181	181
182	182		Reference to <https://asciiwwdc.com/2010/sessions/145> (video:
183	183
184	184
185	185		HD: <https://developer.apple.com/devcenter/download.action?path=/videos/wwdc_2010__hd/session_145__key_event_handling_in_cocoa_applications.mov>)
186	186
187	187		In short, Cocoa's key handling looks like this:
188	188
189	189
190	190		1\. NSApplication.sendEvent (overridable)
191	191
192	192
193	193		2\. Local NSEvent Monitors (local event monitor can change the resulting event. NSEvent. addLocalMonitorForEventsMatchingMask)
194	194
195	195		3\. If modifierFlags include Cmd or Ctrl, then Cocoa tries to perform KeyEquivalent first.
196	196
197	197
198	198		3.1 NSWindow.performKeyEquivalent (Key window)
199	199
200	200
201	201		NSResponder.performKeyEquivalent (called on all views recursively)
202	202
203	203		3.2 NSWindow.performKeyEquivalent (Other "active" windows, but only for views that manage menus)
204	204
205	205		3.3 NSMenu.performKeyEquivalent
206	206
207	207		4\. Check registered hot keys ("app command" keys) (e.g., Cmd~ for Window cycling or Ctrl+F2 to move focus to menu)
208	208
209	209		5\. NSWindow.sendEvent (Key window)
210	210
211	211
212	212		5.1 NSResponder.keyDown (firstResponder)
213	213
214	214
215	215		inherited calls keyDown up the responder chain until it reaches the NSWindow
216	216
217	217		5.2 NSWindow.keyDown
218	218
219	219
220	220		5.2.0 \<undocumented magic\> but for Control+\\
221	221
222	222		5.2.1 NSWindow.performKeyEquivalent (Key Window, possibly for second time if Cmd or Ctrl)
223	223
224	224		5.2.3 NSMenu.performKeyEquivalent (again, possibly the second time)
225	225
226	226		5.2.4 \<undocumented magic\>
227	227
228	228		5.2.5 Beep
229	229
230	230		### Dead Keys
231	231
232	232		By default Cocoa event handling ignores dead-keys (on national keyboards, eg Spanish). For the event to be ready, NSTextInputContext needs to be created and events "translated" through it.
233	233
234	234		The context exists for any NSTextView field(s). Thus native Cocoa TMemo, TEdit or TcomboBox would not be affected. Custom controls however are affected.
235	235
236	236		In order to "translate" a keyboard event TCocoaApplication creates its own (global) NSTextInputContext and passes all events through it. In order to recognize a dead-key combination. Turning the sequence of
237	237
238		-	~~```~~
	238	+	```text
239	239		´ e
240	240		```
241	241
242	242		into
243	243
244		-	~~```~~
	244	+	```text
245	245		é
246	246		```
247	247
248	248		However, if a sequence is invalid, eg:
249	249
250		-	~~```~~
	250	+	```text
251	251		´ s
252	252		```
253	253
254	254		Cocoa reports a single string of character "'s". The Cocoa Widget Set breaks the string into separate characters and reports them as standalone characters, rather than the entire string. (This matches the WinAPI.)
255	255
256	256		## See Also
257	257
258	258		- [[Cocoa Internals]]
259	259
260	260		---
261	261		From the Free Pascal wiki, page "Cocoa Internals/Input", as it stood on 2022-05-07 after a change by Skalogryz, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).
262	262		Category: Cocoa
263	263		Category: Keyboard Input
264	264		Category: Mouse Input
265	265		Category: macOS

Back to history