Cocoa Internals/Input: what changed

Revision 1 to 2. Removed lines have a minus sign; added lines have a plus sign.

Old	New		Line
1	1		## LCL Event Handling
2	2
3	3		For general "event loop" handling you might want to refer to [[Cocoa Internals/Application\|application event loop]] article.
4	4
5	5		## Mouse
6	6
7	7		The biggest difference between Apple and LCL ideology is the way of handling MouseMove events.
8	8
9	9		According to Apple - mouse events can easily flood the event queue and thus are disabled by default, unless a user presses and holds the button.
10	10
11	11		Apple provides a special NSWindow mode to allow MouseMove events to come through even with buttons unreleased. However, such MouseMove events would be reported to the focused controls (aka FirstResponder), rather than the control immediately under the cursor.
12	12
13	13		(MouseDown, MouseUp events are reported directly to the control under the cursor, similar to the LCL).
14	14
15	15		Because of that, the Widgetset mouse event processing routine first checks if cursor is actually above the focused control, and if it's not, it would let Cocoa propagate the mouse event, down (from child to parent) through the controls hierarchy.
16	16
17	17		Such propagation happens, when the NSView inherited mouseMove event is called.
18	18
19	19		If the actual control is found, the propagation of the event is stopped.
20	20
21	21		### Implementation
22	22
23	23		The whole set of methods should be overridden for each sub-classed Objective-C control:
24	24
25	25		``` pascal
26	26		function acceptsFirstMouse(event: NSEvent): Boolean; override;
27	27		procedure mouseDown(event: NSEvent); override;
28	28		procedure mouseUp(event: NSEvent); override;
29	29		procedure rightMouseDown(event: NSEvent); override;
30	30		procedure rightMouseUp(event: NSEvent); override;
31	31		procedure rightMouseDragged(event: NSEvent); override;
32	32		procedure otherMouseDown(event: NSEvent); override;
33	33		procedure otherMouseUp(event: NSEvent); override;
34	34		procedure otherMouseDragged(event: NSEvent); override;
35	35		procedure mouseDragged(event: NSEvent); override;
36	36		procedure mouseMoved(event: NSEvent); override;
37	37		procedure scrollWheel(event: NSEvent); override;
38	38		```
39	39
40	40		Even if a target LCL control doesn't publish the events (eg TScrollBar doesn't have any mouse events published), all those methods should still be overriden. The purpose is to give LCL control over that. For example, if a control is used at design time, ALL of the Cocoa default handling should be blocked.
41	41
42	42		The typical implementation for mouseDown/mouseUp events (including "right" and "other" versions) should look like this:
43	43
44	44		``` pascal
45	45		procedure TCocoaControl.mouseDown(event: NSEvent);
46	46		begin
47	47		if not Assigned(callback) or not callback.MouseUpDownEvent(event) then
48	48		inherited mouseDown(event);
49	49		end;
50	50
51	51		procedure TCocoaControl.mouseUp(event: NSEvent);
52	52		begin
53	53		if not Assigned(callback) or not callback.MouseUpDownEvent(event) then
54	54		inherited mouseUp(event);
55	55		end;
56	56		```
57	57
58	58		Such an implementation notifies the LCL first (via callback) about a mouse event. If the LCL doesn't block it (by returning "true" from MouseUpdownEvent) the event is passed further to Cocoa. Actually there's no harm in non-blocking mouseUp and always inheriting it.
59	59
60	60		However, for some controls, when either implementing drag and drop functionality or some sort of dragging action (eg ScrollBar) mouseDown must be implemented in the following way:
61	61
62	62		``` pascal
63	63		procedure TCocoaScrollBar.mouseDown(event: NSEvent);
64	64		begin
65	65		if not Assigned(callback) or not callback.MouseUpDownEvent(event) then
66	66		begin
67	67		inherited mouseDown(event);
68	68
69	69		callback.MouseUpDownEvent(event, true); // forced mouse-up event
70	70		end;
71	71		end;
72	72		```
73	73
74	74		The reason for that is due to the Cocoa (and Apple in general) implementation of mouse-hold button loops. Once inherited mouseDown is called, the procedure would not return until the mouse button is released or the action is cancelled. It's quite typical to do an event processing loop, until mouse-up event received.
75	75
76	76		The issue is that the actual mouseUp() selector would never be called, and must be emulated via a callback.MouseUpDownEvent(, true) call.
77	77
78	78		The implementation of the mouseUp method should still remain.
79	79
80	80		Also, this approach doesn't apply to rightMouseDown and otherMouseDown. Apple doesn't make any event loops for those.
81	81
82	82		### Ctrl+Left Click / Right Click
83	83
84	84		Apple's classic mouse is a single button mouse. The additional actions are performed by holding the mouse button down. (One might observe the similar behavior for smartphone devices, where holding a finger on the screen for a certain period of time opens up an additional action).
85	85
86	86		However, context menus are also part of the user interface. To open a context menu a user would need to hold the "Control" button down and click the mouse button.
87	87
88		-	~~With the modern mouse device (where having 3 buttons+scroll wheel is the norm) it ended up with <span class="kbd">Ctrl+Left</span> being recognised as a Right click.~~
	88	+	With the modern mouse device (where having 3 buttons+scroll wheel is the norm) it ended up with `Ctrl+Left` being recognised as a Right click.
89	89
90		-	The Cocoa WidgetSet recognizes the <span class="kbd">Ctrl+Left</span> click as a Context Menu request only and forwards an appropriate message (LM_CONTEXTMENU) to the control which ends in a call to the OnContextPopup event. Yet, Cocoa doesn't alternate the information about left-click in any manner. (The Carbon WidgetSet reports the Button as TRightButton, but the shift state indicates the that Left mouse button is actually pressed).
	90	+	The Cocoa WidgetSet recognizes the `Ctrl+Left` click as a Context Menu request only and forwards an appropriate message (LM_CONTEXTMENU) to the control which ends in a call to the OnContextPopup event. Yet, Cocoa doesn't alternate the information about left-click in any manner. (The Carbon WidgetSet reports the Button as TRightButton, but the shift state indicates the that Left mouse button is actually pressed).
91	91
92	92		### Capture
93	93
94	94		Only 1 handle (control) can have the mouse captured. This is typically happening on MouseDown and the capture is released on MouseUp.
95	95
96	96		There are corresponding WidgetSet methods available:
97	97
98	98		`GetCapture `\
99	99		`SetCapture`\
100	100		`ReleaseCapture`
101	101
102	102		There are a few tricks.
103	103
104	104		- most of the Cocoa controls are doing "mouse loop" capture themselves for the MouseDown key. (Mouse events are still propagated to the LCL as if the loop is not happening)
105	105		- if a control is disabled by an EnableWindow() function call, the Capture MUST BE released. This is not documented in WinAPI, yet this is how it's working. (see \#36491)
106	106		- if a control is destroyed with DestoryHandle, the Capture MUST BE released as well.
107	107
108	108		The captured control Handle is stored in the CaptureControl (FCaptureControl) property of TCocoaWidgetSet.
109	109
110	110		The code at TLCLCommonCallback.MouseUpDownEvent checks for the CaptureControl to be assigned. And if it's assigned then it would pass the mouse event to it and would prevent the Cocoa control from processing the event.
111	111
112	112		### Scrolling Wheel
113	113
114	114		Scrolling is different between LCL (WinAPI) approach and macOS (Cocoa or Carbon)
115	115
116	116		- LCL - reports the HARDWARE mouse wheel scroll events (in 120-based deltas). The values ARE NOT a subject to the system configuration of "Number of Lines per scroll" (SPI_GETWHEELSCROLLLINES).
117	117		- Cocoa - reports wheel scrolls as adjustment to lines. (how many lines should be scrolled... and it doesn't matter if there are any kind of lines in the control or what's the size of lines. Just logical lines). The amounts of lines is adjusted by "Scrolling Speed" setting (and thus can be different depending on the setting for the same physical scroll of a line).
118	118
119	119
120	120		scrolling speed can be acquired from either Preferences dialog or via command line:
121	121
122	122		`defaults read .GlobalPreferences com.apple.scrollwheel.scaling`
123	123
124	124
125	125		in runtime:
126	126
127	127		`NSUserDefaults.standardUserDefaults.doubleForKey(NSSTR('com.apple.scrollwheel.scaling'));`
128	128
129	129		Cocoa would increase "wheel" delta if the wheel is scrolled fast enough. WinAPI would continue to report 120.
130	130
131	131		Both WinAPI and Cocoa would "accumulate" the delta, if main thread is working slow. Thus the next wheel event might bring a larger delta.
132	132
133	133		If the hardware allows, WinAPI delta would be different than 120. Cocoa might also start report different amounts.
134	134
135	135		## Keyboard
136	136
137	137		The KeyDown event is handled through the NSWindow sendEvent method. There's explicit code written to pass an event to the focused control (firstResponder).
138	138
139	139		### Virtual Key code
140	140
141	141		The LCL uses the concept of Virtual Key codes.
142	142
143	143		macOS APIs don't follow such a concept. Instead Cocoa is using "characters" for this purpose. For example: Menu Short Cuts or special Key Combinations are typically specified by characters, rather than some special codes. For each key pressed, macOS does provide a hardware key code (available through event.keyCode). The key represents a hardware code. Meaning that on a different keyboard layout the keyCode would remain the same, while character might be different. (For example for QWERTY vs Dvorak vs AZERTY layout, the same "q" key pressed, would produce the same keyCode, yet different character).
144	144
145	145		For this particular reason, Virtual Keycodes are determined with the following approach:
146	146
147	147		- the character (event.characterWithoutModifiers) is examined. If it's available and it can be mapped to a known VirtualKey code - then character to VK would be used and non-numpad key (i.e. "/" on the keyboard is different than "/" on number keypad)
148	148		- otherwise the Virtual Key code is determined based on keyCode.
149	149
150	150		It covers the most common cases. A problem remains for national characters (i.e. German [ß](https://en.wikipedia.org/wiki/%C3%9F)). The character itself is not within the range of VK_(a-z) codes. It's encoded as VK_OEM_xx code. However such a VK code would be different on Cocoa and Windows.
151	151
152	152		### Special Keys
153	153
154	154		Some controls wants special keys, such as Tabs and Arrow (Cursor) keys. For example TMemo with WantsTab set to true, or TTrackBar to change thumb position.
155	155
156	156		Both tabs and arrow keys are used for switching focus within the application itself.
157	157
158	158		Thus, it's critical for a control to let the LCL know that it will handle those keys so that focus is not switched.
159	159
160	160		For this purpose an additional method has been introduced:
161	161
162	162		`lclExpectedKeys:::`
163	163
164	164		This method accepts variable 3 parameters:
165	165
166	166		- wantTabs (default value false), should be set to true, if the control expects to process tab key
167	167		- wantArrows (default value false), should be set to true, if the control expects to process arrow keys (all of them!)
168	168		- wantAllKeys (default value true?)... not sure how it's being used.
169	169
170	170		### Major Patch
171	171
172	172		See <https://bugs.freepascal.org/view.php?id=35449> by Zoë Peterson
173	173
174	174		Reference to <https://asciiwwdc.com/2010/sessions/145> (video:
175	175
176	176
177	177		HD: <https://developer.apple.com/devcenter/download.action?path=/videos/wwdc_2010__hd/session_145__key_event_handling_in_cocoa_applications.mov>)
178	178
179	179		In short, Cocoa's key handling looks like this:
180	180
181	181
182	182		1\. NSApplication.sendEvent (overridable)
183	183
184	184
185	185		2\. Local NSEvent Monitors (local event monitor can change the resulting event. NSEvent. addLocalMonitorForEventsMatchingMask)
186	186
187	187		3\. If modifierFlags include Cmd or Ctrl, then Cocoa tries to perform KeyEquivalent first.
188	188
189	189
190	190		3.1 NSWindow.performKeyEquivalent (Key window)
191	191
192	192
193	193		NSResponder.performKeyEquivalent (called on all views recursively)
194	194
195	195		3.2 NSWindow.performKeyEquivalent (Other "active" windows, but only for views that manage menus)
196	196
197	197		3.3 NSMenu.performKeyEquivalent
198	198
199	199		4\. Check registered hot keys ("app command" keys) (e.g., Cmd~ for Window cycling or Ctrl+F2 to move focus to menu)
200	200
201	201		5\. NSWindow.sendEvent (Key window)
202	202
203	203
204	204		5.1 NSResponder.keyDown (firstResponder)
205	205
206	206
207	207		inherited calls keyDown up the responder chain until it reaches the NSWindow
208	208
209	209		5.2 NSWindow.keyDown
210	210
211	211
212	212		5.2.0 \<undocumented magic\> but for Control+\\
213	213
214	214		5.2.1 NSWindow.performKeyEquivalent (Key Window, possibly for second time if Cmd or Ctrl)
215	215
216	216		5.2.3 NSMenu.performKeyEquivalent (again, possibly the second time)
217	217
218	218		5.2.4 \<undocumented magic\>
219	219
220	220		5.2.5 Beep
221	221
222	222		### Dead Keys
223	223
224	224		By default Cocoa event handling ignores dead-keys (on national keyboards, eg Spanish). For the event to be ready, NSTextInputContext needs to be created and events "translated" through it.
225	225
226	226		The context exists for any NSTextView field(s). Thus native Cocoa TMemo, TEdit or TcomboBox would not be affected. Custom controls however are affected.
227	227
228	228		In order to "translate" a keyboard event TCocoaApplication creates its own (global) NSTextInputContext and passes all events through it. In order to recognize a dead-key combination. Turning the sequence of
229	229
230	230		`´ e`
231	231
232	232		into
233	233
234	234		`é`
235	235
236	236		However, if a sequence is invalid, eg:
237	237
238	238		`´ s`
239	239
240	240		Cocoa reports a single string of character "'s". The Cocoa Widget Set breaks the string into separate characters and reports them as standalone characters, rather than the entire string. (This matches the WinAPI.)
241	241
242	242		## See Also
243	243
244	244		- [[Cocoa Internals]]
245	245
246	246		---
247	247		From the Free Pascal wiki, page "Cocoa Internals/Input", as it stood on 2022-05-07 after a change by Skalogryz, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).
248	248		Category: Cocoa
249	249		Category: Keyboard Input
250	250		Category: Mouse Input
251	251		Category: macOS

Back to history