Cocoa Internals/Input: what changed

Revision 0 to 1. Removed lines have a minus sign; added lines have a plus sign.

Title: Cocoa Internals/Input

New		Line
1	+	## LCL Event Handling
2	+
3	+	For general "event loop" handling you might want to refer to [[Cocoa Internals/Application\|application event loop]] article.
4	+
5	+	## Mouse
6	+
7	+	The biggest difference between Apple and LCL ideology is the way of handling MouseMove events.
8	+
9	+	According to Apple - mouse events can easily flood the event queue and thus are disabled by default, unless a user presses and holds the button.
10	+
11	+	Apple provides a special NSWindow mode to allow MouseMove events to come through even with buttons unreleased. However, such MouseMove events would be reported to the focused controls (aka FirstResponder), rather than the control immediately under the cursor.
12	+
13	+	(MouseDown, MouseUp events are reported directly to the control under the cursor, similar to the LCL).
14	+
15	+	Because of that, the Widgetset mouse event processing routine first checks if cursor is actually above the focused control, and if it's not, it would let Cocoa propagate the mouse event, down (from child to parent) through the controls hierarchy.
16	+
17	+	Such propagation happens, when the NSView inherited mouseMove event is called.
18	+
19	+	If the actual control is found, the propagation of the event is stopped.
20	+
21	+	### Implementation
22	+
23	+	The whole set of methods should be overridden for each sub-classed Objective-C control:
24	+
25	+	``` pascal
26	+	function acceptsFirstMouse(event: NSEvent): Boolean; override;
27	+	procedure mouseDown(event: NSEvent); override;
28	+	procedure mouseUp(event: NSEvent); override;
29	+	procedure rightMouseDown(event: NSEvent); override;
30	+	procedure rightMouseUp(event: NSEvent); override;
31	+	procedure rightMouseDragged(event: NSEvent); override;
32	+	procedure otherMouseDown(event: NSEvent); override;
33	+	procedure otherMouseUp(event: NSEvent); override;
34	+	procedure otherMouseDragged(event: NSEvent); override;
35	+	procedure mouseDragged(event: NSEvent); override;
36	+	procedure mouseMoved(event: NSEvent); override;
37	+	procedure scrollWheel(event: NSEvent); override;
38	+	```
39	+
40	+	Even if a target LCL control doesn't publish the events (eg TScrollBar doesn't have any mouse events published), all those methods should still be overriden. The purpose is to give LCL control over that. For example, if a control is used at design time, ALL of the Cocoa default handling should be blocked.
41	+
42	+	The typical implementation for mouseDown/mouseUp events (including "right" and "other" versions) should look like this:
43	+
44	+	``` pascal
45	+	procedure TCocoaControl.mouseDown(event: NSEvent);
46	+	begin
47	+	if not Assigned(callback) or not callback.MouseUpDownEvent(event) then
48	+	inherited mouseDown(event);
49	+	end;
50	+
51	+	procedure TCocoaControl.mouseUp(event: NSEvent);
52	+	begin
53	+	if not Assigned(callback) or not callback.MouseUpDownEvent(event) then
54	+	inherited mouseUp(event);
55	+	end;
56	+	```
57	+
58	+	Such an implementation notifies the LCL first (via callback) about a mouse event. If the LCL doesn't block it (by returning "true" from MouseUpdownEvent) the event is passed further to Cocoa. Actually there's no harm in non-blocking mouseUp and always inheriting it.
59	+
60	+	However, for some controls, when either implementing drag and drop functionality or some sort of dragging action (eg ScrollBar) mouseDown must be implemented in the following way:
61	+
62	+	``` pascal
63	+	procedure TCocoaScrollBar.mouseDown(event: NSEvent);
64	+	begin
65	+	if not Assigned(callback) or not callback.MouseUpDownEvent(event) then
66	+	begin
67	+	inherited mouseDown(event);
68	+
69	+	callback.MouseUpDownEvent(event, true); // forced mouse-up event
70	+	end;
71	+	end;
72	+	```
73	+
74	+	The reason for that is due to the Cocoa (and Apple in general) implementation of mouse-hold button loops. Once inherited mouseDown is called, the procedure would not return until the mouse button is released or the action is cancelled. It's quite typical to do an event processing loop, until mouse-up event received.
75	+
76	+	The issue is that the actual mouseUp() selector would never be called, and must be emulated via a callback.MouseUpDownEvent(, true) call.
77	+
78	+	The implementation of the mouseUp method should still remain.
79	+
80	+	Also, this approach doesn't apply to rightMouseDown and otherMouseDown. Apple doesn't make any event loops for those.
81	+
82	+	### Ctrl+Left Click / Right Click
83	+
84	+	Apple's classic mouse is a single button mouse. The additional actions are performed by holding the mouse button down. (One might observe the similar behavior for smartphone devices, where holding a finger on the screen for a certain period of time opens up an additional action).
85	+
86	+	However, context menus are also part of the user interface. To open a context menu a user would need to hold the "Control" button down and click the mouse button.
87	+
88	+	With the modern mouse device (where having 3 buttons+scroll wheel is the norm) it ended up with <span class="kbd">Ctrl+Left</span> being recognised as a Right click.
89	+
90	+	The Cocoa WidgetSet recognizes the <span class="kbd">Ctrl+Left</span> click as a Context Menu request only and forwards an appropriate message (LM_CONTEXTMENU) to the control which ends in a call to the OnContextPopup event. Yet, Cocoa doesn't alternate the information about left-click in any manner. (The Carbon WidgetSet reports the Button as TRightButton, but the shift state indicates the that Left mouse button is actually pressed).
91	+
92	+	### Capture
93	+
94	+	Only 1 handle (control) can have the mouse captured. This is typically happening on MouseDown and the capture is released on MouseUp.
95	+
96	+	There are corresponding WidgetSet methods available:
97	+
98	+	`GetCapture `\
99	+	`SetCapture`\
100	+	`ReleaseCapture`
101	+
102	+	There are a few tricks.
103	+
104	+	- most of the Cocoa controls are doing "mouse loop" capture themselves for the MouseDown key. (Mouse events are still propagated to the LCL as if the loop is not happening)
105	+	- if a control is disabled by an EnableWindow() function call, the Capture MUST BE released. This is not documented in WinAPI, yet this is how it's working. (see \#36491)
106	+	- if a control is destroyed with DestoryHandle, the Capture MUST BE released as well.
107	+
108	+	The captured control Handle is stored in the CaptureControl (FCaptureControl) property of TCocoaWidgetSet.
109	+
110	+	The code at TLCLCommonCallback.MouseUpDownEvent checks for the CaptureControl to be assigned. And if it's assigned then it would pass the mouse event to it and would prevent the Cocoa control from processing the event.
111	+
112	+	### Scrolling Wheel
113	+
114	+	Scrolling is different between LCL (WinAPI) approach and macOS (Cocoa or Carbon)
115	+
116	+	- LCL - reports the HARDWARE mouse wheel scroll events (in 120-based deltas). The values ARE NOT a subject to the system configuration of "Number of Lines per scroll" (SPI_GETWHEELSCROLLLINES).
117	+	- Cocoa - reports wheel scrolls as adjustment to lines. (how many lines should be scrolled... and it doesn't matter if there are any kind of lines in the control or what's the size of lines. Just logical lines). The amounts of lines is adjusted by "Scrolling Speed" setting (and thus can be different depending on the setting for the same physical scroll of a line).
118	+
119	+
120	+	scrolling speed can be acquired from either Preferences dialog or via command line:
121	+
122	+	`defaults read .GlobalPreferences com.apple.scrollwheel.scaling`
123	+
124	+
125	+	in runtime:
126	+
127	+	`NSUserDefaults.standardUserDefaults.doubleForKey(NSSTR('com.apple.scrollwheel.scaling'));`
128	+
129	+	Cocoa would increase "wheel" delta if the wheel is scrolled fast enough. WinAPI would continue to report 120.
130	+
131	+	Both WinAPI and Cocoa would "accumulate" the delta, if main thread is working slow. Thus the next wheel event might bring a larger delta.
132	+
133	+	If the hardware allows, WinAPI delta would be different than 120. Cocoa might also start report different amounts.
134	+
135	+	## Keyboard
136	+
137	+	The KeyDown event is handled through the NSWindow sendEvent method. There's explicit code written to pass an event to the focused control (firstResponder).
138	+
139	+	### Virtual Key code
140	+
141	+	The LCL uses the concept of Virtual Key codes.
142	+
143	+	macOS APIs don't follow such a concept. Instead Cocoa is using "characters" for this purpose. For example: Menu Short Cuts or special Key Combinations are typically specified by characters, rather than some special codes. For each key pressed, macOS does provide a hardware key code (available through event.keyCode). The key represents a hardware code. Meaning that on a different keyboard layout the keyCode would remain the same, while character might be different. (For example for QWERTY vs Dvorak vs AZERTY layout, the same "q" key pressed, would produce the same keyCode, yet different character).
144	+
145	+	For this particular reason, Virtual Keycodes are determined with the following approach:
146	+
147	+	- the character (event.characterWithoutModifiers) is examined. If it's available and it can be mapped to a known VirtualKey code - then character to VK would be used and non-numpad key (i.e. "/" on the keyboard is different than "/" on number keypad)
148	+	- otherwise the Virtual Key code is determined based on keyCode.
149	+
150	+	It covers the most common cases. A problem remains for national characters (i.e. German [ß](https://en.wikipedia.org/wiki/%C3%9F)). The character itself is not within the range of VK_(a-z) codes. It's encoded as VK_OEM_xx code. However such a VK code would be different on Cocoa and Windows.
151	+
152	+	### Special Keys
153	+
154	+	Some controls wants special keys, such as Tabs and Arrow (Cursor) keys. For example TMemo with WantsTab set to true, or TTrackBar to change thumb position.
155	+
156	+	Both tabs and arrow keys are used for switching focus within the application itself.
157	+
158	+	Thus, it's critical for a control to let the LCL know that it will handle those keys so that focus is not switched.
159	+
160	+	For this purpose an additional method has been introduced:
161	+
162	+	`lclExpectedKeys:::`
163	+
164	+	This method accepts variable 3 parameters:
165	+
166	+	- wantTabs (default value false), should be set to true, if the control expects to process tab key
167	+	- wantArrows (default value false), should be set to true, if the control expects to process arrow keys (all of them!)
168	+	- wantAllKeys (default value true?)... not sure how it's being used.
169	+
170	+	### Major Patch
171	+
172	+	See <https://bugs.freepascal.org/view.php?id=35449> by Zoë Peterson
173	+
174	+	Reference to <https://asciiwwdc.com/2010/sessions/145> (video:
175	+
176	+
177	+	HD: <https://developer.apple.com/devcenter/download.action?path=/videos/wwdc_2010__hd/session_145__key_event_handling_in_cocoa_applications.mov>)
178	+
179	+	In short, Cocoa's key handling looks like this:
180	+
181	+
182	+	1\. NSApplication.sendEvent (overridable)
183	+
184	+
185	+	2\. Local NSEvent Monitors (local event monitor can change the resulting event. NSEvent. addLocalMonitorForEventsMatchingMask)
186	+
187	+	3\. If modifierFlags include Cmd or Ctrl, then Cocoa tries to perform KeyEquivalent first.
188	+
189	+
190	+	3.1 NSWindow.performKeyEquivalent (Key window)
191	+
192	+
193	+	NSResponder.performKeyEquivalent (called on all views recursively)
194	+
195	+	3.2 NSWindow.performKeyEquivalent (Other "active" windows, but only for views that manage menus)
196	+
197	+	3.3 NSMenu.performKeyEquivalent
198	+
199	+	4\. Check registered hot keys ("app command" keys) (e.g., Cmd~ for Window cycling or Ctrl+F2 to move focus to menu)
200	+
201	+	5\. NSWindow.sendEvent (Key window)
202	+
203	+
204	+	5.1 NSResponder.keyDown (firstResponder)
205	+
206	+
207	+	inherited calls keyDown up the responder chain until it reaches the NSWindow
208	+
209	+	5.2 NSWindow.keyDown
210	+
211	+
212	+	5.2.0 \<undocumented magic\> but for Control+\\
213	+
214	+	5.2.1 NSWindow.performKeyEquivalent (Key Window, possibly for second time if Cmd or Ctrl)
215	+
216	+	5.2.3 NSMenu.performKeyEquivalent (again, possibly the second time)
217	+
218	+	5.2.4 \<undocumented magic\>
219	+
220	+	5.2.5 Beep
221	+
222	+	### Dead Keys
223	+
224	+	By default Cocoa event handling ignores dead-keys (on national keyboards, eg Spanish). For the event to be ready, NSTextInputContext needs to be created and events "translated" through it.
225	+
226	+	The context exists for any NSTextView field(s). Thus native Cocoa TMemo, TEdit or TcomboBox would not be affected. Custom controls however are affected.
227	+
228	+	In order to "translate" a keyboard event TCocoaApplication creates its own (global) NSTextInputContext and passes all events through it. In order to recognize a dead-key combination. Turning the sequence of
229	+
230	+	`´ e`
231	+
232	+	into
233	+
234	+	`é`
235	+
236	+	However, if a sequence is invalid, eg:
237	+
238	+	`´ s`
239	+
240	+	Cocoa reports a single string of character "'s". The Cocoa Widget Set breaks the string into separate characters and reports them as standalone characters, rather than the entire string. (This matches the WinAPI.)
241	+
242	+	## See Also
243	+
244	+	- [[Cocoa Internals]]
245	+
246	+	---
247	+	From the Free Pascal wiki, page "Cocoa Internals/Input", as it stood on 2022-05-07 after a change by Skalogryz, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's [[Licensing]] page. If you have concerns about this material, [message the admin](/wiki-contact).
248	+	Category: Cocoa
249	+	Category: Keyboard Input
250	+	Category: Mouse Input
251	+	Category: macOS

Back to history