[Cuis-dev] [RFC] True Mouse Wheel Support
Gerald Klix
cuis.01 at klix.ch
Fri Aug 13 00:41:34 PDT 2021
Hi Phil,
On 8/13/21 5:34 AM, Phil B via Cuis-dev wrote:
> Gerald,
>
> It looks fine to me testing on 64-bit Debian 11. I confirmed that when you
> toggle the VM setting on and off the image only receives events from the
> appropriate source (i.e. keyboard or mousewheel.) It is odd that the event
> rate of the uncooked events does appear slower.
That surprised me, too. After some thinking -- could not sleep --
it dawned to me that the problem is elsewhere ...
> I would expect it to be
> the other way around (i.e. that the cooked keyboard event rate would be
> slower.) I'm assuming that this is a VM thing that probably needs to be
> fixed there (i.e. higher resolution scroll events are quite handy for
> scrolling long lists, zooming etc)
... Actually the code that generates scroll events from keyboard
events (and vice versa) -- please, don't take this personally -- was
flawed from the very beginning. It generates three/two scroll
events from three/two keyboard events.
If you look at the vm-code, you discover that the VM generates three
or two keyboard events from one mouse-wheel event:
X11:
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/c6c18805f2a5ba03b5ea3e765bb7fec1a2fce248/platforms/unix/vm-display-X11/sqUnixX11.c#L3641
OSX Quartz:
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/c6c18805f2a5ba03b5ea3e765bb7fec1a2fce248/platforms/unix/vm-display-Quartz/sqUnixQuartz.m#L1911
These ones only generates two keyboard events.
Linux/Unix framebuffer:
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/c6c18805f2a5ba03b5ea3e765bb7fec1a2fce248/platforms/unix/vm-display-fbdev/sqUnixEvdevKeyMouse.c#L662
Windows:
https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/c6c18805f2a5ba03b5ea3e765bb7fec1a2fce248/platforms/win32/vm/sqWin32Window.c#L429
(Side note: I might be wrong concerning the event count on windows)
In a nutshell the VM is not really consistent here,
but 2 vs. 3 events did probably go unnoticed.
You can used the attached file-out of a slightly
instrumented HandMorph>>#generateKeyboardEvent:
method to see the three keyboard
events and the resulting three scroll events.
As a fix, I suggest to react only on the simulated
key-down-event and ignore the other events.
We than can adjust the scroll counts in the individual
morph classes. I also suggest to make these scroll counts
preferences. This should lead to consistent behavior across
OSes and scroll-wheel event reporting protocols.
Forthermore advanced users can adjust the scroll rate.
Best Regards,
Gerald
> Nice to finally see those wheel
> side-to-side direction events... nice work!
Thanks!
>
> The only question I have is if there's a way to detect the capability and
> set the VM parameter automatically on image start? If so, we could have a
> preference as to whether or not it should automatically be set. (I imagine
> most would)
Actually we can activate it unconditionally, because
the OSX VM will disregard the VM parameter and sent simulated keyboard
events anyway. It Quartz window code does not refer to
this VM parameter.
>
> Thanks,
> Phil
>
> On Thu, Aug 12, 2021 at 4:20 PM Gerald Klix <cuis.01 at klix.ch> wrote:
>
>> Hi all, Hi Phil,
>>
>> Please find attached a preliminary version of my mouse-wheel support
>> code. Please review the coding style and check for any errors.
>> I only tested it on Linux 64bit, but will also
>> test on Windows tomorrow.
>>
>> The code needs `Smalltalk sendMouseWheelEvents: true` to be active.
>> Vertical scrolling should be smoother/slower,
>> horizontal scrolling should work.
>>
>>
>> Have Fun and Best Regards,
>>
>> Gerald
>>
>>
>>
>> On 8/11/21 11:07 PM, Gerald Klix via Cuis-dev wrote:
>>> Hoi Phil,
>>>
>>> On 8/11/21 10:31 PM, Phil B via Cuis-dev wrote:
>>>> Gerald,
>>>>
>>>> I implemented the wheel event support for Cuis anticipating that at some
>>>> point in the future we'd be able to get 'uncooked' events from the VM.
>>> Cool!
>>>> So
>>>> if you see a way to get them, by all means please do so.
>>> You won't believe it, its dead simple, just copy this SystemDirectory
>>> method from Squeak
>>>
>>> --- snip ---
>>> sendMouseWheelEvents: aBoolean
>>> "The Cog VM can be instructed to deliver mouse wheel events as
>>> mouse wheel events.
>>> By default mouse wheel events are mapped to arrow events.
>>> This flag persists across snapshots, stored in the image header."
>>>
>>> self vmParameterAt: 48 put: ((self vmParameterAt: 48) bitClear: 32)
>>> + (aBoolean ifTrue: [32] ifFalse: [0])
>>> --- snap ---
>>>
>>> and send the apropriate message.
>>>
>>> I verified this behavior by dumping the events with `Sensor test` on a
>>> Linux 64-bit VM. The VM sends events of type 7.
>>>
>>>> It's been a few
>>>> years, but the last time I looked the Unix VM didn't have it yet. IIRC,
>>>> only one platform (Windows?) did and *all* VM platforms need to
>>>> support it
>>>> before you can flip the switch on the functionality.
>>>
>>> I discovered this feature accidentally, while looking at the X11 code;
>>> I wanted all buttons of my trackball supported, some more modifier keys.
>>>
>>> I now see the problem (vm/platforms):
>>> >>> find -name '*.c'| xargs grep sendWheelEvents
>>>
>>> yields:
>>>
>>> ./win32/vm/sqWin32Window.c:extern sqInt sendWheelEvents; /* If true
>>> deliver EventTypeMouseWheel else kybd */
>>> ./win32/vm/sqWin32Window.c:/* if sendWheelEvents is false this maps
>>> wheel events to arrow keys */
>>> ./win32/vm/sqWin32Window.c: if (inputSemaphoreIndex &&
>>> sendWheelEvents) {
>>> ./win32/vm/sqWin32Window.c: if (sendWheelEvents) {
>>> ./unix/vm-display-X11/sqUnixX11.c:extern sqInt sendWheelEvents; /* If
>>> true deliver EventTypeMouseWheel else kybd */
>>> ./unix/vm-display-X11/sqUnixX11.c:/* if sendWheelEvents is false this
>>> maps wheel events to arrow keys */
>>> ./unix/vm-display-X11/sqUnixX11.c:/* if sendWheelEvents is true this
>>> determines how much x & y are incremented */
>>> ./unix/vm-display-X11/sqUnixX11.c: if (sendWheelEvents)
>>> ./unix/vm-display-fbdev/sqUnixEvdevKeyMouse.c:extern sqInt
>>> sendWheelEvents; /* If true deliver EventTypeMouseWheel else kybd */
>>> ./unix/vm-display-fbdev/sqUnixEvdevKeyMouse.c: if (sendWheelEvents)
>> {
>>>
>>>
>>> In this case the old code needs to remain for the unsupported OSes.
>>>
>>> It's now 11pm here, I will look into it tomorrow.
>>>
>>>
>>> Best Regards,
>>>
>>> Gerald
>>>
>>>
>>>>
>>>> Thanks,
>>>> Phil
>>>>
>>>> On Wed, Aug 11, 2021, 3:02 PM Gerald Klix via Cuis-dev <
>>>> cuis-dev at lists.cuis.st> wrote:
>>>>
>>>>> I just noticed that all the events and processing methods are there.
>>>>> I searched for '*Wheel*' instead of '*Scroll*'.
>>>>>
>>>>>
>>>>>
>>>>> On 8/11/21 8:46 PM, Gerald Klix via Cuis-dev wrote:
>>>>>> Hi all, Hi Juan,
>>>>>>
>>>>>> Is there interest in implementing or porting *true*
>>>>>> mouse wheel support for/to Cuis?
>>>>>> I mean switching off the up/down-key emulation
>>>>>> in the VM and add the necessary event classes
>>>>>> and processing methods.
>>>>>>
>>>>>> Benefits:
>>>>>> Mouses with a tilting wheel -- wheel left/right -- will be supported.
>>>>>>
>>>>>> I would like to see all of the keys my Elecom trackballs
>>>>>> provide supported, alas this will require patching the VM.
>>>>>> Maybe I will change the VM's X11 code.
>>>>>>
>>>>>>
>>>>>> Best Regards,
>>>>>>
>>>>>> Gerald
>>>>>>
>>>>>>
>>>>>> Trackball images:
>>>>>> https://i.ebayimg.com/images/i/272219997851-0-1/s-l1000.jpg
>>>>>> https://i.ebayimg.com/images/g/wWAAAOSwB5Fe5vK-/s-l640.jpg
>>>>> --
>>>>> Cuis-dev mailing list
>>>>> Cuis-dev at lists.cuis.st
>>>>> https://lists.cuis.st/mailman/listinfo/cuis-dev
>>>>>
>>>>
>>>>
>>
>
>
-------------- next part --------------
'From Haver 5.0 [latest update: #4743] on 13 August 2021 at 9:29:22 am'!
!HandMorph methodsFor: 'private events' stamp: 'KLG 8/13/2021 08:58:43'!
generateKeyboardEvent: evtBuf
"Generate the appropriate mouse event for the given raw event buffer"
| buttons modifiers type keyValue pressType stamp mouseScrollDirection |
stamp _ evtBuf second.
stamp = 0 ifTrue: [ stamp _ Time millisecondClockValue ]. "VMs report events using #millisecondClockValue"
(evtBuf sixth <= 0 or: [ (keyValue _ Character iso8859s15CodeForUnicodeCodePoint: evtBuf sixth) isNil ]) ifTrue: [ keyValue _ Character macRomanToLatin1: evtBuf third ].
Sensor peekEvent ifNotNil: [ :nxt |
"start: Combining diacritical marks (i.e. accents in the Linux VM)"
(nxt fourth = EventSensor eventKeyDown and: [ nxt third > 255 ]) ifTrue: [
keyValue _ ((Character numericValue: keyValue) withDiacriticalMark: nxt third) iso8859s15Code.
Sensor
nextEvent;
nextEvent;
nextEvent ].
"end: Combining diacritical marks (i.e. accents in the Linux VM)"
"start: Spurious LF after CR on Ctrl-Enter on Windows VM"
((evtBuf fourth = EventSensor eventKeyChar and: [ evtBuf third = 13 ]) and: [
nxt fourth = EventSensor eventKeyChar and: [ nxt third = 10 ]]) ifTrue: [ Sensor nextEvent
"print " ]].
modifiers _ evtBuf fifth.
pressType _ evtBuf fourth.
pressType = EventSensor eventKeyDown ifTrue: [
type _ #keyDown.
lastKeyDownValue _ keyValue ].
pressType = EventSensor eventKeyUp ifTrue: [
(keyValue = 9 and: [(modifiers anyMask: 1) and: [Smalltalk platformName = 'unix']])
ifTrue: [
"Linux VMs don't generate shift-tab keystroke. Turn #keyUp into #keystroke"
pressType _ EventSensor eventKeyChar ]
ifFalse: [type _ #keyUp ]].
pressType = EventSensor eventKeyChar ifTrue: [
type _ #keystroke.
"If Control key pressed, and the VM answers a code below 27,
it means it did the translation, convert it back to regular character:
We want to handle the meaning of ctrl ourselves."
(modifiers anyMask: 2) ifTrue: [
"Control key pressed"
keyValue < 27 ifTrue: [
"But we don't want to do it for Home/End/PgUp/PgDn, just for alphabetic keys"
lastKeyDownValue = keyValue ifFalse: [
"If equal, real Home/End/PgUp/PgDn in Windows => don't translate"
(keyValue + 64 = lastKeyDownValue or: [ "If Equal, Ctrl-alphabetic in Windows => do translate"
lastKeyDownValue < 47 ]) ifTrue: [
"Not on windows. If less (not sure about the bound, but do not translate 48: tab on Mac), alphabetic on Mac => do translate"
keyValue _ (modifiers anyMask: 1)
ifTrue: [ keyValue + 64 ]
ifFalse: [ keyValue + 96
"shift not pressed: conver to lowercase letter" ]]]].
"On Windows, ctrl-backSpace is reported as ctrl-forwardDelete. But keyDown is ok, so we can know and fix."
(keyValue = 127 and: [ lastKeyDownValue = 8 ])
ifTrue: [ keyValue _ 8 ].
"Act as if command/alt was pressed for some usual Windows ctrl-key combinations"
(self shouldControlEmulateAltFor: keyValue) ifTrue: [ modifiers _ modifiers bitOr: 8 ]]].
buttons _ modifiers bitShift: 3.
"Linux and Windows VM send keyboard ctrl-upArrow and ctrl-downArrow when the user tries to scroll using the mouse wheel
Mac VM sends cmd-option-ctrl-shift-upArrow and cmd-option-ctrl-shift-downArrow for trackpad vertical scroll gestures,
and cmd-option-ctrl-shift-leftArrow and cmd-option-ctrl-shift-rightArrow for horizontal scroll gestures.
This way of reporting scroll events by the VM also enables scrolling using the keyboard (actually, we can't tell if user gesture was on Mouse, Trackpad or Keyboard).
But ctrl-shift and cmdAlt-shift are needed used for selecting while moving by word, line, etc.
Additionally, #ctrlArrowsScrollHorizontally allows chosing between keyboard horizontal scroll and moving word by word in text editors."
mouseScrollDirection _ nil.
"Ctrl for Keyboard or Mouse wheel gestures. All modifiers for Trackpad gestures."
(buttons = InputSensor controlKey or: [buttons = InputSensor cmdAltOptionCtrlShiftModifierKeys]) ifTrue: [
keyValue = 30
ifTrue: [mouseScrollDirection _ #up]
ifFalse: [keyValue = 31
ifTrue: [mouseScrollDirection _ #down]]].
"Ctrl for Keyboard or Mouse wheel gestures, only if preference is set. All modifiers for Trackpad gestures."
((buttons = InputSensor controlKey and: [Preferences ctrlArrowsScrollHorizontally]) or: [buttons = InputSensor cmdAltOptionCtrlShiftModifierKeys]) ifTrue: [
keyValue = 28
ifTrue: [mouseScrollDirection _ #left]
ifFalse: [keyValue = 29
ifTrue: [mouseScrollDirection _ #right]]].
mouseScrollDirection ifNotNil: [
Transcript
newLine;
show: 'Scroll event from keyboard event, direction: ', mouseScrollDirection.
^ MouseScrollEvent new
setType: #mouseScroll
position: self morphPosition
direction: mouseScrollDirection
buttons: buttons
hand: self
stamp: stamp ].
^ KeyboardEvent new
setType: type
buttons: buttons
position: self morphPosition
keyValue: keyValue
hand: self
stamp: stamp! !
More information about the Cuis-dev
mailing list