Tk Source Code

Hi,
this may be fixed by further updates in macOS 14...
I've some ttk::scrollbar in my code and when I use the mouse wheel to slowly scroll up (or down), the scrollbar is going back and forth.

it happens with widget demo too.

best regards,
nicolas

Closing this ticket now that TIOP #684 has been merged.

My apologies for whatever caused you to feel that way.  I learned a huge
amount from your comments and tried to use that information appropriately.

Pardon me for saying this. I am no longer interested in this ticket or how it is dealt with.

Re-added <Control-Mousewheel> to ttk/scrollbar.tcl.

Hi Csaba.  Thanks! I think this is a good idea. I think it will make
for a big improvement on the X11 platform, which will continue to report
touchpad events with MouseWheel events after this project is finished.


I haven't tried on linux yet, but I did try replacing those files with
your attachments in the touchpad_events branch on my macBook Air.  I
did not see any change, which is good. The touchpad gestures, which
generate Control-MouseWheel events, were fine. When I used a bluetooth
mouse on the laptop everything worked as expected.  I expect I will
see an improvement on linux.

Marc, I have an idea, which might be worth examining:  Independently of the changes you have made in the C code, modify the mouse wheel event handling in scrlbar.tcl and ttk/scrollbar.tcl in such a way that when both <MouseWheel> and <Shift-MouseWheel> events are received (which is quite common when using the touchpad), count them and ignore the non-dominant ones.  This seems to eliminate the artifacts caused by receiving both x and y deltas.  I have tested it on X11 and Windows, using the trunk version of the C code and the patched versions of the above-mentioned library files.  AFAICT, the results look promising.  Maybe this is yet another way to cope with the undesired effects caused by TIP 563 when using a touchpad.

I am attaching the patched versions of the two library files -- they can be used both with trunk and with the bugfix branch touchpad_events.  It would be nice if you could test them and let me know your opinion.

I am taking the view that TIP 563 was about mouse wheels, which are
inherently 1-dimensional input devices, and not about touchpads, which
are inherently 2-dimensional.  (Or more than 2, since they provide velocity
and momentum).

The TIP only discusses wheels, not touchpads.

Marc, what I wrote about the two-finger gestures over a scrollbar is not my personal opinion.  I just wanted to clarify the expected behavior based on TIP 563.

Dear Csaba, thanks for all this information. I love scrollutil and would have it entierly in the core, of cause.

Dear Marc, thank you for all the great action.

About the pad-scrolling on a horizontal scrollbar: As a user, I really like the horizontal movement and that vertical is not working. It feels very natural. I just did not know that it works like that.

I hope you find an understanding for a usable beautiful solution.

You guys rock ! Take care, Harald

I do not agree with this:

> When the mouse pointer is over the scrollbar then both vertical and
> horizontal two-finger gestures are expected to scroll the scrollbar.

In fact, that is more or less the core problem here.  It leads directly
to the issue that Nicolas was reporting.
 
That behavior is fine if the widget has both horizontal and vertical
scrollbars, but it is extremely confusing if the widget has only one
scrolling orientation available.

When both orientations are available, the user can see the effect and
adjust accordingly.  But if the widget can only scroll, say, vertically
then scrolling with the touchpad produces exactly the artifacts that 
this ticket is about. Minor horizontal deviations produce vertical scrolls
that can cause the widget to scroll up when the user is expecting it
to scroll down.  The widget can appear to go back and forth when the
user is intending for it to move in a consistent direction.

It is very bad UX for vertical movement on the touchpad to produce
horizontal movement of the widget contents.  Similarly, horizontal
trackpad movement should never produce vertical movement of the contents.
In the case of a scrollwheel it does not bother me too much.  The wheel
is not strongly associated with a direction.  Touchpads are different.

Well designed user interfaces never convert horizontal touchpad
gestures into vertical motion. You can test this yourself. Just to pick
one example out of a great many, look at Firefox.  It never produces 
the back and forth behavior reported in this ticket. If a page has both
vertical and horizontal scrollbars then moving two fingers horizontally
while the mouse is in the vertical scrollbar does scroll the page
horizontally. But if a page has only a vertical scrollbar then horizontal
motion while the mouse is in the scrollbar is ignored, just as it is
when the mouse is in the content view.

@Marc,
I wouldn't say that Entry needs MouseWheel binding... I was refering to the scrollbar attached.
bad english...

++

@Harald: When the mouse pointer is over the scrollbar then both vertical and horizontal two-finger gestures are expected to scroll the scrollbar.  In order to minimize the number of artifacts resulting from receiving both vertical and horizontal deltas, one has to avoid bigger deviations from the vertical/horizontal direction.

@Harald:

- The spinboxes in the Widget Demo are Tk core spinbox widgets.  There have never been mouse wheel event bindings for the Spinbox class (only for TSpinbox).

- For the class TScale there are currently no mouse wheel event bindings, neither with Tk nor with Scrollutil.

- For TNotebook the "scrolling" with the touchpad should have the same effect as rolling the mouse wheel (select the next/previous enabled tab).

Marc, thank you so much. I have retired. It was not the version, it was me ;-) You have to make a left/right scrolling on the scroll-pad. I tried up-down. So:

• Entry demo (with scrollbars): mouse over scrollbar: ok

You may communicate here with Csaba. I think, he already did all this in Scrollutil...

Thank you and take care, Harald

Thank you Harald.  I am adding bindings one widget at a time, and I am
not finished.  So I know that some bindings are missing.  But it is
helpful to have a list.

I don't think you had quite the latest version.  The "entry with scrollbars"
demo now works with the touchpad. By that I mean that when the mouse is in
the scrollbar you can scroll the entry with the touchpad or with the wheel
(horizontal scroll either by rolling the wheel, or by tilting it).  As far
as I know, the entry widget never supported scrolling directly with either
wheel or touchpad. You need to put the mouse in the scrollbar.  (Full
disclosure: I haven't tested this on Windows yet.  But it works on macOS and
the only changes were to Tcl code, so it *should* also work on Windows.)

Hi ! I am very interested in this subject. Thanks for caring. My Windows Laptop has a pad. I have build the current branch [touchpad_events] with MS-VS 2015 32 bit.

Two-Finger scrolling mostly works like the Scrollwheel.

Here are the difference using the widget demo:

• Entry demo (with scrollbars): mouse over scrollbar: only the mousewheel scrolls the scrollbar
• Spin Boxes: do not scroll on mouse-wheel or pad
• Combo boxes: ok
• Basic editable text: ok (unfortunately no horizontal scrollbar, so shift+scroll can not be tested)
• The canvas item types: vertical scrollbar: ok, horizontal scrollbar: ok for mouse-wheel, only right-scroll with pad, canvas: no reaction
• Horizontal/Vertical scale: no
• Notebook widget: mouse over tabs: mouse wheel: yes, not with pad

AFAIR, the magic "scrollutil" by Csaba has fixed all those.

Great work and thank you, Harald

It turns out that currently there are no MouseWheel bindings for Entry
widgets - only for scrollbars associated with Entry widgets.  Adding
new bindings for them would be out of the scope of this project.

Hi Nicolas, thanks for reminding me about the Entry widget.  I will add
bindings for it next.  That will be easier than the canvas.

Hi Marc,
here with your latest commit:
-Text is working fine with both touchPad and Mouse
-Listbox is working fine with both touchPad and Mouse

in the widget tour with 2. Entries with scrollbars 
with Mouse, shift+wheel is scrolling horizontally (fine) but not with touchPad.

++

@nab - I think everything is working now, except for canvas scrolling.  I
haven't started that yet, and I am sure it is a mess.  That will be next
on my list.

@Csaba @nab - thanks for the testing.

@nab - The mousewheel scrolling is broken at the moment.  At this point it
is work in progress.  I need to investigate what is happening with the
wheel.

I have tested again the official, trunk version of the mouse wheel event handling using the touchpad of my notebook, whose SSD has partitions for Windows 11, Ubuntu Linux (running GNOME), Linux Mint (running Cinnamon), KDE neon (running KDE Plasma), Linux Lite (running Xfce), and Ubuntu MATE (running, of course, the MATE desktop).  On all of the above-mentioned Linux systems, I couldn't see any problems with the scrolling, except for the ones related to the already discussed TIP 563 regarding the scrolling while the pointer is over a scrollbar (although the artifacts appear by far not as heavy to me).

From the above I conclude that on X11 there is probably no need for any special handling of the mouse wheel events emitted by the touchpad.

@Marc,
with latest commits in the bug fix branch, using the Text example in the widget tour:
-touchPad is working fine
-with the Mouse it only scroll in one direction

++

@François: The implemenation of TIP #570 looks very interesting, and would
allow for a lot of fancy user interactions.  But I am just looking to get
basic touchpad scrolling to work with the usual two-finger gesture.  So I
am doing something much simpler.  It will not interfere with the TIP,
as far as I can tell.

@nab: In the end I decided to use Control-MouseWheel events for high
resolution scrolling, instead of a new event type.  I am not actually
sure why that will make any difference to you, but I am doing what you
suggested because it is much simpler.

Thanks, François. I will look at that.

A major issue is that there is no other way to scroll on a standard laptop
besides the two-finger scroll gesture.  So this is not really a wish-list item.
In the current situation, where scrolling is seriously broken on laptops, it
is a must-have item.

Every standard scrollable widget needs to have bindings for that gesture.
And those which only support scrolling by units, not pixels, will need major
work.

Just a note to mention TIP #570 (Gesture Support for Finger Scrolling and "Pinch to Zoom") with a partial implementation in branch pspjuth-touch. This work originated from the RFE ticket [9c2f8da611].

Marc, your latest comment arrived while I was editing mine.  Your proposal sounds quite reasonable to me.

Sorry Marc, the point is not whether I like your fix or not.  But IMHO the changes you made in the library scripts seem to completely ignore the uniformization of the mouse wheel event handling, which is not just the result of pretty much work, but in addition is the implementation of a TIP (TIP 474).  Any changes in the code have to respect the requirements stated in that TIP.  The problems related to the scrolling with a touchpad are not caused by the uniform nature of the binding scripts.

@Marc,
can it be done using modifiers? instead of new event type that will needed to be included in existing code?

++

@Csaba, here is a proposal.

I will repurpose the bugfix branch, and rename it as touchpad_works.
It will be an implementation for a new TIP.

I will then create a new event type, TouchpadEvent, to be used by
high resolution scrolling devices.  I now know how to recognize
events sent by such devices on Apple and Windows, and I believe
it is possible to do so on linux.  Who knows, maybe someone will
help me with that.

The old MouseWheel bindings can be reverted and new bindings added
for TouchpadEvents.

This will mean that it takes quite bit longer before touchpad
scrolling works, but it will avoid undoing other people's hard work.

What do you think?

> It was intentional that for the scrollbar widget <MouseWheel> and
> <Shift-MouseWheel> should have the same effect.

After reading TIP 563, I gleaned that the point of that change was to add
a quirky convenience feature to Tk which allows activating a horizontal
scrollbar by positioning the mouse over the scrollbar and spinning the
scrollwheel on the mouse as if you were doing a vertical scroll. This
avoids having to use the "tilt wheel" feature and would even work on an
old mouse with no tilt.

Unfortunately, however, that was a really bad idea because it completely
overlooked the fact that touchpads produce both x and y scroll deltas for
every two-finger gesture.  When scrolling a vertical scrollbar in this way
it is not possible to avoid creating some x deltas along with the y deltas.
And with this unfortunate design those x deltas get added to the y deltas
and can make the scrollbar move up and down when you are doing your best to
move it in only one direction.

Currently, in trunk, scrolling with a touchpad, as every laptop user does,
is very badly broken.

In the bugfix branch scrolling with a touchpad at least works.  If you
don't like my fix for the current broken implementation then I would welcome
your help in making it better, or you could provide your own fix.  The
current situation is terrible.  It is not true that it

> has proved OK for quite a long time

In reality it only worked on desktop computers with a mouse.  It did not
work with a touchpad.  People dealt with this by avoiding using scroll
gestures on a scrollbar and by just tolerating the jerky scrolling and
the weird back and forth movements during a normal scroll gesture.  They
assumed that Tk is primitive and so one must tolerate these things.

It was intentional that for the scrollbar widget <MouseWheel> and <Shift-MouseWheel> should have the same effect.

To be frank, I am not at all glad to see the many incompatible changes in the library files listbox.tcl and scrlbar.tcl.  In trunk we have a really uniform handling of the <MouseWheel>, <Option-MouseWheel>, <Shift-MouseWheel>, and <Shift-Option-MouseWheel> events.  The factor argument of tk::MouseWheel is everywhere either -40.0 or -12.0.  This is the result of an intensive collaboration between Jan and myself, and has proved OK for quite a long time.  Library packages like Tablelist and Scrollutil also expect that these values won't change.  You are now using different values, like -30.0 and -10.0 in listbox.tcl and -30.0 and -3.0 in scrlbar.tcl.  In addition:

1. In trunk the mouse wheel event handling for a scrollbar scrolls the connected widget (listbox or ttk::treeview) by the same number of items as the mouse wheel event handling for that connected widget.  IMHO, this is and should remain the expected behavior.

2. You have changed the bindings for Listbox.  The tag TtkScrollable has exactly the same mouse wheel event bindings, and the ttk::treeview widget uses these bindings.  Any change for Listbox should be made in exactly the same manner for TtkScrollable, too.

3. In listbox.tcl and scrlbar.tcl the tk::ScaleNum invocations make no sense.  I had to insert them into text.tcl, because since Tk 8.5 the mouse wheel event bindings scroll a text widget by a certain number of pixels rather than rows, and this amount should be kept in sync with the display's scaling level.

On my comment of 2023-11-18 18:34:16:  Since then I updated my Mac Mini to Ventura 13.6.1.  I am not quite sure, but I think that it had some impact on the mouse wheel event handling. (?)

Hi Marc,
testing the bugfix branch about this ticket and your new code.

let's take 7. A simple scrollable canvas  from widget demon as example, your new code is scrolling perfectly when I move up and a little bit left two fingers on the trackapd, and perfectly fine when moving up and a little bit left.

now in my app I've canvases's text widgets that should only respond to up or down scrolling.
moving up (two fingers on the trackpad) gives positive values, moving down gives negatives values, that's ok. but if I move a little bit left or right when moving up or down I receives both negative and positive values, and it leads to a mess.

Can't we have a <MouseWheel-Up> <MouseWheel-Down> <MouseWheel-Right> <MouseWheel-Left> binding? and a general <MouseWheel> event that is not filtered?

best regards,
Nicolas

Just to clarify: It is only for the Scrollbar widget that I am unable to
bind to <Shift-MouseWheel>.  It works fine for everything else.

Could this be an unintended side effect of TIP 563?  Or maybe it was
intentional?

I have pushed changes which fix the scrollbar bindings.  Here is a synopsis:

* A MouseWheel event repurposes the ButtonPress XEvent, but with the type field
set to MouseWheelEvent.  There are vertical and horizontal MouseWheel events.
The difference is indicated by the ShiftMask bit in the state field.  If the
ShiftMask bit is 1 then the scroll is horizontal.

* There is no way to bind to a <Shift-MouseWheel> event, as far as I can tell.
My workaround for this is for the binding script to actually check the
ShiftMask bit using the state value provided in the %s substitution. 

* Because of the above, bindings for the Scrollbar are only for <MouseWheel>,
meaning that vertical and horizontal scrollbars receive both vertical and
horizontal scroll events.  This was the cause of the weird and unacceptable
behavior when scrolling with a touchpad while the mouse cursor is in a
scrollbar.  Both vertical and horizontal events are generated by a touchpad,
and they were being superimposed, causing the scrollbar to reverse direction
even though the scroll gesture is always moving in the same direction.  A
vertical scrollbar would treat a horizontal MouseWheel event as if it were
a vertical MouseWheel event.

This is definitely related to TIP 563.  I can't explain why it was not dealt
with there.  Perhaps because Tk developers tend not to use touchpads.  But I
can't explain why that would be the case either.

A preliminary investigation indicates that somehow vertical scrollbars receive horizontal touchpad scroll events and horizontal scrollbars receive vertical touchpad scroll events. But I do not yet understand the mechanism underlying this.

That sounds like TIP 563

A preliminary investigation indicates that somehow vertical scrollbars receive horizontal touchpad scroll events and horizontal scrollbars receive vertical touchpad scroll events. But I do not yet understand the mechanism underlying this.

That sounds like TIP 536

Nicolas noticed one more serious problem with this, which I am working on.

While the MouseWheel bindings are working for the Text widget, they are
still not working correctly for the Scrollbar widget (and consequently
for the ttk::Scrollbar widget since they share bindings).  If you use the
two-finger touchpad gesture while the mouse pointer is in a scrollbar there
are similar artifcacts to what Nicolas first reported for the Text widget.

The scrollbar is actually more complicated, since there are two species
of scrollbar.  A preliminary investigation indicates that somehow vertical
scrollbars receive horizontal touchpad scroll events and horizontal
scrollbars receive vertical touchpad scroll events.  But I do not yet
understand the mechanism underlying this.

I have tested the current state of the branch [bug-de3bbcb68] on Aqua and Windows, both with a Logitech mouse only.  The bug reported by Nicolas seems to be fixed.  OTOH, on Aqua an <Option-MouseWheel> event scrolls a listbox by one item only, just like <MouseWheel>.  With the trunk version, <Option-MouseWheel> scrolls by 10 items and <MouseWheel> scrolls by 3 items.  I understand that the bugfix-version makes the scrolling smoother, but IMHO there should still be a difference between the effect of <Option-MouseWheel> and that of <MouseWheel>.  For example, 3 or 4 items vs one (in Tk 8.6 and earlier, it is even 10 items vs one).

A general remark: Since now we are in the Tcl/Tk 9.0b1 release time, it would be very desirable to close this ticket ASAP, after improving the above-mentioned behavior.

About the number of mousewheel clicks per revolution: it is not standardized.
I have an old Microsoft mouse with 18 clicks per revolution and a newer
Logitech mouse with 24 clicks per revolution.

About the conversion factor between WM_MOUSEWHEEL units and pixels:  I
compared the trackpad scrolling speed of Edge to that of Tk, with Tk using
a conversion factor of 1.  The speeds were very close to the same.  That is,
doing the two-finger gesture at about the same speed over the height of
the touchpad scrolled the two programs about the same distance.  So I
think 1 is a reasonable factor.

I see that, when the text widget in the widget demo is scrolled,
the links often fail to become "hot", i.e. turn red, when they pass
under the cursor.  But I think that is a red herring, so to speak.
On my macBook Air the same thing happens, in the same way, with the
current tip of the main branch and with the bug-fix branch.  Both
branches use the YScrollByPixels function.  My guess is that
YScrollByPixels is not generating Enter and Leave events as it
should. That would be a bug, but I think it is unrelated to the
topic of this ticket.

Allowing a choice of unit for MouseWheel events, between "pixel" and
"unit" where a unit is a line, or 30 pixels, or something, might be a
good idea.  I think that would require a TIP.  And I don't know at this
point how the Tk language would need to change to accommodate that.
Perhaps there should be two event types, MouseWheelEvent and TouchpadEvent.

Fixing the broken macOS scrolling by implementing Apple's recommendations
instead of trying to fit Apple's round peg into Microsoft's square hole
is just a bug fix, and should not require a TIP.  The fix is not adding
any feature to the Tk language and would not affect any existing
Tk code.

The Windows situation is less clear, given that Windows may or may not be
suggesting that partial clicks be accumulated.  (Apple is definitely not
suggesting any such thing.)  There is a lot of conflicting language, but
I do have the clear impression that Windows is not *requiring* accumulation,
but just allowing it as an option. My opinion is that a TIP is not
required here, since there are no language changes. but I could imagine
some possible disagreement about that.

If people think I should revert the changes to the Windows port, I would
do that. But the current Windows scrolling behavior definitely seems
broken to me. It is surely not intended for the windows content to move
up and down while the touchpad gesture is always moving in the same
direction.

My suggestion to use non-accumulated precise scrolling is mainly meant to reduce issues in/simplify the platform mouse event code. It may result in smoother than the current behavior, but I do not intend for to meet others’ definition of “smooth scrolling” such as the state-of-the-art discussed in Fatin’s post. Yet even this change is arguably significant enough for a TIP, at least if it intends to survive the next wave of tinkering, and I would not be surprised if bringing it up for discussion results in pushback from those who prefer the current behavior or at least want a way to opt out.

I have only briefly tried Windows touchpad scrolling on an Asus device with quad-core Atom CPU. I find that scrolling the widget demo text widget is not as smooth when it causes the pointer to rapidly highlight and unhighlight widget demo links. So without having tested other Tk programs, I do not know if Tk’s inherently single-threaded layout/drawing/etc. is not too slow for this idea to work well.

Regarding “multiples or divisions of 120”: I interpret that as referring only to integer arithmetic, because Win32 APIs were largely based on Windows APIs for 16-bit Intel x86 CPUs with no FPU, in contrast to Quartz which often uses CGFloat.

Deltas on Windows are defined as corresponding to wheel rotation angle: one notch, a delta of 120, is 15 degrees (which also seems like an abstract amount and not how much a particular hardware mouse wheel physically rotated). One notch is supposed to scroll by either one screen or a certain number of lines as configured in Settings > Devices > Mouse (or Mouse Properties on older Windows); the default is 3 lines. I am not aware that Tk has ever respected this setting, nor how well other 3rd party programs respect it. Maybe it is up to programs to ultimately decide how much to scroll for a given angle delta, and maybe it is convenient to interpret delta as a number of pixels to scroll, but I have not found anything recommending that interpretation. Rather than assigning “number of pixels” meaning to angle deltas, Windows introduced the “Precision touchpad” API for pixel scrolling. As discussed in Fatin’s post, though, if a driver supports precision touchpad but a program does not use the necessary “Direct manipulation” API, then the program is instead sent higher-precision scroll angle deltas. Maybe that implies existence of a conversion factor between pixel and angle deltas somewhere, but I am not aware if it is 1 or something else. But the lack of a way to know for certain whether an angle delta came from a scrollwheel or a higher-precision source seems intentional.

Thank you, Csaba.  No, (2) was not intentional, and neither were the other
errors you mention.  Also, I forgot to look at the scrolling canvas demo.
So I am not done.  I will fix those things.

Just a few remarks regarding the file text.tcl:

1. In line #468 the tk::MouseWheel invocation is missing the factor argument.

2. Is it intentional that in the high resolution case the Option key is simply ignored?

3. You are using expr w/o braces!

I came up with a more robust heuristic for deciding whether a WM_MOUSEWHEEL
message should be considered high resolution.  The answer is YES unless both
the current delta and the previous delta are integer multiples of 120.  The
likelihood of producing two successive deltas that are multiples of 120 with
a trackpad is infinitesimal, based on my experiments.

I tested this by plugging a legacy mouse into my laptop and switching between
scrolling with the touchpad and scrolling with the old mouse.  I saw no
glitches.

I consider this project finished.  (Doing the same for linux looks hard enough
that it should be a separate project - maybe someone who knows more about X11
extensions would help.)

Note to Csaba - this involves no platform dependent code except, of course,
that each platform has its own scheme for generating the MouseWheel XEvents.

Please let me know if there are any remaining issues.  Thanks.

I worried about that. And I would love to know a way to detect whether
a WM_MOUSEWHEEL message is using the high resolution "option".  I couldn't
find one.  I thought of some complicated ways to work around this, but
they all had pitfalls.  Such as what happens if someone plugs a USB mouse
into their laptop, causing the touchpad messages to be interspersed with the
mouse messages.

But then I tested it and I was not able to produce any weird effects caused
by this. Can you?

Basically, it is difficult to produce a value of exactly 120 and extremely
difficult to produce two in a row.  And even if you did produce one, the
glitch would be hard to see since it would be a quite fast scroll anyway.


Of course, if there is a better way I am all ears.

In tkWinX.c:

	    if ( delta % 120 != 0) {
		event.x.xkey.state |= HiresScrollMask;
	    }

A touchpad that normally produces delta other than 120 can still produce a delta of 120, so I think this approach might not be desirable.

Yes, setting  MOZ_USE_XINPUT2=1 smooths it out.

If Tk could arrange to receive the high precision scroll events sent by
the touchpad then I think it would be easy to make the smooth scrolling
work in linux.  But I have no idea how to do that and I have not managed
to find any documentation or examples that provide any hints.

…even Firefox has jumpy scrolling with the trackpad…

Workarounds such as setting either MOZ_USE_XINPUT2=1 or MOZ_ENABLE_WAYLAND=1 environment variables are reportedly sometimes needed for smooth scrolling to work in Firefox on Linux/BSD/etc. depending on whether X11, XWayland, or native Wayland is used.

The bug-fix branch now provides smooth scrolling (of text widgets) for
macOS and Windows.  This is done in a platform independent way, meaning
that no platform-dependent code is needed in the binding for the MouseWheel
event, and that any platform can use the same approach.

The idea is simple.  I borrow a bit in the state field of the MouseWheel Xevent
which is larger than any of the bits used for modifier keys - namely 1 << 9.
(The MouseWheel event is really a KeyPress event with its type set to the 
Tk-specific value MouseWheelEvent.)  The meaning of the flag bit is:
interpet the (signed) integer stored in the (unsigned) keycode field as
being the number of pixels to scroll by.

It works.  It could also be made to work for linux but, according to
[https://pavelfatin.com/scrolling-with-pleasure/] this would involve using
the XInput 2.1 Xorg extension, or something equivalent.  I am not sure how
Tk would go about doing that.

As always, I would appreciate reviews of the changes in the bug-fix branch
and especially reviews that include trying it out.

By plugging several mice into a Windows 11 laptop, including a high-resolution
Apple Mighty Mouse, I have verified that an actual mouse always sends delta
values that are integer multiples of 120 in the WM_MOUSEWHEEL message.

A trackpad, on the other hand, sends arbitrary integer values with magnitudes
up to at least 400 in the WM_MOUSEWHEEL message.  I have not seen any over 500.
(Note that a WM_MOUSEWHEEL message stores the delta as a 16 bit integer.) The
trackpad I was using (on an Asus S3 laptop) also has momentum.

I have seen no evidence to contradict my hypothesis that the values sent by
the trackpad on a Windows laptop can be interpreted as pixels.

Thank you Christopher.  I will fix the casting error immediately.

Currently I am trying to figure out how to assign meaning to the following
statements from:
[https://learn.microsoft.com/en-us/windows/win32/inputdev/wm-mousewheel]

1) The high-order word indicates the distance the wheel is rotated,
   expressed in multiples or divisions of WHEEL_DELTA, which is 120.

2) The wheel rotation will be a multiple of WHEEL_DELTA, which is set at 120.
   This is the threshold for action to be taken, and one such action (for
   example, scrolling one increment) should occur for each delta.

3) The delta was set to 120 to allow Microsoft or other vendors to build
   finer-resolution wheels (a freely-rotating wheel with no notches) to
   send more messages per rotation, but with a smaller value in each
   message. To use this feature, you can either add the incoming delta
   values until WHEEL_DELTA is reached (so for a delta-rotation you get
   the same response), or  scroll partial lines in response to the more
   frequent messages. You can also choose your scroll granularity and
   accumulate deltas until it is reached.

First "multiples or divisions" is not the same as "multiples" if the
multiples are integer multiples.  And the whole business makes no sense
unless the multiples are integer multiples, since any real number is a
multiple of 120.  But then 3) contradicts 2) and my observation also
contradicts 2) - I see values which are arbitrary integers up to about
400 when I scroll with the trackpad.  I am guessing that 2) only applies
to old-fashioned low-resolution scroll wheels.  I am also guessing that
the numbers can be interpreted as pixels, at least for a trackpad.

Regarding [ea7ef17a6b]:

Casting negative floating point values to an unsigned type is undefined behavior. This can be avoided by using xEvent.xkey.keycode = (unsigned int)(int)(delta * MSteryFactor) instead.

Scrolling a text widget is much slower than before, both using a scrollwheel and a trackpad. But the scrolled distance is comparable to TextEdit, so this seems expected, although maybe not consistent with other platforms.

The listbox widget now scrolls by 1 lines per scrollwheel click, rather than 3 as Csaba set out to achieve in [c2be406526]. But the listbox is also stuck in 1994 and only supports displaying and scrolling by an integer number of rows, so getting it to support precision scrolling (i.e. no delta accumulation) would be more work.

I think the cscroll.tcl canvas demo scrolls too fast, however. The right binding at least for trackpads would be something like:

    $c configure -yscrollincrement 1 -xscrollincrement 1
    bind $c <MouseWheel> {
	tk::MouseWheel %W y %D -4.0
    }
    bind $c <Shift-MouseWheel> {
	tk::MouseWheel %W x %D -4.0
    }

I have only briefly tried a Tk 8.7 snapshot on Windows using a scrollwheel mouse, and did not notice Nicolas’ issue despite thinking it could be present due to efforts for [c2be406526]. I have not tried precision scrolling on Windows.

Even with moderately slow trackpad scrolling on Windows the behavior is
unacceptable.  The Y deltas sent in the WM_MOUSEWHEEL message are 1 or 2.
The scroll does not reverse direction, but it takes a lot of scrolling to
build the accumulator to more than 72, which is the threshold for sending
a MouseWheel type Xevent.  The text widget remains fixed for a long time
and then jumps by a few lines.  Not very smooth.

I suspect that the WM_MOUSEWHEEL delta values sent by a trackpad really
are meant to be pixels, as opposed to being 3 degrees of rotation of a
scroll wheel, however that is supposed to translate into pixels.

I am changing the title of this ticket again, because it is not limited to Aqua.
The same behavior, but less extreme, happens on Windows.  I was testing on an
Asus laptop with Windows 11, using the track pad.  When scrolling a text widget
with many lines there are similar artifacts to what Nicolas reported on macOS.

If one scrolls slowly in one direction, using the two-finger gesture on the
trackpad, the Text widget jumps back and forth, sometimes scrolling up and
sometimes scrolling down.  This is almost surely due to the accumulation
scheme. It can be seen in the widget demo on Windows as well as macOS.

The problem is very clearly visible with print statements added to watch
the Y delta values received in the WM_MOUSEWHEEL messages, and the accumulator
values, and the values sent as keycodes in XEvents of type MouseWheelEvent.
The current implementation is a mess.

Windows is capable of supporting smooth scrolling, as can be seen for example
by scrolling in Firefox. And Tk is easily flexible enough to support it on
both Windows and macOS.  (I think it is not possible on Linux, since even
Firefox has jumpy scrolling with the trackpad, but not due to limitations
of Tk.  I will test that more carefully.)

It is important to realize that this is not really an issue with scrollwheels
as much as it is an issue with trackpads.  Scrollwheels may not get used much
but trackpads are probably used more than mice.

I am going to try to implement smooth scrolling for both platforms in this
bugfix branch.

To correct myself, there actually were hidden APIs which could be used for precision scrolling before 10.7: deviceDeltaX, deviceDeltaY, and _continuousScroll. These have the same values as scrollingDeltaX/Y when hasPreciseScrollingDeltas is YES; the runtime deprecation warnings also say to use scrollingDeltaX/Y and hasPreciseScrollingDelta instead of deviceDeltaX/Y and _continuousScroll respectively. So maybe the “spiking” behavior of deltaX/Y was also present on 10.6.

Marc, I am not sure the change you made in text.tcl is OK.  The main goal of the uniformization of the mouse wheel event handling (based on a TIP) was to eliminate any windowing system specific code from the mouse wheel event binding scripts.

I have made a new commit to the bugfix branch which comes close to providing
a smooth scrolling Text with a scroll wheel on Aqua.  It removes the mysterious
factor of 120.  It removes the delta accumulator.  It removes the 30 pixel
minimum for scrolling a Text widget.  It uses the scrollingDelta fields, and
pays attention to hasPreciseScrollingDeltas. For non-precise mouse wheels it
converts the scrollingDelta value to pixels by multiplying by 10 and rounding
away from 0.

I am pretty happy with the results, which I tested with an Apple Mighty Mouse,
a Logitech MX Anywhere 3 bluetooth mouse and the trackpad on a macBook Air.  It
seems to me to scroll as smoothly as Firefox or Safari.

I would be very interested in hearing people's reactions to this attempt.

Another possible reason Apple says to “scroll by scrollDelta points”, while also providing an integer amount for high precision scrollDelta events, is so that apps have the computationally cheaper task of translating already-drawn content by an integer number of pixels, so as to help them stay responsive even when sent scroll events much more frequently.

Regarding this section on momentum in the 10.7 AppKit release notes:

Some Apple mice and trackpads also issue momentum scroll wheel events. That is, the hardware continues to issue scroll wheel events even though the user is no longer physically scrolling. AppKit routes these scroll wheel events to the view the cursor was over when they started. For custom cells, this may not be enough. Using the new methods below, you can now determine when momentum scrolling starts, continues and stops and further route the events appropriately.

I wonder, though, if there is a similar problem trying to handle both click wheel scrolling and precision scrolling, where <MouseWheel> simply does not indicate what kind of input source it is from, nor any intended scroll unit (e.g. pixels or lines, as platform HIG might dictate).

I do not have access to 10.6 Snow Leopard for comparison, but here is my guess at explaining the “spiking” behavior of deltaX/deltaY for certain high-precision input sources (trackpads and similar hardware, but maybe not Apple Mighty mouse).

On 10.6 (and earlier), there was not yet an API for high-frequency events from precision input sources, so the events from these sources had to be accumulated, and then NSScrollWheel events with accumulated delta would only be emitted occasionally—much less frequently than events from input sources.

Then on 10.7, which added the high precision scrollingDelta APIs, NSScrollWheel events now needed to be emitted much more frequently.

So for compatibility with apps still using the 10.6-style delta APIs, the accumulated delta NSScrollWheel events still happen occasionally, as they did on 10.6; they just now have several 0.0 delta NSScrollWheel events in between them on 10.7.

So because I think deltaX/Y can be accumulated amounts, I would think they are not actually used as inputs to scrollingDeltaX/Y. I also think momentum is independent of this accumulation behavior.

The behavior difference between the Apple Mighty Mouse and the macBook Air
trackpad makes a bit more sense if you also look at the momentumPhase field
in the NSScrollWheel event (which I incorrectly referred to as NSMouseWheel
below).

The pattern is that when momentumPhase is non-zero there will be an event
with a fairly large value of deltaY followed by several events with deltaY
equal to 0.0.  These events have scrollingDeltaY values which are non-zero
and of moderate size, so that the sum of the scrollingDeltaY values over
the entire subsequence is approximately 10 times the large initial value
of deltaY.

So scrollingDeltaY is still about 10 times deltaY but the deltaY values
have large spikes followed by zeros, while the scrollingDeltaY values vary
more smoothly.

The Apple Mighty Mouse never generates non-zero values of momentumPhase,
and neither does the Logitech.  I think this confirms that the momentum
behavior of the Logitech mouse is generated by the mouse, not by Apple.

First, a comment in the typographical esoterica category.  When type was
made of lead there were 72.27 points per inch.  That is what TeX and LaTeX
use as the dimension of a point.  Adobe introduced the Desktop Publishing
Point, which is 1/72 inch.  TeX call that unit a Big Point.

I think, Csaba, you are referring to typographical points, as opposed to
Apple points which are the units used for the screen coordinages by Apple.

Now for the results of some experimentation I did:

With an Apple Mouse, in an NSMouseWheelEvent, the value of hasPreciseScrolling
is YES.  The value of deltaY is a float which is always an integer multiple of
1/65536 and the value of scrollingDeltaY is always ceil(10 * deltaY)

With a Logitech mouse the value of hasPreciseScrolling is NO, the value of
deltaY is an integer multiple of 1/65536 and scrollingDeltaY is equal to
deltaY.

With the trackpad on a macbook Air, the value of hasPreciseScrolling is YES,
but the behavior is very different from the scroll ball on the Apple Mouse.
In particular, it is very common for deltaY to be 0.0 while scrollingDeltaY
is 1.0 or -1.0.  Obviously this will have a big (bad) effect with the
current implementation using deltaY.

The reason that scrollingDeltaY is not a function of deltaY is presumably
that the trackpad has "momentum".  The Logitech scrollwheel also has
"momentum", meaning that it continues to send scrollwheel events after
the wheel is stopped, but I suspect that is happening in the mouse, as
opposed to being added by Apple.

1. Regarding the question about the scrolling behavior of the Tk core and Ttk widgets:

- In the early days the text widget scrolled by one line on aqua.  Since tk 8.5 it scrolled by 15 pixels.  The scrolling uniformization (in trunk) changed this to 30 pixels (on all windowing sytems).

- The canvas widget has no built-in mouse wheel bindings, but

    pathName yview scroll number units

  adjusts the view up or down in units of the yScrollIncrement option if it is greater than zero, or in units of one‐tenth the window’s height otherwise.

- The listbox used to scroll by one line on aqua.  The scrolling uniformization has changed this to 3 lines (on all windowing sytems).

2. Regarding the points vs pixels subject: 1 point = [tk scaling] pixels.  On a 96 DPI, unscaled screen this means: 1 point = 1.333... pixels, or 1 pixel = 0.75 points.  However, for quite a long time all this was not valid on aqua, where 1 pixel and 1 point were the same.  A few months ago Jan changed this, and since then the windowing system aqua behaves in this respect just like x11 and win32.

To briefly respond to a few (but not all) points…

WHEEL_DELTA 120 is for TIP 474 so as to match Windows’ <MouseWheel> behavior.

I think the beginning of https://pavelfatin.com/scrolling-with-pleasure/ has a good enough summary of “smooth scrolling”.

The term “point” in Quartz drawing indeed refers to a logical pixel, see https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CocoaDrawingGuide/Transforms/Transforms.html. This comes from a typographical “point” as in 1/72 inches, and since macOS traditionally used a logical pixel density of 72 PPI, 1 point would correspond to 1 logical pixel.

Since Apple says to scroll by scrollingDelta points i.e. logical pixels, I do not think Apple is saying to worry about scrolling by some physical amount specific to your display setup.

As far as Tk Aqua is concerned, one pixel in Tk is drawn as one Quartz logical pixel or “point”. I wish I could leave the discussion at that, and ignore how tk scaling tries to offer a competing definition of typographical “point” inconsistent with other apps. It does not make sense to me whether Apple’s “scroll by one point” guideline still applies when tk scaling could differ from the system logical pixel density. But I feel it would be better for the mouse event code to remain agnostic toward tk scaling rather than try to cancel it out.

Another concrete question:

Where does the factor of 120 come from in the line:
#define WHEEL_DELTA 120
??

Is that, by any chance, meant to be 10 times the default pixel height of
one line in a Text widget?

I am trying to understand the scope of this issue.

The point of Apple's scrolling API seems to be that with a "crude" scroller,
such as a Logitech mouse, widgets should scroll in units of lines of text,
while with a "precise" scroller, such as a trackpad or Apple mouse, widgets
should scroll in units of pixels.

I have a concrete question:

Which Tk or Ttk widgets are capable of scrolling by 1 pixel, and which can
only scroll by at least 1 line?  Can a Text widget scroll by less than 1
line?  Can a Canvas widget scroll by 1 pixel?  What about a Listbox?

I misstated the apparent relation between pixels and points on retina displays.
Apple's display coordinates are floats, and a retina point has a value of 1.0,
which may be the size of 2 physical pixels on the screen.  Anyway, as usual,
conversion between Apple points and Tk pixels is somethng to worry about.

Well, Wikipedia says that an "Apple point" could be converted to what Tk
calls a "pixel" given certain screen resolution information.  I am guessing
that a point is equal to 2 pixels on a retina display.  I an not sure how
that would work on a 3rd party high resolution screen.  And I believe that
the CSS pixel is defined in terms of the visual angle subtended, so I am
not sure that it really is equivalent to an Apple point.

> "Since the advent of high-density "Retina" screens with a much higher 
> resolution than the original 72 dots per inch, Apple's programming
> environment Xcode sizes GUI elements in points that are scaled automatically
> to a whole number of physical pixels in order to accommodate for
> screen size, pixel density and typical viewing distance. This Cocoa
> point is equivalent to the pixel px unit in CSS, the density-independent
> pixel dp on Android[38] and the effective pixel epx or ep in Windows UWP. "

Sorry, the first part of my previous comment is incorrect. Scaling delta does seem to avoid Nicolas’ issue, but I am not sure it is what should be scaled.

Also, the choice of deltaY or scrollingDeltaY seems separate from Nicolas’ issue, because those have the same value when hasPreciseScrollingDeltas is NO.

I do not believe the proposed changes resolve Nicolas’ issue as I had described it on 2023-10-20. I think the issue has to do with the delta accumulation timeout code (WHEEL_DELAY, etc.) introduced for [c2be406526].

The proposed changes only seem to affect the magnitude of the amount scrolled: for non-precision scrollwheels, it is such that they have increased by a noticeable amount, which I find does not work well especially with scrollwheel acceleration in recent macOS (where the scrolling delta is proportional to how fast one scrolled).

Maybe I should also mention an idea, even though it may be too late for 8.7, and I might not attempt to implement it myself anytime soon. Rather than having Tk accumulate scrolling event deltas from the system, Tk widgets should instead support precise scrolling, so that Tk can send non-accumulated high precision scrolling event deltas directly to widgets. That would enable “smooth scrolling” in Tk. See [06537f28d7].

May I suggest that this ticket be retitled, since it is not specifically about the scrollbar widget nor macOS 14 Sonoma?

I believe the current code (as well as the unintended introduction of this ticket’s issue) was meant to address [c2be406526]. I am not certain if that issue was resolved satisfactorily, aside from being closed over 2 years ago. But keep that discussion in mind for possible regressions or platform inconsistencies.

The section “ScrollWheel Events” on AppKit Release Notes for Mac OS X 10.7 discusses how to use scrollingDeltaX/scrollingDeltaY and hasPreciseScrollingDeltas. These APIs are not available on 10.6 Snow Leopard, but are what Apple recommends using instead of deltaX/deltaY.

I may try the proposed changes and comment again later…

The problem seems to be that the code for handling NSScrollWheel events
in 8.7 assumes that the (float) values of deltaX and deltaY will be
integers.  In fact, that is never true for third party mice and it is only
true for an Apple Mouse with a scrollball if you use scrollingDeltaX and
scrollingDeltaY instead of deltaX and deltaY.  More generally, the
scrollingDelta values appear to be integral whenever the value of
hasPreciseScrollingDeltas is YES.

The way you say this in Apple-Speak is:
"When hasPreciseScrollingDeltas is NO, your application may need to
modify the raw value before using it."
But Apple-Speak lacks the words needed to explain what the required
modifications might be.

A modification which seems to work is to multiply by 10.

The actual value of the delta for one click of the Logitech mouse is:
0.100006103515625 = 0.1 + 0.1/16384 = 6554/65536
With an Apple Mouse, the smallest value of a scrollingDelta seems to
be 2.0, which is a bit fast.  So I thought it might be better to divide
by 2 when the scrolling deltas are precise.

When I tested with a trackpad I found that the trackpad has precise scrolling
deltas.  The values of scrollingDeltaX and scrollingDeltaY were always
floating point integers.  The small values of deltaX and deltaY, however,
were various integer multiples of 1/65536. 

For the bugfix branch I used 10 times the scrollingDelta values but then
divided by 2 in the precise case.  It seemed to work OK with both my Apple
Mouse and my Logitech MX Anywhere 3, as well as with the trackpad on a
macBook Air. Some improvement may be possible, though.  And maybe dividing
by 2 is a bad idea.

Hi Marc,
I've tested your bugfix branch about this ticket and it's wayyy better !
it's working fine now.

many thanks for taking care of this bug.

best regards,
nicolas

I can reproduce this on macOS Ventura with a mac mini using a Logitech mouse.

It happens with 8.7 but is fine with 8.6.

I suspect that the code which deals with WHEEL_DELAY is wrong.  That code was
added to 8.7 in ab3c3c1b7 on 2021-03-19).  It was not added to 8.6.

No, that is not related.  There is no replacement for ScrollRect and it is used
in one place by the Text widget.  Even though it is deprecated it still works.
It has been deprecated for about 5 years.

each time I compile Tk I see this warning:
'scrollRect:by:' is deprecated: first deprecated in macOS 10.14 - Use NSScrollView to achieve scrolling views.

isn't it related?

++

thank you Christopher for your investigations.
I do not have a mouse plugged into the Monterey... but I've tried and you're right, it's not specific to Sonoma.

++

I finally tried this on 8.7, and think I have reproduced the issue. If I have, though, then it only affects 8.7, and is not specific to Sonoma (I observe it on e.g. Monterey). It looks like a bug in the scrollwheel delta accumulation which is only noticeable with a mouse scrollwheel, and not a touchpad or other high-resolution scroll input.

If I do this:

--- macosx/tkMacOSXMouseEvent.c
+++ macosx/tkMacOSXMouseEvent.c
@@ -573,6 +573,7 @@ enum {
        }
        tsdPtr->wheelTickPrev = wheelTick;
        delta = [theEvent deltaY];
+       fprintf(stderr, "deltaY %lf\n", delta);
        if (delta != 0.0) {
            delta = (tsdPtr->vWheelAcc += delta);
            if (timeout && fabs(delta) < 1.0) {

and scroll slowly over any widget, the output is either deltaY 0.100006 or deltaY -0.100006 for each click. If I do bind all <MouseWheel> {puts {%%D %D}}, then on 8.7 I expect either %D 120 or %D -120 to be output for each click. At about 3 clicks per second it works correctly; but at about 4 or more clicks per second, either there is no <MouseWheel> event, or the sign of %D is wrong.

after macOS 14.1 update, issue is still there.

++

it happens only with TK app, Xcode is scrolling smoothly.
I use the mouse of a raspberryPi.

++

In the videos, the scrollbar stays perfectly synchronized with the scrolled widget. So I do not think it is a bug in the scrollbar widget, but it could be an issue with the binding of the scrolled widget.

Does this issue happen in any other apps on Sonoma? I assume it does not, but maybe I should ask anyway. What model of mouse is this?

I knew you would say that  :)
here's one with widget demo
https://vimeo.com/869045009

++

The scroll binding on your channels widget seems rather coarse, so I wonder what the effect looks like with the widget demo, which should be smoother. But I am more curious what sort of delta values you are getting and whether they make sense. Maybe try running wish and then doing:

bind . <MouseWheel> {puts {%D}}

Hi Christopher,
here's what I see:
https://vimeo.com/869035497

I'm using the wheel of the mouse. At the beginning I only roll the wheel in one direction to describe the issue.
At some point I've put the scrollbar in the middle of the range and I've rolled the wheel in only one direction.

++

Currently I would say I do not see this issue, but I am not sure I understand what is described. Are you able to provide a screen recording of it?

I have been aware of some quirks in the scrollbar widgets, and not just on Aqua. One example is the thumb height varying while scrolling due to inconsistent rounding. But what is described here sounds more dramatic than that.