Great ! If CI is clean, you may merge to main. Thanks for all, Harald

You can also reuse the ticket.

Just put comments here.

Even if it is closed, you may still edit it.

Great, thank you.

You may also reopen the branch and continue to use it. It is up to you. Click on the commit number, press "edit" and select the cancel "closed" tag option.

Thanks for all, Harald

I see the branch is closed so that answers my question! I will create a new branch and ticket for the updates. :)

Harald,

Yes good point on the documentation. Previously that xrender library was pulled in as a transient dependency from Xft, but now that xrender is used for the Linux TkpPutRGBAImage.

I would like to do a couple follow-up changes, for example, there are a few "ckalloc" &co. function calls that should probably be their Tcl_Alloc analogues. I can work the documentation change in.

Should I make these changes in the existing branch, or create a new one? Also should I create a new ticket?

Thanks,
Mason

The patch introduces the following new dependencies:

• MS-Win: msimg32.lib -> present at least since Windows XP -> ok
• Linux: X-Renderer -> this is optional and tested by the configure script.

There is a new configure switch for Linux: --enable-xrender (default: yes) Is there additional documentation needed ?

Thanks for all, Harald

CI is clean. Schelte is in favor. Merged to main by commit [3bbe54bc].

@Mason: thanks for the awesome work of this very long standing annoyance!

Bug closed.

Take care, Harald

Great. Activated CI on head node. When this is clean, we will erge to main.

Mason, you may also adopt the main text of this ticket, if the original text changed.

Thanks for all, Harald

macOS (arm64):
- Opaque:      resize-bench (diagonal): 52 steps in  995.1ms (19.14ms/step)
- Transparent: resize-bench (diagonal): 52 steps in 1007.9ms (19.38ms/step)  

Windows:
- Opaque:      resize-bench (diagonal): 52 steps in  3001.3ms (57.72ms/step)
- Transparent: resize-bench (diagonal): 52 steps in  3113.8ms (59.88ms/step)

Linux (WSL2):
- Opaque:      resize-bench (diagonal): 52 steps in  1136.6ms (21.86ms/step)
- Transparent: resize-bench (diagonal): 52 steps in  1803.3ms (34.68ms/step)

Info from the core list:

Schelte

In the ticket, Mason reported that transparent and opaque runs have virtually identical runtime on Windows. On linux I still see a huge difference between the two runs, with transparent being more than 10 times slower than opaque. But still the transparent run is similar to the time Mason reported and I'm running this on quite an old machine. So maybe the opaque run is just really slow on Windows compared to linux?

These are my results:
Transparent: resize-bench (diagonal): 52 steps in 8590.6ms  (165.20ms/step)
Opaque: resize-bench (diagonal): 52 steps in 672.6ms  (12.93ms/step)

In any case, both are significant improvements compared to running the same test with Tk 9.0:
Transparent: resize-bench (diagonal): 52 steps in 15718.8ms  (302.28ms/step)
Opaque: resize-bench (diagonal): 52 steps in 2402.1ms  (46.19ms/step)

Mason

Thanks for testing the change out and provide feedback. The gap on Linux is expected at the moment due to the missing TkpPutRGBAImage implementation. So, the gains on Linux are primarily from the caching behavior. On Windows the performance issue was a huge bottleneck to custom theming. Since we can now leverage the nice SVG capabilities in Tk9, it seemed most important to bring Windows down around 100ms (from 500/600ish) in the diagonal resize test with alpha-composited SVG elements.

If folks here review the changes and like how it's all going, I will follow up with the Linux TkpPutRGBAImage. I didn't prioritize this one yet because it was already floating in the 100's on the alpha-compositing anyway, but surely we can boost it a lot more. :)

Harald

thanks for attacking Linux to. I personally would love, that we first could merge the current branch and then do the next step.
But Wizards are always fast, that is life...

The CI run of today is here:
https://github.com/tcltk/tk/actions?query=
Workflow core-tip-754-ttk-composed-pixmaps-cache

It is the older commit without the test reform by Eric.

We see a failure on MacOS:

The nodeCache tests failed.

https://github.com/tcltk/tk/actions/runs/27400538119/job/80977346594

==== nodecache-1.1 Redraw with nothing changed is served from the cache FAILED
==== Contents of test case:

    set ncLog {}
    ncExpose .nc
    llength $ncLog

---- Result was:
6
---- Result should have been (exact matching):
0
==== nodecache-1.1 FAILED



==== nodecache-1.2 Repeated redraws stay cached FAILED
==== Contents of test case:

    set ncLog {}
    ncExpose .nc
    ncExpose .nc
    ncExpose .nc
    llength $ncLog

---- Result was:
18
---- Result should have been (exact matching):
0
==== nodecache-1.2 FAILED



==== nodecache-2.3 Redraw after a state change hits again FAILED
==== Contents of test case:

    set ncLog {}
    ncExpose .nc
    llength $ncLog

---- Result was:
6
---- Result should have been (exact matching):
0
==== nodecache-2.3 FAILED



==== nodecache-3.3 Redraw after a resize hits again FAILED
==== Contents of test case:

    set ncLog {}
    ncExpose .nc
    llength $ncLog

---- Result was:
15
---- Result should have been (exact matching):
0
==== nodecache-3.3 FAILED



==== nodecache-5.2 Content change above leaves the node below cached FAILED
==== Contents of test case:

    set ncLog {}
    nc.over changed 0 0 30 15 30 15
    ncExpose .nc
    list [ncLogged nc.under] [ncLogged nc.over]

---- Result was:
1 1
---- Result should have been (exact matching):
0 1
==== nodecache-5.2 FAILED



==== nodecache-8.1 Stable parcels stay cached through a layout sweep FAILED
==== Contents of test case:

    set ncLog {}
    set x0 [winfo x .sw.nc]
    for {set w 120} {$w <= 220} {incr w 20} {
    .sw.holder configure -width $w
    update
    }
    set x1 [winfo x .sw.nc]
    list [expr {$x1 > $x0}] [llength $ncLog]

---- Result was:
1 36
---- Result should have been (exact matching):
1 0
==== nodecache-8.1 FAILED

Perhaps, the tests are not valid on MacOS as the cache is not used. Then, they should be flagged as "not on MacOS".

Could you look into this ?

Thanks for explaining the choice to create a new test file imgPhInstance.test.
I'm entirely convinced by your arguments. Wise decision.

As for the image cleanup: I believe there is a "testutils forget image" missing
in the section TESTFILE CLEANUP of both test files.

I've had a look at the new test files imgPhInstance.test and
ttk/nodecache.test, mostly from the point of view of maintainability.

Overall: they look fine to me in this respect !

A few recommendations/suggestions/questions:

* Just being curious: why create a new test file imgPhInstance.test instead of
  adding the new tests to the existing test file imgPhoto.test?

* To conform to the usage of comments/headers for standard test sections in
  other test files, I'd suggest a few re-arrangements. Please see the
  attached patch.

* For the cleanup of images, the Tk test suite already provides some standard
  procs for test authors in testutils.tcl: imageInit, imageFinish, imageCleanup
  and friends. You may find these useful instead of manually enumerating image
  names for cleanup. See also their usage in imgPhoto.test.

Regards,
Erik Leunissen.

The branch [tip-754-ttk-composed-pixmaps-cache] had two open leaves:

• (1) 2026-06-11 03:04:04 [3a179196ef] Mason
• (2) 2026-05-31 15:20:15 [d2bcc17325] Marc

I have closed the branch [d2bcc17325] by Marc to avoid the conflict. Better would be to merge in and to resolve the issues. I tried but was not able to do so.

So, here are the commands:

• Remove the closed flag from [d2bcc17325]

fossil merge d2bcc17325
MERGE generic/ttk/ttkImage.c
***** 4 merge conflicts in generic/ttk/ttkImage.c
WARNING: 1 merge conflicts

Resolve the merge conflicts and commit with "closed fork".

Thanks and sorry, Harald

Windows:
- Opaque:      resize-bench (diagonal): 52 steps in 6542.7ms  (125.82ms/step)
- Transparent: resize-bench (diagonal): 52 steps in 6396.4ms  (123.01ms/step)

  nmake -f makefile.vc test-classic TESTFLAGS="-file imgPhInstance.test"
  nmake -f makefile.vc test-ttk     TESTFLAGS="-file nodecache.test"

Thanks, great !

It would be great to focus on the main branch so this change goes into the first beta of tk9.1 expected end of the month.

You can ask for testing to make it work on all platforms. In addition, CI testing may be triggered if interested.

Thanks, Harald

I added a regression patch for 9.0.3 here (tk-9.0.3-ttk-image-el-cache.diff). Strangely, this runs much faster than my previous nmake build. These results are really good:

Windows/MSYS2 (Tk 9.0.3 with TkpPutRGBAImage + cache)
- Opaque:      resize-bench (diagonal): 52 steps in 4657.0ms  ( 89.56ms/step)
- Transparent: resize-bench (diagonal): 52 steps in 6920.1ms  (133.08ms/step)

Okay, so I implemented TkpPutRGBAImage on Windows (see latest ttk-layout-node-cache.diff). With that change plus the layout-node caching mechanism, the results seem pretty solid! The gap between opaque and transparent is ~ 40ms, and the overall cost is dramatically less than without this patch.

Horizontal and vertical resize are better than diagonal, understandably, so the overall performance improvement with this change will be really nice on Windows, and Linux given regular usage (not the benchmark script). Once macOS can take advantage of the caching it will likely see a speedup as well.

Windows (with TkpPutRGBAImage + cache):

Diagonal:
- Opaque:      resize-bench (diagonal): 52 steps in 6827.6ms  (131.30ms/step)
- Transparent: resize-bench (diagonal): 52 steps in 9053.8ms  (174.11ms/step)

Horizontal:
- Opaque:      resize-bench (horizontal): 52 steps in 5495.4ms  (105.68ms/step)
- Transparent: resize-bench (horizontal): 52 steps in 7858.9ms  (151.13ms/step)

Vertical:
- Opaque:      resize-bench (vertical): 42 steps in 3120.4ms  (74.30ms/step)
- Transparent: resize-bench (vertical): 42 steps in 4941.8ms  (117.66ms/step)

You should have commit rights since some weeks... If not, E-Mail me... Harald

The test script now allows testing fully opaque and also transparency. Here are current performance metrics with the latest caching patch. As is evident here, the caching dramatically improves the speed for both fully opaque and transparent image elements. This is without any optimization on the blending path, which should of course be subject to the profiling runs.

Linux (before):
- Opaque:      resize-bench (diagonal): 52 steps in 3410.8ms   ( 65.59ms/step)
- Transparent: resize-bench (diagonal): 52 steps in 20413.2ms  (392.56ms/step)

Linux (after):
- Opaque:      resize-bench (diagonal): 52 steps in 1773.3ms   ( 34.10ms/step)
- Transparent: resize-bench (diagonal): 52 steps in 11097.4ms  (213.41ms/step)

Windows (before):
- Opaque:      resize-bench (diagonal): 52 steps in 25325.8ms  (487.04ms/step)
- Transparent: resize-bench (diagonal): 52 steps in 33213.7ms  (638.72ms/step)

Windows (after):
- Opaque:      resize-bench (diagonal): 52 steps in 8587.4ms   (165.14ms/step)
- Transparent: resize-bench (diagonal): 52 steps in 16579.1ms  (318.83ms/step)

Schelte's report seemed to suggest that all of the "slowness" is from the
blending operation.  That is consistent with your observations since
the caching also eliminates the blending.  So I think we need some
measurements that identify where the time is being spent.  How about
an old-fashioned profiling run?

Marc,
thanks for the post. This is also my knowledge on the MS-Windows side.
In addition, a general solution, and not only for scrollbars, would be great.

Thanks,
Harald

One thing about chatbots is that they **love** to chat with you.  An endless
re-design cycle with ever-expanding scope is just dandy with them.  I suggest
that we give the chatbots a rest and think about the core problem with brains,
looking for a pleasing compromise between simplicity and improvement.

Here are a few ideas.

Claude claims "this produces 6,000-12,000 GDI create/destroy cycles per second".
Maybe we could start there.  I have no information about how expensive these
so-called cycles are, but let's assume that is the key issue.  

Looking at TkPutImage in the windows port I see:

    bitmap = (HBITMAP)SelectObject(dcMem, bitmap);
    BitBlt(dc, dest_x, dest_y, (int) width, (int) height, dcMem, src_x, src_y,
            SRCCOPY);
    DeleteObject(SelectObject(dcMem, bitmap));
    DeleteDC(dcMem);
    TkWinReleaseDrawableDC(d, dc, &state);

where the bitmap which appears as an argument to SelectObject was created by
CreateDIBitmap.  I guess Claude is referring to CreateDIBitmap and
DestroyObject as the cycle.

If CreateBitmap and DestroyObject are what is slow, why not cache
the bitmap in TkPutImage, so if TkPutImage were called many times with the
identical image it could avoid all but one of the CreateBitmap and DeleteObject
calls?

A second idea would be: since Schelte finds that the bottleneck is the alpha
blending, rather than the creation and destruction of bitmaps, how about
trying to implement TkpPutRGBAImage for Windows, so that the GPU could be
used to do the alpha blending.  I am sure Windows supports using the GPU
for such a task.

A third idea would be to treat the special case of an image which is 1 pixel
high (in the vertical case), or 1 pixel wide (in the horizontal case), as the
primary application for the tiling construction, and try to optimize for that
case. I believe that using StretchBlt might allow tiling an entire scrollbar
trough with one function call instead of 6000 -12000 calls to BitBlt.

There are probably many other, possibly relatively simple, optimizations which
humans could come up with, if they were to think about it.

Harald,

> What I understood from the patch is that (semi-)transparent pixels
> are always slow, as they are painted pixel by pixel.

That is not correct.

In generic/tkImgPhInstance.c there is conditional code bracketed by
#ifndef TK_CAN_RENDER_RGBA ... #endif which is responsible for blending
an image with transparency over an opaque image.

When TK_CAN_RENDER_RGBA is not defined the function BlendComplexAlpha
is called.  It does the compositing operation on a pixmap in CPU memory.
This blending involves copying a rectangle from the window into a pixmap,
blending the pixmap with the image, using the image's alpha channel, and
then copying the pixmap back to the window.  This all happens in the CPU.

When TK_CAN_RENDER_RGBA is defined, the function TkpPutRGBAImage is used.
That is a platform-specific function which can use the graphics operations
provided by the OS to do the blending.

The only platform which defines TK_CAN_RENDER_RGBA at the moment is macOS.
(Wayland will also use it, as the Wayland port has to interact directly
with the Mesa graphics driver using OpenGL.)  The macOS port draws directly
to the window's backing store CGImage, so it can use Apple's compositing
features to do the blending, and it happens in the GPU.

What I understood from the patch is that (semi-)transparent pixels are always slow, as they are painted pixel by pixel.

The patch reduces the number of semitransparent pixels by cacheing the image, if the image is composed of multiple images which are superposed and have semi-transparent pixels. If there are any pixels not covered by the cached image (and a background shines through) the performance is bad (but less bad than without the patch, if any semi-transparent pixels change status to non-transparent par the overlay procedure).

The fact that semi-transparent pixels are slow (on MS-Windows) is not addressed. Only, the number of those pixels is reduced.

Does this make sense?

Thanks for all, Harald

--- test_scrollbar_perf.tcl~    2026-06-02 11:12:38.043915660 +0200
+++ test_scrollbar_perf.tcl     2026-06-02 11:37:10.639595544 +0200
@@ -46,10 +46,10 @@
 img_thumb transparency set 13 0 1
 img_thumb transparency set 0 23 1
 img_thumb transparency set 13 23 1
-img_thumb put {{#80505050}} -to 1 0 2 1
-img_thumb put {{#80505050}} -to 12 0 13 1
-img_thumb put {{#80505050}} -to 1 23 2 24
-img_thumb put {{#80505050}} -to 12 23 13 24
+img_thumb put {{#505050}} -to 1 0 2 1
+img_thumb put {{#505050}} -to 12 0 13 1
+img_thumb put {{#505050}} -to 1 23 2 24
+img_thumb put {{#505050}} -to 12 23 13 24
 
 image create photo img_trough -width 14 -height 14
 img_trough put [string repeat {{#2a2a2a} } 14] -to 0 0 14 14

I tested the bugfix branch on Windows with a high-dpi EVO display scaled to
200%.  The test script ran and the scrollbar slider was (barely) visible
as it is on other platforms.  So I take that as empirical proof that there
is not problem with XCopyArea on Windows with a high-dpi display, although
I can't explain what is making it work.

I guess there is currently no issue with high-dpi screens in linux, because
the only scaling possible is to change the screen resolution.  For Tk a
pixel in a pixmap is equivalent to a pixel on the screen, and Xorg manages
that via screen resolution.  A default 200x200 Wish window is not
large enough for the title bar to hold the default window controls on a
high-dpi screen, but that is true no matter how you change resolution.
So XCopyArea should work fine in all cases.

Hi Mason,

I also worked on the code, after your patch, and added comments to try
to explain what was going on there after doing my best to understand the
situation.  I will commit my changes and you can look at them.

However, I think there are a couple of serious issues with this change
which I do not think have been discussed.

1. The code assumes that the destination rectangle will look the same
whenever the widget containing the image element has the same state. I
don't think that is always a valid assumption.  I think it might be valid
it the widget is opaque, so the background of the destination rectangle
gets completely drawn when drawing the parts of the widget that are drawn
before the destination rectangle is tiled.  But I can imagine cases where
that is not the case.  For example, a scrollbar with a transparent trough,
through which one can see the window content underneath the scrollbar.
That content might change without any change to the scrollbar, for example
if the content was animated.  And in that case the cached pixmap would not
match its surroundings if copied into the widget.

2. It is not clear to me that the patch handles high-dpi screens correctly.
When you create an mxn pixmap it has m*n pixels, each represented by 4
bytes of CPU memory.  But an mxn rectangle on a 2x high-dpi screen has
(2*m)*(2*n) screen pixels, representing m*n logical pixels, stored in GPU
memory. I believe that TkPutImage, because it is  using image drawing code
provided by the OS, can render the mxn pixmap into a (2*m)x(2*n) framebuffer
rectangle by linearly scaling the pixmap image.  I could certainly 
be wrong, but I suspect that XCopyArea does not handle the scaling the
same way.  (I do know that this issue is the key blocker for extending
XCopyArea so that it can work with pixmap source or destination drawables.)

What do you know about #2?  How is the high-dpi rendering handled by the
patched code versus the previous code?

I don't know anything about BitBlt (used by TkPutImage on Windows) but
Gemini told me that:
 "The operation is ultimately translated into a GPU instruction or a
  high-speed contiguous chunk transfer (such as a hardware-accelerated
  memcpy) via the Direct3D/DXGI stack or the Desktop Window Manager".
That sort of sounds like a Direct3D version of glBlitBuffer, which
automatically linearly scales an image if the source and destination
rectangles are different sizes.  But it apparently can have a CPU
source.  On the other hand Gemini says that BitBlt does not do
scaling and that one would have to use StretchBlt for that.  In fact,
StretchBlt is used in GdiImage, but it looks like that is only used by
the Canvas widget, not by XPutImage.

And then there is the question of how the XCopyArea and TkPutImage compare
under X11 with a high-dpi screen.

So I really don't know where things stand regarding high-dpi screens.

While I may be making mountains out of molehills, I would feel much better
about this if some of these concerns were addressed by someone who knows
more about the situation than I do.

Mason, I gave you commit rights on the tk repository. Could you update the branch?

Thanks for all, Harald

Never mind -- I see that I can use the performance script to verify
that the scrollbar looks like a scrollbar, and assume that it looks
the same as you intended.

Mason, Do you have a script that I could run to make sure that your
custom scrollbar appears correctly on macOS?

I don't think we need to add a new #define constant which we plan to remove
as soon as possible.  Why not use TK_NO_DOUBLEBUFFERING which is what avoids
essentially the same use of XCopyArea throughout the generic code?  It also
probably represents what is going on here; that is to say that the fact
that macOS draws directly to the screen was probably a big part of why
it did not have the big delays in the first place.

Also, Claude seems very determined to characterize XCopyArea as a "blit" but
I think that is wrong.  You cannot blit from a Pixmap (which lives in CPU
memory) to the screen.  That is always a slow copy operation.  A blit moves
pixel data efficiently between two framebuffers, both living in GPU memory.
I think it is misleading and unhelpful to refer to XCopyArea as a "blit".
Could we please remove that comment?

Marc,

Thank you for finding that issue. I have completed a fix but I don't seem to have write access to the Tk branch.

>> I retry a CI run. If this succeeds, I will merge to main.
>> No merge to core-9-0-branch. 

Harald,

Please do not merge to main until the cache code is made conditional on #ifndef NO_TK_DOUBLE_BUFFERING
It will break macOS. Please don't do that.

IMPORTANT!

I juat remembered why I was so worried about this patch for macOS.  It
woke me up in the middle of the night.  It has a fatal flaw, for macOS
only: It will not render the cached Pixmaps!  The fact that the test
script runs faster is meaningless if the drawing is not happening.

The reason that the drawing will hot happen has two parts:
(1) The drawing code includes the following line:

XCopyArea(Tk_Display(tkwin), imageData->cachedPixmap, d, gc,
		0, 0, (unsigned)dst.width, (unsigned)dst.height, dst.x, dst.y);

(2) The macOS port of XCopyArea is currently incomplete and does not
support a source or destination drawable which is a Pixmap.  If passed
a Pixmap for either it just returns BadDrawable and does no drawing.
(See lines 1125-1128 in macosx/TkMacOSXImage.c.)

While XCopyArea is used in the generic code to do makeshift
double-buffering (copy from window to pixmap, draw on pixmap, copy
back to window), that code is conditional, based on the compiler
constant TK_NO_DOUBLE_BUFFERING, which is defined by the macOS port only.

Finishing XCopyArea for macOS is an important project which needs to be
done.  But it is non-trivial for a few reasons, a major one being that
it needs to deal with Retina displays.  I have been thinking about this
lately, after it came up in the ticket [4af5ca1921].  I think the simplest
approach would be to require that pixmaps are always using logical pixels
and let the OS do the linear scaling to the 2x2 pixels used by a Retina
display.  That will mean that pixmaps displayed on a Retina display are
more pixelated than the origianl and will not be anti-aliased as well. I
don't see a practical alternative to that at the moment, but I am not
sure it will be adequate for projects like the one discussed in
[4af5ca1921].

Another aspect of this is that the scaling which would be required if
XCopyArea always treats pixmaps as consisting of logical pixels is that
that scaling makes rendering considerably slower.  So if the test script
were actually rendering the pixmaps, instead of just checking that they
are pixmaps and returning BadDrawable when the answer is yes, it might
well mean that the macOS speed up would be considerably less, at least
on Retina displays.

In any case, I think the only reasonable way for this ticket to move
forward without breaking the graphics on macOS would be to make the new
caching code conditional on ifndef(TK_NO_DOUBLE_BUFFERING), which would
exclude macOS, and to revert to the original code when
TK_NO_DOUBLE_BUFFERING is defined.

I am also curious about whether the original Windows problem is affected
by whether the computer has a high-DPI screen.

Ok, thanks. Now, I understood the tests. You have to resize the window, not scroll the scrollbar. Improvement is directly visible on Windows, thanks for that.

I retry a CI run. If this succeeds, I will merge to main. No merge to core-9-0-branch.

Any opinions on those proposals?

Thanks for all, Harald

On an M3 macbook Air (with Retina display, which affects XCopyArea) running
macOS 26.4 I got:

Pre-patch: resize-bench (diagonal): 52 steps in 2980.8ms  (57.32ms/step)
Post-patch: resize-bench (diagonal): 52 steps in 1970.2ms  (37.89ms/step)

There were no crashes!

The non-regression test failures were the usual ones that I see on that
system (event-9.18 and event-0.19) plus two new ones (send-13.1 and send-13.2)
which I am sure are unrelated.

So I would say that the patch looks like a significant improvement for
macOS.

To summarize the performance changes below, I am getting about 40% improvement on Windows/Linux and 20% improvement for macOS.

Ideally, the Windows/Linux systems should have sub-100 ms latency on the resizing. Perhaps a follow-up change can be done to improve these systems. However, this minimally-invasive change seems to provide improvements to performance across the board.

macOS (arm64):
- before: "resize-bench (diagonal): 52 steps in 2785.8ms  (53.57ms/step)"
- after:  "resize-bench (diagonal): 52 steps in 2232.5ms  (42.93ms/step)"

Linux (WSL2):
- before: "resize-bench (diagonal): 52 steps in 17462.2ms  (335.81ms/step)"
- after:  "resize-bench (diagonal): 52 steps in  9890.8ms  (190.21ms/step)"

Windows:
- before: "resize-bench (diagonal): 52 steps in 18896.7ms  (363.40ms/step)"
- after:  "resize-bench (diagonal): 52 steps in 11025.1ms  (212.02ms/step)"