Tk Source Code

View Ticket
Login
Ticket UUID: 4af5ca1921122de63f37a99beefb2f5dbef72518
Title: XCopyArea is very (very) slow on macOS with version 9
Type: Bug Version: 9.0.1
Submitter: anonymous Created on: 2025-07-23 12:54:55
Subsystem: 64. Graphic Contexts Assigned To: nobody
Priority: 5 Medium Severity: Important
Status: Open Last Modified: 2026-10-29 15:33:32
Resolution: None Closed By: nobody
    Closed on:
Description:

I would like to report a bug (I hope so) on macOS with the XCopyArea function on Tcl/Tk 9.0.1. Tcl/Tk 8.6 and Tcl/Tk 9 on Windows are not impacted.(I can't tested on linux I don't have this platform)

Context :

I interface Nim and Tcl/Tk and I find a very big difference (x1000) between TK 8.6 and TK9 on this function.

Versions:

Tcl/Tk 8.6.16
Tcl/Tk 9.0.1

I'm not a C programmer, but I tried using this code, which is similar to my Nim code, to show you the difference in time.
In one frame .f, I tried to construct an image.

Note: I couldn't get this code to work on version 8.6.16 on macOS (I don't know why). XCopyArea code returns always BadDrawable. In my Nim code, I create an image with

[image create myIM]
. Perhaps this is necessary for the code below to function, but I do not have the expertise to determine this.

/*
 * xcopyarea.c - XCopyArea with Tcl/Tk (version dylib)
 *
 */

#include <tcl.h>
#include <tk.h>
#include <stdio.h>
#include <X11/Xlib.h>
#include <X11/Xutil.h>
#include <X11/X.h>
#include <time.h>
#include <sys/time.h>
#include <string.h>
#include <stdlib.h>

/* Variables globales */
static Tk_PhotoHandle photo = NULL;
static int width = 800;
static int height = 400;
static Pixmap pixmap;

double get_time_ms(void) {
    struct timeval tv;
    gettimeofday(&tv, NULL);
    return tv.tv_sec * 1000.0 + tv.tv_usec / 1000.0;
}

static int TestImageUpdateCmd(ClientData clientData, Tcl_Interp *interp, 
                             int objc, Tcl_Obj *const objv[]) {
    double start, end;
    int iterations = 100;
    int i, x, y;
    double total_time = 0.0;
    Tk_PhotoImageBlock block;
    unsigned char *data;
    
    if (objc > 2) {
        Tcl_WrongNumArgs(interp, 1, objv, "?iterations?");
        return TCL_ERROR;
    }
    
    if (Tcl_GetIntFromObj(interp, objv[1], &iterations) != TCL_OK) {
        return TCL_ERROR;
    }

    Tk_Window tkwin = Tk_MainWindow(interp);
    if (tkwin == NULL) {
        Tcl_SetResult(interp, "No main window", TCL_STATIC);
        return TCL_ERROR;
    }

    Tk_Window targetWindow = Tk_NameToWindow(interp, ".f", tkwin);

    if (targetWindow == NULL) {
        Tcl_SetResult(interp, "No '.f' frame", TCL_STATIC);
        return TCL_ERROR;
    }

    Tk_MakeWindowExist(targetWindow);

    Window window    = Tk_WindowId(targetWindow);
    Display *display = Tk_Display(targetWindow);

    if (window == None) {
        Tcl_SetResult(interp, "Window not yet created", TCL_STATIC);
        return TCL_ERROR;
    }
    
    XGCValues gcValues;
    gcValues.graphics_exposures = False;
    GC gc = Tk_GetGC(targetWindow, GCGraphicsExposures, &gcValues);

    data = (unsigned char *)malloc(width * height * 4);
    if (data == NULL) {
        Tcl_SetResult(interp, "Memory allocation failed", TCL_STATIC);
        return TCL_ERROR;
    }
    
    for (y = 0; y < height; y++) {
        for (x = 0; x < width; x++) {
            int offset = (y * width + x) * 4;
            data[offset+0] = (x & 0xFF);
            data[offset+1] = (y & 0xFF);
            data[offset+2] = ((x + y) & 0xFF);
            data[offset+3] = 255;
        }
    }

    XImage *ximage = XCreateImage(
      display, 
      Tk_Visual(targetWindow),
      Tk_Depth(targetWindow),
      ZPixmap,    
      0,          
      (char*)data,
      width,
      height,
      32,
      width * 4
    );
    
    
    pixmap = Tk_GetPixmap(display, window, width, height, Tk_Depth(targetWindow));

    if (pixmap == None) {
        Tk_FreeGC(display, gc);
        Tcl_SetResult(interp, "Error Tk_GetPixmap", TCL_STATIC);
        return TCL_ERROR;
    }

    int ximage_status = XPutImage(display, pixmap, gc, ximage, 0, 0, 0, 0, width, height);

    if (ximage_status != 0) {
        Tcl_SetResult(interp, "Error XPutImage", TCL_STATIC);
        return TCL_ERROR;
    }

    for (i = 0; i < iterations; i++) {
        start = get_time_ms();
        
        int status = XCopyArea(display, pixmap, window, gc, 0, 0, width, height, 0, 0);
        printf("status: %d\n", status);
        XFlush(display);
        end = get_time_ms();

        double elapsed = end - start;
        total_time += elapsed;

        if (i < 10 || i % 10 == 0) {
            printf("Iteration %d: %.3f ms\n", i, elapsed);
        }
    }
    

    double avg_time = total_time / iterations;
    printf("\nAverage time: %.3f ms\n", avg_time);
    
    char result[256];
    sprintf(result, "%.3f", avg_time);
    Tcl_SetResult(interp, result, TCL_VOLATILE);
    
    return TCL_OK;
}

#ifdef __cplusplus
extern "C" {
#endif

DLLEXPORT int Imagetest_Init(Tcl_Interp *interp) {
    if (Tcl_InitStubs(interp, "8.6-", 0) == NULL) {
        return TCL_ERROR;
    }
    
    if (Tk_InitStubs(interp, "8.6-", 0) == NULL) {
        return TCL_ERROR;
    }

    Tcl_CreateObjCommand(interp, "test_image_update", TestImageUpdateCmd, NULL, NULL);
    
    if (Tcl_PkgProvide(interp, "imagetest", "1.0") != TCL_OK) {
        return TCL_ERROR;
    }
    
    return TCL_OK;
}

#ifdef __cplusplus
}
#endif

And my test code :


if {$tcl_version eq "8.6"} {
    catch {load libimagetest86.dylib}
    catch {load libimagetest86.dll}
} else {
    catch {load libimagetest90.dylib}
    catch {load libimagetest90.dll}
}

wm geometry . 800x400
wm title . "XCopyArea Test"

frame .f -width 800 -height 400
pack .f -fill both -expand true

update

# test
test_image_update 50

tclsh90 testcode.tcl (simulation code C)

Iteration 0: 18.325 ms
Iteration 1: 17.566 ms
Iteration 2: 17.386 ms
Iteration 3: 16.321 ms
Iteration 4: 18.275 ms
Iteration 5: 19.266 ms
Iteration 6: 19.256 ms
Iteration 7: 18.219 ms
Iteration 8: 18.227 ms
Iteration 9: 16.239 ms
Iteration 10: 17.399 ms
Iteration 20: 18.261 ms
Iteration 30: 18.177 ms
Iteration 40: 19.216 ms

tclsh86 testcode.tcl (simulation code nim)

Iteration 0: 0.325 ms
Iteration 1: 0.566 ms
Iteration 2: 0.386 ms
Iteration 3: 0.321 ms
Iteration 4: 0.275 ms
Iteration 5: 0.266 ms
Iteration 6: 0.256 ms
Iteration 7: 0.219 ms
Iteration 8: 0.227 ms
Iteration 9: 0.239 ms
Iteration 10: 1.399 ms
Iteration 20: 0.261 ms
Iteration 30: 0.177 ms
Iteration 40: 0.216 ms

User Comments: chrstphrchvz added on 2026-10-29 15:33:32:

The improvements for [f215a9fd] were merged, so XCopyArea() will now work again for pixmaps on Tk 9.0+, including the pix.c example. Usage of CGContextSetInterpolationQuality(…, kCGInterpolationNone) which I had mentioned in an earlier comment here has also been added (see [32f2e19d]).

I have not revisited XCopyArea()'s purported performance issues, but in case it needs elaborating, the idea that using XCopyArea() should perform better than repeatedly calling XPutImage() comes from how these APIs behave for actual X11, and for X11 programs in general—not just Tk. If you are running an X client program on a remote computer using SSH forwarding, then XCopyArea() only has to tell the X server on your computer to take the contents of one drawable—from its "backing store" already in memory somewhere on your computer—and put it in another drawable. So it won't surprise me if discussions of those APIs will tend to assume that behavior rather than that of Tk emulation ports.

I did encounter a Tcl 9 porting issue while trying pix.c. UndefinedBehaviorSanitizer (-fsanitize=undefined) reports this error:

tcl91/generic/tclBasic.c:2782:12: runtime error: call to function draw_pix_surface through pointer to incorrect function type 'int (*)(void *, struct Tcl_Interp *, int, struct Tcl_Obj *const *)'
pix.c:308: note: draw_pix_surface defined here

Tcl is calling draw_pix_surface() as if the objc parameter has type int rather than Tcl_Size. The C language considers this undefined behavior, and in practice it can cause program misbehavior.

draw_pix_surface() does not benefit from using Tcl_Size instead of int for objc, so it should use int so that it matches the Tcl_ObjCmdProc function type. There would then be no need for the (Tcl_ObjCmdProc *) cast in Pix_Init(), which the compiler can already warn about:

pix.c:408:13: warning: cast from 'int (*)(void *, Tcl_Interp *, Tcl_Size, Tcl_Obj *const *)' (aka 'int (*)(void *, struct Tcl_Interp *, long, struct Tcl_Obj *const *)') to 'Tcl_ObjCmdProc *' (aka 'int (*)(void *, struct Tcl_Interp *, int, struct Tcl_Obj *const *)') converts to incompatible function type [-Wcast-function-type-mismatch]
  408 |             (Tcl_ObjCmdProc *)draw_pix_surface,
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If using Tcl_Size for objc were desirable, then the function should match the type Tcl_ObjCmdProc2 and instead be passed to Tcl_CreateObjCommand2().

Although, if just the (Tcl_ObjCmdProc *) cast is removed, then using Tcl_Size for objc will not be treated as a mistake by Tcl 9.1. TIP 626 introduced a behavior (not documented in the manual) where, if using a compiler with C11 _Generic support, then passing a function matching the Tcl_ObjCmdProc2 type to Tcl_CreateObjCommand() automatically causes the program to call Tcl_CreateObjCommand2() instead.

Also, draw_pix_surface() ideally should be declared static.

Regarding an API to get the CGContext of a drawable, this is already possible for pixmaps using Tk_MacOSXGetCGContextForDrawable(). Extending that function to work for toplevels may not be difficult, but I do not know if doing so is a good idea, at least without it being clear that whatever uses it will understand things like the context's lifetime, already-applied transforms, Retina dimensions/scaling, etc.

And in case I should restate my earlier comment a bit differently: please don't check the value returned by XCopyArea(). Pretend that it returns junk.


marc_culler (claiming to be Marc Culler) added on 2026-05-20 12:35:42:
I doubt that this answers your question, but the macOS port supports
a mac-specific image type called nsimage:

https://www.tcl-lang.org/man/tcl9.1/TkCmd/nsimage.html

Also, I don't think it is possible, with current implementations, to get
a performance benefit from having a "backing store" (which I take to mean
a framebuffer in GPU memory) when you are using an XPixmap.  Those always
store pixel data in CPU memory. I don't know the extent to which CGImages
take advantage of the GPU.  Their implementation is proprietary, of course.

anonymous added on 2026-05-20 08:38:25:

Thank you for the detailed explanation. I understand now why XCopyArea fails with pixmap drawables on macOS.

As a temporary workaround, I replaced XCopyArea with a direct XPutImage call to the Tk drawable, bypassing the intermediate pixmap entirely. This avoids the BadDrawable error and works correctly for now, though I am aware it does not handle Retina displays and lacks the performance benefit of a backing store.

Before going further, I wanted to ask: is there currently a public Tk API that exposes a CGContext or allows rendering a CGImage to a Tk drawable on macOS? Or is that path only accessible through internal functions like TkMacOSXSetupDrawingContext?


marc_culler (claiming to be Marc Culler) added on 2026-05-20 04:20:00:
My apologies.  My statement that XCopyArea should not return BadDrawable
was just wrong. Or, maybe I should say that, while it *shouldn't* return
BadDrawable, it currently *does*, if you pass it drawables that are
pixmaps rather than windows.

I used the current tip of the core-9-0-branch of Tk to test the Tk
extension generated by your pix.c, which actually did work without
change.  The result of this debugging effort was that I found that
XCopyArea is returning BadDrawable because it computes the size of
the destination to be 0x0.  And that is happening because XCopyArea
does not support copying between pixmaps on macOS.  It assumes that
both the source and the destination drawables are windows, and it
works with the bounding rectangles of the contentViews of those source
and destination windows.

The comment:   // XXXX Need to deal with pixmaps! 

was actually serious.  XCopyArea *does* need to deal with pixmaps
and at the current moment it *does not*.

Making XCopyArea work with window drawables was the key ingredient which
allowed Tk 9.0 to stop having to call the deprecated method
[NSView ScrollRect: by:].  That was replaced by XCopyArea.  And a key step
there was figuring out how to deal correctly with Retina displays.  So
a fair amount of work went into fixing up XCopyArea, but that work did
not include extending it to accept pixmap drawables again.

Sorry that I did not remember all of those details when I said that it
should not return BadDrawable.

For your extension, It looks to me like the step Claude refers to as:
// 4. Fast blit from backing-store to Tk drawable
really means copying the pixmap image to a window. That is the job of a
display proc.  And in some contexts, such as Wayland, that can actually
be done by blitting.

The way to do that would probably be to create a CGImage from the pixel
data in the pixmap and then use the same tools that are already used in
the macOS port for drawing CGImages to windows to render the CGImage.  I
expect that some care would be required in order to deal with Retina
displays.

Claude's incorrect assumption that XCopyArea could do that job does
serve as a demonstration that extending XCopyArea to handle at least one
more of the four cases (namely the pixmap to window case) would probably
enable your code to work. So that would be a good project.

anonymous added on 2026-05-19 19:35:19:

Hello Marc,

Please find attached my C code, translated from my Nim code. As I mentioned, I don’t know C, so the AI’s translation may be incorrect. In any case, this code clearly shows that my code works in versions 8.6 and 9.0.1, but not in version 9.0.3.

load lib9pix.dylib Pix
image create pix myimg
pix::drawPixSurface myimg 200 150
label .l -image myimg
pack .l

You should see an image in a window similar to the one attached (image captured with version 9.0.1)

Below are the command lines I use to compile my library:

gcc -shared -o lib9pix.dylib  -fPIC pix.c -I/usr/local/Cellar/tcl-tk/9.0.3/include/tcl-tk    -L/usr/local/Cellar/tcl-tk/9.0.3/lib    -DUSE_TCL_STUBS -DUSE_TK_STUBS -DMAC_OSX_TK=1 -ltclstub -ltkstub
gcc -shared -o lib86pix.dylib -fPIC pix.c -I/usr/local/Cellar/tcl-tk@8/8.6.16/include/tcl-tk -L/usr/local/Cellar/tcl-tk@8/8.6.16/lib -DUSE_TCL_STUBS -DUSE_TK_STUBS -DMAC_OSX_TK=1 -ltclstub -ltkstub


anonymous added on 2026-05-18 06:46:17:

I’m trying to translate my Nim code into C, but I’m not sure how it will turn out as I know nothing about C. I think the AI might be able to help me.


marc_culler (claiming to be Marc Culler) added on 2026-05-14 20:12:39:
I don't think XCopyArea should produce a BadDrawable result, although as
Christopher says it is not clear what that value means, exactly.  The more
important question is whether the image gets copied.

Could you provide some code demonstrating the failure you are seeing?

anonymous added on 2026-05-12 20:55:28:

Marc,
Sorry for the delay in responding; I’ve had other projects on my mind for a while.
I’m using version 9.0.3 now, and XCopyarea no longer works (it throws a “BadDrawable” error every time). I’ve given up on using it on macOS, so I switched to XPutImage link.
I can’t compare the performance since XCopyarea was slow to begin with, but It is slow again with this command.
In any case, thank you for taking the time to reply.
I think you should add a note somewhere stating that on macOS, using XCopyarea is not recommended. (This may already be the case.)


anonymous added on 2025-08-31 15:14:41:

I corresponded with Marc a few weeks ago by private email, but I haven't had time yet to try out what he asked me to do.
I'll take the liberty of including a part of the conversation here :
Marc wrote :

For example, the arguments src and dst passed to XCopuArea are of type "drawable". In the XCopyArea function body you have:

    MacDrawable *srcDraw = (MacDrawable *)src;
    MacDrawable *dstDraw = (MacDrawable *)dst;
MacDrawable is a struct with a flags field which can be used to decide whether the struct represents a pixmap or a window. One of the flags is TK_IS_PIXMAP. If the drawable represents a window then it is possible to extract from the MacDrawable a pointer to an NSView object, which has an associated graphics context that can be used to draw into the window. If the drawable represents a pixmap then the associated MacDrawable can be used to get a CGImageRef, which is a pointer to a Core Graphics Image. The CGImage also has an associated graphics context which can be used to draw into the CGImage. There are utility functions to convert an XImage to a CGImage and vice versa. There is a tricky point when drawing an XImage to a window, which is that the window might be on a Retina display. In that case the image has to be resampled, which is an expensive operation.

For my part, I tried to improve my binding, to get as close as possible to C functions.
I see from your various comments that you cannot find the correct times, even with the example in C that was reworked by Marc.
I don't know if we should keep focusing on this. XCopyArea might do the job.


chrstphrchvz added on 2025-08-29 12:15:42:

A note for the reporter: be aware that XPutImage(), XCopyArea(), and many related functions do not actually have documented return values. Had these functions not predated C89, they probably would have been defined as returning void rather than int. Consider it unintentional behavior that on Tk Aqua and Win32, some of these functions currently return values such as BadDrawable rather than generate fatal errors as actual X11 would.


chrstphrchvz added on 2025-08-08 18:00:22:

I cobbled together some version of the reporter’s example code. XCopyArea() works again after applying the alternative fix I propose in [f215a9fdaa] but is indeed slightly slower than just using XPutImage() as Marc suggested. I only obtain comparable "slow" performance only on a Retina display; for a non-Retina display it is much faster, about 2ms per iteration. I can use the process sampler in Activity Monitor and see where most of the time is spent inside various internal Core Graphics routines, but I am not certain of their meaning, so my guess is to agree that a resampling operation is what slows down the Retina case. Using CGContextSetInterpolationQuality(…, kCGInterpolationNone) in resetTkLayerBitmapContext seems to speed things up slightly, and to me does not noticeably reduce image quality.

Another thought: the reporter’s animation does not use Retina-aware drawing, so if there is no use for other Tk widgets, or if it is okay to only have non-Retina widgets (buttons, text, etc.) alongside the animation, then I would see if setting NSHighResolutionCapable to false in Wish-Info.plist avoids the resampling overhead.


marc_culler (claiming to be Marc Culler) added on 2025-07-29 12:14:45:
Wow.  Yes I did.  Thank you!

chrstphrchvz added on 2025-07-29 11:34:16:

In [bfebc1f402]:

    // XXXX Need to deal with pixmaps!
    MacDrawable *srcDraw = (MacDrawable *)dst;

Marc, I think you meant to put (MacDrawable *)src here.


marc_culler (claiming to be Marc Culler) added on 2025-07-29 01:05:02:
> Tk Aqua 9 XCopyArea() currently returns BadDrawable for pixmaps
> because it is treating pixmap width and height as zero

I see what you mean.  But it is a pretty roundabout code path that
ultimately leads to that check.  First there is an attempt to get
an NSView for a pixmap which, ignoring the TK_IS-PIXMAP flag, is
done by calling TkMacOSXGetNSViewForDrawable, which return nil of
course.  Then we rely on the obscure fact that the bounds method
applied to a nil NSView returns a rectangle of size 0.

I think we could afford to check a few cases just for the sake of
code readability -- never mind the efficiency cost of making
unnecessary function calls.

Perhaps the "pix" image type would provide a good non-core example.
I can't tell at this point.

I definitely support being careful about changing XCopyArea.

chrstphrchvz added on 2025-07-28 23:00:05:

Tk Aqua 9 XCopyArea() currently returns BadDrawable for pixmaps because it is treating pixmap width and height as zero. I would like to review what happened regarding [df4b603950] first before I can say whether I agree with changes proposed for this ticket.

TkTable only uses XCopyArea() when -drawmode slow is specified. But TkTable is admittedly an obsolete example now that it can instead use clipping regions and avoid fuzzy text on Retina displays. I have not thought of a better non-core example of something that uses XCopyArea(), though.


marc_culler (claiming to be Marc Culler) added on 2025-07-26 22:18:02:
Regarding [685ac30727] - I installed TkTable and ran demos/basic.tcl.  I
was not able to make any overflowing text appear in any other cells.

I am not sure I was doing the right thing in that demo, but I think it
still works as expected with the change to XCopyArea i the pixmap-to-pixmap
case.

I was not able to move the cursor in a cell with the arrow keys, as they
move the focus to a different cell.  But I am assuming that is the
expected behavior.

marc_culler (claiming to be Marc Culler) added on 2025-07-26 21:34:42:
Thank you Christopher.  I was actually quite worried about that.
However, the changes I made in XCopyArea only affect the case where
both source and destination are pixmaps.  It seemed to me that, in that
case, scaling could not possibly be an issue so one could simply extract
the source rectangle from the src drawable as a CGImage and then draw that
CGImage into the context of the dst drawable.  Do you agree?

Of course there are still some weird things.  For example, if you take a
screenshot of a rectangle on a Retina display and then display the resulting
png image in a Tk label, the dimensions of the label image will be twice
as large as the dimentions of the rectangle used for the screenshot.
Apple treats each 0.5x0.5 subpixel as a pixel when it generates the png
file.  But I don't think that Tk can do anything about that.  It has no
knowledge of the source of a .png file, and must display 1 png pixel as a
square with sides of length 1 Apple point on an Apple screen, whether or
not an Apple point corresponds to the width of a screen pixel or to twice
the width of a screen pixel for that particular screen.

I will try to check that this change does not cause a regression of
[685ac30727].

chrstphrchvz added on 2025-07-26 20:57:01:

This ticket is interesting, but I am too far behind and too slow to be of much help. I do want to offer a reminder, however: the emulated X11 APIs in the Tk Win32 and Aqua ports are not just for core Tk, but also for extensions such as TkTable. And the CGImage/non-drawRect implementation was partly motivated by finally getting APIs like XCopyArea() and XGetImage() to work as expected (e.g. [685ac30727]).


marc_culler (claiming to be Marc Culler) added on 2025-07-26 20:41:38:
I did figure out a way to at least partially test the speed of the
new XCopyArea when called by TkImgPhotoInstanceSetSize.  I am seeing
times comparable to what you were getting - somewhere around .001ms.
So I think this worked.  But it would be very helpful if you could
repeat your testcode.tcl test with the bugfix branch.

marc_culler (claiming to be Marc Culler) added on 2025-07-26 18:57:31:
OK, I have created a bugfix branch bug-4af5ca1921 which modifies
XCopyArea in the case when source and destination are both pixmaps.
That is the situation when TkImgPhotoInstanceSetSize calls it.  It
is simpler because rescaling, for example to account for Retina displays,
is never needed.

With luck this will make your timing comparison better.  Please test
it and report the results, since I am not able to test this myself.
Thanks.

marc_culler (claiming to be Marc Culler) added on 2025-07-26 16:36:09:
I am beginning to get an inkling of what this ticket is about.

First, there are two calls to XCopyArea in tkImgPhInstance.c, one of which
never arises for macOS.  The other call is in TkImgPhotoInstanceSetSize.  It
is called to copy the part of the old image which is also inside of the
new resized image.  So XCopyArea is being used to copy from a pixmap to a
pixmap.

That case of XCopyArea was not really ever implemented in 8.0, as evidenced
by the cryptic comment:
// XXXX Need to deal with pixmaps!

The 9.0 XCopyArea does do something in that case, but it is very roundabout.
(I believe it hardly ever matters because in 99% of the applications the
size change is 1x1 -> widthxheight and only happens the first time the
image is drawn.)

I have another theory, which you sort of hinted at.  That theory is that
you are writing your own image type and you are basing your code on
tkPhImgPhInstance.c.  So you copied the Resize function, which contains
the call to XCopyArea, and you are measuring the time for that call
in your own image implementation.

I also suspect that your code is only calling XCopyArea in the case where
the image is being resized from 1x1 to wxh, so the call to XCopyArea is
only changing 1 pixel, and even if it is not actually doing anything at all
in 8.6 it still would not cause any trouble for your code.

Here is my plan.  I am going to fix XCopyArea so it handles the pixmap to
pixmap case efficiently, and not worry about the pixmap to window and window
to pixmap cases for now.  After I do that, I predict that your timing issues
will be gone.  Then I can worry about the other two cases, (or leave them
in their current inefficiant state until someone actually uses them. :^} )

marc_culler (claiming to be Marc Culler) added on 2025-07-26 14:59:11:
BREAKTHROUGH!

To check that I was correct about XCopyArea never being called on macOS
I put a print statement at the start of XCopyArea and ran the widget demo,
trying every demo.  To my shock and surprise, there was one demo which
produced a call to XCopyArea - namely "simple user interface for viewing
images.  With a bit of hacking I tracked it down to line 839 in tkImgPhotoInstance.c, which is part of TkImgPhotoInstanceSetSize.

This seems completely crazy, and may account for what you are seeing.

I am investigating.  Stay tuned ...

marc_culler (claiming to be Marc Culler) added on 2025-07-26 14:08:03:
Here is the thing:  Tk *never* calls XCopyArea (except when scrolling
a Text widget in Tk 9.0 only).

The calls to XCopyArea from generic code are not used by macOS.  They
all look like:
#ifndef TK_NO_DOUBLE_BUFFERING
    [code that uses XCopyArea for linux and windows]
#else
    [macOS code which does not use XCopyArea]
#endif

The only calls to XCopyArea in the 8.6 macOS code are in XCopyPlane
and XCopyPlane is only used for drawing buttons with monochrome bitmap
images.

In 9.0 there is one additional place where XCopyArea is used.  That is for
scrolling a Text widget. In 8.6 a deprecated method [NSView scrollRect: by:]
is used for scrolling; that has been replaced with XCopyArea in 9.0.  XCopyArea
could not be used for this in 8.6 because the 8.6 implemenation of XCopyArea
did not do a copy - it redrew part of the window in a pixmap.  (That didn't
matter for Tk since Tk never uses XCopyArea, but it was a potential problem
for extensions.)

So you are measuring times for something that never happens, and trying to
draw a conclusion from it.  The fact that you get short times in 8.6 and
longer times in 9.0 suggests to me that you are only seeing some misleading
artifacts from the way you are trying to measure the time spent in XCopyArea.

anonymous added on 2025-07-26 11:55:52:

While I still don't understand what "it" refers

The "it" referred to XPutImage or XCopyArea, but that was an assumption, which, based on your comments, seems to be incorrect.

I looked at it, but I don't know how to interpret what I am seeing. If you could provide a little explanation and guidance it would be much appreciated.

The first window that appears corresponds to the code launched with Tclsh86.
The second window that appears corresponds to the code launched with Tclsh90.

Here is my Tcl code below :

proc anim {img1 img2} {
    # Do Something...
    
   # Updates my 2 images
    pix::surfXUpdate $img1
    pix::surfXUpdate $img2

   after 16 [list anim $img1 $img2]

}

package require pix

set w 800
set h 400

# Create my context
set ctx [pix::ctx::new [list $w $h] "rgba(255, 255, 255, 255)"]

# Creating my two images using the Tk_CreateImageType procedure
set img1  [image create pix -data $ctx]
set img2 [image create pix -data $ctx]

# Pack
label .l1  -image $img1 -borderwidth 0 ; pack .l1
label .l2 -image $img2 -borderwidth 0 ; pack .l2

update
# Event loop
after 100 anim

What you see displayed on the terminal corresponds to the calculation time for the XCopyArea function.

tclsh86 testnim.tcl

XCopyArea: 0.008999999999981245ms
XCopyArea: 0.004000000000004ms
XCopyArea: 0.1049999999999662ms
XCopyArea: 0.06699999999998374ms
XCopyArea: 0.006999999999979245ms
XCopyArea: 0.01199999999995649ms
XCopyArea: 0.1129999999999742ms
XCopyArea: 0.04500000000001725ms
XCopyArea: 0.014000000000014ms
XCopyArea: 0.01000000000001ms
XCopyArea: 0.05800000000000249ms
...
tclsh90 testnim.tcl
XCopyArea: 15.05099999999998ms
XCopyArea: 15.21800000000001ms
XCopyArea: 15.63799999999999ms
XCopyArea: 15.71800000000001ms
XCopyArea: 15.57900000000001ms
XCopyArea: 15.57399999999998ms
XCopyArea: 15.83500000000004ms
XCopyArea: 15.64699999999997ms
XCopyArea: 15.76899999999992ms
XCopyArea: 15.53599999999999ms
XCopyArea: 15.75000000000004ms
XCopyArea: 15.68799999999992ms
...

I don't know if this answers all your questions, but it's all I can provide. (Perhaps the Nim code, but I'm not sure that would be of much help.).
Thanks


marc_culler (claiming to be Marc Culler) added on 2025-07-26 02:58:22:
> At first, I thought it might be due to epoll/kqueue **TIP458**

While I still don't understand what "it" refers to, I did look up
TIP #458 and I agree that whether epoll is used in the notifier
is totally unrelated to the time required for updating graphics.
The graphics updates do not involve the notifier at all. No events
are being processed.  No files are being read or written.  And,
even if the notifier were involved, using epoll should only improve
things.

One thing that is happening in CGContextDrawImage, on my laptop anyway,
is that the entire image must be resampled in order to account for
the Retina display.

marc_culler (claiming to be Marc Culler) added on 2025-07-26 01:47:28:
Thank you for the GIF.  I looked at it, but I don't know how to interpret
what I am seeing.  If you could provide a little explanation and guidance
it would be much appreciated.

marc_culler (claiming to be Marc Culler) added on 2025-07-25 22:32:36:
I don't think this is a limitation of Tk 9.  I think it is a limitation
of Apple's CoreGraphics.  Both Tk 8 and Tk 9 use CGContextDrawImage and
that is where the time is being spent in the testmage command.

However there are still many aspects of what you are saying that I do not
understand or that are incomplete.

1.  You have never said what "simulation code nim" means.  So I do not know
what you are comparing against the testimage code.  It appears on the
surface that you may be comparing apples and oranges.

2. The testimage code that I attached will compile as an 8.6 extension,
but it will not update the screen under 8.6.  I know the reason from my
instrumentation: the calls to XPutImage are being made outside of drawRect.
This is a serious limitation of 8.6.  Drawing operations often have to be
repeated because they are attempted outside of drawRect, which (after
macOS 10.14) means that it is impossible to obtain a valid CGContextRef
because the window's contentView is not the focusView.  You see this in
practice as a return value of BadDrawable, as you reported.  The testimage
code has no mechanism for calling XPutImage a second time if the first
attempt fails due to the contentView not being the focusView.  So it
couldn't possibly work with Tk 8.

3. A consequence of (2) is that whatever code you may be using to compare
Tk 8 against Tk 9 (and you still haven't said what that is) it can't
possibly work the same way as the testimage code.  Maybe that just means
that the testimage command is not testing the right thing.

In any case, if there is any way for me to see a comparison of working
Tk 8 code versus working Tk 9 code where the Tk 8 code is significantly
faster, then I would be very interested and it would be very helpful to
me.  But so far I haven't seen anything like that.

anonymous added on 2025-07-25 18:07:36:

Unfortunately for me, even though the focus is on version 9, 8.6 allowed me to compare (I wouldn't have opened a ticket otherwise).

At first, I thought it might be due to epoll/kqueue TIP458 (I don't know if Tk/X11 uses it) but now if you tell me that it's a limitation of Tk on macOS.

Here is a GIF link that shows the difference with my Nim program.

In any case, thank you for considering my request.I consider this to be a limitation on macOS version 9.
I would have liked to give a real example in C with the creation of an image, but I am limited in that way too.

Thank you for your detailed feedback.


marc_culler (claiming to be Marc Culler) added on 2025-07-25 04:06:50:
By the way, the actual bottleneck as far as frame rate is concerned is
probably the frequency with which updateLayer is called.  The job of
updateLayer is to copy the window's backing CGImage to the screen.  Drawing
is done in that CGImage, but the drawing does not appear on the screen
until updateLayer is called.

Tk is not allowed to call updateLayer.  All Tk can do is to set a flag
indicating that an update is needed.  The Aqua window manager calls
uodateLayer if the flag is set and if the time since the last update is
long enough.  Apple is responsible for deciding what the maximum update
frequency should be in order to provide smooth animations.

When I run the testimage command, updateLayer is not called until close
to the last iteration, even though the flag is set on each iteration.
This would not be reflected in your timing results, since they are
checking how long the drawing operation takes, not how long it takes
until the screen is updated.

marc_culler (claiming to be Marc Culler) added on 2025-07-25 01:50:18:
I have done a lot of timing tests inside of the Tk drawing routines
and checked times when running my version of your program, which does
one call to XPutImage per iteration. What I learned is that for an 800x400
image like yours a single call to CGContextDrawImage takes between 5 and 7
milliseconds.  That means that almost all of the time spent drawing in one
iteration of the test program is accounted for by the one call to
CGContextDrawImage that happens during each iteration.

There is no alternative to CGContextDrawImage for drawing an image on
the screen.  This has nothing to do with Tk versions or even with Tk.
It is just what Quartz provides for drawing images on the screen.
So I think there is nothint that could be done to speed up the drawing
in Tk, for any version of Tk.  (Other than getting rid of drawRect, which
forces drawing operations to be repeated multiple times -- and that
is what Tk 9 has already done.)

One thing this means is that if you are trying to do an animation by
drawing into an XImage and copying the Ximage to the screen, then you are
limited to somewhere between 160 and 200 frames per second.  But smooth
animation does not require anything close to that rate.  So I don't
really think that this presents an obstacle to producing smooth
animations with Tk.  If the other timings that you reported were times
for updating the XImage itself, then you should be in good shape.  The
bottleneck will be CGContextDrawImage and the part that matters,
updating your image, will have plenty of time to do its magic without
lowering the frame rate significantly.

The job of CGContextDrawImage is fairly complicated.  If the image is
not the same size as the rectangle it is being drawn into, the image will
be scaled to fit the target rectangle.  If the screen is a Retina screen
the image will be resampled.  It is not totally surprising that this
takes some time.

marc_culler (claiming to be Marc Culler) added on 2025-07-24 22:13:57:
OK, I have a version of your code working.  It does not use XCopyArea
at all.  For each iteration it calls XPutImage once to copy the static
XImage to the screen.  And I see nice triangles containing gradients
tiling the window.

I am seeing comparable times to what you report.  The average is 6.5 ms.
The factor of 3 is probably CPU speed.  (I am using an M3 Macbook Air.)

So the next question would be "How fast would we expect this operation to be?"  Your second list of times is for "simulation code nim".  But I don't
know what that means.  What is being measured by the times in your second
table?

marc_culler (claiming to be Marc Culler) added on 2025-07-24 20:46:40:
When I run the code above with Wish9.0 the calls to XCopyArea all fail
with status 9 (i.e. BadDrawable).  You indicated that you only saw that
with 8.6.

Your code should print out the status for each iteration, even if it is 0
since it contains no test for whether the status is 0.  Yet, your printout
does not contain the status lines.

Also when I built the extension there were two unused variables: photo
and block.

So I am guessing that  the posted code is incomplete.  Do you know what
might be missing?

marc_culler (claiming to be Marc Culler) added on 2025-07-24 19:27:20:
OK, that is interesting.  I will see if I can build your extension and
run this myself.

Comparison with 8.6 would not be useful.  There is no way that Tk 9 can
or should go back to using drawRect.  And 8.6 is obsolete, with releases
ending next year.

We need to focus on Tk 9.

anonymous added on 2025-07-24 18:22:06:

What's interesting is that if I only use XPutImage as you suggested in the C code I provided in my ticket, it's now this function that's slow. Unfortunately, I can't get it to work in 8.6 to compare.


anonymous added on 2025-07-24 16:52:26:

Did you try using XPutImage?

In my implementation, XPutImage and XCopyArea are interconnected, and one cannot function without the other.
Why ?

  1. I use XPutImage to fill a pixmap (server-side buffer) with my image data. This pixmap serves as a cache to avoid retransferring data each time it is displayed.
  2. I use XCopyArea to copy from the pixmap to the window. It is this step that makes the image visible to the user.

Why this dependency on each other?

Without XPutImage: The pixmap remains empty, XCopyArea does not copy anything → black screen.
Without XCopyArea: The image is in the pixmap but is never transferred to the window → black screen.

XPutImage is only called when the image is modified.
XCopyArea is fast and called at each redraw/exposure.
I avoid retransferring image data at each refresh.

That's how I experienced it, I'm not sure if it's the best way. In 8.6, I can get super smooth image animation, which is not the case in 9, so I'm submitting this ticket.


marc_culler (claiming to be Marc Culler) added on 2025-07-24 16:31:58:
Just to clarify my question ... I was trying to ask whether you
tried using XPutImage (or TkPutImage, which happens to be the same
function on macOS) to copy your image to the screen, as opposed to
copying the image to a pixmap.  TkPutImage accepts a window as its
target drawable.  (I imagine that your pixie drawing code is rendering
to something easily convertible to a pixmap and that your goal is to
display that pixmap on the screen.)

marc_culler (claiming to be Marc Culler) added on 2025-07-24 14:00:07:
It sounds like you have an XImage which you would like to copy into a
Tk Widget.  Did you try using XPutImage?

Your project seems similar to the tkpath package, which uses Cairo to draw
into a Tk widget which behaves like a Tk Canvas.  That package ends up
replicating most of the Tk Canvas code (and in particular does not use
XCopyArea on macOS - instead it draws directly into the graphics context of
the toplevel's NSView).  It might be worth looking at their code.

anonymous added on 2025-07-24 07:09:58:

I don't fully understand what you are doing here and will need more explanation. But here is some background.

I create my own image with Tk_CreateImageType.

Can you post a url for your project?

pix

Although the version of my X11 binding is not up to date on GitHub, it gives you an idea of what I want to do.
Here is my link: X11

As I said, I am not able to do this in C, and I tried to create an example in C to get as close as possible to what I do in Nim. The C code example I provided does not work for me on version 8.6.16 on macOS. However, it does work on version 9. I hope you can see that XCopyArea takes time.


marc_culler (claiming to be Marc Culler) added on 2025-07-23 17:34:08:
I don't fully understand what you are doing here and will need more
explanation.  But here is some background.

XCopyArea is heavily used by Tk in the linux and windows ports to copy a
region out of a window, which is then modified and copied back into the
window.  This is possible because on those platforms it is possible to
directly read and write from the window backing store (using the system's
XCopyArea on linux.). However, XCopyArea is never used by the macOS port.
On macOS the modifications are done using drawing functions provided by the
system.  If you look in the code, you will see blocks like
#ifndef TK_NO_DOUBLE_BUFFERING
  ... code which uses XCopyArea ...
#else
  ... macOS specific code
#endif

In Tk 8, which used Apples [NSView drawRect] function, it was not
possible after macOS 10.14 to directly access the backing store of a
window.  The XCopyArea function did no support copying from a pixmap
(which is why you were getting those BadDrawable errors) and the
implementation of XCopyArea in the case of a window drawable was a hack.
It used a feature which allows the drawRect method to draw to a CGImage
instead of to a window.  The way that XCopyArea "worked" was to create
a CGImage, use drawRect to redraw the window, with drawing clipped to the
source rectangle, into the CGImage and then convert the CGImage into
an XPixmap.  There was no copying.  Everything was redrawn.

In Tk 9, [NSView drawRect] is no longer used.  Instead [NSView updateLayer]
is used.  This means that the backing layer of the window is an actual
CGImage which Tk can modify directly.  Wnen an update of the screen is
needed that CGImage is blitted to the screen.  This allowed an
implementation of XCopyArea which worked with pixmaps as well.  However,
it involves a lot of converting between image formats, not to mention
dealing with retina vs non-retina displays.  I would not expect high
performance with XCopyArea on macOS, and I do not understand why you
would ever need to use it on macOS.

My guess is that you should be working with images of type nsimage but,
as I said, I don't understand yet what you are trying to do.

Can you post a url for your project?

Attachments: