| Ticket UUID: | 40dc19ee9e7a41d59237675f87d11e5303efbd1e | |||
| Title: | Regression: Canvas does not immediately display in Tk 9.1 on macOS | |||
| Type: | Bug | Version: | 9.1 | |
| Submitter: | tberg | Created on: | 2026-10-07 22:20:56 | |
| Subsystem: | 04. Canvas Basics | Assigned To: | nobody | |
| Priority: | 5 Medium | Severity: | Important | |
| Status: | Open | Last Modified: | 2026-10-27 21:35:48 | |
| Resolution: | None | Closed By: | nobody | |
| Closed on: | ||||
| Description: |
On macOS (version 26.6.1 and also 15.6.1 on arm64 here), I run this simple script with
Using wish9.0, it draws the black canvas and the blue rectangle immediately. Doing the same with wish9.1 draws noting, I just see the gray window background. Only when resizing the window, the canvas and rectangle are drawn. I see the same behaviour in Tk 9.04 and did not test earlier versions (9.0.3–9.0.1). Note, this is only failing when calling the script from a script file. When writing/pasing the code into a running wish console, everything works as expected. When calling a similar script with a ttk::button instead of the canvas, I see the button immediately. No regression here. Jan did some bisecting:
He says on the core list: This points to this commit: 63230229325efa47. Fix b713c15714: process idle tasks before displaying a TKContentView for the first time | |||
| User Comments: |
marc_culler (claiming to be Marc Culler) added on 2026-10-27 21:35:48:
Thanks for pointing out the collision of WM_HIDDEN. I will change that right away. marc_culler (claiming to be Marc Culler) added on 2026-10-27 21:34:34: I have not seen failures of event-9.2, but I think I may have seen windows appearing in a different order than the one in which they were created. This seemed to occur in sporadic failures in the wm-stackorder tests. (Either they appeared on screen in the wrong order or they failed to move front after being mapped.) What could cause that? I understand that mapping windows to the screen and ordering them is handled asynchronously by the WindowServer process. I heard that WindowServer is not guaranteed to respect requests made with the order(_:relativeTo:) command. Maybe it also does not guarantee that windows will be placed on screen in the same order that they are ordered front. I think that might explain a lot of the sporadic failures that we see. But it seems like a strange behavior. What would be the point of doing that? I have suspected, though, that the Tk test suite can overload the WindowServer process. chrstphrchvz added on 2026-10-26 12:24:46: I have not reviewed tip_765 very thoroughly. I do observe the re-enabled event-9.2 test sometimes failing on a real computer (albeit one older than what macOS 15 Sequoia officially supports). Normally, when event-9.2 starts, the root toplevel iconifies and .top1 appears immediately; after pausing, .top2 then appears at a position overlapping .top1. What I sometimes observe is that .top1 does not appear until after .top2 appears, and .top2 is at +50+50 where it does not overlap .top1.
In
Presumably this should have marc_culler (claiming to be Marc Culler) added on 2026-10-20 15:29:27: Unfortunately the test code does not produce visual artifact that are easy for me to see. It relies on the monitoring code in the dylib. (And, incidentally, to compile the dylib look in the source file - it contains a gcc command that can be adapted to build the library.) However, the visual artifacts are apparently clearly visibile with Nicolas' app. So I relied on his eye for the ultimate conclusion about whether the -hidden attribute solves the problem. marc_culler (claiming to be Marc Culler) added on 2026-10-20 15:23:07: Nicolas gave me permission to attach his testing code, so I will attach a tarball to this ticket containing the files he sent to me plus my modified version of the main script. marc_culler (claiming to be Marc Culler) added on 2026-10-20 14:30:51: One thing that I did when testing tip_765 was to add a print statement in updateLayer which reports the visiblity state of the window. That allows checking whether drawing is happening for off screen windows. marc_culler (claiming to be Marc Culler) added on 2026-10-20 14:17:15: Nicolas provided me with a script written by Claude which creates a window containing a canvas containing a specified number of items (6000 being a typical value but 60000 can be used as well). It withdraws the window, configures the canvas, then deiconifies it. It also loads a custon dynamic library containing code which monitors all calls to updateLayer (by overriding the method on load) and checks pixel colors to determine whether the window is ever visible before the canvas items have been drawn. Changing `wm withdraw .` to `wm attributes . -hidden 1` and `wm iconify .` to `wm attributes . -hidden 0` makes his script report that the window is never visible before the canvas is drawn with the tip_765 branch. That script reports that the window is visible before the canvas is drawn with the main branch. I could provide you with his zip file if you want. I am attaching a simple script which is adapted from one of the tests that I added to the tip_765 branch. chrstphrchvz added on 2026-10-17 03:15:31: Marc, is there a script available for others to try with tip_765 which uses marc_culler (claiming to be Marc Culler) added on 2026-10-16 20:46:40: Too complex. The tip_765 branch fixes this problem as well as Nicolas' problem and some other stuff too, like the failures of tests event-9.11 - event-9.20 and the zombie windows that appear when running the tests. The fix for this problem is just that it removes the event loop from the updateLayer method, which was only there to deal with the artifacts when a window is first shown. With tip_765 the window can be fully configured and drawn while off screen and then presented in a fully rendered state. So if we approve and merge that TIP implementation this will be fixed. tberg added on 2026-10-13 09:49:28: This is all really complex. I wonder how you can even grasp this ... very cool. But what is now the right solution? It seems to me the only fix working for all scenarios is the simple change to Regarding Marc's earlier remark:
As I understand the function marc_culler (claiming to be Marc Culler) added on 2026-10-11 04:08:14: Nicolas reports that the new bugfix branch does not solve the problem with his app, and that the reproducer written by Claude reports OK when things are not OK, but a close examination of the result shows that the script did detect the error but did not recognize it as an error. It sounds like the issue is that the drawing is not being done while the window is withdrawn, so it doesn't help to call updateLayer while the window is withdrawn. I don't see any reason why drawing to a CGImage would depend on whether a window was on or off the screen. But maybe the CGImage is not being created soon enough, or maybe its size is 1x1 while the drawing happens. Of course this does not explain why things work for less complicated windows. But maybe it is too hard to detect the artifacts when you only have a few widgets in the window. marc_culler (claiming to be Marc Culler) added on 2026-10-10 20:17:46: I have now abandoned the inner event loop completely, which should be a relief to all of us, and have created a new bugfix branch branch bug-40dc19ee9e-alt which takes a completely different approach. My free chatbot (Gemini) told me that updateLayer is never called for a View whose window is off scteen, except that there is a trick which will force it to be called on the view of an off screen window. The trick is to set needDisplay to YES and then immediately call displayIfNeeded. So I tried that. And it appears to work! I added those two calls before every call to orderFront. I do not yet know if this fixes Nicolas' app. But Nicolas kindly got Claude to write a reproduction script which simulates his app by withdrawing the root window, adding 6000 widgets and then deiconifying it. He sent the script and supporting files to me. The script loads a dylib which samples pixels on the screen and monitors calls to updateLayer and generates a report claiming to say whether the window is ever visible without having its widgets ddrawn. When I run that script with the new bugfix branch it reports that it is working correctly. Also, looking at the simulated root window being created I see no artifacts. Torsten's 2 line script works correctly, since the event loop is gone, as do the scripts from [f16cdb6d04]. All tests pass for that branch on macOS 27 except that there are sporadic failures of wm-stackorder-2.1, which has been happening for a long time. (Mysteriously, that test fails sporadically on real computers but passes on CI runners.) marc_culler (claiming to be Marc Culler) added on 2026-10-10 17:24:49: Nicolas reports that he sees the artifacts with the bugfix branch. Specifically, his root window appears on the screen at the correct size and then, after a short but noticeable delay, all the widgets appear at once. I think this means that the event loop and the idle task are useless. They are not needed to fix the issue in this ticket. But I am at a loss to explain why the previous incarnation of the event loop inside of updateLayer fixed his problem. All I can guess is that it somehow delayed the deiconization of the window until the drawing finished. But if that is the case then it seems like there should be a script level fix for his problem. Other questions that I don't know the answer to are whether drawing to the CGImage backing store is possible while a window is withdrawn and whether updateLayer is called for a withdrawn window. The fact that the previous attempt worked suggests that the answer to both is yes. But I am not sure about that. I can probably get the answers by experiment. chrstphrchvz added on 2026-10-10 16:53:47: TkWmDeadWindow() releases the dead NSWindow, which in turn can release its contentView (i.e. deadView, because the NSWindow holds a strong reference to it). If deferredUpdateLayer() is called afterward and view was deallocated, then trying to check windowIsDead can crash. If the idle event loop in deferrredUpdateLayer() handles drawing idle tasks, they will call TkMacOSXRestoreDrawingContext(), and so TkMacOSXUpdateViewIdleTask(), which sets setNeedsDisplay:YES, will also get called while in this loop. So doing setNeedsDisplay:YES from deferredUpdateLayer() seems redundant. And since deferredUpdateLayer() is an idle task, wouldn't it also be redundant if all it did was handle other idle events? I tried this change which leaves hasBeenDisplayed and deferredUpdateLayer() unused, and I observe that this ticket's issue is still resolved:
--- macosx/tkMacOSXWindowEvent.c
+++ macosx/tkMacOSXWindowEvent.c
@@ -1040,7 +1040,7 @@
* until after the toplevel has been configured. The configuration is
* done in an idle task, which then marks this view as needing display.
*/
- if (![self hasBeenDisplayed]) {
+ if (0 && ![self hasBeenDisplayed]) {
Tcl_DoWhenIdle(deferredUpdateLayer, self);
[self setHasBeenDisplayed:YES];
return;
Likewise I observe it is resolved with just this change for recent main branch:
--- macosx/tkMacOSXWindowEvent.c
+++ macosx/tkMacOSXWindowEvent.c
@@ -1025,7 +1025,7 @@
* display procs before updating the layer.
*/
- if (! [self onScreen]) {
+ if (0 && ! [self onScreen]) {
//printf("Running event loop.\n");
while(Tcl_DoOneEvent(TCL_IDLE_EVENTS)){}
[self setOnScreen:YES];
Of course, these observations are just for the current state of oddly placed event handling loops in Tk Aqua. I also tried the example in [b41fc99e] with these changes and don't notice a regression, but I agree maybe Nicolas should still try whatever change gets proposed here to make sure artifacts aren't reintroduced in his app. marc_culler (claiming to be Marc Culler) added on 2026-10-10 15:01:27: I think the guard in deferredUpdateLayer is now correct. I added a boolean property windowIsDead to the TKContentView class and I set that to YES in tkWmDeadWindow. I check the property in deferredUpdateLayer. I think this fix is now ready to merge, but I am going to ask Nicolas Bats to test against his app. His report of artifacts when the main window of his app is withdrawn, configured, and deiconified was the original reason for adding the event loop inside updateLayer. Thank you all for your comments and if you have any more, please share them. marc_culler (claiming to be Marc Culler) added on 2026-10-10 13:52:45: > I think the danger in deferredUpdateLayer() is not that (TKContentView > *)clientData might be nil (self is never nil during updateLayer, right?) Right. Thanks. I think I know how to deal with that. I will take another stab at it. chrstphrchvz added on 2026-10-10 08:35:56: The approach in [6964ffb4] does seem better, although I do not fully understand why it is needed. I think the danger in deferredUpdateLayer() is not that (TKContentView *)clientData might be nil (self is never nil during updateLayer, right?), but rather that it might already be deallocated by the time deferredUpdateLayer() is called. I think Tcl_CancelIdleCall(deferredUpdateLayer, deadView) would need to be done from TkWmDeadWindow(), but deferredUpdateLayer() is currently static to tkMacOSXWindowEvent.c. I think the expectation to avoid doing a lot of work in updateLayer is indeed meant not in terms of minimizing instruction count/cycles, but rather keeping it optimized enough so that there is no problem with it getting called 60 or however many times per second per view. This includes avoiding expensive drawing work which should already be performed elsewhere (such as in background threads, although a single-threaded program like Tk is not designed to use those), and especially any potentially unbounded tasks like event handling. jan.nijtmans added on 2026-10-09 21:39:10: So, it looks like we are getting somewhere. Great! marc_culler (claiming to be Marc Culler) added on 2026-10-09 21:12:55: I have just pushed a different solution to this problem to the bugfix branch. It addresses Christopher's concern about doing too much work in updateLayer. There were two flaws in my argument that it is OK to process idle tasks in updateLayer because it only happens once for each toplevel. The first is that it was not true that it only happens once. It was also happening when a window is deiconified. The second flaw is that the idle tasks are arbitrary. Whatever is on the idle queue gets processed, and those tasks could, in principle, take arbitrarily long. So Tk might freeze in updateLayer. I also address the issue that we are never supposed to call updateLayer - only AppKit is allowed to do that. The way it works now is that updateLayer checks if the contentView has never been displayed before. If that is true it posts an idle task and returns immediately. Otherwise it does the normal context swap. The idle task runs the inner event loop which used to be in updateLayer and then sets the flag NSView.needsDisplay, which should cause AppKit to call updateLayer at the next opportunity. The Tcl Notifier is supposedly careful when processing idle tasks not to process any idle tasks which are generated by an idle task. And I don't see any signs of that being a problem. All tests pass and the three test scripts from this ticket and [f16cdb6d04] work correctly. Incidentally, while testing the branch I encountered a segfault in the test event-9.18, which has been consistently failing. The segfault may explain why. The crash was in the idle task, which I think means that that test destroys a window between the first call to updateLayer and the next iteration of the event loop. Destroying a window before it has been displayed on the screen would cause a test for whether the mouse pointer isin the window to fail. tberg added on 2026-10-09 19:04:08: Great, thanks for confirming. I wasn't totally sure here, so it is good to have it confirmed by another pair of independent eyes. I find this event code difficult to understand as it has so many facets. It took me quite some time to understand the relevant parts regarding this phenomenon. marc_culler (claiming to be Marc Culler) added on 2026-10-09 15:22:20: Torsten, I think your analysis is completely correct. I have removed the idle task hack and replaced it with your simple suggestion of not accessing the context until after the mini-event loop has run (in the unique case where it does get run, namely the first time that updateLayer is called for a new toplevel window.) This makes your script work correctly and it makes Christopher's two scripts from [f16cdb6d04] work correctly. I have edited the comments to try explain the situation, including the reason for the event loop, such as it is. I have not renamed the onScreen property yet, but I will. Please retest. marc_culler (claiming to be Marc Culler) added on 2026-10-09 14:47:01: Torsten, I think your suggestion makes sense. And it still satisfies
the condiition I mentioned in my previous comment:
The code:
if (! [self onScreen]) {
//printf("Running event loop.\n");
while(Tcl_DoOneEvent(TCL_IDLE_EVENTS)){}
[self setOnScreen:YES];
}
Runs exactly once for each toplevel.
I think I should rename "onScreen". That name is misleading since its
value does not indicate whether the window is currently on the screen,
but whether it has ever been on the screen before.
marc_culler (claiming to be Marc Culler) added on 2026-10-09 14:40:30: Christopher, without claiming that the idle task is the correct solution, I just want to point out that I don't think that the current updateLayer, breaks your rule: updateLayer is not supposed to do a lot of work, so that there is no problem with it being called often. To me, this means that if queued Tcl events need to be handled to perform drawing, then they should be handled before, rather than during, the call to updateLayer. The reason is that the extra code runs only once in the life of a toplevel. For all calls after the first, the only step which is added is a test of the value of the onScreen property. That should add no time because a test which passes once and fails in all subsequent calls should be predictable by branch prediction. tberg added on 2026-10-09 07:11:59: Yes, that makes it work but I think what I wrote earlier also explains what you are saying. You said
I still think this is because of the
chrstphrchvz added on 2026-10-09 05:02:17: The clientData for
It is also important that But, my impression of how Apple intended
marc_culler (claiming to be Marc Culler) added on 2026-10-08 21:09:24: Tk only deals with user events - mouse, and keys. it does not use the AppKit event loop for that. It only repurposes the AppKit queue. Tcl uses the AppKit event loop to process file and socket events, which is independent of Tk. marc_culler (claiming to be Marc Culler) added on 2026-10-08 21:05:36: An update after looking at the code and doing some debugging.
First, I was wrong when I said that the nested event loop:
while(Tcl_DoOneEvent(TCL_IDLE_EVENTS)){}
is run once per Tk instance. It get run once in the lifetime of a toplevel
window, on the first call to updateLayer for that window, before the layer's
tkLayerBitMapContext is displayed for the first time.
This is what the debugging shows:
% wish9.1 torsten.tcl
ItemCreate
DisplayCanvas
Canvas .c is not mapped
updateLayer
Running nested event loop in updateLayer:
DisplayCanvas
DisplayCanvas
4 idle tasks were run.
So the canvas item is created before the first call to updateLayer and an
attempt to display the canvas is also made before that first call,
but is aborted because the canvas is not mapped yet. Then, in the first
call to updateLayer the nested event loop runs DisplayCanvas twice, along
with 2 other idle tasks. However, for some reason, those two calls to
DisplayCanvas do not seem to do any drawing. Neither the canvas background
nor the rectangle actually get drawn in the backing store CGImage.
This is the debug output when the call to update idletasks is added at the
end of the script:
% wish9.1 /tmp/torsten.tcl
ItemCreate
DisplayCanvas
Canvas .c is not mapped
DisplayCanvas
updateLayer
Running nested event loop in updateLayer:
DisplayCanvas
2 idle tasks were run.
With the call to update idletasks added, the second of the three calls to
DisplayCanvas now runs before the first call to updateLayer as does
one of the two mystery idle tasks. The second call to DisplayCanvas
does draw the canvas during the nested event loop.
And here is what happens when the nested event is removed and the
call to update idletasks is removed from the script:
% wish9.1 /tmp/torsten.tcl
ItemCreate
DisplayCanvas
Canvas .c is not mapped
updateLayer
updateLayer
DisplayCanvas
updateLayer
DisplayCanvas
updateLayer
So there are still three calls to DisplayCanvas. but there are 4 calls to
updateLayer and the second and third calls to DisplayCanvas are each
followed by a call to updateLayer.
Based on this analysys I have created a bugfix branch bug-40dc19ee9e
which appears to fix this issue as well as [f16cdb6d04].
What that branch does is to post an idle task to rerun updateLayer after
the first call to updateLayer. Please test!
tberg added on 2026-10-08 20:52:36: OK, thanks for the explanation! I am not too familiar with these details. I'm, however, still a bit puzzled. You say, Tk does not use the macOS event loop. In the next paragraph you say Tcl_DoOneEvent is using the macOS event loop. OK, that is a Tcl C API command and not a Tk command but still it is used by the Tk code. Anyway, let's assume it is really not a problem of the event loop as such. What is making some Tk drawing visible to the user? For macOS (and I believe this problem is restricted to macOS) it is updateLayer in tkMacOSXWindowEvent.c. The function remembers the drawing buffer in the Despite this, The IT worked in Tk 9.0.0 because there Does that make sense? marc_culler (claiming to be Marc Culler) added on 2026-10-08 17:41:52: Tk does not use the macOS event loop at all. It uses the AppKit event queue, however. The Tk checkproc removes all of the AppKit events from the AppKit queue and translates them to XEvents which are posted on the Tcl event queue. Tcl does not use the macOS event loop to process user events. It uses its own queue. However, as I learned recently when dealing with the macOS 27 menu issue, Tcl_DoOneEvent does create runloops, which means that it is using the macOS event loop. It creates event sources for two runloop modes (one being the default runloop mode and one being a special Tcl-only runloop mode used for all nested runloop runs.). Those event sources are for things like files and sockets. Tcl calls its runloops with an option which makes the call return as soon as one event from one of the sources is processed. In any case, I would not characterize this relationship as "not playing well together". They play well enough with each other. serhiy.storchaka added on 2026-10-08 15:39:52: See also [f16cdb6d04] (reported for 8.6.12): a listbox with a bad -yscrollcommand is not displayed and its error is not reported until an event or update idletasks arrives. It may have the same cause; a fix here should be checked with that script too. tberg added on 2026-10-08 15:14:19: I find it difficult to judge what the right "fix" is here. I agree that these nested event loop calls should never be needed. But what is the alternative? I think the users expect with such a script that the canvas will be visible right away. It is hard to explain why a user needs Personally, I do not want to enforce adding that loop here as well. I can easily just use When it is really just the canvas showing this phenomenon, I would say we can justify adding one more call to the event loop here. If other widgets are affected as well, we might try to make the effort to eliminate the source of the problem ... but as I recall, macOS's event loop and Tk's event loop do not play well together. I may be an impossible task. tberg added on 2026-10-08 15:04:37: Marc Culler wrote on the core mailing list 2026-10-08:
and later ...
| |||
Home
Timeline
Branches
Tags
Forum
Tickets
Download
Wiki