Tk Source Code

View Ticket
Login
Ticket UUID: 7ebdd179743bf25e8b4533287ea214a83cadc498
Title: Yet another idle_curiosity feedback ticket
Type: Bug Version:
Submitter: chrstphrchvz Created on: 2020-07-14 18:20:00
Subsystem: (unused) Assigned To: marc_culler
Priority: 5 Medium Severity: Minor
Status: Closed Last Modified: 2020-09-01 19:54:59
Resolution: None Closed By: marc_culler
    Closed on: 2020-09-01 19:54:59
Description:

I would think the original ticket [06d8246baf] would be the ideal place to discuss idle_curiosity, but it has become quite lengthy. Here is my feedback separately so as to avoid derailing others' feedback tickets—unless it really does not belong in its own ticket.

User Comments: marc_culler (claiming to be Marc Culler) added on 2020-09-01 19:54:59:
I think this ticket can be closed since the idle_curiosity branch was
merged quite a while ago.  That is not to say that there are no more
glitches related to the handling of idle tasks and interactions with
[NSView drawRect:].  I just think that the useful life of this ticket
is over.  Feel free to reopen the ticket if necessary.

marc_culler (claiming to be Marc Culler) added on 2020-07-16 22:41:20:
I applied the hack and tried to see what would happen if event loops were
added to process idle events before each call to nextEventMatchingMask or
CFRunloopRunInMode (called in Tcl_WaitForEvent).

That did seem to eliminate all of the warnings except when resizing a window.
It seems that SetFrame has some different sort of access to drawRect when
liveResize is active.

It also introduced two new regression test failures in main.test.  The error
was:

---- Test cleanup failed:
can't invoke "event" command: application has been destroyed
    while executing
"event generate $w <<ThemeChanged>>"
    (procedure "ttk::ThemeChanged" line 6)
    invoked from within
"ttk::ThemeChanged"

While that could probably be addressed, I also don't see any specific benefit
from adding the extra event loops.  I was not able to detect any difference in
behavior related to whether they are or are not present.

Maybe someone else will see something more revealing.

chrstphrchvz added on 2020-07-16 17:34:43:

Here's small hack for checking for idle events without processing any:

diff --git generic/tclTimer.c generic/tclTimer.c
index 106e2f75aa..4e20097bc1 100644
--- generic/tclTimer.c
+++ generic/tclTimer.c
@@ -719,6 +719,9 @@ TclServiceIdle(void)
     if (tsdPtr->idleList == NULL) {
        return 0;
     }
+    if (getenv("TCL_IDLE_CHECK")) {
+       return 1;
+    }

oldGeneration = tsdPtr->idleGeneration; tsdPtr->idleGeneration++;
diff --git macosx/tkMacOSXWindowEvent.c macosx/tkMacOSXWindowEvent.c
index 5f52ea742..7c29aadf1 100644
--- macosx/tkMacOSXWindowEvent.c
+++ macosx/tkMacOSXWindowEvent.c
@@ -919,6 +919,11 @@ ConfigureRestrictProc(
        fprintf(stderr, "drawRect: drawing %s in %s\n",
            Tk_PathName(winPtr), NSStringFromRect(rect).UTF8String);
     }
+    setenv("TCL_IDLE_CHECK", "1", 1);
+    if (Tcl_DoOneEvent(TCL_IDLE_EVENTS)) {
+       fprintf(stderr, "Warning: idle event(s) queued when entering in drawRect\n");
+    }
+    unsetenv("TCL_IDLE_CHECK");
 #endif

/*


chrstphrchvz added on 2020-07-16 15:28:41:

Marc, thanks once again for your patience. It could've been that in an attempt to not ramble for too long, I neglect to mention something important or phrase something unambiguously.

A quick Google search for "setNeedsDisplay:NO" out of curiosity indicates that such usage should be fine or at least possible. I simply wasn't sure before, as from watching what happens when stepping into setNeedsDisplay:YES from a debugger, I saw that this would then do setNeedsDisplayInRect:[self bounds]; so there was more than a simple assignment going on, but I did not pay attention to what would happen for setNeedsDisplay:NO.

The now-removed first idle event loop I noticed in generateExposeEvents: might resemble how I'd previously tried checking for or making sure any idle events queued when entering drawRect: were handled before Expose events are generated and handled. Maybe something similar is still useful for debugging, but I'm still figuring out what those events would do (i.e. try to draw, or get deferred; if the latter, whether this leads to drawRect: getting called again right away). I'm also still curious if there is/could be a way to "peek" at the idle event queue for already-queued events from Tk without actually handling them; this would be even better for debugging.

I've neglected to share here that the "Flash screen updates" feature in the Quartz Debug utility (from Additional Tools for Xcode) is useful for seeing when and where portions of a toplevel are redrawn. The "No delay after flash" is also useful; however I cannot use the "Flash identical screen updates" feature on the Mac I've been using (possibly because its graphics card isn't supported by Catalina).


nab added on 2020-07-16 14:48:57:
>Nicolas, does the current tip of idle_curiosity fix your issue?
yes. cool !
would you mind to do a merge in stylish_idle_curiosity?
thanks,
++

marc_culler (claiming to be Marc Culler) added on 2020-07-16 14:36:32:
Oh.

Now I get it.  Sorry for being so dense, and thanks Christopher for explaining
again.

I pushed a new commit which, I believe, handles the test correctly.

Nicolas, does the current tip of idle_curiosity fix your issue?

chrstphrchvz added on 2020-07-16 11:56:51:

I had meant to say that any usage of [NSApp isDrawing] could be replaced by ([NSView focusView] != nil). So in the check for (![NSApp isDrawing] || view != [NSView focusView]), the check for ![NSApp isDrawing] is redundant. However the check for (view != [NSView focusView]) is not redundant (as it is not merely checking for [NSView focusView] == nil); removing this check might mean that drawing can occur in the wrong toplevel (i.e. NSView), which I think is the issue Nicolas observes.


nab added on 2020-07-16 09:24:27:
as it may not be very clear, here's the video link:
https://vimeo.com/438838089

nab added on 2020-07-16 09:00:34:
a little bit more about window glitches now I've found a way to replicate them.
please note that both checkins of yesterday suffers from this issue, and I've triple-check that checkin [7ac28ee6] does NOT suffer from this issue.

here's the context, I've two toplevels on screen, the first one is displaying and updating a timer widget (a label) every seconds and the second one displaying a label in a tablelist. 
On the second one, I can keyPress Right key to generate events to update the label and if I hold the key, label is continuously updating.
all that works fine.

but if the label is updated at the same time that the timer of the first toplevel, I can see on the timer's toplevel (the first one) part of the screen of the second  toplevel. And the glitches stays until the timer is updated again...

I can make a video if you want.
++

nab added on 2020-07-16 07:27:31:
Hi,
with checkin https://core.tcl-lang.org/tk/info/87970183c0066afc, I see a lot of window glitches mainly when I deiconify toplevels.
with checkin https://core.tcl-lang.org/tk/info/af841408b2fbd94a, I do not see such glitches.

I used to create toplevel and withdraw it.
now, with both checkins I need to create toplevel, withdraw it and update idletasks otherwise I can see them appearing/disappearing during a very short time.

in terms of CPU, running a massive load in my app:
https://core.tcl-lang.org/tk/info/af841408b2fbd94a = 60% of CPU
https://core.tcl-lang.org/tk/info/87970183c0066afc = 50% of CPU

++

marc_culler (claiming to be Marc Culler) added on 2020-07-16 03:28:12:
Thank you for the feedback Christopher.

It does not seem to be possible to schedule calls to drawRect.  Of course
none of this is doucmented, but your observations indicated that it gets
called from CFRunLoopRunInMode (called in Tcl_WaitForEvent) and from nextEventMatchingMask and no where else.  So I have been assuming that if
the needsDisplay flag is yes when either of those functions are called then
drawRect will be run.  I have seen no indication that this is wrong.  The
needsDisplay flag is a mutable boolean property, so there is no obstacle to
assigning it either Boolean value.

I agree that testing [NSView focusView] seems redundant and unnecessary and
I found that I could remove those tests without causing any problems.  Likewise
I found that the needsRedisplay flag could be removed.

That was an interesting question about whether the first event loop was needed.
I found that I could remove that as well.  But in experimenting with verious
permutations of the code surrounding that loop I found that I was able to
cause Tk_MapWindow to call XMapWindow with a window id of 0, which causes a
segfault.  So I added a guard against that and also removed the loop.

I am sorry that I missed or forgot about your remark about wantsDefaultClipping.
I had only tested on Catalina, but there are definitely reports of the same
behavior on Mojave.  So I edited that omment and th one about setFrame.  And I
added a call to NSRectFromCGRect.

I also looked into the XSync function, which gets called by update after
processing all events.  I decided that there was no need for XXync to call
tkMacOSXFlushWindows, and also no need for the generic ttk code to call it
when the theme is changes.  Since those were the only uses of it, I removcd
that function.  However, I did find that it happens (often in the test suite)
that there can be timer tasks ready to run when XSync is called, presumably
timers that fired while events were being processed in the update command.
So I added an event loop which handles only timer events in XSync.

After fixing a bug in listbox.test I now consistently get 0 failures in the
regression tests on Catalina.

chrstphrchvz added on 2020-07-14 18:20:11:

I am still trying to understand what's changed and whether some of the things that I was previously checking for as indicators of drawing problems still apply. This included idle events being queued when entering drawRect:, widgets trying to draw in wrong drawRect: call, and setNeedsDisplayInRect: being called from drawRect:. At the moment, I observe some of these still happening with idle_curiosity, but have not yet determined if drawing is being lost or delayed. These comments could just be a matter of trying to understand what's now going on, rather than indication something is wrong.

  • Some (container?) widgets are redrawing which did not previously. For example, in the states.tcl widget demo, $w.frame is now redrawn when interacting with $w.frame.scroll> and $w.frame.list. Interesting, but probably not a big deal.
  • Regarding TkMacOSXDrawAllViews():
    The only time when any NSViews have the needsDisplay property set to YES is during execution of this function.
    I am not familiar enough with how AppKit events/notifications work to know if and how this is the case. For example, if macOS (rather than Tk) decide a window needed to be redrawn, can macOS schedule or invoke drawRect: "directly" (i.e. when Tk processes outstanding AppKit events), or do any such requests go through functionality that Tk has implemented/overridden/intercepted?
    If we did not run drawRect, we set needsDisplay back to NO.
    I'd not seen whether NSView supports this being set to NO rather than just YES, but am not objecting to doing this as long as it works/helps.
    Note that if drawRect did run it may have added to Tk's dirty rect, due to attempts to draw outside of drawRect's dirty rect.
    This was something that I previously tried to avoid/workaround, where doing setNeedsDisplayInRect: while in drawRect: wasn't causing drawRect: to be called again right away; is there still a similar problem? I don't remember if the issue of things having to setNeedsDisplayInRect: from drawRect: was completely due events being queued which wouldn't be able to draw for that drawRect: call; but is that not the case anymore or is not a problem? It would seem it should not be the case given what is now done at the end of TkMacOSXEventsCheckProc():
    The drawRect method "borrows" the idle queue while drawing views. … We are trying to arrange for the idle queue to be empty when it starts that process and empty when it finishes.
  • I have wondered if [NSView focusView] != nil should be checked rather than setting/unsetting an isDrawing variable, as focusView is always non-nil when in drawRect: and generally nil when outside of drawRect: (specifically, when outside of lockFocus: or whatever AppKit is wrapping drawRect: with). isDrawing is probably a better-named abstraction, though; or is still useful e.g. for indicating something should be deferred (but in a special way when in drawRect:) instead of allowed to draw.

  • I wonder if this part of drawRect: (and other efforts made in [b1645f5cd3]) is still meaningful:

        if ([self needsRedisplay]) {
    	[self setNeedsRedisplay:NO];
    	[self addTkDirtyRect: [self bounds]];
        }
    
  • What is the first idle event loop for in generateExposeEvents() (i.e. why are there two idle event loops)?

  • In wantsDefaultClipping:: "On Catalina this is never called…". I mentioned in [2a61eca3a8] that this reportedly the case as of Mojave; did that not turn out to be accurate?

  • In TkMacOSXSetupDrawingContext():

    Using [view addTkDirtyRect:clipBounds] is possibly erroneous on 32-bit (if not built with "build 32-bit like 64-bit") without NSRectFromCGRect(clipBounds).

    For some time [NSApp setIsDrawing:YES] has been done only from drawRect:, and not e.g. setFrame:. So the comment for TkMacOSXSetupDrawingContext() (line 1644) may want to be updated.

I think of all this work will one day deserve an entry in macosx/README to give an idea of how macOS deferred drawing differs from what Tk can get away with on other platforms, when and why drawRect: is called, what pitfalls are being avoided.