Tcl Source Code

View Ticket
Login
Ticket UUID: fdfbd5e10fefdb605abf34f65535054c323d9394
Title: Event, Notifier and Timer optimizations / implementation of TIPs #302, #434
Type: RFE Version: 8.5 / 8.6
Submitter: sebres Created on: 2017-07-10 12:17:30
Subsystem: 02. Event Loops Assigned To: nobody
Priority: 5 Medium Severity: Minor
Status: Open Last Modified: 2021-09-03 12:31:16
Resolution: None Closed By: nobody
    Closed on:
Description: (text/html)
<h3>Implementation:</h3>
&nbsp; - <a href="/tcl/timeline?r=sebres-8-6-event-perf-branch&nd&n=200">[sebres-8-6-event-perf-branch]</a> - 8.6 based branch<br/>
&nbsp; - <a href="/tcl/timeline?r=sebres-8-5-event-perf-branch&nd&n=200">[sebres-8-5-event-perf-branch]</a> - 8.5 based branch

<p>First implemented (squashed rebased and cherry-picked) for 8.5-branch, because I made it basically for fork of this branch (it was just easy as direct for 8.6).</p>

<h3>Contains:</h3>
<ul>
<li>massive performance optimization of event-driven handling (test-cases included), so can be see as part of <a href="https://github.com/flightaware/Tcl-bounties/issues/21" class="issue-link js-issue-link" data-url="https://github.com/flightaware/Tcl-bounties/issues/21" data-id="197943681" data-error-text="Failed to load issue title" data-permission-text="Issue title is private">flightaware/Tcl-bounties#21</a>;</li>
<li>whole event-driven functionality is up to 1000-times faster now (especially by large event resp. timer lists, but also basic multi-threaded event-related things like async-IO, reflected channels, etc.);</li>
<li>provided monotonic time (windows/unix);</li>
<li>timer events and wait facilities are time-jump safe at all - implementation for <a href="http://www.tcl.tk/cgi-bin/tct/tip/302.html">TIP #302</a>.</li>
<li>full NRT- resp. RTS-capabilities (micro- resp. nanoseconds precise);</li>
<li>implemented wide-clicks on unix (1 wide-click == 0.001 microseconds (1 nanosecond)), so more precise now (e. g. by time measurement etc.);</li>
<li>windows high-resolution time rewritten without calibration thread now (faster, more robust and provides in addition monotonic time, not affected from the time-jumps);</li>
<li>implements new sub-command <code>after at</code> for async trigger resp. sleep using absolute time (in seconds), that also time-jump safe, and in contrary to <code>after $offs</code> uses absolute based time as due-time.<br>
<code>after at [clock scan "16:00:00"] {do_it_in_1600}</code><br>
<code>after at [clock scan "+1 minute"]; # wait to the next minute</code></li>
<li>thus fully implements <a href="https://github.com/flightaware/Tcl-bounties/issues/2" class="issue-link js-issue-link" data-url="https://github.com/flightaware/Tcl-bounties/issues/2" data-id="190304010" data-error-text="Failed to load issue title" data-permission-text="Issue title is private">flightaware/Tcl-bounties#2</a></li>
<li>timer-events distinguish now between relative time-intervals (which use monotonic based time) and absolute due-time (which uses real-time base);</li>
<li>ultimate solution (resp. fix) for ticket <a href="http://core.tcl.tk/tcl/info/0520d17284500573d7c46aa88e0c6b4ebc9b6a02">0520d17284500573d7c46aa88e0c6b4ebc9b6a02</a></li>
<li>handling round about objects returned by <code>after</code> (and some internal interfaces) are more faster now, because hold the timer-event in the internal representation of object, so e. g. <code>after info</code>, <code>after cancel</code> etc don't need to search an event in event-lists anymore (and therefore don't block the list with lock for the long time);</li>
<li>event (Tcl_Event) prolongation functionality (setting of <code>event-&gt;proc</code> to the callback causes reattach of this event to end of the queue, in contrary to <code>return 0</code>, which leaves the event on the current position, and thus can repeat it too early);</li>
<li>timer prolongation functionality (ATM only in C-API as <code>TclpProlongTimerEvent</code>, can be later used as new sub-command <code>after prolong</code>);</li>
<li>Bonus:
<ul>
<li>commands <code>vwait</code> and <code>update</code> can control which events should be accepted:<br>
<code>update -timer</code><br>
<code>update -noidle</code><br>
<code>vwait -async -timer x</code></li>
<li>commands <code>vwait</code> can use optional timeout:<br>
<code>if {![vwait -timer 10 tmrVar]} continue; # do something other resp. try later</code><br>
<code>if {![vwait 1000 evVar]} { error "timeout occurred" }</code></li>
<li>commands <code>vwait</code> can work similar to <code>update</code>, without waiting for events (process only already occurring events):<br>
<code>if {[info exists evVar] || [vwait -nowait 10 evVar]} { puts "event already launched" }</code><br>
<code>## update and check we are done:</code><br>
<code>while {![vwait -nowait 0 done]} { do something other }</code></li>
<li>commands <code>after</code> and <code>vwait</code> are microsecond precise now (so NRT-capable, also accept time as double):<br>
<code>after 0.01 [list accept $socket]; # do it in 10 microseconds</code><br>
<code>## wait 5 µs for "x" and if not yet ready, 25 µs for "y":</code><br>
<code>if {![vwait 0.005 x]} {vwait 0.025 y}</code></li>
<li>new clock sub-command <code>clock monotonic</code> to provide monotonic time at tcl-level also;</li>
<li>etc. (I'll provide additional info later).</li>
</ul>
</li>
</ul>

<h4>Summaries of the performance test-cases:</h4>
See <a href="https://github.com/sebres/tcl/pull/4#issuecomment-312252977">sebres/tcl#4 (comment)</a> for the performance comparision.
User Comments: chw added on 2021-09-03 12:31:16:
For the POSIX implementation, there's a lack of using CLOCK_MONOTONIC
in the pthread_cond_*() functions, i.e. testing for availability and/or
usability of pthread_condattr_setclock(). This is essential for really
being immune against hiccups in CLOCK_REALTIME.

oehhar added on 2021-09-03 12:06:51: (text/x-fossil-wiki)
Sebres remarked today, that this TIP also solves TIP302 for Unix and Windows.

That is really amazing stuff.
Aparently, we at the TCL side have no Wizards who are even close to understand this.

Thank you again, I appreciate,
Harald

sebres added on 2017-11-07 12:38:40: (text/x-fossil-wiki)
Just noticed that TIPs <a href="https://core.tcl.tk/tips/doc/trunk/tip/434.md">#434</a>, <a href="https://core.tcl.tk/tips/doc/trunk/tip/455.md">#455</a> are also involved (#434 almost completely implemented here, #455 partially resp. affected by introducing of new options).

Has someone already tested it? Reviews, objections, etc?

sebres added on 2017-07-13 16:10:14: (text/x-fossil-wiki)
It looks like I had forgotten to back-port a small but important fix (sometimes causes busy wait in event-cycle).<br/>
Done now and merged in both branches.