TIP 723: timer ensemble for wall clock and monotonic clock events

Login
Core Home Timeline Branches Tags Forum Help 
Author:         Harald Oehlmann <[email protected]>
State:          Final
Type:           Project
Created:        21-May-2025
Tcl-Version:    9.1
Keywords:       after
Tcl-Branch:     tip-723-timer-wall-monotonic
Vote:           Done
Vote-Summary:   Accepted 3/0/0
Votes-For:      AN, HO, KW
Votes-Against:  none
Votes-Present:  none

Abstract

This TIP proposes to add the timer ensemble to execute a command at a given wall clock point or after a given monotonic time period. The timer resolution is set to microseconds instead of milliseconds. All internal time difference functions use a monotonic clock instead of the wall clock. It is proposed to remove the functionality of TIP 233 time manipulation callbacks. It is proposed that "after" now uses the monotonic clock. New C-API functions for monotonic clock and/or microseconds resolution.

Rational

The current after command is used for two purposes, execute at a time delay and at a wall clock instant. This does not work, if time and wall clock differs, like on manual wall clock set, systen suspension etc. after was always documented to invoke at a time distance, but the implementation calculated the wall clock instance and executed at the wall clock instance. Androwish/Undroidwish and other alternate interpreters changed after to work on a monotonic clock since a long time.

For most use cases, like timeouts, gui refresh, periodic processes, the monotonic clock is the right time base. There are use cases for wall clock as a clock widget update.

Those two use cases should be split-up in two distinct commands.

The main API function time base function is TCL_GetTime which returns the seconds like clock seconds and a microseconds fractional part (0-999999). This is always relative to the Unix epoc. Value 0 is start of year 1970 in UTC.

A companion API for monotonic time (Tcl_GetMonotonicTime) is proposed, which is only defined in monotonic time differences. The start point is undefined. In the case of MS-Windows, it is the computer bootup time.

TIP233 allows to register a command to change the result of TCL_GetTime. It was implemented for simulators within a NDC contract where usage is unknown. Its commands are not documented and current usage is unknown. It is supposed, that the after event should be simulated in a different time scale (probably slower) within simulators. It probably never fully worked in all aspects, as it is limited to TCL perception of time, but there are other time aspects run by the operating system like event queue lookup time calculation. It is proposed to remove this functionality, as it does not cover the monotonic clock.

The command after is documented to execute after a time distance, e.g. using a monotonic clock. This is also the supposed use case in 99% of all cases. It is proposed to implement it as documented, e.g. switch from wall clock to monotonic clock.

The command time is documented to measure a time distance. Its usage is switched to a monotonic clock.

The command interp limit is defined to be dependent on the "epoch". A wall clock time point is given. Supposed use case is monotonic clock (e.g. runtime of the interpreter). As a wall clock time point is given (example: clock scan "+1s" [clock now]) and the parameter may be queried, it would be strange, if it does not limit exactly to this time point. Even if it would be much more useful with monotonic clock, it is kept as wall clock. The documentation is updated to clearly state this. E.g. if the intepreter limit in in 1 hour and the wall clock jumpts by 2 hours foreward, the limit fires imediately (with a drift of 1 second due to internal optimizations). The monotonic runtime of an interpreter may not be set.

The after command allowed to specify a script as one or multiple list elements. The timer ensemble only allows one list element.

The after cancel command allowed to specify a command or an after id. The timer cancel command does only support after ids and no commands. Commands are seen as ambiguous and dangerous for a global command.

Specification

Timer resolution

Currently, the timer resolution is milli-seconds, except the function "Tcl_GetTime", which has micro-seconds resolution. It is aimed to have all APIs in micro-seconds resolution. The internal implementation may have a lower resolution. This may evolve over time to get more resolution.

command after

The command after uses the monotonic clock as documented.

The after ensemble is not deprecated.

vwait -timeout ms

The vwait function has an argument "-timeout ms" with a milli-seconds argument. This timeout is changed to monotonic clock.

There is no alternative proposed to specify a timeout in micro-seconds. This may be done in a follow-up TIP.

TIP 233

If Tcl_SetTimeProc or Tcl_QueryTimeProc is called, Tcl_Panic is called with such a message.

Internal functions

Suitable internal functions may use monotonic clock instead of wall clock. Examples are measurements of network delay. This will probably also fix bugs, in the case that the wall clock jumpts back and a time difference gets negative. I don't see nowhere any handling code against this case.

Timer ensemble

The new timer ensemble is proposed with the following subcommands:

timer in time unit script

time is the monotonic time distance and is a wide integer. Negative values are treated as 0.

unit is the unit of time and may be us, ms, s, microseconds*, milliseconds or seconds.

The command will wait the given monotonic delay, if no command given.

The command will be executed after the given delay following a monotonic clock if a command is given. A timer id is returned, which may be used to cancel the event.

timer at time unit script

A command may be executed at a given wall clock time point. Also, a delay until a given wall clock time point is supported.

time is the wall clock instance and is a wide integer. Negative values are treated as 0.

unit is the unit of time and may be us, ms, s, microseconds*, milliseconds or seconds.

A moment in the past will be handled as now.

The command will wait until the given wall clock instance passed, if no command given.

The command will be executed at the given wall clock instance, if a command is given. A timer id is returned, which may be used to cancel the event.

If wall clock and monotonic time increment differs, wall clock events get unreliable, as the underlying timing system may not fire events in time. This was anyway always the case and is not seen as a side effect.

A typical use-case is to pass the result of a clock command for time:

time at [clock scan [clock scan {2030-10-30 05:00:00} -format {%X-%m-%d %H:%M:%S}]\
        seconds $Cmd

or

time at [clock add [clock seconds] 24 hours] seconds $Cmd

timer info ?id?

Return information about a registered timer event. A list of 2 to 3 values is returned, depending on the type. The first list item is the command to execute. The second list item is the type, which may be one of "idle", "monotonic" or "wallclock".

For type "monotonic", the 3rd returned list value is the monotonic time point in micro-seconds, when the event will fire. For type "wallclock", the 3rd returned list argument is the wall clock time point in micro-seconds, when the event will fire. For type "idle", there is no 3rd list value.

The command raises an error, if the passed id argument is unknown.

There is the following difference in comparison to after info:

timer cancel id

Cancel a timer event. The command does nothing, if the given id is unknown.

There is the following difference in comparison to after cancel:

timer idle script

Register a script to be executed on idle time.

Compared to after idle, the script argument must be given as one argument.

timer sleep for delay unit

Sleeps (at least) for the given monotonic time. The unit may be s, seconds, ms, microseconds, us, microseconds.

timer sleep until time unit

Sleep (at least) until the given wall clock time point. The unit may be s, seconds, ms, microseconds, us, microseconds.

clock monotonic

This command returns the current monotonic time point in microseconds.

Sergey has it like this, thanks for that !

Tcl_GetMonotonicTime

The new C API function is proposed:

long long Tcl_GetMonotonicTime()

The difference between two calls is the count of elapsed monotonic microseconds.

Tcl_Sleep

Tcl_Sleep is documented and called with a time interval. It is changed to use the monotonic clock.

Tcl_SleepMicroSeconds

New API:

void Tcl_SleepMicroSeconds(long long Microseconds)

This is the same as Tcl_Sleep, but with a microseconds argument.

Tcl_CreateTimerHandler

The function Tcl_CreateTimerHandler is documented and called with a time interval. It is switched to the monotonic clock.

The call does not return NULL on time value overflow.

Tcl_CreateTimerHandlerMicroSeconds

New API as Tcl_CreateTimerHandler, but using an argument in microseconds.

Tcl_TimerToken
Tcl_CreateTimerHandlerMicroSeconds(
    long long microSeconds,     /* How many micro-seconds to wait before
                                 * invoking proc. */
    Tcl_TimerProc *proc,        /* Function to invoke. */
    void *clientData)           /* Arbitrary data to pass to proc. */

In addition, the function returns NULL, if there is an overflow in timer arithmetic.

Range check

Within this TIP, the maximum time point to address is restraint to LLONG_MAX microseconds (64 bit).

This time point on wall clock time is around year 294441:

expr { (2**63-1) / (1000000 * 60 * 60 * 24 * 365 ) + 1970 }

The reason for this constraint is, that the target time point may always be returned by "timer info". This command returns microseconds.

The sec member of Tcl_Time is limited to (LLONG_MAX/1000000-1). This is a bit earlier than the upper time point but avoids to check the usec part in the overflow check.

This does not apply to the functionality of the clock command. This should work:

clock format 0x800000000

Precision

The precision of the wall clock is not guaranteed. The wall clock may jump at any moment and the system may optimize the query frequency of the wall clock for example to once each 100ms. The monotonic clock should be used for microseconds precission action.

Compatibility

If after is used for a wall clock event, the code should be changed to use timer at.

Demonstration

To test the new functionality on Windows 11:

set now [clock seconds]
for {set t 10} {$t < 10000} {incr t 10} {
    timer at [expr {$now+$t}] 0 [list puts "at $t"]
    timer in $t 0 [list puts "in $t"]
}
vwait a

Discussion

Fix width C types

I would prefer fix length C types in the interface. Currently, "long long" is used. The reasoning is, that Tcl_Time is also using "long long".

I would prefer a 64 bit signed type, like "int64_t". My questions about it got no answers, so dropping the subject.

As a Windows programmer, I am used to fix-size types. This makes many things like bound-check easier and avoids any surprises.

But also, the type "Tcl_WideInt" has no gurantee of size at all. Well, minimum 32 bit probably. So, if this is not 64 bit, anything will break anyway. So, this is a bigger issue in TCL...

interp limit

Christians patch converts interp limit to monotonic clock. This is 99% the cases what should be done, but is not documented like that.

repretitive timer event

There is the proposal of a repetitve timer event. This is sensible but currently not covered. One step at a time.

option name "in" and "info" share same root

The option names "in" and "info" share the same root. This is inconvenient to abreviate the command. The command name "timer after" was proposed. IMHO this is to close to "after" without "timer". Native speakers may comment.

Reference Implementation

A reference impementation can be found in Tcl branch tip-723-timer-wall-monotonic.

Issues

Ashok has reported the following points on 2025-12-13 via core list:

clock format with high values

"clock format 0x800000000" fails

Starvation

9.0 has two queues - event and idle. A well known issue for scheduling long running computations is starving the idle queue (and freezing the Tk UI). The work around is to use the [after 0 after idle] idiom. Now that there is an additional event queue, it seems to me there is a risk in the current implementation that the new queue (wallclock) will be starved by the scheduling queue in a similar scenario. The implementation may need to be changed to alternate between the wallclock and scheduling queues based on the newer time id (Sorry, I have not actually experimented, just looked at the code.)

Windows specific

Frequency is never modified after boot, so instead of calling QueryPerformanceFrequency on every Tcl_GetMonotonicTime invocation, you can just call it once at initialization and cache the value. Also, even if QueryPerformanceFrequency returns 0, you should also check that the returned Frequency.QuadPart is not 0. If it is, use the old method. Microsoft is really opaque with regards to running on virtual machines so better to be safe.

In the Windows version of TclpWaitForEvent, the timeout is truncated. I don't know if that is intentional but I think it should be rounded up. For example, if timePtr->sec is 0, and timePtr->usec is some value < 1000, the timeout in ms is calculated as 0. Otherwise, there is the possibility of a busy-wait.

Patch by Christian Werner

An untested great patch for Linux and MacOS by Christian Werner is found in Tcl branch tkt3328635-posix-monotonic-clock. The basic monotonic implementation was taken from here.

Patch by Sergey

The great patch by Sergey: Ticket fdfbd5e10f At least, the "clock monotonic" command was taken from here. Many other should follow...

Credits

Thanks to Christian Werner and Sergey Brester for implementation and many work around this issue. Thanks to Reinhard, Schelte and Ashok for the proposals on the interface.

Proposals after vote start

Those proposals were also asked within the vote and were accepted.

unit may be abreviated

It was proposed to allow abbreviations of the units, for example "sec". This may be seen as a bugfix, as the current TIP does not mention anything about abbreviations.

unit default value for timer sleep

It is proposed to use default values for the sleeping time units:

The default values were never clearly specified. Here is my proposal:

The reason for this choice:

Copyright

This document has been placed in the public domain.