Tcl Source Code

In threaded unix builds, timer events use pthread_cond_timedwait, which needs an absolute target time. The computation of this target is broken, because it adds the (correct) scaled==real delta to the (incorrect) virtual Tcl_GetTime. The fix is to use real time for the latter.

Moreover, in some unices, the clock used for pthread_cond_timedwait can be selected among several, with pthread_condattr_setclock at condvar creation time. One of them, CLOCK_MONOTONIC, has the extra advantage of being insensitive to system clock tweaks. I think that selecting it when possible can only improve things, since it allows:

     - usual, tip-302-sensitive [after] semantics unchanged, since the timer events are also stored as absolute epoch-time  targets by Tcl
     - fully working Virtual Time with clock_gettime(CLOCK_MONOTONIC) giving tip-302-insensitive [after].

I have a provisional patch for platforms with clock_gettime+CLOCK_MONOTONIC, designed to enable Joe Keen's code in http://groups.google.com/group/comp.lang.tcl/browse_frm/thread/8f820c68143850f9# . Some autogoo would be due before committing.

An alternative to 'select monotonic if possible, epoch if not' would be a runtime setting, like a ::tcl::unsupported::monotonicTime linked var. To be discussed.

Thanks for the info. Commit [2e9374edd5] makes it work on MS-Windows for me. That is great.

I have removed the Windows XP emulation. It did not compile for me. The used interface was introduced with Windows Vista.

This does not allow to merge-back to TCL 8.6, as XP is supported.

The issue with TIP #233 is complicated on the formal level, as this is an approved TIP. I will ask the author of the TIP (Andreas Kupries), if this is really senseful and in conformance. Perhaps, he is in the Telco tonight.

Thanks for all, Harald

And again for the record regarding TIP#233: there ain't no test
cases in the test suite, how can we be sure that it even works?
A section in one of the man pages is missing, too. IMO TIP#233
should be ditched.

Harald,

yes, it is, since that's the precondition to have some monotonic clock
on Win32 using GetTickCount64() or even GetTickCount() as a fall back
on olden OS versions (which could be not old enough even to be recycled).

May I ask, if undroidwish is compiled with "--with-tickcount" to disable TIP 233 https://core.tcl-lang.org/tips/doc/trunk/tip/233.md ?

This activates different code paths...

Thanks, Harald

CHristian, thanks for looking into this. Undroidwish: https://ch-werner.hier-im-netz.de/LUCK/undroidwish-win64.exe

There, the test works like a charme. I will diff the relevant places.

Here is the log from Undoidwish:

% proc pt sec10 {
> 
>     puts "sec10=$sec10, clock=[expr {[clock seconds]-$::now}]"
> 
> }
(Desktop) 2 % 
(Desktop) 2 % set now [clock seconds]
1746963642
(Desktop) 3 % 
(Desktop) 3 % for {set sec 10} {$sec<=180} {incr sec 10} {after [expr {$sec*1000}] [list pt $sec]}
sec10=10, clock=10
sec10=20, clock=20
sec10=30, clock=566
sec10=40, clock=576
sec10=50, clock=586
sec10=60, clock=596
sec10=70, clock=606
sec10=80, clock=616
sec10=90, clock=626
sec10=100, clock=636
sec10=110, clock=646
sec10=120, clock=656
sec10=130, clock=666
sec10=140, clock=676
sec10=150, clock=686

The after is independent on the wall clock. You see the wall-clock jump at 20, but the after continue to work with a time distance of 10.

Thanks for all, Harald

Nope, I was wrong.
Assuming that your patch is what in vanillawish works,
the patchery went wrong. Bummer!

Come on Harald,
a puts and and a flush that's all

It was me. I forgot the "vwait forever", as it is tclsh, not wish.

But for the given test below, there is no difference between the bug branch and tcl9.0.1. The branch does not help and apparently, the after is not independent to the wall clock.

Here is the output:

% proc pt sec10 {
    puts "sec10=$sec10, clock=[expr {[clock seconds]-$::now}]"
}
% set now [clock seconds]
1746909277
% for {set sec 10} {$sec<=180} {incr sec 10} {after [expr {$sec*1000}] [list pt $sec]}
% vwait forever
sec10=10, clock=12
sec10=20, clock=22
sec10=30, clock=32
sec10=40, clock=42
sec10=50, clock=623
sec10=60, clock=623
sec10=70, clock=623
sec10=80, clock=623
sec10=90, clock=623
sec10=100, clock=623
sec10=110, clock=623
sec10=120, clock=623
sec10=130, clock=623
sec10=140, clock=623
sec10=150, clock=623
sec10=160, clock=623
sec10=170, clock=623
sec10=180, clock=623

As you can see, all after events fire together, when the wall clock is advanced.

Do I miss something?

Thanks for all, Harald

I have made the following test on Winddows 11 64 bit with MS-VS2022 64 bit compile of the branch:

proc pt sec10 {
    puts "sec10=$sec10, clock=[expr {[clock seconds]-$::now}]"
}
set now [clock seconds]
for {set sec 10} {$sec<=180} {incr sec 10} {after [expr {$sec*1000}] [list pt $sec]}

Then, I start cmd.exe as Administrator and type "time" and I advance the time by 10 minuites.

The results are as follows:

• TCL 9.0.1: all remaining events fire at once
• Bug branch: no events fire at all, there is no after event any more.

So, there is something wrong with the patch...

Harald

Merged current main by [6ff6edf427]. Only tested on Windows. I was surprised, that there are still "#if TCL_THREADS" in the source. I thought, we nowdays always build threadded.

Next step is to test and to look on Sergeys work.

Thanks for all, Harald

Another attempt to bring tip 302 somewhat forward. The another attached patch
adds handling for the new "-at" attribute for timed after commands. The semantic
is not like "-robust" as laid out in the tip, but instead based on the clock
command, i.e. specifies a point in this (wall clock) time domain to make that
fact explicit to the user. A few test cases are included as well as an update
to the after.n man page.

The patch is against core-8-6-branch and tested on Debian 9 amd64. It runs
all tests without errors. Win32/64 is not verified.

From the Tcl'ers Chat for the record (citing myself, sorry):

Regarding TIP#302: I don't remember any man page version of after(n) and Tcl_CreateTimerHandler(3) due to its relative millisecond-ness mentioning absolute time in the wall clock sense for the last 20 years, which makes CLOCK_MONOTONIC a legit time source. After all, any assumptions, that time computations using [clock seconds] et.al. applied to after(n) are correct, are IMO thus cargo-cultivated programming errors. Then there is interp(n) with its IMO wrongly designed time limit based on absolute time. But that certainly came long after after's invention. And why has it to be based on absolute time? And then there is virtualized time? And why was it? And where is it test cased in the core?

Sooorry, but couldn't resist.

tcl is not dead yet ... https://www.youtube.com/watch?v=oZ2QiyxRiYs&feature=youtu.be&a -- jima, tclers chat, 2017

Time is heavy sometimes; imagine how heavy eternity must be. -- E.M.Cioran, The Book of Delusions, 1936

Latest patch from Christian committed to branch "tkt3328635-posix-monotonic-clock". Applied to trunk in stead of core-8-6-branch, most likely this change is too intrusive for a patch release.

Latest version which is smaller and works hopefully on olden Windows < Vista, and is integrated into AndroWish now, can be found in attachment atip302.diff, please review.

Not sure if this is the right ticket, but my patch to make things more
insensitive against wall clock time changes is somewhat improved now,
see attachment.

status?

Thanks. So this means we do need configure.in/tcl.m4 changes to know of the need for linking against librt, and when.

Have to test on other platforms as well.

Will test and review asap; in the meantime, regarding the clock_gettime() missing symbol on glibc, my wheezy manpage says:

       Link with -lrt (only for glibc versions before 2.17).

The updated patch is now committed to branch "tkt3328635-posix-monotonic-clock".

Pushed revision [265e84eb5e]. I am asking for a general review by Joe and Alexandre.

This code does not work yet! (on linux even).

It compiles on my build system (RedHat FC4 Stentz), yet fails to link. It claims that clock_gettime() is missing (*).

Checking with temporarily added #error lines showed that the code believes that both _POSIX_TIMERS and _POSIX_MONOTONIC_CLOCK are defined. From what I read in the discussion here and referenced web site and manpage that should mean that clock_gettime() is available. And it is ipmlied to be in the (g)libc.

The glibc here is 2.3(.6). Should some other library be linked too ?

Or could be a macro and I need a header which contains the proper mapping to the actual function name ?

Further note that I tried to extend the patch to fall back to CLOCK_REALTIME where possible. Please review this part too.

(Ad *): .../build/tcl-8.5/libtcl8.5.so: undefined reference to `clock_gettime'

The patch was not updated with the discussion about how to configure this.

In the works now.

Working on the Tcl 8.5 branch.

Commit will be on a branch at first, for review (by Joe, who will become a contact here).

A -nocase grep of the Tcl 8.6 sources does not show any use of 'monotonic' except tclCompile.h where it talks about source offsets put into the byte code.

Ditto for Tcl 8.5.

It seems that the patch is not applied.
Bumping prio for self.

status?

status?

Thanks for the POSIX macros: simpler indeed !
For the "#if defined(CLOCK_MONOTONIC)", are we sure that when it's here it's always a macro, not a direct enum ?
Last, is there a typo in your last sentence ? Shouldn't it read "I don't see any reason *not* to preemptively offer this as an option" ?

Also - as a falllback, if _POSIX_MONOTONIC_CLOCK / CLOCK_MONOTONIC is not available, a suitable fallback is CLOCK_REALTIME; this is guaranteed present if _POSIX_TIMERS are available at all, will be no more broken than gettimeofday() (which is what's currently used), and  less broken than the damaged TIP#233 version.

Re:  option to enable/disable this from Tcl scripts or C code -- ISTM this would be an "Old buggy behaviour [Y/N]?" type question, I don't see any reason to preemptively offer this as an option.

You don't need autogoo to detect this, POSIX specifies preprocessor macros you can test instead.  

See <URL: http://pubs.opengroup.org/onlinepubs/7908799/xsh/unistd.h.html >, or `man 7 posixoptions ` on a glibc system, for details.  But basically: #if defined(_POSIX_TIMERS) && defined(_POSIX_MONOTONIC_CLOCK).  Come to think of it, you could just use #if defined(CLOCK_MONOTONIC) ...

Pasting Roy Keene's kind crash-course on autoconf:

[19:49]<rkeene>AC_DEFUN(TCL_CHECK_CLOCK_BLAH, [ AC_COMPILE_IFELSE(AC_LANG_PROGRAM([[contents-outside-of-main]], [[contents-of-main]]), [ AC_DEFINE(HAVE_CLOCK_GETTIME, [1], [Define to 1 if you have whatever clockgettime is]) ]) ])

Then, as Roy suggests, do the runtime part (is CLOCK_MONOTONIC working here ?) at Tcl init time.

Actually, if we choose the explicit selection approach, the toggle should rather be accessible at C level, by the extensions able to benefit from it: Tcl_SetClockKind( 0==normal, 1==monotonic). This way, an extension can enable the tweak-unsensitive [after] in a completely self-contained way. If you like this method I'll write the patch shortly.

File Added - 415876: tim.patch