# TIP 688: clock command revision and speedup
Author: Harald Oehlmann <[email protected]>
Author: Sergey G. Brester <[email protected]>
State: Final
Type: Project
Vote: Done
Created: 27-02-2024
Tcl-Version: 8.7
Tcl-Branch: tip-688
Vote-Summary: 3 / 0 / 0
Votes-For: JN, MC, SL
Votes-Against: none
Votes-Present: none
-----
# Abstract
Fix low [performance](#performance) of the clock command by a C implementation.
Change some corner cases in free scanning and format preferences.
Add new scan/format tolkens for zone and local seconds.
Add option *-now* to **format** to internalize common **clock seconds**
call.
# Rationale
The clock command was seen by FlightAware as critical for server
applications. A bounty was opened.
Sergey Brester developped a clock command rewrite in C in 2017 to get
FlightAware bounty #4. Since then, the branch is in use at FlightAware.
The branch originated for 8.6 was never merged due to small changes in
the command. The massive speedup by a factor of 15 to 100 can be viewed
[below](#performance) and is documented in this [RFC ticket](https://core.tcl-lang.org/tcl/info/ddc948cff9).
Sergey decided in this complete rewrite to do some changes as documented in the upper RFC and listed below.
# Specification
The main specification item is [**SPEED**](#performance) and lower memory footprint.
The proposed changes are:
* clock scan: preference of Gregorian date "%Y%m%d" over Julian date "%Y%j" if both are specified. See this [ticket](http://core.tcl.tk/tcl/tktview?name=e7a722cd35).
Note: this only affects an ambiguous date (if day *%m%d* distinguish from *%j*), so basically only invalid input.
* Different priority of conflicting free form scan items: result for free scanning by relative date with given month. In the following example, current TCL applies "next 1 January" first", the proposal last. (see note on free form below)
~~~
% # FreeScan : relative date with ordinal month (I said January)
% clock scan "5 years 18 months 385 days next 1 January" -base 0 -gmt 1
-Fri Jul 21 02:00:00 CEST 1978
+Sat Jan 21 01:00:00 CET 1978
~~~
* Different priority of conflicting free form scan by relative date with given month and relative weekday. (see note on free form below)
~~~
% # FreeScan : relative date with ordinal month and relative weekday (I said Fri in January)
% clock scan "5 years 18 months 385 days next January Fri" -base 0 -gmt 1
-Sat Jul 22 02:00:00 CEST 1978
+Fri Jan 27 01:00:00 CET 1978
~~~
* additionally scan/format token *%Es* introduced to parse or format local seconds (in opposition to *%s* for posix seconds)
* two extended tokens: *%EJ* (calendar) and *%Ej* (astronomical) Julian day number with time fraction (as floating point number).<br/>
(initially provided in [GH/tclclockmod/PR/16](https://github.com/sebres/tclclockmod/pull/16))
The format group *%Ej* can be also used to convert float date-times of SQLite database, for example:
~~~
% clock format 1514764800 -format %Ej -gmt 1
2458119.5
sqlite> select julianday(1514764800, 'unixepoch');
2458119.5
% clock format [clock scan 2458119.5 -format %Ej -gmt 1] -format {%Y-%m-%d %T} -gmt 1
2018-01-01 00:00:00
sqlite> select datetime(2458119.5);
2018-01-01 00:00:00
~~~
* value *-now* will be accepted as clock value for format or add functions, e. g. **clock format -now -f %u**
* new option clock scan ... **?-validate boolean?** (default 0), if 1 it'd check the scanned input and scan will fail by invalid values (like 30 Feb, or 13th month, or 61 minute, etc)<br/>
(initially provided in [GH/tclclockmod/PR/10](https://github.com/sebres/tclclockmod/pull/10) as a faster, in C-written replacement for tcl'ed variant of [ticket [3475995]](https://core.tcl-lang.org/tcl/info/3475995fffffffff))<br/>
Examples:
~~~
% clock scan "30 February 2018"
1519945200
% clock scan "30 February 2018" -valid 1
unable to convert input string: invalid day
% clock scan "2024-13-01"
1735686000
% clock scan "2024-13-01" -valid 1
unable to convert input string: invalid month
% clock scan "2024-12-01 30:00"
1733007599
% clock scan "2024-12-01 30:00" -valid 1
unable to convert input string: invalid time (hour)
~~~
* own extensions made in ParseClockFormatFormat, ParseClockScanFormat or DateParseActions are no more effective, as the scanning and formatting is pure C-implementation now
## Note about free form scan
The current 8.6.14 manual has the following warning on free form scanning:
If the **clock scan** command is invoked without a *-format* option, then
it requests a *free-form scan. This form of scan is deprecated.*
The reason for the deprecation is that there are too many ambiguities.
(Does the string **2000** represent a year, a time of day, or a quantity?)
No set of rules for interpreting free-form dates and times has been found
to give unsurprising results in all cases.
<h1 id="performance">Performance</h1>
Current performance increase (in comparison vs the original clock-ensemble):
type of clock usage | performance increase to original | new clock-engine | original clock
-------- | -------------------- | ----------- | ------------
`clock format` | 15 - 20 times faster | 0.27 - 4.28 µs/# | 5.45 - 45 µs/#
`clock scan -format` | 40 - 70 times (up to 100 times faster \*)<br/><sub>\* some previously extremely slow scans</sub> | 0.44 - 1.72 µs/# | 21 - 120 µs/#
`clock scan` (freescan) | 15 - 20 times | 0.51 - 5.84 µs/# | 12 - 77 µs/#
`clock add` | 50 - 90 times | 0.31 - 0.68 µs/# | 15 - 45 µs/#
The difference is much more larger, if the tests are running multi-threaded with parasitic load.
#### How the performance can be measured:
Tcl-core has a file [tests-perf/clock.perf.tcl](/tcl/file?name=tests-perf/chan.perf.tcl) which can be used to compare the execution times of original clock and new engine.
It can be also simply performed from the tclsh, using original and new branches.<br/>
Here is a diff illustrating that (which amounted to almost 95x speed-up):
~~~diff
% timerate -calibrate {}
% clock scan "" -timezone :CET; clock scan "" -gmt 1; # warming up
% timerate { clock scan "2009-06-30T18:30:00 CEST" -format "%Y-%m-%dT%H:%M:%S %z" -gmt 1 }
- 62.0972 µs/# 16094 # 16103.8 #/sec 999.392 net-ms
+ 0.654699 µs/# 1437085 # 1527419 #/sec 940.858 net-ms
~~~
# Implementation
Implementation is in [TCL branch "tip-688"](https://core.tcl-lang.org/tcl/timeline?r=tip-688).
# Copyright
This document has been placed in the public domain.