# TIP 688: clock command revision and speedup
	Author:		Harald Oehlmann <harald.oehlmann@elmicron.de>
	Author:		Sergey G. Brester <sebres@users.sourceforge.net>
	State:		Final
	Type:		Project
	Vote:		Done
	Created:	27-02-2024
	Tcl-Version:	8.7
	Tcl-Branch:	tip-688
	Vote-Summary:  3 / 0 / 0
	Votes-For:     JN, MC, SL
	Votes-Against: none
	Votes-Present: none
-----

# Abstract

Fix low [performance](#performance) of the clock command by a C implementation.
Change some corner cases in free scanning and format preferences.
Add new scan/format tolkens for zone and local seconds.
Add option *-now* to **format** to internalize common **clock seconds**
call.

# Rationale

The clock command was seen by FlightAware as critical for server
applications. A bounty was opened.

Sergey Brester developped a clock command rewrite in C in 2017 to get
FlightAware bounty #4. Since then, the branch is in use at FlightAware.
The branch originated for 8.6 was never merged due to small changes in
the command. The massive speedup by a factor of 15 to 100 can be viewed
[below](#performance) and is documented in this [RFC ticket](https://core.tcl-lang.org/tcl/info/ddc948cff9).

Sergey decided in this complete rewrite to do some changes as documented in the upper RFC and listed below.

# Specification

The main specification item is [**SPEED**](#performance) and lower memory footprint.

The proposed changes are:

   *   clock scan: preference of Gregorian date "%Y%m%d" over Julian date "%Y%j" if both are specified. See this [ticket](http://core.tcl.tk/tcl/tktview?name=e7a722cd35).
       Note: this only affects an ambiguous date (if day *%m%d* distinguish from *%j*), so basically only invalid input.

   *   Different priority of conflicting free form scan items: result for free scanning by relative date with given month. In the following example, current TCL applies "next 1 January" first", the proposal last. (see note on free form below)

~~~
 % # FreeScan : relative date with ordinal month (I said January)
 % clock scan "5 years 18 months 385 days next 1 January" -base 0 -gmt 1
-Fri Jul 21 02:00:00 CEST 1978
+Sat Jan 21 01:00:00 CET 1978
~~~

   *   Different priority of conflicting free form scan by relative date with given month and relative weekday. (see note on free form below)

~~~
 % # FreeScan : relative date with ordinal month and relative weekday (I said Fri in January)
 % clock scan "5 years 18 months 385 days next January Fri" -base 0 -gmt 1
-Sat Jul 22 02:00:00 CEST 1978
+Fri Jan 27 01:00:00 CET 1978
~~~

   *   additionally scan/format token *%Es* introduced to parse or format local seconds (in opposition to *%s* for posix seconds)

   *   two extended tokens: *%EJ* (calendar) and *%Ej* (astronomical) Julian day number with time fraction (as floating point number).<br/>
       (initially provided in [GH/tclclockmod/PR/16](https://github.com/sebres/tclclockmod/pull/16))

       The format group *%Ej* can be also used to convert float date-times of SQLite database, for example:

~~~
% clock format 1514764800 -format %Ej -gmt 1
2458119.5                                 
sqlite> select julianday(1514764800, 'unixepoch');
2458119.5

% clock format [clock scan 2458119.5 -format %Ej -gmt 1] -format {%Y-%m-%d %T} -gmt 1
2018-01-01 00:00:00
sqlite> select datetime(2458119.5);
2018-01-01 00:00:00
~~~

   *   value *-now* will be accepted as clock value for format or add functions, e. g. **clock format -now -f %u**

   *   new option clock scan ... **?-validate boolean?** (default 0), if 1 it'd check the scanned input and scan will fail by invalid values (like 30 Feb, or 13th month, or 61 minute, etc)<br/>
       (initially provided in [GH/tclclockmod/PR/10](https://github.com/sebres/tclclockmod/pull/10) as a faster, in C-written replacement for tcl'ed variant of [ticket [3475995]](https://core.tcl-lang.org/tcl/info/3475995fffffffff))<br/>
       Examples:

~~~
% clock scan "30 February 2018"
1519945200
% clock scan "30 February 2018" -valid 1
unable to convert input string: invalid day

% clock scan "2024-13-01"
1735686000
% clock scan "2024-13-01" -valid 1
unable to convert input string: invalid month

% clock scan "2024-12-01 30:00"
1733007599
% clock scan "2024-12-01 30:00" -valid 1
unable to convert input string: invalid time (hour)
~~~

   *   own extensions made in ParseClockFormatFormat, ParseClockScanFormat or DateParseActions are no more effective, as the scanning and formatting is pure C-implementation now

## Note about free form scan

The current 8.6.14 manual has the following warning on free form scanning:

If the **clock scan** command is invoked without a *-format* option, then
it requests a *free-form scan. This form of scan is deprecated.*
The reason for the deprecation is that there are too many ambiguities.
(Does the string **2000** represent a year, a time of day, or a quantity?)
No set of rules for interpreting free-form dates and times has been found
to give unsurprising results in all cases.

<h1 id="performance">Performance</h1>

Current performance increase (in comparison vs the original clock-ensemble):

type of clock usage &nbsp; &nbsp; &nbsp; | performance increase to original &nbsp; &nbsp; &nbsp; &nbsp; | new clock-engine &nbsp; | original clock
-------- | -------------------- | ----------- | ------------
`clock format` | 15 - 20 times faster | 0.27 - 4.28 µs/# | 5.45 - 45 µs/#
`clock scan -format` | 40 - 70 times (up to 100 times faster \*)<br/><sub>\* some previously extremely slow scans</sub> | 0.44 - 1.72 µs/# | 21 - 120 µs/#
`clock scan` (freescan) | 15 - 20 times | 0.51 - 5.84 µs/# | 12 - 77 µs/#
`clock add` | 50 - 90 times | 0.31 - 0.68 µs/# | 15 - 45 µs/#

The difference is much more larger, if the tests are running multi-threaded with parasitic load.

#### How the performance can be measured:

Tcl-core has a file [tests-perf/clock.perf.tcl](/tcl/file?name=tests-perf/chan.perf.tcl) which can be used to compare the execution times of original clock and new engine.
It can be also simply performed from the tclsh, using original and new branches.<br/>
Here is a diff illustrating that (which amounted to almost 95x speed-up):

~~~diff
  % timerate -calibrate {}
  % clock scan "" -timezone :CET; clock scan "" -gmt 1; # warming up
  % timerate { clock scan "2009-06-30T18:30:00 CEST" -format "%Y-%m-%dT%H:%M:%S %z" -gmt 1 }
- 62.0972 µs/# 16094 # 16103.8 #/sec 999.392 net-ms
+ 0.654699 µs/# 1437085 # 1527419 #/sec 940.858 net-ms
~~~

# Implementation

Implementation is in [TCL branch "tip-688"](https://core.tcl-lang.org/tcl/timeline?r=tip-688).

# Copyright

This document has been placed in the public domain.
