Author: Harald Oehlmann <[email protected]>
Author: Sergey G. Brester <[email protected]>
State: Final
Type: Project
Vote: Done
Created: 27-02-2024
Tcl-Version: 8.7
Tcl-Branch: tip-688
Vote-Summary: 3 / 0 / 0
Votes-For: JN, MC, SL
Votes-Against: none
Votes-Present: none
Abstract
Fix low performance of the clock command by a C implementation. Change some corner cases in free scanning and format preferences. Add new scan/format tolkens for zone and local seconds. Add option -now to format to internalize common clock seconds call.
Rationale
The clock command was seen by FlightAware as critical for server applications. A bounty was opened.
Sergey Brester developped a clock command rewrite in C in 2017 to get FlightAware bounty #4. Since then, the branch is in use at FlightAware. The branch originated for 8.6 was never merged due to small changes in the command. The massive speedup by a factor of 15 to 100 can be viewed below and is documented in this RFC ticket.
Sergey decided in this complete rewrite to do some changes as documented in the upper RFC and listed below.
Specification
The main specification item is SPEED and lower memory footprint.
The proposed changes are:
clock scan: preference of Gregorian date "%Y%m%d" over Julian date "%Y%j" if both are specified. See this ticket. Note: this only affects an ambiguous date (if day %m%d distinguish from %j), so basically only invalid input.
Different priority of conflicting free form scan items: result for free scanning by relative date with given month. In the following example, current TCL applies "next 1 January" first", the proposal last. (see note on free form below)
% # FreeScan : relative date with ordinal month (I said January)
% clock scan "5 years 18 months 385 days next 1 January" -base 0 -gmt 1
-Fri Jul 21 02:00:00 CEST 1978
+Sat Jan 21 01:00:00 CET 1978
- Different priority of conflicting free form scan by relative date with given month and relative weekday. (see note on free form below)
% # FreeScan : relative date with ordinal month and relative weekday (I said Fri in January)
% clock scan "5 years 18 months 385 days next January Fri" -base 0 -gmt 1
-Sat Jul 22 02:00:00 CEST 1978
+Fri Jan 27 01:00:00 CET 1978
additionally scan/format token %Es introduced to parse or format local seconds (in opposition to %s for posix seconds)
two extended tokens: %EJ (calendar) and %Ej (astronomical) Julian day number with time fraction (as floating point number).
(initially provided in GH/tclclockmod/PR/16)The format group %Ej can be also used to convert float date-times of SQLite database, for example:
% clock format 1514764800 -format %Ej -gmt 1
2458119.5
sqlite> select julianday(1514764800, 'unixepoch');
2458119.5
% clock format [clock scan 2458119.5 -format %Ej -gmt 1] -format {%Y-%m-%d %T} -gmt 1
2018-01-01 00:00:00
sqlite> select datetime(2458119.5);
2018-01-01 00:00:00
value -now will be accepted as clock value for format or add functions, e. g. clock format -now -f %u
new option clock scan ... ?-validate boolean? (default 0), if 1 it'd check the scanned input and scan will fail by invalid values (like 30 Feb, or 13th month, or 61 minute, etc)
(initially provided in GH/tclclockmod/PR/10 as a faster, in C-written replacement for tcl'ed variant of ticket [3475995])
Examples:
% clock scan "30 February 2018"
1519945200
% clock scan "30 February 2018" -valid 1
unable to convert input string: invalid day
% clock scan "2024-13-01"
1735686000
% clock scan "2024-13-01" -valid 1
unable to convert input string: invalid month
% clock scan "2024-12-01 30:00"
1733007599
% clock scan "2024-12-01 30:00" -valid 1
unable to convert input string: invalid time (hour)
- own extensions made in ParseClockFormatFormat, ParseClockScanFormat or DateParseActions are no more effective, as the scanning and formatting is pure C-implementation now
Note about free form scan
The current 8.6.14 manual has the following warning on free form scanning:
If the clock scan command is invoked without a -format option, then it requests a free-form scan. This form of scan is deprecated. The reason for the deprecation is that there are too many ambiguities. (Does the string 2000 represent a year, a time of day, or a quantity?) No set of rules for interpreting free-form dates and times has been found to give unsurprising results in all cases.
Performance
Current performance increase (in comparison vs the original clock-ensemble):
| type of clock usage | performance increase to original | new clock-engine | original clock |
|---|---|---|---|
clock format |
15 - 20 times faster | 0.27 - 4.28 µs/# | 5.45 - 45 µs/# |
clock scan -format |
40 - 70 times (up to 100 times faster *) * some previously extremely slow scans |
0.44 - 1.72 µs/# | 21 - 120 µs/# |
clock scan (freescan) |
15 - 20 times | 0.51 - 5.84 µs/# | 12 - 77 µs/# |
clock add |
50 - 90 times | 0.31 - 0.68 µs/# | 15 - 45 µs/# |
The difference is much more larger, if the tests are running multi-threaded with parasitic load.
How the performance can be measured:
Tcl-core has a file tests-perf/clock.perf.tcl which can be used to compare the execution times of original clock and new engine.
It can be also simply performed from the tclsh, using original and new branches.
Here is a diff illustrating that (which amounted to almost 95x speed-up):
% timerate -calibrate {}
% clock scan "" -timezone :CET; clock scan "" -gmt 1; # warming up
% timerate { clock scan "2009-06-30T18:30:00 CEST" -format "%Y-%m-%dT%H:%M:%S %z" -gmt 1 }
- 62.0972 µs/# 16094 # 16103.8 #/sec 999.392 net-ms
+ 0.654699 µs/# 1437085 # 1527419 #/sec 940.858 net-ms
Implementation
Implementation is in TCL branch "tip-688".
Copyright
This document has been placed in the public domain.