Author:             Ashok P. Nadkarni <[email protected]>
Tcl-Version:        9.1
State:              Final
Type:               Project
Created:            2025-09-19
Keywords:           Initialization
Tcl-Branch:         tip-732
Vote:               Done
Vote-Summary:       Accepted 6/0/0
Votes-For:          AN, HO, KW, JN, MC, SL
Votes-Against:      none
Votes-Present:      none

---

• Abstract
• Background
  • 9.0 Tcl initialization
  • 9.0 interpreter initialization
• Specification
  • Tcl initialization
  • Interpreter initialization
• Implementation notes
• References
• Copyright

Abstract

This TIP proposes

• Updating and formalizing the method by which Tcl locates its supporting script libraries and encoding files.

• Requiring applications to initialize Tcl using either TclZipfs_AppHook or Tcl_FindExecutable before calling Tcl's C API or creating interpreters.

Background

9.0 Tcl initialization

Historically, Tcl has not clearly documented the calls an application must make to initialize Tcl prior to invocation of Tcl functions or creating interpreters. Manpages, TIP 596, and public documentation like the Wiki and authoritative books, collectively suggest Tcl_InitSubSystems, Tcl_FindExecutable, Tcl_CreateInterp, Tcl_SetPanicProc, and TclZipfs_AppHook may all be used to initialize Tcl.

This TIP proposes to mandate calling either Tcl_FindExecutable or TclZipfs_AppHook before using any other Tcl functions. This is motivated by the following considerations.

• The other functions do not fully initialize Tcl. Nor does the documentation clarify (probably because no one is sure!) as to which Tcl functions can be safely used subsequent to each initialization mode.

• From a maintenance perspective, each results in a different sequence of internal initialization, with differing initialized state, making it difficult to reason about program state.

• As a minor side effect, redundant calls are made, for example Tcl_InitSubSystems is called three times at tclsh startup.

9.0 interpreter initialization

The Tclsh initialization sequence core wiki page shows the call tree for creation of a Tcl interpreter. At a high level, this two has phases:

• Tcl_CreateInterp creates the core components of a Tcl interpreter including commands, globals, and initializing subsystems like I/O.

• Tcl_Init locates and executes Tcl's support scripts that sets up package loading, encodings and ancillary commands such as auto_exec, unknown etc.

In addition to tThis TIP proposes to streamline this latter step which is detailed below. The motivation for the proposed changes is described in a later section.

Tcl executes the following sequence to run an interpreter pre-initialization script, not to be confused with the application initialization script such as .tclshrc or .wishrc.

(1) If the application has registered a pre-initialization script via the call Tcl_SetPreInitScript that is evaluated to initialize the interpreter. The steps below are skipped.

(2) Otherwise, Tcl looks for a procedure called tclInit already defined by the application in the interpreter after the Tcl_CreateInterp call. If found, the script is run and steps below are skipped.

(3) If both (1) and (2) do not hold, Tcl defines its own tclInit procedure which searches for the init.tcl file in several directories as below and sources it. The location of the init.tcl file is also set as the value of the tcl_library global variable in the interpreter. The init.tcl script does the bulk of the work referenced earlier such as setting up auto_path, unknown, encoding directories etc.

This TIP proposes to change the list of directories that Tcl searches in this last step, the result of which is stored in tcl_library and added to the encoding search path.

In Tcl 9.0, this search is implemented by first constructing a list of directories to search. Note the complete list of candidates is constructed before checking if any particular directory contains init.tcl. Directories are added to search path list in the following order:

(3a) If the application has already set the tcl_library variable, its value is added to the search list. Moreover, the remaining search steps 3b-3j are skipped so this will be the only directory in the search path.

(3b) $env(TCL_LIBRARY), $env(TCL_LIBRARY)/tcl9.0 if env(TCL_LIBRARY) defined and not empty.

(3c) The library directory within the ZipFS archive if present

(3d) $tclDefaultLibrary if defined

(3e) [pgkconfig::get scriptdir,runtime] but only if tclDefaultLibrary was not defined.

(3f) $parent/lib/tcl9.0 where parent is [file dirname [file dirname [info name]]]

(3g) $parent/../lib/tcl9.0

(3h) $parent/library

(3i) $parent/../library

(3j) $parent/../tcl9.0/library

(3k) $parent/../tcl9.0.2/library (patchlevel)

(3l) $parent/../../tcl9.0.2/library (patchlevel)

(3j) Every element of the tcl_libPath list if it is defined

(4) Each directory in the list constructed in (3) is examined for the presence of the init.tcl file. If found, it is sourced. On success, the tcl_library variable is set to the directory path and no more directories are examined. If the init.tcl script raises an error, the iteration continues examinining the remaining directories.

(5) The init.tcl script sets up auto_path, encoding directories, unknown handlers for packages and commands, and the auto exec, auto load and auto indexing facilities.

The changes to the current behavior are driven by the following considerations:

• User configuration should not overridden. If user specifies a path, and it does not exist, raise a configuration error. Tcl should not try to guess, at least not until TIP 130 is implemented. This is akin to raising an error on invalid encoding and not silently replacing with a guessed encoding. Based on this, handling of env(TCL_LIBRARY) in (3b) should be similar to that of tcl_library in (3a) in that no further directories or subdirectories should be searched. Note this change would in fact match the documented behavior of env(TCL_LIBRARY).

• Along the same lines, limit the directories searched. Do not search grandparents and further ancestors. The hunt for a suitable directory can be taken too far and has the potential for security holes. The rationale is the same reason that system loaders restrict directories searched to those explicitly listed. As above, layouts involving grandparents etc. can be supported using pkgconfig.

• Mechanisms that are undocumented or duplicated should be removed. Accordingly, eliminate step (2) - check for the application defined tclInit. It is undocumented, not covered by the test suite and there is similar functionality available through Tcl_SetPreInitScript (though it is not exactly the same as it is global and not per interpreter). Similarly, eliminate (3j) - tcl_libPath. Again, undocumented, untested and marked OBSOLETE in the code. Applications should use env(TCL_LIBPATH) or directly set tcl_library instead.

• Tcl expects certain setup to be done by the init scripts, in particular, setting of tcl_library and encodings. Check that the application defined scripts have done this or raise an error. This avoids anomalous behaviour later that takes effort to track down.

• In (4), once init.tcl is found and evaluated, propagate any exceptions. Do not ignore them and do not continue searching for another init.tcl. To reiterate, misconfiguration should not be ignored and attempts made to "fix it up" leading to unpredictable results. Evaluating init.tcl would have already changed interpreter state.

• Eliminate version independent paths that are not explicitly included via configuration or the environment. Thus (3h) and (3i), e.g. $parent/../library, are removed. The latter leads to proven misconfiguration and head scratching as to cause of failures in the presence of multiple Tcl versions.

• Remove patchlevel based search. If at all necessary, the same can be effected through the pkgconfig mechanism (3e) above.

• Explicitly document search paths. Currently, the manpage documentation is both incomplete and incorrect.

• Re-implement the directory search and invocation of init.tcl to from script to C. This is of course solely for faster initialization of interpreters, not for functional reasons. Similarly, move the initialization of auto_path from init.tcl to C. Together these result in an reduction in interpreter initialization time of 25%.

Specification

Tcl initialization

Applications must call one of the following functions prior to making any other calls into the Tcl C API.

• Tcl_FindExecutable may be used to initialize Tcl if the application will not use the ZipFS file system to store Tcl support script files and encodings.

• TclZipfs_AppHook may be used to initialize Tcl irrespective of use of the ZipFS file system. Applications should therefore prefer its use over Tcl_FindExecutable.

Additional threads that access Tcl facilities need not call the above functions and may call Tcl_CreateInterp to create interpreters. Threads that call Tcl functions without creating interpreters beforehand must call Tcl_InitSubSystems first. This is require to initialize the Tcl notifier in a thread that has not previously called any of Tcl_FindExecutable, TclZipfs_AppHook or Tcl_CreateInterp.

Tcl_SetPanicProc does what its name suggests but does not initialize Tcl. It does set up the Tcl stubs structure for stubs-enabled applications.

Tcl_MainEx is modified to not locate the executable or initialize encodings as those tasks have already been executed by one of the above functions.

Interpreter initialization

The resulting initialization sequence with the changes above is shown below:

(1) If the application has registered a pre-initialization script via the call Tcl_SetPreInitScript that is evaluated to initialize the interpreter.

(2) Otherwise, Tcl (via the new LocatePreInitScript internal function) searches for the init.tcl file in several directories as listed in (2a)-(2f).

(2a) If the application has already set the tcl_library variable, the search is ended without checking for the presence of an init.tcl file. This follows 9.0 behavior.

(2b) Likewise, if env(TCL_LIBRARY) is defined, search is ended irrespective of the presence of an init.tcl file. This matches 9.0 documentation, not 9.0 behavior.

(2c) The library directory within the ZipFS archive if present as in 9.0

(2d) $tclDefaultLibrary as in 9.0

(2e) [pgkconfig::get scriptdir,runtime]. This is checked regardless of (2d) unlike 9.0

(2f) $parent/lib/tclVERSION where parent is [file dirname [file dirname [info name]]]

(3) If no directory was found in step (2), an exception is raised. Otherwise the tcl_library variable is set to the found directory.

(4) An attempt is made to source the init.tcl file in the directory. If the file does not exist (this can happen with (2a) and (2b) above) an error is raised. Unlike 9.0, if the found init.tcl raises an exception, it is propagated. In either error case, further directories are not searched.

(5) The init.tcl script calls the tcl::InitAutoPath internal command that sets up auto_path. This command, implemented in C, moved from init.tcl script to append to auto_path the values in env(TCLLIBPATH), tcl_library and its parent, and $parent/lib. In the case of safe interpreters, auto_path is initialized to empty if it did not already exist.

The remaining sections of init.tcl that set up handlers for packages and commands, and the auto exec, auto load and auto indexing facilities, are unchanged.

Performance notes

The following performance improvements are note in TIP 732. Note however, the bulk of the improvement comes from moving search path implementation from Tcl to C, not because of changes in the search path itself.

TIP 732:

% timerate {interp delete [interp create]}
939.971 µs/# 1063 # 1063.9 #/sec 999.189 net-ms

Current Trunk:

% timerate {interp delete [interp create]}
1261.00 µs/# 793 # 793.02 #/sec 999.973 net-ms

Implementation notes

Implementation is in the tip-732 branch.

References

1. TIP 66 describes some aspects of Tcl initialization. While almost a quarter century old, much of its content related to integration with applications still holds.

2. Tclsh initialization sequence page in the core wiki shows the call tree for tclsh startup in 9.0.2.

Copyright

This document has been placed in the public domain.