TIP 753: Windows exec and auto_execok reform

Login
Core Home Timeline Branches Tags Forum Help 
Author:             Ashok P. Nadkarni <[email protected]>
Tcl-Version:        9.1
State:              Final
Type:               Project
Vote:               Done
Vote-Summary:       Accepted 3/0/0
Votes-For:          AN, HO, DKF
Votes-Against:      none
Votes-Present:      none
Created:            2026-04-23
Tcl-Branch:         tip-753

Abstract

This TIP proposes changes and enhancements to auto_execok and exec on Windows and fixes some inconsistencies. Behavior on non-Windows platforms is mostly unaffected except for removal of caches.

See related tickets 4ab2ccc4a2, 1244733, e21fc32c2a, 1c6f6503d4, 4dc35e0c0c, 596936dd2d, 093353fedb and 4fae07ea8a.

Specification

Changes to auto_execok

Support for App Paths (Windows only)

Registered applications on Windows can be invoked directly without being present in the PATH. This is the modern recommended way to be able to invoke applications from outside their installation directory and followed by major applications. In 9.0, Tcl fails to find these applications.

% exec firefox
couldn't execute "firefox": no such file or directory
% auto_execok firefox
%

This TIP proposes to enhance auto_exec to support such applications.

% auto_execok firefox
{C:\Program Files\Mozilla Firefox\firefox.exe}

One limitation of this support is that while the underlying Windows mechanism allows modification of the process environment passed to the child process, this is not possible due to the lack of such a feature in exec to pass a different environment to the child.

Handling of PATHEXT extensions (Windows only)

The PATHEXT environment variable on Windows lists the filename extensions that are searched when looking for a program to execute. Currently auto_execok looks up this environment variable. For extensions other than ".com", ".exe", ".cmd" and ".bat", the returned command is not in a form that can be executed by exec directly. For example, the command

exec {*}[auto_execok myscript.vbs]

will fail.

This TIP fixes this by modifying auto_execok to return the command template in the registry for the extensions listed in PATHEXT. The format of the command template is ill-defined in practice, e.g. spaces in the executable path are not always quoted. Therefore certain heuritics will be employed for parsing the template and performing substitutions on the template before returning it.

Because the substitutions are based on the template, paths returned may be in native form (\ separators). This is unlike 9.0 by design as not all Windows applications support / separators for paths. Note that 9.0 does not make any claim regarding returned paths being in either native or canonical form so this is not regarded as an incompatibility.

One limitation is that the mechanism only works when all arguments are trailing arguments and not interspersed with constant tokens. This is a limitation of the Tcl's auto_exec and exec interface, not specific to this TIP.

This is also really a bug fix. However, it is change in behaviour in that auto_execok for an extension in PATHEXT will now return a list that can be exec'ed rather than the path to a file. Applications (unlikely) that relied on this will break.

Elimination of auto_execs cache (all platforms)

Currently, the auto_execok implementation maintains a cache mapping to resolved commands. This cache does not account for changes that impact resolution, such as the working directory, PATH environment variable etc. resulting in bugs.

The following sequence is just one illustration based on the ticket.

% file exists foo.bat
1
% auto_execok foo
./foo.BAT
% cd ..
% file exists foo.bat
0
% auto_execok foo
./foo.BAT
% auto_reset
% auto_execok foo
C:/Bin/foo.BAT

The example above demonstrates the stale values returned from auto_execok after the directory is changed. An intervening auto_reset is required for the code work correctly. Most applications are going to be unaware of this pitfall and code may not be even aware some other module or component has changed the directory (or PATH, created files etc.). Further, requiring a auto_reset to guard against this defeats the whole purpose of the cache.

Normalizing the returned path does not help as it creates complementary issues.

It is possible that enhancing the cache to be keyed off all factors, the passed name, the working directory, the PATH environment variable might mitigate this but not deemed worthwhile as caching does not help with the real expense which is the exec process creation.

The TIP therefore eliminates this cache. Applications can cache results themselves if they wish.

this cache is publically documented as auto_execs in the library.n manpage even though its content is not described and the variable is not always defined. Attempts to access this cache will result in an undefined variable error being raised.

% info exists auto_execs
0

Changes to auto_exec search path (Windows only)

The search directories examined by auto_execok is consistent with that used by exec. See that section for details. Note this consistency only applies to the search directories. In other respects, exec and auto_execok continue to differ by design and intended purpose.

Changes to exec

Changes to exec search path (Windows only)

In 9.0 and earlier, the exec search path, by design, prioritized the order of extensions over the order of directories in the PATH environment variable. For example, given files C:\a\b\foo.bat, C:\x\y\foo.exe and PATH of C:\a\b;C:\x\y, exec foo would invoke C:\x\y\foo.exe as .exe was prioritized over .bat. This is inconsistent with both Unix and Windows user expectations.

As per this TIP, directories will be searched in the order listed below. In each directory, files will be checked in the order of the passed file name followed by the name with extensions ".com", ".exe", ".bat" and ".cmd" in that order (this is the same order of extensions as 9.0).

The directory search order, consistent with auto_execok, depends on whether the the passed file path contains directory components or not.

If the passed file path contains directory components, only the current directory is included in the search. This differs from 9.0 but is consistent with Unix.

Otherwise, directories are searched in the following order:

NOTE

Unlike auto_execok, exec in 9.0 only searches for exe, com and bat. The extensions in PATHEXT are not examined. This will not be changed for 9.1 either as exec is a mapping into CreateProcess while auto_execok maps into Windows' more general notion of a program or document.

Conditional inclusion of the working directory in the search path

Microsoft has two independent mechanisms for controlling whether the working directory should be included in the search path for executables. The older one is based on the SafeProcessSearchMode registry setting and available since Windows XP. The newer API, NeedCurrentDirectoryForExePathW, available since Vista, controls inclusion of the current directory through the environment variable NoDefaultCurrentDirectoryInExePath. Setting this environment variable does what its name implies.

This TIP uses the NeedCurrentDirectoryForExePathW API to determine whether the current directory is implicitly included the list of directories searched. Setting the the NoDefaultCurrentDirectoryInExePath environment variable will result in the current directory not being included in the directories searched as listed in the previous section. Note that if the current directory, ., is explicitly included in the PATH env, it is not affected by this setting.

Changes to file executable (Windows only)

Windows executables are not compelled to have a .exe extension. This TIP modifies the file executable command, which is documented to check the extension(s), to also return true if the content is a PE32 executable format.

Implementation notes

Implementation is in the tip-753 branch.

Change log

2026-05-07

2026-05-06

2026-05-05

Copyright

This document has been placed in the public domain.