| Ticket UUID: | 596936dd2d55ba70dcdc980b9f58b3dccf248605 | ||
| Title: | auto_execok uses cmd in preference to cmd.exe despite path order (windows) | ||
| Type: | Bug | Created on: | 2023-11-06 07:00:40 |
| Submitter: | juliannoble2 | Assigned to: | nobody |
| Subsystem: | 38. Init - Library - Autoload | Severity: | Minor |
| Priority: | 5 Medium | Last modified: | 2026-05-21 07:01:28 |
| Status: | Closed | Closed by: | apnadkarni |
| Resolution: | Fixed | Closed on: | 2026-05-21 07:01:28 |
| Version: | 8.6+ | ||
| Description: | ||||
init.tcl's auto_execok function has for some time since 8.57 looked for executable files with empty extensions first.
Some tools such as msys2 provide various shims/scripts named like their MS counterparts either with or without a .exe extension.
With c:\windows\system32 earliest in the path - one would expect that typing cmd at a prompt would preferentially find the normal c:\windows\system32\cmd.exe - or use the $env(COMSPEC) value. For shell builtins as defined in the auto_execok function such as 'echo' - the correct cmd.exe is used (from COMPSPEC).
However - in the presence of some other file named cmd in the path - just typing cmd in the shell loads that other path into the ::auto_execs array.
This can be worked around by typing cmd.exe instead of just cmd, but it is surprising, and I'm wondering if at least for cmd - this could specifically use the COMSPEC value - as this is what is chosen for the builtins, and I can't think of a circumstance where cmd should point to a different location than its builtins.
The potential remains for other programs to be selected in an order based on extension that doesn't follow the order listed in PATH - presumably(?) it is done this way for performance so that the entire set of PATH dirs isn't searched for each extension - but I do also wonder at the reason for 'no extension' being ahead of .exe
Current search order is {foreach extension {foreach path ...}}
Generally on windows one expects the earlier entries in the path to be respected - but Tcl on windows has operated in this way for some time.
Might it be worthwhile doing a pre-scan for all extensions in the dirname of $::env(COMPSPEC) (usually something like c:\windows\system32), then, noting the position of that folder in the PATH (or giving lowest pref if not present), only replace it in auto_execs if something was found earlier in the PATH?
That proposal gives the folder of COMSPEC a special privilege - which is a bit ugly.
Rather than that, my preference would be to reverse the the loop nesting and search once through the folders but check each of the 5 extensions - as this is the least surprising in terms of how users have their PATH configured - but I don't know why the current ordering was chosen.
Some very rough testing.. I have a bit over 100 folders in my PATH - which I suspect is somewhat higher than most.
Doing a tcl glob for the 5 extensions (with a specific filename) in one call for a folder with say 50 files - takes less than 1ms
Doing the same on a large folder such as c:\windows\system32 (contains around 5k files) takes less than 3ms
both these rough tests were for non-existent names e.g
glob -nocomplain -dir $d -tail flubber flubber.exe flubber.cmd ...
A single glob like this is a bit faster than testing file exists for each one.
With my changes below - entering a non existent command returns about 4x faster (noticeable in interactive shell) and the selection respects the PATH order. Please review and consider.
set lookfor [lmap ext $execExtensions {string cat ${name} ${ext}}]
foreach dir [split $path {;}] {
# Skip already checked directories
if {[info exists checked($dir)] || ($dir eq "")} {
continue
}
set checked($dir) {}
foreach match [glob -nocomplain -dir $dir -tail {*}$lookfor] {
set file [file join $dir $match]
if {[file exists $file] && ![file isdirectory $file]} {
return [set auto_execs($name) [list $file]]
}
}
}
if 0 {
foreach ext $execExtensions {
unset -nocomplain checked
foreach dir [split $path {;}] {
# Skip already checked directories
if {[info exists checked($dir)] || ($dir eq "")} {
continue
}
set checked($dir) {}
set file [file join $dir ${name}${ext}]
if {[file exists $file] && ![file isdirectory $file]} {
return [set auto_execs($name) [list $file]]
}
}
}
}
| ||||
| User Comments: | ||||
juliannoble2 added on 2023-11-06 07:16:03:
I did another test by copying whoami.exe to the last folder in my path and renaming it to whoami123.exe
I ran unset auto_execs and then time {whoami123} a few times.
Time for first run went from around 180ms for the original version to 190ms to the new code version.
For executables earlier in the path - the new version is considerably faster than the old version - it only gets slightly worse for executables way down in the path list.
juliannoble2 added on 2023-11-06 08:59:30:
I note that it used to be this way around in 2012 and DGP changed it as shown here: https://core.tcl-lang.org/tcl/info/6909864f5655c69 with the comment: Make [auto_execok] search match [exec]. I'd like to understand more about why this was done Can anyone clarify why exec wasn't changed instead and what exactly is going on here? sebres added on 2023-11-06 11:12:53:
> Can anyone clarify why exec wasn't changed instead and what exactly is going on here? Basically no matter what was primary the goal for the change, it got implemented not quite correct. ...
Note that the shell tries all possible file extensions in a specific directory before moving on to search the next directory (if there is one). Since [6909864f5655c69] Tcl does it vice versa (tries all possible directories for a specific file extension and then start from scratch for next extension). juliannoble2 added on 2023-11-06 12:04:28:
Yes, I tested in cmd and pwsh shells and they find things in a different order to Tcl - this really needs to be fixed.
There are other demons here.
The paths retrieved by splitting on the PATH env var sometimes end with a slash.
When you do a glob -dir $dir -tail on a dir ending in a slash - it can return /name instead of just name
Then the file join fails to work as it takes the name as an absolute path and doesn't add the dir.
#with trailing slash in path
% set path {C:\Program Files\PowerShell\7}
C:\Program Files\PowerShell\7
% tcl::unsupported::representation $path
value is a pure string with a refcount of 4, object pointer at 0x1ea036f3270, string representation "C:\Program Fi..."
% append path \\
C:\Program Files\PowerShell\7\
% tcl::unsupported::representation $path
value is a utf32string with a refcount of 2, object pointer at 0x1ea036ef2e0, internal representation 0x1ea038c70b0:0x0, string representation "C:\Program Fi..."
% glob -dir $path -tail pwsh.exe
/pwsh.exe
#without trailing slash
% set path {C:\Program Files\PowerShell\7}
C:\Program Files\PowerShell\7
% tcl::unsupported::representation $path
value is a pure string with a refcount of 5, object pointer at 0x1ea036f3270, string representation "C:\Program Fi..."
% glob -dir $path -tail pwsh.exe
pwsh.exe
Whether or not a trailing slash should strictly exist at the end of a path - I believe it to be fairly common on windows (some installers do it) and it should work.
Tcl8.5 on the other hand does it correctly:
% glob -dir "c:\\" -tail windows-version.txt
windows-version.txt
versus testing in 8.6, 8.7, 9.0
% glob -dir "c:\\" -tail windows-version.txt
/windows-version.txt
juliannoble2 added on 2023-11-06 12:19:01:
Though I see I should have just file normalized that path.. so the original code snippet above needs fixing in that regard too. apnadkarni added on 2026-05-18 10:46:35:
Partial fix with TIP 753 in terms of the search path. Complete fix still pending. apnadkarni added on 2026-05-21 07:01:28:
Closing because
| ||||
