Tcl Source Code

My apologies.. It was an old 8.7 version I setup in 2023 - but I didn't recall that I'd added a file normalize for it in library/init.tcl's auto_execok

Nowadays I extend auto_execok at runtime in my application code for this and for fixing 'reparse point' problems with windows store apps - so my other installations should be the 'vanilla' tcl versions I thought this one was.

Yes, I agree.
I first tried my overridden auto_execok with some auto_resets added to avoid cacheing - but broke everything for the auto_load reason you mentioned!

Then using unset -nocomplain ::auto_execs  I'm pretty happy with the result.

I have about 90 dirs in my path - so I get a perceivable delay if I fat-finger a command and it goes out to search for a non-existant one (~200ms) 

This is no big deal for that case - and also no big deal for my codebase - where there are a few places I can manually cache some paths as you suggest.

It does however unmask an issue, which I now realise is why I was playing with init.tcl in the first place back in '23 

The auto_execok searches foreach extension and then nests foreach directory in the path - Which would make sense if the expectation is that when you type foobar - you must test all paths for that before matching things like foobar.exe or foobar.bat, but I don't *think* that's the way things should work on windows.

If I have a path set as a high precedence - I expect it to match the first executable with that base in the earliest path - not match some exact match later down.
e.g This has caused me some frustrations in the past with Tcl trying to run various inappropriate things such as a bash script named cmd (provided by msys for some reason)

With the auto_execok caching disabled - it's now more apparent that this scanning the entire path list repeatedly for each extension in order is occurring.
As .bat is later in list of extensions - there is a noticable delay typing something vs something.bat

The issue will be worse if/when auto_exec is updated to handle other items commonly in PATHEXT such as .RB .PY 
I think it would be reasonable for auto_execok to determine the right script engine for registered extensions and call them - but perhaps that's a bug/RFE for another time.
(just returning "<resolvedpath>/cmd.exe /c <name>" works for these and those obtuse 'reparse point' based windows installs - but perhaps there's a better way)

At any rate - on my system, with caching disabled - it's now getting near one second to [auto_execok someuniquescriptname] if it's someuniquescriptname.py

From my experiments with cmd.exe and powershell given a non-suffixed command, they both give precedence to the path order first. ie opposite of Tcl's current slow auto_execok method of scanning.
I don't know if this was done deliberately to try to match some cross-platform behaviour(?) - but it has always violated my expections on windows.

Below is my tweak for the searchorder in init.tcl.  It seems to fix the bug in this ticket, improves the uncached auto_execok speed for unsuffixed commands, and provides a result more inline with how windows commandlines do it. 


    set lookfor [lmap ext $execExtensions {string cat ${name} ${ext}}]
    foreach dir [split $path {;}] {
        set dir [file normalize $dir]
        # Skip already checked directories
        if {[info exists checked($dir)] || ($dir eq "")} {
            continue
        }
        set checked($dir) {}
        
        foreach match  [glob -nocomplain -dir $dir -tail {*}$lookfor] {
            set file [file join $dir $match]
            if {[file exists $file] && ![file isdirectory $file]} {
                return [set auto_execs($name) [list $file]]
            }
        }
    }

Versus the original:

       foreach ext $execExtensions {
        unset -nocomplain checked
        foreach dir [split $path {;}] {
            # Skip already checked directories
            if {[info exists checked($dir)] || ($dir eq "")} {
                continue
            }
            set checked($dir) {}
            set file [file join $dir ${name}${ext}]
            if {[file exists $file] && ![file isdirectory $file]} {
                return [set auto_execs($name) [list $file]]
            }
        }
      }

>The resolution from PATH is normalized because the PATH elements were normalized. Duh.

Except that auto_execok has the line:
 set path "[file dirname [info nameof]];.;"

before it adds PATH elements to $path - and $path items then aren't normalized.

I wonder a bit at the executable directory coming before the CWD.
My expectation would be for the CWD to take precedence - but perhaps there's a good reason for it that I'm not considering.

I thought this felt a bit familiar.
I raised the search order in : https://core.tcl-lang.org/tcl/tktview/596936dd2d55ba70dcdc

I think in combination with APN's suggestion of making caching the responsibility of the application - this could resolve a fair chunk of the issues.
(cache wouldn't be so critical if search order matched windows mechanisms and was therefore faster)

To me it feels more like just bug-fixing to get sane and consistent behaviour - but perhaps the cache aspect is different enough that TIPPING is required.

Having rethought this a little and played with various options - I think the main problem is from attempting to resolve plain (no path separator) commands in the CWD.

Although cmd.exe does resolve cmdfoo first in the current directory, unix and powershell both require ./cmdfoo if one wants to run what's in the current directory.

Perhaps from muscle-memory in cmd.exe I'm used to that and found it quite convenient - but microsoft has moved away from it (apparently citing security concerns) and the rest of the world don't seem to do it.

I believe the argument goes that interactive shell users are likely to type common commands such as ls, grep etc whilst looking around various directories - and a) expect the command resolved in the path to be always the one used, b) that theoretically a malicious executable binary could somehow lie in wait in a folder that a user then runs by accident.
I'm not sure how valid the security argument is there - but it seems vaguely plausible.

If Tcl behaved similarly - we could retain the benefits of caching and avoid the anomalies.

Closing because I think the issues raised are addressed in [de961e8d] for 9.1 via TIP 753. There are several issues reported, I am not sure all are fixed in the exact manner suggested but with defined behavior specified in 753.

No plans to backport.

Please re-open if you some issue is still pending or is not resolved satisfactorily.