Tcl Source Code

View Ticket
Login
2026-03-31
05:29 • Closed ticket [108904173c]: glob and file normalize inconsistencies on case inse... artifact: c7e82bac4d user: apnadkarni
2026-03-27
11:16 • Open ticket [108904173c]. artifact: 27f335fe8d user: apnadkarni
2026-03-25
06:48 • Closed ticket [108904173c]. artifact: d7a08b6fa0 user: apnadkarni
06:46 • New ticket [a5ba717243] glob should correctly handle case-sensitive paths on cas... artifact: 58ec856855 user: apnadkarni
05:51
Fix [108904173c] - glob+file normalize inconsistency on Windows check-in: e9168f591b user: apnadkarni tags: trunk, main
05:34
Fix [108904173c] - glob+file normalize inconsistency on Windows check-in: 36e54d2473 user: apnadkarni tags: core-9-0-branch
2026-03-22
15:30 • Ticket [108904173c] glob and file normalize inconsistencies on case insensitive ... artifact: d454c96489 user: apnadkarni
15:11
Bug [108904173c]. Failure to normalize path on Windows. check-in: 84efde4746 user: apnadkarni tags: bug-108904173c
03:34 • Ticket [108904173c] glob and file normalize inconsistencies on case insensitive ... artifact: 6fe05f2c89 user: apnadkarni
2026-03-16
13:05 • Open ticket [108904173c]. artifact: 72658b00bc user: sebres
12:12 • Ticket [108904173c]: 5 changes artifact: 649ee29137 user: juliannoble2
10:43 • Closed ticket [108904173c]. artifact: d81d88f961 user: sebres
05:47 • Ticket [108904173c]: 3 changes artifact: 3c6492178a user: juliannoble2
05:45 • Ticket [108904173c]: 3 changes artifact: 46f25ab6a1 user: juliannoble2
04:22 • Ticket [108904173c]: 3 changes artifact: cf07325ebe user: juliannoble2
04:01 • New ticket [108904173c]. artifact: 5b8b36c28a user: juliannoble2

Ticket UUID: 108904173c1a823a20823885c7b29ab536434a5a
Title: glob and file normalize inconsistencies on case insensitive filesystem
Type: Bug Created on: 2026-03-16 04:01:05
Submitter: juliannoble2 Assigned to: nobody
Subsystem: 37. File System Severity: Important
Priority: 5 Medium Last modified: 2026-03-31 05:29:59
Status: Closed Closed by: apnadkarni
Resolution: Fixed Closed on: 2026-03-31 05:29:59
Version: 8.6,9
Description:
Initial setup is a folder on an NTFS filesystem C:\SDX
file in that folder is sdx1.bat
Platform is windows 11.

When globbing for that file there is a difference in the results depending on whether any glob chars are used, and the result of a 'file normalize' on a result that didn't use glob chars doesn't normalize to the case of the underlying file.

% set files [glob -dir C:/SDX SDX1.bat {[S]DX1.bat} SdX1.bat*]
C:/SDX/SDX1.bat C:/SDX/sdx1.bat C:/SDX/sdx1.bat

%file normalize [lindex $results 0]
C:/SDX/SDX1.bat

I expected to get the result cased as per the filesystem.
To get it to normalize to the underlying filesystem's case, taking a string range works, but other operations such as string cat don't.

%file normalize [string range [lindex $results 0] 0 end]
C:/SDX/sdx1.bat


Due to the inability to normalize to get underlying case - I was tempted to use redundant globbing with square brackets for the case where user input had no glob chars.
ie use the {[S]DX1.bat} pattern if user entered SDX1.bat.

This works on windows - but then doesn't work for the same filesystem mounted on another platform:

(in this case using WSL 2 on same windows machine to test - but presumably would affect a real unix-like system with NTFS filesystem mounted)

%glob -dir /mnt/c/SDX SDX1.bat
/mnt/c/SDX/SDX1.bat

This result passes file exists test - but also isn't normalizable to get the actual underlying case - as presumably file normalize is behaving based on platform rather than attached filesystem properties.

%glob -dir /mnt/c/SDX {[S]DX1.bat}
(no result)

I feel either neither of these should return a result or both should.


I would expect either glob to always return a result that matches the actual filesystem case, or if not, for it to be normalizable without 'string' shennanigans to the underlying case.

(I'd half expect 'file normalize /mnt/c/SDX/SDX1.bat' to normalize to /mnt/c/SDX/sdx1.bat on a unix platform with attached case-insensitive filesystem - but I think that assumption isn't supported in the docs. This behaviour seems like a shortcoming or perhaps a design decision)

For the situation where the case-insensitive filesystem is mounted on a platform that isn't normally used with case-insensitive filesystems - I'd ideally expect it to be able to glob for patterns independent of case on that filesystem - but at the very least I wouldn't expect different results from patterns SDX1.bat vs {[S]DX1.bat}

I'm not sure if this is 1 or 2 bugs. I haven't been able to fully work out from docs what the intended behaviours are.
User Comments:
juliannoble2 added on 2026-03-16 04:22:02:
There is also an anomaly with the curly brace glob mechanism:


  % set results [glob -dir c:/SDX {SDX?.BAT} {SDX*.bat} {SDX[1].bat} {SDX\1.bat} {SDX{1,2}.BAT} SDX1.BAT] 
    c:/SDX/sdx1.bat c:/SDX/sdx1.bat c:/SDX/sdx1.bat c:/SDX/sdx1.bat c:/SDX/SDX1.BAT c:/SDX/SDX1.BAT

sebres added on 2026-03-16 10:43:34:

As already said in chat, it works exactly as expected.

The FS on windows is case-insensitive...
The uniqueness is if $path is equal $path and only $path...
Using case-insensitive comparison operations (e. g. [string equal -nocase]) this equality has been proven.

The "anomaly with the curly brace glob mechanism" from your comment is also not the anomaly, if one consider that the uniqueness is given only in a case-insensitive way (-nocase). The values are different, because windows API for glob cannot handle curly braces directly, so it happens on tcl-side, and since the file exists, the return value is different. The picture would change again, if you would try it with a wildcard like {SDX{1,2}.*}, because then it'd try curly braces against the listing from iterating API calls.

Just note that because uniqueness is guaranteed (in case-insensetive manner) and iterating API calls are time-expensive and costly, glob would avoid iterations unless necessary.

The normalization has also nothing to do with the case-sensitivity...
It is about resolving of ".." segments, relative or volume-relative paths, links, wrapping backslash to slash, etc... nothing else.

Regarding WSL, it also works as expected, because the case-sensitivity of mount is different to case-sensitivity of origin file system.
Therefore the exact glob for {SDX1.bat} find the file, but for {[S]DX1.bat} or some wildcard the glob must iterate over directory listing returned from FS, and in case-sensitive mount point the file simply doesn't exist, because it is "sdx1.bat". The issue of "anomaly" is therefore

Please also consider https://learn.microsoft.com/en-us/windows/wsl/case-sensitivity

I'll close this as incorrect, sorry.


juliannoble2 added on 2026-03-16 12:12:21:
Tell me is this behaviour reasonable? (strong opionion: it's not)


  C:/Users/sleek/AppData/Local/Apps/Tcl90b3/bin
  % set files [glob -dir C:/SDX SDX1.bat SDX1.bat]
  C:/SDX/SDX1.bat C:/SDX/SDX1.bat
  % lassign $files f1 f2

  % file normalize $f1
  C:/SDX/SDX1.bat
  % file normalize $f2
  C:/SDX/SDX1.bat

  % file normalize [string range $f1 0 end]
  C:/SDX/sdx1.bat

  % file normalize $f1
  C:/SDX/sdx1.bat
  % file normalize $f2
  C:/SDX/SDX1.bat

This is a situation that script programmers can't be reasonably expected to reason about to produce reliable code.

I wasn't telling you where the bug is exactly - and I'm willing to accept guidance that for reasons of practicality some things have to be the way they are - but at the very least (even without the 'string range' changing the result) this is an issue of serious underdocumentation of *highly surprising* effects and should at least be left open for others on the team to consider if documentation can be improved around this.

sebres added on 2026-03-16 13:05:59:

Yes, the behavior is reasonable, if you'd forget the case-sensitivity thing (you trying persistently to consider case sensitivity there, where it does no matter).

Regarding normalization from string and "correct" value, I guess one could indeed try to optimize it somehow (e. g. return translated path object instead of normalized path object from glob on windows, so that [file normalize] would be more "consistent" as for case-sensitivity)...

What would be definitely worse to make such kind of implicit "normalization" for EVERY hit in the result of glob by default, because this can cause large performance degradation (by large directory listing), or issues like [02d5d65d70].


apnadkarni added on 2026-03-22 03:34:24:

I'm inclined to agree with Julian there is a bug. Somewhere.

I will ignore cross-mounting etc. because that only confuses me further. Just sticking to Windows and NTFS, the following does not seem consistent or correct.

% set files [glob -dir d:/src RUFF]
d:/src/RUFF
% file normalize [lindex $files 0]
D:/src/RUFF
% file normalize D:/src/RUFF
D:/src/ruff

The unique and case-dependent path terms in the file normalize manpage clearly would lead one to expect those last two commands to return the same value.

The bug manifests in file normalize as glob makes no claim that it returns normalized paths (and nor should it). However, it's possible the cause of the bug is the internal representation being marked as "normalized" by the glob implementation when in fact it isn't (pure conjecture).


apnadkarni added on 2026-03-22 15:30:15:

Proposed fix in bug-108904173c. This may be both incomplete and non-optimal.

The basic issue was that the internal representation generator that checks whether a path needs normalization makes the determination purely based on whether a path component contains . or ... However, normalization as defined in the documentation also requires the path to match the case of the on-disk file entry. This happens automatically on Unix which is case-sensitive but not Windows. Thus just checking for . or .. does not suffice on Windows.

Therefore on Windows, the internal representation is always marked as needing normalization in the branch as a conservative approach as it is not known whether the appended path is exactly the on-disk string. This is non-optimal because when globbing a directory and retrieving the actual name from disk, the passed string is in fact normalized. So the next step would be to pass that information down to the function creating the internal representation so normalization can be skipped for path components read from disk.


apnadkarni added on 2026-03-25 06:48:21:

Fixed in [38f0ce657c] for "native" file systems on Windows.

The situation of case-sensitive file systems (or directories) on case-insensitive platforms remains unresolved. I've added a5ba717243 to track that as a separate issue.


apnadkarni added on 2026-03-27 11:16:04:

Re-opening because although fixed for Windows, apparently the bug also shows up on macOS as exhibited by additional tests added. I do not have or use a mac so someone with access will have to follow up. Was not even aware that macOS also has case-insensitive file system.

==== filesystem-11.1 Bug 10890417 - case of tail is correct after normalization FAILED
==== Contents of test case:

    tcltest::makeFile "" mIxEdCaSe.TxT
    lmap file1 [glob [tcltest::temporaryDirectory]/mixedcase.txt  [tcltest::temporaryDirectory]/MIXEDCASE.TXT  [tcltest::temporaryDirectory]/mIxEdCaSe.TxT  ] {
                        file normalize $file1
                    }

---- Result was:
/Users/runner/work/tcl/tcl/unix/mixedcase.txt /Users/runner/work/tcl/tcl/unix/MIXEDCASE.TXT /Users/runner/work/tcl/tcl/unix/mIxEdCaSe.TxT
---- Result should have been (exact matching):
/Users/runner/work/tcl/tcl/unix/mIxEdCaSe.TxT

To summarize the bug, file normalize should, as per documentation, normalize the file path to reflect the name on disk. This does not seem to happen when normalizing the result of glob when no special characters are present. file normalize [glob MIXEDCASE.TXT] should return mIxEdCase.Txt if that is the actual file name. Instead it returns MIXEDCASE.TXT.


apnadkarni added on 2026-03-31 05:29:59:

Closing because this is a broader macos glob-ing issue now tracked in e6ca0b1b67