Tcl Source Code

View Ticket
Login
2026-10-21
08:47 • Closed ticket [3075027fff]: 8.5 head testsuite failures on windows plus 4 ot... artifact: aa2063ee7d user: serhiy.storchaka
2016-07-09
14:45 • Ticket [3075025fff] 8.6 head testsuite failures on windows status still Open... artifact: 7db572f768 user: apnadkarni
11:05 • Closed ticket [a47641a031]: File normalize is inconsistent with Windows junction... artifact: d128f5f619 user: apnadkarni
2016-07-08
10:30
Merge core-8-6-branch: Bug [a47641a031]. TclJoinPath was calling TclNewFSPathObj with a first argume... check-in: 4589a9d17c user: jan.nijtmans tags: trunk
08:04 • Ticket [9553668fc5] self tests filesystem-1.3, filesystem-1-4, registry-6.21 fai... artifact: cc31ceb5f3 user: apnadkarni
08:02 • Ticket [a47641a031] File normalize is inconsistent with Windows junctions (files... artifact: 8646014971 user: apnadkarni
07:58
Bug [a47641a031]. TclJoinPath was calling TclNewFSPathObj with a first argument that was not an abso... check-in: c96de4ed64 user: ashok tags: core-8-6-branch
2016-02-25
13:58 • Ticket [a47641a031] File normalize is inconsistent with Windows junctions (files... artifact: cf5e87e5ce user: dgp
06:07 • Ticket [a47641a031]: 4 changes artifact: f133748457 user: apnadkarni
06:06 • Ticket [a47641a031]: 3 changes artifact: acf4b3f93b user: apnadkarni
03:39 • Ticket [a47641a031]: 4 changes artifact: b064665573 user: apnadkarni
2016-02-08
11:23 • New ticket [a47641a031]. artifact: 6006ac81c0 user: danckaert

Ticket UUID: a47641a031858cceccf92c9a6da34406b35fed77
Title: File normalize is inconsistent with Windows junctions (filesystem-1.3,4)
Type: Bug Created on: 2016-02-08 11:23:20
Submitter: danckaert Assigned to: dgp
Subsystem: 36. Pathname Management Severity: Important
Priority: 5 Medium Last modified: 2016-07-09 11:05:13
Status: Closed Closed by: apnadkarni
Resolution: Fixed Closed on: 2016-07-09 11:05:13
Version: 8.6.5
Description:
On Windows, [file normalize] gives inconsistent results through junctions:

% cd c:/tmp
% file link foo ../windows
../windows
% file readlink foo
C:\Windows
% file normalize [file join foo bar] ;# this gives the wrong result
C:/Tmp/foo/bar
% file join foo bar
foo/bar
% file normalize foo/bar ;# this gives the right result
C:/Windows/bar

There are already several tickets about junctions in Windows (mostly related to access rights), but his one seems more severe since it involves an EIAS violation.
User Comments:
apnadkarni added on 2016-02-25 03:39:47:
Same issue is also demonstrated by tests filesystem-1.{3,4}. Also this issue does not show up if the link already existed (as opposed to being created within the same process via file link).

apnadkarni added on 2016-02-25 06:06:59:
There is a bunch of code in the execution path [file normalize [file join ...]]
that I find dubious but the root cause of this specific bug is the special
optimization case in TclJoinPath starting at line 874 in tclPathObj.c

After a bunch of checks to ensure certain criteria are met (for example,
only two elements to join), the special case code calls TclNewFSPathObj
to construct the joined path and returns the result. However, as
indicated by the comments and reflected in the code, TclNewFSPathObj
expects the first argument to be an *absolute* path which in the test
case example, it isn't. The returned object's FsPath intrep is essentially
marked as not needing normalization (the TCLPATH_NEEDNORM flag is not set)
though the cwdPtr field is NOT normalized or absolute.

Later, Tcl_FSGetNormalizedPath skips over the crucial normalization code
thinking the FsPath intrep is already normalized. Hence the wrong result.

Options are

- disable the TclJoinPath special case. I've tested that this fixes both
the reported bug as well as the failing test cases. I don't know how
much of a performance impact this has in real life. The time command
does not show any appreciable difference for the file join itself.

- disable the optimized code only for Windows since it appears this bug
does not manifest on Unix. I'm actually curious as to why since it is common
code. My guess is that the file system specific call for normalization does
something different on Unix.

- Add a check to special case criteria that the first argument is indeed
an absolute path.

In addition to fixing file join as above, the question also arises whether
file normalize should even use cached results at all. IMHO semantics of
normalize should require a visit to the filesystem at the time the call
is made. Otherwise, how does it take into account a link created (possibly
even from outside the program itself) after a path is cached?

For the same reason, one might argue that the file link command should
increment the global file system epoch when a link is created to
invalidate all cached representations. This is exactly as what is
done when env(HOME) changes or file system mounts are changed
(see comments in tclioutil.c)

apnadkarni added on 2016-02-25 06:07:37:
Assigned to dgp for comments.

dgp added on 2016-02-25 13:58:31:
I'm happy that someone is looking into this.

On the analysis, I cannot comment on the fine
details, but all the general comments are correct.

I wonder if this other issue is playing any role

http://core.tcl.tk/tcl/tktview?name=961646

I have no confidence that I will be able to confirm
any of the fixes proposed under time pressure in a
way that I'm sure will not risk breaking something
at least as important as what it fixes.  Thus, I
would postpone pushing this fix to trunk until 
after the 8.6.5 release.

apnadkarni added on 2016-07-08 08:02:30:

Fixed in core-8-6-branch by check-in [a47641a031]. TclJoinPath will now only use the optimized branch if the first argument is an absolute path as required by TclNewFSPath.

I'm still of the opinion that file normalize should not use cached information and that file link should update the file system epoch for the reasons listed in a previous comment. But those are separate issues from this fix.