Tcl Source Code

Tcl seems only to support the //?/ dos device syntax for paths referencing a local file volume. 

Use of //?/ on non-volume paths will give erroneous 'file exists' results (possibly due to file pathtype reporting them as 'relative')

It Partially supports //./ syntax 

I know this is a longstanding issue in a curly part of the code - but I'd like to document some of the effects here.

e.g

    %file exists //?/c:/test
    1
    %file exists //./c:/test
    0 (ideally should work - especially given the examples below where //./ syntax is required for any access)

Anomalously the reverse type of must be used for other situations:

    %file exists //?/BootPartition/test
    0 (Not ok)
    %file exists //./BootPartition/test
    1 (OK - but then one would reasonably expect //./c:/test to work too)
    %file exists //?/UNC/LOCALHOST/c$/test
    0 (Not ok)
    %file exists //./UNC/LOCALHOST/c$/test
    1 (OK)

The //?/ is (for me at least) generally the more useful form - as it is the 'non normalizing' version which allows access to otherwise illegal file names for windows (e.g path segment ending in dot or space)
ie it passes otherwise illegal paths through.

As it stands - paths such as "c:/test/illegal." cannot be accessed by UNC or non-volume based dos device paths.

These so called 'illegal' paths can exist quite happily on windows NTFS - either on remote UNC paths or locally for example if a unix directory is copied over/imported.
To test create one on the local machine it can be done from Tcl via for example:
file mkdir //?/c:/test/illegal.

    %file exists c:/test/illegal. 
    0 (ok - not expected to access it via plain path)
    %file exists //?/c:/test/illegal.
    1  (ok - as expected)
    %file exists //./c:/test/illegal.
    0  (ok - dot is the 'normalizing' version of dos device path so it's actually testing c:/test/illegal without the dot)
    %file exists //./c:/test
    0 (Not really ok - if the path exists and is not 'illegal' we should be able to use the //./ path too)

    %file exists //LOCALHOST/c$/test/illegal.
    0 (OK)
    %file exists //?/UNC/LOCALHOST/c$/test/illegal.
    0 (Not ok)
    %file exists //./UNC/LOCALHOST/c$/test/illegal.
    0 (OK)
    %file exists //?/UNC/LOCALHOST/c$/test
    0 (Not ok)

Even without the requirement to access so called 'illegal' windows paths; this makes for a confusing situation regarding handling of existing dos device paths (which sometimes crop up) requiring knowledge of what Tcl can/can't handle, in order for the programmer to translate before passing to Tcl's file functions)

In Powershell - the following example shows how to compare some of this behaviour  (tolerates forward slashes without issue)

    >test-path -literalPath "//?/UNC/LOCALHOST/C$/test/illegal."
    True

>First of all: The windows path prefixes are \\?\ and \\.\, not //?/ and //./. Communicating with the win32 API, always backslashes should be used. Of cource, Tcl should handle that for you.

Yes - I prefer to use the / even in examples, because whether it's windows or Tcl doing the translating - that aspect 'just works' whereas examples using backslashes tend to have escaping complications.

I don't entirely agree with the characterisation of the . and ? dos device paths as a hack - but I do recognise that they also provide access to namespaces and devices that don't necessarily fit with the concept of filesystems - and I don't pretend to understand all the issues.

There are useful filesystem based paths such as //./Volume{<guid>}/ 
These allow access to for example removable media without a drive mapping - or for which the drive mapping is undesirable to use as it may change.
This can be handy for backup related applications for example.
(see output of mountvol command in cmd or powershell)
These can represent filesystem paths available on the system - and I had expected Tcl to be able to access them.

Whilst the MAX_PATH issue has been one of the reasons to use this syntax in the past, I think APN is right that this usecase is now less important.

The //?/ is I think also still important for support of not just trailing dots (and trailing spaces) - but also literal glob chars such as '*' or '?'

I suspect extending special handling to guess when to add the //?/ is not ideal (would have to handle aforementioned globs too?) 


If it's decided to just remove support - I think that'd be unfortunate, but perhaps could be worked around with the use of extensions like Twapi.


If anything - I think volumerelative paths should be a more likely or desirable candidate for removal - as they tend (in my opinion) to be more useful only in an interactive shell - but as Tcl doesn't support the per-volume cwd - there are already bugs in this area when changing drives in Tcl as compared to what happens in cmd.exe or powershell.

Also, perhaps not directly related, but in the area of path handling - 'open c:/test/file:streamb w' creates an extra file using some sort of substitution for the colon instead of accessing an alternate stream in c:/test/file
I can't see any way to read/write NTFS alternate data streams from tcl without resorting to external executables or extensions(?)

That 'open fileswithstream w' clobbers the alternate datastream isn't ideal either.

At the very least I'd have thought we could use the //?/ syntax to bypass whatever Tcl is doing on such paths and let windows give access to the right stream.

And while at it, please consider that file paths longer than MAX_PATH no longer need \\?\ prefixes in modern Windows systems (appropriately enabled in the registry)

First of all: The windows path prefixes are \\?\ and \\.\, not //?/ and //./. Communicating with the win32 API, always backslashes should be used. Of cource, Tcl should handle that for you.

I don't think Tcl should really support the \\?\ and the \\.\ syntax: They are a Windows hack to disable special Windows path processing. Tcl scripts should be portable. Therefore, Tcl already has some special file path processing (in the TclNativeCreateNativeRep() function) to handle this.

See: https://core.tcl-lang.org/tcl/file?name=win/tclWinFile.c&ci=d5bba172c8301ec9&ln=3074-3152

We could extend this special handling: If the path ends with '.' (but doesn't have a slash or backslash immediately before it), also add the \\?\ prefix. That would mean, Tcl scripts don't have to worry about adding the \\?\ prefix any more.

I'll have a further look.

Hope this helps, Jan Nijtmans