| Ticket UUID: | 931847 | ||
| Title: | TclFSNormalizeToUniquePath - incomplete? | ||
| Type: | Bug | Created on: | 2004-04-08 16:57:06 |
| Submitter: | dgp | Assigned to: | vincentdarley |
| Subsystem: | 37. File System | Severity: | |
| Priority: | 5 Medium | Last modified: | 2004-04-26 22:49:59 |
| Status: | Open | Closed by: | |
| Resolution: | None | Closed on: | |
| Version: | obsolete: 8.5a2 | ||
| Description: | ||||
As I understand TclFSNormalizeToUniquePath() it's meant to force paths into one canonical form -- getting rid of symlinks, establishing the case in the filesystem, etc. To do this, it asks the filesystems to do the work, giving the native filesystems first try, then giving all the others a try, calling the Tcl_FSNormalizePathProc of each, if implemented. Each Tcl_FSNormalizePathProc(), though, might not be able to finish the job completely. The "startAt" value returned by TclFSNormalizeToUniquePath() might not point to the end of the returned string value of the path. This is fine, as long as the callers of TclFSNTUP() repeatedly call it until startAt is at the end of the string, or until nothing changes. It appears the existing callers of TclFSNTUP are calling only once, not checking the return value, and expecting TclFSNTUP to have done all the normalization possible. The alternative to changing the callers, is to rewrite TclFSNTUP itself to do what the callers appear to expect, keep looping over normalization attempts until they don't do any more, so the path is as normalized as possible. This bug is difficult to demostrate because it requires a non-native VFS that actually implements a Tcl_FSNormalizePathProc() that would produce a changed path that, say, the native filesystem could normalize further. For example, imagine chaining symlinks into and out of a non-native VFS. | ||||
| User Comments: | ||||
dgp added on 2004-04-09 00:00:55:
Logged In: YES user_id=80530 A related issue: The documentation for new filesystems should more explicitly warn that any Tcl_FSNormalizePathProc implementation might be called on any path whatsoever, not only the ones the filesystem thinks belong to it. dgp added on 2004-04-09 00:04:17:
Logged In: YES user_id=80530 Ack! Expanding a symlink might well introduce new ../ and ./ components to the path, so perhaps the callers must deal with this after all? vincentdarley added on 2004-04-09 00:24:16:
Logged In: YES
user_id=32170
Wow, you've been looking very closely!
I didn't design the code to handle symlinks between
filesystems, although I suppose I didn't actually document
that anywhere. I certainly agree that that would
demonstrate bugs in the existing code.
In particular, however, the basic code assumption is that
the normalization can be achieved by (i) asking the native
filesystem to normalize, then (ii) asking each other
filesystem to normalize exactly once each.
Clearly if filesystems can be nested arbitrarily (e.g.
through your symlink approach), or more simply:
C:/foo/bar.zip/ZIP/foo.kit/KIT/blah.zip/ZIP
Assuming we have a separate .zip and .kit filesystem, each
of which have decided they are case-insensitive and that all
normalized paths should be lower case. Then the above
should be normalized first by the OS (C:/foo/bar.zip) then
by the zipfs (/ZIP/foo.kit) then by the kitfs (KIT/blah.zip)
then by the zipfs again (ZIP).
In the current code structure this would not work.
I guess the correct solution is to first ask the nativefs,
and then, for the filesystem which accepts the path which
has been normalized so far, ask it to normalize some more.
When no filesystem accepts the path, then just stop. So,
we'd have:
Overall path accepted by native fs.
C:/foo/bar.zip normalized by native, leaving 'startAt' at
following '/'
C:/foo/bar.zip/ recognised by zipfs
C:/foo/bar.zip/zip/foo.kit normalized by zipfs.
C:/foo/bar.zip/zip/foo.kit/ recognized by kitfs
etc.
So (pseudo-code):
while (1) {
fs = PathInFilesystem(path, up to 'startAt');
if (fs == null) break;
startAt = fs->normalizePath(path, startAt);
}
with a bit of checking to avoid infinite loops.
This would even solve your symlink issue, I think, although
I haven't thought what it means for 'dir/..' handling.
dgp added on 2004-04-09 00:34:03:
Logged In: YES user_id=80530 FWIW, an arbitrarily nestable filesystem is exactly what I'm working on. I think you've got the right basic idea, on how to remove this limitation, but there may be trouble with PathInFilesystem(). Examining PathInFilesystem() is what got me started. This routine gets passed an arbitrary path, and must decide whether or not to claim it. Generally this can be a hard problem, so it's natural to call Tcl_FSGetNormalizedPath() on the path passed in to have something more predictable to work with. That's fine as long as a working Tcl_FSGNP doesn't depend on a working PIF(). If so, we'll just pass the buck forever. dgp added on 2004-04-09 00:39:46:
Logged In: YES user_id=80530 Now, I'm looking at TclFSNormalizeAbsolutePath() and it appears to have a lot of overlapping function with TclFSNormalizeToUniquePath(). In particular it appears to be trying the same symlink removal. that I understood TFNTUP to be doing. Any prospect for re-factoring some of this so each operation is done in only one place? dgp added on 2004-04-09 01:51:50:
Logged In: YES user_id=80530 OK, I think I've assured myself that Tcl_FSGetNormalizedPath() is safe for the PathInFilesystem() routine to call, so long as the filesystem has its NormalizePath() routine defined to handle any path (if the filesystem has any normalizing to do), and the Separator() routine is defined. vincentdarley added on 2004-04-09 16:31:26:
Logged In: YES user_id=32170 One relevant issue is that TFSNToUniquePath will only want to call PathInFilesystem() for paths that it knows are normalized up to a certain point, (and only wants to know if the path up to that point is in the given fs). I.e. in my example below, we'll want to call PathInFilesystem(C:/foo/bar.zip/), and we know that that path is already normalized, which means we can use 'TclFSMakePathFromNormalized' -(we don't actually want the trailing '/', but that's a detail). This means that we will actually want to construct a new (!) pathPtr containing just this path, which we actually create in the 'pure-normalized' form. Unfortunately, the moment it has been recognized, we throw it away. This means in your arbitrarily nested filesystem, normalization will be a not-stunningly efficient process (but perhaps that doesn't matter too much). This suggests in the future we might want to extend the FS interface to provide a more flexible pathInFilesystem proc (Note: we have a private FS_VERSION_2 we can play with here). Finally, concerning the overlap between the Absolute/Unique routines, yes that is an issue. I should clarify that the Absolute routine is there _not_ to remove symlinks, but rather to remove './' and '../' sequences (and this may result in removing _some_ symlinks, if they are before a '../' sequence). Given your desire to go down the path of arbitrarily nestable/symlinkable filesystems, I think a result of that is we will have to refactor the two functions -- since, in particular, '../' sequences can be re-introduced. p.s. I'm largely away for the next week. dgp added on 2004-04-26 22:49:59:
Logged In: YES user_id=80530 Reminder: when this gets sorted out, look again at Tcl Bug 941108. | ||||
