Tcl Source Code

View Ticket
Login
Ticket UUID: 4ae42446ab6ae605bbe5aaf5cb5905b499ce9794
Title: overlaid zipfs mounts co-mingle directory contents
Type: Bug Created on: 2023-09-09 17:44:12
Submitter: apnadkarni Assigned to: nobody
Subsystem: - New Builtin Commands Severity: Minor
Priority: 5 Medium Last modified: 2023-09-10 10:59:42
Status: Open Closed by: nobody
Resolution: None Closed on:
Version: 8.7
Description:

Easiest described with an example. test.zip contains the directory testdir with the file test2. teststored.zip contains a file abac-repeat.txt.

% zipfs mount //zipfs:/test ../tests/zipfiles/test.zip
% glob -tails -dir //zipfs:/test/testdir *
test2

% zipfs mount //zipfs:/temp ../tests/zipfiles/teststored.zip
% glob -tails -dir //zipfs:/temp *
abac-repeat.txt

Now instead if we mount teststored.zip on a path inside the test.zip archive, we get

% zipfs mount //zipfs:/test/testdir ../tests/zipfiles/teststored.zip
% glob -tails -dir //zipfs:/test/testdir *
test2 abac-repeat.txt

As seen above, //zipfs:/test/testdir now shows files from both archives. While one might be tempted to call this a "feature", I believe it is purely accidental. For one, afaik this is not how most operating system mounts behave. Second, when duplicate names are present, one of the two are hidden depending on order of mounting and are not recovered when the overlaid archive is unmounted, which is confusing.

In my mind, the code should be changed to either

  1. Disallow mount points within mounted archives, or

  2. Disallow mount points if the mount point is an existing directory (may be empty directories could be allowed)

User Comments:
griffin added on 2023-09-09 18:37:07:

FYI: Not to discount this issue, but there does exist "overlay filesystems" that intentionally mingle files from 1 or more "overlays". The intent is to allow access to a collection of files and also allow writing new or modifying exiting files to the same directory, without actually modifying the underlying "pristine" content. This technique can be used to share a base set of files with multiple processes where each process may add or modify files to the same directory. The commingling of files in one process does not interfere with the other process if each process has a separate dependent overlay.


juliannoble2 added on 2023-09-09 21:00:19:
Regarding "Disallow mount points if the mount point is an existing directory (may be empty directories could be allowed)"

I would like to gently argue against this restriction. 

I recall using a 'virtual directory' feature of some half-forgotten webserver years ago where you could override an underlying folder which might have a default page/script for example as a fallback.

I would like to have the ability to mount over a folder with or without existing content if possible.

Features like overlays which combine the contents could be nice if it was well defined and optional/configurable - but that's getting fancy.

My request doesn't make as much sense in an environment where we are restricting zipfs mounts to a specific volume - but I'm hopeful some consideration can also be given as to whether the builtin zipfs can default to using a volume but allow arbitrary mount points in future. I'm guessing that may need a tip - but let's not rule it out unless we have to.

apnadkarni added on 2023-09-10 04:01:30:

@griffin, @juliannoble2, noted. What gives me pause is that the current behavior seems to be purely accidental as a result of a single "flat" hash table being used to store paths.

  • When an overlay has conflicts with the existing mounts, the existing mounts win (i.e. if you mount a.zip and b.zip both containing file X in that order, the a.zip's X is preserved). This seems slightly counterintuitive to me but in any case the comment in the code "This should not happen" which skips b.zip's version implies it was not by design. Any time, I see code that "should not be reached" is reached, it is a point of worry!

  • More important, I would expect that when one of the archives is unmounted, the full contents of the other one that is still mounted are seen. This is not the case as the common entries are simply removed from the hash table. Looking at the code, it seems to me non-trivial re-implementation will be needed to fix this, something that I would be hesitant to introduce for this release. Maybe if @chw's directory caching implementation is picked up, this can be fixed as well.

Perhaps this issue is a corner case not sufficiently important to address at this point in time.


apnadkarni added on 2023-09-10 10:59:42:
Just for the record - dual behavior as file and directory.

```
% glob -dir //zipfs:/test/test *
//zipfs:/test/test/test3 //zipfs:/test/test/test2
% file type //zipfs:/test/test
file
% cd //zipfs:/test/test
couldn't change working directory to "//zipfs:/test/test": no such file or directory
```

I can't put my finger on what problems this might cause but these inconsistencies make me uncomfortable. In any case, will abandon adding test cases in this area until there is some consensus as to the correct behavior.