vfs: add mount names and make the reserved root readable - #66119
pipobscure wants to merge 3 commits into
Conversation
Nothing identified a mounted virtual file system beyond its reserved mount point, which is an opaque implementation detail. A mount made with --vfs-mount in particular could not be told apart from the others, nor be found by the program it was mounted for. VirtualProvider now takes an options bag whose `name` is exposed as `provider.name`, and MemoryProvider, RealFSProvider and ZipProvider pass their options on to it. A VirtualFileSystem exposes the name of its provider as `vfs.name`. --vfs-mount and --vfs-load accept `name=source`, and the name is handed to the provider through a new third argument to a registered provider's create(). Text before the first `=` is only a name if it has no path separator, so a path containing `=` can still be mounted by writing it as `./a=b`. --vfs-load takes the prefix too because a worker re-mounts every source without knowing which one --vfs-load contributed, and must split each value the same way. Add vfs.mounted(), which returns a null-prototype object mapping the name of each mounted file system to its VirtualFileSystem, so code can find a mount by name without being handed the instance. Unnamed mounts are left out, and when several share a name the earliest mount wins. Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
|
Review requested:
|
|
Thanks! Some thoughts:
|
|
@bakkot something you said last night and your comment above ruminated with me and I think you’re right. You said something like “it’s not like you can just fs.readdir /dev/null/vfs” I think that might actually be the natural solution here.
Thoughts? @mcollina this is probably something you should be aware of to vociferously object if you disagree. |
Workers get the same —vfs-mount flags in the same order. Since they are run before startup in he same order and layer numbers are deterministic by order it will have the same initial mounts as the main process. So a worker can be pointed at the same import.meta.dirname relative to the main calling it. This already works and has tests. |
This comment was marked as outdated.
This comment was marked as outdated.
The previous commit named a mounted file system through its provider,
and added vfs.mounted() to look one up by that name. A name identifies a
mount, though, not a provider, and a lookup table beside the file system
is something only VFS-aware code can use.
Take the name as an optional argument to vfs.mount() instead, and drop
provider.name, vfs.name, vfs.mounted() and the options argument added to
the provider constructors and to a registered provider's create(). A
later mount under a name takes it over; unmounting removes the names
linking to that mount. --vfs-mount and --vfs-load pass their `name=`
prefix to vfs.mount().
While anything is mounted, the reserved root `${os.devNull}/vfs` is now a
read-only directory served by the new ReservedRootProvider. It lists
each mount point by its layer id, and each name as a symbolic link to
that id, so plain fs code finds a mount with readdir, readlink and
realpath. The dispatcher follows a name at the start of a path to its
layer, except for the operations that act on a link itself (lstat,
readlink, unlink, rm, rename, symlink and the l* variants), so require(),
import and the fs functions all work through a name, and the loader
identifies modules by their real mount point paths. Loader caches under
a name are purged when the name moves or goes away.
A name must be a single path segment other than `.` and `..`, and must
not be spelled the way a number is, since those segments are layer ids;
`07` is a valid name while `7` is not.
realpath() of a mount point returned it with a trailing separator, from
path.join(mountPoint, '/'); it now returns the mount point itself.
Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
A mount point could be removed: rmdir() of an empty mount point went to the provider, which dropped a nonexistent child of its own root and reported success, and a recursive rm() did the same once the file system was empty. Renaming onto or away from it reached the provider as well. Like any mount point, the root of a VirtualFileSystem now cannot be removed or renamed, nor replaced by a rename: rmdir() and rename() fail with EBUSY, and a recursive rm() empties the file system and then fails the same way. A mount name was refused if it was spelled as any number, which kept names like `-1`, `NaN` and `Infinity` that can never be a layer id. Reserve only the canonical spelling of a non-negative integer. Signed-off-by: Philipp Dunkel <pip@pipobscure.com>
|
Also notice to @jasnell as you were the one to originally critique the version of names I had proposed. I think this actually meets the issues you had as well, but worth double checking. And thanks again to @bakkot ! Pushback makes things better. It did for the original |
vfs.mounted()
mcollina
left a comment
There was a problem hiding this comment.
I like both changes, but I'm unsure if the added machinery is worth it.
How fast is this? It might be simpler to split this into two PRs (one adding the root).
| MapPrototypeForEach(activeNames, (target, name) => { | ||
| if (target === layerId) { | ||
| activeNames.delete(name); | ||
| purgeLoaderCachesForPrefix(getVfsRoot() + sep + name); |
| * }|null} | ||
| */ | ||
| function findVFSOrRoot(inputPath) { | ||
| function resolveVFS(inputPath, followLast = true) { |
There was a problem hiding this comment.
This is a very hot function if I recall correctly. We should do some benchmarks to verify there are no slowdowns.
Also, I don't understand what followLast is used for.
| vfs: activeVFSLayers.get(layerId), | ||
| path, | ||
| normalized: path, | ||
| mountPoint: getVfsRoot() + sep + segment, |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #66119 +/- ##
==========================================
+ Coverage 90.27% 90.30% +0.03%
==========================================
Files 790 791 +1
Lines 271651 272196 +545
Branches 51842 51991 +149
==========================================
+ Hits 245228 245803 +575
+ Misses 16928 16899 -29
+ Partials 9495 9494 -1
🚀 New features to boost your workflow:
|
Nothing identified a mounted virtual file system beyond its reserved mount point, which is an opaque implementation detail. A mount made with
--vfs-mountin particular could not be told apart from the others, nor be found by the program it was mounted for.Mount names
vfs.mount([name])takes an optional name. A named mount can also be reached aspath.join(os.devNull, 'vfs', name), a symbolic link to its mount point. A later mount under the same name takes the name over, and unmounting removes the names linking to that mount. Invalid names throwERR_INVALID_ARG_VALUE. A name must be a single path segment other than.and.., and must not be spelled like a layer id, which is a non-negative integer in canonical decimal form. So17is reserved, while07,-1andNaNare valid names.--vfs-mountand--vfs-loadacceptname=sourceand pass the name tovfs.mount(). Text before the first=is only a name if it has no path separator, so a path containing=can still be mounted by writing it as./a=b.--vfs-loadtakes the prefix too, because a worker re-mounts every source without knowing which one--vfs-loadcontributed, and must split each value the same way.The reserved root is a directory
While anything is mounted,
${os.devNull}/vfsis a read-only directory that plainnode:fscode can read. It lists every mount point by its layer id, and every name as a symbolic link to that id:This came out of the discussion with @bakkot on #65748 and #66116 and a short chat to discuss the shape of it.