Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions benchmark/vfs/fs-resolve.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
'use strict';
const fs = require('fs');
const path = require('path');
const common = require('../common.js');

// Measures what the fs hooks cost a call once a VFS is mounted: a real path
// only has to be told apart from the VFS root, while a path in a layer is
// resolved to the layer, either by its id or through its mount name.
const bench = common.createBenchmark(main, {
target: ['real', 'id', 'name'],
n: [1e5],
}, { flags: ['--experimental-vfs', '--no-warnings'] });

function main({ n, target }) {
const vfs = require('node:vfs');
const layer = vfs.create();
layer.mkdirSync('/dir');
layer.writeFileSync('/dir/file.txt', 'x');
const mountPoint = layer.mount('bench');
const file = {
real: __filename,
id: path.join(mountPoint, 'dir', 'file.txt'),
name: path.join(path.dirname(mountPoint), 'bench', 'dir', 'file.txt'),
}[target];

bench.start();
for (let i = 0; i < n; i++) {
fs.statSync(file);
}
bench.end(n);
layer.unmount();
}
31 changes: 28 additions & 3 deletions doc/api/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -3791,7 +3791,8 @@ Print node's version.
added: REPLACEME
-->

* `source` {string} A directory or an archive file to mount and run.
* `source` {string} A directory or an archive file to mount and run, optionally
preceded by `name=` to name the mount as for [`--vfs-mount`][].

Requires [`--experimental-vfs`][]. May be given at most once.

Expand All @@ -3812,6 +3813,11 @@ from an earlier `--vfs-mount` of the same source.
In worker threads `--vfs-load` mounts but does not load: a worker inherits the
same mounts, in the same order, and runs its own entry point.

The source `--vfs-load` names is mounted at the same reserved mount point in
every thread, whatever else that thread mounts, so a path into it stays valid
in a worker - including one created with its own `execArgv`, which does not
inherit the parent's options and has to be given `--vfs-load` again.

`--vfs-load` is not permitted in [`NODE_OPTIONS`][]: which entry point runs is
the command line's decision, and the environment must not be able to redirect
it.
Expand All @@ -3827,7 +3833,8 @@ $ node --experimental-vfs --vfs-mount=lib.zip --vfs-load=app.zip
added: REPLACEME
-->

* `source` {string} A directory or an archive file to mount.
* `source` {string} A directory or an archive file to mount, optionally
preceded by `name=` to name the mount.

Requires [`--experimental-vfs`][]. May be repeated to mount several sources.

Expand All @@ -3845,6 +3852,23 @@ $ node --experimental-vfs --vfs-mount=a --vfs-load=b --vfs-mount=c
mounts `a`, `b` and `c` in that order and runs `b`. Mounts contributed by
[`NODE_OPTIONS`][] are mounted before the command line's.

A mount is named by writing its value as `name=source`, which mounts it as
[`vfs.mount(name)`][] does: the program can then reach it as
`path.join(os.devNull, 'vfs', name)` without knowing its mount point.

Everything before the first `=` in the value is the name, unless it contains a
path separator (`/` or `\`), in which case the whole value is the source. A
source whose path contains `=` can therefore be mounted without a name by
writing it with a separator:

```console
$ node --experimental-vfs --vfs-mount=assets=./build/assets.zip
$ node --experimental-vfs --vfs-mount=./a=b.zip
```

The first mounts `./build/assets.zip` with the name `assets`; the second mounts
`./a=b.zip` without a name.

The provider backing a source is chosen from the source itself rather than from
its file name:

Expand Down Expand Up @@ -4858,7 +4882,8 @@ node --stack-trace-limit=12 -p -e "Error.stackTraceLimit" # prints 12
[`v8.startupSnapshot.addDeserializeCallback()`]: v8.md#v8startupsnapshotadddeserializecallbackcallback-data
[`v8.startupSnapshot.setDeserializeMainFunction()`]: v8.md#v8startupsnapshotsetdeserializemainfunctioncallback-data
[`v8.startupSnapshot` API]: v8.md#startup-snapshot-api
[`vfs.mount()`]: vfs.md#vfsmount
[`vfs.mount()`]: vfs.md#vfsmountname
[`vfs.mount(name)`]: vfs.md#vfsmountname
[asynchronous module customization hooks]: module.md#asynchronous-customization-hooks
[benchmark runner]: bench.md#command-line-runner
[captured by the built-in snapshot of Node.js]: https://github.com/nodejs/node/blob/b19525a33cc84033af4addd0f80acd4dc33ce0cf/test/parallel/test-bootstrap-modules.js#L24
Expand Down
79 changes: 76 additions & 3 deletions doc/api/vfs.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,12 +173,16 @@ added: v26.4.0
* `emitExperimentalWarning` {boolean} Whether to emit the experimental
warning. **Default:** `true`.

### `vfs.mount()`
### `vfs.mount([name])`

<!-- YAML
added: v26.9.0
-->

* `name` {string} A name for the mount. It must be a single path segment other
than `.` and `..`, and must not be spelled the way a layer id is, as a
non-negative integer in its usual decimal form: `17` is reserved, while `07`
and `-1` are valid names.
* Returns: {string} The absolute mount point.

Mounts the virtual file system and returns the resulting mount point.
Expand All @@ -189,7 +193,7 @@ using paths under the returned mount point.
Mount points always live inside a reserved namespace that cannot have child file system entries,
so virtual paths never conflate with (or shadow) real paths. The virtual path scheme is subject to
change and users should not manually construct them based on assumptions. Instead, obtain
them from what `vfs.mount()` returns or `vfs.mountPoint`.
them from what `vfs.mount()` returns or `vfs.mountPoint`, or mount under a `name`.

```cjs
const vfs = require('node:vfs');
Expand All @@ -203,6 +207,34 @@ const mountPoint = myVfs.mount();
fs.readFileSync(`${mountPoint}/data.txt`, 'utf8'); // 'Hello'
```

A mount given a `name` can also be reached as
`path.join(os.devNull, 'vfs', name)`, a symbolic link to the mount point in
the [reserved root directory][]. This lets code that did not mount the file
system find it without being handed the instance. A later mount with the same
name takes the name over; the earlier file system stays mounted, but can then
only be reached at its own mount point. The name is removed when the file system
it links to is unmounted.

```cjs
const vfs = require('node:vfs');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');

const templates = vfs.create();
templates.writeFileSync('/page.html', '<h1>Hello</h1>');
templates.mount('templates');

// Elsewhere:
const dir = path.join(os.devNull, 'vfs', 'templates');
fs.readFileSync(path.join(dir, 'page.html'), 'utf8'); // '<h1>Hello</h1>'
```

Like any mount point, the mount point cannot be removed or renamed, nor
replaced by renaming something else onto it: [`fs.rmdir()`][] and
[`fs.rename()`][] fail with `EBUSY`. A recursive [`fs.rm()`][] of the mount
point empties the file system before failing the same way.

Each `VirtualFileSystem` instance may be mounted at most once at a
time. Attempting to mount an already-mounted instance throws
`ERR_INVALID_STATE`. Because each instance mounts inside its own
Expand Down Expand Up @@ -380,6 +412,42 @@ The promise namespace mirrors `fs.promises` and includes `readFile`,
`access`, `rm`, `truncate`, `link`, `mkdtemp`, `chmod`, `chown`, `lchown`,
`utimes`, `lutimes`, `open`, `lchmod`, and `watch`.

## The reserved root directory

While any virtual file system is mounted, the directory that holds the mount
points, `path.join(os.devNull, 'vfs')`, can be read through [`node:fs`][]. It
contains a directory for every mounted file system, named like the last segment
of its [`vfs.mountPoint`][], and a symbolic link for every name given to
[`vfs.mount()`][], pointing at the mount it names.

```cjs
const vfs = require('node:vfs');
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');

const root = path.join(os.devNull, 'vfs');
const assets = vfs.create();
assets.writeFileSync('/logo.svg', '<svg/>');
const mountPoint = assets.mount('assets');

fs.readdirSync(root); // e.g. [ '1', 'assets' ]
fs.readdirSync(root, { recursive: true }); // e.g. [ '1', 'assets', '1/logo.svg' ]
fs.readlinkSync(path.join(root, 'assets')); // e.g. '1'
fs.realpathSync(path.join(root, 'assets')) === mountPoint; // true
fs.readFileSync(path.join(root, 'assets', 'logo.svg'), 'utf8'); // '<svg/>'
```

A path through a name works wherever the same path through the mount point
does, including in `require()` and `import`. As with any symbolic link,
[`fs.realpath()`][] resolves it to the path under the mount point, and so does
the module loader: a module loaded through a name is identified by its path
under the mount point.

The root directory itself is read-only. Creating, removing, or changing its
entries fails with `EROFS`, while the file systems its entries lead to can be
written to as usual. When nothing is mounted, the root directory does not exist.

## Module loader integration

Once a `VirtualFileSystem` is mounted, paths under the mount point
Expand Down Expand Up @@ -711,18 +779,23 @@ fields use synthetic but stable values:
[`ffi.dlopen()`]: ffi.md#ffidlopenpath-definitions
[`fs.BigIntStats`]: fs.md#class-fsstats
[`fs.Stats`]: fs.md#class-fsstats
[`fs.realpath()`]: fs.md#fsrealpathpath-options-callback
[`fs.rename()`]: fs.md#fsrenameoldpath-newpath-callback
[`fs.rm()`]: fs.md#fsrmpath-options-callback
[`fs.rmdir()`]: fs.md#fsrmdirpath-options-callback
[`import.meta.resolve()`]: esm.md#importmetaresolvespecifier
[`new ffi.DynamicLibrary()`]: ffi.md#new-dynamiclibrarypath
[`node:fs`]: fs.md
[`require()`]: modules.md#requireid
[`require.resolve()`]: modules.md#requireresolverequest-options
[`url.pathToFileURL()`]: url.md#urlpathtofileurlpath-options
[`vfs.mount()`]: #vfsmount
[`vfs.mount()`]: #vfsmountname
[`vfs.mountPointURL`]: #vfsmountpointurl
[`vfs.mountPoint`]: #vfsmountpoint
[`vfs.unmount()`]: #vfsunmount
[`zipFile.writable`]: zlib.md#zipfilewritable
[`zlib.ZipBuffer`]: zlib.md#class-zlibzipbuffer
[`zlib.ZipFile`]: zlib.md#class-zlibzipfile
[loading from `node_modules` folders]: modules.md#loading-from-node_modules-folders
[reserved root directory]: #the-reserved-root-directory
[the global folders]: modules.md#loading-from-the-global-folders
23 changes: 21 additions & 2 deletions doc/node.1
Original file line number Diff line number Diff line change
Expand Up @@ -1885,7 +1885,8 @@ Print node's version.
.It Fl -vfs-load Ns = Ns Ar source
.Bl -bullet
.It
\fBsource\fR \fB{string}\fR A directory or an archive file to mount and run.
\fBsource\fR \fB{string}\fR A directory or an archive file to mount and run, optionally
preceded by \fBname=\fR to name the mount as for \fB--vfs-mount\fR.
.El
Requires \fB--experimental-vfs\fR. May be given at most once.
Mounts \fBsource\fR exactly as \fB--vfs-mount\fR does, and additionally runs the
Expand All @@ -1901,6 +1902,10 @@ The entry point then comes from the mount \fB--vfs-load\fR itself contributed, n
from an earlier \fB--vfs-mount\fR of the same source.
In worker threads \fB--vfs-load\fR mounts but does not load: a worker inherits the
same mounts, in the same order, and runs its own entry point.
The source \fB--vfs-load\fR names is mounted at the same reserved mount point in
every thread, whatever else that thread mounts, so a path into it stays valid
in a worker - including one created with its own \fBexecArgv\fR, which does not
inherit the parent's options and has to be given \fB--vfs-load\fR again.
\fB--vfs-load\fR is not permitted in \fBNODE_OPTIONS\fR: which entry point runs is
the command line's decision, and the environment must not be able to redirect
it.
Expand All @@ -1912,7 +1917,8 @@ $ node --experimental-vfs --vfs-mount=lib.zip --vfs-load=app.zip
.It Fl -vfs-mount Ns = Ns Ar source
.Bl -bullet
.It
\fBsource\fR \fB{string}\fR A directory or an archive file to mount.
\fBsource\fR \fB{string}\fR A directory or an archive file to mount, optionally
preceded by \fBname=\fR to name the mount.
.El
Requires \fB--experimental-vfs\fR. May be repeated to mount several sources.
Mounts \fBsource\fR as a virtual file system (\fBnode:vfs\fR). Each mount is placed
Expand All @@ -1925,6 +1931,19 @@ $ node --experimental-vfs --vfs-mount=a --vfs-load=b --vfs-mount=c
.Ed
mounts \fBa\fR, \fBb\fR and \fBc\fR in that order and runs \fBb\fR. Mounts contributed by
\fBNODE_OPTIONS\fR are mounted before the command line's.
A mount is named by writing its value as \fBname=source\fR, which mounts it as
\fBvfs.mount(name)\fR does: the program can then reach it as
\fBpath.join(os.devNull, 'vfs', name)\fR without knowing its mount point.
Everything before the first \fB=\fR in the value is the name, unless it contains a
path separator (\fB/\fR or \fB\\\fR), in which case the whole value is the source. A
source whose path contains \fB=\fR can therefore be mounted without a name by
writing it with a separator:
.Bd -literal
$ node --experimental-vfs --vfs-mount=assets=./build/assets.zip
$ node --experimental-vfs --vfs-mount=./a=b.zip
.Ed
The first mounts \fB./build/assets.zip\fR with the name \fBassets\fR; the second mounts
\fB./a=b.zip\fR without a name.
The provider backing a source is chosen from the source itself rather than from
its file name:
.Bl -bullet
Expand Down
45 changes: 35 additions & 10 deletions lib/internal/process/pre_execution.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ const {
ObjectDefineProperty,
ObjectFreeze,
String,
StringPrototypeIncludes,
StringPrototypeIndexOf,
StringPrototypeSlice,
globalThis,
Expand Down Expand Up @@ -223,9 +224,11 @@ let vfsLoadRoot;
// the command line's own node options, in order. NODE_OPTIONS may add mounts
// but not a --vfs-load, so anything it contributed sits ahead of these.
// Returns -1 when no --vfs-load was given.
//
// A worker does not run the --vfs-load entry (see node_worker.cc), but it must
// still recognize which source that entry is, so as to give it the reserved
// layer its mount point has on every other thread.
function getVfsLoadIndex(mountCount) {
if (!getOptionValue('[vfs_load_set]')) return -1;

const execArgv = process.execArgv;
let seen = 0;
let found = -1;
Expand All @@ -252,6 +255,22 @@ function getVfsLoadIndex(mountCount) {
return mountCount - seen + found;
}

// Splits a --vfs-mount/--vfs-load value of the form `[name=]source`. A prefix
// holding a path separator is part of the source, so a path containing `=`
// can still be mounted by writing it with a separator (`./a=b.zip`).
function parseVfsMount(value) {
const eq = StringPrototypeIndexOf(value, '=');
if (eq === -1) return { name: undefined, source: value };
const name = StringPrototypeSlice(value, 0, eq);
if (StringPrototypeIncludes(name, '/') || StringPrototypeIncludes(name, '\\')) {
return { name: undefined, source: value };
}
return {
name: name === '' ? undefined : name,
source: StringPrototypeSlice(value, eq + 1),
};
}

// Mounts every --vfs-mount source. Called from prepareExecution() when there is
// no --import, and otherwise from run_main after the --import loop has run; the
// guard makes the second call a no-op so a provider registered by either a -r or
Expand All @@ -267,14 +286,16 @@ function finishVfsMounts() {
const fs = require('fs');
const path = require('path');
const { selectProvider } = require('internal/vfs/provider_registry');
const { VirtualFileSystem } = require('internal/vfs/file_system');
const { VirtualFileSystem, kLoadLayer } = require('internal/vfs/file_system');

// --vfs-load is forced off in workers (see node_worker.cc), so this records a
// load root only on the main thread; a worker re-mounts the same sources in
// the same order (the reserved paths line up) but runs its own entry.
const loadIndex = getVfsLoadIndex(entries.length);
// [vfs_load_set] is forced off in workers (see node_worker.cc), so a worker
// mounts every source, the --vfs-load one at its reserved layer, and runs its
// own entry rather than the mount's.
const loads = getOptionValue('[vfs_load_set]');
for (let i = 0; i < entries.length; i++) {
const resolvedSource = path.resolve(entries[i]);
const { name, source } = parseVfsMount(entries[i]);
const resolvedSource = path.resolve(source);
let stats;
try {
stats = fs.statSync(resolvedSource);
Expand All @@ -288,16 +309,20 @@ function finishVfsMounts() {
if (provider === null) {
throw new ERR_VFS_INVALID_TARGET(resolvedSource);
}
const vfs = new VirtualFileSystem(provider, { emitExperimentalWarning: false });
const mountPoint = vfs.mount();
const vfs = new VirtualFileSystem(provider, {
__proto__: null,
emitExperimentalWarning: false,
[kLoadLayer]: i === loadIndex,
});
const mountPoint = vfs.mount(name);
// The mount --vfs-load contributed is what the entry is require()d from;
// process.argv[1] names the real source instead, since the reserved mount
// point is an opaque implementation detail.
//
// The source is spliced in rather than assigned over argv[1]: the entry
// comes from the mount, so nothing was consumed as an entry point and the
// first positional argument is the program's own. Overwriting would drop it.
if (i === loadIndex) {
if (i === loadIndex && loads) {
vfsLoadRoot = mountPoint;
ArrayPrototypeSplice(process.argv, 1, 0, resolvedSource);
}
Expand Down
Loading