workspace: sync node_modules by default

.changeset/sync-node-modules.md

@@ -0,0 +1,5 @@
+---
+"@cloudflare/computer": patch
+---
+
+Sync `node_modules` by default so package manager installs persist across workspace runtimes.

docs/02_sync_protocol.md

@@ -405,56 +405,33 @@ first-class conflict primitives.
 
 ## Ignore lists
 
-
-The `ignore` option hides path segments from the pull. Excluded
-paths are still written and read inside the container — the bytes just
-never cross the wire back to the DO. This is essential for any large
-directory of derived files: `node_modules`, `.next`, `target`,
-`__pycache__`, `dist`. Without an ignore, a single `npm install` would
-push tens of thousands of small files through the sync wire on the
-next pull.
-
-The default is `["node_modules"]`, applied server-side when `ignore` is
-omitted. A caller-supplied list **replaces** the default — it does not
-extend it. Pass `[]` to disable ignoring entirely, or pass your full
-list (including `"node_modules"` if you still want it) to customise.
-
-### Ignored entries
-
-Ignored paths are **invisible to the `Workspace.fs` API**. They do not
-appear in `readdir`, `stat` returns `ENOENT`, and `readFile` returns
-`ENOENT`. The bytes still live inside the container, so anything that
-*uses* the ignored files — `exec("node ...")`, build tools, anything
-running container-side — keeps working. The exclusion only affects what
-crosses the wire **and** what the DO-side API surfaces.
-
-This is a deliberately narrow surface for the initial release. Whether
-ignored entries should be representable to the DO at all (as stubs, as
-a separate shell-only namespace, or not at all) is left to a future
-iteration — see [Future considerations](#future-considerations).
+The optional `ignore` list hides matching path segments from a sync
+stream. Filtering is opt-in and applies only while coalescing changes
+for the wire; the filesystem retains and exposes the paths normally.
+An ignored path is therefore not deleted or hidden from `Workspace.fs`,
+but it is omitted from that particular pull. This is useful for large
+directories of derived files such as `.next`, `target`, `__pycache__`,
+or `dist`.
+
+The default is `[]`. A caller-supplied list replaces the configured
+list for that fetch. Pass an explicit list when a sync peer should omit
+paths, or pass `[]` to include every path. To omit `node_modules`, for
+example, configure `ignore: ["node_modules"]` explicitly.
+
+### Filtered entries
+
+Filtering affects only the change stream. Files remain readable and
+writable through `Workspace.fs`, and container-side commands such as
+`exec("node ...")` can use them normally. Filtered entries still advance
+the receiving peer's fetch cursor, so an ignore configuration should be
+kept stable for a sync relationship; removing a pattern does not replay
+changes that were already filtered.
 
 ## Future considerations
 
 Items deferred from the initial design. File an issue if a real use
 case depends on a particular resolution.
 
-### Representing ignored entries to the DO
-
-Today ignored paths are entirely invisible to `Workspace.fs`. That is
-the simplest contract but it loses one piece of information: tools that
-want to enumerate "everything the agent's exec can see" can't get it
-from the DO. Two options worth weighing later:
-
-- **Stub entries with an `ignored` flag** on `stat()`, surfaced via
-  `readdir`. Easy to retrofit; surprising for tools that walk the tree
-  and don't check the flag.
-- **An explicit shell-only namespace** — e.g. `workspace.runtime.readdir`
-  returns container-only entries, `workspace.fs.readdir` stays clean.
-  Cleaner separation, larger API surface.
-
-Either way, the bytes never cross the wire; the question is purely how
-much the DO admits exists.
-
 ### Bloom/cuckoo filter over `vfs_blobs.hash`
 
 Every pull does a `hasObjects` probe round-trip. With tens of thousands

docs/03_filesystem_schema.md

@@ -74,9 +74,10 @@ The `vfs_nodes_by_rev` index supports `coalesceChanges`'s cursor scan
 over live inodes, which the sync protocol calls once per pull to
 enumerate everything modified after the last fetch cursor.
 
-There is no `ignored` column: ignored paths are entirely invisible to
-the DO-side filesystem API (see
-[02. Sync Protocol → Ignored entries](./02_sync_protocol.md#ignored-entries)).
+There is no `ignored` column. Ignore patterns are an ephemeral sync
+option, applied while building a change stream; they do not change the
+filesystem namespace or the paths visible through `Workspace.fs`. See
+[02. Sync Protocol → Ignore lists](./02_sync_protocol.md#ignore-lists).
 
 ### `vfs_dirents` — name → inode mapping
 

docs/06_mount_interface.md

@@ -210,15 +210,15 @@ provider-specific config:
 | Option | Default | Meaning |
 | --- | --- | --- |
 | `mode` | `"read-only"` | `"read-only"` or `"read-write"`. |
-| `ignore` | `[]` | Path segments hidden from the pull *and* from `Workspace.fs`. Composed with the top-level `ignore` by union. See [02. Sync Protocol → Ignored entries](./02_sync_protocol.md#ignored-entries). |
+| `ignore` | `[]` | Path segments omitted from the mount's sync pull. The paths remain visible through the underlying `Workspace.fs`; the option is composed with the top-level `ignore` by union. See [02. Sync Protocol → Ignore lists](./02_sync_protocol.md#ignore-lists). |
 | `writeBack` | `"debounce"` | `"debounce"` (default) or `"manual"`. See “Write-back gating” above. |
 | `writeBackMs` | `500` | Debounce window in milliseconds. Ignored when `writeBack: "manual"`. |
 | `maxBytes` | unbounded | Hard cap on total bytes indexed from this mount. Exceeding throws at index time before any data lands in `vfs_nodes`. |
 | `maxEntries` | unbounded | Hard cap on entry count. Same enforcement timing as `maxBytes`. |
 
 The workspace-level `ignore` option (the renamed `pullIgnore`) applies
 to every mount and to top-level paths. Mount-level `ignore` extends it
-for that mount only.
+for that mount only. No paths are ignored by default.
 
 ## Mount conflicts
 

docs/08_capnweb_interface.md

@@ -312,7 +312,7 @@ type WireError = {
 
 | Code | Meaning |
 | --- | --- |
-| `ENOENT` | Path does not exist on the receiver (covers ignored paths, which are invisible to `Workspace.fs`), or `getExec` / `disposeExec` referenced an unknown id. |
+| `ENOENT` | Path does not exist on the receiver, or `getExec` / `disposeExec` referenced an unknown id. Sync ignore patterns do not alter filesystem path lookups. |
 | `EUNKNOWN_HASH` | `fetchObjects` referenced a hash the receiver has no record of; raised via `createWorkspaceError`. |
 | `EEXEC_BUSY` | `exec` was called with an `id` that's already in use by a live run. |
 | `ELOG_TRUNCATED` | `getExec` resume point is older than the retained log. |

packages/dofs/README.md

@@ -17,7 +17,7 @@ This package exposes a JavaScript module, not a CLI. It bundles three layers tha
 - A `Database` wrapper around Durable Object SQL storage plus `initializeSchema` for the `vfs_*` tables.
 - Filesystem primitives under `src/fs/*` (`mkdir`, `writeFile`, `readFile`, `rm`, `readdir`, `stat`, `lstat`, `chmod`, `find`, `ls`, `grep`, `symlink`, `readlink`, `gc`, `watch`) operating on a `Database`.
 - `SQLiteWorkspaceProvider`, a `@platformatic/vfs` adapter that composes those primitives into a node-shaped filesystem (fd table, positional `readSync`/`writeSync`, `watchSync`, symlinks). This is what `computerd` mounts via FUSE.
-- Sync protocol building blocks operating on the same `Database`: `applyChanges`, `stageBlob`, `materialiseChange`, `coalesceChanges`, `fetchChanges`, `fetchObjects`, `hasObjects`, `pushObjects`, `buildManifest`, `currentRev`, `compareChangeCursors`, `readWatermark`/`writeWatermark`, `assertAppliedPushCursor`, and `DEFAULT_IGNORE`/`isIgnored`. The wire wiring lives in `@cloudflare/computer-rpc`.
+- Sync protocol building blocks operating on the same `Database`: `applyChanges`, `stageBlob`, `materialiseChange`, `coalesceChanges`, `fetchChanges`, `fetchObjects`, `hasObjects`, `pushObjects`, `buildManifest`, `currentRev`, `compareChangeCursors`, `readWatermark`/`writeWatermark`, `assertAppliedPushCursor`, and the opt-in ignore matcher `isIgnored` (the default ignore list is empty). The wire wiring lives in `@cloudflare/computer-rpc`.
 
 Minimal DO-side usage — initialize the schema; the `Database` becomes the handle every other helper takes:
 

packages/dofs/src/sync/ignore.test.ts

@@ -1,10 +1,15 @@
 import { describe, expect, it } from "vitest";
 
-import { isIgnored } from "./ignore.js";
+import { DEFAULT_IGNORE, isIgnored } from "./ignore.js";
 
 describe("isIgnored", () => {
   const list = ["node_modules", ".next", "target"];
 
+  it("has no default exclusions", () => {
+    expect(DEFAULT_IGNORE).toEqual([]);
+    expect(isIgnored("/node_modules/package/index.js", DEFAULT_IGNORE)).toBe(false);
+  });
+
   it("returns false for paths that don't intersect the list", () => {
     expect(isIgnored("/src/index.ts", list)).toBe(false);
     expect(isIgnored("/README.md", list)).toBe(false);

packages/dofs/src/sync/ignore.ts

@@ -1,14 +1,14 @@
-// Path segment matcher for the container-side ignore list. The
-// container uses this to drop paths from coalesceChanges before
-// they hit the wire; the DO's Workspace.fs surface uses the same
-// helper to make ignored paths invisible to API consumers.
+// Path-segment matcher for an explicitly configured sync ignore list.
+// Matching is whole-segment: a pattern matches that segment anywhere in
+// the path, but does not match a longer segment. Patterns are plain
+// strings, not globs; we can extend to globs later if a real case
+// demands it.
 //
-// Matching is whole-segment: "node_modules" matches the segment
-// node_modules anywhere in the path but does not match
-// node_modules_old or my_node_modules. Patterns are plain strings,
-// not globs; we can extend to globs later if a real case demands it.
+// The default is intentionally empty. Ignoring paths is opt-in because
+// the filesystem and sync surfaces should preserve the same namespace.
 
-export const DEFAULT_IGNORE = ["node_modules"];
+// Kept as a named export for callers that need the package default.
+export const DEFAULT_IGNORE: string[] = [];
 
 export function isIgnored(path: string, patterns: string[]): boolean {
   if (patterns.length === 0) return false;

packages/rpc/src/server.ts

@@ -12,7 +12,6 @@ import {
   compareChangeCursors,
   currentRev,
   type Database,
-  DEFAULT_IGNORE,
   fetchObjects,
   hasObjects,
   materialiseChange,
@@ -178,8 +177,7 @@ class SyncRPCServer extends RpcTarget implements SyncRPC {
       }
     }
     const after = input.after ?? { rev: 0, path: null };
-    const ignore =
-      input.ignore ?? (this.options.ignore.length > 0 ? this.options.ignore : DEFAULT_IGNORE);
+    const ignore = input.ignore ?? this.options.ignore;
     const snapshotRev = currentRev(this.db);
     const currentCursor = { rev: snapshotRev, path: null };
     return {

packages/rpc/src/sync-driver.test.ts

@@ -1082,7 +1082,7 @@ describe("sync driver — blob-stage recovery", () => {
       source.mkdirSync("/repo/node_modules/tiny", { recursive: true });
       source.writeFileSync("/repo/package.json", '{"name":"fixture"}\n');
       source.writeFileSync("/repo/dist/result.txt", "installed");
-      source.writeFileSync("/repo/node_modules/tiny/index.js", "ignored");
+      source.writeFileSync("/repo/node_modules/tiny/index.js", "module.exports = 'installed';");
 
       let injected = false;
       const failingDb = new Database({
@@ -1109,7 +1109,9 @@ describe("sync driver — blob-stage recovery", () => {
       const destination = new SQLiteWorkspaceProvider(receiver.db, { now: () => 1 });
       expect(resumed.applied).toBeGreaterThan(0);
       expect(destination.readFileSync("/repo/dist/result.txt", "utf8")).toBe("installed");
-      expect(destination.existsSync("/repo/node_modules")).toBe(false);
+      expect(destination.readFileSync("/repo/node_modules/tiny/index.js", "utf8")).toBe(
+        "module.exports = 'installed';",
+      );
       expect(readFetchCursor(receiver.db)).toEqual({
         rev: currentRev(upstream.db),
         path: null,