feat(desktop): Documents — a local markdown vault, behind a preview flag

[derekross]

Documents — a local markdown vault, behind a preview flag

Adds Documents: a file tree, tabbed editor, and live preview over a folder
of plain .md files on disk. No relay, no events, no new kinds — the vault is
local and per-machine, so Obsidian and git see exactly what you would expect.

Off by default, under Settings → Experiments, following the Projects
pattern (FeatureGate on the sidebar entry, usePreviewFeatureWarning on the
route).

The editor and file browser are ported from Onyx,
an open-source Obsidian-like app. Its Milkdown plugins are plain ProseMirror,
so they port onto Buzz's existing TipTap stack by rewriting imports.

Documents over a real vault — file tree, tabs, and live preview. Files stay plain .md on disk.

Off by default, under Settings → Experiments, following the Projects pattern.

Settings → Documents — the vault folder is global and per-machine, not per-community.

---

The idea the design rests on

A WYSIWYG editor autosaving over someone's real vault is a data-destruction
feature unless proven otherwise. tiptap-markdown is markdown-it plus
prosemirror-markdown, and anything outside the TipTap schema does not survive
the trip.

So rather than guess which constructs are safe, every note is asked one
question before the editor touches it: does serialize(parse(x)) return
x? If not, it opens in source mode — a raw textarea that never touches
the serializer — with a one-line notice. Editing is still possible; silent
reformatting is not.

That guard is lib/roundTripGuard.ts, and lib/markdownRoundTrip.test.mjs is
its corpus test: it pins exactly which constructs survive, so an extension that
changes the answer fails there loudly instead of quietly rewriting a vault.

Measured, not assumed. Byte-exact comparison passed 4% of a real
470-note vault, which is the same as not shipping. Bisecting the failures found
six differences no reader can see — a lone trailing space, blank lines that
only separate blocks, *** vs ---, table padding, _em_ vs *em*, and runs
of spaces inside a line. Tolerating exactly those took it to 63%. The
remaining 176 failures span 164 distinct causes: a real long tail, mostly
callouts, footnotes and raw HTML, which the guard should keep catching.

The trade this encodes, stated plainly: saving a note may rewrite those
cosmetic differences into canonical form. Nothing rendered changes, but a
git-tracked vault will show the diff on first save.

Two exceptions are load-bearing and pinned by tests: a blank line between list
items makes a list loose, and a blank line between blockquotes is the only
thing keeping them from merging. Both survive the editor intact, so collapsing
them as "block separation" would have hidden a real change.

Security

The vault commands are the security-sensitive surface, so:

• The renderer never names the vault root. It lives in VaultState on the
  Rust side. Onyx's commands took (path, vault_path) — letting a caller supply
  both sides of the containment check — and five of them skipped validation
  entirely.
• Containment is lexical, never canonicalize(). A folder deliberately
  symlinked into a vault is an intentional grant and keeps working; .. escapes
  are still rejected. There is a ported regression test for this, and
  ValidatedVaultPath makes "use the returned path, not the string you passed
  in" a compile error rather than a review note.
• Reads are capped at 2 MB. Every read loads the file into memory, ships it
  over IPC as JSON, and gets parsed twice frontend-side; one stray export ending
  in .md would otherwise freeze the app.
• Deleting a linked folder unlinks it and leaves the target alone. Verified
  empirically, then pinned — it depends on remove_dir_all refusing to descend
  through a symlink (the CVE-2022-21658 fix).
• Hostile markdown never becomes live markup. A vault is not trusted input;
  editorSecurity.test.mjs feeds script tags, event handlers, javascript:/
  data:/vbscript: links and image-URL escapes through the real editor and
  asserts against the parsed DOM that none materialise. It also asserts an
  ordinary link still works, so it cannot pass by rendering nothing.

One accepted residual, called out for review: set_active_vault takes a
renderer-supplied path, so a compromised renderer could point the vault
somewhere sensitive. Given terminal_attach + terminal_input already grant
arbitrary command execution to the renderer, this is not an escalation — but the
defence-in-depth fix (persist the granted path backend-side, accept only a
dialog-picked or previously-granted one) is worth considering separately.

In scope

Tree, tabs with dirty state, live preview, raw source mode, 2s debounced
autosave, ⌘S, filesystem watcher, create/rename/delete, wikilinks with
click-through, backlinks (linked and unlinked), heading outline with scroll-spy,
Obsidian syntax decorations, tables, session restore.

Out of scope for v1: frontmatter UI, daily notes, templates, vault search,
quick-switcher, graph view, ![[embeds]], and any Nostr sync.

Wikilinks, tags, highlights and task markers are decoration-only — they stay
plain text in the document so they serialize back byte-identically. That is why
they render with their [[brackets]] visible.

Notable fixes found by dogfooding

• Autosave accused the user of a conflict on nearly every save.
  write_vault_file replaces the file with a rename; the watcher fires on that
  rename and its event routinely beats the command's own response back to the
  webview, so the mtime echo-suppression needed had not been recorded yet.
  Comparing mtimes cannot win a race against the thing that produces the mtime,
  so reconciliation now compares bytes, recorded synchronously before the
  write starts. This also fixes byte-identical external rewrites (what sync
  clients and formatters do constantly) raising a spurious conflict.
• Switching notes took ~2s. getBacklinks scanned every note on every
  switch, compiling a fresh RegExp per line across ~75k lines: 103ms → 22ms.
  The bigger cost was re-parsing markdown on every tab switch; handing
  setContent the parsed document from last time is 30–47× faster
  (249ms → 8ms on a 110KB note). The per-file editor remount is kept — it is
  what stops undo resurrecting another note's text, and creating the editor is
  only ~10ms.

Testing

Gate	Result
pnpm test (desktop unit)	4512 pass
cargo test --lib (Tauri)	2267 pass
Documents E2E	32 pass
Full smoke suite	965 pass / 5 fail — all pre-existing or flaky, triaged below
cargo clippy -D warnings, pnpm check, tsc	clean

One new Rust dependency: notify (MIT OR Apache-2.0).

lib.rs was at 991 lines on main and this branch's vault registration pushed it
over the ratchet, so the pending_sync flush loop moved into
persona_events::spawn_flush_loop — next to the function it drives — rather
than raising the limit.

The five smoke failures

Each was triaged to a cause rather than waved through. None is a regression
from this branch:

Failure	Cause
inbox-edit.spec.ts:175	Fails on unmodified origin/main too — verified in a detached worktree
thread-focus-mode.spec.ts:139	Same — fails at baseline
relay-reconnect.spec.ts:158	Flaky: 7 passed / 1 failed over 8 runs
community-rail.spec.ts:509	Passes in isolation; fails only under full-suite CPU contention
community-rail.spec.ts:1078	Same

[wesbillman]

@derekross thanks for the PR but this is a significant change to the Buzz core. Also, we have a "nest" in buzz already that might be useful for this usecase but without the UI. If you check in your home folder ~/.buzz on mac, the agents can already store and manage markdown files there for guides, research, etc. I could see a feature in the future where we visualize something like that, but we might not be ready for the vault implementation here.

Also, we should look into a way to store these on the relay vs. locally so they would be available to all your devices connected to the relay vs. just local file system.

But this is a dope idea! Def gets the ideas flowing

[derekross]

Yes, those documents are stored there, but you have to open an application outside of Buzz to view them. That was the whole point of the PR so that don't need to leave Buzz to view documents that you and agents create together. It's a better workflow.

Yes, we can store them on the relay. I had planned on that being a phase two after the initial UI was approved and merged. It would have been a larger undertaking to try and accomplish both right away. But yes, sharing documents would be the ultimate goal.