perf: investigate in-memory state heap footprint (~60x on-disk-to-heap expansion)

[themightychris]

Context

Surfaced during an incident (PR #131): a cold boot rebuilding in-memory state from the full published import OOM'd at the 1536 MiB V8 old-space ceiling. Mitigated by raising the heap to 3072 MiB + container limit to 3.5Gi, but the underlying footprint is worth a real look.

The smell

• On-disk record data on published is only ~25 MB of TOML (uncompressed blob bytes across people/projects/updates/buzz/memberships/tag-assignments).
• A cold boot building the in-memory representation needs >1.5 GB of V8 heap — roughly a 60× expansion.
• The full-text store is better-sqlite3 FTS5 (apps/api/src/store/fts.ts), which is off-heap (native memory), so it does not explain the V8 heap blowup. The heap cost is the record maps + secondary indices.

Dataset shape (published)

• people: 31,832
• tag-assignments: 10,353
• tags: 1,005
• projects: 268
• project-memberships: 822 / project-updates: 503 / project-buzz: 112

Worth investigating

• Duplication: does each record exist in more than one in-memory structure (raw map + multiple indices + gitsheets' cached per-sheet dataTree snapshot)? See apps/api/src/store/memory/ (loader.ts, state.ts) and apps/api/src/store/sheet-indices.ts, plus the gitsheets openStore data-tree caching noted in apps/api/src/store/public.ts / memory/reload.ts.
• Per-record overhead: 31.8k people at >40 KB heap each would be ~1.3 GB — are person objects carrying large/derived fields, retained TOML source strings, or boxed structures?
• Index sizing: how many secondary indices, and are any of them O(records) string-keyed Maps holding full objects rather than ids?
• gitsheets retention: confirm whether the parsed blob contents / tree objects are held alongside the typed record maps (a second full copy).

Goal

Bring the cold-boot heap footprint down meaningfully (target: comfortably under 1.5 GB for the current import, with margin to grow), or document why the current footprint is irreducible. This also de-risks prod, where the dataset only grows.

Related: PR #131 (the heap/limit bump mitigation).

[themightychris]

Root-caused — it's the TOML parser, fixed upstream in gitsheets

Measured the boot heap against the full published import and isolated the cause. It is not index sizing, per-record object overhead, or a gitsheets second-copy (dropping the store freed 0 MB) — all the issue's original hypotheses are refuted.

Root cause: @iarna/toml's parser emits string field values as V8 sliced/cons-strings that transitively pin large parser buffers. Each parsed record retains ~12× its source size (~17 KB per ~750-byte person record). People are 90% of the footprint (31.8k records).

Evidence chain:

• People = 525 MB / 581 MB retained; 17.3 KB/record vs 745 B on-disk.
• The person object is 251 B / 10 fields — not the bloat.
• Drop-store + gc → frees 0 MB (state holds it via the records).
• Symbol-strip / ref-copy → no change; JSON roundtrip → 141 MB ⇒ bloat is in the string values, not keys.
• Isolated @iarna.parse retains 5,183 B/parse (422-B record) vs smol-toml 741 B and the Rust toml crate 1,100 B.

Fix

Parser swap in gitsheets: @iarna/toml → smol-toml for parse, keep @iarna for serialize (preserves the byte-stable, human-readable canonical form — zero write churn). Lossless on a 4,310-record corpus; TOML dates stay instanceof Date. gitsheets full suite green (353/353).

→ gitsheets PR JarvusInnovations/gitsheets#197 (+ JarvusInnovations/gitsheets#196 recording the serialization-format analysis for the eventual Rust core).

End-to-end on our real published dataset, via our boot path, with the swapped gitsheets:

Metric	before	after
Retained heap	581 MB	88 MB	6.6×
Transient boot peak	532 MB	71 MB	7.5×
Total heapUsed	609 MB	116 MB	5.3×
RSS	806 MB	292 MB	2.8×

The parser swap fixes both retained footprint and the transient boot spike (an in-app value-flatten would only have fixed retained).

Remaining work here (this repo)

Blocked on the gitsheets release: bump gitsheets once #197 merges/publishes, then we can walk the boot heap (--max-old-space-size) and container limit back down toward node-safe values with large margin — closing this out and de-risking prod growth.

[themightychris]

Resolved — deployed and verified in sandbox

Root cause (@iarna/toml parser pinning ~12× each record's source in V8 sliced/cons-strings) fixed upstream in gitsheets 1.4.1 (smol-toml parser), consumed via #145, and memory limits resized to the real working set via #146.

Live on next-v2.codeforphilly.org (:sandbox image, gitsheets 1.4.1):

	leak era	now (deployed)
Pod memory (cluster top)	~353Mi	244Mi
Container limit	2560Mi	1Gi
--max-old-space-size	2048	512
memory request	1Gi	512Mi
Boot	spiked >1.5Gi (OOM'd at 1536)	clean, 0 restarts, reconciled, HTTP 200

Boot now fits comfortably under a 512MB heap ceiling that would have OOM'd the old parser's ~532MB transient peak — confirming the fix both at retained footprint and the boot spike. Hands the ~3.9Gi nodes ~1.5Gi back (limit 2.5Gi→1Gi; request 1Gi→512Mi), removing the node-starvation risk behind the earlier NodeNotReady.

Follow-up for the future Rust core: JarvusInnovations/gitsheets#196.