Skip to content

Derive toolkit MCP executors over the daemon's open database#1409

Open
jackulau wants to merge 1 commit into
UsefulSoftwareCo:mainfrom
jackulau:fix/1406-toolkit-mcp-data-dir-lock
Open

Derive toolkit MCP executors over the daemon's open database#1409
jackulau wants to merge 1 commit into
UsefulSoftwareCo:mainfrom
jackulau:fix/1406-toolkit-mcp-data-dir-lock

Conversation

@jackulau

Copy link
Copy Markdown
Contributor

Fixes #1406.

The bug

The daemon's boot bundle holds the data dir's ownership lock (a BEGIN EXCLUSIVE held open on data.db.owner-lock, per-connection, busy_timeout = 0) for its whole lifetime.

The toolkit MCP resource did not reuse that bundle. It built its executor with:

const handle = await createExecutorHandle({ activeToolkitSlug: resource.slug });

which goes createLocalExecutorLayer -> openOwnedLocalDatabase -> acquireDataDirOwnership, taking the same exclusive lock the process already holds. That can never succeed while the daemon runs, so every /mcp/toolkits/<slug> request hit SQLITE_BUSY and returned a 500 (-32603), while the default /mcp resource kept working.

The fix

activeToolkitSlug only varies the plugin set (it adds a tool policy provider to the toolkits plugin). Tenant, subject, and data dir are fixed for the process, so the second database open was never needed.

This extracts the makeExecutor(plugins) seam over the bundle's already-open handle and derives toolkit executors from it, mirroring how self-host pairs a long-lived database with per-request plugins (SelfHostPluginsProvider).

The seam builds over the inner FumaDB handle (sqlite.db), not the owning { db, close } wrapper. That is load-bearing: createExecutor only closes the database when it was handed a wrapper, so a scoped executor's close() tears down its own plugins and never the shared database.

Verification

The reproduction is e2e/local/toolkits-mcp.test.ts, which boots a real server and drives /mcp/toolkits/<slug>:

  • With the fix stashed it fails with exactly the reported error: {"error":{"code":-32603,"message":"Internal server error"}}, HTTP 500.
  • With the fix applied it passes.

Gates: format:check clean, lint 0 warnings / 0 errors, typecheck 43/43, test 37/37.

Why this shipped unnoticed, and what now guards it

The e2e-local job ran only stdio-mcp.test.ts, so the toolkit path had no coverage at all. This adds toolkits-mcp.test.ts to that job (neither scenario drives a browser, so the job's browser-flakiness rationale does not apply to them).

That job is gated if: github.event_name != 'pull_request' though, so on its own it would only catch a regression after merge. The more useful guard is two new cases in apps/local/src/executor.test.ts, which run in the normal test chain on every PR:

  1. A toolkit-scoped executor can be derived while the bundle holds the data dir. This is the Local daemon: /mcp/toolkits/<slug> always 500s — toolkit resource re-opens the owned data dir and deadlocks on its own lock #1406 condition, at unit level.
  2. Disposing a scoped executor leaves the shared database open.

The second is the one worth having. I verified it by mutation: changing db: sqlite.db to db: sqlite in apps/local/src/executor.ts still passes typecheck (the type system does not catch it, since SqliteFumaDb structurally satisfies ExecutorDb) but fails the new test with LibsqlError: CLIENT_CLOSED: The client is closed. Without that test, a future refactor could silently make every /mcp and /api request fail the moment any toolkit session ended.

Notes

  • No changeset: @executor-js/local is in the changesets ignore list and is private.
  • Worth a maintainer's call, not done here: now that e2e-local runs only headless scenarios, the if: github.event_name != 'pull_request' skip may be worth revisiting. Flipping it makes every PR pay the job, so I left it alone.

The daemon's boot bundle holds the data dir's ownership lock (a BEGIN
EXCLUSIVE held open on `data.db.owner-lock`, per-connection, with
`busy_timeout = 0`) for its whole lifetime. The toolkit MCP resource built
its executor with `createExecutorHandle({ activeToolkitSlug })`, which opens
a second owned database and so deadlocks against the lock this very process
holds. Every `/mcp/toolkits/<slug>` request failed with SQLITE_BUSY,
surfaced as a 500 (-32603), while the default `/mcp` resource kept working.

`activeToolkitSlug` only varies the plugin set: it adds a tool policy
provider to the toolkits plugin, while tenant, subject, and data dir are
fixed for the process. The second open was never needed. Extract the
`makeExecutor(plugins)` seam over the bundle's already-open handle and
derive toolkit executors from it, mirroring how self-host pairs a
long-lived database with per-request plugins.

The executor is built over the inner FumaDB handle rather than the owning
`{ db, close }` wrapper, so a scoped executor's `close()` tears down its own
plugins and never the shared database.

Cover both invariants in `apps/local` unit tests, which run on every PR, and
add the toolkits scenario to the e2e job that previously ran only the stdio
one. That gap is why this shipped unnoticed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Local daemon: /mcp/toolkits/<slug> always 500s — toolkit resource re-opens the owned data dir and deadlocks on its own lock

1 participant