Skip to content

fix(openfeature-provider/go): prevent silent SIGSEGV when recovery closes an instance mid-resolve - #512

Merged
fabriziodemaria merged 5 commits into
spotify:mainfrom
tmlye:fix-closed-instance-resolve-crash
Aug 12, 2026
Merged

fix(openfeature-provider/go): prevent silent SIGSEGV when recovery closes an instance mid-resolve#512
fabriziodemaria merged 5 commits into
spotify:mainfrom
tmlye:fix-closed-instance-resolve-crash

Conversation

@tmlye

@tmlye tmlye commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Fixes #511.

Problem

RecoveringResolver.startRecreate closes the old wazero instance in a background goroutine. A concurrent resolve can still be mid-call on that instance, because pool slots only take a shared RLock per resolve. WasmResolver.call holds r.mu for the duration of a call, but Close does not take it. The close therefore lands mid-call. A resolve that races with the close can get a nil api.Function from ExportedFunction, and exportedFunction caches the nil. The next fn.Call dereferences the nil interface. The Go runtime cannot attribute that fault, so it kills the process silently (sigfwdgo → dieFromSignal(SIGSEGV), exit 139, no traceback). See #511 for the core dump analysis.

Changes

  1. WasmResolver.Close now takes r.mu around instance.Close. This serializes close with in-flight calls and removes the race window. Primary fix.
  2. exportedFunction panics with a clear message instead of returning nil, and never caches a nil handle. This is hardening. withRecover already catches panics on this path, so a residual nil becomes a recovered evaluation error instead of silent process death.

Testing

  • New TestResolveOnClosedInstancePanics pins the closed-instance contract. A resolve on a closed instance must end in a recoverable panic. It must never reach fn.Call through a nil handle. The test closes the raw instance directly, because WasmResolver.Close flushes logs first and that would warm fnCache. The direct close exercises the post-close ExportedFunction path.
  • go test ./confidence/internal/local_resolver/ passes.
  • Honest limitation: the silent-death mode only reproduces under the production race, where the close lands inside the mid-call window. A fully closed instance returns a non-nil handle, and its Call fails with a recoverable error. The test therefore pins the contract, not the exact race. The two production cores in Silent process crash (SIGSEGV, no traceback): recovery closes the wasm instance while concurrent resolves still use it #511 are the evidence for the race itself.
  • The TestFlagResolve_* e2e tests fail with "ClientSecret is required" both with and without this change. They need credentials this environment does not have.

@fabriziodemaria fabriziodemaria left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — this looks like the right fix for the cored crashes in #511.

Minor follow-up (non-blocking): consider also skipping cached nil entries on load, not just guarding the fresh lookup path:

if fn, ok := r.fnCache.Load(name); ok && fn != nil {
    return fn.(api.Function)
}

Prevents a stale cached nil from ever reaching fn.Call if one was stored before this fix.

fabriziodemaria and others added 3 commits August 12, 2026 11:05
…ecret is unset

Fork PRs do not receive repository secrets, so CONFIDENCE_CLIENT_SECRET
is empty. Skip e2e and flag-logs tests gracefully instead of failing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…signature

NewWasmResolverFactory now takes a second bool argument (useInterpreter)
after the UseWasmInterpreter feature merged to main.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@fabriziodemaria
fabriziodemaria merged commit e7355f0 into spotify:main Aug 12, 2026
2 checks passed
@github-actions github-actions Bot mentioned this pull request Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Silent process crash (SIGSEGV, no traceback): recovery closes the wasm instance while concurrent resolves still use it

2 participants