Skip to content

fix(pxe): prevent contract sync deadlock on nested syncs#23951

Merged
nchamo merged 1 commit into
merge-train/fairies-v5from
nchamo/sync-deadlock
Jun 9, 2026
Merged

fix(pxe): prevent contract sync deadlock on nested syncs#23951
nchamo merged 1 commit into
merge-train/fairies-v5from
nchamo/sync-deadlock

Conversation

@nchamo

@nchamo nchamo commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Why

The contract sync service uses a semaphore to limit how many scope syncs run concurrently. The problem: a scope sync can trigger a nested sync (a cross-contract utility call from sync_state) that needs a slot of its own. So when 5 syncs each hold a slot and each waits on a nested sync, no slots are left for those nested syncs, and everything deadlocks. And because the semaphore was shared across the whole PXE, unrelated transactions could fill it and trigger the same deadlock.

Fix

Give each sync call its own semaphore, rate-limiting only the scopes within that single call. Those scopes don't depend on each other, so bounding them can't deadlock. A nested sync builds its own limiter, so it always has a slot and can make progress. We still cap resource usage when a single call has many scopes.

@nchamo nchamo self-assigned this Jun 9, 2026
@AztecBot

AztecBot commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Flakey Tests

🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry.

\033FLAKED\033 (8;;http://ci.aztec-labs.com/d1a3d99f7f8f3ac8�d1a3d99f7f8f3ac88;;�): yarn-project/scripts/run_test.sh bb-prover/src/avm_proving_tests/avm_check_circuit_token.test.ts (5s) (code: 0)

@nchamo nchamo requested a review from mverzilli June 9, 2026 01:03

@mverzilli mverzilli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oof, very nice catch. How did you stumble upon this?

@nchamo

nchamo commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

I got the deadlock here: #23938

@nchamo nchamo merged commit aa81397 into merge-train/fairies-v5 Jun 9, 2026
12 checks passed
@nchamo nchamo deleted the nchamo/sync-deadlock branch June 9, 2026 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants