Skip to content

Reset a statement after run() the way better-sqlite3 does - #238

Open
penberg wants to merge 1 commit into
mainfrom
reset-statement-after-run
Open

penberg wants to merge 1 commit into
mainfrom
reset-statement-after-run

Conversation

@penberg

@penberg penberg commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

Builds on #237 by @ejc3 (the commit keeps their authorship) and widens the fix to match better-sqlite3 exactly. Supersedes #237. Report with reproductions: tursodatabase/libsql-client-ts#352.

Problem

run() reset a statement only before stepping it, and returned as soon as the step returned. SQLite leaves a statement in progress in two cases that this missed:

  • The step failed with SQLITE_BUSY. SQLite suspends the statement so the caller can step it again, and it keeps counting as an active writer. Every COMMIT on the connection then fails with SQLITE_BUSY: cannot commit transaction - SQL statements in progress until the statement is reused or garbage collected, although the lock is free. Through @libsql/client, one write batch that fails busy makes the following batches on the same client fail too.
  • The step returned a row. libsql's run() treats SQLITE_ROW as success, so INSERT ... RETURNING or PRAGMA journal_mode=WAL run through run() stopped after the first row. In autocommit mode the write is only committed when the statement halts, so the row stayed invisible to other connections, the write lock stayed held, the connection's own next COMMIT failed with the same error, and changes was reported as 0 because SQLite records it at halt.

Probe on main, run() on INSERT INTO t VALUES (?) RETURNING x:

                              better-sqlite3      libsql (main)
changes reported by run()     1                   0
row visible to 2nd conn       yes                 no
2nd connection write          OK                  SQLITE_BUSY
same connection BEGIN..COMMIT OK                  SQLITE_BUSY: cannot commit transaction -
                                                  SQL statements in progress

Fix

better-sqlite3 steps and then unconditionally resets in run(), and reads changes and lastInsertRowid after the reset. Do the same in both the sync and the promise API. In the promise API the reset runs inside the tokio future, on the thread that stepped the statement, before the result is handed back to the JavaScript thread.

Tests

In both sync.test.js and async.test.js, each keeping the statement referenced so that garbage collection cannot hide the bug:

  • BEGIN IMMEDIATE fails with SQLITE_BUSY against a second connection holding the write lock, and a write transaction on the same connection must then commit (from Reset a statement when run() fails #237).
  • INSERT ... RETURNING via run() reports changes of 1, is visible to and does not block a second connection, and the same connection can then run a transaction.
  • SELECT via run() does not keep a read transaction open in rollback-journal mode.
  • PRAGMA journal_mode=WAL via run() does not block the next COMMIT.

All of them pass on better-sqlite3 as well. Before the fix, the busy and RETURNING cases fail on libsql. With the fix: sqlite 56, libsql 56, async 71, extensions 29, concurrency 4, connections 2.

`run()` reset a statement only before stepping it, and returned as soon
as the step returned. SQLite leaves a statement in progress in two
cases that this missed:

* The step failed with `SQLITE_BUSY`. SQLite suspends the statement so
  the caller can step it again, and it keeps counting as an active
  writer. Every `COMMIT` on the connection then fails with
  `SQLITE_BUSY: cannot commit transaction - SQL statements in progress`
  until the statement is reused or garbage collected, although the lock
  is free. Through `@libsql/client`, one write batch that fails busy
  makes the following batches on the same client fail too.

* The step returned a row. libsql's `run()` treats `SQLITE_ROW` as
  success, so `INSERT ... RETURNING` or `PRAGMA journal_mode=WAL` run
  through `run()` stopped after the first row. In autocommit mode the
  write is only committed when the statement halts, so the row stayed
  invisible to other connections, the write lock stayed held, the
  connection's own next `COMMIT` failed with the same error, and
  `changes` was reported as 0 because SQLite records it at halt.

better-sqlite3 steps and then unconditionally resets in `run()`, and
reads `changes` and `lastInsertRowid` after the reset. Do the same in
both the sync and the promise API. In the promise API the reset runs
inside the tokio future, on the thread that stepped the statement,
before the result is handed back to the JavaScript thread.

Tests, in both `sync.test.js` and `async.test.js`, keep the statement
referenced so that garbage collection cannot hide the bug:

* `BEGIN IMMEDIATE` fails with `SQLITE_BUSY` against a second
  connection holding the write lock, and a write transaction on the
  same connection must then commit.
* `INSERT ... RETURNING` via `run()` reports `changes` of 1, is visible
  to and does not block a second connection, and the same connection
  can then run a transaction.
* `SELECT` via `run()` does not keep a read transaction open in
  rollback-journal mode.
* `PRAGMA journal_mode=WAL` via `run()` does not block the next
  `COMMIT`.

All of them pass on better-sqlite3 as well. Before the fix, the busy
and RETURNING cases fail on libsql.

Co-authored-by: Pekka Enberg <penberg@turso.tech>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants