Skip to content

Commit 7931dcc

Browse files
authored
fix(file-search): store workspace file revisions at millisecond precision (#7941)
* fix(file-search): store workspace file revisions at millisecond precision The content revision round-trips through JavaScript Date and JSON, both of which truncate to milliseconds, while now() stores microseconds. A sub-millisecond revision stops comparing equal to its own round-trip, so every SQL equality keyed on it matches zero rows and the file is never claimed, indexed, or cleaned up. Truncate at the column default and enforce it with a BEFORE trigger for the writers a default cannot reach, then retire the rows minted before the invariant existed in a batched script migration. * fix(file-search): normalize revisions promoted by metadata-only writes A chat upload is materialized into a workspace file by setting context alone, so the revision is never written and the search-index trigger would key the promoted file to a value nothing can claim. Fire the normalizer on every update; `UPDATE OF` is evaluated against the statement's target columns, so this does not demote provenance on a metadata write. Accept the shared integration database in the repair suite the way the sibling suites do, and record the new script migration in the registry order assertions. * chore(db): drop the 0357 migration ahead of renumbering * chore(db): renumber the content revision migration to 0358 Staging took 0357 for the outbox autovacuum migration. The SQL is byte for byte the previously reviewed file; only the number and the references to it changed.
1 parent 2112998 commit 7931dcc

10 files changed

Lines changed: 28029 additions & 2 deletions

‎.github/workflows/test-build.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -222,7 +222,10 @@ jobs:
222222
working-directory: packages/db
223223
env:
224224
KNOWLEDGE_ACL_TEST_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_auth_scim
225-
run: bunx vitest run script-migrations/0016_backfill_search_vectors.postgres.test.ts
225+
run: >-
226+
bunx vitest run
227+
script-migrations/0016_backfill_search_vectors.postgres.test.ts
228+
script-migrations/0018_repair_workspace_file_content_revision.postgres.test.ts
226229
227230
- name: Verify Search progress, pagination, and outbox scheduling in PostgreSQL
228231
working-directory: apps/sim
Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
ALTER TABLE "workspace_files" ALTER COLUMN "content_updated_at" SET DEFAULT date_trunc('milliseconds', now());--> statement-breakpoint
2+
-- The trigger is the invariant; the truncating default above only keeps the common insert path from
3+
-- paying for it. A default cannot cover the writers that matter here: explicit `CURRENT_TIMESTAMP`
4+
-- expressions and raw SQL inserts.
5+
--
6+
-- It fires on every UPDATE rather than `UPDATE OF content_updated_at`, because a row can enter the
7+
-- search index without its revision being written: materializing a chat upload sets `context` alone, and
8+
-- `workspace_files_search_index_pending` then indexes whatever revision the row already carried. `UPDATE
9+
-- OF` is evaluated against the statement's target columns, so widening it does not make
10+
-- `workspace_files_secret_provenance_demote` fire on those metadata writes — normalizing a legacy row
11+
-- keeps its tracked provenance. The WHEN guard keeps the plpgsql call off the hot path for the compliant
12+
-- writes that are the norm, and the name sorts before both other triggers so they see the normalized
13+
-- value. Rows minted before this runs are retired by script migration
14+
-- `0018_repair_workspace_file_content_revision`, which the runner applies after every SQL migration.
15+
CREATE OR REPLACE FUNCTION workspace_file_content_version_millisecond()
16+
RETURNS trigger
17+
LANGUAGE plpgsql
18+
AS $$
19+
BEGIN
20+
NEW.content_updated_at := date_trunc('milliseconds', NEW.content_updated_at);
21+
RETURN NEW;
22+
END;
23+
$$;--> statement-breakpoint
24+
DROP TRIGGER IF EXISTS workspace_files_content_version_millisecond ON workspace_files;--> statement-breakpoint
25+
CREATE TRIGGER workspace_files_content_version_millisecond
26+
BEFORE INSERT OR UPDATE ON workspace_files
27+
FOR EACH ROW
28+
WHEN (NEW.content_updated_at <> date_trunc('milliseconds', NEW.content_updated_at))
29+
EXECUTE FUNCTION workspace_file_content_version_millisecond();

0 commit comments

Comments
 (0)