fix(provenance): close durable writer and reader gaps

[icecrasher321]

Summary

• Preserve memory provenance across tool and Agent writes, sanitized messages, and concurrent first appends. Recover usable historical bindings without newly blocking legacy reads.
• Keep newly combined file content unsafe when unrecorded bytes are mixed with known secrets. Preserve existing legacy and unrecorded-only behavior.
• Count distinct secrets across durable message and column bindings, retain selection and source scope, and enforce byte budgets incrementally. Remove redundant hash limits and intermediate copies.
• Apply the existing policy before knowledge reranking and table snapshot mounts. Retain all rollout flags, improve writer/refusal diagnostics, and clarify the documented compatibility policy.

Type of Change

• Bug fix

Testing

• 414 tests passed across 28 suites, including file append-to-model projection, V2 route-to-provider payload checks, and real PostgreSQL memory/table integration tests under both flag settings.
• PostgreSQL regressions cover concurrent first appends, 17,000-message histories, 11,000 column bindings, derived transformations, and real secret/byte limit failures. These suites run in CI.
• Lint, all 46 repository audits, generated-artifact checks, and all 26 workspace type checks passed on the latest staging base.

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 8, 2026 4:32am UTC

[greptile-apps]

Greptile Summary

This PR closes durable secret-provenance gaps across memory, knowledge reranking, table snapshot mounts, workspace files, and client-tool completion handling.

• Serializes concurrent memory writers and binds provenance to the exact sanitized messages persisted.
• Recovers usable historical memory bindings while preserving legacy compatibility and bounded failure behavior.
• Applies provenance checks before knowledge reranking and table snapshot model mounts, with rollout-aware handling of unrecorded data.
• Adds bounded writer, refusal, recovery, decryption, and client-seal diagnostics without logging secret material.
• Adds route-level, unit, and PostgreSQL integration coverage and documents the knowledge-search conflict response.

Confidence Score: 5/5

The PR appears safe to merge; no concrete correctness, security, or repository-rule violations remain.

The changed model-egress paths either import and redact exact provenance, preserve explicitly flag-controlled legacy compatibility, or fail closed when enforcement requires it. Memory concurrency and historical recovery are bounded and covered by PostgreSQL and route-level regressions.

Important Files Changed

Filename	Overview
apps/sim/executor/handlers/agent/memory.ts	Sanitizes messages before binding, serializes memory writes, and safely preflights historical provenance recovery.
apps/sim/lib/memory/secret-provenance.ts	Adds per-message provenance binding, bounded historical recovery, and more precise unknown-write diagnostics.
apps/sim/lib/memory/locks.ts	Introduces a transaction-scoped PostgreSQL advisory lock keyed by workspace and conversation.
apps/sim/lib/knowledge/application/search.ts	Imports stored result provenance before reranking and refuses unsafe provider egress under the configured policy.
apps/sim/lib/table/rows/secret-provenance.ts	Aggregates snapshot safety classifications and attributes non-exact writes after their sidecars are bound.
apps/sim/lib/uploads/contexts/workspace/workspace-file-secret-provenance.ts	Adds safe diagnostics for non-exact writes and provenance-based model-egress refusals.
apps/sim/lib/copilot/request/tools/client-completion-seal.server.ts	Classifies sealed completion and context failures without exposing encrypted or decrypted contents.
.github/workflows/test-build.yml	Adds PostgreSQL integration coverage for table provenance classification and concurrent memory writes.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Tool or Agent output] --> B[Sanitize persisted value]
  B --> C[Bind secret provenance to value]
  C --> D[(Durable store and sidecar)]
  D --> E{Model-facing read}
  E --> F[Validate sidecar binding]
  F -->|Exact| G[Import scoped or anonymous secrets]
  F -->|Unrecorded, flag off| H[Proceed and report diagnostic]
  F -->|Unavailable or enforced| I[Refuse model egress]
  G --> J[Redact provider payload]
  J --> K[Knowledge reranker or agent model]

Loading

_{Reviews (1): Last reviewed commit: "fix(provenance): close durable writer an..." | Re-trigger Greptile}

[icecrasher321]

@cubic review

[cubic-dev-ai]

@cubic review

@icecrasher321 I have started the AI code review. It will take a few minutes to complete.

[icecrasher321]

@cubic review

[cubic-dev-ai]

@cubic review

@icecrasher321 I have started the AI code review. It will take a few minutes to complete.

[icecrasher321]

@cubic review

[cubic-dev-ai]

@cubic review

@icecrasher321 I have started the AI code review. It will take a few minutes to complete.