feat(cas): cache blob existence checks for uploads by migmartri · Pull Request #3562 · chainloop-dev/chainloop

migmartri · 2026-10-08T10:51:32Z

Implements spec issue-3543. Closes #3543.

When an upload starts for evidence that is already stored, the Artifact CAS now answers from a cache of positive existence results instead of reading credentials from the secrets manager, building a storage client, and asking the backend. Repeated uploads of the same evidence, from chainloop attestation add or the trace push, return after the first round trip. The CLI and the API do not change.

Existence cache: entries are namespaced by the organization from the signed token, the backend type, the hashed secret reference, and the digest. Only positive results with a known size are cached, so audit events for cached hits keep their size. The cache is always on with a 24h lifetime. It uses a dedicated NATS KV bucket when the CAS has a NATS connection, and an in-memory LRU otherwise, also when the bucket cannot be set up at startup. Cache errors are treated as misses.
Backend client cache: clients loaded for uploads are reused in process memory for 5 minutes. A reused client that fails is dropped and loaded again one time.
Metrics: hit and miss counters for both caches.

Differences from the spec, to be recorded in a separate spec amendment:

R-004: the cache has no settings. It is always on with a fixed 24h lifetime, and there is no off switch.
If the NATS bucket cannot be set up at startup, the CAS falls back to the in-memory cache instead of failing to start.
The client cache key also includes the organization, because the AWS S3 access point backend keeps per-organization session credentials inside the client.
The client cache is only used on the upload path. Downloads load the backend as before.

This change was written with AI assistance (Claude Code).

🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri

Cache positive existence results in the Artifact CAS, namespaced by organization and backend, so repeated uploads of the same evidence skip the secrets manager and the storage backend. Backend clients loaded for uploads are reused for a short time in process memory. The cache uses NATS KV when available and an in-memory LRU otherwise, and can be turned off or tuned through the CAS config and the Helm chart. Implements R-001 to R-009 of spec issue-3543. Refs #3543 Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: 9b6de27d-6322-4cf7-8e72-5e56d858c783

chainloop-platform · 2026-10-08T10:51:41Z

PR validation — ✅ 3 passing

Status	Policy	Material	Messages
✅ Passed	`pr-min-approvals`	`pr-info`	-
✅ Passed	`pr-description-required`	`pr-info`	-
✅ Passed	`pr-user-story-linked`	`pr-info`	-

_{View attestation ↗}

AI Session Checks — 🟡 80% · ⚠️ 1 failing

Avg score	Sessions	Failing policies	Attribution	Files	Lines	Total Duration
🟡 80%	1	⚠️ 1	100% AI / 0% Human	18	+934 / -195	1h24m59s

🟡 80% — 100% AI — ⚠️ 1 policies failing

Oct 8, 2026 10:10 UTC · 1h24m59s · $18.78 · 560 in / 210.9k out · claude-code 2.1.294 (claude-opus-5-5)

_{View session details ↗}

Change Summary

Adds Artifact CAS upload-side existence caching and backend-client reuse.
Uses NATS KV when available, with memory fallback when NATS is absent or bucket setup fails.
Removes the proposed configuration knobs, leaving the cache always on with a fixed 24h TTL.
Adds and reruns cache-focused tests, including fallback and tenant-isolation mutation checks.

AI Session Overall Score

🟡 80% — Well-verified work, but planning and requirement drift deserve review.

AI Session Analysis Breakdown

🟢 92% · verification

🟢 AI validated the change with repeated suite runs and mutation checks. · High Impact

🟢 88% · solution-quality

🟢 Cleanup moved fallback logic into the shared cache constructor, not a caller special case. · High Impact

🟡 72% · user-trust-signal

No notes.

🟡 68% · alignment

🟠 Two user corrections were needed to remove config knobs and keep the spec out of this PR. · Medium Severity

💡 After a second boundary change, ask the AI to restate the final contract before more edits land.

🟡 65% · context-and-planning

🟢 AI read the issue and merged spec before the first implementation edits. · Medium Impact

🟠 The feature started from an issue URL with no visible shared plan for a large multi-phase change. · Medium Severity

💡 For issue-sized work, require a short visible plan before the first code edit.

abstained · scope-discipline

🟡 Scope discipline was not assessed because the final diff stat was unavailable after several reverts. · Low Severity

Missing criteria: scope-discipline

File Attribution

████████████████████ 100% AI / 0% Human

Status	Attribution	File	Lines
modified	ai	`app/artifact-cas/internal/service/bytestream_cache_test.go`	+326 / -35
modified	ai	`app/artifact-cas/internal/service/cache.go`	+148 / -1
modified	ai	`pkg/cache/casexistence/casexistence.go`	+113 / -15
modified	ai	`app/artifact-cas/internal/service/bytestream.go`	+84 / -42
created	ai	`pkg/cache/casexistence/casexistence_test.go`	+84 / -0
modified	ai	`app/artifact-cas/cmd/wire.go`	+50 / -27
modified	ai	`app/artifact-cas/cmd/main_test.go`	+26 / -26
modified	ai	`docs/specs/issue-3543-cas-existence-cache.md`	+17 / -17
modified	ai	`pkg/cache/memory_test.go`	+23 / -0
modified	ai	`app/artifact-cas/internal/conf/conf.proto`	+11 / -11
modified	ai	`deployment/chainloop/values.yaml`	+7 / -7
modified	ai	`app/artifact-cas/internal/service/bytestream_test.go`	+12 / -1
modified	ai	`deployment/chainloop/README.md`	+5 / -5
modified	ai	`pkg/cache/cache.go`	+9 / -1
modified	ai	`pkg/cache/memory.go`	+7 / -2
modified	ai	`app/artifact-cas/internal/service/service.go`	+7 / -0
modified	ai	`deployment/chainloop/templates/cas/configmap.yaml`	+3 / -3
modified	ai	`deployment/chainloop/Chart.yaml`	+2 / -2

Policies (4, 1 failing)

Status	Policy	Material	Messages
✅ Passed	`ai-config-ai-agents-allowed`	`ai-coding-session-9b6de2`	-
✅ Passed	`ai-config-no-dangerous-commands`	`ai-coding-session-9b6de2`	-
⚠️ Failed	`ai-config-no-secrets`	`ai-coding-session-9b6de2`	Secret (generic-password) detected in session content [turn=762, source=tool_result, line=10]: INF redacted secrets from the AI coding session before upload count=50 rules=["generic-[REDACTED:generic-password]"]
✅ Passed	`ai-config-mcp-servers-allowed`	`ai-coding-session-9b6de2`	-

Security Checks — ✅ 5 passing

✅ `secret-scan`

Status	Policy	Messages
✅ Passed	`secrets-detection`	-

✅ `sast-scan`

Status	Policy	Messages
✅ Passed	`owasp-top10-2025`	-
✅ Passed	`sast`	-
✅ Passed	`cwe-top25`	-
✅ Passed	`cwe-top26-40-cusp`	-

Scans not applied (3)

Scan	Reason
`vulnerability-scan`	no manifest/lockfile changed
`github-actions-scan`	no workflow files changed
`iac-scan`	no IaC files changed

_{View attestation ↗}

Security context

This change touches code with 15 recorded security-fix advisories. These are pointers to what past fixes established, not findings in this diff, and they never fail the check.

_{View in Chainloop ↗ · How this works ↗}

Powered by Chainloop and Chainloop Trace

cubic-dev-ai

All reported issues were addressed across 18 files

_{Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic}

Share one backend client cache between services, collapse the client reload retry into a loop, move the NATS to memory fallback into casexistence.New, and simplify the cache tests. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: 9b6de27d-6322-4cf7-8e72-5e56d858c783

Remove the existence cache settings from the CAS config and the Helm chart. The cache is always on with a 24h lifetime, in NATS KV when available and in memory otherwise, also when the NATS bucket cannot be set up at startup. Record the build-time decisions in spec issue-3543 (R-010, D-010 to D-013). Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: 9b6de27d-6322-4cf7-8e72-5e56d858c783

cubic-dev-ai

All reported issues were addressed across 13 files (changes from recent commits).

_{Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic}

The spec amendment for issue-3543 moves to its own spec PR. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: 9b6de27d-6322-4cf7-8e72-5e56d858c783

cubic-dev-ai Bot reviewed Oct 8, 2026

View reviewed changes

Comment thread deployment/chainloop/README.md Outdated

Comment thread pkg/cache/memory.go

Comment thread app/artifact-cas/cmd/wire.go Outdated

migmartri requested a review from a team October 8, 2026 10:59

migmartri added 2 commits October 8, 2026 13:07

cubic-dev-ai Bot reviewed Oct 8, 2026

View reviewed changes

Comment thread docs/specs/issue-3543-cas-existence-cache.md Outdated

migmartri assigned javirln Oct 8, 2026

chore: drop spec amendment from the implementation PR

c6d1fc4

The spec amendment for issue-3543 moves to its own spec PR. Assisted-by: Claude Code Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev> Chainloop-Trace-Sessions: 9b6de27d-6322-4cf7-8e72-5e56d858c783

migmartri commented Oct 8, 2026

View reviewed changes

Comment thread app/artifact-cas/internal/service/bytestream.go

javirln approved these changes Oct 8, 2026

View reviewed changes

migmartri merged commit 843e69a into main Oct 8, 2026
16 of 17 checks passed

migmartri deleted the feat/cas-existence-cache branch October 8, 2026 12:31

BrewTestBot mentioned this pull request Oct 8, 2026

chainloop-cli 1.119.0 Homebrew/homebrew-core#316548

Merged

Navigation Menu

Repository navigation

feat(cas): cache blob existence checks for uploads - #3562

feat(cas): cache blob existence checks for uploads#3562
migmartri merged 4 commits into
mainfrom
feat/cas-existence-cache

migmartri commented Oct 8, 2026 •

edited

Loading

Uh oh!

chainloop-platform Bot commented Oct 8, 2026 •

edited

Loading

🟡 80% — 100% AI — ⚠️ 1 policies failing

Change Summary

AI Session Overall Score

AI Session Analysis Breakdown

🟢 92% · verification

🟢 88% · solution-quality

🟡 72% · user-trust-signal

🟡 68% · alignment

🟡 65% · context-and-planning

abstained · scope-discipline

File Attribution

Policies (4, 1 failing)

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

migmartri commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chainloop-platform Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR validation — ✅ 3 passing

AI Session Checks — 🟡 80% · ⚠️ 1 failing

🟡 80% — 100% AI — ⚠️ 1 policies failing

Change Summary

AI Session Overall Score

AI Session Analysis Breakdown

🟢 92% · verification

🟢 88% · solution-quality

🟡 72% · user-trust-signal

🟡 68% · alignment

🟡 65% · context-and-planning

abstained · scope-discipline

File Attribution

Policies (4, 1 failing)

Security Checks — ✅ 5 passing

✅ secret-scan

✅ sast-scan

Security context

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

migmartri commented Oct 8, 2026 •

edited

Loading

chainloop-platform Bot commented Oct 8, 2026 •

edited

Loading

✅ `secret-scan`

✅ `sast-scan`

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading