Skip to content

feat(cleanup): add per-type row limits to existing jobs - #7842

Merged
TheodoreSpeaks merged 6 commits into
stagingfrom
feat/bounded-cleanup
Sep 15, 2026
Merged

TheodoreSpeaks merged 6 commits into
stagingfrom
feat/bounded-cleanup

Conversation

@TheodoreSpeaks

@TheodoreSpeaks TheodoreSpeaks commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Accept per-type row limits on the existing log and soft-delete cleanup endpoints, returning HTTP 202 and one queued job ID.
  • Pass a shared remaining budget through the existing cleanup selectors so limits apply across every workspace and organization chunk. Omitted types are skipped.
  • Serialize retention jobs, disable automatic task retries, and use smaller owner-discovery pages for manual runs. Fail manual jobs on owner subscription lookup errors. Preserve existing retention rules, billing, storage, and child cleanup behavior.

Type of Change

  • New feature

Testing

  • 107 focused contract, route, dispatch, worker, and cleanup-helper tests passed.
  • Lint, all 46 audits, block registry, docs manifest, and app/auth type checks passed.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Sep 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Sep 15, 2026 6:25pm UTC

Request Review

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

@greptile-apps

greptile-apps Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge; no actionable new defects or outstanding previous findings remain.

Summary

This PR adds authenticated, per-type row limits to the existing log and soft-delete cleanup endpoints while retaining the existing behavior for parameterless scheduled calls.

  • Queues each bounded request as one serialized Trigger.dev job with automatic retries disabled.
  • Shares mutable row budgets across workspace and organization scopes, counting selected roots before destructive side effects.
  • Extends cleanup helpers for bounded log, metadata, snapshot, file, knowledge-base, and other soft-delete cleanup.
  • Adds route, contract, dispatcher, worker, and helper coverage plus operational documentation.

Diagram

sequenceDiagram
    participant Cron as Maintenance caller
    participant API as Cleanup API
    participant Trigger as Trigger.dev
    participant Worker as Cleanup worker
    participant Owners as Owner discovery
    participant Cleanup as Existing cleanup helpers

    Cron->>API: GET with per-type limits
    API->>API: Authenticate and validate limits
    API->>Trigger: Queue one bounded job
    API-->>Cron: 202 + runId + limits
    Trigger->>Worker: Run serialized job
    Worker->>Worker: Create shared row budgets
    loop Until every budget is exhausted or owners end
        Worker->>Owners: Read next owner page
        Owners-->>Worker: Retention scopes
        Worker->>Cleanup: Run scope with shared budgets
        Cleanup->>Cleanup: Select roots and consume budgets
        Cleanup-->>Worker: Cleanup result
    end
Loading

Reviews (6) · Last reviewed commit: "fix(cleanup): fail manual jobs on owner ..."

Comment thread apps/sim/background/cleanup-logs-bounded.ts Outdated
Comment thread apps/sim/background/cleanup-logs-bounded.ts Outdated
Comment thread apps/sim/background/cleanup-soft-deletes-bounded.ts Outdated
Comment thread apps/sim/lib/cleanup/bounded-delete.ts Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

Comment thread apps/sim/background/cleanup-logs-bounded.ts Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

Comment thread apps/sim/lib/cleanup/storage-outbox.ts Outdated
Comment thread apps/sim/background/cleanup-bounded.test.ts Outdated
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

@TheodoreSpeaks TheodoreSpeaks changed the title feat(cleanup): add bounded manual retention runs feat(cleanup): add per-type row limits to existing jobs Sep 15, 2026
@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

Fixed the owner-lookup finding in 8e834e3. Manual cleanup now propagates personal, organization-workspace, and organization subscription lookup errors instead of reporting success after skipping those owners. Added regression coverage for all three paths; existing scheduled dispatch behavior is preserved.

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@greptile

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@TheodoreSpeaks I have started the AI code review. It will take a few minutes to complete.

@TheodoreSpeaks
TheodoreSpeaks merged commit 8fea7cf into staging Sep 15, 2026
33 of 34 checks passed
@waleedlatif1
waleedlatif1 deleted the feat/bounded-cleanup branch September 15, 2026 19:24

This branch was previously deployed

1 inactive deployment
Preview 8e834e33 Deployed Sep 15, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant