fix(search): isolate Google crawl failures and bound embedding writes

[waleedlatif1]

Summary

• Continue central Google crawls past narrowly identified user-access failures, retain diagnostics, and hold deletion reconciliation until the listing is complete.
• Report inaccessible external nested groups as partial Drive permission syncs while preserving verified membership freshness and blocking directory-wide failures.
• Reduce embedding insert batches while keeping document replacement, search projections, and provenance atomic; add the rollback regression to PostgreSQL CI.
• Update Google connector troubleshooting documentation.

Type of Change

• Bug fix

Testing

• 1,380 connector, crawl-engine, access, and document-source tests passed.
• 22 PostgreSQL permission tests and a real second-batch rollback/retry test passed.
• Type-check, lint, block-registry validation, all 46 repository audits, and generated-artifact checks passed.

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 17, 2026 6:32am UTC

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; the changed failure-isolation, permission-freshness, reconciliation, and embedding-transaction paths preserve their required safety boundaries.

Summary

This PR makes Google Workspace crawls resilient to narrowly classified user-access failures, preserves bounded failure diagnostics through resumable listings, allows partial Drive directory refreshes for inaccessible external nested groups, and reduces embedding insert batch size without weakening transaction atomicity.

• Failed Gmail or Calendar user listings no longer prevent later users from being crawled.
• Listing failures suppress deletion reconciliation and are surfaced as partial sync diagnostics.
• External nested-group access failures retain last-known membership subject to the existing freshness limit.
• Embedding writes use 100-row statements inside the existing document transaction, with PostgreSQL rollback coverage added to CI.
• Google connector troubleshooting documentation reflects the new behavior.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Start connector sync] --> B[Refresh mirrored directory]
  A --> C[List Google Workspace users]
  B -->|Complete| D[Directory refreshed]
  B -->|Only external nested groups inaccessible| E[Partial directory warning]
  B -->|Directory-wide or unknown failure| F[Abort sync]
  C --> G[List current user's documents]
  G -->|Success| H[Process documents]
  G -->|Narrow user-access failure| I[Record bounded listing failure]
  G -->|Credential, quota, or unclassified failure| F
  I --> J[Continue with next user]
  H --> J
  J -->|More users| G
  J -->|Listing complete| K{Any listing failure?}
  K -->|Yes| L[Keep unlisted documents and report partial sync]
  K -->|No| M[Reconcile deletions]
  L --> N[Retry in next generation]
  M --> O[Complete sync]
  E --> L

Loading

_{Reviews (1) · Last reviewed commit: "fix(search): isolate Google crawl failur..."}

[cubic-dev-ai]

No issues found across 28 files

Confidence score: 5/5

• Automated review surfaced no issues in the provided summaries.
• No files require special attention.

_{Re-trigger cubic}