Skip to content

v0.6.85: mothership stream, resource column spacing, prospeo, findymail integrations, markdown google docs creation#4659

Open
waleedlatif1 wants to merge 10 commits into
mainfrom
staging
Open

v0.6.85: mothership stream, resource column spacing, prospeo, findymail integrations, markdown google docs creation#4659
waleedlatif1 wants to merge 10 commits into
mainfrom
staging

Conversation

@waleedlatif1
Copy link
Copy Markdown
Collaborator

icecrasher321 and others added 10 commits May 17, 2026 15:05
…he cleanup (#4646)

* improvement(redis-cleanup): schedule, async workflow, hitl bae64 cache cleanup

* address comments
* improvement(mothership): abort path race preventing persistence

* address comments

* address bugbot comment
…hunk viewer (#4643)

* fix(knowledge): preserve scroll position when toggling tokenizer in chunk viewer

* fix(knowledge): skip scroll restore on initial mount of chunk editor

* chore(dev): add dev:clean script to purge Turbopack cache
…o fix heap growth (#4652)

* fix(memory): prune toolSchemaCache and semaphores to prevent heap growth

toolSchemaCache (lib/copilot/chat/payload.ts): module-level Map keyed by
userId:workspaceId never deleted expired entries, only checked TTL on read.
With 100K+ unique user/workspace pairs each holding 50-200KB of tool schemas,
this was the primary driver of the 24MB -> 25GB heap growth observed in
CloudWatch. Add a setInterval sweep every 30s (matching the TTL) with .unref()
so it does not prevent graceful shutdown.

semaphores (lib/core/async-jobs/backends/database.ts): acquireSlot created
Semaphore entries that releaseSlot never deleted. With per-execution UUID keys
(e.g. scheduleJobId), each scheduled workflow run would add a permanent entry.
Store the concurrency limit on the Semaphore struct and delete the entry from
the Map when all slots are free and no waiters remain.

validatorCache (lib/copilot/tools/server/generated-schema.ts): validated as
bounded (93 tools x 2 schema kinds = 186 max entries, ~2-9MB). No fix needed.

isolated-vm nativeContexts: validated as deferred GC, self-healed by worker
rotation at MAX_EXECUTIONS_PER_WORKER=200. externalMB spikes trace to
concurrent isolate heaps at peak load (128MB limit x active isolates), not a
reference leak. No fix needed.

* fix(memory): prune effectiveEnvCache and instrument cache sizes in telemetry

effectiveEnvCache (lib/environment/utils.ts): same unbounded accumulation
pattern as toolSchemaCache — module-level Map keyed by userId:workspaceId
with a 15s TTL that is only checked on read, never proactively evicted.
Adds a periodic sweep matching the TTL interval with .unref().

cache-registry (lib/monitoring/cache-registry.ts): lightweight registry
so modules can expose their cache sizes to telemetry without coupling.
toolSchemaCache and effectiveEnvCache both register on module load.

memory-telemetry: emits cacheSizes in every Memory snapshot log so
CloudWatch can confirm the caches stay bounded post-deploy.

* improvement(memory): replace manual TTL Maps with lru-cache for toolSchemaCache and effectiveEnvCache

Replaces the homegrown Map + setInterval sweep pattern with LRUCache from
the lru-cache npm package, which is the standard Node.js solution for
bounded in-process caching with TTL.

Changes per cache:
- Removes manual ToolSchemaCacheEntry / EffectiveEnvCacheEntry types
- Removes setInterval sweep timers (and the .unref() boilerplate)
- Removes the two-phase promise->value entry update inside the IIFE
- Stores Promise<T> directly — in-flight and resolved states share one type
- max: 200 (toolSchemaCache) / max: 500 (effectiveEnvCache) as hard ceilings
- TTL behaviour and concurrent-request deduplication are preserved exactly
- cache-registry .size reporting works unchanged via lru-cache's .size prop

* fix(memory): remove redundant waiters guard in releaseSlot
… search (#4653)

* feat(prospeo): add Prospeo integration for B2B contact enrichment and search

Adds 8 operations: enrich person/company, bulk enrich person/company,
search person/company, search suggestions, and account information.
Uses X-KEY header auth.

* refactor(prospeo): extract shared parse helpers into utils.ts
* feat(findymail): add Findymail B2B contact data integration

Adds 11 tools covering verified email lookup (by name, LinkedIn, domain
roles), email verification, reverse email lookup with profile enrichment,
company info, employee discovery, phone lookup, technology stack
detection, and credit checks. Single API-key block with operation
dropdown, gradient-rendered icon, and generated docs.

* fix(findymail): handle HTTP errors and surface last_detected_at

- All 11 tools now check response.ok and return success:false with the API error message on non-2xx responses
- search_technologies now maps last_detected_at to match lookup_technologies and the shared output schema
- Restore file_v3 in docs icon-mapping (translated docs still reference it)

* improvement(findymail): exclude operation from params transform

Match the convention used by enrich/apify/box/calendly — destructure out
operation before forwarding the rest to the tool call, so the operation
key doesn't leak into the tool payload.
…st stacking (#4655)

* improvement(workspace): fix resource table column proportions and toast stacking

* fix(resource): restore scrollbar-gutter stable on table scroll container

* fix(resource): remove scrollbar-gutter stable — single-table layout doesn't need it

* fixed files cols

* fix(findymail): add required enabled field to wandConfig entries

* fix(findymail): remove optional from block outputs — not valid on BlockConfig output type

* fixes

* fix(files): use folderSizeMap for sort value so size sort matches display

* files ref
…4657)

* fix(tables): type-aware SQL casts for range filters on date columns

* improvement(table): tighten filter-cast types & workspace guards

- Drop redundant tableId/workspaceId from BulkUpdateData and BulkDeleteData; service uses table.id / table.workspaceId so column metadata and DB scope can't drift apart.
- Add missing workspace-id guards to copilot user-table cases (insert_row, batch_insert_rows, update_row, batch_update_rows, rename); collapse duplicated rename check.
- Add service-level integration tests that buildFilterClause/buildSortClause receive table.schema.columns from queryRows, updateRowsByFilter, deleteRowsByFilter.

* improvement(table): cast jsonb date filters/sorts to timestamptz

::timestamp strips timezone offsets from ISO strings, making comparisons
depend on the server TimeZone setting. ::timestamptz preserves the offset
so chronological comparisons are correct regardless of server config.

* improvement(table): correct JSDoc examples for required columns arg

* improvement(table): validate range operator value types at SQL builder
…4656)

* feat(google_docs): opt-in Markdown formatting for create operation

* fix(google_docs): harden multipart boundary handoff and align postProcess guard
@vercel
Copy link
Copy Markdown

vercel Bot commented May 19, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped May 19, 2026 2:42am

Request Review

@cursor
Copy link
Copy Markdown

cursor Bot commented May 19, 2026

PR Summary

Medium Risk
Medium risk due to changes in copilot chat finalization/persistence (new transactional locking and cancelled-stream handling) and table filtering now depends on column-type-aware SQL casts, which could affect production query behavior. Most other changes are additive docs/UI tweaks and new integrations.

Overview
Adds two new sales integrations: Prospeo and Findymail are introduced end-to-end (new block configs, icons, docs pages, and integrations catalog entries), including their operation lists and API-key auth.

Fixes cancelled copilot stream persistence/races: POST /api/copilot/chat/stop and the chat lifecycle now use a new transactional finalizeAssistantTurn (row-level lock + optional “active-or-cleared” policy) and withStoppedContentBlock to reliably append partial assistant output and republish completed when appropriate; tests updated accordingly.

Refines table + workspace UI behavior: table row filtering now passes schema columns into buildFilterClause for type-aware casts (notably dates), resource tables get sticky headers and tuned column width multipliers (plus folder size rollups and a standardized EMPTY_CELL_PLACEHOLDER), and toast notifications are capped/stacked with offset-aware animations.

Operational cleanup: scheduled and async workflow execution now always clears the per-execution base64 cache in finally, and copilot tool-schema caching is replaced with a bounded LRUCache registered for monitoring.

Docs updates: File tool docs switch to file_v4 across locales, expand the EN tool surface (file_fetch/file_read output shapes), and CloudWatch docs add mute/unmute alarm operations; Google Docs block adds an opt-in Markdown formatting toggle for create.

Reviewed by Cursor Bugbot for commit 6827be7. Configure here.

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 6827be7. Configure here.

}
for (const folder of folders) getTotal(folder.id)
return totalSize
}, [files, folders])
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Folder size computation ignores files outside current view

Medium Severity

The folderSizeMap computation iterates over files to sum sizes per folder, but files likely represents only the files currently loaded/visible (possibly filtered or paginated), not all files in the workspace. This means folder sizes will be inaccurate — showing partial totals when files are filtered, searched, or paginated, and potentially showing EMPTY_CELL_PLACEHOLDER for folders that actually contain files not in the current view.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 6827be7. Configure here.

onDismiss={dismissToast}
/>
)
})}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stacked toasts overlap hiding content behind newer ones

Medium Severity

All toast items use [grid-area:1/1] causing them to stack on the same grid cell. The stackOffset applies a horizontal translateX shift of only 3px * depth, meaning multiple toasts are nearly completely overlapping. With MAX_VISIBLE set to 4, four toasts would pile on top of each other with just 3–9px horizontal offset — making older toast content unreadable and their dismiss buttons inaccessible.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 6827be7. Configure here.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 19, 2026

Greptile Summary

This release bundles several independent improvements: a race-condition fix in the mothership stream's cancelled-completion path, LRU-backed server-side caches to bound heap growth, type-aware SQL casts for date/number range filters, and three new integrations (Prospeo, Findymail, Google Docs Markdown creation).

  • Mothership cancel fix: finalizeAssistantTurn now runs inside a FOR UPDATE transaction and both the streaming completion callback and the /chat/stop endpoint use 'active-or-cleared' marker policy, eliminating the race that previously caused partial content to be lost on cancel.
  • SQL type casts: buildFilterClause and buildSortClause now accept column definitions to emit ::numeric or ::timestamptz casts, so date-range filters compare chronologically instead of lexicographically.
  • Prospeo & Findymail: Full B2B enrichment and search tool suites; API keys correctly use user-only visibility throughout.
  • Google Docs Markdown: Uses the Drive multipart upload endpoint to let Google natively convert Markdown, with the boundary stashed on the params object between headers() and body() calls.

Confidence Score: 4/5

Safe to merge; no data-loss or security regressions found.

The cancel-path race fix uses a sound FOR UPDATE transaction approach. The SQL type-cast refactor is well-tested. The open questions around the Google Docs markdown path (string body serialization and boundary param mutation) should be confirmed before that feature is exercised in production, but neither represents a guaranteed breakage today.

apps/sim/tools/google_docs/create.ts — confirm the tool execution framework handles a string return value from body() without JSON-stringifying it.

Important Files Changed

Filename Overview
apps/sim/lib/copilot/chat/terminal-state.ts Wraps finalizeAssistantTurn in a FOR UPDATE transaction with streamMarkerPolicy support; adds structured FinalizeAssistantTurnResult return type to surface outcome to callers.
apps/sim/tools/google_docs/create.ts Adds Markdown-to-Google-Doc upload via Drive multipart endpoint; uses a boundary string stashed via param mutation between headers() and body().
apps/sim/lib/table/sql.ts Adds jsonbCastForType helper centralising numeric/timestamptz cast logic; propagates column type map into buildFilterClause and buildSortClause for type-aware JSONB comparisons.
apps/sim/lib/environment/utils.ts Replaces hand-rolled promise cache with LRUCache (max 500, TTL 15s); retains .catch() eviction on failure; registers with monitoring registry.
apps/sim/lib/copilot/tools/server/table/user-table.ts Adds workspace ownership check to all table mutation and query operations, and adapts to new function signatures.
apps/sim/app/_styles/globals.css Adds --stack-offset CSS variable to toast keyframe animations; violates the custom rule discouraging globals.css edits, though keyframes are inherently global.
apps/sim/lib/core/async-jobs/backends/database.ts Semaphore cleanup: tracks limit on struct and deletes map entry when all slots are returned with no waiters, preventing unbounded growth.
apps/sim/lib/copilot/request/lifecycle/run.ts Cancelled-path now calls onComplete with accumulated content/blocks instead of returning empty, and guards against double-invocation with onCompleteStarted flag.

Sequence Diagram

sequenceDiagram
    participant Client
    participant StopRoute as /chat/stop
    participant PostCB as onComplete
    participant DB as Postgres

    Note over Client,DB: Cancel path - both paths run concurrently

    Client->>StopRoute: POST stop
    Client->>PostCB: abort signal fires

    par Stop endpoint
        StopRoute->>DB: SELECT FOR UPDATE WHERE chatId AND userId
        DB-->>StopRoute: row with conversationId and messages
        StopRoute->>StopRoute: ownsTurn check active-or-cleared policy
        StopRoute->>DB: UPDATE messages append assistantMsg and stopped block
        DB-->>StopRoute: updated
        StopRoute->>Client: publishStatusChanged completed
    and Completion callback
        PostCB->>DB: SELECT FOR UPDATE WHERE chatId
        DB-->>PostCB: row
        PostCB->>PostCB: alreadyHasResponse check
        alt not yet persisted
            PostCB->>DB: UPDATE messages append assistantMsg and stopped block
        else already persisted by stop
            PostCB->>PostCB: outcome AssistantAlreadyPersisted
        end
        PostCB->>Client: publishStatusChanged if warranted
    end
Loading

Comments Outside Diff (1)

  1. apps/sim/tools/google_docs/create.ts, line 144-146 (link)

    P2 body() returns a raw string — verify framework handles non-object bodies

    For the markdown upload path, body() returns a string rather than the JSON-serializable Record<string, unknown> every other tool returns. If the tool execution framework unconditionally calls JSON.stringify(body), the multipart string would be double-encoded and the Drive API would reject the request. Confirm the framework branches on typeof body === 'string' and passes it through raw.

Reviews (1): Last reviewed commit: "feat(google_docs): opt-in Markdown forma..." | Re-trigger Greptile

Comment on lines 878 to 898
@keyframes toast-enter {
from {
opacity: 0;
transform: translateY(8px) scale(0.97);
transform: translateX(var(--stack-offset, 0px)) translateY(8px) scale(0.97);
}
to {
opacity: 1;
transform: translateY(0) scale(1);
transform: translateX(var(--stack-offset, 0px)) translateY(0) scale(1);
}
}

@keyframes toast-exit {
from {
opacity: 1;
transform: translateY(0) scale(1);
transform: translateX(var(--stack-offset, 0px)) translateY(0) scale(1);
}
to {
opacity: 0;
transform: translateY(8px) scale(0.97);
transform: translateX(var(--stack-offset, 0px)) translateY(8px) scale(0.97);
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The keyframe animations in globals.css were updated to support the --stack-offset custom property for toast stacking. Per the project style rule, globals.css edits should be avoided and styles moved to local component files instead. CSS @keyframes are globally scoped and can't be co-located with a component module today, but the same visual effect is achievable without a globals.css change by applying the translateX directly on the element's style prop (which already sets --stack-offset) and using a Tailwind transition on transform. If the keyframe approach is kept, the impact here is minimal.

Suggested change
@keyframes toast-enter {
from {
opacity: 0;
transform: translateY(8px) scale(0.97);
transform: translateX(var(--stack-offset, 0px)) translateY(8px) scale(0.97);
}
to {
opacity: 1;
transform: translateY(0) scale(1);
transform: translateX(var(--stack-offset, 0px)) translateY(0) scale(1);
}
}
@keyframes toast-exit {
from {
opacity: 1;
transform: translateY(0) scale(1);
transform: translateX(var(--stack-offset, 0px)) translateY(0) scale(1);
}
to {
opacity: 0;
transform: translateY(8px) scale(0.97);
transform: translateX(var(--stack-offset, 0px)) translateY(8px) scale(0.97);
}
}
@keyframes toast-enter {
from {
opacity: 0;
transform: translateY(8px) scale(0.97);
}
to {
opacity: 1;
transform: translateY(0) scale(1);
}
}
@keyframes toast-exit {
from {
opacity: 1;
transform: translateY(0) scale(1);
}
to {
opacity: 0;
transform: translateY(8px) scale(0.97);
}
}

Rule Used: Avoid editing the globals.css file unless absolute... (source)

Learned From
simstudioai/sim#367

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines 107 to 120
@@ -69,18 +115,30 @@ export const createTool: ToolConfig<GoogleDocsToolParams, GoogleDocsCreateRespon
throw new Error('Title is required')
}

const requestBody: any = {
const folderId = params.folderSelector || params.folderId
const metadata: Record<string, unknown> = {
name: params.title,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Boundary stashed via param mutation between headers() and body()

headers() writes _boundary onto the params object so body() can read it, creating a hidden execution-order dependency. If the tool execution framework ever evaluates body() before headers(), _boundary will be undefined and the function throws — producing an HTTP 400 to Google with a mismatched Content-Type. The loud throw is a good guard, but the better fix is to generate the boundary once outside both functions or derive it deterministically from params.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants