Skip to content

Commit 3e1e1fc

Browse files
authored
feat(jev): add TypeSafe evaluation models to Agent (#8103)
* feat(jev): add TypeSafe decision integration * fix(jev): validate batch answers against requested questions * improvement(jev): refine criteria helpers and suggested skills * fix(jev): sanitize generated JSON inputs * fix(jev): clarify empty Noul canvas sentence * fix(jev): preserve Agent tool criteria and rubric schema * fix(jev): expose text inputs to Agent models * feat(jev): integrate native evaluation into Agent providers * fix(jev): harden evaluation transport and output discovery * fix(jev): bound evaluation payloads before serialization * chore(jev): use the standard hoisted test mock * fix(jev): expose typed answer fields in reference pickers * feat(jev): support hosted TypeSafe keys and default to latest * improvement(agent): place credentials beside the model selector
1 parent c28366c commit 3e1e1fc

43 files changed

Lines changed: 1600 additions & 57 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎apps/docs/components/icons.tsx‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9936,3 +9936,16 @@ export function PitchBookIcon(props: SVGProps<SVGSVGElement>) {
99369936
</svg>
99379937
)
99389938
}
9939+
9940+
/** TypeSafe’s official mark from https://typesafe.ai. */
9941+
export function TypeSafeIcon(props: SVGProps<SVGSVGElement>) {
9942+
return (
9943+
<svg {...props} viewBox='0 0 21 32' fill='currentColor' xmlns='http://www.w3.org/2000/svg'>
9944+
<path
9945+
fillRule='evenodd'
9946+
clipRule='evenodd'
9947+
d='M10.36 0.071C10.55 -0.054 10.707 -0.008 10.757 0.166L15.57 2.792L15.33 3.23L15.748 2.953V8.81L20.542 11.425L20.224 12.008L20.742 11.663V23.613C20.742 23.889 20.556 24.238 20.326 24.391L10.383 31.02V30.402L10.12 30.885L5.323 28.269C5.124 28.365 4.972 28.262 4.972 28.012V22.153L0.398 19.659C0.176 19.797 0 19.697 0 19.428V7.478C0 7.201 0.186 6.853 0.416 6.7L10.36 0.071ZM6.27 27.645L10.49 29.947L19.434 23.985L15.212 21.684L6.27 27.645ZM10.775 12.743V18.218C10.775 18.494 10.589 18.842 10.359 18.995L5.804 22.032V26.957L14.915 20.882V9.983L10.775 12.743ZM15.747 20.827C15.747 20.83 15.746 20.832 15.746 20.835L19.91 23.107V12.219L15.747 9.948V20.827ZM1.34 19.033L5.401 21.249L9.435 18.56L5.373 16.345L1.34 19.033ZM0.832 7.423V18.373L4.972 15.613V10.139C4.972 9.863 5.158 9.515 5.388 9.361L9.944 6.324V1.349L0.832 7.423ZM5.804 15.44L9.943 17.697V12.913L5.804 10.655V15.44ZM6.271 9.771L10.335 11.988C10.343 11.982 10.351 11.975 10.359 11.97L14.374 9.291L10.313 7.077L6.271 9.771ZM10.776 6.189L14.916 8.447V3.573L10.776 1.316V6.189Z'
9948+
/>
9949+
</svg>
9950+
)
9951+
}

‎apps/docs/content/docs/platform/self-hosting/environment-variables.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ import { Callout } from 'fumadocs-ui/components/callout'
4747
| `MISTRAL_API_KEY` | Mistral |
4848
| `XAI_API_KEY_1` | xAI |
4949
| `KIMI_API_KEY_1` | Moonshot Kimi |
50+
| `TYPESAFE_API_KEY_1` / `_2` / `_3` | TypeSafe Jev hosted key rotation |
5051
| `ZAI_API_KEY_1` | Z.ai |
5152
| `TOGETHER_API_KEY` | Together AI |
5253
| `FIREWORKS_API_KEY` | Fireworks AI |

‎apps/docs/content/docs/workflows/blocks/agent.mdx‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,30 @@ For a custom cloud deployment, enter its provider prefix and model ID: `azure/my
3131

3232
Ollama Cloud, OpenRouter, Fireworks, Together AI, Baseten, Ollama, vLLM, and LiteLLM load their available models from the configured provider. New models appear through that discovery without a Sim catalog release. You can also enter a namespaced ID directly, such as `ollama-cloud/deepseek-v4.1-flash`, `openrouter/provider/model`, or `ollama/my-local-model`. Provider prefixes are case-insensitive; the model ID after the prefix keeps its original casing.
3333

34+
### Jev evaluation models
35+
36+
Select `jev-latest` from TypeSafe in the Agent model selector. Hosted Sim supplies a key and bills model usage through the normal credit system; workspace or organization BYOK keys override the hosted key without model charges. Self-hosted users enter their TypeSafe key in the block. Use `jev-1.13.0` to pin a version or `jev-preview` to follow preview releases. These models use **State** and **Questions** in place of conversational messages. State accepts text or a reference to a JSON object or array. Questions is a JSON object keyed by the answer names you want:
37+
38+
```json
39+
{
40+
"route": {
41+
"type": "choice",
42+
"instructions": "Which team should handle this request?",
43+
"criteria": { "billing": "Payments and invoices", "support": "Product issues" }
44+
},
45+
"urgency": {
46+
"type": "score",
47+
"instructions": "How urgent is the request?",
48+
"criteria": ["Routine", "Soon", "Immediate"]
49+
},
50+
"resolved": { "type": "noul", "instructions": "Has the request been resolved?" }
51+
}
52+
```
53+
54+
Read results from `<agent.answers>` or expand an answer in the reference picker, such as `<agent.answers.route.choice>`. Each Choice answer includes `choice`, `probabilities`, and `confidence`; each Score answer includes `score`, `legend`, `probabilities`, and `confidence`; each Noul answer includes `noul`, a probability from 0 to 1. `content` contains the same answers as JSON text, and the standard model, token, timing, and cost outputs remain available. Use a Condition block to route on these results.
55+
56+
Jev evaluates the supplied state in one request. Chat messages, files, tools, skills, conversation memory, response-format schemas, and chat model fallbacks are hidden for these models. Saved settings return when you switch back to a chat model. TypeSafe documents a 64,000-token total request limit and a 32,000-token limit for state plus the longest question. See [TypeSafe's model documentation](https://docs.typesafe.ai/models) and [question formats](https://docs.typesafe.ai/api).
57+
3458
### Files
3559

3660
Files for the model to read: images for a vision-capable model, or documents for text. Upload them on the block, or pass a file from an earlier block, such as an upload trigger or an [API](/workflows/blocks/api) response, with a connection tag.
@@ -104,7 +128,7 @@ Some settings live under advanced, or appear only for models that support them:
104128
- **Max output tokens.** Caps the response length. Defaults to the model's full limit.
105129
- **Reasoning effort / Thinking level.** For models with extended reasoning, how much the model thinks before answering. Higher is more thorough but slower and costs more tokens.
106130
- **Prompt caching.** For Anthropic Claude models, reuses the system prompt and tool definitions between runs instead of re-reading them every time. Cached input costs a tenth of the normal rate, but writing the cache costs 1.25x, so leave it off for one-off runs and turn it on when the same agent runs repeatedly. The cache covers a prefix only if it reaches 1,024 tokens (2,048 on Haiku) — below that Anthropic ignores it and nothing changes. Entries expire after five minutes of no use.
107-
- **API key.** Your key for the chosen provider. Hidden on hosted Sim, which supplies one.
131+
- **API key.** Your key for the chosen provider. Hidden when hosted Sim supplies a key for the selected model, including Jev.
108132
- **Fallback models.** An ordered list of up to five models to try when the request to the selected model fails, whether the provider is overloaded, rate-limited, or down. Sim tries the 2nd choice, then the 3rd, and so on, once each, and `<agent.model>` reports the model that answered. On hosted Sim, hosted models use your workspace's BYOK or platform credentials; local and self-hosted installations may still require a key. A model that needs its own key takes it from a workspace environment variable you pick on the row; a model on the same provider as the selected model reuses the block's key. A stored row key stops applying when its key field is hidden. Providers that require family-specific credentials, such as Vertex, can only be fallbacks for a selected model of the same family. The Auto model cannot be a fallback. A fallback runs with the selected model's settings where its provider accepts them: temperature and max output tokens are clamped to the fallback's limits, and when the fallback has a reasoning effort, thinking level, or verbosity setting that the selected model's value does not fit, the row shows that field so you can pick a value for it; leave it empty and the provider's default applies.
109133
- **Retry on fail.** Retries the selected model after a failure, up to a maximum number of tries with a wait between them. When its tries run out, the fallback models are tried in order, once each, with no wait before the first of them. A fallback is never retried. See [Retries and fallbacks](#retries-and-fallbacks) for how recorded tool results are reused and when a tool can execute again.
110134

‎apps/sim/app/workspace/[workspaceId]/settings/components/byok/byok.tsx‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ import {
3939
SerperIcon,
4040
TinyFishIcon,
4141
TogetherIcon,
42+
TypeSafeIcon,
4243
WizaIcon,
4344
xAIIcon,
4445
ZaiIcon,
@@ -130,6 +131,13 @@ const PROVIDERS: (BYOKManagerProvider & { id: BYOKProviderId })[] = [
130131
description: 'LLM calls',
131132
placeholder: 'sk-...',
132133
},
134+
{
135+
id: 'typesafe',
136+
name: 'TypeSafe',
137+
icon: TypeSafeIcon,
138+
description: 'Jev evaluation models',
139+
placeholder: 'Enter your TypeSafe API key',
140+
},
133141
{
134142
id: 'fireworks',
135143
name: 'Fireworks',
@@ -352,6 +360,7 @@ const PROVIDER_SECTIONS: BYOKProviderSection[] = [
352360
'cohere',
353361
'xai',
354362
'kimi',
363+
'typesafe',
355364
'fireworks',
356365
'together',
357366
'baseten',
Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
/** @vitest-environment node */
2+
import { resetEnvFlagsMock, setEnvFlags } from '@sim/testing'
3+
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
4+
import {
5+
getEffectiveBlockOutputPaths,
6+
getEffectiveBlockOutputs,
7+
getEffectiveBlockOutputType,
8+
} from '@/lib/workflows/blocks/block-outputs'
9+
import { getBlockReferenceTags } from '@/lib/workflows/blocks/block-reference-tags'
10+
import { evaluateSubBlockCondition } from '@/lib/workflows/subblocks/visibility'
11+
import { AgentBlock } from '@/blocks/blocks/agent'
12+
import { getAgentModelOptions, getModelOptions } from '@/blocks/utils'
13+
import { getBaseModelProviders } from '@/providers/models'
14+
import { Serializer } from '@/serializer'
15+
import { useProvidersStore } from '@/stores/providers/store'
16+
import type { BlockState } from '@/stores/workflows/workflow/types'
17+
18+
const { mockGetBlock } = vi.hoisted(() => ({ mockGetBlock: vi.fn() }))
19+
20+
vi.mock('@/blocks', () => ({ getBlock: mockGetBlock }))
21+
22+
describe('Agent evaluation configuration', () => {
23+
afterEach(resetEnvFlagsMock)
24+
beforeEach(() => {
25+
mockGetBlock.mockReturnValue(AgentBlock)
26+
})
27+
28+
it.each(['jev-1.13.0', 'jev-latest', 'jev-preview'])(
29+
'shows native fields and credentials for %s',
30+
(model) => {
31+
const visible = AgentBlock.subBlocks
32+
.filter((field) => evaluateSubBlockCondition(field.condition, { model }))
33+
.map((field) => field.id)
34+
expect(visible).toEqual(['model', 'apiKey', 'evaluationState', 'evaluationQuestions'])
35+
}
36+
)
37+
38+
it('keeps evaluation inputs configurable for a model reference', () => {
39+
for (const field of AgentBlock.subBlocks.filter((field) => field.id.startsWith('evaluation'))) {
40+
expect(evaluateSubBlockCondition(field.condition, { model: '<start.model>' })).toBe(true)
41+
}
42+
})
43+
44+
it.each([false, true])('shows TypeSafe credentials only when needed, hosted=%s', (hosted) => {
45+
setEnvFlags({ isHosted: hosted })
46+
const apiKey = AgentBlock.subBlocks.find((field) => field.id === 'apiKey')!
47+
expect(evaluateSubBlockCondition(apiKey.condition, { model: 'jev-latest' })).toBe(!hosted)
48+
})
49+
50+
it.each(['jev-1.13.0', '<start.model>', '{{MODEL_ID}}'])(
51+
'exposes answers for %s in downstream selectors',
52+
(model) => {
53+
const values = { model: { value: model } }
54+
expect(getEffectiveBlockOutputs('agent', values)).toHaveProperty('answers')
55+
expect(getEffectiveBlockOutputPaths('agent', values)).toContain('answers')
56+
expect(getEffectiveBlockOutputType('agent', 'answers', values)).toBe('json')
57+
}
58+
)
59+
60+
it('does not expose evaluation answers for a known chat model', () => {
61+
expect(getEffectiveBlockOutputs('agent', { model: { value: 'gpt-4o' } })).not.toHaveProperty(
62+
'answers'
63+
)
64+
})
65+
66+
it.each(['jev-1.13.0', '<start.model>'])(
67+
'keeps answers accessible with a saved chat schema for %s',
68+
(model) => {
69+
const outputs = getEffectiveBlockOutputs('agent', {
70+
model: { value: model },
71+
responseFormat: {
72+
value: { schema: { type: 'object', properties: { title: { type: 'string' } } } },
73+
},
74+
})
75+
expect(outputs).toHaveProperty('answers')
76+
if (model === 'jev-1.13.0') expect(outputs).not.toHaveProperty('title')
77+
else expect(outputs).toHaveProperty('title')
78+
}
79+
)
80+
81+
it('shows Jev only in the model picker that supports evaluation inputs', () => {
82+
useProvidersStore.getState().setProviderModels('base', Object.keys(getBaseModelProviders()))
83+
expect(getAgentModelOptions().map((option) => option.id)).toContain('jev-1.13.0')
84+
expect(getModelOptions().map((option) => option.id)).not.toContain('jev-1.13.0')
85+
})
86+
87+
describe('evaluation answer references', () => {
88+
const questions = {
89+
category: { type: 'choice', instructions: 'Choose a category', criteria: { a: 'A', b: 'B' } },
90+
rating: { type: 'score', instructions: 'Rate the result', criteria: ['Low', 'High'] },
91+
passed: { type: 'noul', instructions: 'Did it pass?' },
92+
}
93+
94+
it.each(['jev-1.13.0', 'jev-latest', 'jev-preview', '<start.model>', '{{MODEL_ID}}'])(
95+
'exposes typed question fields for %s without an execution result',
96+
(model) => {
97+
const values = {
98+
model: { value: model },
99+
evaluationQuestions: { value: JSON.stringify(questions) },
100+
responseFormat: {
101+
value: { schema: { type: 'object', properties: { title: { type: 'string' } } } },
102+
},
103+
}
104+
const tags = getBlockReferenceTags({
105+
block: { id: 'agent-test', type: 'agent', name: 'Evaluate', subBlocks: values },
106+
})
107+
const fields = {
108+
'answers.category.choice': 'string',
109+
'answers.category.confidence': 'number',
110+
'answers.category.probabilities': 'json',
111+
'answers.category.type': 'string',
112+
'answers.rating.score': 'number',
113+
'answers.rating.confidence': 'number',
114+
'answers.rating.legend': 'json',
115+
'answers.passed.noul': 'number',
116+
}
117+
for (const [path, type] of Object.entries(fields)) {
118+
expect(tags).toContain(`evaluate.${path}`)
119+
expect(getEffectiveBlockOutputType('agent', path, values)).toBe(type)
120+
}
121+
expect(getEffectiveBlockOutputType('agent', 'answers', values)).toBe('json')
122+
expect(getEffectiveBlockOutputType('agent', 'answers.category', values)).toBe('json')
123+
expect(tags).not.toContain('evaluate.answers.passed.confidence')
124+
expect(tags.includes('evaluate.title')).toBe(!model.startsWith('jev-'))
125+
}
126+
)
127+
128+
it.each([undefined, '', '{', '<start.questions>', '{{QUESTIONS}}', [], null, { unknown: {} }])(
129+
'keeps the answers object selectable when questions cannot be inferred: %j',
130+
(value) => {
131+
const values = { model: { value: 'jev-latest' }, evaluationQuestions: { value } }
132+
expect(getEffectiveBlockOutputPaths('agent', values)).toContain('answers')
133+
expect(getEffectiveBlockOutputType('agent', 'answers', values)).toBe('json')
134+
}
135+
)
136+
137+
it('uses structured questions and follows edits without leaking fields into chat models', () => {
138+
const values = {
139+
model: { value: 'jev-latest' },
140+
evaluationQuestions: { value: { result: questions.category } },
141+
}
142+
expect(getEffectiveBlockOutputPaths('agent', values)).toContain('answers.result.choice')
143+
expect(
144+
getEffectiveBlockOutputPaths('agent', {
145+
...values,
146+
evaluationQuestions: { value: { result: questions.passed } },
147+
})
148+
).not.toContain('answers.result.choice')
149+
expect(
150+
getEffectiveBlockOutputPaths('agent', {
151+
...values,
152+
model: { value: 'gpt-4o' },
153+
}).some((path) => path.startsWith('answers'))
154+
).toBe(false)
155+
})
156+
157+
it('does not offer ambiguous reference paths for special question IDs', () => {
158+
const values = {
159+
model: { value: 'jev-latest' },
160+
evaluationQuestions: {
161+
value: {
162+
'with.dot': questions.passed,
163+
'with space': questions.passed,
164+
'with[0]': questions.passed,
165+
'<start.question>': questions.passed,
166+
'valid-id_1': questions.passed,
167+
},
168+
},
169+
}
170+
const paths = getEffectiveBlockOutputPaths('agent', values)
171+
expect(paths.filter((path) => path.startsWith('answers.'))).toEqual([
172+
'answers.valid-id_1.noul',
173+
'answers.valid-id_1.type',
174+
])
175+
expect(getEffectiveBlockOutputType('agent', 'answers', values)).toBe('json')
176+
})
177+
178+
it.each(['type', 'properties', 'description', '__proto__'])(
179+
'resolves the question named %s through schema properties',
180+
(id) => {
181+
const values = {
182+
model: { value: 'jev-latest' },
183+
evaluationQuestions: { value: { [id]: questions.passed } },
184+
}
185+
expect(getEffectiveBlockOutputPaths('agent', values)).toContain(`answers.${id}.noul`)
186+
expect(getEffectiveBlockOutputType('agent', `answers.${id}.noul`, values)).toBe('number')
187+
}
188+
)
189+
})
190+
191+
it.each([false, true])(
192+
'serializes native fields without requiring messages, advanced=%s',
193+
(advancedMode) => {
194+
const values = {
195+
model: 'jev-1.13.0',
196+
apiKey: '{{TYPESAFE_API_KEY}}',
197+
evaluationState: '42',
198+
evaluationQuestions: '{"passed":{"type":"noul","instructions":"Did it pass?"}}',
199+
messages: JSON.stringify([{ role: 'user', content: 'Old chat prompt' }]),
200+
}
201+
const block: BlockState = {
202+
id: 'agent-test',
203+
type: 'agent',
204+
name: 'Evaluator',
205+
position: { x: 0, y: 0 },
206+
enabled: true,
207+
advancedMode,
208+
outputs: {},
209+
subBlocks: Object.fromEntries(
210+
Object.entries(values).map(([id, value]) => [
211+
id,
212+
{ id, value, type: AgentBlock.subBlocks.find((field) => field.id === id)!.type },
213+
])
214+
),
215+
}
216+
const result = new Serializer().serializeWorkflow({ [block.id]: block }, [], {}, {}, true)
217+
expect(result.blocks[0].config.tool).toBe('typesafe')
218+
expect(result.blocks[0].config.params).toMatchObject({
219+
evaluationState: '42',
220+
evaluationQuestions: values.evaluationQuestions,
221+
})
222+
expect(result.blocks[0].config.params).not.toHaveProperty('messages')
223+
}
224+
)
225+
})

0 commit comments

Comments
 (0)