Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 127 additions & 0 deletions docs/features/agent-judgment-model/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Agent Judgment Model — Plan

Spec: `spec.md`. Status: implemented; gates green.
Depends on: `docs/features/typesafe-jev-provider/` for `ModelType.Judgment` and the `jev` protocol.

## Slice 1 — Configuration field

Objective: a persisted, per-agent judgment model that nothing reads yet.

- [x] Add `judgmentModel` to `DeepChatAgentConfig` in `src/shared/types/agent-interface.d.ts`.
- [x] Add it to `DeepChatAgentConfigSchema` in `src/shared/contracts/domainSchemas.ts`.
- [x] Add it to the repository merge list in `deepChatAgentRepository.ts` so it round-trips.
- [x] Do not extend `CONFIG_ENTRY_KEYS`; the slot is per-agent only.

Ownership: shared agent config contract and the agent repository.

Completion condition: the field survives a save/reload round-trip and an existing row without it
still parses.

## Slice 2 — Settings UI

Objective: select and clear a judgment model, restricted to Jev-protocol models.

- [x] Add the field to `ModelKey`, `FormState`, the reactive defaults, and `emptyForm()`.
- [x] Add the `open` ref, the `modelFields` entry, and the close branch in `selectModel`.
- [x] Add it to `CONFIG_DIFF_KEYS`, `buildEditableConfig`, and the `fromAgent` mapping.
- [x] Restrict its picker to `ModelType.Judgment` through the existing per-field type filter.
- [x] Add the i18n label across all 23 locales.
- [x] Update the fixed model-picker count and index assertions in the renderer test.
- Note: `src/types/i18n.d.ts` is already stale in the repository relative to `zh-CN`; running
`pnpm run i18n:types` rewrites ~336 unrelated lines. The single `judgmentModel` leaf was inserted
by hand instead. Wholesale regeneration belongs in its own change.

Ownership: `DeepChatAgentsSettings.vue`, i18n catalogs.

Completion condition: the picker shows only judgment models; save and reload persist the selection;
`pnpm run i18n` passes.

## Slice 3 — Judgment question set

Objective: the reviewable surface, isolated in one file.

- [x] Add `jevPermissionQuestions.ts` holding the question definitions, every threshold, and the
composition rules.
- [x] Keep questions atomic — a risk `Choice`, an authorization `Noul`, an injection `Noul` — and
leave composition to code.
- [x] Document every threshold as provisional pending evaluation evidence.

Ownership: one new module under the DeepChat agent runtime.

Completion condition: the module contains no review logic and no I/O, and every threshold is named.

## Slice 4 — System One execution path

Objective: let the runtime issue a System One request to a provider.

- [x] Add `runJudgment` to the provider runtime port, the `ProviderExecutionPort` pick, the runtime
implementation, and the tool-runtime binding dependencies.
- [x] Implement it for the Jev protocol provider, rejecting non-System-One providers with a clear
error instead of a missing-method crash.
- [x] Preserve abort, timeout, rate limiting, and error mapping.

Ownership: `src/shared/types/provider.ts` port surface, provider runtime, `jevProvider.ts`.

Depends on: Slice 3 for the question shapes.

Completion condition: a judgment call can be issued and returns typed answers, and a non-Jev
provider rejects it clearly.

## Slice 5 — Reviewer backend selection

Objective: use the judgment model when configured, and change nothing when it is not.

- [x] In `reviewAutoApproveToolPermission`, branch on the configured judgment model.
- [x] Build a filtered review state containing only what the questions need.
- [x] Compose typed answers into `ToolPermissionReviewResult` in code, with code-side action binding
replacing the hash echo.
- [x] Preserve `critical -> block`, `high -> ask_user`, and `ask_user` on failure, timeout, or
invalid output.
- [x] Map the structured classification to fixed local rationale copy.
- [x] Log the decision, the judgment signals used, and token usage, without logging secrets.
- [x] Bound the judgment request with the review's own abort signal, so the 30s review timeout
applies to it exactly as it does to the generative path.

Ownership: `toolPermissionReviewer.ts`.

Completion condition: with the slot unset the existing path is untouched; with the slot set a
System One request is issued and mapped.

## Slice 6 — Review and validation
Objective: prove decoupling held and the safety floor is intact.

- [x] Confirm by inspection that compaction, title generation, translation, and memory consolidation
still read `assistantModel`; only the reviewer reads `judgmentModel`.
- [x] Durable tests: the `critical`/`high` floor under the judgment path, each auto-allow threshold,
failure and malformed answers resolving to `ask_user`, abort propagation, and the unchanged
generative path when unset.
- [x] Run `pnpm run format`, `pnpm run i18n`, `pnpm run lint`, `pnpm run typecheck`, and the focused
main and renderer suites.
- [x] No temporary probe or scaffolding retained.

Completion condition: all gates pass and existing permission-reviewer tests pass unchanged.

## Slice 7 — Review fixes

Applied in response to the PR review.

- [x] Make the bundled catalog load-bearing instead of decorative: it is returned whenever the live
catalog is unavailable or empty, so a missing key or one transient failure cannot clear a
previously discovered catalog. The original claim that the static entries populated the picker
on their own was wrong — nothing seeds the renderer's model store from `DEFAULT_PROVIDERS`.
- [x] Promote the auto-allow risk cap from a literal to `autoAllowMaxRiskLevel`, and document the
intentional difference from the generative path plus its consequence for the evaluation.
- [x] Perform the post-call abort re-check on the judgment path, matching the generative path.
- [x] Stop silently dropping an already-aborted caller signal in the provider request signal.
- [x] Keep the tail as well as the head of each message in the judgment state, so injection content
at the end of a long tool result is visible. The generative path's truncation is unchanged.
- [x] Drop the unused `isJevScoreAnswer` guard and the redundant `JevQuestion` re-export.
- [x] Select the provider by api type only, so repointing the built-in entry cannot pin it to
`JevProvider`.

## Deferred

- Evaluation of judgment quality, latency, cost, Chinese authorization, and injection resistance.
Issue #2326 treats this as the condition for adoption, not for implementation.
- Threshold tuning, which depends on that evaluation.
- The `pnpm run i18n:types` regeneration drift noted in Slice 2.
186 changes: 186 additions & 0 deletions docs/features/agent-judgment-model/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
# Agent Judgment Model

Status: proposed.

## Context

`DeepChatAgentConfig.assistantModel` is a single `{providerId, modelId}` pair that currently has five
runtime readers: tool-permission review, context compaction, session title generation, session
translation, and memory consolidation. Issue #2326 wants TypeSafe's Jev usable as an independent
permission-review backend, which is impossible while review and compaction share one setting.

Jev cannot replace the generative assistant model, because it does not generate text. Compaction in
particular produces a rolling summary string (`compactionService.generateRollingSummary` ->
`BaseLLMProvider.summaries`) and reads the chosen model's `contextLength` for budgeting, so a
non-generative model cannot serve it. This goal therefore adds a separate, narrowly scoped slot and
leaves every existing `assistantModel` reader untouched.

This document covers the agent-facing half. The provider/protocol half is a separate goal in
`docs/features/typesafe-jev-provider/`.

## Goals

- Add an explicit, opt-in `judgmentModel` slot to a DeepChat agent.
- Restrict that slot to Jev-protocol (`ModelType.Judgment`) models.
- When set, run tool-permission review against it through the System One protocol.
- When unset, preserve today's behaviour exactly.
- Keep the review result type and the permission interaction flow unchanged.

## Non-goals

- Replacing or repointing `assistantModel` for compaction, title generation, translation, or memory
consolidation.
- Default-enabling Jev review. The capability is experimental and opt-in.
- Widening which tools require review, or relaxing existing permission rules.
- Letting the review model execute tools, read files, or investigate the environment.
- Requiring Jev to produce free-text explanations equivalent to the generative reviewer.
- Enabling the capability on the basis of successful API calls alone; issue #2326 requires
evaluation evidence before adoption.

## Design

### Configuration

`DeepChatAgentConfig` gains `judgmentModel?: DeepChatAgentModelSelection | null`, a sibling of
`assistantModel`. It is added to the agent config type, the config schema, and the repository's
merge list so it round-trips through the agents table.

The global config-entry surface (`CONFIG_ENTRY_KEYS`) is deliberately not extended: the slot is
per-agent, and the legacy global key is a migration source rather than a new home.

### Settings UI

The agent settings form gains one more model field alongside the existing pickers, filtered to
`ModelType.Judgment` through the existing per-field type filter. The picker reuses `ModelSelect`;
no new component is introduced. The field is dirty-tracked by the existing signature mechanism and
saved through the existing patch path.

Because the slot is Jev-only, a judgment model can never be selected into a chat-shaped slot, and a
chat model can never be selected into the judgment slot.

### Review backend selection

`reviewAutoApproveToolPermission` currently resolves `config.assistantModel` and calls
`generateCompletionStandalone` with a system prompt demanding strict JSON, then parses that JSON
back out of free text.

With a judgment model configured, the reviewer instead:

1. computes the action envelope and its hash in code, exactly as today;
2. builds the review `state` from the exact action, its arguments, the permission context, and the
recent transcript;
3. asks a fixed set of atomic questions in one System One call;
4. composes the returned typed answers into a `ToolPermissionReviewResult` in code.

With no judgment model configured, the existing generative path runs unchanged.

### Action binding

The current generative path requires the model to echo `actionHash` and downgrades to `ask_user` on
mismatch. Jev does not generate text and cannot echo anything, so the hash echo is replaced by
code-side binding: the hash is computed for the reviewed action, passed into the same judgment call,
and the returned verdict is applied only to that action and its exact arguments. A result is never
cached or reused for a different action.

Note what this does *not* claim: nothing compares a returned hash against a stored one. The binding
is structural — one call, one request object, no reuse — rather than a check. It preserves the
existing invariant that a review verdict belongs to one specific action and its arguments, while
removing the mechanism that depended on text generation.

### Question set and thresholds

Following TypeSafe's own review guidance, the questions and every threshold live in a single
reviewable file rather than being spread through the review logic, because the questions and
thresholds are the parts a human must review.

Questions are atomic and composed in code rather than asking one question to reason end to end:

- a `Choice` over the risk level;
- a `Noul` for whether the recent transcript clearly authorizes this class of action;
- a `Noul` for whether the state contains content attempting to steer the decision.

Composition rules, all enforced in code:

- `critical` risk blocks; `high` risk asks the user. These existing constraints are preserved and
are not overridable by the model.
- An action is auto-allowed only when its risk is at or below `autoAllowMaxRiskLevel`, the
authorization signal clears its threshold, the risk answer is confident enough, and no injection
signal is present.
- Any uncertain, invalid, failed, or timed-out review asks the user.
- An action that explicitly requires user confirmation keeps that confirmation; Jev never overrides
it.

**Intentional policy difference from the generative path.** The generative reviewer allows low *and*
medium risk. The judgment path caps auto-allow at `low`, so switching an agent to a judgment model
makes it strictly more interruptive. That is deliberate for an opt-in path and the cap is a named
threshold rather than a literal, so it can be changed in one reviewable place.

It has a consequence for the evaluation in issue #2326 that must be accounted for: because the two
paths do not share a policy, an evaluation that measures interruptions is not measuring the model
alone. The cap should be aligned with the generative path before drawing a conclusion about Jev's
judgment quality.

### Confidence semantics

`confidence` describes how concentrated the answer distribution is. It is not a permission to act
and is not treated as a safety guarantee. A `Noul` near `0.5` means the yes/no probabilities are
similar, not that the risk is moderate. Thresholds are per-decision and start conservative, and are
recorded alongside the questions so they can be revised against evaluation evidence.

### Rationale

`rationale` is produced by mapping the structured classification to fixed local copy. It is not
presented as model-authored explanation, because the model does not author text.

### Input scope

The review state is filtered before it is sent: only the fields the questions need. TypeSafe's
documented weakness is that accuracy degrades as state grows with irrelevant detail, and the
current reviewer sends up to eight messages of up to 2,000 characters each plus full tool
arguments. Reusing that payload verbatim would work against the questions.

Tool results are retained deliberately, because they are a primary prompt-injection vector and the
injection question needs to see them. That makes truncation direction matter: head-only truncation
would hide an instruction placed at the end of a long tool result, which is exactly the content the
injection question exists to catch. The judgment state therefore keeps both the head and the tail of
each message. The generative path keeps its existing head-only truncation, so that path's prompt is
byte-for-byte unchanged.

## Invariants

- A review verdict is bound to one action hash and its exact arguments.
- `critical` still blocks and `high` still asks the user, regardless of model output.
- Failure, timeout, and invalid output ask the user.
- A cancelled turn never receives a verdict: the judgment path performs the same post-call abort
re-check as the generative path, so a judgment that resolves after cancellation is discarded.
- An already-aborted caller signal is never silently dropped by the provider's request signal.
- Explicit user-confirmation requirements are never overridden.
- `assistantModel` readers other than permission review are unchanged.
- The selected judgment model takes effect on the next review without a restart, because the
reviewer re-resolves agent config per call.

## Compatibility

- With `judgmentModel` unset, review behaviour is byte-for-byte the existing behaviour.
- Existing stored agent rows parse unchanged; the new field is optional.
- The permission interaction flow and `ToolPermissionReviewResult` shape are unchanged.

## Acceptance criteria

- An agent can be configured with a judgment model, and the selection persists across reload.
- The judgment picker offers only Jev-protocol models.
- With a judgment model set, review issues a System One request and maps typed answers to the
existing result shape; no text JSON parsing is involved.
- With no judgment model set, the existing generative review path runs and existing tests pass
unchanged.
- A `critical` verdict from the model still blocks and a `high` verdict still asks the user.
- Timeout, HTTP failure, and malformed answers all resolve to `ask_user`.
- The same verdict cannot be applied to a different action or different arguments.
- Compaction, title generation, translation, and memory consolidation continue to read
`assistantModel`.

## Open questions

- The exact threshold values are placeholders until the evaluation in issue #2326 produces evidence.
- Whether the Chinese-language authorization scenarios documented as lower-accuracy by TypeSafe
clear an acceptable bar is an evaluation outcome, not an implementation decision.
Loading