Skip to content

provider: add clf-ai-gateway (OpenAI-compatible, 9 models) - #6046

Open
bap1106 wants to merge 6 commits into
anomalyco:devfrom
bap1106:add-clf-ai-gateway
Open

provider: add clf-ai-gateway (OpenAI-compatible, 9 models)#6046
bap1106 wants to merge 6 commits into
anomalyco:devfrom
bap1106:add-clf-ai-gateway

Conversation

@bap1106

@bap1106 bap1106 commented Sep 1, 2026

Copy link
Copy Markdown

Add provider: CLF AI Gateway

OpenAI-compatible gateway at https://api.clfaigateway.dev/v1, reselling
Cloudflare Workers AI open-weight models (GLM, Kimi, DeepSeek, Qwen) on prepaid
credits. We are transparent about running on Workers AI upstream — each model's
upstream @cf/... id is exposed via GET /v1/public/models (no auth required),
which is also the live source for pricing and capabilities.

What's included

  • provider.toml — name, @ai-sdk/openai-compatible, env CLF_AI_GATEWAY_API_KEY, base URL, docs link.
  • logo.svg — currentColor, no hard-coded colors.
  • models/ — 9 models. All numbers are measured against the live endpoint,
    not copied from upstream marketing pages:
    • limit.context verified per model (the 1M models were probed to the
      1,048,576 boundary; we deliberately do NOT repeat the 1,310,720 figure that
      upstream catalog reports — see issue cloudflare-workers-ai: glm-5.3-flash limit 1310720 exceeds gateway max 1048576 (same as #4422 for glm-5.2) #5824 for that discrepancy).
    • limit.output = 131,072, verified by sending max_completion_tokens probes.
    • attachment only on the 4 models that actually accept images (probed);
      the gateway returns a named 400 for image input on the other five.
    • cost = current promotional pricing (40% off list), which is what the API
      actually bills today. We will PR updates when pricing changes; live prices
      are always at https://clfaigateway.dev/models and GET /v1/public/models.

Verification for reviewers

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [critical] [violation] providers/clf-ai-gateway/models/deepseek-v4-flash.toml:3 - Check: Non-lab hosts must use base_model (AGENTS.md blocker). Why: CLF AI Gateway is a multi-model relay of lab models (DeepSeek, Z.ai/GLM, Moonshot Kimi, Alibaba Qwen), not the model creator. All nine entries are full standalone TOMLs with no base_model, which also restates lab facts that already exist under models/. Action: Point each file at the lab entry and keep only host-specific fields/deltas (cost, reasoning_options, real limit/modality overrides, etc.): deepseek/deepseek-v4-flash, deepseek/deepseek-v4-pro, zhipuai/glm-4.7-flash, zhipuai/glm-5.2, zhipuai/glm-5.3-flash, zhipuai/glm-5.3, moonshotai/kimi-k2.6, moonshotai/kimi-k2.7-code, alibaba/qwen3.8-27b. Strip duplicated identical name/description/family/capabilities/[modalities]/[limit] unless they truly differ on this host.
  • [high] [violation] providers/clf-ai-gateway/models/deepseek-v4-flash.toml:14 - Check: Relay reasoning_options must follow lab + same-surface peers; [] means no caller control, not uncertainty. Why: Every model sets reasoning = true with reasoning_options = [], but the stated upstream (Cloudflare Workers AI) and existing CF peers expose controls for almost all of these IDs (e.g. DeepSeek V4: toggle + high/max; GLM-4.7-Flash/5.2/5.3 and Kimi K2.6/K2.7 Code: toggle + low/medium/high; Qwen3.8-27B: toggle + low/medium/xhigh). Empty options hide those controls to clients. Action: Author per-model options matching the Workers AI / established peer surface for this OpenAI-compatible path (with leading wire-path comments for any toggle), or document affirmative proof this gateway strips those fields. Only keep [] where peers truly have no control (CF’s glm-5.3-flash is the nearby empty peer).

@bap1106

bap1106 commented Sep 1, 2026

Copy link
Copy Markdown
Author

Both review items addressed in e57d223:

  • All 9 models now reference their lab entries via base_model (zhipuai/*, deepseek/*, moonshotai/*, alibaba/*) and carry only this surface's deltas (cost, limits, modalities, reasoning options).
  • reasoning_options now lists the real per-model reasoning_effort sets this gateway enforces server-side — an invalid value returns 400 listing the valid set, and the sets are exposed publicly in GET /v1/public/models as capabilities.reasoning_efforts. They are measured on this surface, not copied from peers: e.g. glm-5.3 accepts none/low/medium/high/max (and rejects xhigh), while glm-5.3-flash accepts low/medium/high/xhigh. Wire-path comments added; native template toggles are not forwarded by this gateway.
  • [modalities] kept only where they differ from the lab (text+image — this OpenAI-compatible surface forwards image_url parts only, no video/pdf). [limit] values are probed on this surface, including qwen3.8-27b output = 131,072 verified via max_completion_tokens probes (the lab file lists 32,768).

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [high] [possible mistake] providers/clf-ai-gateway/models/deepseek-v4-flash.toml:8 - Check: Relay reasoning_options must follow lab + same-surface peer baseline for the model, not a generic GPT ladder (AGENTS.md → Reasoning options; DeepSeek V4 is not L/M/H). Why: This file (and deepseek-v4-pro.toml) sets effort = ["low", "medium", "high", "xhigh"]. First-party DeepSeek is toggle + low|high|max (Flash) / high|max (Pro); Cloudflare Workers AI (claimed upstream) is toggle + high|max. Inventing L/M/H/xhigh risks wrong client controls and misdocuments the host. Action: Align with the values this gateway actually accepts and that map meaningfully (prefer lab/CF high/max, plus Flash low if real). If L/M/H/xhigh are truly enforced, cite the exact capabilities.reasoning_efforts (or equivalent) per model and note alias→native mapping; do not keep an unverified GPT-style ladder.
  • [high] [possible mistake] providers/clf-ai-gateway/models/qwen3.8-27b.toml:14 - Check: Costs are USD/MTok and should match the PR’s own pricing story (promo = fraction of list / live /v1/public/models). Why: cache_read = 0.27 equals input = 0.27. Cloudflare Workers AI list is input = 0.45, cache_read = 0.05; 40% off implies ~0.27 / 0.03, not cache_read = full input. Action: Correct cache_read to the billed USD/MTok (or justify why cache is priced at input rate) and keep it consistent with the cited public models payload.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/glm-5.3.toml:9 - Check: Effort must not invent an off switch the model/host does not support. Why: Values include none, but first-party Z.ai documents GLM-5.3 as always-on thinking with low|high|max only (no disable). Action: Drop none unless this gateway really accepts it and turns thinking off; if off exists via another wire field, document that path (toggle + wire comment) instead of inventing none.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/glm-5.2.toml:9 - Check: Relay effort set = host-real levels, grounded in lab/peer baseline (Z.ai effective high|max; CF low|medium|high). Why: ["low", "medium", "high", "xhigh"] (and the same shape on glm-5.3-flash.toml) mixes a broader ladder than lab effective levels and adds xhigh beyond the CF peer set without in-repo proof of distinct behavior. Action: Set each GLM file to the gateway’s actual accepted efforts; if values are aliases (e.g. L/M→high, xhigh→max), author the effective set (or document aliases in a leading comment) rather than an unverified superset.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/kimi-k2.6.toml:9 - Check: When lab/upstream expose on/off and this host only forwards reasoning_effort, off must still be represented if the API allows it. Why: Moonshot lab is toggle-only; CF is toggle + low|medium|high. These files (and several siblings) use graded effort with neither toggle nor none, so clients cannot express “reasoning off” even though peers can. Action: For each model, either add none (if omitting/none disables thinking), add toggle with a leading wire comment if a separate on/off field is forwarded, or keep effort-only / [] only when the host truly has no off control—and state that in the leading comment.

OpenAI-compatible gateway on Cloudflare Workers AI upstream. Models reference lab entries via base_model; files carry only this surface's deltas: measured limits, promo pricing (matches GET /v1/public/models, no auth), per-model reasoning_effort sets enforced server-side, and text+image modalities where the lab lists more. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bap1106

bap1106 commented Sep 1, 2026

Copy link
Copy Markdown
Author

All five items addressed in 4627786 — by adding per-file evidence rather than changing values, because these values are the enforced, measured surface of this gateway (not a template ladder):

  • Effort sets (deepseek, glm-5.2, kimi): enforced server-side per model — an out-of-set value returns 400 naming the exact set (live example: sending max to deepseek-v4-flashdoes not accept reasoning_effort 'max'. Supported: low, medium, high, xhigh). The same sets are public in GET /v1/public/modelscapabilities.reasoning_efforts. Values are forwarded verbatim upstream, no alias mapping; leading comments now state this per file.
  • Thinking toggle: no on/off field exists on this surface — only reasoning_effort is forwarded; omitting it applies the gateway default. Stated in each file's leading comment.
  • glm-5.3 none: accepted and validated by the API (part of the public set); comment now carries the measured caveat (2026-08-28) that upstream may still emit some reasoning at none — minimize, not hard-off.
  • qwen3.8-27b cache_read = input: intentional. This model has no verified cached tier on this surface, so cached tokens bill at the input rate; the public endpoint shows cached_input == input for both promo and list (270/270 now, 450/450 list). We don't advertise a cache discount we can't prove. Comment added.

@bap1106 bap1106 closed this Sep 2, 2026
@bap1106 bap1106 reopened this Sep 2, 2026
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [possible mistake] providers/clf-ai-gateway/models/deepseek-v4-flash.toml:60 - Check: Relay reasoning_options must match this host’s real controls, using lab + same-surface peers as baseline (not invented GPT-style L/M/H). Why: First-party DeepSeek and Cloudflare Workers AI (the claimed upstream) use toggle + high/max (Flash also allows low), not effort-only low/medium/high/xhigh. Listing a broader set without toggle risks wrong client controls if the gateway only remaps or partially accepts those strings. Action: Confirm against GET /v1/public/models and live requests that this exact set is enforced and that no on/off control exists; if the surface is CF-faithful, align with toggle + high/max (and Flash low if applicable) and add the required leading wire comment for any toggle.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/deepseek-v4-pro.toml:90 - Check: Same DeepSeek V4 baseline as above. Why: Lab/CF peers are toggle + high/max; this file uses effort-only low/medium/high/xhigh, which is a different control surface than the documented upstream. Action: Verify the public capability list and live 400 behavior for Pro specifically; correct reasoning_options (and toggle wire comment if needed) to the host-true set.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/glm-5.3.toml:219 - Check: none in effort means a real off/minimize control that callers can rely on; baseline for GLM-5.3 is lab/OpenRouter low/high/max (always-on). Why: The file includes none and medium, and its own comment says upstream may still emit reasoning at none with no hard off switch. Publishing none then misleads clients that expect reasoning off. Action: Drop none unless it truly disables (or clearly minimize-maps) on this API; keep only levels that match measured host behavior and lab/peer baseline (low/high/max, plus any extra only with proven effect).
  • [medium] [possible mistake] providers/clf-ai-gateway/models/glm-5.3-flash.toml:181 - Check: Effort set vs lab/peer baseline for GLM-5.3-Flash. Why: Lab/OpenRouter use low/high/max; this file uses low/medium/high/xhigh, adding medium/xhigh and dropping max without a clear same-surface peer match. Action: Verify capabilities.reasoning_efforts and live acceptance for this model ID; align values to the enforced set (prefer lab low/high/max unless the gateway truly differs).
  • [low] [possible mistake] providers/clf-ai-gateway/models/glm-5.2.toml:151 - Check: Extra effort levels beyond lab/same-surface peers need host-proven effect. Why: Zhipu lab is effectively high/max; Cloudflare Workers AI peers use low/medium/high. This adds xhigh on top of L/M/H. Action: Confirm xhigh is in the public set and has a distinct effect on this model; otherwise use the CF-peer low/medium/high set (or lab high/max if that is what the gateway enforces).

@bap1106

bap1106 commented Sep 2, 2026

Copy link
Copy Markdown
Author

Re the remaining [low] note on glm-5.2 xhigh: it is in the public enforced set — GET /v1/public/models lists ["low","medium","high","xhigh"] for glm-5.2 (and the same shape for glm-5.3-flash / both DeepSeek V4), while sending it to a model outside its set returns a 400 naming that model's exact set. These are live-acceptance values on this surface, not a copied ladder.

The API still accepts 'none' (part of the public accept-set), but measured 2026-08-28 it does not reliably disable thinking upstream - so we stop advertising an off switch that does not switch off. File comment explains. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bap1106

bap1106 commented Sep 2, 2026

Copy link
Copy Markdown
Author

Addressed the glm-5.3 item in 1ab2a9e: dropped none from the advertised set. Fair catch — the API does accept it (it's in the public accept-set), but our own measurement says it doesn't reliably disable thinking upstream, so we won't advertise an off switch that doesn't switch off. The file comment now states exactly that.

The four remaining "confirm" items are confirmed, per the two comments above: each listed set is the live GET /v1/public/modelscapabilities.reasoning_efforts for that model id, and sending any out-of-set value returns a 400 naming that model's exact set verbatim (e.g. deepseek-v4-flash: Supported: low, medium, high, xhigh). These are host-enforced acceptance values on this surface — where they differ from lab/peer files, the difference is measured, not invented.

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [high] [possible mistake] providers/clf-ai-gateway/models/deepseek-v4-flash.toml:11 - Check: Relay reasoning_options must follow lab + same-surface peers; do not invent a GPT-style L/M/H set for DeepSeek V4. Why: Lab DeepSeek and Cloudflare Workers AI (this gateway’s stated upstream) use toggle + high/max (Flash also allows low); this file advertises effort-only low/medium/high/xhigh and omits max. That conflicts with the documented DeepSeek V4 baseline and the anti-pattern of forcing L/M/H onto V4. A public URL claim alone does not establish meaningful host behavior. Action: Align with lab/CF (toggle + high/max, plus Flash low if this host exposes it), or keep a host-specific set only with concrete evidence that this API rejects max, accepts these levels as distinct non-alias efforts, and has no thinking on/off control—cite the exact accept-set from GET /v1/public/models / live 400 text in the PR body mapped to each claim.
  • [high] [possible mistake] providers/clf-ai-gateway/models/deepseek-v4-pro.toml:11 - Check: Same DeepSeek V4 baseline for relays. Why: Lab/CF Pro is toggle + high/max; this entry uses effort-only low/medium/high/xhigh, inventing low/medium and dropping max/toggle. Action: Match lab/CF Pro controls, or justify each host-only level and the missing toggle/max with host-specific wire evidence (not only “measured” assertions).
  • [high] [possible mistake] providers/clf-ai-gateway/models/glm-5.3-flash.toml:11 - Check: Effort set must match lab/peers unless this host’s real control surface differs with evidence. Why: Z.ai / OpenRouter / Vercel use low/high/max (always-on); Cloudflare Workers AI documents reasoning_options = [] (no caller control). This file uses low/medium/high/xhigh, adding medium/xhigh and dropping max, against both the lab baseline and the claimed upstream. Action: Prefer lab low/high/max, or CF [] if this surface truly exposes no control; if keeping the advertised set, prove this host accepts those values as real graded efforts (and not aliases) via the public models payload / live API.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/glm-5.2.toml:11 - Check: GLM-5.2 reasoning baseline (lab vs this host). Why: Lab Z.ai is effectively high/max; CF peers use toggle + low/medium/high. This file uses effort-only low/medium/high/xhigh, which matches neither without host proof that xhigh is a first-class level and that there is no thinking toggle. Action: Align to lab or CF same-surface controls, or document/verify this gateway’s exact accept-set and whether off is unsupported.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/glm-5.3.toml:12 - Check: GLM-5.3 effort list vs lab/peers. Why: Lab and most relays use low/high/max with no toggle; this adds medium relative to that baseline (while dropping the earlier none claim). Extra levels need host proof of distinct effect, not only acceptance. Action: Use low/high/max unless GET /v1/public/models (or equivalent) shows medium as a real distinct effort on this host; note that in the PR body.

… effect

Measured on the live surface 2026-09-02: same prompt, 3 samples per level, median reasoning_tokens. Levels the API accepts but that do not change behavior are documented as aliases in each file and no longer advertised; models with no measurable graded control get reasoning_options = []. Evidence table in the PR body. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@bap1106

bap1106 commented Sep 2, 2026

Copy link
Copy Markdown
Author

Add provider: CLF AI Gateway

OpenAI-compatible gateway at https://api.clfaigateway.dev/v1, reselling
Cloudflare Workers AI open-weight models (GLM, Kimi, DeepSeek, Qwen) on prepaid
credits. We are transparent about running on Workers AI upstream — each model's
upstream @cf/... id is exposed via GET /v1/public/models (no auth required),
which is also the live source for pricing and capabilities.

What's included

  • provider.toml — name, @ai-sdk/openai-compatible, env CLF_AI_GATEWAY_API_KEY, base URL, docs link.
  • logo.svg — currentColor, no hard-coded colors.
  • models/ — 9 models referencing their lab entries via base_model (DeepSeek → the dated
    snapshots matching the upstream ids, deepseek-v4-flash-0731 / deepseek-v4-pro-0813),
    carrying only this surface's deltas. All numbers are measured against the live endpoint:
    • limit.context probed per model (the 1M models to the 1,048,576 boundary — we
      deliberately do NOT repeat the 1,310,720 figure from the upstream catalog, cf. issue cloudflare-workers-ai: glm-5.3-flash limit 1310720 exceeds gateway max 1048576 (same as #4422 for glm-5.2) #5824).
    • limit.output = 131,072, verified with max_completion_tokens probes.
    • [modalities] only where they differ from the lab: this surface forwards image_url
      parts only (no video/pdf); attachment inherited from the lab where it matches.
    • cost = currently billed (promotional, 40% off list) prices; live at
      GET /v1/public/models (nano-USD per token, ÷1000 = USD/MTok).

Reasoning options — measured effect, not accept-set

Wire: only reasoning_effort is forwarded, verbatim; no on/off toggle field exists on
this surface
, so there is no toggle option. The host accepts a fixed set per model
(out-of-set → 400 naming the set; public as capabilities.reasoning_efforts), but only
some accepted levels change behavior. reasoning_options.values lists only the levels
with a measured distinct effect
— same prompt, 3 samples per level, median
reasoning_tokens (2026-09-02); models whose accepted levels are inert get [].
Accepted-but-inert levels are documented in each file as aliases.

model host accept-set (exact 400 text) measured medians (reasoning_tokens) values
glm-5.3 Supported: none, low, medium, high, max low 26 · medium 330 · high 32 · max 368 low, max — {low,high} minimal, {medium,max} full; none accepted but does not disable
glm-5.3-flash Supported: low, medium, high, xhigh low 27 · medium 289 · high 31 · xhigh 295 low, xhigh — {low,high} minimal, {medium,xhigh} full
glm-5.2 Supported: low, medium, high, xhigh low 471 · medium 367 · high 356 · xhigh 666 low, xhigh — low/medium/high overlap
glm-4.7-flash Supported: low, medium, high 1294 · 1451 · 567 (spreads overlap) [] — accepted but inert
deepseek-v4-pro Supported: low, medium, high, xhigh low 282 · medium 214 · high 746 · xhigh 4124 low, high, xhigh — medium ≡ low
deepseek-v4-flash Supported: low, medium, high, xhigh 563 · 242 · 199 · 226 (no consistent order) [] — accepted but no reliable control
kimi-k2.6 Supported: low, medium, high 528 · 577 · 456 [] — always-on (matches lab toggle-only)
kimi-k2.7-code Supported: low, medium, high 141 · 155 · 283 (spreads overlap) []
qwen3.8-27b Supported: low, medium, xhigh low 138 · medium 166 · xhigh 245 low, xhigh — medium ≡ low

Sample 400 (verbatim): Model 'deepseek-v4-flash' does not accept reasoning_effort 'max'. Supported: low, medium, high, xhigh.

Why not lab/peer label names (e.g. high/max for GLM-5.2): this host returns 400 for
max on glm-5.2 — a value the API rejects cannot be advertised; the labels used are the
accepted names for each measured band.

Verification for reviewers

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [high] [possible mistake] providers/clf-ai-gateway/models/deepseek-v4-flash.toml:7 - Check: Relay reasoning_options must reflect real caller controls; [] means no control, not weak/noisy effect (AGENTS.md Reasoning options; audit skill evidence bar). Why: This file sets reasoning_options = [] while its own comments say the gateway still accepts and validates reasoning_effort (low/medium/high/xhigh) and lab/same-surface peers expose real DeepSeek V4 controls (lab/peers: toggle + graded effort, commonly high/max). n=3 high-variance samples are not enough to treat a documented accept-set as “no caller control,” and clients will stop sending a field the API still takes. Action: Restore non-empty effort options that match this host’s public accept-set (and drop only proven no-op aliases), or show stronger host evidence that reasoning_effort is rejected/ignored end-to-end—not merely statistically noisy.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/glm-5.2.toml:9 - Check: Baseline effort labels for a relay should follow lab/peer canonical levels for the model, not invent alternate names for the same bands. Why: Lab Z.ai/Zhipu entries treat effective GLM-5.2 levels as high|max (low/mediumhigh, xhighmax). This file advertises ["low", "xhigh"], which mislabels the default band as low and skips the lab’s canonical pair even though the comment already describes two behaviors (default cluster vs xhigh). Action: Prefer lab-aligned wire names for the two distinct bands (typically high + max/xhigh if those are accepted here), and keep aliases out of values.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/deepseek-v4-flash.toml:1 - Check: base_model must point at the actual underlying lab identity when the host resells a specific snapshot. Why: Provider text says this gateway runs on Cloudflare Workers AI upstream, and Workers AI catalogs DeepSeek as dated snapshots (deepseek-v4-flash-0731, deepseek-v4-pro-0813), while these entries use generic deepseek/deepseek-v4-flash and deepseek/deepseek-v4-pro (different lab metadata/dates). Action: Confirm the upstream @cf/... ids from GET /v1/public/models; if they are the dated Workers AI builds, retarget base_model to deepseek/deepseek-v4-flash-0731 and deepseek/deepseek-v4-pro-0813 (same for the pro file).
  • [low] [possible mistake] .pr-review/pull-request.json (PR body) - Check: Material factual changes should cite mapped first-party evidence in the PR body. Why: Commit 3’s measured effort pruning is the main reasoning-data change, but the PR body still only points at the public models endpoint and does not include the per-model median/reasoning_tokens table claimed in the TOML comments. Action: Add that measurement table (or equivalent host docs) to the PR body, mapped model-by-model to the final reasoning_options sets.

… measured effects documented

reasoning_options.values now list exactly what this host accepts (the public capabilities.reasoning_efforts; anything else is a 400) - the client-facing control surface - with the measured per-level effect (3 samples/level medians) documented in each file and in the PR body. DeepSeek entries now reference the dated lab snapshots that match the upstream ids (deepseek-v4-flash-0731, deepseek-v4-pro-0813). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/clf-ai-gateway/models/kimi-k2.6.toml:11 - Check: Relay reasoning_options must match lab/same-surface peers and real caller control (AGENTS.md Reasoning options; audit skill baseline / []). Why: Lab providers/moonshotai and peers (OpenRouter, Vercel) are toggle-only or []; this file advertises effort low|medium|high while its own comment says those values are accepted but ignored and thinking is always on. Graded effort here invents a control surface clients do not have. Action: Set reasoning_options = [] (or toggle only if this host actually exposes an on/off wire field—document it in a leading comment).
  • [high] [violation] providers/clf-ai-gateway/models/kimi-k2.7-code.toml:11 - Check: Same baseline rule for Kimi K2.7 Code. Why: Lab and most relays use reasoning_options = []; the file’s measured note says low|medium|high are accepted but ignored. Advertising graded effort is false caller control. Action: Set reasoning_options = [].
  • [high] [violation] providers/clf-ai-gateway/models/deepseek-v4-flash.toml:11 - Check: Do not invent L/M/H (or similar) for DeepSeek V4; [] only when there is affirmatively no caller control. Why: Lab/CF/OpenRouter baselines are toggle + high/max (or low|high|max), not low|medium|high|xhigh. The file’s own 2026-09-02 medians show no consistent ordering and call the set “not a reliable control,” yet still publish four effort levels. Action: Use reasoning_options = [], or only levels with a verified distinct effect on this host—and never a GPT-style L/M/H/xhigh dump that lab/peers do not use.
  • [high] [violation] providers/clf-ai-gateway/models/glm-4.7-flash.toml:11 - Check: effort values must be real controls on this host, not accept-only aliases. Why: Comment states low|medium|high are accepted but ignored / no measurable graded control. Publishing them as reasoning_options tells clients there is graded reasoning when there is not. Action: Set reasoning_options = [] (or add toggle only if this surface forwards a real thinking on/off field).
  • [medium] [possible mistake] providers/clf-ai-gateway/models/deepseek-v4-pro.toml:11 - Check: Extra effort levels beyond lab/peers need meaningful host effect; avoid alias noise. Why: Lab is toggle + high/max; this host omits toggle (OK if true) but lists low|medium|high|xhigh. Own medians: medium ≈ low; low/high/xhigh distinct. Including medium (and framing four GPT-like grades) overstates control vs lab baseline. Action: Drop non-distinct aliases (at least medium); keep only levels with a measured distinct effect on this host (e.g. low|high|xhigh), and note the wire field in the leading comment.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/glm-5.3.toml:11 - Check: Effort labels must match real behavior; accept-set completeness must be consistent. Why: Lab is low|high|max (always-on). Own medians: {low,high} minimal vs {medium,max} full, so advertising high as a higher tier is misleading; comment also says none is in the public accept-set but is omitted while the file claims “exactly that host accept-set.” Action: Publish only levels with distinct measured effect (or lab-aligned low|high|max if those map correctly here). Either include none if it is truly accepted, or stop claiming a complete accept-set copy.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/glm-5.3-flash.toml:11 - Check: Same semantic-effort rule. Why: Medians show {low,high} minimal and {medium,xhigh} full, so listing all four implies a normal L→H→xhigh ladder that this host does not implement. Action: Restrict values to levels with distinct measured effect (e.g. low|xhigh or whatever pair is stable), matching lab/peer intent rather than a full alias dump.
  • [medium] [possible mistake] providers/clf-ai-gateway/models/glm-5.2.toml:11 - Check: Extra levels need distinct host effect. Why: Lab effective set is high|max; file lists low|medium|high|xhigh while medians say only xhigh is distinct and L/M/H overlap as default aliases. Action: Advertise only levels that change behavior on this host (and drop pure aliases), with a short leading wire comment.

…re inert, no toggle wire field

Per the latest review round: reasoning_options lists only levels with a measured distinct effect on this host (3 samples/level medians, 2026-09-02); models whose accepted levels are inert get [] with a leading comment stating that no on/off wire field is forwarded either. Accept-sets stay documented in each file and in the PR body. DeepSeek base_models keep the dated snapshots. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewer: ready Automated review found no actionable items

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant