Skip to content

feat: add Lyceum provider - #6117

Open
Casparlehmkuehler wants to merge 1 commit into
anomalyco:devfrom
Casparlehmkuehler:add-lyceum-provider
Open

feat: add Lyceum provider#6117
Casparlehmkuehler wants to merge 1 commit into
anomalyco:devfrom
Casparlehmkuehler:add-lyceum-provider

Conversation

@Casparlehmkuehler

Copy link
Copy Markdown

Lyceum (https://lyceum.technology) is an EU-hosted, OpenAI-compatible inference API.
42 model ids at https://api.lyceum.technology/openai/v1, auth LYCEUM_API_KEY.

Lyceum hosts other labs' models, so every provider entry is base_model +
overrides. Six lab entries did not exist yet and are added under models/:
nousresearch/hermes-4-{70b,405b}, openbmb/minicpm-v-4_5,
google/gemma-3-27b-it, nvidia/cosmos3-super-reasoner,
alibaba/qwen3-30b-a3b-instruct-2507, alibaba/qwen3-embedding-8b.

Every provider-side value was measured against the live API on 2026-09-02, not
assumed:

  • cost: GET /api/v2/external/pricing?resource=serverless_inference, USD per
    token converted to USD/MTok.
  • reasoning_options: each reasoning_effort value posted to
    /chat/completions (3 attempts, to separate real rejections from transients).
    none is listed only where it demonstrably suppresses reasoning. [] means
    the host accepts no caller control, not that it went untested.
  • interleaved: which response key carries the reasoning text. true where it
    is reasoning, which the schema's field enum does not cover.
  • attachment: an image_url content part posted per model; text-only models
    reject it, so the signal discriminates.
  • limit.output: the proxy rejects max_tokens above 65536 for every model, so
    entries resolve to at most that. Three models cap lower and were bisected:
    openbmb/minicpm-v-4_5 24576, qwen/qwen2.5-vl-72b-instruct and
    qwen/qwen3-32b sit under the global cap already.

Two id families are Lyceum-specific aliases, both resolved server-side and
priced identically to their target: lyceum/{simple,complex,reasoning} (router
aliases) and the -instant suffix (pins enable_thinking = false for clients
that cannot set request parameters).

bun validate passes.

Happy to supply a test key to a maintainer out of band if CI needs one.

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/lyceum/models/deepseek/deepseek-v4-pro.toml:2 - Check: Reasoning options must follow lab/peer baseline for this host, not a full schema enum dump. Why: Lyceum is a multi-model OpenAI-compatible relay. First-party DeepSeek V4 Pro is toggle + high/max (Flash is toggle + low/high/max); established relays stay near that narrow set. This entry (and deepseek-v4-flash-0731.toml) authors the full effort enum none|minimal|low|medium|high|xhigh|max, which AGENTS.md / the audit skill forbid as invented universal levels—API acceptance alone is not distinct effort. Action: Replace with the host-true subset of DeepSeek’s controls (typically none in effort or a documented toggle, plus only levels that change behavior on Lyceum, e.g. high/max or Flash’s low/high/max), with a leading wire comment for the exact field.
  • [high] [violation] providers/lyceum/models/z-ai/glm-5.2.toml:2 - Check: GLM reasoning_options must match lab/same-surface peers, not a full enum. Why: First-party Z.ai GLM-5.2 is effort high/max (with none/minimal as skip); GLM-5.3 / 5.3-Flash are low/high/max. Lyceum dumps the full enum on glm-5.2, glm-5.3, glm-5.3-flash, and the lyceum/reasoning alias, and GLM-5.1 uses a graded list while the lab is toggle-only. That mis-documents caller controls vs the underlying models. Action: Author only levels Lyceum actually exposes with distinct effect, aligned to each GLM generation (5.1 toggle or equivalent; 5.2 high/max ± real off; 5.3/Flash low/high/max ± real off), and mirror the same set on router aliases.
  • [high] [violation] providers/lyceum/models/moonshotai/kimi-k2.6.toml:2 - Check: Kimi reasoning_options baseline is lab + peers, not a full dump. Why: Moonshot K2.6 is toggle-only; K3 is toggle + low/high/max. Lyceum sets K2.6 to the full effort enum and K3 to only none/max, which neither matches the lab surface nor a documented peer intersection. Action: Set K2.6 to the control Lyceum actually forwards (toggle and/or the real effort values), and set K3 to the measured subset of low/high/max plus off via none or toggle—not a full enum or an unexplained none/max-only pair.
  • [high] [violation] providers/lyceum/models/qwen/qwen3.5-9b.toml:2 - Check: Qwen3.5 relay options should follow Alibaba/peer controls for this API shape. Why: First-party Qwen3.5 is toggle (+ often budget_tokens on Alibaba chat), not a seven-value effort ladder. Lyceum applies the full enum on qwen3.5-9b, qwen3.5-397b-a17b, and lyceum/simple. That invents GPT-style graded effort for a toggle/budget family. Action: Use the controls Lyceum really implements for these IDs (e.g. reasoning_effort values that map to thinking on/off and any real grades, or toggle + budget if that path exists)—not the full schema enum—and keep router aliases in sync.
  • [medium] [violation] providers/lyceum/models/nvidia/nemotron-3-super-120b-a12b.toml:2 - Check: Nemotron options must not invent graded effort when lab/peers are toggle/budget. Why: NVIDIA first-party Super/Ultra/Nano are toggle (Omni adds budget_tokens). Lyceum sets Super to low/medium/high, Ultra and Omni to the full effort enum, and Nano to [] while still marking interleaved reasoning. That conflicts with the lab baseline and the “no full enum / no invented L/M/H” rule unless each level is proven distinct on this host. Action: Align each Nemotron entry to measured Lyceum controls without dumping the schema enum; prefer lab-like toggle (and Omni budget only if present), or a proven effort subset—not Ultra/Omni full dumps or Super L/M/H by default.
  • [medium] [possible mistake] providers/lyceum/models/z-ai/glm-5.1.toml:2 - Check: Provider overrides of lab modalities/attachment must be real host deltas. Why: Lab zhipuai/glm-5.1 is text-only (attachment = false, input ["text"]). This file forces attachment = true and input = ["text", "image"]. That is a large capability claim beyond the shared lab identity; a single image_url accept is weak evidence that the model is multimodal. Action: Confirm vision is truly supported end-to-end on Lyceum for GLM-5.1; if not, drop the attachment/modalities overrides and keep lab text-only metadata.
  • [medium] [possible mistake] providers/lyceum/models/nvidia/llama-3_1-nemotron-ultra-253b-v1.toml:2 - Check: reasoning = false overrides on a reasoning lab model need host-specific evidence. Why: nvidia/llama-3.1-nemotron-ultra-253b is reasoning = true. Overriding to false without a leading rationale (unlike MiniCPM / -instant aliases) risks cataloging a reasoner as non-reasoning if the host still emits CoT. Action: Document/verify that this Lyceum ID never returns reasoning; if it does, keep reasoning = true and set accurate reasoning_options / interleaved.
  • [low] [possible mistake] models/nousresearch/hermes-4-405b.toml:16 - Check: New lab entries should include complete limit defaults when known. Why: Hermes 4 and Cosmos3 Super Reasoner lab files omit limit.output (providers only patch output). That leaves incomplete lab metadata for other hosts and makes inheritance easy to get wrong. Action: Add authoritative lab limit.output (and any other known lab defaults) from first-party/model cards where available, keeping Lyceum’s 65_536 as a provider override only when the host is lower.

@Casparlehmkuehler

Copy link
Copy Markdown
Author

Thanks, all eight items addressed in the amended commit.

The full-enum lists were the wrong call: I had measured which reasoning_effort
values the API accepts, and you are right that acceptance is not a distinct
level. Re-measured with 5 attempts per value (the first pass had transients that
made a couple of models look narrower than they are) and authored the
intersection of the first-party entry and what this host really exposes:

  • deepseek-v4-pro toggle + high/max; deepseek-v4-flash-0731 toggle +
    low/high/max, matching providers/deepseek.
  • glm-5.1 toggle; glm-5.2 toggle + high/max; glm-5.3 and
    glm-5.3-flash toggle + low/high/max, matching providers/zhipuai. The
    toggle is real on top of the graded set: none suppresses the reasoning,
    which is also what the -instant aliases pin.
  • kimi-k2.6 toggle; kimi-k3 toggle + low/high/max. medium is the
    only value K3 rejects, so it now matches providers/moonshotai exactly.
  • qwen3.5-9b and qwen3.5-397b-a17b toggle. No budget field is reachable
    here, so unlike Alibaba's own entries there is no budget_tokens.
  • qwen3.8-27b and qwen3.8-flash-next toggle + low/medium/xhigh, which
    is providers/alibaba/models/qwen3.8-flash.toml's effort set; high is
    rejected upstream. qwen3.8-2.4t-a95b gets the same effort set with no
    toggle, since it rejects none.
  • Nemotron: Omni none/low/medium/high/max and Ultra
    none/medium/high, following the relay peers. Super keeps
    low/medium/high because this host rejects none, so there is genuinely
    no off switch; Nano-30B keeps [] because none is accepted but does not
    suppress anything. Both now say so in a leading comment.

Every toggle and effort entry now carries a leading comment naming the wire
control, per AGENTS.md.

On the two possible mistakes:

  • glm-5.1 vision: you were right to doubt it. Sent a solid-colour test image
    and asked for the colour; GLM-5.1 answers "White" for a red image, so it
    accepts the part without reading it. Dropped the attachment and modalities
    overrides and kept the lab's text-only metadata. Re-ran the same check on the
    entries that do claim vision beyond their lab row: qwen2.5-vl-72b-instruct
    and minicpm-v-4_5 name the colour correctly, so those overrides stand.
  • llama-3_1-nemotron-ultra-253b-v1: also right. With detailed thinking on as
    the system prompt it does reason, into a <think> block inside content.
    Removed the reasoning = false override; it is now reasoning = true with
    reasoning_options = [] and a comment explaining that the switch is the
    system prompt, not a request field. The same test caught qwen3-32b, which
    reasons via Qwen's /think convention, so it changed the same way. Neither
    sets interleaved, since the block is in content rather than a side channel.

Lab limit.output added: Hermes 4 70B/405B 131_072, Cosmos3 Super Reasoner
256_000, MiniCPM-V 4.5 32_768. Lyceum's own ceilings stay as provider overrides
(65_536 platform-wide, 24_576 for MiniCPM).

bun validate passes.

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/lyceum/models/deepseek/deepseek-v4-flash-0731.toml:4 - Check: When off is reasoning_effort = none among graded levels, author effort with none in values and no toggle (AGENTS.md → Reasoning options; same-surface peers such as NVIDIA NIM / Baseten / Above / Kosmik / Groq). Why: These files document off as reasoning_effort = none on the same field as graded effort, then encode toggle + effort without none. That is not a separate on/off control; clients will not learn to send none, and the shape conflicts with established OpenAI-compatible peers. Action: Drop toggle and put none in effort.values. Expected sets from the file comments: deepseek-v4-flash-0731none|low|high|max; deepseek-v4-pro, lyceum/reasoning, z-ai/glm-5.2none|high|max; moonshotai/kimi-k3none|low|high|max; qwen/qwen3.8-27b, qwen/qwen3.8-flash-nextnone|low|medium|xhigh; z-ai/glm-5.3, z-ai/glm-5.3-flashnone|low|high|max. Keep a leading wire comment like # Effort: reasoning_effort = none|…. Apply the same fix in: providers/lyceum/models/deepseek/deepseek-v4-pro.toml, providers/lyceum/models/lyceum/reasoning.toml, providers/lyceum/models/moonshotai/kimi-k3.toml, providers/lyceum/models/qwen/qwen3.8-27b.toml, providers/lyceum/models/qwen/qwen3.8-flash-next.toml, providers/lyceum/models/z-ai/glm-5.2.toml, providers/lyceum/models/z-ai/glm-5.3.toml, providers/lyceum/models/z-ai/glm-5.3-flash.toml.

@Casparlehmkuehler

Copy link
Copy Markdown
Author

Fixed. toggle is gone from every entry whose off switch is
reasoning_effort = none on the same field as the graded levels; those are now
a single effort list with none in values, exactly the sets you listed:

  • deepseek-v4-flash-0731, kimi-k3, glm-5.3, glm-5.3-flashnone|low|high|max
  • deepseek-v4-pro, glm-5.2, lyceum/reasoningnone|high|max
  • qwen3.8-27b, qwen3.8-flash-nextnone|low|medium|xhigh

The leading comments now say which way the unlisted values go, since it differs
by host: kimi-k3 rejects medium, the qwen3.8 pair rejects high and max,
and the GLM and DeepSeek entries accept everything but collapse it onto the
listed levels the way their first-party APIs document.

toggle survives only on the five ids with no graded levels at all
(kimi-k2.6, glm-5.1, qwen3.5-9b, qwen3.5-397b-a17b, lyceum/simple),
which matches their lab entries.

bun validate passes.

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/lyceum/models/meta-llama/llama-3.3-70b-instruct.toml:3 - Check: Override-only base_model files must not restate identical base fields. Why: models/meta/llama-3.3-70b-instruct.toml already has attachment = false and text-only modalities, so this restates an unchanged value. Action: Remove the redundant attachment = false (and keep only real host deltas such as cost).
  • [medium] [possible mistake] providers/lyceum/models/z-ai/glm-5.3.toml:4 - Check: Relay reasoning_options must match this host’s real controls and must not invent levels beyond lab/peer baseline without verified effect. Why: First-party providers/zai/models/glm-5.3.toml and established peers author low|high|max and document that GLM-5.3 always reasons (thinking cannot be disabled). This entry adds none as an off switch, which contradicts that baseline. Action: Re-verify on Lyceum whether reasoning_effort=none actually suppresses reasoning end-to-end; if not, drop none and use ["low", "high", "max"] (and apply the same correction to providers/lyceum/models/z-ai/glm-5.3-instant.toml only if that alias is meant to stay non-reasoning via name encoding, not via effort).
  • [medium] [possible mistake] providers/lyceum/models/qwen/qwen3.8-flash-next.toml:4 - Check: Effort sets for always-on lab models should not list none unless this host truly turns reasoning off. Why: Lab metadata for alibaba/qwen3.8-flash-next states thinking is always on with low|medium|xhigh, matching peers such as Cortecs; this file still includes none as a disable value. Action: Confirm reasoning_effort=none actually disables reasoning on Lyceum; if it is rejected or ignored, remove none and keep ["low", "medium", "xhigh"].
  • [medium] [possible mistake] providers/lyceum/models/minimax/minimax-m3.toml:3 - Check: On relays, reasoning_options = [] means affirmative no caller control, not “only reasoning_effort was tried.” Why: First-party MiniMax M3 and several same-surface peers expose a real on/off control (toggle), while this entry sets [] after testing only reasoning_effort. The same pattern appears on providers/lyceum/models/lyceum/complex.toml (alias of M3). Action: Verify whether Lyceum forwards any non-reasoning_effort disable/enable control used by MiniMax/peers; if yes, author that control (with a leading wire comment), otherwise keep [] and note the affirmative no-control finding in the leading comment.
  • [medium] [possible mistake] providers/lyceum/models/nvidia/nvidia-nemotron-3-nano-30b-a3b.toml:3 - Check: [] on a relay of a lab model that has caller controls needs affirmative host evidence. Why: First-party providers/nvidia/models/nvidia/nemotron-3-nano-30b-a3b.toml authors toggle, while this file sets [] because reasoning_effort=none does not suppress reasoning. That may miss a different wire control the host still accepts. Action: Confirm whether Lyceum exposes any on/off control for this model beyond reasoning_effort; if a real toggle exists, author it with a leading wire comment, otherwise keep [] with an explicit no-control note.

Lyceum is an EU-hosted OpenAI-compatible inference API. 42 model ids, all
third-party models, so every provider entry uses base_model.

Adds the lab metadata the listing needs: nousresearch/hermes-4-{70b,405b},
openbmb/minicpm-v-4_5, google/gemma-3-27b-it, nvidia/cosmos3-super-reasoner,
alibaba/qwen3-30b-a3b-instruct-2507, alibaba/qwen3-embedding-8b.

Cost, reasoning_options, interleaved, attachment and limit.output were each
read off the live API on 2026-09-02 rather than assumed, and every reasoning
entry carries a leading comment with the wire control it describes.
@Casparlehmkuehler

Copy link
Copy Markdown
Author

Two of the four doubts were real bugs, one was a good question with a
measurable answer, and one rests on a lab row that says the opposite.

Fixed: nemotron-3-nano-30b-a3b does have a toggle. You were right that
[] after testing only reasoning_effort was not affirmative. The control is
chat_template_kwargs.enable_thinking = false, and it is a real disable, not
response filtering: on a fixed prompt the completion drops from 278 to 131
tokens and the reasoning field disappears. Now [{ type = "toggle" }] with
that field named in the leading comment.

Checked and kept [], now stated affirmatively. For minimax-m3,
minimax-m2.5, kimi-k2.7-code, qwen3-next-80b-a3b-thinking and the
lyceum/complex alias I tried reasoning_effort, enable_thinking,
chat_template_kwargs.enable_thinking, thinking, thinking.type and
reasoning.enabled; every one leaves the reasoning on. Each file's leading
comment now lists what was tried.

One trap worth recording: on minimax-m2.5, reasoning.exclude = true makes
the reasoning vanish from the response, which looks like a toggle. It is not.
With max_tokens = 600 the completion still bills all 600 tokens and returns
empty content, so the model reasoned and the output was discarded. Not authored
as a control.

none genuinely disables on the GLM, DeepSeek, Kimi and qwen3.8 entries.
Re-verified end to end with a 400-token budget and a reasoning-inducing prompt,
three repeats each. Without the flag every one returns a populated reasoning
channel; with reasoning_effort = none all three repeats come back with no
reasoning channel and no <think> block:

model no flag none x3
glm-5.1 reasoning_content off, off, off
glm-5.2 reasoning off, off, off
glm-5.3 reasoning off, off, off
glm-5.3-flash reasoning_content off, off, off
qwen3.8-27b reasoning_content off, off, off
qwen3.8-flash-next reasoning_content off, off, off
deepseek-v4-pro reasoning off, off, off
deepseek-v4-flash-0731 reasoning off, off, off
kimi-k3 reasoning off, off, off

So none stays in those values. Lyceum serves GLM-5.3 and the qwen3.8 pair
through an upstream that honours the switch even where the lab row says thinking
is always on; if you would rather the catalog not diverge from the lab baseline
on that point, say so and I will drop none and lose the off switch in the
metadata.

llama-3.3-70b-instruct: the attachment = false override is a real delta.
models/meta/llama-3.3-70b-instruct.toml:6 is attachment = true, sitting next
to text-only modalities, so the lab row is internally inconsistent rather than
already false. Lyceum rejects an image_url content part outright, so I kept the
override and the leading comment now explains why. Happy to drop it if the lab
row gets corrected to false instead.

bun validate passes.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewer: ready Automated review found no actionable items

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant