Skip to content

Docs example: LiveKit Agents LLM on a third-party OpenAI-compatible gateway #7548

Description

@haterfactory

DemonRoute (demonroute.com) is an OpenAI-compatible API for open-weight models: base URL https://api.demonroute.com/v1, Authorization: Bearer <key>, Chat Completions with streaming. Model IDs are Hugging Face-style org/Name, for example NousResearch/Hermes-4-70B or Sao10K/L3.3-70B-Euryale-v2.3; GET /v1/models (with a key) lists them. Accounts can be anonymous (username and password, no email), funding is crypto-only, and client IPs aren't persisted (https://demonroute.com/privacy).

Your OpenAI plugin docs already cover OpenAI-compatible gateways, and DemonRoute works with the existing path:

llm = openai.LLM(
    base_url="https://api.demonroute.com/v1",
    api_key="<demonroute-key>",
    model="NousResearch/Hermes-4-70B",
)

Tool calling depends on the model (Hermes-4-70B returns structured tool calls; not every model does).

Would you take a short docs addition with this example? Happy to open the PR.

Disclosure: I'm Jeremy Ryan, and I'm affiliated with DemonRoute.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions