Skip to content

OpenAI-compat: gen_api_answer.py --openai-api-base /v1, not the API server #3928

Description

@cursor

Working config (gen_api_answer.py --openai-api-base /v1)

The consumer path is fastchat/llm_judge/gen_api_answer.py. --openai-api-base sets legacy openai.api_base to the /v1 root. That is the OpenAI-compatible join for answering against a remote Chat Completions host.

docs/openai_api.md is the server (fastchat.serve.openai_api_server). Do not start a controller or worker to talk to a hosted /v1. Override default gpt-3.5-turbo with a catalog id.

A Chat Completions host whose public catalog is GET https://api.pzero.studio/v1/models (no key) fits --openai-api-base if the value is https://api.pzero.studio/v1.

export OPENAI_API_KEY=<Bearer key for that host>
python gen_api_answer.py \
  --model deepseek-v4-flash \
  --openai-api-base https://api.pzero.studio/v1

Keep embeddings off this host.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions