refactor(api): replace provider if-else chains in simple_chat with a ChatStreamer

[GdoongMathew]

Summary

Extracts provider-specific streaming into a ChatStreamer hierarchy.

Changes

• New chat module: abstract ChatStreamer + one subclass per provider (Ollama,
  OpenRouter, OpenAI, Azure, Bedrock, DashScope, Google, LiteLLM). Subclasses self-register via
  __init_subclass__; ChatStreamer.create(...) returns the right one. Adding a provider
  = adding a class. Each exposes respond_stream(prompt) -> AsyncIterator[str].
• simple_chat.py & websocket_wiki.py: reduced to prompt_builder builders plus
  stream_and_fallback(), which retries once with the context-free prompt on token-limit
  errors. Streamers now propagate exceptions, so the fallback works for all providers.

[gemini-code-assist]

Code Review

This pull request refactors the chat streaming and prompt building logic by introducing a unified ChatStreamer abstraction and helper functions in a new api/chat module. This refactoring significantly simplifies api/simple_chat.py and api/websocket_wiki.py by removing duplicate provider-specific streaming and fallback implementations. Additionally, unit tests have been added to verify provider registration. Feedback on the changes highlights potential KeyError exceptions across multiple ChatStreamer subclasses due to direct dictionary access on model_config keys (such as temperature, top_p, top_k, and num_ctx), and notes that LiteLLMChatStreamer is currently missing from the unit tests.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Accessing model_config["temperature"] directly can raise a KeyError if the key is missing from the configuration. It is safer to only add temperature to model_kwargs if it exists in model_config, similar to how top_p is handled.

Suggested change

	def __init__(self, *, model: str, model_config: MODEL_CFG):
	self.client = self._build_client()
	self.model_kwargs = {
	"model": model,
	"stream": True,
	"temperature": model_config["temperature"]
	}
	# Only add top_p if it exists in the model config
	if "top_p" in model_config:
	self.model_kwargs["top_p"] = model_config["top_p"]
	def __init__(self, *, model: str, model_config: MODEL_CFG):
	self.client = self._build_client()
	self.model_kwargs = {
	"model": model,
	"stream": True,
	}
	if "temperature" in model_config:
	self.model_kwargs["temperature"] = model_config["temperature"]
	if "top_p" in model_config:
	self.model_kwargs["top_p"] = model_config["top_p"]

[gemini-code-assist]

Accessing model_config["temperature"], model_config["top_p"], and model_config["top_k"] directly can raise a KeyError if any of these keys are missing from the configuration. It is safer to dynamically build the GenerationConfig parameters based on what is present in model_config.

Suggested change

	def __init__(self, *, model: str, model_config: MODEL_CFG):
	import google.generativeai as genai
	from google.generativeai.types import GenerationConfig
	self.client = genai.GenerativeModel(
	model_name=model,
	generation_config=GenerationConfig(
	temperature=model_config["temperature"],
	top_p=model_config["top_p"],
	top_k=model_config["top_k"],
	)
	)
	def __init__(self, *, model: str, model_config: MODEL_CFG):
	import google.generativeai as genai
	from google.generativeai.types import GenerationConfig
	generation_kwargs = {}
	for key in ("temperature", "top_p", "top_k"):
	if key in model_config:
	generation_kwargs[key] = model_config[key]
	self.client = genai.GenerativeModel(
	model_name=model,
	generation_config=GenerationConfig(**generation_kwargs)
	)

[gemini-code-assist]

Accessing model_config["temperature"], model_config["top_p"], and model_config["num_ctx"] directly can raise a KeyError if any of these keys are missing from the configuration. It is safer to dynamically build the options dictionary based on what is present in model_config.

Suggested change

	def __init__(self, *, model: str, model_config: MODEL_CFG):
	from adalflow.components.model_client.ollama_client import OllamaClient
	self.client = OllamaClient()
	self.model_kwargs = {
	"model": model,
	"stream": True,
	"options": {
	"temperature": model_config["temperature"],
	"top_p": model_config["top_p"],
	"num_ctx": model_config["num_ctx"]
	}
	}
	def __init__(self, *, model: str, model_config: MODEL_CFG):
	from adalflow.components.model_client.ollama_client import OllamaClient
	self.client = OllamaClient()
	options = {}
	for key in ("temperature", "top_p", "num_ctx"):
	if key in model_config:
	options[key] = model_config[key]
	self.model_kwargs = {
	"model": model,
	"stream": True,
	"options": options
	}

[gemini-code-assist]

Accessing model_config["temperature"] directly can raise a KeyError if the key is missing from the configuration. It is safer to only add temperature to model_kwargs if it exists in model_config.

Suggested change

	def __init__(self, *, model: str, model_config: MODEL_CFG):
	if not OPENROUTER_API_KEY:
	logger.warning("OPENROUTER_API_KEY not configured, but continuing with request")
	# We'll let the OpenRouterClient handle this and return a friendly error message
	from api.openrouter_client import OpenRouterClient
	self.client = OpenRouterClient()
	self.model_kwargs = {
	"model": model,
	"stream": True,
	"temperature": model_config["temperature"]
	}
	if "top_k" in model_config:
	self.model_kwargs["top_k"] = model_config["top_k"]
	def __init__(self, *, model: str, model_config: MODEL_CFG):
	if not OPENROUTER_API_KEY:
	logger.warning("OPENROUTER_API_KEY not configured, but continuing with request")
	# We'll let the OpenRouterClient handle this and return a friendly error message
	from api.openrouter_client import OpenRouterClient
	self.client = OpenRouterClient()
	self.model_kwargs = {
	"model": model,
	"stream": True,
	}
	if "temperature" in model_config:
	self.model_kwargs["temperature"] = model_config["temperature"]
	if "top_k" in model_config:
	self.model_kwargs["top_k"] = model_config["top_k"]

[gemini-code-assist]

Accessing model_config["temperature"] and model_config["top_p"] directly can raise a KeyError if any of these keys are missing from the configuration. It is safer to only add them to model_kwargs if they exist in model_config.

Suggested change

	def __init__(self, *, model: str, model_config: MODEL_CFG):
	from api.dashscope_client import DashscopeClient
	self.client = DashscopeClient()
	self.model_kwargs = {
	"model": model,
	"stream": True,
	"temperature": model_config["temperature"],
	"top_p": model_config["top_p"],
	}
	def __init__(self, *, model: str, model_config: MODEL_CFG):
	from api.dashscope_client import DashscopeClient
	self.client = DashscopeClient()
	self.model_kwargs = {
	"model": model,
	"stream": True,
	}
	for key in ("temperature", "top_p"):
	if key in model_config:
	self.model_kwargs[key] = model_config[key]