Documented steps

Install the client: `pip install openai`.
Instantiate with the target base: `client = OpenAI(base_url="http://localhost:8000/v1", api_key="…")` (vLLM port 8000; Ollama http://localhost:11434/v1 any api_key; OpenRouter https://openrouter.ai/api/v1 real key).
Call `client.chat.completions.create(model=..., messages=[...])`.
Read the reply from `response.choices[0].message.content`.

Known gotchas

Key semantics differ per target: vLLM enforces a key only if started with --api-key; Ollama requires the param but ignores its value; OpenRouter needs a valid key.
OpenRouter attribution: pass `extra_headers={"HTTP-Referer": "<url>", "X-OpenRouter-Title": "<name>"}`.
Server-only sampling params not in OpenAI's schema (e.g. vLLM top_k) go in `extra_body={"top_k": 50}`.
Docs: https://docs.vllm.ai/en/stable/serving/online_serving/openai_compatible_server/, https://openrouter.ai/docs/api/reference/overview, https://docs.ollama.com/openai

docs.vllm.ai · 5 steps · unrated

Serve LLMs with vLLM's OpenAI-compatible server

docs.vllm.ai · 6 steps · unrated

vLLM: serve a model behind an OpenAI-compatible HTTP API using `vllm serve`

ml-ops · 6 steps · unrated

Give your agent this knowledge — and 16,900+ more routes

One MCP install gives any agent live access to the full route map across 5,900+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans

Query an OpenAI-compatible LLM API with the official OpenAI Python SDK using a custom base_url (vLLM, Ollama, OpenRouter)

Documented steps

Known gotchas

Related routes

Give your agent this knowledge — and 16,900+ more routes

Need this verified for your stack — or a route we don't have yet?