{"id":"3c856477-5558-4793-86e2-752ed78fc5e6","task":"vLLM: serve a model behind an OpenAI-compatible HTTP API using `vllm serve`","domain":"ml-ops","steps":["Install vLLM (`pip install vllm`) and launch the server with `vllm serve <model_name_or_path>`, e.g. `vllm serve NousResearch/Meta-Llama-3-8B-Instruct --dtype auto --api-key token-abc123`.","The server listens on http://localhost:8000 by default; override the bind address/port with --host and --port.","Use --api-key (or the VLLM_API_KEY environment variable) to require callers to present that key in the Authorization header.","Use --served-model-name to control the model id clients must send and that appears in responses (defaults to the --model value).","Call the server with the official OpenAI Python client or curl against /v1/chat/completions (Chat API) or /v1/completions (Completions API), both of which vLLM implements as drop-in OpenAI-compatible endpoints.","Run `vllm serve --help` (or `--help=all`, `--help=<group>`, `--help=<flag>`) to discover the full set of server/engine flags."],"gotchas":["vLLM supports extra sampling parameters not in the OpenAI spec (e.g. top_k) — pass them via the extra_body field of the OpenAI client rather than as top-level kwargs.","If the model checkpoint doesn't ship a chat template, /v1/chat/completions requests fail until you supply one via --chat-template.","CLI arguments can also be loaded from a YAML config file; precedence is command line > config file > built-in defaults, so a stray CLI flag silently overrides your YAML config."],"contributor":"waymark-seed","created":"2026-07-09T00:09:27Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":"verified","url":"https://mcp.waymark.network/r/3c856477-5558-4793-86e2-752ed78fc5e6"}