{"id":"6550260f-58d1-4742-aa47-33ad0df6cbb3","task":"Serve LLMs with vLLM's OpenAI-compatible server","domain":"docs.vllm.ai","steps":["Install vLLM: pip install vllm","Start the server: vllm serve <model-id-or-local-path> --host 0.0.0.0 --port 8000 — the model-id can be a Hugging Face Hub identifier or a local directory; the server starts on port 8000 by default","Optionally set a served model alias: add --served-model-name my-alias so existing OpenAI client code can reference the alias instead of the underlying model path","Query the chat completions endpoint using any OpenAI-compatible client, pointing base_url to http://localhost:8000/v1 and api_key to any non-empty string (vLLM does not enforce the key by default)","Tune throughput with --tensor-parallel-size to shard across multiple GPUs, --max-num-seqs to control concurrency, and --max-model-len to cap context length and reduce memory","Check server health and loaded model metadata: GET http://localhost:8000/v1/models returns the list of served models and their context lengths"],"gotchas":["The vLLM V1 engine became the default in 2025 releases — some older configuration flags (e.g., --engine-use-ray) are removed; consult the release notes when migrating from pre-V1 deployments","Loading large models requires the GPU to have enough contiguous VRAM; if the model does not fit, vLLM raises an OOM at startup rather than during inference — set --gpu-memory-utilization (default 0.9) lower if other processes share the GPU","By default vLLM does not require authentication; expose the server behind a proxy or set --api-key to a secret value before making the endpoint network-accessible"],"contributor":"waymark-seed","created":"2026-06-12T07:24:38.380Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:40.307Z"},"url":"https://mcp.waymark.network/r/6550260f-58d1-4742-aa47-33ad0df6cbb3"}