vLLM: serve multiple LoRA adapters from a single base model deployment (multi-LoRA)

domain: ml-ops · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Start the server with --enable-lora and register one or more adapters via --lora-modules {name}={path}, e.g. `vllm serve meta-llama/Llama-3.2-3B-Instruct --enable-lora --lora-modules sql-lora=jeeejeee/llama32-3b-text2sql-spider`.
  2. Set sizing flags that apply to all adapters: --max-loras, --max-lora-rank (match the largest rank among the adapters you're serving, don't over-set it), and --max-cpu-loras.
  3. Confirm registration by calling `GET /v1/models` — each LoRA adapter is listed alongside the base model (with the JSON form of --lora-modules, its parent field points back to the base model id).
  4. Send inference requests specifying the adapter name in the model field, e.g. `curl http://localhost:8000/v1/completions -d '{"model": "sql-lora", "prompt": "San Francisco is a", "max_tokens": 7}'` — base-model and multiple LoRA requests are served concurrently.
  5. To add/remove adapters without restarting the server, set VLLM_ALLOW_RUNTIME_LORA_UPDATING=True and POST to /v1/load_lora_adapter and /v1/unload_lora_adapter with a lora_name/lora_path JSON body.

Known gotchas

Related routes

Serve LLMs with vLLM's OpenAI-compatible server
docs.vllm.ai · 6 steps · unrated
Serve an LLM with vLLM using tensor parallelism across multiple GPUs
docs.vllm.ai · 6 steps · unrated
Serve an LLM through vLLM's OpenAI-compatible API server
docs.vllm.ai · 5 steps · unrated

Give your agent this knowledge — and 15,500+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans