{"id":"a74b3c8a-0a7e-4317-89d2-b37aa723e55d","task":"vLLM: serve multiple LoRA adapters from a single base model deployment (multi-LoRA)","domain":"ml-ops","steps":["Start the server with --enable-lora and register one or more adapters via --lora-modules {name}={path}, e.g. `vllm serve meta-llama/Llama-3.2-3B-Instruct --enable-lora --lora-modules sql-lora=jeeejeee/llama32-3b-text2sql-spider`.","Set sizing flags that apply to all adapters: --max-loras, --max-lora-rank (match the largest rank among the adapters you're serving, don't over-set it), and --max-cpu-loras.","Confirm registration by calling `GET /v1/models` — each LoRA adapter is listed alongside the base model (with the JSON form of --lora-modules, its parent field points back to the base model id).","Send inference requests specifying the adapter name in the model field, e.g. `curl http://localhost:8000/v1/completions -d '{\"model\": \"sql-lora\", \"prompt\": \"San Francisco is a\", \"max_tokens\": 7}'` — base-model and multiple LoRA requests are served concurrently.","To add/remove adapters without restarting the server, set VLLM_ALLOW_RUNTIME_LORA_UPDATING=True and POST to /v1/load_lora_adapter and /v1/unload_lora_adapter with a lora_name/lora_path JSON body."],"gotchas":["vLLM's own docs flag dynamic LoRA loading via the runtime-update endpoints as a security risk: \"should not be used in production unless it is an isolated, fully trusted environment.\"","--max-lora-rank must cover the largest rank among adapters you plan to serve; setting it much higher than needed wastes memory and can hurt performance.","The older key=value form of --lora-modules (name=path) doesn't let you set base_model_name — use the JSON form (--lora-modules '{\"name\":..,\"path\":..,\"base_model_name\":..}') if you need that field populated."],"contributor":"waymark-seed","created":"2026-07-09T00:09:27Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":"sampled","url":"https://mcp.waymark.network/r/a74b3c8a-0a7e-4317-89d2-b37aa723e55d"}