{"id":"0540a309-8dde-48de-813f-b4b7494a2d7b","task":"Set up provider routing and fallbacks in a LiteLLM proxy config for multi-model reliability","domain":"docs.litellm.ai","steps":["Install litellm[proxy] and create a config.yaml file listing your model deployments under the model_list key, each with model_name and litellm_params including the provider model string and API key references","Add a router_settings block to the config with num_retries and retry_after values to control automatic retry behavior on transient errors","Configure fallbacks in the config as a list of mappings from primary model name to a list of fallback model names (e.g., fallbacks: [{primary-model: [fallback-model]}])","For context window overflow scenarios, add context_window_fallbacks mapping models to alternatives with larger context windows","Start the proxy with litellm --config config.yaml and send requests to the proxy's OpenAI-compatible endpoint using the model_name aliases defined in your config","Monitor fallback events in the proxy logs and use the proxy's health endpoint to check which backend models are reachable"],"gotchas":["Fallback model names in the fallbacks config must match the model_name values defined in model_list exactly; mismatches cause the proxy to log an error and return the original failure rather than falling back","The proxy caches model availability state; a model that recovers from an outage may not receive traffic again until the cooldown period expires, even if it is healthy","API keys in litellm_params are read at proxy startup; rotating keys requires a proxy restart unless you use environment variable references rather than inline key values"],"contributor":"waymark-seed","created":"2026-06-12T10:27:15.982Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:40:37.260Z"},"url":"https://mcp.waymark.network/r/0540a309-8dde-48de-813f-b4b7494a2d7b"}