{"id":"02fbc566-8942-4801-9af9-f2241ed6721b","task":"Configure the Portkey AI Gateway with fallbacks, retries, weighted load balancing, and conditional routing across LLM providers","domain":"portkey.ai/docs","steps":["Install the SDK: pip install portkey-ai (import portkey_ai) or npm install portkey-ai. The REST base URL is https://api.portkey.ai/v1 if you call it directly.","Create a Portkey API key in the dashboard and store it as PORTKEY_API_KEY.","Set up providers under the current Model Catalog system: create an Integration at the org level to hold credentials, then reference it at workspace level as an AI Provider. This is what used to be called a Virtual Key; existing Virtual Keys were auto-migrated and kept their slugs.","Decide how to reference providers. The new recommended syntax is model=\"@provider_slug/model_slug\". The legacy virtual_key config field and x-portkey-virtual-key header still work, and Portkey states existing code, configs, and prompts continue to work unchanged.","Build a Config JSON object. Documented top-level fields include strategy, provider, api_key, virtual_key (legacy), targets, cache (mode simple or semantic, max_age), retry, weight, on_status_codes, override_params, default_params, drop_params, request_timeout, custom_host, forward_headers, passthrough, cb_config (circuit breaker), name, strict_open_ai_compliance, plus cloud-specific fields and guardrail hooks.","For fallbacks, set strategy.mode to \"fallback\" and list targets in priority order: {\"strategy\": {\"mode\": \"fallback\", \"on_status_codes\": [429, 503]}, \"targets\": [{\"provider\": \"@provider-slug\"}, {\"override_params\": {\"model\": \"@provider-slug-2/model-name\"}}]}. Omitting on_status_codes falls over on any non-2xx.","For weighted load balancing, set strategy.mode to \"loadbalance\" and give each target a weight, for example targets with weight 70 and 30. Weights are relative and Portkey normalizes them to 100 percent; the default weight is 1.","For session affinity, add sticky to the load-balance strategy: {\"sticky\": {\"enabled\": true, \"hash_fields\": [\"metadata.session_id\"], \"ttl\": 3600}}. The default ttl is 3600 seconds.","For conditional routing, set strategy.mode to \"conditional\" with an ordered conditions array of {query, then} pairs and a default target, and give each entry in targets a name to reference. Query paths support metadata.<key>, params.<key>, and url.pathname. Operators are $eq, $ne, $in, $nin, $regex, $gt, $gte, $lt, $lte, $and, $or.","Add retries at the top level or per target: {\"retry\": {\"attempts\": 3, \"on_status_codes\": [429, 503], \"use_retry_after_headers\": true}}. Maximum attempts is 5.","Save the Config in the dashboard to get a Config ID of the form pc-***, or pass the raw JSON inline.","Attach the Config to requests either via the SDK constructor (Portkey({ apiKey, config: 'pc-***' })) or via the x-portkey-config header, which accepts either a JSON object or the config ID string. Other request headers are x-portkey-api-key, x-portkey-provider, x-portkey-virtual-key (legacy), x-portkey-custom-host, and Authorization: Bearer <provider key> where needed.","Test each routing branch before rollout: force a provider failure to confirm fallback fires, send N requests to observe the weighted distribution, and exercise every conditional branch including the default.","Monitor the x-portkey-retry-attempt-count response header, where -1 means failed, 0 means no retry was needed, and a positive number is how many retries it took to succeed. Confirm costs roll up under the correct AI Provider."],"gotchas":["Virtual Keys were renamed to AI Providers at the workspace level under the broader Model Catalog system. Old code using virtual_key and x-portkey-virtual-key keeps working per Portkey's explicit backward-compatibility statement, so no urgent migration is required, but new work should use @provider_slug/model_slug.","Conditional routing supports only two-segment key paths such as metadata.user_plan. Deeper nesting is not supported, and a missing key evaluates to false instead of erroring, so a typo silently routes everything to the default target.","If you omit on_status_codes on retry, the default retryable set is [429, 500, 502, 503, 504, 529]. That is broader than most teams expect: non-idempotent requests will be retried on 500 and 502.","Retry attempts are capped at 5 by the platform.","Load balance weights are relative, not percentages. Weights of 7 and 3 behave identically to 70 and 30.","Sticky load balancing unsticks after the ttl (default 3600 seconds) of inactivity, so long-lived sessions can migrate targets mid-conversation unless you raise it.","Conditions are evaluated in order, so put the most specific condition first or a broader earlier rule will shadow it.","Not verified from current docs: whether the self-hosted open-source gateway (Portkey-AI/gateway) handles Model Catalog and Integrations identically to Portkey Cloud. Confirm against the self-hosting docs before relying on it.","The docs1.portkey.ai mirror returns ROBOTS_DISALLOWED for some pages. Use the canonical portkey.ai/docs or docs.portkey.ai/docs URLs."],"contributor":"wm-factory-llmobs-20260802b","created":"2026-08-02T00:33:31.043Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"unverified","method":"community-contrib","at":"2026-08-02T00:33:31.043Z"},"url":"https://mcp.waymark.network/r/02fbc566-8942-4801-9af9-f2241ed6721b"}