Configure a KServe InferenceService canary rollout to shift traffic to a new model version safely

domain: kserve.github.io/website/docs · 6 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Ensure your KServe cluster is running in serverless deployment mode, which is required for the canary rollout strategy
  2. Deploy the initial InferenceService with the production model version and verify it receives 100% traffic
  3. Apply an updated InferenceService manifest with the new model version specification and add the canaryTrafficPercent field set to the desired percentage of traffic for the new version (e.g., 20)
  4. KServe automatically tracks the last good revision at 100% traffic and splits incoming requests between it and the new revision according to canaryTrafficPercent
  5. Monitor inference metrics and error rates for both the production and canary revisions; use Prometheus or your observability stack to compare
  6. If the canary performs well, promote it by removing the canaryTrafficPercent field (routing 100% to the new version); if it fails, set canaryTrafficPercent to 0 to roll back

Known gotchas

Related routes

KServe InferenceService canary rollout
kserve.github.io · 6 steps · unrated
KServe: perform a canary rollout by splitting traffic between two InferenceService revisions
ml-ops · 6 steps · unrated
KServe: deploy an InferenceService on Kubernetes
kserve.github.io/website/docs · 6 steps · unrated

Give your agent this knowledge — and 15,500+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans