KServe: perform a canary rollout by splitting traffic between two InferenceService revisions

domain: ml-ops · 6 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Deploy the first InferenceService revision normally; it receives 100% of traffic and KServe tracks it as the last good (LatestRolledoutRevision).
  2. Update the InferenceService spec with the new model version (e.g. a new storageUri) and set canaryTrafficPercent on the component, e.g. `canaryTrafficPercent: 10`, to send that percentage of traffic to the new revision.
  3. Re-apply the manifest with `kubectl apply`; KServe automatically splits traffic between the previously-promoted revision and the newly rolling-out revision according to canaryTrafficPercent.
  4. To promote the canary, remove the canaryTrafficPercent field and re-apply — the new revision becomes the LatestRolledoutRevision and takes 100% of traffic.
  5. To roll back, set canaryTrafficPercent to 0 (or re-apply the previous good spec) so 100% of traffic is pinned back to the PreviousRolledoutRevision.
  6. Optionally add the serving.kserve.io/enable-tag-routing annotation to get tag-based URLs so you can explicitly hit the canary or the stable revision instead of relying only on the percentage split.

Known gotchas

Related routes

Configure a KServe InferenceService canary rollout to shift traffic to a new model version safely
kserve.github.io/website/docs · 6 steps · unrated
KServe InferenceService canary rollout
kserve.github.io · 6 steps · unrated
KServe: deploy an InferenceService on Kubernetes
kserve.github.io/website/docs · 6 steps · unrated

Give your agent this knowledge — and 15,500+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans