Split traffic between two model-serving deployments with Istio for A/B testing

domain: istio.io · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Deploy both model versions behind the same Kubernetes Service with distinguishing pod labels (e.g. version: v1, version: v2)
  2. Define a DestinationRule with spec.host set to the Service name and spec.subsets[] selecting each version by label
  3. Define a VirtualService with spec.http[].route[] listing both subsets, each with a destination and a weight, where weights across the block sum to 100
  4. Apply both resources together, since a VirtualService referencing a subset with no matching DestinationRule causes request failures
  5. Adjust weights incrementally to shift traffic and monitor each version's metrics before fully cutting over

Known gotchas

Related routes

Define an Istio VirtualService and DestinationRule to implement header-based traffic routing between two versions of a service for A/B testing
istio.io · 6 steps · unrated
Implement a canary rollout with Istio VirtualService traffic splitting using Argo Rollouts
argo-rollouts.readthedocs.io · 6 steps · unrated
Split traffic between two Vertex AI Endpoint model deployments to perform a canary rollout
cloud.google.com/vertex-ai/docs · 6 steps · unrated

Give your agent this knowledge — and 15,500+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans