Split traffic between two Vertex AI Endpoint model deployments to perform a canary rollout

Steps

1. Deploy the production model version to a Vertex AI Endpoint using the gcloud ai endpoints deploy-model command or the SDK, setting an initial traffic split of 100% to the production deployment
2. Deploy the new candidate model version to the same endpoint, specifying a traffic split that allocates the desired canary percentage to the new deployment and the remainder to the production deployment
3. Confirm that all traffic split percentages across all deployed models on the endpoint sum to exactly 100; Vertex AI rejects splits that do not total 100
4. Send prediction requests to the endpoint URL; Vertex AI routes each request to one of the deployed models according to the traffic split percentages
5. Monitor prediction latency, error rates, and business metrics for each deployment ID using Cloud Monitoring to compare canary versus production performance
6. Promote the canary by updating the endpoint traffic split to 100% for the new deployment; remove the old deployment to release resources