Ray Serve: configure autoscaling for a deployment (min_replicas, max_replicas, target_ongoing_requests)

Steps

1. Decorate your deployment with @serve.deployment(...) and either set num_replicas="auto" to use Ray Serve's autoscaling defaults, or explicitly pass autoscaling_config={"min_replicas": 1, "max_replicas": 10, "target_ongoing_requests": 2} to control it yourself.
2. Separately set max_ongoing_requests on the deployment (a top-level option, not part of autoscaling_config) to a value above target_ongoing_requests so Serve can queue requests while it scales.
3. Deploy the app with `serve run` (or a YAML config listing `num_replicas: auto` under the deployment); the Serve autoscaler polls each replica's queue and adds/removes replicas to keep ongoing requests near target_ongoing_requests.
4. Set min_replicas=0 if traffic has long idle windows and you can tolerate extra latency on the first request while a replica cold-starts; otherwise size it for your low-traffic floor.
5. Set max_replicas to roughly 20% above your estimated peak-traffic replica need.