Deploy a custom model on Replicate and expose it as a production API deployment with auto-scaling

domain: replicate.com/docs · 6 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Package your model using Cog (Replicate's containerization tool): define predictor.py with a Predict class implementing predict(), and a cog.yaml specifying the base image and Python dependencies
  2. Build and push the Cog model to Replicate with cog push, which creates a new model version on the platform
  3. Create a Deployment using the POST /v1/deployments API endpoint or the Replicate dashboard, specifying the model, version, hardware type, min_instances, and max_instances for auto-scaling
  4. The deployment provides a dedicated URL distinct from the shared model endpoint; reference the deployment name in API calls rather than the model version directly
  5. Invoke the deployment via POST /v1/predictions, setting the version field to the deployment's model version; the deployment auto-scales between min and max instances based on queue depth
  6. Monitor deployment metrics (request volume, latency, instance utilization, error rates) from the Replicate dashboard and adjust min/max instance counts as traffic patterns evolve

Known gotchas

Related routes

Replicate: run a model via the API
replicate.com/docs · 6 steps · unrated
Ray Serve: configure autoscaling for a deployment (min_replicas, max_replicas, target_ongoing_requests)
ml-ops · 5 steps · unrated
KServe: deploy a model as an InferenceService with autoscaling on Kubernetes
ml-ops · 5 steps · unrated

Give your agent this knowledge — and 15,500+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans