Steps

Lay out the model repository in S3 using the standard structure: `<model-name>/config.pbtxt` and `<model-name>/<version>/<model-file>` under your bucket path
Start the server pointing at the bucket: `tritonserver --model-repository=s3://bucket/path/to/model/repository`; for a local/private S3-compatible endpoint use `s3://host:port/bucket/path/...` (prefix the host with `https://` inside the URL to force HTTPS)
Provide AWS credentials and default region via `aws configure` or the standard AWS environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`) — environment variables take priority over the CLI-configured credentials
Optionally control where Triton stages its local copy by setting `TRITON_AWS_MOUNT_DIRECTORY` to an existing, empty local directory (by default Triton copies the remote repo into a temp folder that is deleted on shutdown)
For multiple credential sets across buckets/providers, use the beta `TRITON_CLOUD_CREDENTIAL_PATH` JSON credential file instead of environment variables
`--model-repository` can be passed multiple times to combine the S3-backed repo with other local/GCS/Azure repositories on the same server

Known gotchas

Triton doesn't cache cloud-backed repositories across restarts by default — it re-copies the entire repository to local disk (or your configured mount directory) on every startup, and there's no built-in file caching outside of writing a custom repository-agent proxy
If you set `TRITON_AWS_MOUNT_DIRECTORY`, that directory must already exist and be empty or server startup will fail
Credentials resolution order matters: environment variables override `aws config`-sourced credentials, so mismatched env vars in a container can silently point Triton at the wrong account/region

docs.nvidia.com · 6 steps · unrated

NVIDIA Triton Inference Server: set up a model repository and serve

docs.nvidia.com/deeplearning/triton-inference-server · 6 steps · unrated

configure nvidia triton inference server explicit model control mode for load/unload via api

docs.nvidia.com/deeplearning/triton-inference-server · 5 steps · unrated

Give your agent this knowledge — and 15,500+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

NVIDIA Triton Inference Server: configure a model repository backed by Amazon S3 instead of local disk

Steps

Known gotchas

Related routes

Give your agent this knowledge — and 15,500+ more routes

Need this verified for your stack — or a route we don't have yet?