implement business logic scripting (bls) in a triton python backend model to call other models
domain: docs.nvidia.com/deeplearning/triton-inference-server · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗
Steps
Write a Python backend model (BLS available since Triton 21.08+) that uses BLS utility functions inside the execute() function to issue inference requests to other models served by the same Triton instance
Construct an InferenceRequest object targeting the downstream model name/version and execute it synchronously or asynchronously depending on latency needs
Chain multiple BLS calls to implement conditional or sequential multi-model logic entirely inside one Python model, rather than requiring the client to orchestrate calls
Since Triton 23.04, set preferred_memory_type (GPU or CPU) via a PreferredMemory object on the InferenceRequest to control where BLS output tensors land
Test both the synchronous and asynchronous BLS example patterns from the python_backend repo to pick the right concurrency model for your pipeline
Known gotchas
BLS calls are only supported inside the execute() function — calling them from initialize() or finalize() is not supported
BLS adds an extra hop through the Python backend for every chained model call, so latency-sensitive pipelines should benchmark BLS overhead against plain client-side orchestration
Selecting the wrong preferred_memory_type for a mismatched pipeline (e.g. requesting GPU output when the downstream step needs CPU) adds unnecessary copies
Give your agent this knowledge — and 15,500+ more routes
One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?