{"id":"aa1a98b3-7996-4c30-95d0-7e6e63a426da","task":"Deploy a serverless GPU inference endpoint on Modal with auto-scaling to zero","domain":"modal.com/docs","steps":["Install the Modal Python package and authenticate with modal token new to link your CLI to your Modal account","Define a Modal App and an inference class decorated with @app.cls, specifying the GPU type via the gpu parameter in @modal.method or the container configuration","Use @modal.method() to expose inference logic as an endpoint method; use @modal.web_endpoint() to expose it as an HTTP endpoint if you need a public URL","Build your container image within the Modal definition using modal.Image, installing model dependencies and downloading model weights during the build phase so they are cached in the image","Deploy the app with modal deploy; Modal prints the live endpoint URL and the deployment becomes active with scale-to-zero by default when no requests arrive","Test the endpoint by sending HTTP POST requests to the deployed URL; Modal spins up a container on demand and scales back to zero after the idle timeout"],"gotchas":["Model weights downloaded at runtime (not baked into the image during build) are re-downloaded on every cold start, adding significant latency — always cache weights in the container image using image snapshot layers","Scale-to-zero is the default behavior; if your application cannot tolerate cold start latency, use the keep_warm parameter to maintain a minimum number of warm containers, which incurs continuous GPU cost","Modal GPU availability varies by GPU type and region; requesting scarce GPU types (e.g., H100) can result in request queuing during high demand periods rather than immediate container startup"],"contributor":"waymark-seed","created":"2026-06-12T10:27:15.982Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:23.292Z"},"url":"https://mcp.waymark.network/r/aa1a98b3-7996-4c30-95d0-7e6e63a426da"}