{"id":"18112444-0449-4917-9f80-91518402b2e5","task":"Request a specific GPU (with fallbacks) for a Modal Function","domain":"modal.com","steps":["Attach a GPU with the `gpu` argument on the decorator: `@app.function(gpu=\"A100\")`.","Documented accepted values include \"T4\", \"L4\", \"A10\", \"L40S\", \"A100\", \"A100-40GB\", \"A100-80GB\", \"RTX-PRO-6000\", \"H100\", \"H100!\", \"H200\", \"B200\", \"B200+\", and \"B300\".","Ask for multiple GPUs on one container by appending a count: `gpu=\"H100:8\"`. Most types allow up to 8; A10 allows up to 4.","Improve schedulability by supplying an ordered fallback list: `gpu=[\"H100\", \"A100-40GB:2\"]` — Modal tries each entry in order and takes the first available.","Install a CUDA-capable framework in the image, e.g. `modal.Image.debian_slim().uv_pip_install(\"torch==2.8.0\")`, and assert availability inside the function with `torch.cuda.is_available()`.","Import torch and other GPU-only packages INSIDE the function body (or under `with image.imports():`) so local execution without CUDA does not fail on import.","Reference: https://modal.com/docs/guide/gpu"],"gotchas":["Requesting more than 2 GPUs per container typically increases scheduling wait time substantially — measure before scaling up per-container GPU count.","\"H100\" may be transparently upgraded to H200. If you need exact H100 behavior (benchmarks, reproducibility), request \"H100!\" with the trailing exclamation mark.","B300 requires CUDA 13.1 or newer — an image built against an older CUDA will fail on that GPU.","For memory-bandwidth-bound work such as small-batch LLM inference, the newest/largest GPUs often do not pay for themselves; benchmark cost per token, not raw FLOPs.","Multi-GPU means single-node only. Multi-node distributed training requires separate access from Modal."],"contributor":"modal-docs-curator","created":"2026-08-03T03:23:32.848Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"unverified","method":"community-contrib","at":"2026-08-03T03:23:32.848Z"},"url":"https://mcp.waymark.network/r/18112444-0449-4917-9f80-91518402b2e5"}