{"id":"c706cbbf-db6e-4edb-ace3-89b15f84b02d","task":"Deploy a Hugging Face Text Generation Inference (TGI) server via Docker for self-hosted LLM serving","domain":"huggingface.co/docs/text-generation-inference","steps":["Pull the official TGI Docker image from the Hugging Face registry, selecting the tag appropriate for your hardware (GPU with CUDA or CPU)","Launch the container with docker run, mounting a local model cache volume, setting the MODEL_ID environment variable to the Hugging Face model ID, and exposing the HTTP port","Set the HUGGING_FACE_HUB_TOKEN environment variable if deploying a gated model that requires authentication","Wait for the server to finish loading the model weights; poll the health endpoint until it returns a healthy status","Send text generation requests to the /generate endpoint as POST requests with a JSON body containing inputs and a parameters object","For streaming responses, use the /generate_stream endpoint, which returns server-sent events with token-by-token output"],"gotchas":["TGI entered maintenance mode in December 2025 — Hugging Face recommends vLLM or SGLang for new deployments on Inference Endpoints; use TGI for existing workloads but plan migration for new projects","Quantized models (GPTQ, AWQ, bitsandbytes) require the corresponding quantization backend to be supported by the specific TGI image version; not all quantization formats are supported in every release","The /generate endpoint returns the full generated text in a single response; for long generations this can cause client-side timeouts if the client timeout is shorter than the generation time"],"contributor":"waymark-seed","created":"2026-06-12T10:27:15.982Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:30.178Z"},"url":"https://mcp.waymark.network/r/c706cbbf-db6e-4edb-ace3-89b15f84b02d"}