MLflow Deployments Server (AI Gateway): stand up a gateway server to proxy a third-party LLM provider endpoint

Steps

1. Install MLflow with GenAI extras and start the tracking server: pip install 'mlflow[genai]' then mlflow server --port 5000. The AI Gateway is built into the MLflow Tracking Server itself and comes up at http://localhost:5000 (no separate gateway process needed).
2. Note the backend requirement: the gateway needs a SQL-based backend store (SQLite, PostgreSQL, MySQL, or MSSQL) plus the FastAPI tracking server; the default `mlflow server` setup (SQLite + FastAPI) already satisfies this for local use.
3. In the MLflow UI, go to Settings > LLM Connections > Create, name the connection, pick the provider (e.g. OpenAI), and paste that provider's API key — MLflow stores it encrypted server-side.
4. Go to the Endpoints tab > Create Endpoint, give it a name, pick the provider/model (e.g. gpt-4o), and attach the LLM Connection you just created.
5. Query the endpoint either via a plain POST to http://localhost:5000/gateway/<endpoint-name>/mlflow/invocations with a {"messages": [...]} body, or by pointing an OpenAI-SDK client's base_url at http://localhost:5000/gateway/mlflow/v1 (no client-side API key needed since it's configured server-side).
6. Use the gateway's traffic-routing features (documented under Traffic Routing & Fallbacks) to configure A/B splits across models/providers or automatic failover chains for high availability.