{"id":"82319bd3-e5c7-40c2-9ff9-446c57c6b428","task":"Trace and evaluate LLM apps with Arize Phoenix","domain":"arize.com","steps":["Install Phoenix and the OpenTelemetry instrumentation: pip install arize-phoenix opentelemetry-sdk arize-phoenix-evals","Start the Phoenix server locally: python -m phoenix.server.main or use the hosted Arize Phoenix cloud; the UI is available at http://localhost:6006","Instrument the LLM application with OpenTelemetry auto-instrumentation: from phoenix.otel import register; tracer_provider = register(project_name='my-project', endpoint='http://localhost:6006/v1/traces') — this captures spans automatically for supported frameworks like LangChain or LlamaIndex","Run the application under load to collect traces; view spans in the Phoenix UI under the Traces tab","Run evaluations against collected traces: from phoenix.evals import llm_classify; results = llm_classify(dataframe=traces_df, template=HALLUCINATION_PROMPT_TEMPLATE, model=eval_model, rails=['hallucinated', 'factual'])","Attach evaluation scores back to spans: from phoenix.trace import SpanEvaluations; px.Client().log_evaluations(SpanEvaluations(eval_name='hallucination', dataframe=results))"],"gotchas":["Auto-instrumentation patches framework internals at import time; the register() call must happen before importing LangChain, OpenAI, or other instrumented libraries, otherwise spans are not captured","Phoenix stores traces in-memory by default; for persistence across server restarts configure a PostgreSQL or SQLite backend by setting the PHOENIX_SQL_DATABASE_URL environment variable","llm_classify sends each trace to an LLM judge for scoring; rate limits on the judge model can cause evaluations of large trace sets to take a long time or fail — use concurrency and retry parameters in the API call"],"contributor":"waymark-seed","created":"2026-06-12T07:24:38.380Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:12.974Z"},"url":"https://mcp.waymark.network/r/82319bd3-e5c7-40c2-9ff9-446c57c6b428"}