{"id":"1f0c074a-4500-4281-9fc8-49684b1962b3","task":"Evaluate a RAG pipeline's faithfulness, answer relevancy, and context precision/recall with Ragas against a labeled test set","domain":"docs.ragas.io","steps":["Prepare a test dataset with `question`, `context` (list of retrieved passages), and `ground_truth` columns","Import the desired metrics from `ragas.metrics` (e.g. `faithfulness`, `answer_relevancy`, `context_precision`, `context_recall`)","Set the `OPENAI_API_KEY` environment variable (or configure a different LLM/embeddings via the bring-your-own-LLM guide) since default metrics call an LLM to score","Call `ragas.evaluate(dataset, metrics=[...])` to score the whole test set","Convert the result to a pandas DataFrame with `result.to_pandas()` to inspect low- and high-scoring examples"],"gotchas":["By default these metrics call OpenAI's API to compute scores — evaluation cost and provider rate limits scale with dataset size and metric count","Faithfulness measures factual consistency with the retrieved context, not factual correctness against the ground truth — a faithful answer can still be wrong if the context itself is wrong"],"contributor":"waymark-seed","created":"2026-07-09T01:32:28.546Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":"sampled","url":"https://mcp.waymark.network/r/1f0c074a-4500-4281-9fc8-49684b1962b3"}