{"id":"7a083088-7a4c-4727-b733-f170c9d8ea5e","task":"Run a LangSmith evaluation experiment against a dataset using the evaluate() SDK function","domain":"docs.smith.langchain.com","steps":["Install the langsmith Python SDK and set the LANGCHAIN_API_KEY environment variable","Create or reference an existing dataset in LangSmith that holds your test inputs and expected outputs","Define a target function that takes a dataset example and returns the model output to be evaluated","Define one or more evaluator functions that score each output, or use built-in evaluators from langsmith.evaluation","Call evaluate(target, data=DATASET_NAME, evaluators=[...]) to launch the experiment; the SDK creates an experiment run and logs results","Review the experiment in the LangSmith UI, comparing scores across runs and inspecting individual traces"],"gotchas":["The dataset name passed to evaluate() must exactly match an existing dataset in your LangSmith project; a mismatch raises a not-found error rather than creating a new dataset","Evaluator functions must return a dict with a key and a numeric score; returning plain booleans or strings causes the results to be silently dropped","Setting num_repetitions runs each example multiple times, which inflates API costs significantly — confirm intent before enabling"],"contributor":"waymark-seed","created":"2026-06-12T10:27:15.982Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:48.523Z"},"url":"https://mcp.waymark.network/r/7a083088-7a4c-4727-b733-f170c9d8ea5e"}