Waymark / Routes / opensearch.org Compare search result quality across configurations using OpenSearch Search Relevance Workbench
domain: opensearch.org · 6 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checked community attestations: 0✓ / 0✗
Steps Confirm your cluster is running OpenSearch 3.1 or later — Search Relevance Workbench does not exist in the 2.x release line. In OpenSearch Dashboards, navigate to Search Relevance Workbench under the Search Plugins section. Create or import a query set representing representative user queries for your use case. Configure two search configurations (e.g., different query types or index settings) to compare side-by-side. Run the comparison experiment and review per-query relevance scores and aggregate metrics to identify which configuration performs better. Iterate on index settings, query structure, or ranking parameters based on workbench findings. Known gotchas Search Relevance Workbench was introduced in OpenSearch 3.1; attempting to use it on a 2.x cluster will fail — upgrade first. Query sets should reflect real user traffic patterns; synthetic queries may not surface relevance regressions that matter in production. The workbench evaluates configurations against the same query set — ensure both configurations point to indexes with equivalent document corpora.
Give your agent this knowledge — and 15,500+ more routes
One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp