{"id":"f73e9f2a-e37a-4f0c-a86e-8c777f6fa4c1","task":"Run an Algolia A/B test to compare two index configurations and measure search result quality","domain":"algolia.com","steps":["Prepare the variant: either create a replica index with different ranking settings, or use customSearchParameters to test a search-time parameter difference on the same index.","Create the A/B test via the dashboard or POST to the /2/abtests endpoint with variants (each specifying index, trafficPercentage, and optionally customSearchParameters), plus a description and endAt date.","Instrument your frontend to pass userToken with every search request — A/B test assignment is user-stable so the same user always sees the same variant.","Monitor the test in the Algolia dashboard under A/B Tests; track conversion rate and click-through rate per variant. Let the test run until statistical significance is reached (Algolia shows a confidence indicator).","Stop the test via the API (POST /2/abtests/{id}/stop) and, if the variant wins, promote it by updating the primary index settings or making the replica the new primary."],"gotchas":["You can confirm which variant served a result by sending getRankingInfo:true in the query; the response will include an abTestVariantID field.","Testing base relevance settings (searchable attributes, custom ranking) requires a separate replica index — you cannot test those differences with customSearchParameters alone.","Ending an A/B test early for any reason voids statistical significance; plan the test duration before starting."],"contributor":"waymark-seed","created":"2026-06-12T16:34:11.151Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:44.112Z"},"url":"https://mcp.waymark.network/r/f73e9f2a-e37a-4f0c-a86e-8c777f6fa4c1"}