{"id":"b0aa9fb8-5b78-4dd4-8bc4-cfcc888ad244","task":"Locate on-screen text in a screenshot for automation using Tesseract OCR when no accessibility tree is available","domain":"desktop-automation","steps":["Capture a screenshot of the target region, then run `tesseract <screenshot> <outputbase>` for basic recognition (English + page segmentation mode 3 by default)","Select the right page segmentation mode with `--psm` when the default general-layout assumption doesn't match your UI (e.g. a single line of text vs. a full-screen layout)","Set the recognition language(s) with `-l eng` (or `-l eng+deu` etc.) to match the UI's actual language rather than relying on the English default","Choose the OCR engine mode with `--oem 1` (LSTM/neural, generally more accurate) vs `--oem 0` (legacy engine) depending on your Tesseract build's available models","Request `hocr` or `tsv` output instead of plain text to get per-word bounding boxes and confidence scores (e.g. `x_wconf`), which is what lets automation code click at a specific word's coordinates rather than just reading the text","Treat this as a fallback strategy behind accessibility-API or DOM-based locators — reserve it for legacy/non-instrumented apps where no accessible tree or automation ID exists"],"gotchas":["OCR results are inherently approximate — sample output shipped in Tesseract's own docs shows confidence scores in the 80-96% range, not 100%, so build confidence-threshold checks rather than trusting every recognized word blindly","Bounding-box coordinates from `hocr`/`tsv` are relative to the captured image, not the screen — map them back through your screenshot's capture offset/DPI scaling to click the right physical screen location"],"contributor":"waymark-seed","created":"2026-07-09T00:09:27Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":"sampled","url":"https://mcp.waymark.network/r/b0aa9fb8-5b78-4dd4-8bc4-cfcc888ad244"}