Extract structured product data from a product detail page (PDP) without an official API

Steps

1. Request the PDP with a well-identified User-Agent and check for a JSON-LD or Open Graph data layer first, as these are cheaper to parse than full DOM traversal.
2. If no structured markup is present, parse the HTML DOM to locate the price element (commonly identified by CSS classes, data attributes, or itemprop values), title (h1 or product title element), and availability indicator.
3. For variant-based products (size, color, etc.), identify the variant selector elements; note that selected-variant data is often injected via JavaScript into a window.__INITIAL_STATE__ or similar global variable — extract this JSON blob from the script tags.
4. Normalize extracted price strings: strip currency symbols, handle locale-specific thousands separators and decimal points, and pair with a currency code inferred from the page locale or URL TLD.
5. Validate the extracted data for internal consistency: price should be a positive number, availability should map to a known state, and the product title should match the URL slug or breadcrumb to detect off-target extraction.
6. Version your extraction selectors: when a merchant redesigns their PDP, selectors break silently; add a freshness check that flags extractions that produce suspiciously empty or null fields for manual review.