Build a price and stock monitoring pipeline with change detection

Steps

1. Define a watchlist of product identifiers (URLs, SKUs, or feed item IDs) and store them with their last-known price and availability in a persistent store (database or KV store).
2. Schedule periodic fetch jobs at an interval appropriate to the merchant's update cadence (e.g., every 15–60 minutes); use conditional GET requests with If-None-Match or If-Modified-Since headers to minimize bandwidth.
3. After each fetch, diff the new price and availability values against the stored snapshot; compute percentage price change and flag transitions in availability (InStock → OutOfStock and vice versa).
4. Emit structured change events (product_id, field_changed, old_value, new_value, detected_at) to a queue or webhook for downstream consumers such as a purchasing agent or notification service.
5. Implement exponential backoff and circuit-breaker logic: if a merchant endpoint returns repeated errors, reduce polling frequency and alert rather than hammering the server.
6. Persist a rolling history of price points with timestamps to support price trend analysis and price-protection claim lookups.