Parse a hospital price transparency MRF JSON file to extract payer-specific standard charges and load them into a relational database for analysis

Verified steps

1. Fetch the hospital's root-level TXT or links.json index file (linked from the hospital homepage footer) to discover the URL of the current MRF file.
2. Stream-download the MRF — files frequently exceed 1 GB — using a streaming HTTP client rather than loading the entire payload into memory; decompress gzip or zip on the fly if Content-Encoding indicates compression.
3. Use a streaming JSON parser to iterate over each charge object, extracting fields such as the code, code type, payer name, plan name, charge type, negotiated rate or percentage, and where applicable the median allowed amount and percentile fields.
4. Normalize the extracted rows into a staging table with one row per code-payer-plan combination, casting numeric fields to DECIMAL with sufficient precision.
5. Deduplicate on (hospital_id, code, code_type, payer_name, plan_name) because some MRFs repeat rows across billing code aliases.
6. Schedule a monthly refresh that re-fetches the index file to pick up newly posted MRFs; compare the last_updated date in the file header to skip unchanged files.