Paginate through College Scorecard API results using page and per_page parameters

Steps

1. Register for a free API key at https://api.data.gov/signup/ and use it as the api_key query parameter in all requests to https://api.data.gov/ed/collegescorecard/v1/schools.
2. Add page (0-indexed integer, default 0) and per_page (integer 1–100, default 20) parameters to your request URL to control pagination; example: ?api_key=YOUR_KEY&per_page=100&page=0.
3. Parse the metadata object in the JSON response; it contains total (total matching records), page (current page number), and per_page (records per page) — use metadata.total and per_page to calculate the number of pages needed: ceil(total / per_page).
4. Iterate from page=0 up to the last page, incrementing by 1 each request; stop when the results array is empty or when you have fetched ceil(total / per_page) pages.
5. Combine the results arrays from all pages into a single dataset; each element represents one school with the fields you specified using the fields parameter.
6. Cache paginated results where possible; the dataset is large (thousands of schools) and re-fetching all pages on every run is slow — store locally and refresh periodically.