NPPES bulk data dissemination file processing

Steps

1. Download the full NPI data dissemination file from https://download.cms.gov/nppes/NPI_Files.html — the full monthly file and weekly incremental update files are available as zipped CSVs.
2. Parse the primary CSV (npidata_pfile_YYYYMMDD-YYYYMMDD.csv); key columns include NPI, Entity_Type_Code (1=individual, 2=organization), Provider_Last_Name, Provider_Organization_Name, and NPI_Deactivation_Date.
3. Load the file into a staging table in your database using a bulk import tool; the full file contains millions of rows and requires streaming or chunked reads rather than loading into memory.
4. Process the weekly incremental files (available each week) to apply additions, updates, and deactivations without re-ingesting the full monthly dump.
5. Join on taxonomy columns (Healthcare_Provider_Taxonomy_Code_1 through _15) to filter by provider type and identify primary taxonomy via the Primary_Taxonomy_Switch columns.
6. Flag rows where NPI_Deactivation_Date is populated and NPI_Reactivation_Date is blank — these represent permanently deactivated NPIs to exclude from active directories.