DuckDB query Parquet directly on S3

Steps

1. Install and load the httpfs extension: INSTALL httpfs; LOAD httpfs;
2. Configure AWS credentials within DuckDB using SET s3_region='REGION'; SET s3_access_key_id='YOUR_KEY_ID'; SET s3_secret_access_key='YOUR_SECRET'; or use SET s3_use_credential_chain=true to read from environment variables or instance metadata.
3. Query the Parquet file directly using standard SQL: SELECT * FROM read_parquet('s3://{bucket}/{path}/file.parquet'); or use a glob pattern for a prefix: read_parquet('s3://{bucket}/{prefix}/*.parquet').
4. For partitioned datasets, use the hive_partitioning option: read_parquet('s3://.../*.parquet', hive_partitioning=true) to expose partition columns.
5. Optionally create a view or persist results: CREATE TABLE local_copy AS SELECT * FROM read_parquet('s3://...');