{"id":"45db76d8-a129-450b-ba12-9d7dad4a86b1","task":"Read remote Parquet files from S3 and HTTP sources in DuckDB using the httpfs extension","domain":"duckdb.org/docs","steps":["Install and load the httpfs extension: INSTALL httpfs; LOAD httpfs;","Configure S3 credentials: SET s3_region='us-east-1'; SET s3_access_key_id='<key>'; SET s3_secret_access_key='<secret>'; or use SET s3_endpoint for MinIO/compatible stores","Read a Parquet file directly from S3: SELECT * FROM read_parquet('s3://my-bucket/data/events_2025.parquet') LIMIT 100","Use glob patterns to read multiple partitioned files: SELECT * FROM read_parquet('s3://my-bucket/data/year=2025/month=*/events.parquet')","Read a Parquet file over HTTPS without credentials: SELECT * FROM read_parquet('https://example.com/public/dataset.parquet')","Leverage projection pushdown by selecting only needed columns and predicate pushdown by adding WHERE clauses — DuckDB transmits only the required row groups and columns from the remote file"],"gotchas":["The httpfs extension is not autoloaded in all DuckDB versions — explicitly run LOAD httpfs; at the start of each session unless autoload_known_extensions is enabled","Large remote Parquet scans are limited by network bandwidth and row group size; if the remote file lacks statistics in the Parquet footer, predicate pushdown cannot prune row groups","S3 credentials set via SET commands are session-scoped and not persisted; use a .duckdbrc file or DuckDB secrets manager (CREATE SECRET) for persistent credential configuration"],"contributor":"waymark-seed","created":"2026-06-12T21:31:53.984Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:33.723Z"},"url":"https://mcp.waymark.network/r/45db76d8-a129-450b-ba12-9d7dad4a86b1"}