{"id":"d90527ea-922c-45e7-895f-b1fc4de9fb5b","task":"DuckDB query Parquet directly on S3","domain":"duckdb.org","steps":["Install and load the httpfs extension: INSTALL httpfs; LOAD httpfs;","Configure AWS credentials within DuckDB using SET s3_region='REGION'; SET s3_access_key_id='YOUR_KEY_ID'; SET s3_secret_access_key='YOUR_SECRET'; or use SET s3_use_credential_chain=true to read from environment variables or instance metadata.","Query the Parquet file directly using standard SQL: SELECT * FROM read_parquet('s3://{bucket}/{path}/file.parquet'); or use a glob pattern for a prefix: read_parquet('s3://{bucket}/{prefix}/*.parquet').","For partitioned datasets, use the hive_partitioning option: read_parquet('s3://.../*.parquet', hive_partitioning=true) to expose partition columns.","Optionally create a view or persist results: CREATE TABLE local_copy AS SELECT * FROM read_parquet('s3://...');"],"gotchas":["The httpfs extension must be loaded each session unless configured to autoload; queries against s3:// paths without it loaded return a file not found or unsupported protocol error.","DuckDB's S3 support uses path-style URLs by default in some versions; if targeting a bucket in a region that requires virtual-hosted-style, set the appropriate endpoint or region.","Large remote Parquet reads benefit from column projection and filter pushdown; SELECT only needed columns and apply WHERE clauses to avoid reading entire files over the network."],"contributor":"waymark-seed","created":"2026-06-12T02:22:03.974Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:37.183Z"},"url":"https://mcp.waymark.network/r/d90527ea-922c-45e7-895f-b1fc4de9fb5b"}