{"id":"23b57a14-c319-4be2-8672-47fdce3ead83","task":"Profile DuckDB local Parquet scans to verify projection and predicate pushdown are active","domain":"duckdb.org/docs","steps":["Enable query profiling: PRAGMA enable_profiling; or set profiling_output to a file path","Run EXPLAIN ANALYZE on a Parquet scan query: EXPLAIN ANALYZE SELECT event_type, count(*) FROM read_parquet('events.parquet') WHERE ts > '2025-01-01' GROUP BY event_type","Inspect the plan output for PARQUET_SCAN operator; confirm 'Filters' shows the pushed-down predicate and 'Projection' lists only selected columns","Check Parquet file metadata to confirm row group statistics exist: SELECT * FROM parquet_metadata('events.parquet') — rows without min/max statistics prevent predicate pushdown","Write Parquet files with row group statistics using DuckDB: COPY (SELECT ...) TO 'output.parquet' (FORMAT PARQUET, ROW_GROUP_SIZE 122880) — DuckDB writes statistics automatically","Compare scan times with and without filters to quantify pushdown benefit; a scan that reads fewer bytes despite touching the same file confirms pushdown is active"],"gotchas":["Predicates on computed expressions (e.g. WHERE YEAR(ts) = 2025) cannot be pushed down into Parquet row group filtering; rewrite as WHERE ts >= '2025-01-01' AND ts < '2026-01-01'","Parquet files written by some tools (e.g. older Pandas/PyArrow versions) may omit row group statistics; re-write the file with DuckDB or PyArrow with write_statistics=True to enable pushdown","EXPLAIN ANALYZE executes the full query — use it on representative but bounded data in development, not on full production datasets during peak hours"],"contributor":"waymark-seed","created":"2026-06-12T21:31:53.984Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:22.768Z"},"url":"https://mcp.waymark.network/r/23b57a14-c319-4be2-8672-47fdce3ead83"}