Set Delta Lake VACUUM retention period and safely reclaim storage from old data files
domain: docs.delta.io · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗
Steps
Check the current retention threshold with SHOW TBLPROPERTIES delta.`/path/to/table` and look for delta.deletedFileRetentionDuration (default 7 days / interval 168 hours).
Set a custom retention with ALTER TABLE delta.`/path/to/table` SET TBLPROPERTIES ('delta.deletedFileRetentionDuration' = 'interval 14 days').
Before running VACUUM, optionally do a dry run to see what would be deleted: VACUUM delta.`/path/to/table` RETAIN 168 HOURS DRY RUN.
Execute VACUUM: VACUUM delta.`/path/to/table` RETAIN 168 HOURS; confirm the files deleted and total bytes reclaimed from the output.
If you must drop retention below 7 days (not recommended), first set spark.databricks.delta.retentionDurationCheck.enabled = false or the equivalent Delta OSS config to bypass the safety check.
Known gotchas
Running VACUUM with a retention period shorter than the longest active reader transaction will cause those readers to fail mid-query because the underlying files will be deleted.
VACUUM does not respect time-travel or snapshot history beyond the retention window; once files are vacuumed, time-travel queries referencing those snapshots will fail.
Delta Lake on OSS Spark requires delta.retentionDurationCheck.enabled=false to lower retention below 168 hours; this safety check exists specifically to prevent accidental data loss.
Give your agent this knowledge — and 15,600+ more routes
One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?