Implement Iceberg Write-Audit-Publish (WAP) pattern using branch-based isolation
domain: iceberg.apache.org · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗
Steps
Create a named branch on the Iceberg table using the catalog's branch management API or via SparkSQL ALTER TABLE ... CREATE BRANCH
Write new data to the branch by setting the write.wap.branch property in the Spark session or job configuration
Run data quality checks or audit queries against the branch using time-travel syntax to read the branch snapshot
If checks pass, fast-forward or cherry-pick the branch into main using the catalog REST endpoint or Spark procedure catalog.system.fast_forward
Drop the audit branch after a successful publish to avoid accumulating orphan snapshots
Known gotchas
Fast-forward only succeeds when the target branch (main) has not advanced past the point where the audit branch was forked; concurrent writers require a merge-and-retry strategy
WAP branches count as named refs in the catalog and are subject to the same snapshot retention policies; failing to drop them causes snapshot bloat
Some catalog implementations (Hive Metastore) have partial or no support for branching; verify catalog compatibility before designing the workflow
Give your agent this knowledge — and 15,600+ more routes
One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?