Compare Apache Hudi and Apache Iceberg table service operations (compaction, cleaning, clustering) and select the right tradeoffs

Steps

1. Hudi compaction (MOR tables): the compaction service merges delta log files into columnar base files on a configurable schedule (inline or async); tune hoodie.compact.inline.max.delta.commits to control how often compaction triggers relative to ingestion commits.
2. Iceberg compaction (rewrite_data_files): Iceberg has no built-in background service; compaction is an explicit user-triggered action via the rewrite_data_files procedure; it is better suited for batch-schedule maintenance jobs than continuous streaming ingestion.
3. Hudi cleaning: the cleaner service deletes old file versions based on a retention policy (number of commits or time); configure hoodie.cleaner.policy and hoodie.cleaner.commits.retained to bound storage growth.
4. Iceberg cleanup: use expire_snapshots and remove_orphan_files as separate steps; there is no unified cleaning service, giving more explicit control but requiring more operational orchestration.
5. Hudi clustering: reorganizes data within partitions by a sort key for query performance, similar to Iceberg's sort-order compaction; set hoodie.clustering.inline or run async clustering via a Spark job.
6. Choose Hudi for high-frequency upsert workloads where continuous background services reduce operational burden; choose Iceberg for batch-oriented workloads, polyglot engine support, or when tight control over maintenance timing is needed.