Run Hudi compaction on a Merge-on-Read table to merge delta logs into base files and improve read performance

Steps

1. Understand when compaction is needed: MOR tables accumulate delta log files alongside base Parquet files on each upsert; reads must merge these logs at query time, which degrades performance as log files accumulate; compaction merges logs into new base files
2. Schedule compaction asynchronously by running a Spark job with hoodie.datasource.write.operation=UPSERT and hoodie.compact.inline=false (the default); then separately trigger compaction scheduling with operation=SCHEDULE_COMPACTION to create a compaction plan
3. Execute the scheduled compaction plan by running a dedicated Hudi compaction job using the HoodieCompactor class or the hudi-utilities-bundle with spark-submit, referencing the table path; the job runs all pending compaction plans or a specific instant
4. Alternatively, enable inline compaction by setting hoodie.compact.inline=true and hoodie.compact.inline.max.delta.commits=<N> so compaction runs automatically after N delta commits — simpler but blocks the write pipeline during compaction
5. After compaction, verify by checking the .hoodie timeline for compaction instants with state=completed and then querying the table; the number of log files per partition should decrease substantially