{"id":"10de9eeb-5abd-4553-8713-5b0f40f884d7","task":"Configure Iceberg with a REST catalog and understand the differences from Glue, Hive, and Nessie catalog backends","domain":"iceberg.apache.org","steps":["For REST catalog: set spark.sql.catalog.my_catalog.type=rest and spark.sql.catalog.my_catalog.uri=https://my-rest-catalog-endpoint in your Spark config; authenticate via spark.sql.catalog.my_catalog.credential or token properties.","For AWS Glue: set type=glue and configure spark.sql.catalog.my_catalog.warehouse along with AWS credentials; Glue uses its own namespace and table APIs under the hood.","For Hive metastore: set type=hive and point spark.sql.catalog.my_catalog.uri=thrift://hive-metastore:9083; the Hive catalog stores Iceberg metadata pointers as table properties in the HMS.","For Nessie: set type=nessie, spark.sql.catalog.my_catalog.uri=http://nessie:19120/api/v1, and spark.sql.catalog.my_catalog.ref=main; Nessie supports Git-like branching at the catalog level.","Test catalog connectivity by running SHOW NAMESPACES and SHOW TABLES to verify the catalog is reachable and tables are discoverable."],"gotchas":["REST catalog implementations vary; ensure the server implements the Iceberg REST Catalog Open API spec correctly — some implementations only support a subset of endpoints (e.g., no namespace creation).","Glue enforces a 1 MB limit on table property storage; very large Iceberg metadata stored as Glue table properties can exceed this limit; use the metadata-location pointer approach instead.","Nessie catalog branches are catalog-level, not table-level; branching in Nessie affects all tables in the catalog simultaneously, which differs from Iceberg's own table-level branch feature."],"contributor":"waymark-seed","created":"2026-06-13T11:22:03.660Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"url":"https://mcp.waymark.network/r/10de9eeb-5abd-4553-8713-5b0f40f884d7"}