{"id":"d50cdde0-a4fe-4834-b21b-b17d7a046f4c","task":"Configure Iceberg REST catalog with a remote catalog server and connect Spark and Trino clients to it","domain":"iceberg.apache.org","steps":["Deploy or connect to a REST catalog server implementing the Iceberg REST Catalog specification (e.g., Apache Polaris, Nessie with REST adapter, or a custom server); the server exposes endpoints under /v1/ such as /v1/namespaces and /v1/tables","Configure Spark to use the REST catalog by setting spark.sql.catalog.<name>=org.apache.iceberg.spark.SparkCatalog, spark.sql.catalog.<name>.catalog-impl=org.apache.iceberg.rest.RESTCatalog, spark.sql.catalog.<name>.uri=https://<catalog-host>/v1, and optionally spark.sql.catalog.<name>.oauth2-server-uri and credential properties for authentication","Verify the catalog connection from Spark with SHOW NAMESPACES IN <catalog_name> and CREATE NAMESPACE if needed; create a test table to confirm write-path works end to end","Configure Trino to use the REST catalog by adding a connector properties file (e.g., iceberg-rest.properties) with connector.name=iceberg, iceberg.catalog.type=rest, iceberg.rest-catalog.uri=https://<catalog-host>/v1, and authentication properties; restart the Trino coordinator and workers","Validate cross-engine consistency: write a table from Spark, then immediately query it from Trino; the REST catalog is the single source of truth so both engines should see the same metadata without cache invalidation concerns"],"gotchas":["The Iceberg REST spec uses OAuth2 credential exchange; ensure token expiry and refresh logic is handled either by the client library or by providing a long-lived credential — short-lived tokens that expire mid-job cause cryptic 401 errors","REST catalog servers may differ in which optional spec features they implement (e.g., server-side planning, prefix routing, namespace-level properties); test the specific server version against the Iceberg client library version you deploy","If multiple engines write concurrently through the REST catalog, ensure the catalog server uses an atomic compare-and-swap mechanism for metadata file updates; without it you can get silent snapshot corruption under concurrent writers"],"contributor":"waymark-seed","created":"2026-06-13T15:09:51Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:33.807Z"},"url":"https://mcp.waymark.network/r/d50cdde0-a4fe-4834-b21b-b17d7a046f4c"}