Deploy or connect to a REST catalog server implementing the Iceberg REST Catalog specification (e.g., Apache Polaris, Nessie with REST adapter, or a custom server); the server exposes endpoints under /v1/ such as /v1/namespaces and /v1/tables
Configure Spark to use the REST catalog by setting spark.sql.catalog.<name>=org.apache.iceberg.spark.SparkCatalog, spark.sql.catalog.<name>.catalog-impl=org.apache.iceberg.rest.RESTCatalog, spark.sql.catalog.<name>.uri=https://<catalog-host>/v1, and optionally spark.sql.catalog.<name>.oauth2-server-uri and credential properties for authentication
Verify the catalog connection from Spark with SHOW NAMESPACES IN <catalog_name> and CREATE NAMESPACE if needed; create a test table to confirm write-path works end to end
Configure Trino to use the REST catalog by adding a connector properties file (e.g., iceberg-rest.properties) with connector.name=iceberg, iceberg.catalog.type=rest, iceberg.rest-catalog.uri=https://<catalog-host>/v1, and authentication properties; restart the Trino coordinator and workers
Validate cross-engine consistency: write a table from Spark, then immediately query it from Trino; the REST catalog is the single source of truth so both engines should see the same metadata without cache invalidation concerns
Known gotchas
The Iceberg REST spec uses OAuth2 credential exchange; ensure token expiry and refresh logic is handled either by the client library or by providing a long-lived credential — short-lived tokens that expire mid-job cause cryptic 401 errors
REST catalog servers may differ in which optional spec features they implement (e.g., server-side planning, prefix routing, namespace-level properties); test the specific server version against the Iceberg client library version you deploy
If multiple engines write concurrently through the REST catalog, ensure the catalog server uses an atomic compare-and-swap mechanism for metadata file updates; without it you can get silent snapshot corruption under concurrent writers
Give your agent this knowledge — and 200+ more routes
One MCP install gives any agent live access to the full route map, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp