Delta Lake enforces schema by default; attempt to write a DataFrame with a mismatched column to observe the AnalysisException and understand the enforcement boundary.
To add new columns from an incoming DataFrame, enable schema evolution with df.write.format('delta').option('mergeSchema', 'true').mode('append').save('/path/to/table').
For DDL-driven evolution, use ALTER TABLE delta.`/path/to/table` ADD COLUMNS (new_col STRING) to add columns explicitly without a write operation.
Set the table-level property delta.columnMapping.mode=name if you need to rename or drop columns without rewriting data (requires Delta protocol version 2).
Validate the resulting schema with DESCRIBE delta.`/path/to/table` or spark.read.format('delta').load('/path/to/table').schema to confirm the evolution applied correctly.
Known gotchas
mergeSchema only handles additive changes (new columns, widened types in some cases); it does not allow dropping columns or changing column types incompatibly — those require overwriteSchema=true which rewrites the table.
Column mapping mode (rename/drop support) requires upgrading the Delta protocol version, which is a one-way operation; older Delta readers that do not support the new protocol version will be unable to read the table.
Automatic schema evolution via spark.databricks.delta.schema.autoMerge.enabled applies globally in a session; prefer per-write mergeSchema options to avoid unintentional schema changes from unexpected source data.
Give your agent this knowledge — and 200+ more routes
One MCP install gives any agent live access to the full route map, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp