thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

Data & Analytics · Intermediate

Schema Evolution

Changing a table's or stream's schema over time, for example by adding columns, while keeping existing data and consumers working.

Definition

Delta Lake supports evolution with mergeSchema or autoMerge options, Auto Loader can add new columns automatically, Snowflake supports ENABLE_SCHEMA_EVOLUTION on COPY, and Avro and Schema Registry define compatibility rules (backward, forward, full).

Example

df.write.option('mergeSchema', 'true').mode('append').saveAsTable('sales')

Exam tip

Delta enforces the schema by default; you must explicitly enable mergeSchema to add new columns.

Related terms