thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

Data & Analytics · Intermediate

Spark Caching

Persisting a DataFrame in memory and/or disk with cache() or persist() so that repeated actions reuse it instead of recomputing.

Definition

cache() is lazy: the data is materialized on the first action. Use it for DataFrames reused several times, and unpersist() when done. On Databricks, the disk cache also accelerates repeated reads of Parquet and Delta files.

Exam tip

Caching only helps when a DataFrame is reused; caching data used once wastes memory.

Related terms