thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

Data & Analytics · Intermediate

Auto Loader

A Databricks feature (cloudFiles source) that incrementally and efficiently ingests new files from cloud storage, with schema inference and evolution.

Definition

Auto Loader tracks processed files in its checkpoint (RocksDB) and uses directory listing or file notification mode. It supports schema hints, a rescued data column and schema evolution modes.

Example

spark.readStream.format('cloudFiles').option('cloudFiles.format', 'json').load('/landing/orders')

Exam tip

Incremental file ingestion from cloud storage on Databricks = Auto Loader (cloudFiles).

Related terms