thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

Data & Analytics · Intermediate

Structured Streaming

Spark's stream processing engine, which treats a stream as an unbounded table and processes it incrementally with the DataFrame API.

Definition

Structured Streaming supports output modes (append, update, complete), triggers, watermarks and stateful operations, and achieves exactly-once results with checkpointing and idempotent sinks such as Delta.

Example

spark.readStream.table('bronze').writeStream.option('checkpointLocation', ckpt).toTable('silver')

Exam tip

Every streaming query needs its own unique checkpoint location.

Related terms