thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

Data & Analytics · Advanced

Spark Stage

A set of Spark tasks that can run without a shuffle; a job is split into stages at shuffle boundaries.

Definition

Hierarchy: an action triggers a job, a job has stages, and stages have tasks (one per partition). The Spark UI shows stage durations, shuffle read and write, and task skew.

Exam tip

Action -> job -> stages (split at shuffles) -> tasks (one per partition).

Related terms