thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

Data & Analytics · Advanced

Watermark (Streaming)

A threshold in stream processing that estimates how far event time has progressed, determining when windows can close and how late data is handled.

Definition

In Spark, withWatermark('eventTime', '10 minutes') lets state be dropped for data older than the threshold. Beam and Dataflow use watermarks plus triggers and allowed lateness.

Example

df.withWatermark('ts', '10 minutes').groupBy(window('ts', '5 minutes')).count()

Exam tip

Watermarks bound streaming state; without one, stateful aggregations grow without limit.

Related terms