thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

Data & Analytics · Intermediate

Duplicate Record Detection

Identifying records that represent the same entity, through exact key matches or fuzzy matching of attributes.

Definition

Exact duplicates are found with GROUP BY or ROW_NUMBER. Near-duplicates need fuzzy matching (edit distance, phonetic algorithms) and survivorship rules to choose the best values.

Example

SELECT email, COUNT(*) FROM customers GROUP BY email HAVING COUNT(*) > 1;

Exam tip

GROUP BY ... HAVING COUNT(*) > 1 is the standard SQL duplicate check.

Related terms