---
title: Deduplication
url: http://0.0.0.0:4000/master/deduplication
content_type: landing
section: Master
---

# Deduplication

The goal of deduplication is to eliminate duplicate records by merging them together into a single, accurate, and consolidated golden record. This process maintains full traceability, allowing you to identify contributing records for the resulting golden record and providing the possibility to revert changes if necessary.

> **Important:** You can reduce the number of duplicates in the system proactively even before creating a deduplication project. For this purpose, CluedIn provides the possibility of merging by identifiers: those [data parts](http://0.0.0.0:4000/get-started/core-concepts/data-life-cycle) that have identical primary identifiers or additional identifiers are merged during processing. For more information, see [Identifiers](http://0.0.0.0:4000/ingest/mapping/review-mapping#identifiers).

The following diagram shows the basic steps for merging duplicates in CluedIn.

![How to merge duplicates in CluedIn: 1 configure matching rules, 2 review groups of duplicates, 3 merge duplicates, 4 reevaluate merged records, 5 optionally revert merges](http://0.0.0.0:4000/assets/images/management/deduplication/dedup-main.gif)

This section covers the following areas:

- [Concept of deduplication](http://0.0.0.0:4000/master/deduplication/concept-of-deduplication) – explore how deduplication works through a simple yet detailed example.

- [Deduplication in practice](http://0.0.0.0:4000/master/deduplication/deduplication-in-practice) – discover the practical application of deduplication guidelines through an example.

- [Creating a deduplication project](http://0.0.0.0:4000/master/deduplication/create-a-deduplication-project) – learn how to create a deduplication project and configure matching rules.

- [Managing a deduplication project](http://0.0.0.0:4000/master/deduplication/manage-a-deduplication-project) – learn how to effectively manage a deduplication project, including generating and discarding results, editing project configuration, and more.

- [Managing groups of duplicates](http://0.0.0.0:4000/master/deduplication/manage-groups-of-duplicates) – learn how to process groups of duplicates, including merging duplicates, verifying merged records, and reverting merges.

- [Reference information about deduplication projects](http://0.0.0.0:4000/master/deduplication/deduplication-reference) – find information about matching functions, rule conditions, deduplication project statuses, and group statuses.

- [Matching one of multiple fields](http://0.0.0.0:4000/master/deduplication/match-one-of-multiple-fields) – learn how to identify duplicates when any value from a group of fields matches across records.