Customer experience | Sales, Customer Insights,...

Clean Duplicates

(2) Share

Report

Posted on by Crownd

Hi there!

What is the best way to clean lots of duplicate Contact/Lead entries in Dynamics 365?
I can easily detect all the duplicate entries when exporting all leads/contacts into an (online) excel file. I created a temporary boolean field "isDuplicate" to flag all records that were duplicates.
However, when trying to import them back into dynamics i can not update the records as i throws me an error for wanting to import duplicates...

Is there any better way to clean/delete multiple duplicate records?

I also tried using OOTB function of dynamics to detect duplicates, however, this only allows to merge/delete records one at a time which would take for ever in our case.
(Moreover, the Interface for this funcitonality looks like it is from 1995...)

Best Regards
Matthias

I have the same question (0)

All responses (6)

Answers (0)

Inogic 639 on at

Like (1)

Report
Clean Duplicates

Hi Matthias,

You're encountering a common limitation with Dynamics 365's out-of-the-box (OOTB) duplicate detection and data import functions. While exporting Contacts/Leads to Excel helps in identifying duplicates (e.g., via a custom "isDuplicate" flag), re-importing them for update typically fails due to CRM’s enforced duplicate detection and record ownership rules. Additionally, the native duplicate detection tool only supports single-record merge/delete actions, making it impractical for bulk cleanup.
When addressing duplicate record issues in Dynamics 365 CRM, the following functionalities are essential:

Rule-Based Matching: Supports exact and fuzzy logic across multiple fields (e.g., Email, Full Name, Phone).

Bulk Processing: Detects and processes thousands of duplicate records in structured batches.

Automated Merge/Delete: Executes bulk merge/delete operations with intelligent merge logic.

No Excel Import Needed: Works within CRM—no need for Excel export/import cycles, avoiding import errors.

Field-Level Merge Control: Allows precise selection of field values to retain in the master record during merges, ensuring data accuracy and integrity in high-volume Dynamics 365 CRM environments.

Audit & Rollback: Maintains action history with rollback options for data safety.

Modern UI: Provides an intuitive, performance-optimized interface for faster deduplication workflows.

All these capabilities are built into the DeDupeD app, which is highly preferred to address the duplicate record challenges commonly faced by Dynamics 365 CRM users dealing with large volumes of data. Unlike the limited out-of-the-box features, DeDupeD provides a scalable, enterprise-grade, and CRM-native solution for efficiently identifying and resolving duplicate Contacts and Leads—ensuring clean, consistent, and reliable data across your CRM.

Thanks!

Was this reply helpful? Yes No
Suggested answer

Pradeep Rai 5,489 Moderator on at

Like (1)

Report

RE: Clean Duplicates

If you enable the allow duplicates while importing then it will create duplicate records.

As naveen said, you can use power automate to delete records.

Which means, During import allow duplicates and after import complete you can run the power automate to delete the record.

First try on existing record once it works then import and run the power automate.

Instead delete you can try the Merge Request class as described in below link:
https://sureshmaurya.me/2021/06/13/record-merging-and-merge-tracking-in-depth-in-microsoft-dynamics-365/

Was this reply helpful? Yes No
Crownd 19 on at

Like (1)

Report

RE: Clean Duplicates

I'll attach the screenshot to this message. It is in German unfortunately, but it essentially says that the lead could not be imported/updated since there exists a duplicate.

However, what would happen if I allow duplicates while importing, I would not want to create even more dups in my system?

Best Regards

Matthias

Was this reply helpful? Yes No
Naveen Ganeshe 3,397 User Group Leader on at

Like (1)

Report

RE: Clean Duplicates

can you share the screenshot of the error while importing? I believe we can allow dups in the system while import

Was this reply helpful? Yes No
Crownd 19 on at

Like (1)

Report

RE: Clean Duplicates

Hi Naveen!

thanks for the fast reply and your input.

This would seem like a nice way to go about it, but, unfortunately, we do not have a license for power automate.

And i would assume a CRM the size of Dynamics should have some sort functonality do deal with multiple duplicate records

Best Regards

Matthias

Was this reply helpful? Yes No
Suggested answer

Naveen Ganeshe 3,397 User Group Leader on at

Like (2)

Report

RE: Clean Duplicates

Hi Matthias,

You can use power automate to do this job for you. Here are the link:
technologyblog.rsmus.com/.../
https://powerusers.microsoft.com/t5/Power-Automate-Community-Blog/Delete-duplicate-records-items-from-Dynamics-365-CRM-CDS/ba-p/334853

Was this reply helpful? Yes No