Customer experience | Sales, Customer Insights,...

How to delete duplicate records

(0) Share

Report

Posted on by HimaM

135

Hello All,

We have inserted 50k duplicate data how can i detect and delete the records in dynamics 365?

When i tried duplicate detection rules it says as per the below SS.

8358.Capture.PNG

Please suggest.

Regards,

Hima

I have the same question (0)

All responses (11)

Answers (0)

Umair Iqbal on at

Like (0)

Report

Hi Hima,

Please see the below two links which will help you to resolve your issue:

Link 1 - community.dynamics.com/.../274295

Link 2 - docs.microsoft.com/.../run-bulk-system-jobs-detect-duplicate-records

Please let me know if these links were helpful for you to resolve your issue.

Kind Regards

Umair

Was this reply helpful? Yes No
LuHao 40,892 on at

Like (0)

Report

Hi HimaM,

I would suggest that you first find all duplicate records through XrmToolBox, then merge or delete part of them, and wait until there are less than 5000 duplicates, you can ues OOB delete duplicate records feature.

Alternatively, narrow down the scope of the duplicate detection rule, first find duplicate records within 5000, delete them, and then find other duplicate records and delete them.

Deduplicator for XrmToolBox

Deduplicator helps you find duplicated records in Microsoft Dynamics 365 CE / CRM with ability to overcome CRM limitations.

This is the download page of XrmToolBox: https://www.xrmtoolbox.com/

After downloading and unzipping it, open 'XrmToolBox.exe'.

Contact it to your D365 organization. At Tool Libarary, install 'Deduplicator'.

First Load Entities, then select Entity and fields, finally Find duplicates.

Was this reply helpful? Yes No
HimaM 135 on at

Like (0)

Report

Hello Hao,

Thank you so much for your response.

Using XRM Toolbox i found the duplicate records. There are around 35000 records duplicated. How do i delete the duplicate one?

Regards,

Hima

Was this reply helpful? Yes No
Suggested answer

ashish12 3,079 on at

Like (1)

Report

docs.microsoft.com/.../run-bulk-system-jobs-detect-duplicate-records

Was this reply helpful? Yes No
LuHao 40,892 on at

Like (1)

Report

Hi HimaM,

Since 35000 records are too many, you have to take another solution, which is to reduce the scope of duplicate detection.

Detect less than 5000 records at a time and delete them, execute them in batches.

1. First, create a personal view of the entity that need to delete duplicate records, set the filter of the view a liiter more strictly, that is you need to control the number of records in this view to be around 5000 or slightly more than 5000.

Use Advanced Find to quickly create personal views.

Then, repeatedly create personal views, the filtering of each personal view should be different, and control the number of records in this view to be around 5000, until all personal views contain all records of the entity.

In this case, we divide the huge number records in one view into the appropriate number of records in multipe personal views.

Then run bulk system jobs to detect duplicate records on these personal views.

In this way, the scope of detect duplication is reduced to a single personal view and the detection result must be less than 5000 records, so they will be successfully deleted.

Was this reply helpful? Yes No
LuHao 40,892 on at

Like (1)

Report

Hi HimaM,

Since 35000 records are too many, you have to take another solution, which is to reduce the scope of duplicate detection.

Detect less than 5000 records at a time and delete them, execute them in batches.

1. First, create a personal view of the entity that need to delete duplicate records, set the filter of the view a liiter more strictly, that is you need to control the number of records in this view to be around 5000 or slightly more than 5000.

Use Advanced Find to quickly create personal views.

Then, repeatedly create personal views, the filtering of each personal view should be different, and control the number of records in this view to be around 5000, until all personal views contain all records of the entity.

In this case, we divide the huge number records in one view into the appropriate number of records in multipe personal views.

Then run bulk system jobs to detect duplicate records on these personal views, create bulk system jobs multipe times, selecting a different personal view each time.

In this way, the scope of detect duplication is reduced to a single personal view and the detection result must be less than 5000 records, so they will be successfully deleted.

Here are the relevant documents for reference.

How to Create and Share Personal Views in Microsoft Dynamics 365: https://www.magnetismsolutions.com/blog/adammurchison/2017/09/14/how-to-create-and-share-personal-views-in-microsoft-dynamics-365

Run bulk system jobs to detect duplicate records: https://docs.microsoft.com/en-us/power-platform/admin/run-bulk-system-jobs-detect-duplicate-records

Was this reply helpful? Yes No
DynamicsNik 75 on at

Like (0)

Report

excellent solution!

Was this reply helpful? Yes No
Shaheer Ahmad on at

Like (0)

Report

I have found the duplicate records using the deduplicator. But how do i delete them? there's no option to delete them

Was this reply helpful? Yes No
Inogic 709 on at

Like (0)

Report

Hi,

You cannot detect more than 5000 duplicate records due to OOB Bulk Duplicate Detection Job limit, which restrains you from detecting more than 5000 duplicates.
But with Dynamics 365 CRM data hygiene solutions like DeDupeD App by Inogic, you can easily detect and clean up/delete more than 50,000 duplicates.
With DeDupeD, you can also detect duplicate records within the same entity and across different entities and merge duplicate records of the same entity into one; define rules to minimize and prevent duplicate records entering the CRM system from the client side or server side; and merge historical duplicate records in bulk.
Additionally, the app lets you identify duplicates based on changes in fields or columns and also upon saving the record. This data management tool proves helpful in streamlining the data deduplication process for CRM users.
Download your free 15-day trial from the Inogic website or Microsoft AppSource to get started!

Thanks!

Was this reply helpful? Yes No
Community member 2 on at

Like (0)

Report

Hi,

Did you find any solution? What deduplicator do you use to find any duplicate records?

Was this reply helpful? Yes No