Skip to main content

Notifications

Announcements

No record found.

News and Announcements icon
Dynamics 365 Community / Forums / Microsoft Dynamics CRM (Archived) / [RANT] Why are duplica...
Microsoft Dynamics CRM (Archived)

[RANT] Why are duplicate detection rules honored when re-importing existing data?

(0) ShareShare
ReportReport
Posted on by f23fzf34ffavcz2t56 221

It's infuriating and baffling to me that duplicate detection rules are applied to data that is being re-imported after being modified in Excel. Duplicate detection rules ONLY make sense in the context of new records! I'm at my wits end with CRM and all the silly little things it does that directly inhibit productivity.

Take for example the concept/functionality of Save & New. Why can't we Save & New? It takes 7 seconds, and several clicks+page loads, for one of my most proficient users to add new Opportunity Products. A Save & New button would cut that down to 1-2 seconds. This time adds up, Microsoft!!!

If duplicate detection rules are applied when importing existing data, why aren't they applied when modifying records normally? Oh, let me guess. Because it doesn't make sense to apply duplicate detection rules to existing records??? Could that possibly be the reason why?

If the developers of CRM applied duplicate detection rules to existing records, being edited normally, you couldn't get any work done because you'd trigger a rule every single time! SO WHY DOES IT APPLY WHEN IMPORTING DATA?

It doesn't make any sense. It's been this way for years and no one has fixed it? Is not one else bothered by this? Am I the only person that runs into this?

*This post is locked for comments

  • Robert Wittig Profile Picture
    Robert Wittig 1,885 on at
    RE: [RANT] Why are duplicate detection rules honored when re-importing existing data?

    Keep in mind this turns it off for new imports that create records and imports that update existing records.

  • Suggested answer
    f23fzf34ffavcz2t56 Profile Picture
    f23fzf34ffavcz2t56 221 on at
    RE: [RANT] Why are duplicate detection rules honored when re-importing existing data?

    Meant to reply to this a few days ago...

    You bring up a good point. Thanks. While I blame it all on Microsoft's poor messaging, I had a lot of misunderstandings about duplicate detection. I was able to clean up our duplicate detection rules a bit to clear up some of the false negatives during imports but I'm still left with phantom duplicates. For those I chose to allow duplicates during the import.

    While working through this I found another interesting option in Settings > Data Management > Duplicate Detection Settings. You can explicitly turn of duplicate detection for imports. I don't see this as being more convenient than just allowing duplicates during import but it's there.

    2016_2D00_05_2D00_17-14_5F00_54_5F00_48_2D00_Duplicate-Detection-Settings.png

  • Community Member Profile Picture
    Community Member Microsoft Employee on at
    RE: [RANT] Why are duplicate detection rules honored when re-importing existing data?

    Duplicate detection *does* apply to existing records. If you have two contacts, one with an email address and one without, then you add an email address to the second one that is the same as the first one this will trigger a duplicate detection warning.

    So if you export data and modify it and then import it, the same applies, because you could have added or changed the data to a field that now identifies a record as a duplicate.

    To avoid this happening, as you have now seen, you can tell the import process to ignore duplicate detection.

  • f23fzf34ffavcz2t56 Profile Picture
    f23fzf34ffavcz2t56 221 on at
    RE: [RANT] Why are duplicate detection rules honored when re-importing existing data?

    Thank you for making it clear but why would they word it this way? Never did I think, "Allow duplicates: Yes" meant "Don't perform duplicate detection rules when importing existing data." I only ever thought of that option as being relevant to NEW data.

    If the prompt were worded more sanely like, "Ignore Duplicate Detection rules during import" all of this could have been avoided. But, no, that would have taken some deep thinking.

    I doubly didn't consider changing the value of that field because when I first came across this fist-shaking behavior I reached out to CRM Support. They were baffled at first too. After a few days they came back and said it was by design but never offered this as a workaround. Instead, their workaround was to temporarily disable duplicate detection rules.

  • f23fzf34ffavcz2t56 Profile Picture
    f23fzf34ffavcz2t56 221 on at
    RE: [RANT] Why are duplicate detection rules honored when re-importing existing data?

    No need to apologize. I appreciate you taking the time to confirm.

  • Verified answer
    JDMCRM Profile Picture
    JDMCRM 1,795 on at
    RE: [RANT] Why are duplicate detection rules honored when re-importing existing data?

    Hi,

    By exporting all of your Records in a 'Dynamic Worksheet' the Unique Identifiers are exported with the Records.

    When importing, as shown in my screenshot above, if you allow duplicates it will simply disregard the duplicate detection rules and UPDATE the existing Records. It will not create duplicates.

    -----

    e.g. A.Datum - Account Number = A100

         Contoso  - Account Number = A200

    &

    Duplicate Detection Rule in place that doesn't allow for Account Numbers to be a match.

    ----

    If you:

    1. Export all of your Account Records in a Dynamic Worksheet

    2. Update the Contoso Account number to A100

    3. Set 'Allow Duplicates' to 'no'

    The import will fail.

    However, if you set 'Allow Duplicates' to 'Yes' the existing Account 'Contoso' will have its Account Number updated to the same as A.Datum.

  • Robert Wittig Profile Picture
    Robert Wittig 1,885 on at
    RE: [RANT] Why are duplicate detection rules honored when re-importing existing data?

    You might be right for 2016, I only did it at a few clients and I know I did it for 2015 and down, I just went and looked and I cannot find the command for save and new anywhere.  Sorry!

  • f23fzf34ffavcz2t56 Profile Picture
    f23fzf34ffavcz2t56 221 on at
    RE: [RANT] Why are duplicate detection rules honored when re-importing existing data?

    Just letting you know you left your name, URL, and company names visible but it looks like that may have been unintentional.

  • f23fzf34ffavcz2t56 Profile Picture
    f23fzf34ffavcz2t56 221 on at
    RE: [RANT] Why are duplicate detection rules honored when re-importing existing data?

    I looked at the Connect site (thanks) and will add my suggestion though I don't have any hope for it ever being seen or acted upon by MS.

  • f23fzf34ffavcz2t56 Profile Picture
    f23fzf34ffavcz2t56 221 on at
    RE: [RANT] Why are duplicate detection rules honored when re-importing existing data?

    > I personally prefer it does the check when re-importing especially in the case of accounts/contacts/leads.

    I don't understand this. The original export explicitly identifies the records being modified with a GUID and a timestamp. There's no chance of duplication if those fields are left intact.

    On the Save & New front, I was told this feature was completely removed from CRM 2016 which is the version I am on. Do you know otherwise? But what really gets me about this is that Microsoft believes this feature to be unnecessary. That fact is that it's not unnecessary even if it is little used (conjecture on my part.)

    This is a time-saving button. Microsoft is making CRM less performant and more click heavy. The new live grid (if that's the right term) is absolutely useless for anyone with customized entities because you can't add your custom fields to the grid. You can't execute JS against this grid either.

    All these decisions are crazy.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

December Spotlight Star - Muhammad Affan

Congratulations to a top community star!

Top 10 leaders for November!

Congratulations to our November super stars!

Tips for Writing Effective Suggested Answers

Best practices for providing successful forum answers ✍️

Leaderboard

#1
André Arnaud de Calavon Profile Picture

André Arnaud de Cal... 291,280 Super User 2024 Season 2

#2
Martin Dráb Profile Picture

Martin Dráb 230,235 Most Valuable Professional

#3
nmaenpaa Profile Picture

nmaenpaa 101,156

Leaderboard

Featured topics

Product updates

Dynamics 365 release plans