Skip to main content

Notifications

Announcements

No record found.

News and Announcements icon
Dynamics 365 Community / Forums / Microsoft Dynamics CRM (Archived) / Optimze perfomance for...
Microsoft Dynamics CRM (Archived)

Optimze perfomance for batch update to 1 million contacts in CRM

(0) ShareShare
ReportReport
Posted on by Community Member Microsoft Employee

Hello, 

We have a business need where we have to update 1 million records in CRM every month. The update requries updation of 11 fields on the Contact entity. 

We developed a console app for that and tried to do the Bulk update using Execute Multiple, but then we faced the time out issue. We tried to reduce the batch size from 1000 to 500, but it seems like it will take days to update 1M records. 

The maximum time we have to run the batch job is 5 hours. I need to know what is the best practice or solution to perform this operation. Do we need any third party tool, or we can modify and optimize our console app to run the operations fast. 

Need expert advise here. 

Thanks, 

Fahad

*This post is locked for comments

  • Suggested answer
    Guido Preite Profile Picture
    Guido Preite 54,081 Super User 2024 Season 1 on at
    RE: Optimze perfomance for batch update to 1 million contacts in CRM

    direct update of SQL records is not supported, people should stop to suggest this approach

  • Suggested answer
    JayeshAC Profile Picture
    JayeshAC 1,157 on at
    RE: Optimze perfomance for batch update to 1 million contacts in CRM

    Hi Fahad,

    If you want to update millions of records then better option would be using SQL scripts instead of CRM API.

    Though it is not recommended but it will solve your problem and update million of records in few minutes.

    Click on below link to know more details about calling SQL script.

    community.dynamics.com/.../207020

    Regards,

    Jayesh

  • Suggested answer
    Community Member Profile Picture
    Community Member Microsoft Employee on at
    RE: Optimze perfomance for batch update to 1 million contacts in CRM

    Hi Fahad,

    when you have to deal with millions of records you have to forget about standard CRM Service.

    Although it is not supported, go straight to SQL Server doing a lot of testing in a cloned environment (using SSIS, console application or commercial software like Scribe).

    In the past, when I worked hard on OnPremise environments, I did it without any issue (console application).

    Hope it helps.

    If you found the answer helpful, please mark as Verified 

    Thank You & Best Regards

    Francesco Picchi

    Microsoft Dynamics CRM Consultant, Bologna, ITALY.

    Independent Contractor

    http://www.francescopicchi.com

  • Suggested answer
    Guido Preite Profile Picture
    Guido Preite 54,081 Super User 2024 Season 1 on at
    RE: Optimze perfomance for batch update to 1 million contacts in CRM

    if you are onpremise you can definitely optimize your console application.

    The ideal will be to read first all your CRM records (and because you are onpremise you can use the SQL filtered view, it should speedup the process.

    After you compare in memory the actual changes you need to update, you wrote about 11 fields, probably these 11 fields do not change always all and not for all the contacts, so when you update a contact you need to be sure that it requires to be updated (so you don't update a contact that had no changes) and just update the changed fields (for example 3 fields of 11)

    hope it helps

  • Community Member Profile Picture
    Community Member Microsoft Employee on at
    RE: Optimze perfomance for batch update to 1 million contacts in CRM

    CRM 2016 on-Premise

  • Friyank Profile Picture
    Friyank 944 on at
    RE: Optimze perfomance for batch update to 1 million contacts in CRM

    instead of Console try powershell script.  

  • JayeshAC Profile Picture
    JayeshAC 1,157 on at
    RE: Optimze perfomance for batch update to 1 million contacts in CRM

    Hi Fahad,

    Which version are you using - CRM online or OnPremise?

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

December Spotlight Star - Muhammad Affan

Congratulations to a top community star!

Top 10 leaders for November!

Congratulations to our November super stars!

Tips for Writing Effective Suggested Answers

Best practices for providing successful forum answers ✍️

Leaderboard

#1
André Arnaud de Calavon Profile Picture

André Arnaud de Cal... 291,280 Super User 2024 Season 2

#2
Martin Dráb Profile Picture

Martin Dráb 230,235 Most Valuable Professional

#3
nmaenpaa Profile Picture

nmaenpaa 101,156

Leaderboard

Featured topics

Product updates

Dynamics 365 release plans