5 Tips to improve the performance in a DMF import project

Data integration is a key success factor in almost every implementation. A primary concern of my customers is the speed that data can be imported, especially if there are vast amounts of data and a small-time window.

Here you have 5 tips to improve the performance:

1. Composite entities are not recommended. For example, use Sales Order Headers V2 and the Sales Order Lines V2 entities instead of Sales orders composite entity.

2. Limit use of XML files. XML files are slow. XML files need to go through complex parsing and transformations, which can take up considerable processing power and time. For high volume, it is recommended to use multiple CSV files processes in parallel.

3. Set up Configuration Parameters. Navigate to the Data Management Workspace – Framework parameters – Entity settings – Configure entity execution parameters:
   
   For example, I set up 100 Import threshold count and 4 in the Import task count for the Sales Order Headers V2 and the Sales Order Lines V2.
   
   
   
   1. Import threshold record count – This field defines how many records (lines) D365 should upload per ‘Task’.
           b.Import task count – This field defines the number of tasks that should run simultaneously
   
   
4. Increase the number of batch threads. As the DMF import and export projects use bath jobs, it is a good idea to increase the number of batch threads.
   
   
   
   
5. Clean the data. A lot of time is spent on validations and error reporting. Consider this fact when you import a high volume of invalid or inconsistent data. We recommend that you try to fix and reduce errors that are related to data quality. In this way, you help prevent unnecessary executions of validation and error handling. Also, you can disable the Run business validation. Navigate to the Data Management Workspace – Data entities – Select the Entity – Entity structure – uncheck the ‘Run business validations’ checkbox.
   
   

If you apply all these recommendations and your DMF import project still slow, please review the features flighted in data management. These features are enabled via flighting. Flighting is a concept that allows a feature to be ON or OFF by default.

Data management overview - Finance & Operations | Dynamics 365 | Microsoft Docs

Thanks for reading,

Said

Acknowledgments: I want to thank my colleague Amy Flowers (Customer Engineer) for her contributions and peer review.