64
hi,
I have a requirement to upload 5 million customers in D365 FO and need to explore diffferent option which are faster and better.
I also want to understand how can we explore the option of "ODBC" and "azure sql db " while importing the records via DMF in a way that it can help me import the records faster.
I think we can use csv instead of excel and it would shrink the file size.
Kindly share your experience on this.
Thanks,
VK
As far as I know, ODBC isn't and never was supported by F&O.
And if think that "Azure SQL DB" is used only by BYOD export.
Hi Martin
thanks for the reply.
I have a requirement to import around 5 million customers in the D365 FO environment during the go live phase and hence I am planning out different options that I should consider for importing it.
Also since the client is having a third party software however it uses sql as backend, I wanted to check if I can leverage the option of "Azure Sql DB" and "ODBC" connection .
I haven't used this option before but would like to understand more on this.
Sometimes is benefitial to create a custom data entity with just those things that you need for the particular performance-critical import. Removing unnecessary logic or taking some shortcuts (that you can affort because you know the imported data) may have significant impact on performance.
My strong recommendation is to avoid continuous number sequence whenever possible. Why exactly do you want to use it for customers?
Manual intervention is not required. It is something like a normal batch job but here the set batch jobs will run simultaneously. Earlier I did the same for my custom table.
Are you telling if set to continuous, the number sequences will hung? I guess there wont be any issue on that. But if the number sequence exceed then it will cause an issue. Also while setting this execution parameters, you should be very careful about setting the tasks because if it is more than the capacity of system, it will cause the d365 to hang. So be very careful while setting this.
Nice thread Bharani Preetham Peraka.
One question for my own understanding.. as I see all these batches are running at the same time which is pre-defined as per the framework parameters and no need any manual intervention eveytime to load data.
But will it impact anyway to the number sequence in case if customer account number is set to 'Continuous' ?
And how such batches usually work.. means is it like one batch will start after the first one ends? Because I see atleast 3-4 batches are executing at the same time!
Regards,
Sourav
We have a standard function for this in the DMF. Use the following link.
Hi vicky1234,
As of now.. data management is the best possible way to load huge data at a time.
Another option is Excel add-ins but it has limitation and cannot push much faster than Data management.
Also you can check with developer if they create any staging table and from there they pick the data to push in main table. It will also help you to track what are the data that are pushed and what is still pending.
Example, we created a custom form called "Customer entry #" interface where data is getting loaded first and from there, through batch process, new record (and updated record) are pushed in the main Table.
But unfortunately I cannot share the code detail as part of privacy. You can show your developer and discuss.
Regards,
Sourav
Hi vicky1234,
You can use Data management for "Customers V3" data entity.
Now talking about huge data.. you can create multiple load one after another. Every load may contain say around 25k new records. Once one batch ends, you can load another batch with new 25k records.
Breaking data in multiple loads will give you much faster master data upload in system.
Regards,
Sourav
#1
André Arnaud de Cal...
293,278
Super User 2025 Season 1
#2
Martin Dráb
231,983
Most Valuable Professional
#3
nmaenpaa
101,156
Moderator
Share
Report
All responses (
Answers (
