Problem exporting bulk entities (more than 10) through Data Management Framework

(1) Share

Report

Posted on by BADRI NARAYANAN G

218

Hi Friends

We are planning to export more than 20 entities through Data Management Framework(D365) to BYOD(Azure SQL).During this process there are few entities in /executing/ status and few are in /not run/ status.It remains the same even if we run that export for more than a day.Please give me few solutions on how to rectify this issue.Attached the image below.Is there a limit to run certain number of entities.Thank you.

I have the same question (0)

All responses (8)

Answers (1)

Suggested answer

Ramit Paul 22,803 Most Valuable Professional on at

Like (0)

Report
Copy link

Link copied!

Hi,

You can try by restarting the data management service from LCS

Was this reply helpful? Yes No
AR-01121610-0 4,971 Moderator on at

Like (0)

Report
Copy link

Link copied!

Depending on the size and rowcounts it may also be a lack of resources on the Azure SQL DB. Try to monitor resource usage ob the BYOD to be sure this is not the case.

Was this reply helpful? Yes No
Suggested answer

Kevin Xia Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

Hi,

Make sure that the system allocates enough resources to the export process. Insufficient resources can cause timeouts or process stalls. As Ramit said, you can try restarting the service to resolve this issue. Are there any logs with any errors or warnings? Check to see if the export process is generating any logs, or to see if there are any errors or warnings that may have occurred that could be causing the process to break.

Best regards,

Kevin

Was this reply helpful? Yes No
BADRI NARAYANAN G 218 on at

Like (0)

Report
Copy link

Link copied!

Hi Friends,thanks for all your response.

@Kevin Xia - Export process is not generating any errors or warnings it is however generating logs saying '9' 'FNBI_ExchangeRateType' record(s) inserted in staging.Few export process are always in executing status and few remain stuck in not run status.

Thank you

Badri

Was this reply helpful? Yes No
Suggested answer

saurabh bharti 15,045 Moderator on at

Like (0)

Report
Copy link

Link copied!

Hi Badri,

Seems technical issue for optimizing the export batch or resources assigned but from my functional perspective , isn't it possible to breakdown in to multiple export projects?

Was this reply helpful? Yes No
BADRI NARAYANAN G 218 on at

Like (0)

Report
Copy link

Link copied!

Hi Friends,
We have tried to increase the Azure SQL database transaction unit (DTU) size.There is also sufficient storage on the Azure SQL side of about 50GB(Free storage)
After doing this i have tried to export data entities but still facing the same issue i.e the export gets stuck in exporting status.Please let me know if there is any solution for this issue.@saurabh bharti There is requirement to export bulk quantities at once and hence we are creating only one export project.Thank you.

Was this reply helpful? Yes No
Verified answer

Alireza Eshaghzadeh 14,839 Super User 2026 Season 1 on at

Like (0)

Report
Copy link

Link copied!

Hi BADRI,
Do you perform incremental exports for all entities? If so, kindly verify that the "Skip staging" flag is set to Yes.
Combining various entities with different types (Config, master, transaction) in a single data export job may result in issues, particularly as the data volume increases.
I recommend structuring the BYOD (Bring Your Own Database) jobs based on their priority, company-specific/shared nature, and export frequency. Subsequently, consider breaking them down into separate export jobs.

Was this reply helpful? Yes No
BADRI NARAYANAN G 218 on at

Like (0)

Report
Copy link

Link copied!

Hi Friends,
@Alireza Eshaghzadeh - Thanks for your response.I am running the export of entities with full push and i have not marked skip staging for any of the entities.In total i am exporting 114 entities.Actually there is requirement from client to export all the data entities at once to SQL and hence we are creating only one data export job.As of now i have run it through batch.I will update here once it is completed.

Thank you

Badri

Was this reply helpful? Yes No