Views:

Applies to Product - Dynamics 365 Commerce
 

What’s happening?
Customers are experiencing issues with data export to their BYOD database, including long processing times for batch jobs and failures due to timeout errors.
 

Reason:

  • For the first case, the batch processes are taking too long, potentially due to complex entities being used for exports, which can create issues when using in conjunction with change tracking queries.
  • In the second case, the inability to "skip staging" during the export process is caused by the presence of virtual fields, container fields, composite entities, or bitmap fields in the entity. The entity also has a post-load method that prevents skipping staging.
  • The third case indicates that jobs are failing with a timeout error, which may be related to slow job execution within the defined timeout period.

 

Resolution:

  • For performance issues with BYOD exports:
  • Avoid using very complex entities for exports; consider creating simpler entities for data warehouse exports.
  • Export using batch instead of synchronous exports. Create a separate batch group with N-1 AOS and process exports during off-business hours.
  • Use only required fields and remove unnecessary fields from mapping.
  • Utilize the skip staging option wherever possible.
  • Implement incremental exports and limit the number of columns tracked using only primary table change tracking or custom queries.
  • Use full push for data entities with large numbers of joined data sources, avoiding enabling change tracking for all data sources.
  • Regularly clean up staging tables using the job history clean-up feature to improve performance.
  • Ensure the BYOD database tier is sufficient (preferably P2 or higher).
  • Limit export projects to up to 10 entities; if more, break them down into additional projects.
  • For the inability to skip staging:
  • Remove any virtual fields or other problematic fields from the entity as confirmed by the user.
  • If the issue persists, consider testing the V4 entity, which may not have the same limitations as the V2 entity.
  • For timeout errors during data export:
  • Increase the timeout value in Framework Parameters > BYOD tab to allow for longer processing times, especially for entities with large volumes of data.
  • Investigate any custom queries that are executing sow and exceeding the job time out period.
  • If the resolution steps are not sufficient or the issue remains unresolved, further investigation may be required with support.