76,267 Records to import to target Will it Allow ?

(0) Share

Report

Posted on by Sakthiananth

2,127

Hi All,

I had been struggling with the new website design of Dynamics AX, Some how managed to come over but the Issue is the same like website for AX.

Here's my Problem.

One of client imported the External item details in AX through Data Migration Framework with 76,267 records for Staging It hardly took 2 Min to insert into staging when it comes to Target table (eg: SuppItemTable) It's taking 32 Hrs to import 35,000 records.

Note: We're using Pipe File format to import the data.

Rest of the records are not inserted as well.

Does any one came across such a problem.

*This post is locked for comments

I have the same question (0)

All responses (7)

Answers (0)

Sort by

Suggested answer

André Arnaud de Cal... 305,797 Super User 2026 Season 1 on at

Like (0)

Report
Copy link

Link copied!

Hi Sakthi,

It all depends on configuration.

You can see if you have enabled to run business logic during copy to target or not. Also check the indexes used on the target table. With mass import indexes might become fragmented. You can run a reindex to see if it helps. Possibly you can also disable some indexes during copy to target. Make sure you enable them again.

Also watch at hardware resources on the servers involved to see if there is a bottleneck on e.g. memory.

Insert into staging is indeed much faster as SSIS is used for it.

Was this reply helpful? Yes No
Suggested answer

Malcolm Burtt 1,475 on at

Like (1)

Report
Copy link

Link copied!

Hi Sakthi,

My guess is that you are processing the import file using a single thread. There is a feature in DIXF that allows you to run the staging to target process using multiple batch threads which should allow you to compress the total time taken for the import.

The only trouble is I can't recall exactly how you do this at the moment. :-)

I'll look into it and post the answer unless someone else beats me to it.

Regards

Malcolm Burtt

Was this reply helpful? Yes No
Suggested answer

Community Member on at

Like (0)

Report
Copy link

Link copied!

To use multiple threads/tasks to import data from staging to target, change Number of Tasks in the Target Data Execution form.

1) Bring up the Target Data Execution form

a. Data Import Export Framework > Common > Processing Group

b. Select the row of the appropriate Group Name

c. Click Copy Data To Target

d. Select the appropriate Job ID and click OK

2) Change the Number of Tasks

a. For example, increase the value from 1 to 6 (AX recalculates Bundle Size)

b. Click Run

Was this reply helpful? Yes No
Sakthiananth 2,127 on at

Like (0)

Report
Copy link

Link copied!

Thank you for your time Burtt.

Was this reply helpful? Yes No
Sakthiananth 2,127 on at

Like (0)

Report
Copy link

Link copied!

Thanks Andre, I'll do import by disabling the indexes in the target table and will update you the results ASAP.

Was this reply helpful? Yes No
Sakthiananth 2,127 on at

Like (0)

Report
Copy link

Link copied!

Hey. Thanks Rick. But I found some errors while validating the records in the staging and I came to know that this because some one wiped the related records in the DB because of this I'm not able to import those data to the target.

Once again thanks for for your valuable input.

Was this reply helpful? Yes No
Sakthiananth 2,127 on at

Like (0)

Report
Copy link

Link copied!

Hey. Thanks All . I found some errors while validating the records in the staging and I came to know that this because some one wiped the related records in the DB because of that I'm not able to import those data to the target. Will update the solution ASAP.

Once again thanks for your valuable inputs.

Thank's

Was this reply helpful? Yes No