Process runs slow in Batch job when record count greater than 15K

(0) Share

Report

Posted on by RahulD365

Hi Team

I came across a weird issue - I have a class which does some process and it's running in batch job

Scenario 1:

When I have 5000 record in my table, and the batch job loops thru the table records(5000) to do some operations on those records, basically it's just taking approx 10-12 minutes to process 5000 records in batch

whereas,

Scenario 2:

Processing 15K records, I observed via SQL that it's running slower than Scenario 1, It was been observed that for 5K record - it took approx. 30min, and to process whole 15K record it took approx. 2 hours.

Did anyone face similar issue anytime?

Am I missing anything ?

I tried implementing - clearing cache, queryrun.next(), following all BP etc..

Please help. thanks!

I have the same question (0)

All responses (4)

Answers (0)

André Arnaud de Cal... 303,679 Super User 2026 Season 1 on at

Like (0)

Report
Copy link

Link copied!

Hi Rahul,

Can you share your coding, so we can try to assist? On what level(s) do you have try/catch and/or ttsbegin and ttscommits? Have you also considered splitting the batch job into multiple tasks; each with a different part of the dataset?

Was this reply helpful? Yes No
zhifeng Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

Hello Rahul,

Agree with Andre, we need to understand how your code is designed. And did you put 15K records in one large transaction or split to multiple transcations? Have you tried multiple task?

Was this reply helpful? Yes No
RahulD365 89 on at

Like (0)

Report
Copy link

Link copied!

Sorry I forgot to mention that :)

Yes, it's running in multithread and it's simple run method where it has ttslevel correctly placed.

Run method is just trying to loop the record from the table and generate XML in the Azure file storage folder.

ttsbegin;

While select SomeTable

where SomeTable.status = 'Pending'

{

// splitting the task upto 1000 records in one batch task

//Via using contract class, trying to place the XML generated to Azure storage folder

}

ttscommit;

I'm just trying to understand why it's taking so much time for 15K record, whereas 5K record just get executed in 10-12 min.?

Was this reply helpful? Yes No
zhifeng Microsoft Employee on at

Like (0)

Report
Copy link

Link copied!

Why do we need ttsbegin, per my understanding we just query data and send it to xml.

Was this reply helpful? Yes No