In the 10.0.39 release,
a significant change has been made regarding batch jobs, particularly regarding how job history is handled for large batch jobs with more than 5,000 batch tasks. Previously, all batch tasks were stored in the job history, regardless of the job size. However, to address potential performance issues and prevent blocking of batch-related tables due to the size of such jobs, a new approach has been implemented.
Now, if a batch job contains more than 5,000 tasks, only the first 2,500 tasks will be saved in the job history. However, it's crucial to note that the system prioritizes which tasks to save based on their status. Tasks with the status of Error, Canceled, Finished, and Not Run will be saved first, in that order of priority.
To ensure that you don't lose any critical job history data, it's essential to understand this new behavior and take proactive steps to mitigate any potential impact. One approach could be to regularly monitor and archive job history data for large batch jobs, especially those exceeding the 5,000-task threshold. By periodically archiving historical data, you can maintain a comprehensive record of job history while also managing database performance effectively.
Additionally, consider optimizing batch job configurations to minimize the number of tasks generated for large jobs wherever possible. This could involve breaking down large jobs into smaller, more manageable batches or optimizing job scheduling to distribute workload more evenly over time.
Overall, by staying informed about these changes and implementing proactive measures to manage batch job history effectively, you can ensure that your organization continues to benefit from efficient batch processing without sacrificing critical historical data.
