You’re offline. This is a read only version of the page.
Announcements
Hello,
I have a question about batch parallelism and transaction scope. Is it possible to commit the changes made in a batch job with multiple asynchronous tasks only if there are no errors in any of the tasks and roll all changes back by the batch job run if any of the tasks contained an error?
For context, my company has a weekly batch job that performs posting of approx.400 project hours journals sequentially. This process is taking a long time and so I've created a proof of concept using the top picking technique described here:
https://learn.microsoft.com/en-us/archive/blogs/axperf/batch-parallelism-in-ax-part-iii
This is working great, I've cut the processing time significantly but my company has a requirement that if any error occurs on any of the journals, they would like all of the postings that occurred during the batch to roll back. This was easy when it was one big sequential process. Now that I've implemented this with multiple batch tasks, each journal is posting and committing individually and if there is an error, only that one journal fails to post and the rest are left posted.
I wasn't sure if there is a change to the way I am creating the batch tasks that would indicate to fail and roll back the entire batch job?
Thanks, Martin! I agree with what you've written here. We do have the import and posting process as separate batch jobs so we are good there but I believe I need to investigate more on the process that generates the journals and work with the business to create these journals that can be posted independently without being dependent on one another. Either that or give the business a convenient way of correcting whatever issues may be caused by posting some of the journals but not others.
If you want a transaction, you must give up parallelization. If you run several batch tasks at once, they can be executed on different computers (using independent DB connections). Therefore "Transaction Scope Across Batch Tasks" can't be done, IMHO.
You'd need to approach your problem differently, ideally starting from the solution design. The idea of having separate journals that can be posted independently, but they shouldn't, looks strange to me. Also, I wouldn't merge data import and journal posting in a single step, if possible. If import succeeds, an unposted journal is saved. Then you try to post it, and if it fails, you can try it again (if it fails because of a DB error, for example), or you can correct data before subsequent postings. When you first import the data and only then run the posting, you can then identify related journals and post them together (e.g. creating a batch task with the "batch").
But of course, I'm just speculating without knowing the actual requirements.
After more research and review, I believe that it's not possible for the tasks in a batch job to share a transaction scope. I'm not sure if I'm right but I believe each batch task is probably running independently and has it's own transaction scope. I was hoping to identify some approach to creating the batch tasks or some configuration on the batch header that would allow me to roll back the entirety of the batch job if a single error occurred but it doesn't look like that is easily achievable. I really don't like this problem because I believe transaction scope should be kept smaller and not large like this. Each journal should post and then the system should move to the next. That is working great. Unfortunately, my business has what I believe to be a poor design in the journals that this batch job is posting where some of the journals are loosely related to other journals. They would like the posting of these journals to be all or none so that in the case of an error, they can delete all of the journals and reimport them if needed. I think in absence of a straight forward way to roll the entire batch back, I will investigate the problem up stream and look at how my business can react if errors do occur by either re-importing the journals that need to be re-imported or reversing the ones that posted with invalid values.
This whole topic did get my curiosity though and I'm still curious if anyone is familiar with a pattern that has been done in AX where processes are done in parallel and but the results of the process are committed all-or-none? I started down the path of a batch task control table that would maintain the status of the batch tasks and once the batch tasks were done with their work they would only commit their transaction if all other batch tasks also completed their work and would throw an error and abort their transaction if any batch task reported an error status. This design feels fragile and I'm not very comfortable with it. I worry about the possibility of infinite loops or deadlocking.
Maybe I'm thinking about the problem incorrectly. Is there another way to perform asynchronous operations but keep them within the same database transaction scope? Any help to point to an example that exists in the system or documentation that explains how to do something like this would be greatly appreciated!
In case it helps, here's the controller for my batch job:
Share
Report
All responses (
Answers (
public class ABCPayrollPostController extends SysOperationServiceController
{
List taskList;
BatchGroupId groupId;
SysRecurrenceData recurrenceData;
utcDateTime startDateTime;
}
public Batch doBatch()
{
// add all needed tasks
this.addAdditionalTasks();
// add the normal info - the super() call is suppressed - the task is added in the method above
info(strFmt("@SYS73254", this.caption()));
return null;
}
public void addAdditionalTasks()
{
// create the list of tasks to be added
this.buildTaskBatchList();
// add the tasks to the batch
this.addTasksBatchHeader();
}
private void buildTaskBatchList()
{
taskList = new List(Types::Class);
// add the task 5x - hard coded for now, will parameterize later
taskList.addEnd(ABCPayrollPostTaskController::construct());
taskList.addEnd(ABCPayrollPostTaskController::construct());
taskList.addEnd(ABCPayrollPostTaskController::construct());
taskList.addEnd(ABCPayrollPostTaskController::construct());
taskList.addEnd(ABCPayrollPostTaskController::construct());
}
private void addTasksBatchHeader()
{
BatchHeader header;
ListEnumerator taskListEnumerator;
SysOperationServiceController batchTask;
SysOperationServiceController prevBatchTask;
BatchInfo taskBatchInfo;
header = this.batchInfo().parmBatchHeader();
if (recurrenceData != conNull())
{
header.parmRecurrenceData(recurrenceData);
}
if (!groupId)
{
groupId = this.batchInfo().parmGroupId();
}
if (header != null &&
taskList != null)
{
taskListEnumerator = taskList.getEnumerator();
taskListEnumerator.reset();
while (taskListEnumerator.moveNext())
{
batchTask = taskListEnumerator.current();
taskBatchInfo = batchTask.batchInfo();
taskBatchInfo.parmGroupId(groupId);
header.addTask(batchTask);
prevBatchTask = batchTask;
}
header.parmCaption(this.caption());
header.save();
}
}
public static void main(Args args)
{
ABCPayrollPostController::construct().startOperation();
}
public static ABCPayrollPostController construct()
{
ABCPayrollPostController serviceOperation;
serviceOperation = new ABCPayrollPostController();
serviceOperation.parmClassName(classStr(ABCPayrollPostController));
serviceOperation.parmMethodName(methodStr(ABCPayrollPostController, runOp));
serviceOperation.parmExecutionMode(SysOperationExecutionMode::Synchronous);
serviceOperation.parmDialogCaption("Parallel Payroll Post");
return serviceOperation;
}
public void setRecurrenceData(SysRecurrenceData _recurrenceData)
{
recurrenceData = _recurrenceData;
startDateTime = SysRecurrence::getRecurrenceStartDateTime(recurrenceData);
}
public void runOp(ABCPayrollPostDataContract _dataContract)
{
ABCPayrollPostService::newFromDataContract(_dataContract).run();
}
BatchGroupId parmGroupId(BatchGroupId g = groupId)
{
groupId = g;
return groupId;
}
Here's the service code that each of the tasks runs. Ideally, I would like to get rid of the try/catch here and allow everything to roll back if any error occurred during the posting.
public void run()
{
ABCPayrollImportJournals importJournals;
JournalId currentJournalId;
importJournals.readPast(true);
do
{
try
{
ttsBegin;
select firstOnly pessimisticLock importJournals
index hint ABCPayrollPostingStatusId
where importJournals.ABCPayrollPostingStatusId == ABCPayrollPostingStatus::Ready;
info(strFmt("Batch %1 processing journal %2", processId, importJournals.JournalId));
if (importJournals)
{
currentJournalId = importJournals.JournalId;
importJournals.ABCPayrollPostingStatusId = ABCPayrollPostingStatus::Processing;
importJournals.update();
}
ttsCommit;
ttsBegin;
ABCPayrollCheckPost::PostJournal(importJournals, true, true);
if(importJournals)
{
importJournals.ABCPayrollPostingStatusId = ABCPayrollPostingStatus::Posted;
importJournals.update();
}
ttsCommit;
}
catch
{
if (importJournals && currentJournalId)
{
ttsBegin;
select forUpdate importJournals
where importJournals.JournalId == currentJournalId;
if (importJournals)
{
importJournals.ABCPayrollPostingStatusId = ABCPayrollPostingStatus::Error;
importJournals.update();
}
ttsCommit;
}
}
}
while(importJournals);
}
#1
André Arnaud de Cal...
294,075
Super User 2025 Season 1
#2
Martin Dráb
232,860
Most Valuable Professional
#3
nmaenpaa
101,158
Moderator
