Question on how to use multi-threading with SysOperations Framework

(2) Share

Report

Posted on by jmcmillan

Greetings,

I'm trying to create some classes that take advantage of the multi-threading capabilities of the batch framework. I understand that in order to use the top-picking methodology, I need to basically set some records in a staging table that the work item classes will operate on with a pessimistic selection on the table. My question is what is the best practice to restock the table once it reaches the end? I can create a table method that can add more records, but how do I coordinate the work item classes to not try to do this at the same time while one of them has called this method?

I'd love it if there was a way to share an instantiated class among the threads that could better coordinate their efforts. I guess this isn't possible, or else the staging table wouldn't necessarily be needed in the first place. Is there a good way to have the threads communicate to each other? Is there any metadata on the tables that could be shared with the threads to show the state of the call of the restocking method?

Categories:

Dynamics 365 Finance

I have the same question (0)

All responses (3)

Answers (0)

Mariano G. 3,598 on at

Like (1)

Report

I would recommend you this excellent post about different multithreading approaches: Multithreading in D365 Batch Jobs - Dynamics 365 Musings

Was this reply helpful? Yes No
jmcmillan 98 on at

Like (0)

Report
@Mariano G., that is indeed a great article. I'm absolutely a fan of Peter Ramer's Blog. It's been a great resource in learning how to program with x++ and design custom solutions for D365 F&O. However, it only touches on the three major design methodologies for multi-threading batch jobs.

I'm wondering if anyone has found a way to better coordinate the threads with an instantiated class that resides in memory. From what I've gathered, each batch task (thread) uses its own session, which could potentially be on a completely separate AOS. I'm wondering if there is a way with the SysOperations Framework to store certain states of the process (such as a call to a method that populates the staging table).

What I envision is the following:

Service class pre-populates staging table with records that need to be processed (up to a certain number as this could be a great amount).

Service class then creates the different batch tasks (threads) using the allowed max threads as set in the Contract class.

Each thread continues to work through the staging table until it reaches the end, in which case the first thread that reaches this point calls a table/class method to repopulate the staging table with more records.

I don't really want multiple threads to call this method at the same time, so if there is some way to coordinate this without having to create another table, then that's preferable.

I get that having the ability for true parallelism introduces possible race conditions, but that's why you adjust for that. I'm just trying to understand if there is a base class that handles this. I haven't seen it in any documentation for SysOperations that is easily available or explained thus far.

Was this reply helpful? Yes No
Martin Dráb 237,990 Most Valuable Professional on at

Like (0)

Report

The solution described in Batch Parallelism in AX – Part - III uses a staging table, but it isn't the idea is different than yours. The design there isn't meant as something running continuously. It's a solution for a situation when you have a certain amount of records (such as invoices ready to be posted) and you want to process them - the batch run ends when they're processed. This might be an one-off task or something running, say, every night. The parallelism is used to process the records by several tasks at once, but it's still true that when all the records (that were available when the batch started) are processed, the batch run ends.

If you want to have a batch tasks running continuously, consider if you can't get the data prepared by normal business processes. For example, I used this approach for a queue of data to be send asynchronously; a business process added new records for processing and the batch tasks took them. It's just important to avoid blocking, e.g. the other processes shouldn't try to change existing records.

Anyway, if you want one task creating staging data and other tasks waiting for it to complete, notice that top-picking technique already does something similar. A task uses locks in database to inform other tasks which record it's processing. You can have a table for this purpose; the difference will be that you won't use readPast(true) in this case. This approach, in a bit more complicated way, it's also used by SysAutoSemaphore class.

Was this reply helpful? Yes No

Join the conversation

Community site session details

Question on how to use multi-threading with SysOperations Framework

Helpful resources

Quick Links

Responsible AI policies

Neeraj Kumar – Community Spotlight

Congratulations to the November Top 10 Community Leaders!

Leaderboard > Finance | Project Operations, Human Resources, AX, GP, SL

Featured topics

Product updates

Community site session details

Question on how to use multi-threading with SysOperations Framework

Helpful resources

Quick Links

Responsible AI policies

Neeraj Kumar – Community Spotlight

Congratulations to the November Top 10 Community Leaders!

Subscribe to this forum!

Select categories

Leaderboard > Finance | Project Operations, Human Resources, AX, GP, SL

Featured topics

Product updates