Queuing Background Tasks

I recently faced an interesting problem. The customer had production terminal at the end of a line, were the operator proceed the output. When he marked the production order as completed he clicked the button on the terminal and the request to post the output was sent to Business Central. For that I created the custom API. Nothing special.

On a very high level this looked like this:

Very strait-forward, worked like a sharm... until we started to test this on the floor. It appeared (oh my god who could predict that :) that the responses could be not only successful, there could be the errors as well. And the reasons were in 100% cases in Business Central: no location code, date is closed, item is blocked and so many other things related exclusively to Business Central itself. And of course the operator near the terminal had nothing to do with that! It's completely outside of his responsibilities and knowledge. He is responsible for the production line and his terminal, and the financial people are responsible for Business Central data to successfully register the output.  

So, what could be the possible solution? Split the responsibilities! And as a consequence - split the Request and PostOutput. I created the staging table, which i exposed as an API. I got the requests, saved and then immediately ran the background task to post the output. The errors I saved to the error log table with the manual opportunity to solve error in BC and run the post output once again. 

On a high level the first diagram changed into this one:

A bit more complicated, but still strait-forward. Operator was happy, financials were happy. Me was happy as well :)

However things become to break when there appeared more than one operator. Did you guess what happened?

A Locking problem!

When multiply terminals were sending requests to my BC CustomStaging API - all was perfect. I just saved the requests - that's it. But, when I run the background task for each individual request record to post output we stucked into the locking problem. As a result the error log was full of errors not related to BC data itself. And financials users had to manually start output process once again, at the same time when the operators were sending requests from terminals... This just had to be resolved somehow. 

Queuing Background Tasks

That could be a solution. But not possible ... outside of the box. Anyway, let's try to implement this. 

Just for simplicity, i will show the logic on a slight different example. 

Creating Items through API in a background

I have a staging table. Here I will save incoming requests. For the simplicity - only Item description.

Also created the API page. When I receive the request I save it to the staging table (line 25), and run the Background task to create item (line 28). In case of errors I save the error to the error log.

The background task logic looks like this. I create the item and delete the staging entry. To simulate the hard work and to get into the the locking problem during multiply calls in (that i will show in a minute) i also do such strange stuff in the beginning. 

In case of errors I save the error in my own error log entry table

Let's try to run this

And in Business Central we will see new Item. 

So far so good.

Creating multiply Items through batch API in a background

Now let's try simulate several services creating items in BC in a parallel. For that we will use batch calls.

The initial services received success responses. However if we look at Business Central items list we will see

Only 1 new item was created! Were is the second one? Let's go and see the error log table that we created.

And here we come into the necessity of queuing background tasks.

Creating multiply Items through batch API in a background with queuing 

The thing is that you cannot queue background tasks. There only two parameters that you can control : IsReady, NotBefore.

NotBefore will not help us, cos we don't know when to start the task. The incoming requests could come continuously, the execution time of each task could be different, there could be posting process inside business central itself. It's just not possible to calculate when task should be started. 

Let's have a closer look at IsReady. 

Sets the task to the ready state. A task cannot run unless it is ready.

Here it is! We can set the IsReady to true for the first task in the list, and for the all other tasks we will set IsReady to false. So the first task will be executed and the other tasks will be set OnHold. When the first task will be executed, we will find the second task and set the IsReady to true. And so we will start the loop. Simple and elegant!

First we will change the TaskScheduler.CreateTask into this

and when the first task will be executed we will mark the next one as IsReady.

Let's try to create two items in a batch now.

So far so good. And in Business Central

Great! Hope this will help some of you

The sourcecode is available here https://github.com/dkatson/Blog-Queuing-background-tasks