Skip to main content

Notifications

Announcements

No record found.

Finance | Project Operations, Human Resources, ...
Answered

workflow records are taking long time to process

Posted on by 796

Dear All,

 

Workflow records are taking long time to process and batch job level there was no errors and batch completing without any errors but taking long time to finish. Batch recurrence is everyone min.

In event viewer we are getting below message regularly

Workflow Instance ID: 101484 failed to find a bookmark for the workflow message id e359312a-f235-49fe-ad23-efae105eecf9 message type AcknowledgeTaskCompleted.

For each condition validation it take more time.

 

we do not have any changes/customizations relating to workflow. All over sudden we are getting this issue. We have two AOS's one is acting a batch server and we did full compilation and CIL and restarting the services. Even though issue is same

Note  :- Even Tutorial_WorkflowProcessor taking long time to process.

  • kvnkk Profile Picture
    kvnkk 796 on at
    RE: workflow records are taking long time to process

    Dear Andre,

    From 5-days we are not getting any issue, all records are processing without any issue, I had a discussion with my DB, he also monitored, and he confirmed that there was no blockings and long running queries.

    We will keep monitor if we found something will update on the same request. Thanks for your valuable suggestions.

  • André Arnaud de Calavon Profile Picture
    André Arnaud de Cal... 291,280 Super User 2024 Season 2 on at
    RE: workflow records are taking long time to process

    Please update this thread if you found some information during the monitoring or if you need some more guidance.

  • kvnkk Profile Picture
    kvnkk 796 on at
    RE: workflow records are taking long time to process

    first we identify the workflow batch issue (workflow records are taking long time to process) then after few days users are reporting about worklist items issue in EP.

    find workflow instances in the workflow message table(s) with the provided GUID IDs and see if there is some data corruption. Are you aware of possible data manipulation in the database to fix a certain issue?

    R :- we are doing analysis about data manipulations and even our DB also looking at the issue that is there any blockings, log running quires etc...

    If you have a copy of the database in a test environment, you can also pause the batch execution and use the Tutorial workflow processing class while debugging the scenario.

    R:-  yes it can be done. We are working on it.

    Note :- But end of the day all records getting processed with out any issue.

  • André Arnaud de Calavon Profile Picture
    André Arnaud de Cal... 291,280 Super User 2024 Season 2 on at
    RE: workflow records are taking long time to process

    PS: Did the issue start after the actions you mentioned in your other question? See some users unable to view worklist items (EP) - Microsoft Dynamics AX Forum Community Forum

  • André Arnaud de Calavon Profile Picture
    André Arnaud de Cal... 291,280 Super User 2024 Season 2 on at
    RE: workflow records are taking long time to process

    Hi knvkk,

    Your numbers doesn't sound alarming. It could be related to the errors.

    You can try to find workflow instances in the workflow message table(s) with the provided GUID IDs and see if there is some data corruption. Are you aware of possible data manipulation in the database to fix a certain issue?

    If you have a copy of the database in a test environment, you can also pause the batch execution and use the Tutorial workflow processing class while debugging the scenario.

  • kvnkk Profile Picture
    kvnkk 796 on at
    RE: workflow records are taking long time to process

    Can you provide some more details? Are at the end all workflow messages processed? Do you know how many workflow messages are processed per batch run? You can monitor the number of tasks created during the execution. The messages will be bundled with 100 instances per task.

    • Yes, All workflow messages are processed without any issue but took long time some time one hour or two hours to clear the queue

    • There only 4 to 5 workflows having nearly 25 to 30 tasks and reaming workflows are complete in 1 to 5 levels (simple once).

    How many different workflow types do you have? How many legal entities? Do you have workflow configurations, which do have many nodes and are complex?

    • Other than standard we have nearly 25 workflow type

    • One legal entity

    • Complex once only 5(Most used once are 3) reaming are completes with in 1to 5 levels.

    Have you checked with the database administrator if the database is healthy or are there some blocks, long running queries or fragmented indexes related to the workflow tables?

    • Will check with my admin about this.

    Recently we are getting below error messages for some different instances and still we are working on it to track the issue.

    Error1:-

    Workflow Instance ID: 102115 failed to find a bookmark for the workflow message id f9610ac1-c72e-4a58-b141-4712125e4f05 message type AcknowledgeWorkItemCompleted.

    Error2 :-

    Workflow Instance ID: 102403 System.Activities The instance ID provided by the workflow host, 583d9eae-28ad-404d-ad6b-d8e37ae209bd, doesn't match the ID stored in the serialized instance, 3f8809a4-e724-404c-8ede-8b158330d287. A host may not deserialize an instance using a different ID than when the instance was serialized.    at System.Activities.Runtime.ActivityExecutor.OnDeserialized(Activity workflow, WorkflowInstance workflowInstance)

      at System.Activities.Hosting.WorkflowInstance.InitializeCore(IDictionary`2 workflowArgumentValues, IList`1 workflowExecutionProperties)

      at System.Activities.Hosting.WorkflowInstance.Initialize(Object deserializedRuntimeState, DynamicUpdateMap updateMap)

      at System.Activities.WorkflowApplication.LoadCore(DynamicUpdateMap updateMap, TimeoutHelper timeoutHelper, Boolean loadAny, IDictionary`2 values)

      at System.Activities.WorkflowApplication.Load(Guid instanceId, TimeSpan timeout)

      at Microsoft.Dynamics.AX.Framework.Workflow.Runtime.WorkflowInstanceManager.Execute(WorkflowMessage message, Guid instanceId, String instanceNumber, String originator, String submitter, Guid affinity)

  • Verified answer
    André Arnaud de Calavon Profile Picture
    André Arnaud de Cal... 291,280 Super User 2024 Season 2 on at
    RE: workflow records are taking long time to process

    Hi kvnkk,

    Can you provide some more details? Are at the end all workflow messages processed? Do you know how many workflow messages are processed per batch run? You can monitor the number of tasks created during the execution. The messages will be bundled with 100 instances per task.

    How many different workflow types do you have? How many legal entities? Do you have workflow configurations which do have many nodes and are complex?

    Have you checked with the database administrator if the database is healthy or are there some blocks, long running queries or fragmented indexes related to the workflow tables?

    A lot of questions, but they might all be relevant to answer. I have seen a really bad scenario where messages did pile up and project invoicing got delayed as the messages did process too slowly. I did a lot of optimizations to get their situation under control.

Under review

Thank you for your reply! To ensure a great experience for everyone, your content is awaiting approval by our Community Managers. Please check back later.

Helpful resources

Quick Links

December Spotlight Star - Muhammad Affan

Congratulations to a top community star!

Top 10 leaders for November!

Congratulations to our November super stars!

Tips for Writing Effective Suggested Answers

Best practices for providing successful forum answers ✍️

Leaderboard

#1
André Arnaud de Calavon Profile Picture

André Arnaud de Cal... 291,280 Super User 2024 Season 2

#2
Martin Dráb Profile Picture

Martin Dráb 230,235 Most Valuable Professional

#3
nmaenpaa Profile Picture

nmaenpaa 101,156

Leaderboard

Product updates

Dynamics 365 release plans