SBX - Search With Button

SBX - Forum Post Title

AX Batch jobs get stuck (AX2012 R1 CU5)

Microsoft Dynamics AX Forum

Willem v. Duren asked a question on 24 Apr 2014 7:04 AM
My Badges

Question Status

Verified

Hello everone,

I have a strange problem for a longer time which I can not see to solve (AX2012 R1 CU5)

Some batch jobs get stuck, which I have to cancel manually before other batch jobs are processed.

It concerns:

- "Create a scheduled task that will execute the batch transfer of subledger journal entries."

- "Named user license count reports processing"

Both last succesful 18-03-2014 23:00

When I manually cancel them batch processing continues.

- Automatic role assignment works fine

- Some other batch processes work fine

The conclusion is that our AOS (batch) server works fine.

I have a 3 tier setup. I tried to debug but it looks like the batch job is never really started. The batch jobs do not fail, it just hangs for days saying "executing", and debugger does not jump in.

Our system also does not generate SEPA files anymore (also generated by batch process).

What would be the right approach? Anyone can put me in the right direction ?

Any help is appreciated.

Kind regards, Willem.

Reply
Suggested Answer

Hi Willem,

When something is stuck in the workflow you could monitor the SQL database for deadlocks. I have seen stuck batches due to deadlocks before.

Reply
Willem v. Duren responded on 24 Apr 2014 8:59 AM
My Badges

Hi André, would that make sense if it only affects two batch jobs ? I just checked the log files of the SQL but couldn't find anything strange. I will google tonight to see how I can monitor for deadlocks.

Thank you for your help.

Reply
Rudhra Kumar responded on 24 Apr 2014 9:35 AM
My Badges
Suggested Answer
Willem v. Duren responded on 24 Apr 2014 1:41 PM
My Badges

Hello Rudra! Thanks again for your help.

I tried the SQL trace in AX, but no deadlocks found. Also in SQL logs no deadlocks. The SQL trace in AX show all kinds of SQL statements made, like: 

100x:

"SELECT TOP 1 T1.NAME,T1.TYPE,T1.USECOUNT,T1.MODIFIEDDATETIME,T1.CREATEDDATETIME,T1.RECVERSION,T1.RECID FROM SYSUTILELEMENTSLOG T1 WHERE ((TYPE=?) AND (NAME=?))"


"SELECT OWNERGROUP, OWNERCHILDNAME, RELATIONNAME, EFFECTIVEACCESS, SYSMANAGED, MANAGEDBY, OBJECTGROUP, OBJECTNAME, OBJECTCHILDNAME, KERNELTYPE FROM [dbo].[ModelSecurityPermission] WHERE OWNERHANDLE = ? AND OWNERLAYER = ? AND OWNERGROUP = ? AND OWNERCHILDNAME = ? ORDER BY RECID"

"SELECT TOP 1 T1.RECID FROM SYSFILESTORE T1 WHERE (BATCHJOB=?)"

"SELECT TOP 1 T1.BATCHJOBCANCELED,T1.BATCHJOBENDED,T1.BATCHJOBERROR,T1.BATCHJOBID,T1.USERID,101090 FROM BATCHJOBALERTS T1 WHERE ((BATCHJOBID=?) AND (CREATEDBY=?))"

Etc.

Batch process still freezes.

 

Seems to be a tough one...

Reply
Rudhra Kumar responded on 24 Apr 2014 10:40 PM
My Badges
Suggested Answer

Could you please increase the maximum buffer size in the server configuration utility to troubleshooting on the same.

Is it possible to split the batch job into two to the check the freezing issue.

Reply
Willem v. Duren responded on 25 Apr 2014 12:10 AM
My Badges

Hi Rudra,

My current buffer size is 25. What do you recommend? Can I just do that in the production environment to test and afterwards change it back? Or will the changing back lead to problems ?

I am not sure how to split the batch into two. They are system classes.

Kind regards, WIllem.

Reply
Rudhra Kumar responded on 25 Apr 2014 1:46 AM
My Badges
Suggested Answer

Max Buffer Size should not exceed 27 as per MS recommendation.But for testing purpose we can double the size and check then revert it back, we can do in production or test environment and this will not affect anything but when the buffer size increases then overall performance decrease this is the formula given by MS

(Note :- When you change the buffer size then AOS service asks you restart the services so check the online users and transaction if any at that time then you can change the buffer size)

technet.microsoft.com/.../aa569634.aspx

Splitting of Job is like you can splitup the criteria's you were passing or functionalities of the existing job.

Reply
Willem v. Duren responded on 25 Apr 2014 4:13 AM
My Badges

OK! Helps a lot.

I just set it to 40. The batch job is executing for 40 minutes now, seems like it's hanging again. When a batch job is executed that normally hangs, the batch jobs that do not normally hang and are executed also hang ("executing"), Also when I cancel the first job that always hang. Strange behaviour...

Reply
Rudhra Kumar responded on 25 Apr 2014 4:24 AM
My Badges

oh..is your batch job getting completed without any issues after a 40 mins but till it completes it will be in hang state, once completed it will come back to normal or something strange lik still hang

Reply
Willem v. Duren responded on 25 Apr 2014 11:57 AM
My Badges

Hi Rudra,

I just checked. The batch is still "busy" so it hangs, even with buffer size 40.

I am thinking, is it an idea to manually run the class without batch to see if it runs then ? Using a Job for example ?

Reply
Rudhra Kumar responded on 25 Apr 2014 12:09 PM
My Badges

Yes you can try running classes one by one will make you get completed but you can try to compile all clases whichever is used by your batch job if it is getting hang then we need to reindex the tables involved in your batch job if data is populated with huge numbers then we are in need to create a new index for tat tables.

Did you try to split your existing job criterias into two or three jobs to avoid the load with one job?

Also could you brief about ur existing batch job?

Reply
Willem v. Duren responded on 25 Apr 2014 12:24 PM
My Badges

Update:

I just tried.

- SubledgerJournalTransferController gives an error (Object not initialized) on line 42 of the run class:

    map = SubledgerJournalTransferRule::getDocumentTypesAndSummarizationSetting(subledgerJournalTransferRequest.parmledgerId(),subledgerJournalTransferRequest.parmSourceDocumentType());

- SysUserLincenseMiner worked for 10 minutes, looked like my AX environment was freezing, and then gives the error "Error in getting SID"

- A Batchjob that normally works (SysSecurityDynamicRoleAssignment class) works without errors as expected.

A batch does not give any arguments right ? In other words, a batch job also runs the class.run() method ? Or does it pass any arguments ? (company for example?)

I am testing one class per time, for example:

static void WillemBatchJobTest25042014(Args _args)

{

    SysUserLicenseMiner SysUserLicenseMiner;

   SysUserLicenseMiner =new SysUserLicenseMiner();

   SysUserLicenseMiner.run();

}



Now I think of it: both batch jobs that are not running are made bu user "Admin". Could that be the cause? The "Error in getting SID" message means something is corrupt with the executing user I guess? Is that the service account ?

Reply

Hi Willem,

Did you apply a change on the admin user recently? You can check the SID in AX stored with the user and compare ths one with the SID of the admin when this user is logged in in Windows. The SID can be seen e.g. in de registry editor. I don't know if there are some other tools for it.

Reply
Rudhra Kumar responded on 26 Apr 2014 12:21 AM
My Badges
Verified Answer

Hi Williem,

Please follow the below to get the SID value for the user your getting trouble, also double check the current SID is used for the admin user in the AX database "userinfo" table-->"SID" column 

To get SID for the current logged in domain user, please type the following in your command prompt--> C:\> whoami /user

To get SID for all domain users, please type the following in your command prompt--> wmic useraccount get name,sid

Also refer the below link for your reference:-

http://technet.microsoft.com/en-us/sysinternals/bb897417.aspx

http://www.windows-commandline.com/get-sid-of-user/ and http://pcsupport.about.com/od/registry/ht/find-user-security-identifier.htm

Reply
Willem v. Duren responded on 26 Apr 2014 11:17 AM
My Badges

Now we are getting somewhere. I just checked and indeed someone deleted our "Admin" (named "AXadmin" in active directory) account. Probably to save SBS licenses.......

I cannot make a new user and change the SID of that user to the old SID, because it is auto generated. So I will have to update the user in AX (in the SQL database).

I will make a new user and update the SID in SQL next Monday and let you two know the outcome.

PS; Can't I just update the SID of the AX Admin account to the sid of my own admin user ? Or doesn't  AX accept double SID's for different users ? 

I keep repeating it, but really: thanks for your help so far!! You will hear from me on Monday pr Tuesday (I will have to catch our system administrator for the active directory).

Reply
Rudhra Kumar responded on 26 Apr 2014 12:21 AM
My Badges
Verified Answer

Hi Williem,

Please follow the below to get the SID value for the user your getting trouble, also double check the current SID is used for the admin user in the AX database "userinfo" table-->"SID" column 

To get SID for the current logged in domain user, please type the following in your command prompt--> C:\> whoami /user

To get SID for all domain users, please type the following in your command prompt--> wmic useraccount get name,sid

Also refer the below link for your reference:-

http://technet.microsoft.com/en-us/sysinternals/bb897417.aspx

http://www.windows-commandline.com/get-sid-of-user/ and http://pcsupport.about.com/od/registry/ht/find-user-security-identifier.htm

Reply
Verified Answer

Hi Willem,

The network alias and SID are checked in AX. So only changing the SID with your SID is not working. A dedicated Admin account would be the best.

you can update the network alias and the SID to the new information. you can also empty these two fields and then start AX directly with this new iser account. Then the credentials of this user will be used for the admin user in AX.

Reply
Suggested Answer

Hi Willem,

When something is stuck in the workflow you could monitor the SQL database for deadlocks. I have seen stuck batches due to deadlocks before.

Reply
Rudhra Kumar responded on 24 Apr 2014 9:35 AM
My Badges
Suggested Answer
Rudhra Kumar responded on 24 Apr 2014 10:40 PM
My Badges
Suggested Answer

Could you please increase the maximum buffer size in the server configuration utility to troubleshooting on the same.

Is it possible to split the batch job into two to the check the freezing issue.

Reply
Rudhra Kumar responded on 25 Apr 2014 1:46 AM
My Badges
Suggested Answer

Max Buffer Size should not exceed 27 as per MS recommendation.But for testing purpose we can double the size and check then revert it back, we can do in production or test environment and this will not affect anything but when the buffer size increases then overall performance decrease this is the formula given by MS

(Note :- When you change the buffer size then AOS service asks you restart the services so check the online users and transaction if any at that time then you can change the buffer size)

technet.microsoft.com/.../aa569634.aspx

Splitting of Job is like you can splitup the criteria's you were passing or functionalities of the existing job.

Reply

SBX - Two Col Forum

SBX - Migrated JS