AX 2012 R2 - Unwanted massive project stages modification

(0) Share

Report

Posted on by DavidGermain1980

330

Hello!

We're currently using AX 2012 R2 since more than a year, and we did face an unusual issue recently for the second time, months apart.

By an unknown trigger, the project stage for all the projects in the system is modified to Completed. We have thousands of projects and subprojects, from internal Time projects to billable T&M projects, and everything was modified to Completed. No more transactions (journals, timesheets) were possible because of this massive change, and we rushly had to generate a script to change the project stages to In process to allow the operations to resume.

Our assumption: the change on a single project stage did trigger and unwanted batch that did the same modification on all projects. There is no way in AX to process a batch to change massively the project stage, unless the subprojects from a parent project, so we don't understand.

Is it something that happened somewhere else? I can't find any other information about a similar issue.

Thanks,
David

*This post is locked for comments

I have the same question (0)

All responses (19)

Answers (1)

Brian Welcker on at

Like (0)

Report

Sorry this happened to you. I have never seen this behavior nor had it reported by any other customers. We don't have any batch processes to update project stages (parent / subproject modifications are done interactively) so I'm not sure how this happens. It would be good to get the SQL statement that performed the update but it sounds like you don't have a repro.

Do you have any customizations that modify the stage on the project?

Was this reply helpful? Yes No
DavidGermain1980 330 on at

Like (0)

Report

Thanks for this reply Brian.

We don't have any customizations that work on the project stage, so I don't think it's coming from this area.

I'll take a look with my team (Infra / DBA) to see if we can have any clue on the SQL database.

Was this reply helpful? Yes No
DavidGermain1980 330 on at

Like (0)

Report

Below is a screenshot of a request that was found by my DBA in his monitoring tool.

It is the request that triggered the massive project stages modification.

Can it be helpful?

Thanks,
David

Was this reply helpful? Yes No
Sandeep Chaudhury 3,303 on at

Like (0)

Report

Hello David,

I concur with Brian here.

I have never seen this behavior with any of the customers so far.

Is there any integration points with your AX application, which creates projects in AX? or Do you just manually create and manage projects and sub-projects in AX ?

I believe a "Database Logging" might help you to watch it for the next time, as you mentioned it already happened two times now. (But, this will have additional burden on the memory and resource)

Was this reply helpful? Yes No
DavidGermain1980 330 on at

Like (0)

Report

Hi Sandeep,

Yes, we have a tool to upload projects in AX, and of course it's a customization. We'll take a look at this mod to investigate; thanks for the idea.

However, we're not using this tool on a daily basis, only for big imports, and outside of working hours, so the trigger didn't come directly from this mod. Anyway, we'll take a look nevertheless.

Thanks for your reply,

David

Was this reply helpful? Yes No
Brian Welcker on at

Like (0)

Report

I've sent the trace to one of our engineers to see if he can identify it - I'll let you know what he finds. In the meantime, if you are able to isolate the issue, you should open a support incident. I am still guessing it is from some customization.

Was this reply helpful? Yes No
DavidGermain1980 330 on at

Like (0)

Report

Thank you Brian for the follow-up.

Regards,

David

Was this reply helpful? Yes No
Brandon Ahmad 2,465 User Group Leader on at

Like (0)

Report

You aren't imagining this. I've seen it also on an implementation. After nearly a year of using AX, this suddenly happened to one of my clients. My clients swore that no one had did this, and didn't know how the problem occurred.

I couldn't reproduce the problem, but I had a suspicion that it was related to all projects being checked when they changed the staging.. Something there.. but I couldn't reproduce it.

Unfortunately or fortunately, this only happened once and my client wasn't tied in enough to the Production Module for this to affect postings. So, I just changed all the project stages back -- though time consuming.

I would have needed to see what the client click-through sequence was that led to these events by implementing a client trace, and it wasn't occurring enough for me to do that -- at least as of now.

If I could get the event to repeat itself 2 or 3 more times, I think that I could figure out. Let us know if you see it again.

Was this reply helpful? Yes No
DavidGermain1980 330 on at

Like (0)

Report

Thanks Brandon, I'm happy to see we're not alone.

It is effectively impossible (as far as I know) to reproduce it. The only other hint that we have is a server that went down in the hours before the problem.

On our side, we're working on an action plan to fix the project stgaes rapidly if it happens again.

Was this reply helpful? Yes No
Community Member on at

Like (0)

Report

This just happended to us today! A user wanted to change project stage from Created to In process, and all project's stages were changed to In process (we have some 25000 projects). We use AX2012R2 CU7 - and have been in production for half a year now.

Luckily we had database log on ProjTable (and therefore could exactly pinpoint who and when), and the user told me that all he did was "... open the project, and change stage to In process - it froze AX for some 10 min, and then I shut AX down". (Another thing is, he now says the project he did it on is gone... Have to look into that tomorrow)

However, we did a "custom" rollback from the database log with a .doUpdate() on ProjTable setting Status to its old value - maybe not the most elegant solution.... But I'm more worried that this could happened in the first place.

Was this reply helpful? Yes No