Personalized Community is here!
Quickly customize your community to find the content you seek.
Latest TechTalk Videos
Have questions on moving to the cloud? Visit the Dynamics 365 Migration Community today! Microsoft’s extensive network of Dynamics AX and Dynamics CRM experts can help.
2021 Release Wave 1Discover the latest updates and new features to Dynamics 365 planned April 2021 through September 2021.
Release overview guides and videos Release Plan | Preview 2021 Release Wave 1 Timeline
The FastTrack program is designed to help you accelerate your Dynamics 365 deployment with confidence.
FastTrack Program | Finance and Operations TechTalks | Customer Engagement TechTalks | Upcoming TechTalks | All TechTalks
First off, you may be shocked to even see this written about, surely it's not necessary these days, with Microsoft hosting in Azure and a Software As A Service (SAAS) provision for Production environments. Right?
Wrong, say a number of larger organisations who have already gone through the pain; I've helped pick up the pieces in many cases, alongside colleagues, since working on "AX7" with Microsoft support in the early days (as it was known then) shortly before general availability. And here's the spoiler, it's not all Microsoft's fault.
Well, there are several things which Microsoft have little or no control over and indeed partners may also have little or no control over after you’ve gone live. Here are some examples of why such performance issues require a combined effort:
This post is primarily aimed at non-technical readers and those responsible for coordinating or managing general performance issues, but with links drilling through to more in depth resources containing further details.
By ‘general performance’, I mean a set of unidentified performance issues across one or more modules, or indeed the entire application.
This is intended to provide a suggested approach drawing on experience, to get you started in what can be a challenging area.
General performance issues can often be highly political and complex, partly due to the combined effort which is required and common misconception that it’s all on Microsoft, as explained above. Performance can also be subjective; it’s a feeling to some extent. As such, collaboration and user perception is key, hence the importance of understanding the issues and setting the right level of expectation from the start. Generally speaking though, the important thing to remember is the same relatively straightforward approach applies every time when analysing these performance issues.
In the most successful performance projects I have seen, there has been a spirit of openness and collaboration towards the shared goal of improving performance, without which people may tend to try to defend their own areas and can be less willing to contribute information if they feel there is a risk in doing so. If you haven’t already, depending on the scale of the issue you may wish to consider setting up a regular conference call to share information, agree and assign actions and review progress, for example on a weekly basis.
These steps are based on typical approaches I’ve followed and seen others follow in similar situations. For the most part, the management of such issues is technology agnostic and therefore not too dissimilar to my previous blog post. Whilst it may be tempting to immediately log an ‘everything is slow’ support case with Microsoft, the engineer on the other end, lovely people that they are, is only likely to ask the same questions as below initially, it’s necessary; in my experience, Microsoft cases can be handled much more expediently and effectively if there are specific facts readily available from the start - especially, as it goes in the famous eighties film: ”Man who catch standard code running slow with latest update accomplish anything” (or something like that).
Right, so enough of the throwbacks, Glen, what should I do?
Setting the right levels of expectation from the start is key to keeping any performance tuning project within scope. I say project here deliberately because general performance issues should be treated as such, including scoping, timelines and allocation of multiple resources. There may be questions like ‘Could [X technical issue] in some way relate to this performance issue?’; bear in mind that while positive contributions should be encouraged, be careful how you use the information – be careful to stick to your original goals and not be sidetracked.
Get a list of processes and validate expected durations for those processes from the end users, i.e. whether they are actually realistic or not based on the underlying logic. If you don’t think it’s realistic, say so, it’s better to have these conversations as early as possible; ask key users to define in business terms what the requirement is and if possible, provide supporting information to go with it. Ideally, get the target defined in terms of volume and concurrent users as well. For example: “With 200 concurrent users, we expect process ‘X’ to take a maximum of 30 seconds and an average of 10 seconds, for a 100 line order. We have calculated this based on the order volume a user in that area would typically need to process to meet their targets.” It relates back to the principle of SMART: Specific, Measurable, Achievable, Realistic and Time related.
Users saying “AX is generally slow” or “‘X’ process is slow” may be valid, but it doesn’t provide enough information on its own to properly analyse the problem; the tools (below) can help but there is nothing better than first-hand information from the users. It can help to stress the importance of their role in this process. This brings us on to point 2.
Ask for further details and if you can’t get the required information by simply asking, try other approaches such as shadowing users while they are experiencing the issues (ok, more difficult right now, but hopefully you’re getting the idea). If you don’t get your answers, keep asking – you may well find that a lot of “noise” simply disappears and some specific issues start bubbling to the surface which you can then begin to address.
Once you do start getting the information though, users need to see that their efforts are worthwhile to offer you continued support (i.e. first hand information), so ensure you at least demonstrate that you are working on it and ensure they are being kept informed (ideally, directly if you can). If you can also achieve some quick wins, even better.
Some examples of questions you might want to ask the end users:
Even if it is described only as a ‘general performance’ issue:
Where an issue is identified with a specific process:
Fortunately for us, Microsoft have provided multiple tools out of the box which can help us with this task and because it’s on Azure and SAAS, there’s no installation required. Happy days (or it will be once these issues are over, I hear you cry)!
Check out this Tech Talk from the awesome Markus Otte and Davy Vliegen in the Microsoft R & D Solution Architect team.
Finance and Operations: Performance Troubleshooting Tools for Dynamics 365 | December 14, 2018
(Edit) Plus these recent posts from Todd Jeska in Microsoft's Premier Field Engineering team:
Wait, what? I’ve installed this thing On Premises! Keep calm, the good old DynamicsPerf and tracing can still help (and Query Store mentioned above is also available On Premises for SQL versions 2016+).
Additionally, if you’re responsible for writing code, you most definitely want to be running the Customisation Analysis Report as it may yield a lot of quick wins before you even start using those tools:
Additionally, if you're working with Dynamics 365 Commerce (previously known as Retail), then take a look at this - with thanks to Chris Bittner (also in the Microsoft SA team) for putting me on to that:
By this point (if not earlier), you should be in a position to formulate an action plan based on:
Keep in mind that the plan may well change as time goes on and further issues/actions are identified, but obviously the key is to have one!
There is less to check these days, assuming you’re in the majority who are using the cloud version of Dynamics 365 Finance (if not that’s for another blog post). However:
You can use the following resources as a guide:
Finance And Operations: Microsoft Managed Continuous Updates | April 2, 2019
Finance and Operations: Regression Suite Automation Tool - Testing Lifecycle Demo | May 29, 2019
Finance and Operations: Regression Suite Automation Tool - Background & Setup | May 28, 2019
This next phase can also overlap to a degree; having said that, it’s important to get as much done at the ‘general’ end first when dealing with general performance issues, to avoid costly and potentially unnecessary additional monitoring / analysis time later.
Performance tuning is iterative, where the cycle includes analysis, corrective actions/tuning, deployment of changes then review.
Even when investigating general performance, as mentioned above you should still get some examples from users of where processes are particularly slow. This is for 2 main reasons:
As with previous versions, you can still analyse traces using the Trace Parser tool:
Review should be a regular part of the process because as mentioned already, performance tuning is iterative; resolution of some fundamental issues may help, but then more can be identified after changes are deployed and the performance tuning is able to become more focused and in depth.
However, at the same time it’s important to be able to recognise when to stop tuning and move on. There is generally a ‘law of diminishing returns’ to be applied here, meaning in each iteration of the performance tuning of a specific process, you would expect the potential for improvement to reduce exponentially. Some kind of exit criteria should be applied (and predefined as early as possible). For example, you may have simply reached the target duration agreed with the end user or some kind of cost / benefit decision was made, such as:
As well as reviewing performance fixes that are deployed to address customisation issues, consider QA (quality assurance) processes and having at least best practice checks put in place for performance of every code deployment. One benefit of this is that the customer can feel more reassured that any downturn in performance is not seen to be a result of a recent code deployment. Other things I have seen (following a similar principle) are: putting all other code deployments (i.e. anything other than performance fixes) on hold during the period of performance tuning; being prepared to reverse deployments if there is any doubts over whether or not they caused a performance issue.
Review what could have been done pro-actively that can be applied in future to avoid the issue happening again and plan to have it in place on every implementation project. Have another look at the Fast Track checklist and the following Tech Talk, to establish what could have been missed and how.
Finance and Operations: Performance Benchmark for Dynamics 365 | January 30, 2019
Don’t lose heart, break your problem down into smaller pieces or if it's all a bit too overwhelming, you can always find a good partner like QUANTIQ to guide and support you.
Business Applications communities