Finance | Project Operations, Human Resources, ...

Performance unacceptably slow 3 months post go live with D365 F&O; Partner / Microsoft not fixing

(2) Share

Report

Posted on by ZL-21052041-0

We are experiencing significant performance slowness across our D365 F&O application and need help. For example, when we open a sales order with one line, it can take over a minute to open. In addition, there are times when changing the warehouse on a sales line can take over 1 minute. Also, the order confirmation and invoicing process can take a very long time and even time out. We have done tickets with Microsoft, talked to our FastTrack people and had our implementation partner working every angle to address this. We have gone through multiple tickets and provided updated Usage data in the excel template. We were also told that only the license count determines what resources are allocated to our environment, so why does the transactional volume from the usage profile even matter? We are willing to spend more money on licenses if we know that will help but no one can tell us how many we should get or if it will definitely help. When the customer service person showed us the AOM and Batch counts for the next "tier" they were the same as the tier (L2 and L3 in screenshot) we are in, so do they have more juice!? We are also going to try and find a consulting partner that focuses on performance if anyone has suggestions.

We are a mid-size business with 11 locations in 10 US States and have been live in D365 for 3 months. We have a number of integrations including Avalara for tax, Red Maple for CC, True Commerce for EDI among others. We also have 4 sales order import integrations including our ecommerce website, an OCR platform, a DMF import, and EDI(True Commerce). Also please note that we are using global unified pricing which based on our understanding has known performance issues. We only have 50 enterprise licenses even though we have 150 users. Most of our users that do not have a "full" license use but use advanced warehousing on handhelds in the warehouse. A significant portion of our sales orders are imported through one of those integrations and sometimes we have orders with 200-300 lines that can take hours to import. We do about 500 orders per day and about 400 invoices. We have an integration with Pacejet for freight out and currently invoice at the time of shipment. We have about 10,000 customers and 50,000 parts. We also use the configurator for custom assemblies and create parts on the fly we call "NCIs". I always say we do a little bit of a lot because we have so many different things that we have implemented.

Has anyone experienced similar performance issues? This is crippling our business because we are waiting for the system constantly.

TLDR. We have serious performance issues and can't get an answer on what to do to fix it.

Categories:

Dynamics 365 Finance

All responses (5)

Answers (0)

ZL-21052041-0 13 on at

Like (0)

Report

Performance unacceptably slow 3 months post go live with D365 F&O; Partner / Microsoft not fixing

Thanks Anton and Navneeth. The detail in your responses in addition to Martin's has given us multiple opportunities to identify and resolve the issues with our performance. I will respond to this thread or contact you all directly with follow ups. In addition, we are using Yammer now to discuss detailed technical issues.
Suggested answer

Navneeth Nagrajan 1,794 Super User 2025 Season 1 on at

Like (1)

Report

Performance unacceptably slow 3 months post go live with D365 F&O; Partner / Microsoft not fixing

Hi ZL-21052041-0,

Looks like there will be remediation steps that will be required to resolve these issues.

1. Usually these issues arise because of data base loads and too many custom indexes being created on certain tables in the database. You can get data from the performance team within Microsoft that could give you the following:

- Slow performing screens (User interfaces)

- Throttling interfaces

- Slow performing tables and indexes. Even the usage of indexes can be determined through the performance team.

Sales order lines turn around times are a serious concern. This is definitely a slow performing screen. Check the production environment configuration (CPU processing cores, database tiers) with Microsoft.

2. With Customer V3 or V2 would recommend removing all the unwanted and non-required localization fields from the integration schema. There are more than 40 fields in that entity which do cause performance issues especially when mapped to an integration layer. Would recommend staying with Customer V3 entity (considering platform upgrades and schema changes, if any that come up in the future).

3. Flights being turned on in D365 F&O database - Performance team or Advanced cloud engineering team (ACE) team can definitely help you resolve this issues. Have flights turned off if they are not being used or if there are flights enabled through Feature management. Feature management features like

4. OData calls throttling is not a surprise for pricing or inventory data. Would recommend moving away from OData and move into something like recurring integrations with Azure Logic Apps through custom web services. (https://learn.microsoft.com/en-us/dynamics365/guidance/techtalks/integrate-finance-operations-overview).

5. Batch optimisation - It is a good approach where batches are prioritised as critical, normal and low, however recommend you to have separate batch groups for various categories of critical batch jobs, normal batches etc. For example, if there is a batch job running for updating prices of products through a legacy system and if there is another batch running for order processing and invoicing and if both are marked as critical and are heavy batches then recommend two separate batch groups marked as critical.

Hope this helps. Happy to answer questions, if any.
Anton Venter 19,959 Super User 2025 Season 1 on at

Like (1)

Report
Performance unacceptably slow 3 months post go live with D365 F&O; Partner / Microsoft not fixing

Hi,

Performance issues can sometimes be challenging to pinpoint and solve. In my experience, the following are mostly responsible for poor performance:

Using unoptimized queries, missing indexes, etc.

Using row-based operations instead of set-based operations.

Code not optimized for performance.

Because the environment is running in the cloud, if there is high demand for computing resources by other cloud customers, your environment's resources can temporarily be reduced. This could explain why you are experiencing spikes in poor performance in standard Dynamics 365 FSCM functionality. Moving your Dynamics 365 FSCM environment from the cloud to on-premises might be an alternative to consider. I have not compared running in the cloud with on-premises, so I can’t comment much on it.

I don’t think moving to the next licensing tier will solve all performance issues. Because you have a complex Dynamics 365 FSCM environment, someone will need to investigate each bottleneck in depth to determine the root cause. Importing an order with 300 lines should definitely not take hours to import. It should not take longer than a few minutes or so. You mentioned OData; OData is notoriously slow, and I strongly recommend (in most cases) using the Data Management Framework instead for recurring integrations, especially for bulk data.

If standard functionality, like opening a sales order, is performing poorly and there are no related customizations involved, Microsoft should investigate and resolve any issues. In my experience, standard Dynamics 365 FSCM performs at acceptable levels overall in most cases.

My experience with support tickets is that escalation through the partner is often required to achieve better results. The "higher" you can escalate, the better.

Similarly, with ISV solutions, if there are performance issues, the ISV should investigate and resolve them.

If custom functionality is performing poorly, the customizations should be investigated and reviewed in depth to pinpoint the performance bottlenecks.

Wherever possible, plan and schedule batch jobs to run outside office hours as much as possible.

The chances of this are low, but have you ruled out office network bandwidth bottlenecks or other networking issues, such as VPN connections or DNS? Have you tested the system performance from other locations, like from your home or using your mobile phone’s Wi-Fi connection? Which browser are your employees using? Could the browser be causing the performance issues? Is there some kind of third-party add-in running in the browser?
ZL-21052041-0 13 on at

Like (0)

Report
Performance unacceptably slow 3 months post go live with D365 F&O; Partner / Microsoft not fixing

Thanks Martin. Good points on the more detailed analysis and understanding of the performance. Regarding the performance in different environments. Yes, this is an issue in our test/dev enviros, but those are tier 2, so we didn't know if that was a fair comparison. There has been quite a bit of talk about the tiers of different instances and how that may or may not impact performance so any guidance on that is helpful. We don't have nearly the transactional volume or user activity in test.

I am the CFO of our business and have a background in ERP implementations and integrations but not especially knowledgeable in technical fields.

We have received 6 points that Microsoft mentioned to us as impacting performance. We have been working to implement updates to address each of them. Some of them we have already implemented and have not seen much of a change. We still see spikes with very poor performance (screens locked for 10+ minutes) and our technical consultants have used LCS and telemetry to investigate. We have seen the spikes in AOMs and that other AOMs may be underutilized at that time. Most of the time 3 will be at capacity and one hardly used during the worst slowdowns. The 6 points are below:

The work id counting (already resolved) - this was custom code to count the number of work IDs that a previous implementation partner did and we have removed.

He pointed out that we are using Customer V3 and that it is not recommended. We use this for integration with our ecommerce platform to make sure the databases are in sync. We are exploring changing to V2.

ODATA calls are throttling. He recommended using async over sync for ecomm except for pricing/inventory (things you need live calls on). Move sales order entry to async this is in progress but a big change

The SELECT statement for pricing is sometimes timing out after running for 30 minutes - He recommended getting with unified pricing expert but Microsoft has a flight on our sandbox/test that they put in so hopefully this is a big help. We have sent dozens of traces related to this.

Batch prioritization – everything was set to normal and we have adjusted this

Product Search Performance - This is an ISV solution that we have removed
Martin Dráb 234,037 Most Valuable Professional on at

Like (1)

Report

Performance unacceptably slow 3 months post go live with D365 F&O; Partner / Microsoft not fixing

You can check LCS to see if the existing application servers are under heavy load. If not, adding more of them won't make a big difference.

Performance problems are typically related to database, but a wrongly implemented integration, for instance, could cause such problems as well.

You haven't mentioned any results of the analysis by your partner and Microsoft. At this point, you should know better than just that opening of the sales orders form takes a minute. You should know the part of code where the time is spent, how much time takes the query itself (= whether it's the culprit or not) and such things.

Do you have the same problem in non-production environments? If so, you know that it's not a problem with a particular environment and you have much more options for analysis. You can access the database, therefore you can monitor it, maintain indexes, try different execution plans and so on. You can also debug code, for instance.

If it's specific to production, you don't need to look for a problem in code (because it works fine in elsewhere), but you depend on Microsoft a lot, because you can't access the database. But you still have access to logs in LCS, you get query store data when copying the production database, you can create traces and so on.