What is the Best Tool for Dynamics 365 Performance Testing?

Objective

Have you ever asked the questions below when considering what tool you should use for Dynamics 365 Performance testing?

"What is the best tool to automate performance testing?"
"How can I choose the right tool in the market for my specific needs?"
"Should I develop my own tool rather than purchase a tool for performance testing?"

Performance testing is mandatory and an important aspect of Dynamics 365 development and maintenance, as it helps to ensure that Dynamics 365 performs well under expected and peak load conditions. Once you understand the importance of solution performance testing, the next decision is which tool to use.

It is important to choose the right tool for your specific testing needs, based on the characteristics of the system under test and the performance criteria that you need to measure. In some cases, it may be necessary to use multiple tools in order to fully test the performance of a system. The right performance testing tool can help you reduce the testing cost and bring up testing quality, as well as validating the performance of the system and helping to identify bottlenecks should they exist. . The purpose of this blog is to help customers and partners identify the best tool for their requirements.

Requirement first, selection second

Every project is unique, and its performance goals and objectives vary. There are several factors to consider when evaluating a performance testing tool, and it is crucial to consider as many scenarios as possible. Essential factors to consider include total cost, testing options for users, and support. Hence, there is no standard answer to: “What is the best tool for performance testing?”. There is no one-size-fits-all solution for all testing scenarios, and you need to choose the tool based on the project's specific requirements. The first step is to understand the business requirements and application architecture, and then select performance tools that meet the performance objectives.

FastTrack presented a series of Performance Testing in Microsoft Dynamics 365 TechTalk Series to help you define and execute your performance testing strategy. The series provides guidance on identifying the appropriate tool for each project. Before deciding on a performance testing tool, it is important to fully comprehend the business needs and processes, and then determine the necessary tools and techniques for each step.

For example, in a procurement operation scenario, there are various steps involved, each with its own specific requirements and simulations. By determining the type of each process (such as integration, user interaction, or batch job), you can search for tools that fulfill the technical needs for that type of process. This can aid in selecting the most suitable performance testing tool for your requirements.

Figure 1 – Tools required in the scenario.

In Process #1, "The Planned Order Interface Integration," there is a requirement for an external system to push datasets to the system and generate planned orders based on the pushed data. To simulate this process, the following steps are involved:

1. Creating the dataset.
2. Pushing the created dataset to the system.
3. Generating the planned order in the system.

To address these requirements, decide to develop a console program for generating the required data, a load test project in Visual Studio 2019 for sending data to the system using a multi-threading process to maximize throughput, and a task scheduler for automating the integration on a specified schedule. Finally, the master transactional data are stored in an SQL database for maintenance.

In Process 2, "Planned Order Submission," it primarily involves user activity. Upon completion of Process 1, thousands of planned orders are generated in the system, and end users are required to access the system to submit the planned orders. To simulate this, we need to capture the end-to-end flow from accessing the system through to completing the submission, then convert it to a testing script and use a task scheduler to trigger the process automatically on a schedule.

Once the scenarios and processes are defined, it is easier to decide on the tools and technical requirements for each process and present an overview in a table (refer to Figure 1). For the Purchase Order scenario, we need a total of 7 tools/technical solutions.

1. Console program
2. Load test in VS
3. Task scheduler
4. SQL database
5. Tool to emulate multiple users (e.g. JMeter)
6. SQL agent
7. System default batch job

In conclusion, it is crucial to thoroughly evaluate your automation requirements and select a tool that satisfies those needs in the most effective and efficient manner. By considering these and other relevant factors, you can ensure that you make the appropriate tool choice for your objective.

Manual performance testing and automated Testing

Performance testing for Dynamics 365 usually involves simulating a particular workload with hundreds or, in some cases, thousands of concurrent users, based on business requirements and the agreed-upon transaction volume and performance objectives. Utilizing a performance testing tool prior to deploying Dynamics 365 can lower the risk of potential problems and enhance the overall system quality and reliability. These tools aid in simulating various workloads and evaluating system performance under different scenarios, helping to detect and resolve any issues before they impact users. This results in improved user experience, stable and dependable systems, and decreased risk for the company.

"Is it possible to conduct performance testing manually?"

Yes, it is possible in less complex implementations with simple scenarios, however, it is important to review each scenario as some scenarios may require a customer to simulate hundreds of concurrent users with multiple steps coordinated at the same time? For example, consider the scenario where a customer needs to simulate 500 users creating1,000 purchase orders within 5 minutes, running concurrently other users need to approve the purchase orders to complete the process. If you plan to conduct performance testing without a tool, it means you would need to coordinate all the users to perform the testing scenarios at the same time. This can be very costly and not efficient especially if multiple rounds of performance testing are required.

Closing

This article aims to highlight that there is no one-size-fits-all performance testing tool for Dynamics 365. Various tools are designed for different purposes and cater to different workloads, protocols, and performance standards. There are numerous other performance testing tools available, and selecting the right tool for a project will depend on the project's unique needs and requirements. To determine the best fit, it may be helpful to evaluate multiple tools and compare their features and capabilities. Although the objective of automated performance testing is to minimize manual testing, it is crucial to note that complete automation may not be feasible, but the goal should be to reduce as much manual testing as possible.

Due to the presence of numerous tools in the market, it is highly recommended to conduct a Proof of Concept (POC) exercises to determine the most suitable toolset.