Part 2 - Dynamics 365 finance and operations apps performance testing with JMeter - Execution
Like
Share
ReportIntroduction
In Part 1 of this blog series, we looked at Apache JMeter as an alternative tool for conducting performance testing in Dynamics 365 finance and operations apps. By leveraging JMeter and its test script generated through the Correlation recorder, users can efficiently perform both single-user and multi-user testing, ensuring the application's robustness and reliability under varying workloads. Please note that the following approach has been tested and is applicable in Sandbox (T2+) environment.
Configure a development environment
To begin our performance testing journey with JMeter for Dynamics 365 finance and operations apps, we will first focus on setting up the development environment for scripting and testing. One of the key advantages of using JMeter is that it eliminates the need for a Dynamics 365 finance and operations apps development machine in the entire performance testing.
1. Begin by downloading the JDK execution file from the official website and follow the installation instructions provided.
2. Install the JDK execution file.
3. After installing Java JDK, verify if the necessary environment variables have been added.
JAVA_HOME=C:\Program Files\Java\jdk-1.8 (depended on your Java installed path)
-
4. Open Command Prompt and type Java --version or Java -version to check the installed version.
-
5. Next, download the JMeter files apache-jmeter-5.5.zip. (Showcased in this blog post)
-
6. Once the JMeter files are downloaded, proceed to unzip the contents into a directory of your preference (e.g. C:\apache-jmeter).
Install Plugin (bzm – Correlation Recorder)
To enable efficient correlation recording in JMeter, follow these steps to install the necessary plugin.
-
1. Begin by downloading the Plugin Manager and place it in the JMeter directory's lib\ext folder.
-
2. Go to the \bin folder and run jmeter.bat as an Administrator.
-
3. Once the JMeter GUI is launched, go to the Options and select JMeter Plugins Manager.
-
4. Within the JMeter Plugins Manager, locate the Available Plugins section and search for the BlazeMeter - Correlation Recorder Plugin. Proceed to install it to facilitate correlation recording. (For this blog post, we will demonstrate using Correlation Recorder version 1.2.2. However, you may download the latest version if available.)
-
Installing the JMeter CA certificate for HTTPS recording and Configure your browser to use the JMeter Proxy
For this demonstration, Firefox was used to record the scenario. However, feel free to choose Edge or any other browser you prefer. To enable secure recording in Firefox, you need to set up the CA Certification. Refer to here for detailed instructions on configuring CA Certification.
-
• When you launch JMeter for recording purposes, the JMeter certificate is automatically generated or updated. This certificate is essential for recording secured applications in Dynamics 365.
-
• Keep in mind that the root CA certificate remains valid for 7 days from the date of generation. If the certificate expires, you can remove the old certificate and import the newly generated one.
-
• You can locate the JMeter root CA certificate in the \bin folder of your Apache JMeter installation.
To facilitate recording, we need to set up a Proxy server in Firefox. The port number provided in Firefox should match the Correlation Recorder's port number. Refer to here for step-by-step instructions on configuring the Proxy server in Firefox or other browsers.
Development of Test Script
Utilizing JMeter Templates for Scenario Recording
JMeter offers a convenient set of templates that come pre-configured with essential elements for scenario recording. We will use the bzm - Correlation Recorder template, which we installed during the pre-requisite process.
Go to the \bin folder within the JMeter directory, and locate the jmeter.bat file. Run it as an Administrator to launch the JMeter. Once the JMeter UI is open, go to the File menu and select Templates. From the list of templates available, choose bzm - Correlation Recorder and click on Create.
Give it an appropriate name (e.g. CreatePOAndSubmitWorkFlow) and save it.
Configurations for Recording D365 F&O Scenarios
Before recording scenarios in JMeter for Dynamics 365 finance and operations apps, some essential one-time configurations are required for each Test plan. However, you can copy and reuse your Test plan (.jmx) for other scenarios, making the process more efficient.
User Defined variable
To begin, declare the host name and scheme in the variable list. You have the option to include a description for better clarity.
| Name | Value |
| host | <company>.sandbox.operations.dynamics.com |
| scheme | https |
| tenant | <GUID of tenant> |
HTTP Request Defaults
Declare the default protocol (HTTPS) and server name (F&O Sandbox environment) in the HTTP Request Defaults. This settings value based on User Defined Variables for all HTTPS requests within the Test plan.
- Protocol [https]: ${scheme}
- Server Name or IP: ${host}
bzm - Correlation Recorder
As previously mentioned, we will be utilizing the Correlation Recorder to record scenarios in Dynamics 365 finance and operations apps. Proper configuration in the correlation recorder is crucial for accurate recordings. The recorder acts as an HTTP(S) proxy server, capturing the interactions between JMeter and the application.
Global Settings
-
• Port: 8888 or to be same as the browser proxy port settings
Test Plan Creation tab
-
• Target Controller: leave default Use Recording Controller
-
• Grouping: leave default Do not group samples
-
• Capture HTTP Headers: Checked
-
• Regex Matching: Checked
-
• Type: Java
-
• Use Keep-Alive: Checked
-
• Follow Redirects: Unchecked
Requests Filtering
Download the filters template DefaultRequestFilters.txt from here. Copy all filter rules from the text file (Ctrl + C). In JMeter, go to the filters section, and click on Add from Clipboard to paste the filter rules. This template helps exclude unnecessary URLs, optimizing the recording process.
Correlation Rules
We have prepared a template of correlation rules, these are the rules that will determine what information will be extracted and replaced from each response and request message when recording scenarios in Dynamics 365 finance and operations apps.
-
1. Download the correlation rules template file (D365Login_correlation_template-1.0.2.json) from here.
-
2. Move the downloaded template to the correlation-templates folder within the JMeter directory \apache-jmeter\correlation-templates\.
-
3. In the JMeter, go to Correlation Recorder, access the correlation rules and click on Load template.
-
4. In the Templates Manager, select the Available tab, choose the downloaded D365Login_correlation_template-1.0.2.json, and click Install. (Restart JMeter if you cannot find the downloaded template).
-
-
Noted: If you see a blank page in the Installed or Available tab, and your JDK version is higher than 13, you may need to uninstall it and install JDK 13 instead. Newer JDK versions may affect the Templates Manager.
5. In the Templates Manager, go to the Installed tab, select D365Login_correlation_template-1.0.2.json, and click Load.
Noted: Please enable the "Enable Correlation (Legacy)" option in the latest version (e.g. V2.1) to achieve a similar experience to the older version of Correlation Recorder.
View Result Tree Configuration
Under the bzm - Correlation Recorder section, set the desired file path for the results location in the View Result Tree \bin\FnO_Sample\CreatePOAndSubmitWorkFlow.xml.
Click on button Configure next to the Filename and ensure that all checkboxes are selected.
Similarly, under the Test Plan, set the desired file path for the results location in the View Result Tree \bin\FnO_Sample\CreatePOAndSubmitWorkFlow_Run.xml.
Starting the Tasks Recording
Before starting the scenario recording, it is crucial to ensure that the setup and configuration in the above sections are completed. Furthermore, it is essential to confirm which specific scenarios need to be captured. To achieve this, it is highly recommended to collaborate with your business team to define the end-to-end scenarios.
In this blog post, we will showcase a simple scenario: “Create purchase order and submit purchase order to approval workflow”, using Contoso data. The scenario has the following flow:
-
• Sign into Dynamics 365 finance and operations apps using your credentials.
-
• Go to Procurement and sourcing module then Create a Purchase Order with 2 items.
-
• Save the Purchase Order and submit to workflow.
-
• Sign out from the application.
-
1. Go to the bzm – Correlation Recorder section. Click on the Start button to initiate the scenario recording process.
-
2. When you come across the Recorder: Transactions Control prompt, you are clear to proceed with walking through the designated scenarios.
-
3. Let's get started by inputting the transaction name as LoginD365. After that, proceed to access the URL: https://<environment_name>.sandbox.operations.dynamics.com/. Following this, provide your email address and password on the credential page.
4. Once you've successfully obtained access to the Dashboard, designate the transaction name as GoToPurchaseOrderListPage. Then, go to Procurement and Sourcing > Purchase orders > All purchase orders.
5. Input the transaction name as CreateNewPO and proceed by clicking the New button to generate a new Purchase Order.
6. Input 1001 in Vendor account field, leaving all other settings as their default values. Subsequently, click the OK button to continue.
7. Enter FirstLineDetail as the transaction name. Then, input the subsequent details into the line:
- Product number: 1000 (Surface Pro 128 GB)
- Site: 1
- Warehouse: 11
- Quantity: 20
8. Enter AddSecondLine as the transaction name. Then click the button Add line to add a new line below first line.
-
9. Enter SecondLineDetail as the transaction name. Then input the subsequent details into the line:
- Product number: A0001 (HDMI 6' Cables)
- Site: 1
- Warehouse: 11
- Quantity: 10
10. Enter the transaction name as SavePO and subsequently, click the Save button to ensure that the Purchase Order is successfully saved.
11. Input the transaction name as SubmitWorkFlow and go to the Workflow section. Proceed by selecting the Submit button, and within the prompted dialogue box, input the comment “Please help to approve my purchase order by JMeter”. Finally, click the Submit button to initiate the submission of the purchase order workflow.
12. Confirmed the PO has been in review state, then Input the transaction name as SignOut and proceed to perform the sign out action.
13. Once you have successfully signed out, click the Stop button.
Upon finishing the recording process, it is important to ensure that you save the file. Once the recording is concluded, you can the opportunity to review the recorded samples within the Recording Controller.
Post-Recording
After recording the scenario in JMeter, there are several essential tasks that should be carried out.
Correlation using PowerShell script for Dynamics 365 finance and operations transactions
Correlation involves capturing and preserving dynamic responses from the server and effectively passing them to subsequent requests. Recognizing the limitations of the bzm-correlation recorder in constructing correlation rules, we have developed a dedicated PowerShell script tailored for automatic correlation of control IDs. These control IDs are encountered in requests other than the login operation and are distinctive for each session.
To achieve automated correlation, follow these steps:
-
1. Exit the JMeter before proceeding with the following steps. This prevents overwriting the test plan after executing the Correlation matching.
-
2. Begin by downloading the “CorrelationScript.ps1” from here and save it within the JMeter directory's \bin folder.
-
2. Open the automatic correlation script, “CorrelationScript.ps1,” using PowerShell.
-
3. Utilize the Browse button to select the script file path, as well as the path of the test script (.jmx).
-
4. Similarly, click the Browse button to specify the result file, which is the .xml (Noted: .jtl if you are using higher version e.g. V2.1 of correlation recorder) located within the view result tree under the bzm-correlation recorder.
5. Click the OK button. The auto-correlation script will initiate a scan of both the .jmx file and the .xml result file. Please refrain from closing the PowerShell window until the script's execution is complete.
-
6. Once the previous step is completed, then open JMeter again.
Optimizing Test Script
-
• Check for any correlation failures, if found, assign parameters manually.
-
• Review and identify any invalid Requests that need to be disabled.
-
• Add Think time intervals within the script if necessary.
-
• Organize Requests by grouping them according to transaction names or types, and then add these groups to the Transaction Controller. This approach simplifies recording management and enhances the ability to reuse the test plan for different scenarios.
Execution of Single-User Tests (GUI Mode execution)
Once you've completed the JMeter script and provided all necessary parameters, it's time to verify the validity of the recorded test. This verification step is important to ensure the test has the necessary elements to run or to identify if any further adjustments are required. Alternatively, you can proceed with a run-through of the process to confirm its functionality.
It's worth noting that performing multi-user testing in GUI mode is not recommended. GUI mode is primarily used for scenario recording and validation purposes. In this context, it's important to confirm that the Thread Group is set up correctly.
Go to Test Plan > Thread Group and ensure that Number of Threads, Ramp-up Period, and Loop Count are all set to 1. This ensures an accurate representation of a single-user scenario. For more detailed information on each parameter refer to this link.
Initiate the verification of the test script by clicking on the Start button. To obtain an overview of the execution, refer to the Summary Report. For more detailed insights into each transaction, especially when debugging is necessary, consult the View Result Tree section.
Execution of Multi-User Tests
Within JMeter, the execution of multi-user testing can be initiated through either the GUI or command mode. The recommended approach is to utilize the command mode, primarily due to its advantages such as efficient resource utilization, separation of controllers and agents, and lower resource overhead.
Create testing users
Prior to commencing multi-user testing, it's imperative to include test users within both the target environment and Azure Active Directory (AAD). This inclusion is essential to simulate concurrent user activity. Before proceeding with the import of users into Dynamics 365 finance and operations apps, ensure the testing user accounts are created within Azure Active Directory. Please make sure to collaborate with an Azure Administrator to gain a clear understanding of any potential additional charges.
-
1. Go to Azure Active Directory, then proceed to All users. From there, click on Bulk operations, followed by selecting Bulk create. Download the CSV template provided.
-
2. Utilize an editor to open the downloaded CSV template. Input the necessary information into the designated columns.
-
3. Upload the completed CSV template and click the Submit button. This action will lead to the creation of testing users in Azure Active Directory upon successful validation.
-
4. After successfully creating the testing users in AAD, move on to Dynamics 365 finance and operations apps. Go to System administration > Users. Click on Import Users.
-
5. Choose the previously created testing users from the preceding steps and initiate the user import process.
-
6. Assign the Purchasing Agent role to all the created testing users.
CSV Data Set config
-
1. Create a CSV file (the headless comma separated list of userEmail, password) duplicating the authentication credentials that were established in Azure Active Directory (AAD).
-
2. To create a list of testing users for testing, move to JMeter and Right-click on the Test plan, Add > Config element > CSV Data Set Config.
-
3. Outline the particulars as follows:
- Filename: Indicate the path to the CSV file containing the credentials
- Variable Names: userEmail, password
-
4. Go to Thread Group and expand group “1. Access to finance and operations apps”
-
5. Identify the request responsible for transmitting the credentials to login.microsoftonline.com/tenantId/Login.
-
6. Subsequently, using the CSV Data Set Config variables, substitute the occurrences of login, loginfmt and password in the above-mentioned request.
Execution
In our defined scenarios, the performance objective requires utilizing 10 test users to individually initiate the creation of Purchase Orders and subsequently submit them through to workflow. The target response time for each of actions is set at under 2 seconds.
-
1. To configure this, Go to Test Plan and access the Thread Group. Make sure to set the Number of Threads parameter to 10, and retain all default settings for the other relevant parameters.
2. Deactivate all listeners and non-test elements, retaining only the executable transaction script within the Test Plan.
-
3. Indicate the host name, save the project, and subsequently close JMeter.
-
4. Launch Command Prompt with Administrator privileges.
-
5. Navigate to the \bin directory where jmeter.bat execution application stored.
-
6. Execute the following command, referring to additional information here for command parameters:
-
Jmeter.bat -n -t “C:\apache-jmeter\bin\FnO_Sample\CreatePOAndSubmitWorkFlow.jmx” -l “C:\apache-jmeter\bin\FnO_Sample\Result\Results.csv” -e -o “C:\apache-jmeter\bin\FnO_Sample\Result\Report”
-
7. A successful execution of the command will be indicated as follows:
8. The successful test outcome is represented as follows: all Purchase Orders have been generated and submitted to the workflow successfully, and they are presently awaiting approval.
9. Verify the workflow history, you can access detailed information within it. This includes the workflow name, version, the individual who submitted it, as well as any accompanying comments.
HTML Report
Since the parameter -o is utilized, an analysis report will be generated automatically upon the completion of testing.
To access this report, go to the directory located at \bin\FnO_Sample\Result\Report\. Inside this folder, you will find the dashboard containing the relevant information.
Double click on index.html to open it. There is performance information provided in the report. The JMeter HTML report serves as a comprehensive summary of performance test results, presenting essential metrics such as response times, throughput, and error rates in a visually accessible format. This report aids in analyzing the performance of the tested application, identifying bottlenecks, validating performance goals, and making informed decisions for optimization and further testing efforts.
In the Response Time overview, we can observe most response times where less than 1,500ms which aligns with our performance goal (less than 2 seconds). A more in-depth analysis of the results will be available in Part 3 – Dynamics 365 Finance and Operations Apps Performance Testing with JMeter – Result Analysis.
Azure Load Testing
Azure load testing is a load testing service designed to facilitate the generation of extensive load for your applications. This service adeptly simulates traffic, irrespective of the hosting location of your applications. For further details, you can access the comprehensive documentation available here.
In our defined scenarios, the performance objective requires utilizing 100 test users to individually initiate the creation of Purchase Orders and subsequently submit them through the workflow. The target response time for each of actions is set at under 2 seconds.
-
1. Begin by establishing the Azure Load Testing resource. Detailed guidance can be found here. Go to Azure portal > Azure Load Testing, create new Azure Load Testing resource, Instance name is D365FOPerfTests
-
2. Navigate to the newly created Load Testing resource D365FOPerfTests. Access the Tests section and initiate the creation process. Choose to upload a JMeter script to shape the test. Comprehensive steps can be referred to here.
-
3. Fill in the essential information on the Basics tab, including the Test name and Test description. For instance, "Create PO and Submit PO Workflow".
-
4. In the Test plan tab, upload the script files, namely CreatePOAndSubmitWorkFlow.jmx and Login.csv. Successful validation will be indicated by a "Completed" status.
5. While optional, parameterizing variables is advantageous for executing the JMeter .jmx file across different environments. In JMeter, access User Defined Variables to incorporate environment-specific parameters below, then save it. Find detailed steps here.
6. With the test configured, initiate the execution by clicking Run. Offer a concise description for this test run, such as "Create PO and submit Workflow with 100 users."
7. A successful execution of the Azure Load Testing will be indicated by specific information shown below. In this case, the test concluded in approximately 4 minutes, generating a total of 10,900 requests. The average response time remained under 1 second, aligning with our performance target.
8. The successful test outcome is represented as follows. A total of 100 Purchase Orders were generated and successfully submitted to the PO Workflow without errors, currently awaiting approval. The result highlights an average rate of around 7 orders per second.
Summary
In Part 2 of this blog series, we explored how Apache JMeter is proficient in performing performance tests for Dynamics 365 finance and operations apps. By using JMeter and Azure Load Testing along with generated test script, you can conduct both single-user and multi-user tests with ease.
Sample script can be downloaded through link:
Blog series included:
- Part 1 - Dynamics 365 finance and operations apps performance testing with JMeter - Introduction
- Part 2 – Dynamics 365 finance and operations apps performance testing with JMeter - Execution
- Part 3 – Dynamics 365 finance and operations apps performance testing with JMeter - Result Analysis
- Part 4 – Dynamics 365 finance and operations apps performance testing with JMeter - Tips and Troubleshooting
- Part 5 – Dynamics 365 finance and operations apps performance testing with JMeter - MFA
Comments
-
Part 2 - Dynamics 365 finance and operations apps performance testing with JMeter - Execution
Hi All,
Thank you for your feedback, questions, and for taking the time to read this blog post. I’d like to share my comments below for recently questions:
- This approach can be applied to Dynamics 365 Sales, however, it’s not exactly the same. I’m planning to release something specific for Dynamics 365 CE soon, so please stay tuned.
- If I understand correctly, you're looking to handle multiple requests. In that case, you should switch to multiple-user testing and increase the number of threads. This will simulate the requests from different threads.
- The PowerShell script helps map the Dynamics values, which can also be done manually, but it requires more effort.
- I have successfully tested the full Sales order cycle (Creation, Confirmation, Packing, Invoicing).
- The provided D365Login_correlation_template only supports Tier-2 or higher environments because the authentication methods differ between Tier-1 and Tier-2 or higher environments. Consequently, the script that works in Tier-2 or higher environments will not work in Tier-1. You can verify the authentication variables and values, then proceed with the extraction. Alternatively, you may consider moving Tier 1 from LCS to UDE.
-
-
-
Part 2 - Dynamics 365 finance and operations apps performance testing with JMeter - ExecutionThank for the wonderful post. I was looking for something similar, But I have below question -1. Will the same Methodology will be applicable to Dynamics 365 Sales?2. Since you are using simple transaction controller all the http request will be executed in sequential order, how will this replicate browser UI behaviour where request are hit in overlap ?3. Due to company restriction I will not be able to download power shell script for auto correlation used in this blog, are there any alternate methods available ?
-
Dynamics 365 finance and operations apps performance testing with JMeter - ExecutionHi Rahul, Thank you for your reading and your questions. Experiencing a high number of errors during the recording of steps is not a common occurrence. The quickest way to address this issue is to consider re-recording the steps. Alternatively, troubleshooting may be necessary.
For the multi-user test, please verify that the option "Same User on Each Iteration" is not selected in the thread group. If this option is chosen, JMeter will employ the same user for each iteration. -
Dynamics 365 finance and operations apps performance testing with JMeter - ExecutionAt the time of recording, the result comes up with many error, after repeatedly recording, one thread group come up with correct recordings, Can you tell me, it is same happening for me or for you also.
and i tried running multi user test, it creating for 2 times but with same user only, even i am giving diffrent users in csv file.
Thanks,
*This post is locked for comments