Regression Suite Automation Tool (RSAT) - Components and Usage

1. Overview​
2. Task recorder, Lifecycle services (LCS) – Business process modeler (BPM), Azure DevOps (ADO) – Test plans, Remote Server Administration Tools (RSAT)​
3. End to end process​

1. Overview:

Microsoft's "One Version"​

• Microsoft’s vision is that all customers will be on the latest product version (achieving predictable and continuous upgrades). ​
• Thus, there are 2 major updates per year (April and October).​
• Additionally, Microsoft releases monthly platform updates. ​
• Example: Preview January, General available February (self-update), auto-update schedule March (via LCS) ​
• Customers may choose to pause up to 3 consecutive updates in a row.​
• Customers are required to take a minimum of 3 updates per year (resulting in the need of at least 3 regression tests/year). ​
• If auto-update is chosen (LCS), then 7 days are given to validate in UAT sandbox and sign-off before the updates are made in production.

What is a RSAT?​

• RSAT stands for Regression Suite Automation Tool and is a test automation tool/application for Microsoft Dynamics 365 for Finance and Operations installed locally or on a Virtual Machine.​
• It facilitates regression testing of business processes as a result of continuous platform and application updates and other changes during the implementation’s UAT and/or after go-live (such as new modifications, configuration changes, hotfixes).​
• Therefore, RSAT can reduce the time and cost of user acceptance testing as well as regression testing post go-live.​
• RSAT is connected to Azure DevOps’ testing suite and loads test plans from there.​
• For an implementation, RSAT will usually be pointed to the UAT Tier 2 environment. Here, the test manager will conduct regression testing with the RSAT Tool.​
• After successful validation of test results in Azure DevOps, code updates can be pushed to the production instance (new modifications, hotfixes, continuous updates) and/or configuration changes can be moved to the production instance.​

Components:

RSAT - Process flow:

Prerequisites:

• D365FO Tier 1 Dev or Tier 2 UAT box - A Dynamics 365 for Finance and Operations environment v 8.1 or newer is needed for this workshop, though for customers on v7.3, PU20 or higher is supported as well. System parameters and Help settings needs to be set up pointing to the BPM library.​
• LCS project/BPM library – LCS project needs to be created, BPM needs to be created and BPM processes need to be build out. VSTS sync needs to be enabled.​
• DevOps project/DevOps Test plan – DevOps project as well as Test plan needs to be created. Synced test cases need to be added to appropriate test plans.​
• RSAT – RSAT tool needs to be installed and set up for D365FO as well as CPOS (if needed).​
• Excel – Excel is required for parameterizing and configuring RSAT test runs.​
• Web browser – A Web browser (I personally would recommend Chrome) needs to be installed. ​
• Operating system - Windows 10 needs to be installed on machine where RSAT is installed/ran from.​
• Selenium - Selenium needs to be installed on machine where RSAT is installed/ran from.
• Permissions/Security/licenses:
  • The test manager needs to have system administrator security role assigned in Dynamics 365 F&O environment. ​
  • The test manager need the Azure DevOps Test Manager license for DevOps (or one of these licenses – Visual Studio Enterprise, Visual studio Test Professional, MSDN Platform subscriber) ​
  • The test manager user will need to be in the same tenant than D365FO.
  • The test manager need to have the LCS owner role assigned in LCS.​
  • Authentication Certificate is to be installed on the local/virtual machine where RSAT is ran from

2. Task recorder, Lifecycle services (LCS) – Business process modeler (BPM), Azure DevOps (ADO) – Test plans, Remote Server Administration Tools (RSAT)​

Task recorder for D365FO:

• The task recorder is part of the out-of-the-box D365FO application.​
• A user is able to start recording business processes step by step using the task recorder under settings in D365FO. ​
• Task recorder supports Microsoft Edge, Internet Explorer 11, and the latest version of Google Chrome. ​
• Task recorder will be used to record functional scenario and business process to replay them as support for users as well as to leverage them for RSAT regression testing.​
• A user can record processes also on the CPOS. However, the register needs to be set up for it and the recording needs to be manually uploaded to BPM.

Business process modeler (BPM):​

• The Business process modeler (BPM) is a tool within Microsoft Dynamics Lifecycle Services (LCS) to support maintaining business process libraries and business process flowcharts. ​
• The out of the box BPM library is based on industry-standard processes that are described by the American Productivity & Quality Center (APQC). ​
• LCS and BPM are both tools only available with the use of Microsoft Dynamics 365 for Finance and Operations. ​
• This RSAT BPM library will be referenced under System administration/Help in D365FO. After doing so, a user will be able to upload and reference a task recording directly from D365FO to a specific business process in BPM. ​
• The RSAT BPM library will be enabled for VSTS sync in LCS what essentially seamless integrates business process and task recordings from BPM to Azure DevOps test plans as test cases.
• It is recommended to create dedicated business libraries for all your business processes in general as well as core business processes required to be regression tested via RSAT as of continuous updates. Short, for the purpose of RSAT, a new process library should be created in BPM that captures the business-critical scenarios that need to be tested for continuous updates.​
• This can be subset of all business process documented initially for the implementation.​
• Processes can be documented in BPM in such a manner that it flows in the same sequence as within Dynamics 365 for Finance and Operations, based on the task recordings.​
• Leverage the import template to upload, structure and sequence your BPM processes more efficiently. ​
• Synchronize with Azure DevOps (One – way Sync) to create DevOps work items corresponding to each business process and the attached task recordings​
• When you synchronize a BPM library with an Azure DevOps project, an Azure DevOps work item is created for each business process line in the BPM library.​
• This synchronization is a one-way synchronization. Changes in LCS are reflected in Azure DevOps, but changes in Azure DevOps aren't reflected in LCS.

Azure DevOps - Test plans:

• Azure Test Plans or the Test hub in Azure DevOps Server provides three main types of test management artifacts: test plans, test suites, and test cases.​
  • Test plans group test suites and individual test cases together. ​
  • Test suites group test cases into separate testing scenarios within a single test plan. Grouping test cases makes it easier to see which scenarios are complete.​
  • Test cases validate individual parts of your code or app deployment. You can ensure your code works correctly, has no errors, and meets business requirements. ​
  • Test cases integrate from BPM via VSTS sync into DevOps.

Remote Server Administration Tools (RSAT):

• See overview information for RSAT in 1. Overview above.

3. End to end process

1. Create Task Recording for creating a new product in D365FO.​
2. Save the Task recording to the appropriate BPM process directly from D365FO.​
3. Validate in LCS BPM if task recording is attached to prior selected business process.​
4. Validate in DevOps test plan that the test case was successfully integrated from BPM to Azure DevOps.​
5. Add test case to test plan in DevOps.​
6. In RSAT:​
   • Load test cases.​
   • Generate and modify parameters file when running for the first time.​
   • Update the parameters file.​
   • Run the test cases.​
   • Review test results in Azure DevOps.