Should we create test automation? This is a question that countless development teams have discussed. Below are a few of the questions that are inevitably discussed.
When I started at Microsoft years ago, my team’s assumption was that we would attempt to automate everything. We devoted resources to create thousands of automated test cases. Automation was created for our Dynamics GP desktop client, windows applications, installs, and browser applications. Fortunately, our Desktop client test automation is based on a macro tool, which is built into and ships with Dynamics GP. This makes the test automation incredibly stable and effective.
Much of our other test automation has not fared as well as the desktop client test automation. It was based on frameworks that were not always reliable. The applications under test were difficult to automate and included external application dependencies that increased the complexity. A strategic investment in some test automation would have been good. Many in our software industry have come to realize that automating 100% of an application is difficult and may not be a good investment of time. This in no way is to suggest that test automation is a bad thing. It is a great tool when used properly in the right scenarios.
If we shouldn’t assume that test automation is the answer, we should consider whether creating test automation for our new Dynamics GP service based architecture (SBA) is worthwhile. I would propose that it is for the following reasons.
Hopefully you’ve decided to invest in creating test automation for SBA. The test framework that was developed and used internally has been made available externally. This code enables Dynamics GP partners and customers to quickly and easily write test automation for their service endpoints. It provides the means for communicating with the endpoints and verifying the service response and SQL results. The rest of the content is devoted to providing instructions and examples to aid in this effort.
The following are the requirements for a machine or machines that will be used in the creation and execution of test automation.
Working Dynamics GP 2015 environment with GP services installed and configured. (If desired, this can exist on a machine other than the machine executing the test automation.)
Visual Studio 2013 install that supports unit testing (Visual Studio Ultimate is an example.)
Fabrikam data (This is not an absolute requirement. It is simply required to run the sample test cases without modifying any code.)
Building The Sample
After successfully building the solution, navigate to Test Explorer (Test > Windows > Test Explorer). This is where the current test cases are displayed.
The automation supports executing with both json and xml. It's currently set to use json. Running with xml simply requires a change to the config file. Modify Config/GPServices/ContentTypes to set the desired content type.
Running Test Automation
Creating New Test Automation
The following are the steps required to create automation for new service endpoints.
2. Open Microsoft.Dynamics.TestUtilities.GPService > Common > UriEntitiesMetadata.cs. Within the AddEntities() method, add metadata describing the endpoint. The following would be added for the Employees endpoint. This information is used when building up the complete url that will be used to communicate with the service endpoint.
UriEntityDetail(Entity.Employees, "Payroll/", "Employees", false));
3. Within the Microsoft.Dynamics.GP.Svc.Tests project, add a new “module” folder under the Tests folder. I would add a “Payroll” folder for the Employees endpoint. Add another folder below the newly created named for the endpoint. Employee would be created for the Employees endpoint.
4. Create a new class in the above folder for the entity. Employee.cs would be created for the Employees endpoint.
5. Copy the contents of Template.cs into the new class file and modify for the endpoint.
TestEnvironment.BaselinePath = @"\GPSvc\Microsoft.Dynamics.GP.Svc.Tests\Tests\" + CurrentTestCategory + @"\Employee\Baselines\";
6. Create a test case based on the template provided. The example below will communicate with the Employees endpoint for the given employee ID.
The following are some other helpful tips.
1. If the test automation doesn’t seem to be communicating with the desired endpoint, set a breakpoint on the above line: this.ExecuteAndVerify(request). Debug the test case, and create a watch on request.CompleteUrl to verify that the url is correct. If a portion of it is not, modify the appropriate metadata to correct. For any Get (read) requests, copying the url into the browser and executing is an easy, quick test.
2. Data is very important when building up the automation environment. Much care should be given to what data is added and how it is maintained for the future. If a test case requires data that is not available in the current dataset, you can use the following to modify:
this.SqlServer0.ExecuteSQLQuery(“Insert query here”, databaseID); (databaseID is from the config mentioned above.)
3. Any test cases that are expected to modify the database should define test case details. The test case details define any tables that should be dumped and where clauses that should be applied when dumping the tables. Below is an example that would dump three tables with the defined where clauses. Not defining a where clause results in the whole table being dumped.
When new test automation is initially run, it is guaranteed to fail. This is due to the framework’s built in functionality to compare a given test execution test outputs with baselines. Since the baselines don’t yet exist, the test cases will fail. The test automation will save all test outputs to a TestOutputs folder (relative to given tests, i.e. Payroll/Employee/TestOutputs). Once these test outputs have been verified as correct, they should be moved to the Baselines folder (next to TestOutputs) where they will serve as baselines. Future test executions will compare test outputs to the files that were promoted to serve as baselines.
The code in the above mentioned zip file enables the following:
Creating service requests (setting urls, verb, headers, attaching objects, defining credentials, etc.)
Serializing and de-serializing objects using json or xml (Required for most create or update requests)
Executing service requests
Capturing service responses
Saving and restoring sql table states
Executing sql queries
Capturing the contents of sql tables
Verifying service responses
Verifying sql table contents
Filtering service responses or sql table contents of data that diffs every time.
Test template code for all test types
Code reads from config, making code machine agnostic
Test case examplesCode to verify newly defined tests
Hopefully you find the SBA test framework easy to use and valuable in improving the quality of your service endpoints. If you run into questions or issues, feel free to add a comment. This will allow any discussion to be shared among all those creating test automation. Thanks and happy testing!