Functional regression test automation in Dynamics AX 2012

(0) Share

Report

Posted on by Tina Vyf

I am looking for a tool to automate functional regression tests in Dynamics AX 2012:

1. I looked at third party (ISV) tools like Axeptance and Executive Automats. Does anyone have experience with these tools? Could you provide me pro's/cons and your experience when you made it part of your testing process?

2. I read Dave Froslie's Testing Best Practices white paper and am also interested in using Microsoft Test Manager in Visual studio. Does anyone have any videos/examples/blog posts using Test Manager with Dynamics AX? So far all I have found is videos showing web application testing, but I would like to see it in action with Dynamics AX.

Any other suggestions, tips and tools regarding automated functional regression testing also welcome!

P.S. - I am not interested in unit tests or SysTest yet; hopefully that will be phase 2 of our new testing process! :-)

*This post is locked for comments

All responses (4)

Answers (0)

Nickr on at

Like (0)

Report

RE: Functional regression test automation in Dynamics AX 2012

I seem to remember Microsoft released a video from Convergence/Tech conference where they used a test automation tool which was based on the task recorder. Not sure if this is something that was released or is still in Beta.

I'll try and track down the video..
Martin Dráb 230,445 Most Valuable Professional on at

Like (0)

Report

RE: Functional regression test automation in Dynamics AX 2012

I think the best way indeed is if you try it, familiarize yourself with Test Manager / Test Runner and show them how it works. I would try to explain them than defining manual tests in this structured and repeatable way is exactly what they need for automation, so it's exactly the right step to start with. They will immediately benefit test runner instructions and fast-forward (e.g. that the test runner automatically opens the form to test) and full automation will be developed iteratively later. (Or won't be developed, when they realize the costs...)

You don't have to install TFS by yourself - you can simply use Visual Studio Online. I would recommend that even if you're going to use on-premise TFS at some point - it's a great way to start and it takes just a few minutes. You can also download a VM (e.g. this one) with TFS and labs demonstrating various features (this is without AX, of course).

You'll find a plenty of documentation for TFS, this shouldn't be a problem. You can also look at some courses at Microsoft Virtual Academy.

Unfortunately I have only limited success so far in convincing people. It's easy to demonstrate, it looks great, everybody agrees it will save them time... and then they return to how they worked before. But I had the same experience with version control in AX and the situation has improved dramatically in recent years. So hopefully this is not a lost case either. I also assume it will all be easier in the next version of AX (and TFS will again be the tool to use).
Tina Vyf on at

Like (0)

Report

RE: Functional regression test automation in Dynamics AX 2012

Hi Martin,

Thank you very much for you answer. I really appreciate your insight.

When I said "I am looking for an automation tool", I actual meant "My company is looking at automation tools"! We have a team (developer, tester and functional consultants) tasked to evaluate the two above tools and decide if it is something we want to incorporate into our process; I am the developer on the team.

I also feel automated GUI testing it not worth the effort and leads to more overhead for testers than it actually saves time. I have tried functional automation in Dynamics and it seems to lead to a lot of scripting and calling of static methods to simulate user interaction. I my option the whole point of having a functional tool is that it can be used by functional testers and business and not cause extra coding work for developers in order to simulate what a user would do.

I agree that the way to go is creating automated Unit Tests (development) and supporting testers with VS Test Manager (functional). My follow up question is: How do I convey to/convince business that this is the route to go? Besides Dave Froslie's paper I cannot find any 'proof' that Test Manger works well with Dynamics AX. I am willing to try it myself, but I have no experience working with or installing TFS.

- Tina
Martin Dráb 230,445 Most Valuable Professional on at

Like (0)

Report

RE: Functional regression test automation in Dynamics AX 2012

I don't have any video showing Test Manager with AX, nevertheless it's a great tool for manual tests with fast-forward, not mentioning the tight integration with TFS (such creating a bug with recording with just a few clicks).

Nevertheless I think that trying to build fully automated GUI tests is not worth the effort. Especially things like lookups are difficult to handle. I believe that fully automated unit tests of business logic plus manual tests as efficient for testers as possible is the way to go.

It seems that you follow the usual path - "testing is what testers do, therefore test automation is simulating what human testers do" (clicking on buttons etc,.). Unfortunately it's the most difficult, slowest and most fragile type of automated tests. People should start looking at these things when they have done every else, not as the first introduction to test automation. Many people give it up later because of all the complexity of GUI testing and never return to automated tests again, including such crucial types of tests as unit tests. It's a loss for everybody then.