Dynamics Community Forum Thread Details

Performing test the right way

I'm not familiar with this product, but it seems that is allows you to record actions and reply them, using image recognition or skipping client and sending network commands directly.

This is what people usually imagine by "automated test", because it basically means replacing a human tester with a script, except the fact that the tester can use his/her brain to decide whether something succeeded, what to do next etc.

On the other hand, it's a poor choice for automation, because:

It's extremely fragile. A simple change in user interface (even caused by things like security) can break all your tests. If it uses image recognition, the problem is insanely huge - I see people struggling with tests if they change screen resolution, for example. You want tests fail if your application is broken, not because the resolution has changed.
It lacks isolation. If something fails, you know that your have a problem somewhere in a business process, calling several thousand methods. Or maybe you have a problem with data. You simply don't know.
That's why automated tests should test small isolated units.
You must use AX database, therefore you have to maintain test data. Because database is shared for all tests, you can get into all the problems associated with shared test fixture (such as that one test can influence other tests, or you have to expensively cleanup the state before/after each test).
It's the slowest type of test you can imagine.

From my opinion, this is a kind of testing that you should consider when you already have all easier, faster and more reliable tests in place, you have experience with test automation and you want to get rid of the last manually intervention in integration and UI tests. If this is your first attempt for automated testing, it will very likely fail (you won't achieve it at all or maintenance cost will negate all gains) and people often give up automated testing completely because of this experience. But the problem was that they knew nothing about automatic testing and therefore they made wrong decisions.

I would recommend two things:

Use unit tests as the primary mean for verification of implementation. If you don't test code at the lowest level and get poor implementation of methods, you'll never build a high-quality product from them. Don't worry about code coverage too much, but ensure yourself that all complicated logic you write is covered by unit tests.
Use Microsoft Test Manager (you could use some other similar tool) to plan and design your manual tests. Designing tests and expected results is the first and mandatory step towards automated UI tests, because scripts (unlike human testers) can't just figure it out by themselves. It's surprising how often people want to automated tests they never defined.
Then record these tests and use fast forward for automating some steps. You'll see that some actions can't be replayed, therefore many tests can't be fully automated in this way. That's the moment when you can start thinking of converting test recordings to coded UI tests and implement logic needed for full automation. Nevertheless even if you never do that, you'll already have quite a lot of automation and tools to highly improve productivity of human testers.

Quick Links

Leaderboard