We discussed about benefits of test automation. We also looked at some best practices when automating tests. One of the major criteria that has an effect on the schedule and effort of an automation project is Selecting Tests. There are a lot of things that need to be weighed-in while selecting tests for automation.
A test case is typically based on a end user workflow (a scenario). We simulate the actions performed by the user, look at the results and assign a pass or fail status to a test. Depending on the application under test you have to decide on what scenarios to automate. Still, there are criterions that can be used to make automate or manual decision.
1. Automate Basic Application Functions
There are some functionalities of the application that forms the critical path. Whenever you make a release, you will be checking these functionalities. These tests will be a good start for automation.
These tests tend to be routine tests and run often. Manually running these tests leads to fatigue and might lead to errors.
2. Automate Tests Targeted for Differing Configurations
Most of our applications are delivered for multiple platforms and operating systems. So automating tests that you run on different configurations (Operating Systems, Versions, Platforms) will save you lot of time and effort.
3. Automate Tests that have Consistent Outcomes
One of the criteria for automating a test is that when the steps are performed, the outcome should always be the same. You can't automate a test that requires a subjective verification to find whether it succeeded or failed.
That said most scenarios, even when the outcomes do not match at different times, can be automated by looking at the common patterns in the output. For example, a Google search might return different outputs for the same search term — but you can be sure that the search term appears in (at least few) of the search results. We can still automate such scenarios.
4. Capture the Low Hanging Fruits
There are some tests that are simple to automate. For example, checking for successful and failure of login from a dialog is simple to automate. These kinds of tests when automated adds value to the test suite and improves ROI.
5. Automate Tests that are Impossible to Run Manually
Some tests are not possible to run manually. For example, your test might be based on some timing or an action from external system. Automating these scenarios resolves most of the issues and helps in delivering a robust system.
6. Automate Tests that Follow the Same Steps, but Differ in Input and Expected Results
Most of our application does some logic processing at some time. These are tests that check for given input what is the output. Think of a calculator. These tests are best performed through automation as data driven tests.