When we talk about test automation, usually we refer to unattended testing. However, test automation tools like MarathonITE can also be used to reduce the effort required for executing other types of testing – semi automated or manual/exploratory tests.
You get the best results from a test automation effort when you can run your test suites unattended. You can achieve this by integrating test automation with external systems like your source control system or a continuous integration server.
Planning for Unattended Testing
Identify a Trigger Point
When do you want the test suites to be executed? The simplest solution may be to trigger a test run at some specific time every day. Another way is to schedule a test run whenever developers update your source control system. If you go this way, you may want to schedule the run only when a specific action is taken – like a feature complete option of git-flow. You may also use a CI server and use its functionality to schedule the test run.
Nothing in the above precludes you from using multiple strategies together. You can schedule a test run every day at 2AM as well as whenever a module is updated in the code repository. You can be creative and flexible in identifying the trigger points.
If your test scripts are interdependent, then you will be forced to run the tests in a particular sequence. Besides being a bad practice, this will preclude you from executing the tests in parallel for reducing the .
Most interdependent test scripts can be made independent of each other using Fixtures to setup a known environment for each of the test scripts.
Run your tests on dedicated machines to avoid disruptions by other users.
Easily accessible reports
Ensure that the test results are available in a central repository accessible to all the stake holders. It is a good practice to send a message to all the team members with the test result – specifically if they are errors.
Setup a Process to fix the issues
If you do not work and fix the issues raised by the test execution as soon as possible – the failures rolls over. Over a period of time the failures overwhelm your team and the reports gets ignored. Ensure that you have a process in place for working on the test failures.
One of the teams I worked on used to give it the highest priority. Any failures identified by the CI Server used to be worked on and any other work is kept pending till all the failures are resolved.