1. Preface
    1. About MarathonITE
    2. Supported Platforms
      1. Java/Swing™
      2. Java/FX™
      3. Web Applications
    3. What's New in MarathonITE 5.0
    4. Change Log a.k.a Version History
    5. System Requirements
    6. Copyright Notice
  2. Getting Started
    1. Introduction to Test Automation
      1. Unattended Testing
      2. Semi Automated Testing
      3. Exploratory Testing
    2. MarathonITE Projects
    3. Your First Project
      1. Java/Swing™ Test Project
        1. Application Under Test - SwingSet3
        2. Creating a Project
        3. Recording a Test
        4. Anatomy of a Test Script
        5. Running Tests
        6. Looking at Results
      2. Java/FX™ Test Project
        1. Application Under Test - Ensemble
        2. Creating a Project
        3. Recording a Test
        4. Anatomy of a Test Script
        5. Running Tests
        6. Looking at Results
      3. Web Application Test Project
        1. Application Under Test - DuckDuckGo Search
        2. Creating a Project
        3. Recording a Test
        4. Anatomy of a Test Script
        5. Running Tests
        6. Looking at Results
    4. MarathonITE Sample Projects
  3. User Guide
    1. Installation and Startup
    2. MarathonITE User Interface
      2. Editors
      3. Output Views
    3. Creating Marathonite Test Projects
    4. Recording Tests
    5. Managing Checklists
    6. Exploratory Tests
    7. Semi Automated Tests
    8. Executing Tests
      1. Executing a Test from Editor
        1. Debugging Scripts
        2. Using Script Console
      2. Executing Tests from Test Runner
      3. Executing Tests in Batch Mode
    9. Organizing Tests
      1. Organizing Tests in Folder Heirarchy
      2. Organizing Tests as Features and Stories
      3. Organizing Tests in Suites
      4. Linking Tests to TMS and Issue Manager
    10. Modularizing Test Scripts
      1. Module Methods
      2. Extract Method Refactoring
      3. Using Data Loops
      4. Convert to Data Loop Refactoring
    11. Data Driven Tests
      1. Convert to DDT Refactoring
  4. Advanced Scripting
    1. Ruby Programming Language
    2. Marathon and Ruby
    3. Selenium/WebDriver Bindings
      1. Java/Swing™ Components
      2. Java/FX™ Controls

2.1.Introduction to Test Automation

Industry surveys suggest that over 75% of all functional testing is still done Manually. In order to optimise testing, it is important that we understand the applications and limitations of Automated testing. Read on for an overview of what Test Automation is, why and when should you use it and what advantages it can give you.

The beginning

When Test Automation made its debut some two decades ago, the IT industry heaved a great sigh of relief. The advent of client/server architecture had introduced so many more layers of software complexity, and all of it needed testing! A lot of Tools dominated the test automation landscape. They came in colourful packaging with a promise to bid all testing problems a cheerful adieu!

The end?

However, automation users quickly realised that there were many loopholes to using these new tools. For starters, every time the application-under-test changed, the automated tests had to change as well. While the tool developers added features over time to help reduce the testers efforts, users began to feel that the time taken for automation development and upkeep, far outweighed any benefits derived from it.

A new beginning

Over the years, however, automation tools have been improved with additions of new techniques and better understanding of what testers are looking for. Tools like MarathonITE do not hide the working behind no-code facade. Most tools generate either keyword based tests or script based tests. With the realization that there is no code-free automation, the automation field opened up. With the ever increasing complexities of applications, automated tests have become a necessity and an useful weapon in tester's arsenal.

Lets learn a little about Test Automation.


Why Test Automation?

  • Removing human error. Computers are good at mundane/repeated tasks, while humans are prone to error while carrying out such tasks. Thus, test automation allows you to let computers do what theyre good at, while freeing up the teams time to do tasks that are more important.
  • For faster and accurate test execution. When we compare the speed and accuracy of running tests, computers are far superior to their human counterparts.
  • Continuous Integration and continuous delivery. Test automation is more important now than ever! With current development practises such as Agile, DevOps, Continuos Integration and Delivery, we require tests that can run quickly and reliably - you can't be successful in any of these development methods without some degree of test automation.

Automated Testing

When should you start to consider Automation?

The best time to start automation, is at the very beginning of the testing process.

Start where you are. Use what you have. Do what you can.

During your application testing process, the more steps that are not automated, the more steps will you have to add to automation later - thus, more work! The best decision is, to automate the testing process from the very beginning. As your application grows, the automated tests will also grow.

What should not be Automated?

  • One off items. There are exceptions of course. If a simple test can save time and energy, then go ahead!
  • Tests that require manual steps. However, have a look at MarathonITE's semi automated tests to simplify these tests.
  • Non deterministic tests. Scenarios whose behaviour can't be predicted, should not be automated. Its impossible to automate something when, you do not know what the output is going to be.
  • Look and feel. It is subjective and need manual intervention.

What should you Automate?

Deciding what to automate is a gruelling task. The following points should help you in deciding what you can automate:

  • Smoke Tests. The smoke tests provide you a go/no-go decision for further effort in testing.
  • Scenarios or User Stories. Creating tests for scenarios help you to check the functionality in your application.
  • Buggy Modules. Bugs tend to group together in modules. If you find a particular module to be buggy, put in effort to have automate tests around that module.
  • Critical and Risky portions. One area that you mustn't forget during automation is to create tests for the most critical and risky portions of your Application.

Test Automation Pitfalls and How to Overcome them

  • Forgetting that test automation is a software development project. Like any computer based automation effort, test automation also is a software development project. Consider it as one, plan it as one.
  • Concentrating and creating tests for easily verifiable scenarios. Easily verified tests, things that can be easily verified as a human, need not be converted to an automated test.
  • Not having tests that are reliable. A test should identify whether a specific functional unit is working or not.
  • Not having checkpoints in the tests. A functional test should ensure that result that you expect is what you get if it is successful.
  • Not having similar setup as your production system. Ensure that the system setup that you use the same (or atleast similar) to what your end users use. If your end users use Windows™ and you test on Mac OS™, how much ever fun it is, is not going to help your project.

Finally, Make sure that the Application development goes hand in hand with the testing, or else, important test results may go to waste!

Some pointers to keep in mind:

  • Automation should not be the only type of testing. The best practise is to mix it up; manual, semi- automated and completely automated testing.
  • Automation can generate fake secure feeling. If all your tests run smoothly, it may mean that they are not doing what they're meant - catch errors.

Manual vs Automated

What advantage does Automated testing give you?

  • Test Efficiency
    Test automation makes the testing process superbly efficient. While your automated software completes the repetitive, time consuming, mundane tests, the testing team can pay attention to more important and non- deterministic tasks. Thus saving time, resources and money!
  • Durability
    Test Automation allows you to reuse written tests. This saves time and energy while the previously written tests can be modified to fit the ever-growing software. This makes Automation software long-lasting, reusable and completely worth it!
  • Coverage
    Test Automation allows you to cover a larger area for testing as compared to manual testing, thus it can save time. In the long run, maintenance also reduces as a wider area is tested by automated testing.
  • Consistency
    Once the automation testing software is set up for your Application, it acts as a consistent platform for your testing needs. By simply running pre-recorded instructions, the regression tests can verify if the new versions of the software are compatible with pre-existing functionalities or not. This novel consistency provides a much needed reliability for your testing protocols.
Suggest Edit
Close Menu