How not to automate..!

       Whatever you do, do not simply distribute a testing tool among your testers and expect them to automate the test process. Just as you would never automate accounting by giving a program compiler to the accounting department, neither should you attempt to automate testing by just turning a testing tool over to the test group.
   It is important to realize that test automation tools are really just specialized programming languages, and developing an automated test library is a development project requiring commensurate skills.


Automation is more than capture/replay
If you acquired a test tool with the idea that all you have to do is record and playback the tests, you are due for disappointment. Although it is the most commonly recognized technique, capture/replay is not the most successful approach. Selecting an Automation Approach, capture and replay does not result in a test library that is robust, maintainable or transferable as changes occur.


Don’t write a program to test a program!
The other extreme from capture/replay is pure programming. But if you automate your tests by trying to write scripts that anticipate the behavior of the underlying program and provide for each potential response, you will essentially end up developing a mirror version of the application under test! Where will it end? Who tests the tests? Although appealing to some, this strategy is doomed - no one has the time or resources to develop two complete systems.   Ironically, developing an automated test library that provides comprehensive coverage would require more code than exists in the application itself! This is because tests must account for positive, negative, and otherwise invalid cases for each feature or function.


Duplication of effort
The problem is, if you just hand an automation tool out to individual testers and command that they automate their tests, each one of them will address all of these issues - in their own unique and Introduction personal way, of course. This leads to tremendous duplication of effort and can cause conflict when the tests are combined, as they must be.


Automation is more than test execution
So if it isn’t capture/replay and it isn’t pure programming, what is it? Think of it this way. You are going to build an application that automates your testing, which is actually more than just running the tests. You need a complete process and environment for creating and documenting tests, managing and maintaining them, executing them and reporting the results, as well as managing the test environment. Just developing scores of individual tests does not comprise a strategic test automation system.



Need for a framework
Instead, approach the automation of testing just as you would the automation of any application - with an overall framework, and an orderly division of the responsibilities. This framework should make the test environment efficient to develop, manage and maintain. 

Remember, test tools aren’t magic - but, properly implemented, they can work wonders!

Challenges of Test Automation

1. Selection of Automation Tool – Today there are n number of automation tools available in the market and choosing a good automation tool is one of the major challenges that an organization often faces. This is majorly because there are several commercial tools which are expensive, there are open source tools which might not be reliable and there are tools of which an organization may not have sufficient expertise to make optimum use of.

2. No Defined Process for Executing Automation Project within an organization – Automation is like a project execution ,wherein you start with Requirement, then you design the framework as per your requirements and finally you roll it out for Test Case Execution. Lack of systematic approach and process will make successful automation really tough. As we have processes, guidelines and checklist defined for our development project; we should also have guidelines, processes and checklist available for Automation as well.

3. Availability of Right Resources – The right set of resources is a must when you are doing automation. This means the resources should be skilled enough to design and code robust scripting, so that it requires minimum time for debugging during maintenance.

4. Commitment from Customer or Management – Automation is time consuming and resource intensive task. Customer or management commitment is required if one wants to get the real benefit of automation. This paper will highlight the approach wherein you can maximize your benefits within reasonable period of time.

Divide and Conquer in Software Testing

This is all about testing.

To break down the testing tasks, the following “what,” “when,” “how,” and “who” questions should be answered.

What Should be tested? During the test-planning phase, what to test and what not to test will have been determined as part of the scope of testing.

When should test procedure be developed? Test procedure be developed as soon as requirements are available. (unfortunately we don't have that Test Procedures). Once it has been determined what to test, the sequence of tests must be established. What needs to be tested first? The test planner should get to know the testing priorities, and should become familiar with the build and release schedule. Procedures for testing the high-priority items should be developed first.

How should test procedures be designed? No single testing solution can effectively cover all parts of a system. Test procedures for the different parts of the system must be designed in the manner most appropriate for effectively testing each of those specific parts.

In order to design the appropriate and most effective tests, it is necessary to consider the parts that make up the system and how they are integrated.

Who should develop the tests? Once it has been determined what must be tested, when it is to be tested, and how the test is to be accomplished, it will be easier to decide to whom the various testing tasks should be assigned.

Since user-interface, or black-box, testing does not exhibit all defects, gray-box testing must also be applied.

Keep in mind that black-box and gray-box testing combined are still not sufficient to procedure a quality product. Testing processes, procedures, inspections, and walk-through s, along with unit and integration testing, are all important parts of effective testing. Black-box and gray-box testing alone cannot identify all of the defects in a software program.