Key Challenges to Test Execution and Reporting
Some of the key challenges while executing tests and reporting results include the following:
1. Test cases are not detailed enough or not understandable
Recommendation – It is important for the test team to document the test case steps and narrations to make the test cases easier to understand. Also, while acquiring an existing test suite, the test team should undertake a knowledge transfer.
2. Test environment is not controlled by test team (controlled by development or other team)
Recommendation – Test team should have a connect established with environment support team. Also, service level agreements for the environment support requests should be established and adhered to.
3. Test data is not prepared or no tools exist for test data creation, especially in case of need for large
amount of test data
Recommendation – Test team should get involved early and the need for required test data should be established early. Use of test data management tools is recommended in case of large amounts of test data or when dynamic test data is required.
4. Unreachable code (to achieve 100% code coverage)
Recommendation – Functional testing should be black-box (not code-dependent). The coverage should be measured through coverage of requirements and not code. Code coverage should be the criteria primarily in unit test cases.
5. Not knowing what is new when conducting regression tests
Recommendation – A log of new functionality for planned releases should be maintained. Also, the test management tool should be updated with specific new functionality added in specific releases. Test cases should be tagged to specific functionality.
6. Insufficient time remaining for test execution due to delay in development activities
Recommendation – Testing team involvement should be planned early in life cycle. Testing should be carried out in parallel to development life cycle. Also, test cases should be categorised with classification of ‘Must Do – Mandatory’, ‘Should Do – Preferable’, ‘Could Do – if time permits’. In case of shorter window for test execution, the tests should be executed in this order of priority.
Source: “Software Testing – Effective Methods, Tools and Techniques“ book authored by our director, Mr Pradeep Oak and published by McGraw Hill.