Just as in any other application software testing, we need to define our strategy for carrying out the testing of mobile applications. The strategy is developed based on the goals/objectives of the software, target user profile, functional and non-functional requirements, schedules and availability of resources among other things. Extent of test automation during development and for regression cycles, choice of tools need to be defined. Test strategy would also depend on the type/domain of mobile application (banking, social, health-fitness, game,)
Due to the fast pace of changes in the mobile market, large number of mobile handsets are being used by people with variety of OS versions, display sizes, network technologies, browsers and such other characteristics. Many of these characteristics can impact the rendering of mobile application on that mobile device. It would be prohibitive to test the application on all the mobile devices and hence, it is recommended to choose a subset of mobile devices (and OS/browsers) which are more likely to be used by the target users.
Typically, following aspects needs to be considered while preparing the test strategy –
- Functional testing – Feature tests, end-to-end transaction tests.
- Usability testing – To test for ease of use and satisfactory user experience, multiple language tests.
- Performance testing – To check performance of the mobile application in terms of response time under various connection options, file transfers, etc. Responsiveness of user interface.
- Compatibility testing – To test the application on a variety of mobile devices with different screen sizes running different operating systems and browsers; upward and backward compatibility; interoperability with other applications
- Installation testing – Testing the installation and uninstallation of the mobile application under stand alone and shared environments. Tests for user provisioning & de-provisioning.
- Stress testing – Application stability and behaviour with low memory, low speed and high transaction volume.
- Security testing – Security of data during transmission and on device.
- Operational testing – Tests for built-in health-check, backup, recovery, data integrity during upgrades, etc.
- Conformity Testing – Tests to check adherence to industry standards and guidelines (telecom, W3C, financial, PCI-DSS, prohibited contents, App store policies)
Generally, it is suggested to carry out all the tests – both functional and non-functional with one combination of handset-OS-browser. And, on all the other combinations, one can carry out a subset of tests of all types especially compatibility testing and user experience testing.
Source : “Software Testing – Effective Methods, Tools and Techniques“ book authored by our director, Mr Pradeep Oak and published by McGraw Hill.