Testing
Failing to adequately test an app before launching can severely affect the chances of success. This is especially important because the publishing process may take some time (up to two weeks) depending on the platform. Furthermore, if an app is launched with bugs, it will quickly get negative reviews which may discourage potential users from ever trying it out.
Test categories
Development: includes unit and integration tests created by developers; each test verifies a small subset of the overall system.
Functional: verifies that the system satisfies requirements.
Experiential: verifies that the software meets user expectations; includes
exploratory, usability, acceptance and alpha/beta testing.Non-functional: validates that the software meets architectural quality requirements; includes performance, load and security testing.
Test environments
Framework: unit and integration tests are run automatically by every change to the source code.
Emulation: test the app using software that simulates physical devices.
Device-in-hand: test an app directly with a physical device.
Device lab: a vendor-hosted or on-premise software as a service (SaaS) platform that gives access to a broad spectrum of vendor-managed devices for automated testing.
Crowdsourcing: a managed relationship of individual testers who are contracted to test on their personal devices.
Choosing tests categories and environments for apps
| Test Category | Suitability | Test Environment | |
|---|---|---|---|
| Internal Apps | External Apps | ||
| Development | High | High | Unit and integration tests can be carried out automatically by a continuous server. Device-in-hand and emulation are well suited for development tests due to the need for rapid feedback and automation. |
| Functional | High | Medium | Device-lab, device-in-hand and emulation are suitable for functional testing due to the opportunity to create repeatable test scripts and the need to test the complete business process. |
| Experiential | Medium | High | Crowdsourcing is suitable for experiential testing because independent users can discover issues not anticipated or encountered during development. |
| Non-functional | High | Low | Device lab and device-in-hand are optimally suited for non-functional testing due to the tool requirement that accompanies testing software under load and assessing application. |