With over 3 million apps available between Android and Apple, to remain competitive you must provide users with high quality, high value apps. Consumers expectations for quality and performance continue to increase. Users will quickly lose patience if there are app defects and delays. Statistics by Slick Text have shown that 74% of customers won’t wait more than five seconds for an app to load before abandoning it. Eighty percent of mobile users won’t try an app more than twice. A third of those users will go to your competitor. You need to ensure that your application is top notch. To do this you will need to properly test your app for its functionality, usability, and consistency. There are some very important things that must always be considered when doing mobile testing.
1. Understand the type of mobile application
Consider the type of mobile application that you are testing. This will help to determine the elements that you will be testing. The differences mainly come from the design of the application. There are three main types of mobile applications:
- Native Apps- These have been designed for a particular operating system or device. They have access to the various features of a phone such as the camera, GPS etc.
- Mobile Web Apps- Stored on a server and delivered via the internet through a browser. They are more like websites that are made to look and feel like native apps. When testing mobile web apps, you will have to test on various mobile web browsers.
- Hybrid Apps-This is a combination of the Native and Mobile Web Applications. They run on the device and utilize web technologies such as HTML. These are created using some tools like PhoneGap or Cordova to create cross-platform apps which would dictate the type of tests to be done and the restrictions of the apps themselves.
2. Use a check-list
Make a test plan of the items and functionalities that need to be tested. Here are a few:
- Battery usage
- Speed of the application
- Data Plan/Wi-Fi requirements
- Memory requirements
- Functionality of the application (all requirements/user stories)
3. Use both emulators and real devices
Emulators allow you to test multiple devices, however they do not have the specific faults and characteristics that real devices will have. Physical devices allow you to test the exact firmware, model, and operating system that will be used by the customer. This is an expensive practice and may result in the Software Quality Engineer having a small range of physical devices at their disposal.
4. Test on a wide variety of device/OS combinations
In order to increase test coverage across devices and platforms as well as reduce risk to your app’s functionality, it is paramount to test on various devices and platforms. For example:
– Application memory leaks with larger images displayed on larger screens
– Visual alignment and design could be off depending on the screen resolution and size
– App integration with social applications may respond differently on different operating systems
– Responsive websites may appear differently depending on the browser and browser version used on the mobile device
There are some questions that you will need to consider when choosing the devices and platforms.
- What are the requirements around OS/platform support?
- Are there features in your application that are only on specific versions of iOS or Android?
To keep up with what are the most popular device/OS combinations you can go here:
- http://www.idc.com/prodserv/smartphone-market-share.jsp
- http://www.idc.com/prodserv/smartphone-os-market-share.jsp
- http://www.statista.com/statistics/266572/market-share-held-by-smartphone-platforms-in-the-united-states/
5. Automate your tests.
It is important for you to automate your tests (especially if you are testing several devices). This allows for faster testing across multiple devices and platforms. This will also provide you with quick and precise results. You can consider using tools such as: Appium, Calabash, MonkeyTalk, iOS-driver, UI Automator among others.
6. Test the orientation
Some apps can only be played in landscape mode or only portrait. Which mode will the app be played in? Is it restricted to one mode? If not does it crash when switching from one mode to another? You should also test the following:
- Is the screen re-drawn correctly? The User Interface (UI) code will change in the orientation. Does your app handle this well?
- Is the state of the application maintained? Tests should check whether or not the information that was entered by the user is lost or retained. The application should be in the same state that it was in before its orientation changed.
7. Test interrupts to the application.
How does the application deal with interrupts? These interruptions include the following:
- Phone Calls
- Text Messages
- Battery dying/died
- No Wifi/No Data
- No/Low Storage
- Other application notification
8. Test Resource Dependencies
Most applications depend on network access, SMS, Bluetooth, or GPS. If your application depends on external resources then you should test how the app responds when the resource(s) are not available. Test if the app shuts down gracefully or if it just crashes.
9. Test the Security of the Application
What measures are in place to protect user information? Is there a privacy policy that the user has to agree to? Does the application give out personal information of its users? Does the app require the users to enter information such as credit card information? Is the user informed of the use of the information that they are providing?
10. Test Application Usability
- Will users be able to navigate the app?
- Is there a help menu?
- Are the instructions given clear and concise?
It is a good idea to have a diverse group of persons test the app to see the ease of use of the application and to identify various issues across a wide user base. Additionally, you should also test the familiarity of the app. This is as people tend to respond well to familiar things.
11. Don’t forget to test on tablets
Most developers do not test their application on tablets which results in low user satisfaction and low app ratings. When testing on a tablet check to see if the developer created a different application for the tablet or a different set of resource files. This is as the tablet would have more space which could result in a lot of white space and an undesirable user interface if not utilized properly. Also the tablet does not have a cellular network connection like phones. Hence, the app should be able to handle this.
There are several elements to test:
- Was higher quality graphics used such as HD (the tablet is much larger than a phone)?
- Does the application use all the available space?
- Does the layout, font and spacing fit the tablet size?
To have a high quality app, it is very important to utilize these best practices in mobile testing.
Let’s review:
It is important to understand the type of mobile application, always have a checklist, use both emulators and real devices, test on a wide variety of devices/OS combinations, automate your tests, test the orientation and interrupts to the application, test resource dependencies and the security of the application, test application usability, and don’t forget to test on tablets.
Author: Julia Pottinger – QualityWorks
References
http://istqbexamcertification.com/mobile-application-development-and-testing-checklist/
http://www.softwaretestinghelp.com/beginners-guide-to-mobile-application-testing/
http://developer.android.com/tools/testing/what_to_test.html
