In this article, we are going to discuss the State of Mobile Gaming and examine mobile game testing with the key aspects of mobile game performance evaluation. To understand more about the mobile gaming performance, let us take a look at some interesting numbers to relate.
The above numbers give us a clear picture of the diversity of mobile games and the performance of any mobile game is closely related to its user experience. This makes us think more about the performance factor so that the players don’t get disappointed.
Striking graphics, rich media, animations and their effects makes the mobile game attractive, but if the performance of the mobile lags, then they are pretty much useless.
In the gameplay, gamers expect consistent progress and should experience a personalized touch of the game, and nothing should slow down their gaming experience.
This explains the importance of performance evaluation of every mobile game.
There is the wide range of methods and tactics to test the performance of a mobile game. Naturally, this leaves us with the option of using real devices (phone, tablet) and preferably different versions of those.
It’s not overstating to initiate performance tests in the early phase of mobile game development. The sooner we spot the performance bottlenecks; the better the game will be. Before the game gets published in Google Play or App Store, games should have run through different types of performance tests on all possible device configurations.
For example, the most common performance test done for mobile games is the general stress test to determine the responsiveness and stability of the game. The game should have the consistent performance for certain load or user interaction pattern. Below discussed are the mobile game performance testing checklist which should be considered before publishing a game.
There are three pillars Mobile Game Performance Testing
Device Performance Testing
When the user experiences the slow game, they get annoyed.
For device performance, the following should be checked –
Game Start Up
How much time the game takes to start up?
It is the first game performance parameter adjudged by the user. As a thumb rule, after the user taps on a game icon the first screen should be shown in 1-2 seconds.
Battery Time while playing
Constant use of mobile games consumes a high amount of battery life and heat the phone. This factor contributes a lot to the performance of any mobile game and could normally happen when the game is using more resources than required. Excessive resource usage creates a load on the processor and phone gets heated up.
Memory Consumption
When testing a mobile game, the memory consumption should be given priority. Adding more functionalities to the game increases the memory consumption.
For example, in a game with push notifications and social integration consumes more memory compared to other games.
In some cases, it has been observed that memory usage by the whole OS is mere 14%, but the game by itself consumes 11%.
So, this must be handled before launching the game or by launching it for beta testing.
Hardware/Software Variation
We have already discussed in this blog about the diversity of the mobile devices. It is mandatory to validate the game performance on different devices.
It could be the case that the game is running smoothly on one device but not on other. Like for different vendors of Android devices, the game should be checked on Samsung, HTC, and Lenovo phones.
Similarly, the game needs to be tested with different RAM and processor specifications.
Usage with Other Apps
When the game under test is running in parallel with other apps, there should be no interference. The best way to check it is by switching game under testing and other apps.
App in background
An app that is running in the background is retrieved, it should remain in the same state as it was before. If this scenario is not handled properly, then the data get lost. Again you have to enter data from scratch upon retrieving the app.
Testing Server/API Performance
When the game is interacting with the server via API, the response time becomes critical to evaluate the game performance.
For Server performance, the following factors are to be considered-
Data to and from server
The game should handle data efficiently that is sent from the server. It must not take too much time while loading the data. In certain games, data is sent in a specified format. So, before displaying it in the game, it should be converted to the relevant format. In this process, games sometimes become slower and response time becomes longer.
API Calls Generated from Games
The number of calls raised from the game under test should be lesser to the calls generated by the server. In some cases, multiple API calls are made for the same functionality. For better performance, this should be handled with less number of calls.s
Server Down Time
Due to some reason if the server is down or unreachable then the data can be saved in the native database. So, whenever the server is down, the data which is saved in the native database can be shown. Another solution could be the failover database servers i.e. if one of the servers is down or in maintenance phase the backup server should be available to switch over. The failover/backup server should always be in continuous replication and synchronisation with the main server.
Network Performance Testing
The performance of the game on different networks and network properties need to be measured.
For Network performance testing, the following things should be checked.
Jitters
When there is a delay in receiving information on the network, then it is termed as jitters. It is a problem with the connectionless networks or packet switch networks. As the information is distributed into packets, packets can travel by a dissimilar path from the sender to the receiver. When data arrives at the intended location, it becomes scrambled than it was originally sent. In the case of Jitters, the game should be built in a such a way to handle it.
Appropriate notification should be sent to the end user, either to resend the request or wait till the system responds again.
Packet Loss
In the case of complete packet loss, the game should be able to resend the request for the information or should generate the alerts accordingly. If data is incomplete, then the player will not be able to comprehend information displayed in the game. This can be stressful for the player. So, it is better to display a suitable message or prompt player to try again.
Network Speed
The app needs to be checked on a variety of networks with variable speed. The app should be tested on 2.5G, 3G & 4G networks. Both Wi-Fi and mobile networks are included in this. Also, the behavior should be monitored according to the network. Especially, when both networks are available, and switching between one network to another.
For example, an issue may arise in the game for the users while switching phone network from 4G to WIFI and vice versa. In this case, the game becomes unresponsive and may require a restart before the gameplay.
That’s pretty much about the performance factors of Mobile Game Testing.
Happy Testing!
