I always try to write about how to make testing easier to remember and understood by everyone. System performance is crucial for project success and ensuring the satisfaction of the end-users. In this article, I will identify the meaning of performance. What comes to mind is that each of the 11 letters of the word “PERFORMANCE” tells the whole performance story. Each of these 11 letters communicate about a quality attribute and all of them are required in overall PTLC-performance testing life cycle for ensuring end-to-end system performance.
Below are the 11 letters of the word “PERFORMANCE” – each representing a different quality attribute in the overall performance testing procedure.
1. Predictability
2. Efficiency
3. Responsiveness
4. Failover/ Fault tolerance
5. Observability
6. Resilience
7. Magnitude
8. Availability
9. Notification
10. Capacity
11. Extensibility
Let’s talk about each of the 11 letters. By understanding these 11 quality attributes related to performance, it will be easier to understand the whole performance testing story which covers performance testing process, it’s requirement, different types of performance testing and why they are required in PTLC.
Performance Testing is mainly carried out in pre-production environment to get an idea well in advance, how the system is going to behave in live. This reduces the overall project risk, gives assurance ahead of deployment. This is in line with first letter P. For me, this P stands for Predictability. Performance testing results gives a predication ahead of deployment to the system owner and stakeholders, it confirms whether to go ahead for live system, tells the current state (performance) of the system in terms of different SLAs- service level agreements and lastly identifies the performance improvement scope, if any.
Moving on to the second letter of the word – for me, this E stands for Efficiency. When I say efficiency, I mean how efficient the system is compared to different SLAs – like response time, throughput, resource utilization (like, % Processor Time (_Total) or % Committed Memory), availability, scalability etc. Efficiency doesn’t highlight any potential problems but it’s useful as an indication of how well it will perform in live. Efficient system gives assurance to the system owners, guarantee that end-users will be happy, and the product will endure in the market for a long time. Both these P & E letters convey the importance of performance testing.
The third letter, R stands for Responsiveness. We know load testing is conducted to measure the responsiveness (i.e. capturing response time) of a system under a specific user-load. Load testing, a type of performance testing, assists to capture the speed/responsiveness of the system. This is required to compare with the SLA, to know the current situation and finding the scope for improvement, if any. Normally, load testing can be done with average load, peak load, or even little higher than peak load looking at the future user’s perspective.
Next is F. This can be either Failover or Fault tolerance. Failover testing is conducted to ensure that the system can handle extra load during a failure. We can think about load balancing concepts here. A good example is having a system with 4 web servers and due to heavy load, if one of the web servers fails, the remaining 3 web servers can handle the existing load. Fail over ensures that the end-users of the failed server are automatically redirected to the working servers. Whereas fault tolerance ensures that a system can continue operating properly in case of one or more failures of its components. Fault tolerance is mainly for high-availability or life-critical systems. Fault tolerance would automatically help the system recover without changing to another server.
The fifth letter, O stands for Observability. Observability or monitoring in performance testing plays a major role. Observability will be completed only when monitoring data is captured. Monitoring collects and displays this observable data. After the monitoring data is available, performance test result analysis will be done to see if there are any performance issues. Monitoring is a critical part of performance testing execution. It collects all the performance counter metrics which assist to analyse the performance test results, pinpoint where exactly performance issues lie and scope for improvement, if any. Observability or monitoring is required both in performance environment while conducting performance testing and in production environment for analysing live user experiences. Today’s APM- application performance monitoring tool (or even DEM – digital experience monitoring for both the digital agents: human as well as machine, supports both synthetic & real user monitoring) proactively identify the performance issues, if any, locate and resolve them immediately before the actual end-user gets affected and ensure customer satisfaction.
Next we will talk about R. For me, it is Resilience – recovering quickly from the sudden difficulties. Spike testing is a type of performance testing which ensures the system is ready for extreme conditions. It is conducted to see the behaviors of the system when suddenly increasing/decreasing the user load by a very large amount, and whether the system can come back to its normal state after the extreme duration. It is to measure how much the system performance deteriorates when there is a sudden high load and how quickly the system gets back to normal.
The seventh letter, M stands for Magnitude i.e. database size, or even the data size (large data). Volume testing, a type of performance testing, is done to verify how the system will behave on a large database or large data and even when the database grows, looking at the future. Overall, volume testing assists to identify the potential performance problems that are likely to occur with increased the magnitude of data and resolve them well in advance.
Eight letter of the word “PERFORMANCE” is A. For me, this stands for Availability. 24*7 availability is common for all the applications. Soak or endurance testing is conducted to determine if the system can sustain or survive for prolonged durations under the continuous expected load. Think about sustainability perspective or to find issues like memory leak or whether the database connections are getting closed properly in longer durations etc. All of these will be seen with endurance testing and the application’s stability and availability will be confirmed.
Ninth letter, N stands for Notification. When I mentioned, observability or monitoring earlier, that would not be fully completed with notification. Monitoring notification or alerts assist to let you know well in advance if there are potential performance issues. It boosts operational efficiency level in production and you can proactively identify the potential glitches and resolved them before they affects the actual end-user. Different levels of notifications assist to avoid any missing issues. As your system is 24*7 available, it is simply impossible to watch and monitor the system all the time. Notifications assist on this. Thresholds based notifications are based on utilisation, timing or frequency level. Basically, the operational team can’t afford to miss any performance level notifications. This is very important in ensuring the overall system performance.
Tenth letter of the word “PERFORMANCE” is C. This C stands for Capacity. To know the application’s capacity, we stress test and identify the scalability of the application. This stress testing determines how many users or transactions a given system will support whilst still meeting performance goals and shows the overall capacity of the system. It suggests the capacity planning of the system to plan growth like increased user load or increased transactions. It also determines whether scale up or scale out is required.
Last letter, E stands for Extensibility. For me, extensibility is an absolute requirement for any system and extensibility is for future growth. In today’s changing and highly competitive market extensibility should be always part of the system. We need to remember when extensibility is supported by the system, performance should be similar. Response time should be consistent, increased numbers of users will be supported, more transactions will be supported, system will handle the sudden spike situation & overall resource utilisation will be similar (should be optimum).
So, the above 11 letters tell the whole performance story that every company is looking for its end-users and for its product / systems to be in the market for a long time. In a nutshell, we can say that meaning of the word performance is in each of its letters which makes it easy to remember!
