Being a tester by profession, I always try to write on testing related topics. Even though, I started my career in development, I switched to testing after just 6 months. Now after 13.5 years, I am looking back and thinking what could be the inherent meaning of test. I think inherent meaning of TEST is Trial of different thought processes, then Error finding, if any. Next, Solve those errors, if appropriate and lastly, it is Time bound together.
So, if I put this as easier way to understand, it will be like below:
• T- Trial
• E- Error
• S- Solve
• T- Timebound
We all know testing is nothing but trialing different thought processes and includes both verification and validation processes. These trial processes consist of different possible scenarios or scenarios based on customer/project/ market requirements or service level agreements for end-users. These scenarios can be thought of as looking at different quality attributes and overall they come under either functional or non-functional perspectives. For example, when we say functional testing, we are thinking about unit testing (done by developers), integration testing, system testing, and acceptance testing (done by testers). When we say non-functional testing, we are talking about performance testing (load, stress, endurance, volume, spike testing), security testing. We need to remember that these trials are not limited, however they can be extended to compatible testing, network testing, compliance testing or even software regulatory verifications.
In a nutshell when I say trial, I mean end-to-end comprehensive testing trials. Trials should be well defined thought processes & fully covered in all the above mentioned testing aspects, popularly known as test case scenarios and it must be properly documented for future references. Trials are like experiments and confirm if there are any errors. If you conduct trials/experiments effectively, there will be less chance of potential risk in a live system and your end-user will be much happier. The more you do different trials, the more likely the product will be better. Not only that, the earlier you do the trials, the better. This is like identifying the issues/defects at the early stages of SDLC – software development life cycle, which is less costly. Though traditional waterfall methodologies don’t support early trials, recent agile/dev-ops are in favour of early trials and continuous testing.
Errors are the next step – after conducting trials (like executing test scenarios) when the actual results or outcomes don’t match with anticipated results. Normally, after finding errors, defects need to be created. Even though it is an error as it does not match with expected result, it doesn’t mean it will be always a defect which will need to be solved later. This is applicable to both functional or non-functional testing. That decision (whether it is a defect) needs to be taken with care by the application owners or stakeholders looking at the end-user perspective or project requirement perspective, or if it is reasonable. If you think about the inherent meaning of test, trial and error are the most critical stages for successful software. When developers create the system, it is expected that it will work perfectly fine, but these trial and error processes verify and confirm that the system is ready for production. Overall, these trial and error processes reduce the product risk and give more confidence ahead of deployment. It is also more cost effective.
If you found errors after trials, the same trials are executed multiple times to reproduce the same error and then a decision is made by the application owner, as discussed above, before raising a defect. Defects are well documented and kept in a test management tool for tracking and from a future references perspective. Yes, I am talking about all aspects including a functionalities standpoint, usability or accessibility standpoint or a non-functional attribute like performance, security, availability, stability, scalability, reliability or vulnerability perspective.
As a next step, application owners or stakeholders need to solve those identified defects before the production release so that end-users are not affected. The developers need to work on the defects and solve the errors. These errors can be a functional issue like a button is not working or a non-functional issue like upgrading the server-side parameter to make the response time within the acceptable service level limit. Again, once the developers are done with changes or have updated the defect to resolved in the defect tracking tool or have sent a communication, testers need to do the trial again like re-trial (or retest) to ensure that the earlier error or defect is now solved and the expected outcome value is matching. Based on the re-trial result, defects can be closed in the defect management tool. Again, we also need to think about whether the changes made by development to resolve the defects do not adversely affect the existing functionalities and also from the non-functionality perspective. Therefore, we need to do re-trials like regression testing (may not be the case for all trials) to ensure that there are no additional or new errors. If an error is found during the re-trial, then that will now need to be solved. We need to understand that this trial, error and solve method is an iterative process. Re-trial or regression test helps to ensure that there will not be any surprises in production.
Lastly, the trials, error and solve process must be carried out in a timely manner. We need to understand, this must be so as we need to deliver the product within a certain time frame. We can’t do trial, error and solve process iteratively for long durations. If you do that, then the competitor will potential get ahead, and you will be lagging behind the market. The expectation for these 3 processes is that they will consume as little time as possible. Whether it is legacy methodology like waterfall (slow and only started once development gets fully completed) or recent agile/devops methodology (fast and testing begins during development early stages like shift-left testing), testing is a timebound process. So, it must be done within a certain time and these time periods are getting shorter and shorter. This is mainly due to a fast moving, tough and competitive market. Nowadays, end-users are looking for new features and they want them now. They want them everywhere, in every medium. As a result, not only accelerated development but also accelerated testing is now becoming a requirement.
We all know automation in testing is a major trend. The objective is to create fast paced testing. If you look at the inherent meaning of test, I think automation is absolutely assisting. First in trials, trials must be executed as only then will you find defects and if there are defects then they need to be solved and then re-trials are required. As I’ve highlighted above, overall this must be completed in a timely manner. Automation is hugely beneficial for continuous regression testing. Automation testing is fully in line with what I believe is the inherent meaning of test.
On one hand, automation testing is helpful to speed up the process and will ensure the timely delivery of the process. On the other hand, to make it more time efficent, I think proactive monitoring would be also helpful. Recently there are so many varieties of (standalone and combined) trials that you can possible think of, for example different types of devices, different operating systems, different browsers, different networks etc. It means there are many possible trials or scenarios that needs to be executed. Again, we are in the age of both the digital agents i.e. human and machine. So, both real user monitoring and synthetic monitoring assist to identify the errors proactively and solve those errors before it’s end-user are getting affected. This is again due to the timebound process and you can’t be late. Rather, you must deliver the product quickly for end-user happiness and do the business in long run.
In a nutshell, we can say that the inherent meaning of TEST is Trial, Error, Solve and it is undeniably Time bounded.
BIO:
Arun earned a degree in Computer science from Govt. Engg. College, India (college topper). He has 13.5 years of managing E2E testing delivery experience in different types of applications. He has a keen interest in reading and writing different technical papers. He was selected in multiple international conferences; global webinars and his papers have been published in multiple forums. Currently, he is working as a Senior Test Manager in Atos-NAO & Global subdomain leader for Atos Expert: Application-Testing.
