What are the The Principles Of Test Automation? Let’s go back to the start. Today’s organisations must adapt faster to keep up with the changing needs of customers. This places major demands on the development of applications. Customers not only demand speed, they also demand a large degree of user-friendliness. If, for instance, a web shop does not function properly, an alternative can quickly be found. Agile software development closely matches these change demands and has already found its way into a large number of organisations.
In the quest to keep up with increasing amounts of (test) work, the contribution by, and importance of, test automation is growing. However, for test automation to support organisations, it requires changes to people and (test) processes. As a consequence, a good understanding of the impact of test automation is important. Architecture can assist because it can provide insight. It allows organisations to make a structured plan that incorporates all the important aspects.
As its starting point, architecture uses a set of principles that indicate what is important. Because, for the most part, these principles are identical for all organisations, we have identified a generic set of principles for test automation. We have applied our testing, test automation and architecture knowledge and experience to describe what we believe to be important. These principles are likewise a starting point for a book about test automation that we are writing, which further details the principles and the architecture, while at the same time serving as an ‘umbrella’ that covers the rest of the book. The following principles of test automation shed light on the organisational and information systems aspects respectively.
Organisational Principles of Test Automation
Principle 1: Test automation aligns with the objectives and maturity of an organisation.
Test automation is not an objective in itself. Instead, it contributes to realising the objectives of an organisation. The extent of its contribution varies as per organisation. It requires an investment that must be justifiable and, in particular, requires organisational change. For example, it leads to a shift in the duties of the test engineer, who receives more software development tasks. Because of this, a thorough analysis of the organisational context and the cost-benefit of test automation (business case) is essential. Maturity models can assist in defining realistic objectives and provide a better overview of the work required.
Principle 2: Test automation is based on a defined vision, policy and architecture.
The implementation of test automation must not be taken lightly. It is a relatively complicated topic that demands a range of decisions. These include decisions on the positioning of test automation in the organisation and its setup. Making conscious decisions will help set up test automation in such a way that it is future proof. But it is also possible to overshoot here, so a one-size-fits-all approach to test automation must be avoided. By compiling a test vision, insight can be obtained into which particular objectives are pursued by test automation and which particular risks are addressed. Test policy establishes the most important decisions in terms of how tests are handled and the role played in this by test automation. Test architecture describes the key practicalities of tests, such as the processes to be employed and the tools to be used.
Principle 3: Test automation takes the human factor into account.
Humans are the crucial factor, also when it comes to test automation. The term ‘automation’ might suggest that people become less important, but nothing could be further from the truth. Tasks are rearranged and actually place greater demands on people. Everyone involved in testing must understand why test automation is important and what their involvement entails. This means that the impact on roles required for test processes and on associated competences will have to be closely examined. On one hand, repetitive tasks will fall away and the emphasis will be placed on analysis of test results. On the other hand, the importance of the role of the test engineer will increase. A change-based perspective on test automation is therefore very important.
Principle 4: Test automation requires a balance between comparison risk and effort.
Testing and the automation of tests take time, but testing everything (repeatedly) takes too much time in many cases. It is therefore important to focus testing efforts on issues that require the most attention, or on items threatened by the greatest risk. In the past, calculations on whether testing added sufficient value have not always been accurate. But test automation demands even greater amounts of work, so it is more important to accurately define the added value in advance. It is therefore essential to use a calculation model that defines all the factors that determine risk and effort. Such a calculation model will have to be intelligent so that items influenced by automation can be distinguished from items that are not. A more coarse-grained version of the calculation model will also have to be applied when it comes to compiling the test automation business case.
Information systems principles of Test Automation
Principle 5: Test automation is model-based.
During the test process, various models and data are used that relate to the application being tested. These models can be applied directly in the test process for the purpose of e.g. the automatic generation of test scripts and expected test results. This ensures that development and control efforts are minimised, that inconsistencies are prevented as best possible and that changes to the application require the least possible changes to test scripts, which can then be regenerated. Establishment of functionality in a structured, model-based form places greater demand on analysis and design processes. Functionality is preferably described in pseudocode.
Principle 6: Test automation data is explicitly governed.
Data plays an important role in test automation. Because process quality is highly dependent on data quality, the latter requires categorical attention. The same therefore applies to the data used during test automation. Attention paid to data governance involves – amongst others – the clear definition of tasks and responsibilities. Who, for example, will be responsible for the testware once a project is complete? If the right arrangements are not in place, there is a realistic chance of testware being inconsistent and therefore unusable. For this reason, version and configuration management of the data involved is essential.
Principle 7: Test automation explicitly takes data security into account.
Customers expect that organisations handle their data with care so that it does not end up in the hands of unauthorised parties. Clear restrictions related to this are defined by legislation and regulations, such as the Data Protection Act. This means that test automation must also pay specific attention to data security. It should not be possible to trace data used during testing to individuals (e.g. customers). Data of European organisations should remain within European borders and should not be influenced by specific authorities, within for instance the context of the Data Protection Act. In a more general sense, attention must be paid to data security risks and precautions and the policy necessary to guarantee security.
Principle 8: Test automation tools are necessary but should not be leading.
Test automation requires support from tools, which serve to make the automation possible and carry it out. However, it is important to realise that tools only offer support, while the real challenges are organisational by nature. Tools should above all support the objectives and risks, and not more than that. A set of simple tools can be sufficient for some organisations and applications, although more a more elaborate toolset can be required in other situations. The selection of test tools should therefore be based on the test vision and test policy. Expensive tools are not, by definition, better tools.
Initiative
In this article, we have described the things that we believe are important when it comes to the setup of test automation. In our opinion, an architectural approach contributes to better understanding and greater insight. Test architecture is therefore an important competence within the context of test automation. We are curious to know whether others recognise the principles of test automation that we have outlined so that we can improve them.
Who are the authors?
Danny Greefhorst is director at ArchiXL. Jos van Rooyen is principal consultant at Bartosz. Marcel Mersie is a project manager and consultant at Bartosz.
