Testing is divided into manual and automated testing. The long term goal is to increase the automated testing. Manual testing is possible to start with minimal prerequisites and it is the task for QA engineers. The idea is to relieve developers of a difficult coding load with switching to test automation – to write simple automated tests for a short period of time (month). Consequently, it assures requirement coverage before (or simultaneously with) implementation. These approaches can be brought together using smart fuzzing.
On the other side, it’s possible to write simple automated tests for requirement coverage before implementation; such tests can be written by developers.
The presented SFTDD method integrates random testing (fuzzing methods) with simple automated tests in following steps:
- Create simple automated tests for requirement coverage
- Boundary value analysis
- Mutation-based fuzzing based on boundary values
The SFTDD is implemented with Robot Framework and Selenium 2.0. The proof of concept is confirmed in the real software development process (product size = 10 Engineer Years).
Diversity
At the highest level, the presented solution integrates three diverse areas: Management, QA approaches, and R&D actions.
Integration of diverse management approaches:
- Managing developers to efficiently execute diverse tasks: functionality development and test automation.
- Managing the complete team to systematically execute manually exploratory testing during a “testing day”.
- Managing of QA for test planning and coordination of test automation.
Integration of diverse QA approaches:
- Boundary value analysis
- ATTD (Robot Framework)
- Data driven testing (Selenium WebDriver)
- Fuzzy testing to cover random boundary values (it is not possible to cover all of it)
- Detection of modules for further code review and static analysis
Integration of diverse R&D actions:
- Feature-driven functionality development
- Test automation
Innovation
Agile test driven development as direct connection of testing and development implemented in the SFTDD model is in the process of becoming a European patent (EPC) – details are excluded in this presentation.
Leadership
Besides integrating diverse management approaches, SFTDD integrates flexible HRM and punctual fulfilment of requirements.
The Most Important Takeaways
First Takeaway
Fuzz testing is an efficient testing approach that ensures higher quality of the product in given time and resources. Smart fuzzing evolved from dumb fuzzing which evolved out of Monkey testing. Efficient smart fuzzing integrates domain knowledge into the testing process with a study of boundary value analysis.
Second Takeaway
The most important step for test-driven development (TDD) is a testing framework. The presented real-life example of a development project, using a testing environment with Robot Framework and Selenium WebDriver proves that writing automated test cases is very effective and attractive for developers. It is both a relieve and a reward. Application of procedures behind buzzwords “Data Driven Testing” and “Keyword Driven Testing” in a real development process indicates that precise managed test automation is very efficient from two diverse points of view:
- Requirement coverage is reached as a consequence of automated tests
- Verification of requirements from client’s and developer’s perspective
Third Takeaway
Which are the most important rules to keep in mind when planning the SFTDD process? How to convince the management to adopt SFTDD model?
Three steps of the innovative SFTDD model:
- Create simple automated tests for requirement coverage.
- Perform boundary value analysis.
- Mutation-based fuzzing based on boundary values.
The adoption of the SFTDD model:
- Developers start with the implementation without previously written test cases.
- Developers realize complexity of requirements.
- Developers realize that it is easy to write simple automated test cases for requirement coverage and that this is a very fast way to find out and understand requirements from client’s perspective.
- Start with the execution of a SFTDD model.
About the Author
Gregor Molan works for Comtrade as head of research group. His obsession is to merge concepts from graph theory into efficient QA models. He uses previous experiences from development of complex system models. He sets up QA model for Comtrade software products as a conjunction of R&D and QA expertise to assure the highest software quality with highly efficient test automation. Each QA model coordinated by him has at least one novelty imported from research or development.
