In recent years we have heard a lot of buzz around the terms DevOps, Continuous Delivery and more recently Programmable Infrastructure.
All these names are new but hiding behind them are the same challenges we been dealing with for the last 15 years.
History of Testing
Software testing was introduced as a separate discipline in 1979 by Glenford J. Myers. Before the 80s, testing was an integral part of development and mainly included debugging. During 80s and 90s, testing grew from a small team within a development organisation to a separate department, creating a new discipline typically called Quality Assurance. This allowed developers to focus on writing code without the worry of testing it on growing variety of platforms in almost infinite number of situations.
At the end of the 90s testing was well defined and test automation was in daily use by growing testing departments.
Agile Movement
The growing efficiency of both development and testing teams led to the exposure of a new pain point the handover between the teams. Finding and fixing bugs were relatively simple tasks, but the time from the discovery to the fix constantly grew. Before introduction of Agile, testing was done at the end of the widely used waterfall development process. Prerelease integrations took weeks or even months making release planning almost impossible and causing degradation in the quality of produced code due to shortcuts taken under heavy pressure of approaching release date.
Agile was introduced as the way to break out of specialisation silos within development organisation and reduce the cost of handover between development and test teams to an absolute minimum.
Continuous Integration (CI) was a key technical practice to allow smooth flow of changes from the beginning of development till the release day. It was designed to expose integration problems as early as possible, significantly improving the quality of the code and eliminating costly prerelease integrations.
This was the point where I started my career, first as a Software Developer in 1999 and later as a Software Configuration Manager in 2005. During these years CI was the holy grail of the software development and many companies were practicing or experimenting with it and I had an opportunity to be part of many transitions to Agile and attempts to implement CI.
Agile teams
Typical transitions to Agile at the beginningmid 2000’s started by mandating Scrum but without making any significant changes to the organisational structure. Dev teams simply became Scrum teams with a new way for planning their work and different parts of CI implementation were carried out by by specialised siloed teams such as development, build and test.
The success of these siloed teams led to great improvements in each stage of the delivery pipeline that increased the productivity of each siloed team. This exposed a new bottleneck the coordination of changes requiring the interaction of multiple teams.
In many cases, just assigning an issue to the right team took longer than all the other fix activities all together.
This was the perfect time to implement the organisational change needed to reducing the cost of handover between the specialised teams. A new type of crossfunctional team, able to perform 99% of their daily activities without external dependencies started to become a norm.
This organisational change had significant impact of the tools used for development.
Unifying toolset
Up to this point test automation tools were replacing humans by imitating their actions. Most of test automation tools at 90s were UI oriented and heavily used recording to repeat the same action over and over again.
Such tools were difficult to maintain as development was done in the Dev team but test automation created and owned by a separate team. This situation made such tools less approachable by developers, increasing the time it took to understand the real problem behind an alert received from such systems.
Newly created agile teams quickly replaced these tools with the new type of fully programmable testing frameworks such as Xunit, Selenium etc. This allowed the crossfunctional teams to define the tests as an integral part of the software moving it to much earlier stages in the delivery pipeline. Some teams were even able to practice TDD and write tests before even single line of code was written.
Repeat of the history, DevOps and Programmable Infrastructure
The story of software testing splitting to a separate discipline to allow local optimisation and later absorbed back into development to reduce the handover costs and by that creating a new type of fully programmable testing is now repeating itself.
- The new players are Continuous Delivery (CD) which is effectively CI taken all the way to the clients
- DevOps teams which are nothing more than Agile teams that have responsibility for deployment and maintenance of the software
- Programmable Infrastructure (PI) which is the same as build and test automation done by developers within Dev teams.
In the last five years we see the clear approval by the market for the need for the DevOps teams and the creation of the first generation of the PI tooling such as Puppet, Chef, etc. Those tools are very similar to the test recording tools that we saw in the 90’s and in the same way as they were replaced by programmable tools such as xUnit and Selenium, we are witnessing now a rise of the new, fully programmable tools to manage infrastructure with software containers as their core. The popularity of Docker, the current leader in this field, is a clear sign for the need for this change.
Conclusion
Every now and then we see a new set of buzzwords to explain some new phenomenon happening around us. Very rarely they represent something truly unique.
These days we hear a lot about DevOps, Continuous Delivery, Programmable Infrastructure and other interesting concepts in the same are of delivering our software to our clients. Although these terms look like something new and fresh, I think we are experiencing just another cycle of the same Agile transition that was started 15 years ago.
And other the practical note it means that we don’t need to reinvent the way we implement these practices but we can learn from our own experiences from just few years ago.
About the Author
Pini is the CTO of Container Solutions, he has 15+ years of experience in delivering software in Israel and Netherlands. Starting as a developer and moving through technical, managerial and consulting positions in Configuration Management and Operations areas, Pini acquired deep understanding of the software delivery processes and currently helping organisations around Europe with improving software delivery pipeline by introducing Docker and other cutting edge technologies.