SWIFT’s Global Test Strategy
Shifting left in a critical quality environment
SWIFT has always been known for our relentless focus on quality, security, reliability and delivering on our world class 99.999% availability. This laser focus is supported by two key success factors – our software delivery process and our FNAO mindset, deeply embedded in our organisation.
In the Volatile, Uncertain, Complex and Ambiguous (VUCA) world we live in today, there’s another factor that allows us to stay relevant – our emphasis on continuous improvement. How can we maintain our level of quality and excellence whilst delivering more and at a much faster pace? How can we adapt more quickly to evolving market and customer requirements? And how can we speed up the adoption of new technologies and continue to stay ahead of the game?
A holistic global test strategy
Delivering on our commitments to excellence and quality, our software delivery life-cycle comprises numerous checkpoints from code review and functional testing to resiliency and security testing. With the ever-changing competitive landscape in the financial industry, the importance of constantly re-evaluating our processes has never been more important, asking ourselves questions such as:
- Are we running the right tests – and at the right test level?
- Do we have redundancies across the different levels of testing?
- At which point do we identify the most critical issues?
Our test strategy is made up of in-depth tests at multiple levels of development, but having a holistic, end-to-end view on how the different levels of testing fit together could bring even more benefits to SWIFT and our customers.
After extensive process reviews, a new global test strategy was implemented, aiming to reduce time to market and continuously improve our quality. It comprised two main guidelines; test effectiveness, or finding defects as early as possible in the software delivery cycle, and quality predictability, providing feedback as early as possible and making it fully transparent.
Shifting left in an operationally sensitive organisation
Our community expects excellence on a daily basis, so we decided on a gradual approach in implementing our new strategy – this would help avoid any impact on the quality expertise we had developed over many years. So just how did we handle such a major change?
Start small
We began by focusing on a small number of products across a number of different portfolios. Each product delivery team provided their view on the global test strategy and shared their findings on a monthly basis. This feedback was then used to define the standard framework from the bottom up.
Single unique framework with management alignment
After the pilot phase, the GTST was incorporated in our software delivery process and tracked by project managers. A series of roadshow presentations was organised to explain the key benefits to internal stakeholders and build solid senior management support.
Quality dashboard
Being able to get critical insights into our testing process was also key. A simple reporting tool was developed, with a series of graphs helping us visualise different aspects of quality in real time:
- Defects leakage: every defect was tracked to determine where it was found versus where it should have been found.
- Test automation pyramid: the number of tests created per test level was tracked. More automation should exist at earlier test levels as defined in our test pyramid (see below). A high level of automation later in the testing levels (so higher up the pyramid) could impact speed and quality.
Testers and developers, together from the start
As is often the case, we found that our testers were often working one release behind our developers. Embedding these two critical roles together from the start of the cycle, the quality and speed of feedback improved significantly as did the alignment between development and testing teams.
Tickets to EuroSTAR 2019 Now Available – Book Now and Save!
Continuing excellence
Our transformation is still underway, but we have already seen a reduction in time to market from 12 to 6 months for newly launched products, and from 2 weeks to 3 days for the delivery of security patches – with no impact on quality. The Global Test Strategy continues to re-define our role as a quality organisation, allowing our teams the freedom and empowerment to focus even more on improving the way we deliver quality.
Ready for DevOps
The Global Test Strategy is speeding up our move into DevOps. The GTST describes the full testing flow from concept to delivery stage plus the build promotion criteria between each test level. These are now being used to design the automated software delivery pipeline to production. If all build promotion criteria are met per test level, the pipeline ensures that our high quality objectives are met and as a consequence the software could be deployed automatically in production.
