Let me explain. Did it ever happen to you that you read something or was told something, you believed you understood 100% of whatever the concept was about and then single-mindedly go forward in a direction that you felt had 100% focus. Only to realize, in the end, you are about 100% off target relative to someone else’s understanding.
The same thing just happened to me. I saw the EuroSTAR Call for Papers; I saw: “2015 Programme Chair, Ruud Teunissen is challenging you to share your experiences in software testing with the wider community!” Naturally, I thought I would write a 2,000-word life story of test-related experiences and share it with the wider community.
Then you do that thing where you send copies around to your friends and family for feedback. Even though you feel you are 100% on target, somehow, independent confirmation of 100% makes that 100% feel a little more like 100% and less like zero percent.
The one person I included in the initial review was Emma Connor who replied (and I am paraphrasing a little…) “That’s great, Dave. The only thing I would change is … all the words.”
Then Emma showed me why: “… implement the ideas, models or approaches that caught your eye. How did you “sell” it to management and your team? What challenges did you face when you started implementing the ideas? How did you manage the transition from old to new?”
Now, I am not a tester! I am not a sales-man! Certainly, I am no speaker or writer for that matter and my inspirations usually arrive once every 25 years. So, at the risk of stating the obvious, what am I doing here?
The answer to that is simple. The journey to “here” started when, one day, a dazzling flash of inspiration hit me. Suddenly, life took a new meaning and I wanted to make the world a better place! However, before I go further, let me just back up a bit and fill in some of the context.
I am not a tester. Actually, I started out as a Computer Programmer. I was young, the force was strong and the Dark Side seduced me. After spending a few years “Forrest Gumping” my way through various large-scale transformation programmes, one day, my development braces flew off and next thing, I was running into the arms of Business Analysis.
When I look back at those projects, there is without a doubt one common denominator: communication (more precisely, miscommunication). What would happen, without fail, is that everyone in the project would be 100% certain that they understood everything 100% and then go single-mindedly forward with 100% focus. Only to realize that, somewhere towards the end, the project is 100% off target relative to other expectations.
This sort of thing happens all the time but I did not realize it happens all the time until much later in my career: specifically, when I worked at Credit Suisse, Switzerland. Due to language barriers, writing Word documents full of Business Requirements was pointless. No one would read nor understand and sign-off went through the motions without conviction.
Then I tried using no words – just, models and diagrams (Enterprise Architect) to express concepts. If you have ever engaged a project during the modelling phase – you get this strange dynamic between the analysts and the developers and the boundary between requirement and solution becomes blurred by both sides. Again, misunderstandings appear in the model and bugs show up just before go-live. Models and diagrams – great in theory but in practice, the problem remains the same and worse, the business felt excluded.
Then I tried something else – quite by accident. I created Excel sheets. In Excel, I used tables to describe expectant results for a given set of input values. It just seemed a natural fit to the problem domain at the time. The impact to the team surprised me: developers “got it” and understood the detail clearly. The business “got it” and understood the detail clearly. Suddenly, engineers and business were speaking the same language without ambiguity.
Lack of ambiguity was most noticeable. For example, if the detail requirement exists in cell E1 = (A1*B1) / (D1–C1) then the developers had clear instructions irrespective of mother tongue language. Moreover, developers gained fresh insights into business assumptions: what should happen if D1-C1 caused divide by zero?
What I did not realize until later in the project (credit to a genius computer programmer called Sam Blume) was that I used the same Excel document to test code. I viewed the specification another way: the Excel document had all use-case scenarios to hand. Testing meant running code over the input values from Excel and checking that the output matched expectations.
At the time, testing was manual and error prone. Sam Blume had an insight: he could see how I used the Excel documents and so wanted to build a test-automation framework around the Excel documents.
Proving that neither Sam nor I are sales-men, we asked our Project Manager (Holger Knöfel) if we could build the framework. Holger said “no” but we understood “yes”. We built the framework and installed the framework on a continuous integration server (Jenkins). When developers checked in their code, a fresh build would generate automatically once all tests passed successfully.
The framework was a fundamental step-forward in a most subtle way that neither Sam nor I could have predicted. Jenkins would run through tens of thousands of tests in a matter of minutes and generate a new build once all tests passed. We deployed exactly what the business specified and the deployment never failed – no bugs, no noise and no politics.
With benefit of hindsight, Holger could see that using this framework significantly cut development costs. The framework “sold” itself. Later, Holger requested we present the framework to other teams within Credit Suisse. I created a PowerPoint presentation and I wrote a catchy title called “test-driven development”. I had no idea that TDD or Kent Beck meant something to others. TDD just sounded exactly what Sam and I were doing at the time.
Briefly, times were good. We had a cool framework that combined specification with test automation. Team productivity was an all-time high. We accelerated changes and all deployments worked– no bugs, no noise and no politics.
What happened next absolutely shocked me: a real-life Dilbert moment!
Do you know those production systems or strategic programmes that behave like a drunken uncle at some family wedding? They are loud, obnoxious, noisy and embarrassing. Individually, the components seem to work but fail when put altogether. Part of you wants to distance yourself but inevitably, you clean up the mess anyway. Well, our framework transformed the project from drunken uncle to the quiet little golden girl, sitting quietly in the corner.
When it came to next year’s budget, the steering committee forgot they even had a cash flow hedge tool (sitting quietly in a production corner) and forgot to allocate budget. Holger tried to argue the case but the response was “we forgot you had a production system. Too late now; all budgets have been allocated.” Consequently, without budget, my contract terminated and I joined Julius Baer.
The inspirations hit me about one week into my new contract. I had become used to our framework and it was horrible to switch back to the old ways. The project was crashing, falling over, needing constant support and consuming far too many resources than was healthy.
Now I was on a mission: determined to implement the framework.
First, I asked Project Management and users but the result was “no“. For Project Management, so long as the core components work, having known-bugs in production justifies on-going budget. Users did not want to impose any process over IT in case things backfired and excuses for non-delivery addressed project process. Developers were indifferent and business analysts were confused: testing is someone else’s problem and nothing to do with Business Analysis. Surprisingly, the only people who shared my passion for quality and efficiency were testers.
Selling to middle management is pointless. Middle managers focus too much on their daily process. They want to consume 100% of budget and claim another 30% next year to cover burn-in costs. Cost savings put their positions into question and guarantees less budget for next year.
Next attempt, I tried to get the concept in front of c-level management. Getting the airtime was surprisingly easy, provided I compressed the sales pitch onto one page of key comparable metrics. However, because the concept is new with no point of reference, they immediately pass down to middle management to review. Consequently, I was back at square one.
I am convinced the right way to sell is via c-level management. However, to solve the awareness, I decided to go public, meaning: social media, blogs and speaking at test conferences like EuroSTAR.
YouTube was easy. Getting past EuroSTAR was less easy. Ironically, the talk I submitted last year was much like this talk but the content felt too “salesy” and so after rejection, I just give up. Knowing when not to sell is important too, after all, no one buys a scratched record.
However, giving up is easier to say than do. Inspiration is like a good book: so hard to put down! The idea to introduce the test framework in Julius Baer never left my mind and I waited patiently for the right opportunity.
Then, my opportunity came. I was to participate on a regulatory programme and the Project Manager allowed me to use my framework.
This time, I played down the technology. I agreed with developers that I would do all their testing, provided I specify using my Excel documents. This was agreed and soon the developers signed-up to the method. Conversations would go: “I deployed to UAT…” – “I know, I can see the defects are still there!” Followed by shock, disbelief and insistence “I thought I fixed that…”
The cool thing about the framework is that, as designed and built by developers for developers, the test results are non-negotiable, unambiguous and highlight exactly the entry points.
Thus, with my team working at capacity, we were releasing new features every two weeks. Suddenly, Julius Baer was able to offer service updates every month: derivate, valuations, collateral reporting, etc. The legal department were constantly revising their contracts as new regulatory services came on-line every month.
This time, I refused to be the quite, forgotten project. This time, I planned to make the right level of noise at the right time. It was the dream project. It had visibility across many departments and playing down the technology paradoxically added fuel to fire of curiosity: “How can you test so fast? How you release every week? Why is the system not stable? What is going wrong?”
By November 2014 and this time deploying one release every two weeks, I had to handle a meeting very carefully. A key stakeholder was furious at having all these production releases charged to the project. Carefully, I explained we have a new development methodology and we deliver new features, not bugs. In fact, we just released a new patch to offer portfolio reconciliation services, which, I understand, is the most popular service with our clients.
Suddenly, the conversation switched and next thing I knew was that Central Projects Office arranged a meeting between Software Solutions Development and me to discuss ways to integrate my framework into the bank’s development stack.
This is how I managed to sell my ideas and this is what I learnt: never give up, create awareness, trust your inspirations and always capitalise opportunities that promote popularity.
I am Dave Cobbe, thank you for listening and see you all again soon!
