Release Day: The Good, the Bad and The Ugly?

[Ronan Healy]

I thought this was an apt title for this discussion, I’m sure you all have different experiences of release day. Just like there are so many methods of testing, I imagine there are many different roads to release day.

I am curious on how everyone manages release days. Are they always the same or does every release day bring different challenges?

I would love to hear people’s stories about release day; the good, the bad and the ugly!

[Neil]

Well, I can start you off with one of the very strangest…

I was working for a (now-defunct) web design company, who’d been contracted to produce a portal for a particularly large government-backed organisation. In their infinite wisdom, the client decided they wanted the new site to go-live at midnight, but for some reason we didn’t invest any time/effort in coding for this to happen automatically.

In other words, release day night involved the development team going to the pub, and heading back to the office just before midnight. Those of us who’d drank the least (myself included) were tasked with deploying the site and watching the logs; one of my colleagues was being sick in the toilets at the time…

I’ve been through some stressful releases, but I’m proud to say that at least I’ve stayed sober through the rest of them 🙂

[Ronan Healy]

So for some, does Release day happen something like this?

[Andrés]

@Ronan

Firstly, this is some great topic you have brought up 🙂
Secondly, I had a great laugh at the picture you shared illustrating (to a certain extent) what happens during “Release Day”.

I’ve had several experiences, some were good (very few of them), and some were not so good (most of them), I’ll share one with a little disclaimer “Be warned, the following story is based upon real events, any resemblance with any other project… is not a coincidence :P”

— The Ugly

We had to support the release to production of a major version of one of the company’s flagship throughout a weekend. The release took longer than anticipated, so everyone was soon as… “Houston… we have a problem!” The group chat we were in was red hot, and though we spoke our opinion on the matter, none seemed to care to listen.

— The Bad

At certain point throughout the Nth attempt of deploying the build, and having it failed for the Nth+1 times, someone just out of the blue happened to be enlightened and said “Could it be possible we were having a problem with the build?”. Literally, I facepalmed and laughed at it, we had spoken that out loud several times with none willing to listen, but at least, they were now going to check it. After a few hours, the problem was addressed, new commits were pushed to the CI server and after a while, a new build was available for its deployment.

— The Good

Once the fix was implemented and there was a brand new build, not only was it successfully deployed, but also did it work as intended. The tests passed, the people in the chat was advised and they were all happy, reports were written and sent, certain stakeholders praised the effort the entire team engaged into.

Now, I’d like including another point here which I’d like calling: The “Funny”

— The “Funny”

As we had the chance of talking with the developer who solved the problem, he mentioned having done so without his laptop, he had to use his smartphone instead. You see… the thing is that he was on vacation with his family, they were out of town, and because they wanted to have some family time together, the only one piece of technology he was willing to take along with him was his smartphone, so against all odds (because he was not a part of the project anymore) he was paged, and connected with a remote desktop app to a computer, and that was why having a new build available for its deployment took several hours. See? I told you it would be “funny” 😀

A question to the community, what have your experiences been like in the matter? What would you like sharing with us?

Looking forward to your comments.

Cheers,

-Andrés

[Archana]

There was this “release day” one day later and the code was frozen with all major defects fixed. One of the developers happened to make a “minor” change in the code or BRE rules (which was obviously without approval and without knowledge of anyone else).  And on the release date, someone thought of running a quick smoke test just to find out the major functionality was broken. Release had to be postponed for a day.

I am sure this must be a pretty common scenario happening very often

[Author]

Posts