An Agilist's Tale - Great Expectations

PART 1 

A three part blog series about a history of Agile told from our protagonists view. 

I’ve always been a methods nerd… most people won’t know that I started my career in the world of science working in a laboratory. Maybe that's influenced me more then I’ve realised? Repeating experiments to notice a result, an eye for variation, and a bent towards empiricism may have come from my job in Quality Control at Australian company Ego Pharmaceuticals for five years.

I’ve always been pretty good at logic and reasoning and those tendencies are a good match for someone entering the world of technology. I understood programming languages and could get along as a developer, although I bowed out of more complex technology roles fairly early in my tech career, favouring process, management and leadership. 

I think this logic and reasoning preference is why I gravitated towards Agile approaches to software development. Or perhaps it was because, like the majority of technologists in the 90’s, I had experienced some terrible software project fails. I was seeking a better way - it seemed like the obvious thing to do. 

In my beginning, before the term Agile had been coined, us idealists were reading books, going to meet-ups and trying stuff. I was working in London at the time and fell in with a crowd that were exploring DSDM and XP. On reflection I look back at this time as if I was witnessing a kind of socialist movement. We believed inequality and suffering on software projects were the results of really terrible ideas of how to get large complex pieces of software to integrate and work together.  Perhaps calling it inequality and suffering is a little extreme, what was certain however was the god awful quality of the stuff we were building, the dysfunction in the team members who disagreed about how we should go about building it, and the waste of time and energy spent on things other than getting code written. 

For example, I worked on a big project as a programmer in 1997, our code was large and separate swathes of functionality, all written on different computers, loosely adhering to poorly communicated ‘rules’ of architecture and naming convention by a frequently churning team, more than half of who were novices at programming. Version control was a concept poorly supported by primitive programs, checking in… happened, we were told it had to happen and we had to check in our code… when it was finished. Sometimes this was once a fortnight, sometimes once a week, sometimes, not at all. 

“I don’t know if you’ve ever seen a 200 person call centre all rise from their desks as one unit and throw their hands up in the air in wonderment at their frozen, defunct, not working, and dead CRM software on their screens.”

When we entered the so called ‘Integration’ phase, very little of our software worked nicely together, and the bugs started piling up. When the project manager told us we were now in the 'System Test’ phase we frantically fixed the top 10% of really frightening bugs in the time allocated - while a few of our most experienced programmers finished writing the bulk of the actual system. We were asked to come in to support the ‘UAT phase' one Sunday because the call centre who were going to use the software were going live the following day. We had a dependency on an external system that fired off a customer validation check each time customer details were entered. It perpetually crashed in our test versions of systems. Turns out it would perpetually crash in production also. It seemed like the only thing it could be relied upon to do, was crash. 

I don’t know if you’ve ever seen a 200 person call centre all rise from their desks as one unit and throw their hands up in the air in wonderment at their frozen, defunct, not working, and dead CRM software on their screens. It’s quite a sight, something that stays with you forever, and makes you think “I do not want to build software like this ever again”.  We actually had to teach the call centre staff how to kill off programs using those three little words 'Ctrl-Alt-Delete’. It passed the time while they were not able to do their jobs.

The funny thing is we all kind of knew it wasn’t going to work. We all had good intentions but we were swept up with the excitement of the long and dreadful project coming to any kind of conclusion. Our Project Manager was a good person, and this project had burned through several Project Managers and team members already. We wanted to support her, and of the three axis: Time, Scope and Cost, she had managed to stick to Time for the client except for a few small extensions. In the 90’s that was a statistic that was in good company: over 85% of software projects were failing back then. 

We knew on the Sunday we supported UAT that the software was in bad shape, but the client went ahead with the go-live anyway. I expect someone was trying to prove something to someone else, we were thankfully shielded from this kind of management level politics. I had a brief Emperor's New Clothes moment where I was imploring the team that we had to do something about not going live the next day. I was told to shut up, take my Sunday overtime payment and watch it all unfold. So I did. We all did. Hence the 200 people simultaneously standing up visual, and presumably the 200 apologies that closely followed the entire and complete system failure. 

“I’m sorry there appears to be something wrong with our system”

We had known all along the thing was a dud. Every day we worked too late against unrealistic deadlines to achieve a build against requirements that were partially documented and incorrect. It was not a team following a process, it was a mad scramble masquerading as a profession. Every evening, fuelled by expensed drinks and food, we the workers would get together and rant endlessly about everything wrong with the project, the software, and the managers. It was exhausting. 

Many months later the remnants of our team were co-opted onto another project at this client with a different Project Manger who was keen to try new things.

Enter stage left DSDM! DSDM, the 'Dynamic Systems Development Method' arrived in the form of two chirpy consultants looking for pet projects to try it on. The consultants told us they had a DSDM two day course we could attend. Our PM said “Great! You have one afternoon, go for it”. And we did. 

Half of our cobbled team digested DSDM at lightning speed that afternoon and started applying it straight away. DSDM was logical. It was business focussed. It had ways to trade off scope and prioritise things in priority order. It had time boxes where you did stuff that didn’t run over. It had principles about collaborating. It was HEAVEN! 

We became the poster kids for DSDM. Our project was a success, we delivered in phases, our users used our working software and then we IMPROVED it and kept giving them more. We hit our Time Scope and Cost deadlines - almost unheard of in those days. I studied to become a DSDM certified professional and everyone in the entire software community all lived happily ever after…

Except we didn’t…

 ***

See Part 2 - A tale of Two Cities for the next installment of An Agilist’s Tale.