The Death Of Scrum
I remember in 2001 when I first heard about Agile software development and Scrum. It was a free lecture by one of the early Agile software practitioners, Allan Shalloway, at the Bellevue University. After a few years working in organizations that practiced Waterfall, I was one of the early Agile champions at Microsoft. The Agile ideology spread like wildfire. A few years later no one argued about the benefits of standups, stickies on the wall, rapid releases, and the rest of Scrum practices. Unfortunately, today the practice has run its course. I would not be putting familiarity with Scrum on your resume. What happened? To understand that we need to go back 20 years when Agile was just starting out.
Agile exploded in popularity for two reasons; one was technological, and another one was emotional. The technological reason is well explored: companies went from shipping software on CDs to using the Internet. While continuous deployment was still its infancy the delivery of new versions of your code was much faster than shipping them on physical media. Of course, it took some time for a complete changeover, for years and years, you could still buy a copy of Office in a box at your local Office Max, but more and more companies focused on delivering their software via the web. It made a lot of rigidity of waterfall irrelevant. You should ship bug fixes faster, you could watch people use your software in real time, you could fix up their data if something went wrong. Agility allowed you to get feedback from the market faster. Companies adopting agile processes made software that was better suited to solve customer problems.
The emotional reason was that Waterfall generally sucked the joy from writing software. Software enthusiasts loved tinkering, prototyping, seeing the results of their ideas come produce visible results on a screen almost instantaneously. Instead of the ability to make computers do magical things engineers were stuck for months writing and debating design on paper, which could be so complex that you didn’t even know that it would really stay the same during the implementation phase. When Agile shrunk the amount of time between ideation and code most of the engineers eagerly participated in it.
What happened next? We got a whole industry of Agile software and certifications to be Scrum masters. The process got more complex because you had myriad fields, Fibonacci points and other junk that had nothing to do with software but was forced onto engineers without any real reasoning as to why everyone was doing it. Agile in 2001 was stickies on the wall, Agile a decade later was an implementation of JIRA that took a certified consultant and then constantly got in the way of doing the work.
I must confess, there was a time when I was one of the people drinking the cool aid. Watching the burndown charts felt like work was being done, never mind the fact that the whole thing could be easily gamed by assigning extra points to the same amount of work. The sprint releases were not actual releases, yet we pushed engineers hard to complete them generating a mild feeling of stress and hurry-up mode no matter if we had to actually hurry.
The Agile movement had a bad side effect, since the idea was that you didn’t need to plan far ahead, and everything can be adjusted in the next sprint a lot of people got Project Management titles that had no idea how the software was built and what was really involved in the delivery of the project. Instead of getting ahead of large roadblocks, they focused more on making sure that all the fields were filled out in JIRA, adding little value to the project and most of the time just doing annoying pestering of people doing real work.
What do we do now? Is the idea to get back to sticky notes? Of course not, if sticky notes worked, we would just continue using them. We also know that putting more metadata about the work doesn’t make the work go any faster. We need to get back to things that are real and connected to real needs of the software teams:
- What are our actual delivery deadlines? These are different for internal automation tool vs an iOS app.
- What are the meaningful milestones? Why do we have them?
- How do we manage dependencies between team effectively?
- How do we manage capacity while accounting for the fact that context switch can be extremely expensive.
- What is the right way to communicate with customers and business stakeholders?
Tools can and should help manage and plan at scale. The answer is not to avoid planning but to plan the right amount given the particulars of a project and technology stack. The good things about Agile like the predictable times to adjust the plans are a good habit. The things like estimating every task in meaningless Fibonacci points needs to go.
New paradigms and project management schemes are emerging as we speak and what has been working in particular situations is probably a subject for another post.
