Agile, Improvement, Leadership, Opinion, Sprint Planning

The Perils of Swarming

I’ve been hearing a lot lately about ‘swarming’, especially in regards to getting things over the line. This often seems to be in the context of a sprint end, or some other self-imposed deadline, with a request for developers to swarm on a small set of stories or tasks that have not been completed on time and need to be finished before the current block of work can be closed.

Swarming is often used as a short term solution, bring an engineer or two (or maybe from other disciplines) onto a task to support the existing developer(s) to finish off the things they’re unable to get to. I’ve seen instances where these additional engineers are coming from stories parallel to the work they’re being asked to support, but also from stories completely unrelated to it.

This is obviously different from bring a couple of engineers into a story, breaking up the work and attacking it in parallel from the start.

It pains me that this still happens in 2017.

It’s as if the significant amount of practical and theoretical research into the way people approach work, how people are motivated and invest in the work they’re doing simply has not happened.

But there are obvious drawbacks to this approach that should be clear to everyone involved that swarming as a short term solution to finish a piece of work simply does not work.

Adding manpower to a late software project makes it later.

Brooks Law – Fred Brooks 1975

Swarming Does Not Make Things Go Faster

Support Overhead

Lets say an engineer is working on a story that’s taken longer to complete than their estimates indicated it would. This is OK, and it happens – it’s not a bad thing – because you learn from that experience and apply that to your next set of estimates.

But instead of taking the learning and moving on, we’ll swarm on this story for the last couple of days to get it finished. And to do that, let’s imagine we bring on two engineers, one familiar with the work, one not so (but still from the same team).

helloquence-61189

So we went from 12 engineer-hours (I work under the assumption that engineers should get 6 hours of uninterrupted work done per day – if they’re lucky) to 36.

But actually, we have various ceremonies on the second day (sprint review/retrospection) to the actual engineer hours are 9, meaning we have 27 of productive time.

  • But before they’re able to be production within those 27 hours of work, we have to get them up to speed
  • How far along was the original developer in the work they were doing
  • Which task(s) was the original developer focused on, and how can that work now be done in parallel
  • Are there any tasks that have been prepared but not worked on, and what was that preparation
  • What learnings do the new developers need to understand that they otherwise wouldn’t know
  • How much has been QA’d, how much still needed to be QA’d and what progress has been made on those tasks half way through

You could argue that the developers should know this, they’ve been to the morning meetings and are always up to speed on what the other developer is doing. But this is not the case, those morning meetings are signposts for the work in progress, but they cannot and should not provide all the information above on a daily basis.

So this support is not only taking away time the original developer could have been working on the task, it’s now taking away time the other developers could have been working on other stories and tasks that they no longer can be. So rather than gaining and addition 18 hours of productive work, you’re actually reducing it across the project as a whole.

 

Context Switching is More Than Just Checking Out Twitter

One of the benefits of working on a specific story for a few days, broken down into tasks or not, is that as an engineer you’ll always be thinking about where the work goes next. While you’re completing the current task, you’re subconsciously working on the next set of tasks within that story.

Since you’re in the same domain, this is productive mind work, because you’re not context switching from one domain to another and your brain is subconsciously piecing together the bigger picture allowing you to join those dots without you even knowing about it.

Even when you’re sleeping, your brain will be piecing these together, making you even more productive within that domain (and that’s one of the very strong arguments for 8 hours of sleep and <40 hour weeks to really be productive).

But if an engineer is in the middle of a piece of work, or comes to the end of a story and needs to jump into the middle of a different domain, all that ‘free’ work goes out of the window.

That subconscious planning is no longer useful, the context that has been built around the work being done falls apart. Instead, the engineer needs to start again, building up that context from scratch and given the short space of time that swarming usually occurs over, there’s not enough time to build it up anyway, meaning the work is often being done on a constantly blank canvas.

People are starting to understand that context switching throughout the day (such as browsing Facebook, Twitter or YouTube) while working on something of value drastically reduces the productive of the person doing the work. But this also applies to larger time frames, as context switching between domains of equal value can have the same impact but on a bigger scale.

This is why I’m also a fan of not always taking the highest priority story from a backlog, and instead reviewing the top few and picking the one that best suits the space the engineer is in at that moment in time.

Not All Engineers Are Equal

I’ve covered this before in a few places but agile has a tendency to try to treat all engineers as equal. Even people who have been working on the same products for same length of time will have very different skill sets that don’t always apply the same way in the same situations.

Swarming emphasises the desire to have fungible ‘resources’. When swarming is requested, it often means that who ever is free should start to look at the tasks outstanding and start working.

You can argue that the engineers in question should discuss and apply their skills relevantly (or not at all as the case may be) but don’t underestimate the peer pressure of being asked to swarm for the greater good regardless of the specific skillset present on the team at that time.

And given that you may often have the wrong skill set applied to the wrong solution, you’re simply increasing the overhead of both QA feedback loops and a significant increase of technical debt, all of which have further negative impacts on the delivery of future stories.

How Can You Avoid Swarming

Swarming happens when you have a specific deadline, have tasks which don’t look like they’ll make it over the line, and people on the team available to be reassigned. All three of these should be flagged as warning signs

Question Your Deadlines

Often the deadlines we assign ourselves, especially within time boxed development methodologies like Scrum. While I am a fan of regular and short check points, I am not a fan of the behaviour artificial deadlines can often bring. With bi-weekly deadlines, it’s common to see short cuts being made to hit the delivery date, often explicitly adding technical debt and moving the more work into a future sprint.

Ask yourself, do we really need to take on the disadvantages of swarming, and the possibility of slowing down future development for the sake of delivering this one story by a date we imposed on ourselves? This is one of the main reasons I prefer Kanban over Scrum, while the speed of delivery and iteration is just as fast (if not faster in my experience), you do not have to content with arbitrary lines in the sand as you approach the end of the week.

Of course, there are real deadlines that we have tied to external stakeholders or financial implications. But as you near those deadlines, if you’re in a position where something of technical depth is still needed in a rush, you have a bigger problem with your project and swarming certainly won’t help, other than give you an artificial sense of accomplishment.

Constantly Evaluate Your Progress

How far off completion the tasks are will often determine who or how many people swarm. If it’s a couple of hours off completion, then often swarming won’t occur because the overhead is quite clear on such short time scales.

But if it’s looking to be running days over, then we need to question why the delay was not highlighted sooner, and action taken to remedy it when the overhead vs. the percentage of work remaining was significantly more favourable.

richard-tilney-bassett-183124

If these issues are highlighted sooner, then we have more opportunity to reassign people to different tasks, and spread the work across the team rather than doing it when we’re in full flow. It also allows us to, if we have those deadlines we’ve imposed, to start to message the impact on delivery before it comes as a surprise to anyone.

Ruthless Prioritisation

If you’re willing to pull someone off an existing task or story to complete another, then the question is why were they working on that story in the first place.  If that task is not as high a priority, and can support multiple engineers, then maybe those engineers should have been working on it in the first place.

If you have stories correctly prioritised, and the scope of work correctly identified, you should be starting to work on the highest priority work with as many suitable engineers as can comfortably work on it without stepping on each other’s toes.

If multiple engineers couldn’t work on the task at the start, why can they work on the task when it’s in full swing?

When You Think You Need To Swarm

There’ll come a time when you feel it’s right to swarm on a task, but you need to ask yourself these questions

  • How much delay will we add to the current work if the engineer(s) already on the task need to bring other engineers up to speed
  • Are the engineers we want to move onto the work actually suited to the work, and able to bring explicit value in the short space of time they’ll be working on it
  • What are you avoiding by focusing on a specific task, and will the delay in that work cause the same problems in the future
  • Are the engineers you want to move onto the outstanding work already highly invested in the work they’re doing and what cost will moving them have on their focus and the work they’ll come back to?

What Swarming Isn’t

Swarming is not assigning multiple engineers to a story at the beginning of it’s development, assuming that those engineers can efficiently work on a story in parallel. That is good planning and good prioritisation as long as their effectiveness continues to be high.

I’ve seen swarming used to describe this process in the past, but that’s not what swarming on task is, and shouldn’t be mistaken for the practice described above.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s