Backlog Grooming Bugs Me

by | Articles

I wrote this article, a rant really, in April 2014. It was lost in a website shuffle a couple years ago, and someone recently asked if I could put it back. So here it is. But, it’s been slightly renovated from the original version to bring it up to 2017 code.

tl;dr:

Backlog grooming meetings are often the most unproductive and painful meetings in Scrum. If yours are great, then skip this article. It isn’t for you.

Here are the key points:

  • The most successful organizations have balanced product teams that meet routinely, usually weekly, to review and reorganize the backlog, and plan the work they need to do to get ready for the next sprint.
  • The best product owners have routine story discussions with team members to talk through options and details for the work they’re doing. 
  • Hold routine story workshops more frequently, at least once or twice a week.
  • Let participants “opt in” because you won’t have productive discussions with people who don’t want to be there.
  • Keep the discussions small. Smaller discussions are more productive. But always make sure you include someone who understands the functionality, someone who can write the code, and someone who can test it. 
  • Move conversations out of the tool and back to the whiteboard.
  • Bring in whatever you need into the conversation to help you tell the story. But plan on changing and refining what you bring in.
  • Acceptance criteria comes after building shared understanding of what we’ll build and answers the question “what would we check to confirm it’s done?”
  • Photograph the results of your collaboration. Put those photos along with notes in the tool later. Don’t confuse collaborating with documenting.

Most of what I said, and a lot more can be found in User Story Mapping. You should buy a copy. If you already have one, they make great gifts.

Stories and backlog grooming

Adopting new process language bothers me.  I was introduced to the Extreme Programming term “story” in 2000 and it wasn’t until 2005 that I stopped rolling my eyes when I heard the term and could actually begin to use it in sentences myself.

“Backlog grooming” isn’t a new term in the Scrum community, but it’s a term that still drives me up the wall.  Let me explain why.

The backlog grooming meeting as team torture

Years ago, I sat in on a painful meeting with a software development team I was just beginning to coach.  It was important that I learn the way they were working today, and respect the process choices they’d made so far.  It was my goal to observe and understand.  

In this meeting, the whole team gathered in a conference room and projected a screen from a popular Agile management tool onto the wall.  The screen showed the backlog.  One by one the team went through backlog items, opened each item’s document containing a user story, and then read through the details with the Scrum Master facilitating and driving the keyboard.  Team members asked questions about the story.  The product owner did his best to answer questions.  Guided by the Scrum Master, the team tried to write acceptance criteria for each story in the document using a Behavior Driven Development style “given-when-then” format.  

It sounds pretty straightforward.  At worst it sounds a bit boring. But it was worse, much worse.

It was clear by all the restless shifting in chairs and smartphone fiddling that no one wanted to be there.  The team didn’t seem to understand the product and  struggled to ask useful questions.  There were a couple developers that seemed to have an easy job participating and they seemed to delight in asking difficult questions the product owner couldn’t answer, or explaining to the product owner how the product “couldn’t do that.”  The more junior people seemed bewildered.  When they tried to ask questions they were often answered with eye-rolling and patronizing answers that made it clear to them that if they only knew the product or the domain, they’d have never asked in the first place.  The ScrumMaster tried to keep the team focused, but she had most of her attention on the tool trying to find stories and trying to get acceptance criteria keyed in.  She spent a fair bit of time correcting team member’s attempts at forming good acceptance criteria.

I sat at the back of the room watching, feeling the pain of the people in the room as the meeting dragged on for over two hours.  Toward the end, it was clear that people just wanted to escape.  When asked if they understood the story, or if the acceptance criteria made sense, they were quick to nod.  Their “yes’s” felt more like confessions extracted during torture than genuine agreement.   

At the end when people filed out of the room I heard grumbling comments like, “How can we get any work done if we spend all our time in meetings?”  

I asked someone, “How often do you do this?”  

“Every week” he winced.  

“What’s this meeting called?” I asked.

“It’s the backlog grooming meeting.  We have to do it in Scrum.”

It’s from this and many similar experiences with backlog grooming that I’ve come to develop a healthy disdain for both the term and the meeting it now refers to.

What I think Scrum creators really meant

I originally attended a Scrum class taught by Ken Schwaber in 2005.  When I dig back in my cobwebbed memory, I recall Ken saying that the Product Owner needs to keep the backlog groomed.  He explained that the backlog can turn into a mess of items expressed differently, overlapping each other, some no longer needed, and no longer sorted in priority order.  “Every sprint you’ll need to spend time keeping the backlog cleaned up,” is what I recall Ken saying.  Back then, he referred to this cleaning up as “grooming.” This made sense to me.

On the other hand, the meeting I keep seeing that’s called a “backlog grooming” doesn’t make sense.  In fact, any Product Owner who participates routinely in this meeting knows the one thing they can’t do in the backlog grooming meeting is tidy up the backlog.  It had better be pretty well groomed before the grooming meeting or things will go badly.    

As I read the current Scrum Guide,  I notice it doesn’t agree with what I recall hearing from Ken many years ago.  Possibly I’m imagining I heard it.  Today’s “Definitive Guide to Scrum” by Ken Schwaber and Jeff Sutherland does describe grooming as the process of, “Adding detail, estimates, and order to backlog items.” While doing that work makes sense, “grooming” may not be the word I’d choose.   The meeting I commonly see seems the poorest possible process solution for doing the work we need to do to get stories ready for upcoming sprints.

“Grooming” has been deprecated – call it refinement

The first time I said “grooming” in a class I was teaching in the UK I got a several snickers from people in the room. Do a quick google search on the term “grooming” and you’ll see the kinds of things the term commonly refers to.  One common meaning is pretty disturbing, and it’s not “dog grooming.” For that reason most of the Scrum now tries to use the term “backlog refinement” – but sometimes they still slip.

What went wrong

In trying to better understand the work successful product owners do to fill their role, I’ve spent a lot of time interviewing them.  I always ask product owners to “tell me about your worst day on the job.”  At least half the time I hear them describe an awful experience during a sprint planning meeting – that meeting where the whole team gets together to plan their work for the upcoming development cycle.  They describe the team asking them questions they weren’t prepared to answer.  They describe the team complaining that they didn’t have enough information to estimate work, to determine if they could or couldn’t get it done in the next couple weeks.  They describe the team and the company leadership blaming them for not doing enough work to prepare for the upcoming sprint.  Through hearing these stories, I feel a lot of their pain.

When I asked successful Product Owners what they did to feel ready for the upcoming sprint, almost all of them described lots of frequent conversation and collaboration with team members to talk through upcoming work before the Sprint.  Many of them described learning the hard way that on their own they could never come up with the right information to support a story.  They needed the discussion with team members to help them work through the details and to understand what options were feasible and what details were necessary to help team members proceed.

So, here’s where I believe the Scrum community went a little sideways.  This is the pretzel logic I think happened somewhere along the line:

  • Sprint planning is painful when we’re not prepared
  • Collaboration with team members before the sprint helps prepare
  • Product Owners are supposed to keep the backlog groomed
  • The whole team is responsible for the work done in the Sprint
  • [Big idea] Let’s create a new whole team meeting called the “backlog grooming meeting” where the team and product owner collaborate to get ready for the upcoming sprint!

I’m hoping you have trouble with that logic, because I sure do.

What really works

The Product Owner does need to keep the backlog in good order and deliberately budget time every week to look over the backlog pull out things we no longer need, reorganize and rewrite items, and prioritize work so that we know what’s coming up at least in the next sprint and often, a couple sprints out.  

I advocate widening product ownership to a team.  In working with my friend, Marty Cagan, we refer to this team as the “discovery team” or the “core product team.”  It’s their job to discovery what should be built.  A balanced discovery team includes someone who understand the business concerns that motivate building software, users and their concerns and how they’ll use the software, and someone who understands the engineering concerns and can help identify what’s feasible to build.  It’s from this intersection of valuable, usable, and feasible that good products spring forth.  A single Product Owner may lead the team, but it’s still a team.

The most successful organizations have balanced product teams that meet routinely, usually weekly, to review and reorganize the backlog, and plan the work they need to do to get ready for the next sprint.

Product Owners need frequent collaboration with team members to help everyone think through and understand what we can build.

I remember the good old days in software development that when we were faced with a challenge, we rushed to a whiteboard to talk things through.  A few of us would have often loud discussions, drew lots of pictures, argued a bit, shared ideas, and eventually we’d agree on what we could do.  Often we’d end up realizing we might need to prototype something or talk to users to learn something we didn’t understand.  This worked well and it still does.

The best product owners have routine story discussions with team members to talk through options and details for the work they’re doing.  

Good Product Owners keep these discussions small and include the people passionate about the work and helpful in thinking through the details.  They know we’re here to learn, to consider options, and make decisions.   They know that we’re not meeting to communicate requirements, but that together we’ll make decisions about what to build that best solves problems.  Ultimately, participants drive to some agreement on exactly what we’ll build.  They’ll answer the question, “What would we check to confirm it’s done?”  The answer to that question is acceptance criteria or story tests.    

Acceptance criteria comes after building shared understanding of what we’ll build and answers the question “what would we check to confirm it’s done?”

One of the better product Ownership groups I worked with holds Story Workshops at least once a week, sometimes twice.  They post on a visible calendar when the workshops will be and write in the names of the stories they’ll be workshopping in the next session.  The stories to be discussed in the next meeting look a little like “daily specials” at a restaurant.  Team members “opt-in.”  They decide if they want to attend or not.  If a workshop starts and they’re missing someone important, like a tester who’ll be responsible for verifying the work is done right, then the Scrum Master rushes out to see if they can round someone up to participate.  But, for the group I’m thinking of, most of the time the right people do opt-in.  Discussions are animated and fun.  Team members look forward to this opportunity to collaborate.  

Hold routine story workshops more frequently, at least once or twice a week.

Let participants “opt in” because you won’t have productive discussions with people who don’t want to be there.

Keep the discussions small. Smaller discussions are more productive. But always make sure you include someone who understands the functionality, someone who can write the code, and someone who can test it. 

One of groove-killers for backlog grooming meetings is projecting your backlog management tool on the wall and asking one person to type. So, stop it. Move your conversations back to what we know works, the whiteboard. Stand up. Draw pictures. Bring in pictures to post on the wall. Make sure they’re on paper so team members can scribble changes and notes on them – because if it’s going well, they will.

Move conversations out of the tool and back to the whiteboard.

Whatever you bring into a story discussion is legal. Yes this includes formal specs, UML, use cases, workflow models, wireframes, or pixel-perfect mockups. They’re legal because you’ll use story telling and back and forth conversation to make sure everyone understands. And, because you know that everything is open to change, you’l plan on making changes and refinements during the discussion. It’s a workshop, so plan on working. And, it’s called refinement because we plan on refining things.

Bring in whatever you need into the conversation to help you tell the story. But plan on changing and refining what you bring in.

When you’re done with your workshop, someone will need to get all this stuff into the tool. Make it easy by photographing what you’ve done and putting that photo in the tool. You may need to transcribe a few notes from your conversation too. But, make documenting someone’s chore after the fact. Don’t confuse effective collaboration with documenting. The documents should be evidence that you did collaborate. They don’t replace collaboration.

Photograph the results of your collaboration. Put those photos along with notes in the tool later. Don’t confuse collaborating with documenting.

Later, at Sprint Planning, team members are familiar with the stories.  They helped decide on them.  They helped author the acceptance criteria, and are quick to help explain details to other team members who didn’t help workshop the story.  And, when “that guy” – you know the one – that guy who delights in poking holes in everything pipes in, team members fight back.  I’ve heard team members ask, “Hey – where were you when we workshopped this over the last couple weeks?!  It would have been good to hear this then.  Right now you’re just slowing us down.”

Use product team planning and story workshops to prepare

The combination of product team planning and story workshopping works.  

Yes, keep the backlog groomed and in good order, but don’t make the whole team do it.  If you’re the product owner, do a bit of it on your own, and lots of it with your closest collaborators – a small balanced product team.  

Use product team planning meetings to discuss how the release is progressing and to choose stories for upcoming Sprints.  Use the time together to identify the work you’ll need to do to get stories ready for the upcoming Sprint.

Use routine workshops with team members to work through the story details.  Keep these workshops small and productive.  Don’t exclude those who want to be there. However, don’t make people attend who don’t have a desire to contribute.  

Pay attention to what happens during sprint planning, everyday during the sprint, and later at a sprint review.  You’ll get a feel for the kinds of things you need to be discussing during workshops to help things go smoother.

 

Most of what I said, and an lot more can be found in User Story Mapping. You should buy a copy. If you already have one, they make great gifts.

 

 

Opportunity Canvas

by | Articles

An Opportunity Canvas is a simple one-pager I like to use to facilitate a discussion about a new feature or capability. It’s put together in the spirit of the Business Model Canvas or Lean Canvas which are fabulous tools for discussing new products. But, unlike those canvases, the Opp Canvas assumes you’ve got a product already out there so you won’t need to reconsider the operational or revenue model – which is what those other canvases ask you to deal with.

filling-out-an-opp-canvas  2-opportunity_canvas

Because we’ve got so many good ideas for features we could add, we should consider the specific user’s problems we’re solving, and how we think they would use your solution to solve them. And, since it’s not only about users, you should discuss how users being challenged hurts your business, and how helping them helps your business. This all helps to figure out how we’ll measure success. The Opp Canvas helps drive that discussion.

If you just want to give the thing a try, go ahead and download a pdf here.

If you want to tailor it to fit your needs, here’s the original document. It’s laid out in PowerPoint, so it’s nothing fancy.

Stay tuned

I’ll write a more full featured article on this later. For now, I needed to get the files in a more obvious place to download.

 

 

More Like This

Story Mapping Example Problems

by | Articles, Quick References

I have a fundamental problem when teaching Story Mapping in a class or workshop. That is, participants never have enough time to build a complete enough backlog to explore all the different release and development strategies that having a full backlog gives you. That sucks. But, I have a simple solution.

I’ve borrowed an idea from cooking shows. You know how when you watch a cooking show the chef cuts up and prepares some of the ingredients – but not all of them? Someone behind the scenes and ahead of the show has cut up all the ingredients and got them ready so the chef can just pull them out and mix them in. They may have even cooked up whatever the chef will put into the oven so at the right time in the show, the chef can pull something out of the oven ready to taste. That’s what I’ve done with a couple story mapping problems I use in classes. Basically, I’ve chopped, mixed, and cooked up a small software product. It’s nearly done. But to finish it, you’ll need to add a little of your own effort.

Lists – mobile planning and to-do lists

The lists problem is yet another mobile app to create simple to-do lists, plans, wish lists – or whatever your target users might wish to create. Who your target user is, and what’s in scope is up to you.

You can see a directory of all the lists parts here.

In it you’ll find:

  • background on the problem
  • an opportunity canvas all filled out – but with some vague stuff you could improve on
  • 3 possible personas in a  2-column proto-persona format with room for you to fill in your solution ideas
  • a story map made to print out on Avery pre-perforated business cards
  • some really rough UI sketches so you can get some ideas what the app could look like – but they’re pretty uninspired. You could do better!

Here’s what it looks like printed and spread out on my office floor.

lists problem

Film Finder – a place to find info on movies

Yes I know there’s imdb.com and lots of other sites and apps out there… but maybe a trimmer app focused on a particular type of user might find a different niche.

You can find this older problem here.

 

If you find these examples useful to learn to work with story maps, or if you have any comments or questions, please get in touch.

More Like This

Slow Software

by | Articles

In August 2007 I was lucky enough to be able to edit an issue of Better Software Magazine. I was able to choose my authors: Alistair Cockburn, Jim Highsmith, and Jim Shore. I did this because I was lazy – I wanted great articles without a lot of effort. Choosing these authors guaranteed it.

This “Technically Speaking” column from the editor allowed me a page to rant about what was important to me – namely the tendency for software development to dig too quickly into building software before being clear about the context their software will be used in, the goals it should reach, and the problems it should solve.

Read Slow Software

More Like This

Common Agile Practice Isn’t for Startups

by | Articles

Why common Agile processes like Scrum and Extreme Programming can cause startups to focus on the wrong thing

When working with startups individually or coaching in an accelerator, I often get asked to do a basic Agile tutorial. And, every time I’m asked, I get this sinking feeling. What most people believe Agile development is today isn’t at all what you need if you’re a startup. In fact, trying too hard to get Agile “right” is a good way to slow down the stuff that should really be happening.

Common Agile Practice is for Delivery

When most people think of common Agile practice today, they think of this Scrum/Extreme Programming hybrid – or XcrumP as I like to call it. It goes a little like this.

  1. A product owner builds a backlog of user stories that describes your Minimum Viable Product – the smallest product you think will succeed in the market (stories came from XP, backlogs from Scrum).
  2. Prioritize the backlog highest priority first, (which is hard because we need it all, at least we’re pretty sure we do).
  3. Use a Sprint Planning Session to plan the details for 2 weeks worth of work (but we’re hopelessly optimistic so we often over-commit to 3 weeks or more).
  4. Start a 2-week Sprint (Scrum’s name for a time-boxed development cycle, and they could be as short as one week, or as long as a four).
  5. Every day hold a standup meeting (which actually feels like a status report to the people who are stressed because development isn’t moving fast enough).
  6. Every day move sticky notes around a task board (which looks more like a Kanban board).
  7. Try using some sort of tracking tool to track the backlog and progress (and show burndown charts which for some reason people really hate to create by hand).
  8. Use engineering rigor like test-driven development, automated acceptance tests, and continuous integration (but throw it out when it looks like we’re not going to get everything done in time).
  9. As stories are finished coding, the tester thoroughly tests them, and the product owner reviews them (but usually we hand off a pile of finished stories to the tester the last day of the Sprint and tell them to test it all, then watch the tester melt down).
  10. At the end of 2-weeks, stop and do a Sprint Review where we briefly show what we got done, and reflect on how we could improve (but we spend a lot of time arguing about whether things are really “done-done” because we want a high velocity, and then we make excuses for why we couldn’t get 3 weeks of work done in 2 weeks).
  11. Repeat steps 2-10 till done or out of cash, whichever comes first.

That’s the basic flow. And my snarky comments in parens are what I see usually go wrong. And sadly, they may sound way too familiar to some of you.

I remind people often that:

process isn’t skill

Understanding a process isn’t the same thing as doing it well. And, it takes a while to build the skill and discipline necessary to predictably deliver at high quality. But happily with a common Agile process like Scrum, you’ll get to stop and reflect on where you’re really sucking every couple weeks, and you’ll see that get better over time.

Common Agile practice today uses time-boxed development cycles to put just enough structure around the process of planning work, doing work, and then stopping to evaluate and course correct. This simple execution loop uses velocity, or the rate we’re delivering software as its most visible metric. And that makes sense considering of the key Agile principles (mentioned in the Agile Manifesto) is: “+Working software is the primary measure of progress.”

If you’re going to execute well, you’ll need to build these basic Agile muscles.

If you want a more thorough drawing, look at my Agile & Scrum Quick Reference Guide.

But all that emphasis on delivery velocity leads people to believe that that’s all Agile processes are about. And when you want more stuff faster, which we all do, that can be a pretty seductive draw.

Agile in the Startup – the Anti-pattern

A few guys in a garage programming day and night stopping only to eat pizza and play Call of Duty.

Is that the mental model you have for a startup? This is the sort of startup where the founders believe their idea is fantastic, so fantastic that their biggest problem is getting their big idea built faster so they can release it to much fanfare at a big launch party. A startup thinking this way might adopt an Agile development approach with the hopes of building faster, or to at least get a bit of visibility into how fast the team is actually going.

Imagine a Silicon Valley startup adopting Scrum so they can go faster.  Actually, don’t imagine it.  Watch this short excerpt from the HBO series Silicon Valley – but watch out, because there’s lots of “adult language.”  (If you offend easily – go back to just imagining it.)

Thanks to my friend Aaron for first showing me this. If you’re in a startup, start watching Silicon Valley for a good “scared straight” experience.

As silly as this video makes Scrum look, the video points out a few of the key benefits of an Agile process like Scrum, benefits like making a plan, and keeping work visible. In the video Dinesh says: “You see what he’s doing right? He’s trying to get us to compete so we work faster…” And, in spite of being mad about being manipulated, the programmers code faster so they won’t be outdone by each other.

OK, all that’s great. But for some of you, you already know that building more faster isn’t really your biggest risk. At least not to start with.

Most Startups are About Discovery

These days, thanks to Lean Startup thinking and Customer Development, we now know our biggest risk isn’t getting the software built, but validating that there are customers out there that want our product, and can use it. I’m going to call that stuff product discovery.

Now I suspect if you’re reading this, you’re familiar with Lean Startup’s build-measure-learn loop, but I’ll explain it anyway – from my perspective.

lean startup model

  1. Describe our big product idea (but recognize that it’s a hypothesis, we can’t assume people want it)
  2. Identify the riskiest assumptions hidden in our product idea (assumptions about who our customers are, and their needs, as well as assumptions about how they’d value and use our product)
  3. Separate out the few biggest assumptions (those that would sink your product if they weren’t correct)
  4. Identify an approach to validating those assumptions – your Minimum Viable Product Test (customers you could talk to, and potentially prototypes or examples they could engage with to validate your assumptions)
  5. Build your simple prototype (and write a few notes about how you’ll engage with customers to test your assumptions)
  6. Engage with customers to test your assumptions (pay attention to what you’ve learned as you do)
  7. Make sense of what you’ve learned, and rethink your big product idea
  8. Repeat steps 2-7 until you’re confident you’ve identified the smallest product you really need to build, or until you’re out of cash

There’s something very different about this process loop: the primary measure of progress during discovery isn’t delivery velocity, it’s learning velocity. And sadly, we can’t measure its features or stories completed. And, even worse, we can’t plan two weeks of it in detail because what we learn today can and should change what we do tomorrow.

When you’re doing discovery right, it could look a little like this team at the Nordstrom Innovation Lab.

Notice the Nordstrom Lab still uses time-boxes, 1 week in this case. But, they didn’t start the time-box by predicting how much they’d deliver, but with learning goals in mind. Then they iterated around the build-measure-learn loop as fast as they could.

What the Nordstrom team ended up with wasn’t really “high quality software” – it was more of a jankety prototype that worked well enough so that they could leave it behind and measure the stuff that mattered most: did sales people actually use the application? Did they actually sell more as a consequence? And, did Nordstrom customers like the experience of buying this way better? Honestly, the jury is still out on all those questions. Notice that you won’t see this application running in a Nordstrom store near you. But that’s not failure. It’s learning. In the words of my friend David Hussman:

“The difference between learning and failure is how much money you spend to do it.” — David Hussman @davidhussman

Discovery teams quickly figure out that what takes the longest time isn’t building working software, it’s getting time with customers in order to learn something that really slow you down. The Nordstrom team solves this problem by placing their small discovery team inside a teaming pool of customers that they could learn from. This allowed them to get around the learning loop many times a day. That’s a super-fast learning velocity.

Keep the spirit of Agile, and discount the dogma

If you’re in software development today, you really need to understand Agile development. But, if you’re a startup, be aware that common Agile practice is about predictable delivery and is best suited for the times when you’re confident you’re building the right thing. By using an Agile time-boxed development cycle, you can more predictably build it right.

If you’re biggest risk is building the right thing, focus on building your learning muscles.

Go back to original Agile values and principles. Remember that Scrum and Extreme Programming are good delivery approaches to using Agile thinking, but not the only approaches. As a team frequently ask yourselves these questions:

  1. Are we working together as a team collaboratively and effectively?
  2. Are we making our ideas visible fast so we can learn fast, whether it’s working software, or a simple paper prototype?
  3. Are we learning directly from our real customers, the people that will buy and use our product?
  4. Are we stopping frequently to take stock of what we’ve learned and to re-think our product idea, our plans, and the way we’re working?

Those four questions are what the manifesto really means from a startup’s perspective. Keep those basics in mind, forget all the other agile dogma you hear, and you’ll be using Agile right.

More Like This

Design Studio Recipe

by | Articles

A simple recipe for using collaborative sketching with Agile and Lean UX teams

design studio pitchDesign Studio is a quick collaborative approach to considering lots of possible solution ideas for a new product, a feature, or enhancement.

In a design studio, everyone sketches and shares ideas. By everyone I really mean everyone: UI designers, developers, testers, product managers, business stakeholders, end users, customer service people, and anyone else interested. Everyone feels involved and engaged. Everyone quickly learns that the best ideas aren’t any one person’s but a mashup of everyone’s. And, on top of all this, it’s fun. And, you deserve a little fun.
(more…)