Guideline: how to do a learning review when something goes wrong

Summary

When something goes wrong, do a learning review as soon as you can. This can be useful for teams in a work setting or for you as an individual (i.e. school projects).

Examples & Stories

A couple of years ago I did some research on best practices for doing project/incident postmortems. This is a summary of what I learned.

The phrase “Never let a good crisis go to waste” is sometimes attributed to Winston Churchill. It’s a phrase I heard a number of times in Ireland during the financial meltdown of 2008-2011. Ever since, if something goes wrong at home or at work, the quote pops into my head.

In my day job, we sometimes encounter situations that don’t go a well as they could have; a customer issue takes too long to solve or some process or product doesn’t work as well as it should. To learn from the situation, we usually do a blameless postmortem or a learning review. Typically, in work situations, more than one person is involved in the situation and the review.

I have also done a personal learning review when something goes wrong or I receive feedback on something I could have done better. In general, I am trying to focus on building on my strengths but if an issue causes a significant impact, I have found it is worth spending time to do a learning review.

Camille Fournier in her excellent book The Manager’s Path, suggests using the term “learning review” instead of postmortem. In the book Camille explains:

“the “postmortem” process is a critical element of good engineering. In fact, instead of calling the process a postmortem, many have started calling it a “learning review” to indicate that its purpose is not determining cause of death but learning from the incident.”

Etsy: an online marketplace, have published a postmortem facilitation process. Their 20-page process builds on blog post by John Allspaw in 2012. In the process Etsy describe:

“the goal is not to find the cause of an accident. The goal is to seize the opportunity for an organization to learn as much as they can, in a relatively short period of time, about how people normally perceive and perform their work. Because the people involved were doing their normal work on a normal day when the event in question happened.”

We’ll see more details on how Etsy run postmortems below.

Google have documented their postmortem process in the book “Site Reliability Engineering: How Google Runs Production Systems”. Like Etsy they also strive for blameless post mortems:

“For a postmortem to be truly blameless, it must focus on identifying the contributing causes of the incident without indicting any individual or team for bad or inappropriate behavior. A blamelessly written postmortem assumes that everyone involved in an incident had good intentions and did the right thing with the information they had.”

How?

At a high level I recommend doing a learning review in three separate phases:

• Document what happened at the time - an initial timeline. Try to avoid blaming yourself or others during this phase. Focus on the facts and rationale for decisions during the event.
• Decide what to do - find remediation actions to resolve the issue or change processes to reduce the likelihood the issue will happen in the future.
• Implement the remediation actions and follow-up later.

A learning review is more effective when you separate the reflection about what happened (past) from deciding what to do about it (future).

If you’re trying to understand a personal issue, use a notebook (paper or electronic). Ask anyone who might observed something during the issue, to provide input.

When documenting what happened for a group learning review, you can use a shared electronic document. Ask everyone involved to add to the timeline of what happened. Remember, the people involved did what they thought was the right thing at the time.

Try not to ask “why” something happened because that could imply a judgement. Instead, ask “how”. What was the rationale? What data was the decision or action based on? This is a subtle but important distinction.

Camille Fournier makes the following recommendations in the The Manager’s Path:

• Resist the urge to point fingers and blame
• Look at the circumstances around the incident and understand the context of the events
• Be realistic about which takeaways are important and which are worth dropping

Etsy recommend a four step process:

• Gather the initial timeline - the people closest to the event gather objective data and construct a timeline of events.
  • Sometimes the data can be scattered in different places (support cases/tickets, emails, chat messages, phone logs).
  • The timeline doesn’t need to include every detail.
• Hold a debriefing meeting - to gather subjective data (opinions, judgments, assumptions, beliefs).
  • The attendees read the initial timeline before the meeting
  • Up front, the facilitator should try to establish a sense of trust with all participants. “We will be focusing on the HOW of what happened, not the WHY.”
  • Walkthrough the timeline - everyone asks questions. Look for descriptions not explanations.
  • “Pause at intervals during debrief discussions and ask people to raise their hands if they have learned something they didn’t know before.”
  • The facilitator adds additional notes to the timeline. The main focus is the timeline but if remediation ideas come up, note them to discuss later.
• Pivot towards learning
  • This can happen in a separate meeting or a distinct phase of the debriefing meeting.
  • Brainstorm “learning points” which may include remediation actions
  • Choose a small subset of the team to take the brainstorm list of “learning points” and reflect on them.
• Report on the viability of the “learning points” and remediation actions
  • Two or three days later, the sub-team reports back on the viability of the remediation points.

Google’s process has more suggestions on what to do after the “learning points” are decided. They also have ideas on keeping the process working:

• Review each postmortem (or learning review) later
• The Google example postmortem document has the following main sections
  • Authors/Date
  • Status
  • Summary
  • Impact - the effect on customers/revenue
  • Root Causes - “An explanation of the circumstances in which this incident happened. It’s often helpful to use a technique such as the 5 Whys”
  • Trigger
  • Resolution
  • Detection
  • Table of Action Items (each has an owner)
  • Lessons Learned
    • What went well
    • What went wrong
    • Where we got lucky
  • Timeline
• Add an interesting or well-written postmortem to the monthly newsletter.
• Get newly hired engineers to role-play a previous postmortem. Get someone involved in the previous incident to be the incident commander.
• Ask for feedback on postmortem effectiveness.

Why?

Things will always go wrong. We’ll learn more from mistakes or omissions if we can be as open and honest with ourselves as possible. This is easier if you can separate the activity of gathering the facts from deciding what to do.

Why ask “how” instead of “why”? In the Etsy guide they explain:

“Asking “how” enables us to hear other people’s stories. Asking “why” creates a story of our own and tends to elicit only the evidence that supports our story.”

This is somewhat ironic considering I am including this in a section called “Why”.

In finance Michael Batnick writes

"The difference between normal people and the best investors is that the great ones learn and grow from their mistakes, while normal people are set back by them"

Ray Dalio explains why learning is crucial in the book Principles: Life and Work:

"Stretching for big goals puts me in the position of failing and needing to learn and come up with new inventions in order to move forward. I find it exhilarating being caught up in the feedback loop of rapid learning—just as a surfer loves riding a wave, even though it sometimes leads to crashes. Don’t get me wrong, I’m still scared of the crashes and I still find them painful. But I keep that pain in perspective, knowing that I will get through these setbacks and that most of my learning will come from reflecting on them."

The bigger picture

This is an example of a guideline that we use in this blog/publication. Guidelines are also known as:

• best-practices
• principles
• tips
• hacks
• mental models
• directives

In future posts we’ll curate individual guidelines or glean collections of them from articles, books and other content. We’re planning posts on:

• Productivity
• Starting a business
• Managing cash flow in a growing business

If you’d like to write for LearnShareDo, please send an email to (write at learnsharedo dot com). Thanks for reading!

Please follow, share or comment on: twitter, medium or linkedin.

This post was first published on Learn Share Do.

Credits: Photo by Eugenio Mazzone on Unsplash

Disclosure: some links in this post may use Amazon affiliate links.