A productive post-mortem can’t change the past, but it can almost always alter the future.
In the modern business world, retrospectives have long been an industry standard. Yet some organizations resist the post-mortem for a variety of reasons, from concerns about negativity to worries that extended reflection is a waste of time and resources. However, according to the Harvard Business School’s Francesca Gino and three colleagues, taking time to reflect on work significantly improves job performance in the long run.
Gino and her research team conducted three experiments that focused on the dual process theory of thought, which posits that humans learn in two distinct ways: fast-paced learning by doing and slow-paced learning by thinking and reflection. In all three experiments, those who were told to reflect and document what they learned markedly improved their job performance over those who continued working without reflection. In the final experiment, individuals who shared their reflections with others — and explained their thoughts in person — improved their job performances even more.
A reflective post-mortem can redirect team performance toward far more fruitful projects in the future. To ensure meaningful improvements, it is imperative to gather the team, assess what went right or wrong on both a macro and micro level, and adapt accordingly. What’s more, listening to the suggestions of everyone involved — a key factor in the scrum model — can lead to effective solutions that resonate far in the future.
We sat down with 17 tech companies to discuss how they make the most of a project post-mortem. We learned that comfortable formats, clear communication, deliverable action points and positive accountability can change both the tone and the content of a post-mortem, creating far better results.
Project Post Mortem Tips
- Create tickets for follow-up items right away
- Create a few, impactful action items
- Rotate post mortem leaders
- Provide descriptions and timelines before meeting
- Utilize the time to express appreciation for your team's work
Nurx
Spencer Spezzano
ENGINEERING MANAGER
At Nurx, a platform for accessible healthcare, Engineering Manager Spencer Spezzano knows that stakeholder alignment and cross-team planning before and during a post-mortem can help teams hit their delivery dates with more success and less churn. Furthermore, by democratizing the prioritization of action items, the collaborative environment creates more teamwork and better problem-solving than before.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you're making the most of that time?
Nurx conducts hour-long post-mortems after every major project, whether that’s launching a new service line or adding a major new feature. These meetings are attended by all stakeholders to ensure alignment on the learnings and outcomes.
But rather than waiting until the end of a project to start evaluating what worked and didn’t, we provide space to collect feedback in real-time from the start of a project. We share a Trello board with three sections: “working well,” “can be improved” and “kudos.” Each section orients the individual into the right mental space to evaluate things as they go: “Working well” documents things people think are helping our process, “can be improved” collects areas of opportunity, and “kudos” provides a space to call out wins for our teammates. Managers keep an eye on the Trello board to catch issues that should be addressed immediately, rather than at the post-mortem.
During the post-mortem we go through each section and individuals read out their feedback, and each team member votes on which items in the “can be improved” section should be top priorities. With these items identified the team works together to come up with action plans to improve our process.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
“Weeks of coding can save you hours of planning,” meaning that being intentional and strategic upfront saves time and prevents inefficiency and frustration. One of the biggest determinants of a successful project is hitting the delivery date, but with large, multi-department projects that can be a challenge. With a new service line we launched in the past, we dove in as a company before thoroughly investigating what was needed from each team, what the dependencies were, and how long everything would realistically take. As a result, we had to push back the launch date.
For subsequent service line launches, we’ve worked to align all stakeholder departments around a high-integrity commit date. We make sure everyone is aware of executables tracked on a project plan spreadsheet, which allows us to quickly identify dependencies and holds everyone accountable. We make dedicated time upfront for all teams to do thorough investigations and work through unknowns to de-risk the delivery date.
This method of upfront cross-team planning was born out of a post-mortem and it has helped transform our process and so we can consistently hit delivery dates with less churn.
It is important that we foster a collaborative environment where people’s voices are heard and recognized.”
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
A key goal of post-mortems is iterating on our process, but since there is only so much time we need to ID iterations that have the biggest impact. In the past, we would spend the post-mortems looking at each area of opportunity on the Trello board to figure out an action plan, but this led to overly long meetings, a lack of focus and ultimately limited improvements on process iteration.
As an alternative, we started a process where we vote on which “can be improved” items to prioritize, and each team member is given a set number of votes. With the introduction of voting, we’ve been able to focus the team and our efforts on high-impact areas. The voting drives buy-in, as everyone gets aligned that while we might have 20 things to improve, we voted to prioritize these top five. We’ve seen our process improve faster which has led to decreased churn and more confidence in hitting our dates.
Another benefit of voting is democratizing the prioritization effort, which helps us move to a bottom-up approach versus a top-down one. It is important that we foster a collaborative environment where people’s voices are heard and recognized.
Wish
Andrew Potapov
SENIOR ENGINEERING MANAGER
At Wish, a mobile e-commerce platform, Senior Engineering Manager Andrew Potapov believes that the most important aspects of a successful post-mortem are the completion of preliminary work and finding the right audience to unlock the best and most innovative solutions. When these findings are documented and then shared in a published internal report, the results can be far ranging.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you're making the most of that time?
The most important thing to ensure the post-mortem meeting runs smoothly is to have a post-mortem document prepared and reviewed by the team prior to the meeting. Sometimes, if folks haven’t had a chance to review the document, we will take the first five minutes of the meeting to read the document and make sure everyone is on the same page. The document should describe the timeline of the incident, the impact, the steps taken to resolve the issue as well as the safeguards to be added to prevent similar issues in the future.
The second most important thing is to select the right audience for the meeting and keep the discussion focused on the incident and the resolution. We usually try to keep the attendance limited to key stakeholders from engineering, product and operations. During the meeting, it's extremely important to not get into finger-pointing, but to use this as a learning opportunity for us as an organization. This is especially important with more junior team members, who, if they haven’t been through a production incident previously, can be pretty hard on themselves. We strive for blameless post-mortems and really focus on improving our processes as a team and company.
We strive for blameless post-mortems and really focus on improving our processes as a team and company.”
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
The product our team works on is still early in its lifecycle. While our main product has solid deployment processes and pipelines, the product our team is working on is still developing a lot of its processes and automations.
We discovered an issue recently where we packaged and deployed our staging and production mobile app differently. This led to unexpected problems that were not detectable during the testing stage. For example, the way translation strings were being packaged was different in the two environments. At one point, this caused the production app to crash for our international users. What’s worse, it crashed in such a way as to not be detectable by our monitoring system. The only way we found out about the issue was through negative app reviews. You never want your users telling you your app is broken.
This issue taught us a number of things. First, we needed to have consistency in how we package and release our applications across our environments. Second, we needed much better monitoring and alerting so we can be proactive about fixing bugs. Third, we decided to start prompting our users for app reviews. This caused our app rating to go from 3.7 to 4.8.
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
We’ve done a few things to improve our processes around the post-mortems. First, we created a consistent post-mortem document format to make sure all the aspects of the incident, resolution, learnings and follow-up steps were covered. Second, we started to ensure that we create tickets for each follow-up item and prioritize them appropriately in the upcoming sprints. Third, we started to include some of the more junior members on the team in the discussion for training purposes. Finally, we started publishing our findings to the broader engineering team to hopefully help other teams avoid some of the pitfalls.
MasterClass
Win Raguini
DIRECTOR OF ENGINEERING — DEVICES
Win Raguini is director of engineering at MasterClass, a streaming platform for high-profile instruction and classes. He’s learned that if post-mortems are split into specific categories and the data collection is done ahead of time, both team focus and overall velocity will be increased. Moreover, Raguini found that there is an important holistic need for self-care, particularly in the remote environment, that affects both the success of the project and the long-term potential of the post-mortem.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you're making the most of that time?
We have two types of post-mortems: five-whys meetings after emergency incidents and retrospective meetings that happen after every sprint. Five-why’s meetings are targeted meetings to understand the root problem of a particular incident, usually resulting in a process improvement or more research. To make this meeting as effective as possible, it’s important we stick to the five-whys structure as closely as possible when appropriate with the understanding that not all situations will necessarily take this exact form and adjustments should be made to the structure as needed. Retrospectives are made efficient by talking about the most high-priority items first and then recording action items to be sent out to the team immediately to support accountability.
Retrospectives are made efficient by talking about the most high-priority items first and then recording action items...”
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
The biggest revelation over the past three months have come from realizing that interviews, ad-hoc meetings and task randomizations take up more time than expected, which has affected team velocity. Since then, the team has made more of an effort to record these events as part of the sprint to better understand the frequency of these events and better plan the sprint.
Quite honestly, the most valuable thing we identified was the need for self-care during COVID-19. Learning that everyone was struggling with sleep and perma-WFH, getting sunlight and exercise was good. We made a point of asking folks to figure out how to do good stuff for their future self on Friday’s, then followed up on Monday's to see how it went.
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
On one team, we’ve had a different person lead the post-mortem, which has led to a greater feeling of ownership and accountability.
Webflow
James Nimlos
SENIOR SOFTWARE ENGINEER
James Nimlos is the senior software engineer at Webflow, a platform for building websites without coding. He learned that by using a philosophy of shared ownership over critical actions and inactions, the team became more cohesive, responsible and adept at finding solutions.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you're making the most of that time?
Post-mortems have a very simple structure on my team: one person writes the question we need answered, the motivation for the question and sometimes a more concrete description of what they expect to be done with the answer. We have a weekly meeting where we discuss multiple questions with all ranges of impact on how to improve outcomes in the future and how we work together as a team. We value this time because it explicitly maps out workflows and how we should react to situations. It also provides time to share knowledge as well as align assumptions about our work.
We agree that this meeting is important and helps us move confidently when we are working individually. We ensure value in this time by requiring an outcome on each point, which is usually either documentation or code to write so that we don’t repeat history. Each meeting, someone volunteers to facilitate and is responsible for keeping us on topic and ideally within the allotted time. Finally, as an all-remote team, we give ourselves some time to chat at the beginning of each meeting since we don't always have group conversations like this due to proximity.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
The best revelations come when we recognize a more systemic root cause than our initial assessment. Recently, we solved an issue for a new team member by writing a checklist for common debugging steps with users. However, we expanded our session to examining the meta of how we support users and what we could do better. We immediately started brainstorming around our support workflow and where problems stem from — this led to a three-part strategy and the long-term effect is a better experience for users and less time chasing down bugs for us! However, the largest lesson was remembering to search for more systemic solutions to problems because they have larger residual benefits compared to over-scoped fixes.
The best revelations come when we recognize a more systemic root cause than our initial assessment.”
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
I’ve learned to make explicit statements clarifying what it means to have a “blameless” analysis to ensure we all start with the same assumptions. Blameless doesn’t mean we pretend there weren’t individuals behind the actions, but instead we acknowledge we all share fault for not preventing the incident. Instead of “blameless” we all hold blame because doing nothing is a choice as well.
Choosing to not review code that ended up being the cause of a major incident is also an action, but because there isn’t a log of that decision, it’s frequently overlooked. I’m not saying everyone should review everything, that’s not scalable, but having shared ownership of code means you all share the blame when problems arise. Ultimately it’s not about where blame lies, but what you can do about it. Acknowledging that everyone has some responsibility means you can quickly move beyond reactionary arguments and begin analyzing the problem and what different solutions you can implement to prevent it next time.
Justworks
Justin McKee
ENGINEERING MANAGER
McKee doesn’t believe that a project ever actually dies. Rather, it continues carrying important lessons on how to better plan for the future. And he and his team act accordingly.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you’re making the most of that time?
At Justworks, we refer to these meetings as Post-Project Kaizens (PPK). The term “kaizen” refers to a system of continuous improvement and aligns with our goals of these after-action meetings: to identify key learnings and opportunities for improvement. We chose to avoid the term “post-mortem” because it connotes a sense of finality. In our experience, a project is rarely ever “dead” – it lives on in lessons learned and future maintenance.
Our Post-Project Kaizen template includes: description, timeline/key dates, accomplishments, problem areas, lessons learned, opportunities for improvement, and post-project action items.
A great way to start the PPK is to celebrate the team’s accomplishments through the project. Then, we spend most of our time discussing lessons learned and opportunities for improvement that were surfaced in the previous two sections. Any action items are assigned a responsible party and a due date.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
When the U.S. government passed COVID-19 legislation providing assistance to businesses, Justworks immediately began supporting customers in navigating the complex legal requirements and transferring emergency funds.
The organization mobilized to work together to provide an accurate and swift turnaround. This entailed working across functions and fundamentally rethinking boundaries of responsibility. Our PPK highlighted teams’ tremendous accomplishments, but it also identified challenges raised by this new way of working.
We faced moments of uncertainty around ownership and expectations, inefficient resourcing and avoidable bugs. Rather than relying on familiar team-specific ways of working, we could have coordinated better with centralized knowledge repositories and designated point people.
These learnings were invaluable when the organization needed to quickly implement subsequent pieces of legislation. Due to our PPK, we can still work together quickly without uncertainty or frustration. We now create a separate mission-specific project board rather than distributing tickets across team boards.
In our experience, a project is rarely ever dead – it lives on in lessons learned and future maintenance.”
What’s one thing you’ve done to improve your post-mortems over time?
One of the best-received changes we made was to publish our description and timelines in advance of the meeting, rather than attempting to reconstruct relevant events in real time. As a result, we all start the PPK with a shared understanding of the project details, and we have more time to discuss the juicy takeaways rather than sweat details.
Bluecore
Colin Allen
PROGRAM MANAGER
At Bluecore, Allen prepares his team to ask the “five whys” at each post-mortem. The method derives from a technique used in retrospectives to discover the root cause of an issue. Asking “why” five times per problem informs the basis of subsequent inquiries.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you’re making the most of that time?
To get the most of a post-mortem, good preparation is key. Before the meeting, document the incident timeline to capture the known facts (who, what, when and where). Distribute this to the participants before the meeting, so that the team steps into the meeting ready to ask the “five whys.”
In the meeting, it is key to have a blameless environment to ensure a constructive conversation. Those with the most context should feel safe to contribute to the conversation without fear of repercussion. Dig into systemic issues and focus on actionable changes as outputs.
Time management is also key. Ensure that there are 10 minutes at the end of the meeting to review the action items generated so that they can be assigned and scheduled for completion. Without follow up, little value is gained from any retrospective.
The most interesting revelations from a post-mortem come from understanding how culture manifests itself in code.’’
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
I find the most interesting revelations from a post-mortem come from understanding how culture manifests itself in code. Past decisions, team structure and product prioritization can help explain a feature’s complex systems working in unexpected or unplanned ways.
The challenge with this type of fragility is that it is very hard to predict future trends. Through post-mortems, the team has learned that the only sustainable approach to maintaining a complex system is by shifting their efforts from reactive to proactive.
The team is taking two different tacks to holistically reduce the flakiness in the system. One is to commit to fixing the system after the immediate issue is remediated. Secondly, the team is investing in proactive programmatic alerts, which are preferable to bugs, a lagging indicator.
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
Traceability has been key. Over time, I have gotten better at ensuring that action items are written up into Jira tickets and that they are completed in a reasonable timeframe. Every team is busy, so accountability is key to ensure that technical debt is addressed.
Lever
Melissa Skevington
ENGINEERING MANAGER
At Lever, project pain points are typically discussed in the department’s bi-weekly engineering roundtable, where developers are outcomes-focused in order to address all questions and preventative measures.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you’re making the most of that time?
A typical post-mortem for our team was made with efficiency in mind and templated and stored for the team to use for consistency in our company-wide Atlassian Confluence page.
Each post-mortem is documented in writing, using the template. To use the template you simply use the one-click button and fill out the following: a summary of the incident, mitigation steps taken, a timeline of activities, a list of action items to be taken as a result of the incident, and links, screenshots and dashboards of the incident.
Once the document is filled out, it’s shared with the full engineering team. Having this standardized template decreases the time we spend on post-mortems and provides transparency across the team to see what the incident was and learnings. That way, if a meeting is needed, everyone has all of the information in advance.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
The most valuable lesson we’ve learned from our post-mortems is not only solving the technical issue at hand and preventing it from happening in the future, but understanding what impact the project has had on our customers. This has helped our team improve communication company-wide to ensure everyone has the latest on what’s working or what’s not.
Taking a moment to recognize the impact we have on our users has helped our team work closely with the support and sales teams to anticipate questions that may arise and set expectations moving forward.
We’ve updated our template to be very clear regarding our customer impact.”
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
One of the biggest changes we’ve done to improve our post-mortems is moving them from our original platform, Github, to Atlassian Confluence. By doing this, we were able to templatize our post-mortem written documents, provide the company with visibility into the post-mortems, and search for documentation. Additionally, it created a lower barrier for edits for engineers to make to the post-mortem document itself. Originally with Github, engineers writing the post-mortem had to submit a specific code for other engineers to approve or make changes to the document.
Now in Confluence, all engineers involved able to pop in and out of the document to make live changes with ease. The results are substantially better as there’s such a low barrier of entry in editing the documents. We love the documentation here. We’ve built a good culture of creating guides on how to handle certain incidents, so newcomers don’t feel stranded and everyone is on the same page.
Amount
Michael Zarnitsa
DIRECTOR OF PRODUCT DEVELOPMENT
Recently, Zarnitsa and his team discovered a direct correlation between the number of service incidents the fintech company was seeing and the age of a system component. Thanks to the post-mortem, they were able to clear up inefficiencies causing issues while maintaining the integrity of their customizable dashboards.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you’re making the most of that time?
At Amount, each service incident recorded requires a post-mortem procedure. Post-mortem involves researching the exact circumstances of the problem and completing a form. The form has specific questions like owner, incident link, priority, status and due date. It also has detailed explanations of what happened for engineers as well as non-technical people, system components and the cause of the issue, as well as a follow-up task list.
After a service incident is handled, product managers and engineering leaders complete a post-mortem form and make a plan of action to prevent such incidents in the future.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
We conduct periodic post-mortem meetings where we summarize service incidents and their causes. We collect statistics and try our best to notice trends and patterns. We recently noticed a direct dependency of service incident count on the age of a system component. The longer the component had been in service, the more updates and changes it accumulated, some of which potentially deviated from the original design, causing unexpected behavior in edge cases.
After this realization, our executive team launched an initiative to refactor some of the oldest system components. The team updated requirements and design principles. We are building the new components in a way that allows quick partner launch. The new subsystems shape out to be more focused and less cross-functional.
We collect statistics and try our best to notice trends and patterns.’’
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
We recently switched tooling for our post-mortem process. We moved from PagerDuty to Confluence to report our post-mortem forms. This gave us a better templating system. All reports are uniform and the questionnaires are pre-filled. Because the company uses Confluence heavily for various internal documentation purposes, it became much easier to reference post-mortem reports. Confluence also integrates better with Jira (our ticketing system), as it comes from the same vendor.
Homebot
Chad Calhoun
DIRECTOR OF ENGINEERING
Chad Calhoun is director of engineering at Homebot, a real estate technology company. He believes that by creating both organized formats and team autonomy, post-mortems have a higher chance of producing successful changes going forward. Moreover, the ability of post-mortems to identify pain points such as missed scope or scope creep has led to his team being more mindful of patterns and risks in future projects, which further increase the chance of a project’s success.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you're making the most of that time?
We find post-mortems to be a great tool to diagnose hidden problems or blockers, identify root causes, improve processes and build team morale. We most commonly use them at the end of a sprint, but we also have a quarterly cadence with the full product and engineering organization.
Our post-mortems involve engineers, a product manager and a designer. We have a cadence of two-week sprints and we schedule our post-mortems at the end of these. Each team has ownership of deciding which tools they use to run the meeting, but in going remote, we have found Miro to be one that works really well. The Miro board is labeled with primary section headers: “Where could we improve;” “Any learnings you want to share;” “What went well / Appreciations;” and “Actions.” Appreciations are where individuals call out who and what they are thankful for amongst each other. Before we wrap, we ensure actions have been defined, have owners and finally, have a vote from the team through a fist of five (a voting convention) on how they feel it went overall. As each team has the autonomy to run their own flavor of the above, we always aim to address the same core values.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
As we traverse different types of problems and navigate the evolution of a team, keeping participants engaged and seeing output with clear, defined actions is always important. It’s important to find ways to keep those actions apparent. Some of our teams will follow up in Slack after the ceremony with each action and their owners. We also review them at the beginning of each post-mortem. We highly value solution-oriented feedback and lean on that to adapt to the always evolving dynamics of the team. Seeing the actions follow through is important to keep trust within the team.
Keeping participants engaged and seeing output with clear, defined actions is always important.”
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
As teams identify pain points in their processes they want to track more actively, we use this forum as a discussion opportunity. We expect these items to evolve over time as the teams evolve. A couple good examples of those have been “missed scope” and “scope creep.” At this stage, we talk through cards from the sprint that were labeled with either issue. The impact with these examples may have been that we weren’t able to identify all the factors involved in implementing the effort leading to “missed scope” or it could have been that there was extra work required beyond the initial defined scope that wasn’t planned.
After clearly identifying those points through conversations with the team, we then talk through how we could approach challenges like this in the future with questions like, “How could we have foreseen this ahead of time?” or, “Are we sure this effort was needed?” This has led to the team being more mindful of patterns or known types of risks that could fall into that bucket of scope creep or missed scope, enabling us to be more informed in the next planning effort.
Beyond Finance
Matt Pyra
LEAD SOFTWARE ENGINEER
Matt Pyra is a lead software engineer at Beyond Finance, a platform for customized financial products. He learned that the resistance many workers have toward post-mortems centers around preconceived notions about tone, negativity and a lack of impact. To turn this resistance around, Pyra focused on post-mortems that were positive, empathetic and packed with clear action items to utilize in the next project.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you're making the most of that time?
Of all the ritual meetings in software development, none is more gruesome than the post-mortem. It’s right there in the name. To encounter one of these meetings is, all too often, to dissect that which just went wrong — to peer into the grim reflection of failure itself! But, for all the dread this meeting can dredge up, it is an invaluable tool for improving team performance.
In terms of format, a productive post-mortem follows the structure of a regular sprint retrospective, with the difference being the topic of discussion. Rather than focusing on the previous sprint, the focus is the previous project. Leveraging a familiar format helps to alleviate some of the stress that can be associated with a post-mortem.
BEYOND FINANCE'S TIPS FOR A PRODUCTIVE POST-MORTEM
- Avoid the Term “Post-Mortem”: The phrase reinforces the negative and puts team members in a defensive posture. Try a more constructive term like “retrospective.”
- Meet Regularly Beforehand: Most sizable projects have multiple phases. More frequent retrospection will improve team performance.
- Facilitate Communication: Your team has its own identity and preferred style of communication. Learn the style of your team and embrace it.
- Produce Meaningful Action Items: Teams enjoy self-regulation, and the “action item” has become the currency of legislation. Favor producing fewer, more impactful action items.
- Encourage Empathy Toward the Failures: Utilizing empathy generously is a great strategy for any relationship. Remind team members that the purpose of this meeting is improvement, not blame.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
A previous post-mortem improved our clarity on roles and expectations. On a recent project, there were assumptions made about who would provide requirements and coordinate cross-team collaboration. That assumption led to failures in these regards. Requirements were missing, and teams were unaware of their responsibilities. Now, those responsibilities are stated more clearly and checks are in place to ensure smoother roll out.
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
The best improvement to our post-mortems is checking in more regularly. Having a post-mortem only at the end of a project means it is clearly too late to course-correct. Instead of meeting just at the end, we now meet at regular intervals. We keep track of issues as they arise and address them before the next meeting.
Pareto Intelligence
Jerome Karaganis
ENGINEERING MANAGER
At Pareto Intelligence, a healthcare solutions company, engineering manager Jerome Karaganis knows that asking the right questions and staying away from self-implication or blame can get employees talking and sharing during a post-mortem meeting. The result: more impactful suggestions and changes that can be used in important projects going forward.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you're making the most of that time?
I see retrospectives as a great way to improve the development process. My first tip is that post-mortems are old school. Why wait until the end of a project to find out that your team has been doing something wrong for a long time? The whole idea of Agile development is to foster continuous improvement, and retrospectives are a key part of that process.
Over the years, I’ve heard from team and project managers who have either been hesitant to add retrospectives or have given them up. Some give them up because they think they take too much valuable time. Believe me, I understand the crunch to get things done. However, if you’re not talking with your team about how time gets wasted every sprint — and more importantly, how to fix those problems — I guarantee you can find more than half an hour of productive time that can be reclaimed.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
Asking “What went wrong?” may return nothing but the sound of crickets. On the other hand, directly asking each team member, “What went well for you this sprint?” is more likely to get an answer. Even better, asking leading questions like, “Oleks, I know that Ming helped you last week, can you elaborate on how that went?” is almost guaranteed to get a solid response. Encourage the team to cheerlead each other and to recognize both individual contributions and group collaborations — this is the start of building a team that likes to work together.
Another thing learned is that retrospectives should never be about blaming. They are not an opportunity for outsiders to talk to the team. Retrospectives are a chance for the team to talk to itself and improve organically. Creating an environment where people feel safe to admit their own mistakes and work together to find solutions is the primary goal. As a leader, one of the best ways to show the team what retrospectives are all about is to model the desired behavior: Admit your own mistakes, detail how you plan to improve and then follow through.
Retrospectives are a chance for the team to talk to itself and improve organically.”
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
Some resist retrospectives because they believe it leads to complaining or venting without any real change. While it’s easy for a retrospective to become an extended chat session, even then, it’s not a waste of time. People like being heard and knowing that someone is paying attention, but the best way to pay attention is to guide the team to come up with actionable solutions to the issues that arise. One of the best outcomes of a good retrospective is finding items that can be turned into Jira tickets and tracked, so that the next retrospective can naturally begin with a discussion of progress on those items.
When I first started running post-mortems and retrospectives, I was apprehensive because I didn’t know how the team would take it. I realized, though, that at a bare minimum, they are a chance to tell my team how much I appreciated all the hard work they put in. Who wouldn’t want to sit through that? And finally, a post-mortem is the idea of reviewing a project with the hope of learning from the past. Retrospectives are the exact same thing — just more frequent — and this advice applies equally to both.
Buildout, Inc.
Alan Spadoni
VICE PRESIDENT OF ENGINEERING
Alan Spadoni is vice president of engineering at Buildout, a platform for commercial real estate. He believes that the key to a quality post-mortem is not only a consistent format to keep everyone on track, but also making sure post-mortems are conducted after all projects, including the most successful ones. With an ongoing attitude that even the best products and features can be updated and improved, the post-mortem becomes more positive and productive than before.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you're making the most of that time?
Each development team hosts retrospective meetings each month to discuss internal team process and sprint-level issues. We also host larger, cross-team retros on project milestones. We use a specific format for consistency to ensure we stay on track. Furthermore, each meeting’s notes are written down and shared with the department in confluence.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
In September 2020, when going through the retro on a project to build a Google and Outlook contact sync that resulted in some customer attrition from our CRM product, we noticed that a number of steps in the project went “just OK,” which compiled into a bad user experience. This was a key revelation in both the root cause of our issues, and as a reminder that “just OK” isn’t what we should strive for.
We consistently schedule retros now, even after an extremely successful project milestone.”
What’s one thing you’ve done to improve your post-mortems over time? What were the results?
The biggest improvements to our process have been threefold. First, we consistently schedule retros now, even after an extremely successful project milestone. Constant iteration and improvement are necessary. Cargo culting also applies to process — you can’t copy something you’ve done at another company and never change it, nor is any process ever truly perfect. Secondly, on the team level, we host a 5-minute mid-week check-in to identify and address any blockers that haven’t been identified in daily scrum due to the inherent myopia of daily meetings. Finally we call these meetings “retrospectives” instead of “post-mortems.” It sounds silly, but the attitude shift toward positive, constructive feedback with the name change was noticeable. The only remaining “post-mortems” we create are for service outages.
ezCater
Stacy Wells
SENIOR SOFTWARE ENGINEER
Stacy Wells is a senior software engineer at ezCater, an online catering marketplace. She learned that by using a familiar format and combining action items with positive accountability, a post-mortem can unlock both individual and team-level improvements going forward.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you're making the most of that time?
Each team at ezCater has the ability to set up their ritual meetings in a way that works best for them. My team’s post-mortems resemble our weekly retrospective meetings. We use this format because it makes everyone comfortable and familiar, and allows us to jump right in rather than figuring out where observations fit best.
We share a doc with three distinct sections: “What went well?”, “What could have gone better?”, and “Anything else on your mind?” As items are added to the document, everyone is encouraged to “plus one” any point that they agree with or want to emphasize. These “plus ones” highlight the points that we should spend the majority of our time on. Focusing on these points allows us to use our time more productively and leads to higher-quality conversation.
Ultimately, our goal is to ensure that our team does better each time we take on a project. We do this by identifying meaningful action items from our discussion and assigning them to an owner. Designating an owner creates accountability and guarantees that the task is prioritized appropriately. These action items drive incremental change to our process and are how we improve as a team.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
One of the most important things I have learned from post-mortems is not to wait until a post-mortem to talk about how things are going. I’m very lucky to work with a team that values collaboration and welcomes ideas from all perspectives. Post-mortems really highlight the team dynamic as we discuss our approach to solving challenging and interesting problems. But the points shared in the post-mortem are unlikely to be something we have not talked about previously. That sets us up for quick iteration and knowing when we need to reprioritize. If we wait to bring these processes up until the post-mortem, we lose the opportunity to make immediately impactful changes.
It is important to note that even though we may have talked about many of the points in the post-mortem prior to the actual event, the higher-level discussion is still incredibly valuable. The difference for post-mortem conversations is that we typically have results so we can have a more reflective conversation versus solving a problem in the moment.
What's one thing you've done to improve your post-mortems over time? What were the results?
A recent post-mortem was set to happen shortly after the winter holidays when all of our team took some time off. In order to capture thoughts before leaving, my manager set up the doc well in advance and invited us to add things as they came to us. In addition to having the doc ready to go, she added weekly Slackbot reminders with a link to the document. This removed friction and supported asynchronous communication.
Setting up our post-mortem this way enabled us to remember valuable observations that may have been forgotten during time off, but it also made it okay to really take that time away from thinking about our project that was wrapping up during that time. Our team came back refreshed and ready to talk about our successes and how we could do even better next time.
AdmitHub
Manan Thakkar
ENGINEERING MANAGER
At AdmitHub, creators of an AI-powered edtech chatbot, Engineering Manager Manan Thakkar knows that productive post-mortems have long-lasting results. By creating accountability boundaries and establishing workable action points, post-mortems can create valuable insights, such as understanding when tech debt is a good or bad proposition.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you're making the most of that time?
At a high level, our project post-mortems involve a small group of people engaging in a focused discussion regarding the processes and practices (or lack thereof) that had a notable impact on our ability to complete a given project based on its requirements. We note any new processes that worked well, what issues significantly slowed us down, and whether any mid-project course corrections could have been preempted. The outcome is generally a set of action items, which can range from individual tasks that can be scheduled into the next project, to changes in process or simply a known concern we need to passively keep an eye out for in future projects.
Product and engineers should both be included, but we don’t let our post-mortem be purely product-driven, and it shouldn’t be a review of the feature or deliverable itself.
Proper boundaries between product and engineering is crucial. While post-mortems should be blameless, we nonetheless try to identify proper owners for the resulting action items and growth opportunities, as engineers shouldn’t be attempting to own or solve product prioritization issues, nor should the product team be responsible for resolving technical missteps.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
Two things we bring up every post-mortem are tech debt and scope creep. One thing we’ve found is that accumulating tech debt isn’t inherently bad, and scope discussions don’t have to be just about reducing scope. We’ve had projects nearly derailed when we’ve attempted to haphazardly reduce scope after discovery. Once a project is underway, there is significant cost to reducing its scope. Our post-mortems have taught us that we need to identify from the start which projects aspects are less necessary and how cutting them might impact the implementation.
We’ve also found the need to better identify when it is or isn’t productive to create tech debt. When part of a project doesn’t have any cross-dependencies and won’t be expanded upon, we might have wasted resources and time on premature optimization. Conversely, we’ve designed systems that needed to be overhauled just a few sprints later because a known future project required the data structures to be built in a more extensible way. By calling these out in post-mortems, we’ve learned to consider the entirety of the upcoming product roadmap to identify which aspects warrant being built out in a way that's conducive to future expansion.
What’s one thing you've done to improve your post-mortems over time? What were the results?
We’ve learned not to treat post-mortems as one-off meetings where participants simply need to be present. Participants should come prepared, and it may even be helpful to keep a running list of unexpected issues during the project itself.
Everyone involved should be given ample time before the post-mortem to gather their thoughts on the project’s implementation, as well as ample opportunity afterwards to address those issues. One of the most demoralizing things that can happen during a post-mortem is realizing that you were bitten by the same issue that was brought up at the end of the previous project, simply because no one tackled the action item that would’ve prevented that issue. Collecting feedback is only useful if it has the possibility of being acted upon. Feedback is most effectively incorporated when it is organized into a set of simple actionable items that can be scheduled into the next project.
Lastly, remember that not every action item from a post-mortem will be a huge success. It’s important to look back at the process changes that came about from previous post-mortems, and cut the ones that didn’t actually succeed in resolving the issue they set out to address.
Ellevation Education
Dan Milstein
CHIEF TECHNOLOGY OFFICER
Dan Milstein is chief technology officer at Ellevation Education, a software platform for ELL educators and students. By working to perfect their post-mortems over time, Milstein has been able to split and add roles, make meaningful structural changes, and conduct preemptive work that facilitates extra time for deeper-level discussions. The result: positive, impactful adjustments that have been incorporated into a variety of projects going forward.
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you’re making the most of that time?
We distinguish between “post-mortems” (for incidents or outages) and “retros” (for projects).
For incident post-mortems, we have people who were close to the issue build up a timeline in a Google doc immediately after the event, which is often a few days before we can meet. We then spend the post-mortem clarifying the customer impact and then digging in and asking a lot of “why” questions and coming up with proposed improvements or remediations.
We try to look at the full cycle — from the triggering event through detection — to develop and deploy a fix. We often ask, “Why did we only find out about this when customers complained?”
We try to have folks from customer-facing teams join us for our post-mortems — that’s been incredibly valuable for building empathy and understanding, as well as identifying valuable fixes around how we communicate with customers during incidents.
For project retros, it’s somewhat similar — we’ll build up a timeline before the meeting itself to create shared understanding. Then we’ll simply ask, “What went well?” and, “What could go better?” and give room for a real conversation to develop.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
One post-mortem drew clear attention to a flaky and unreliable test suite, which engineers were routinely ignoring. Not only did we clean up a bunch of those tests as a remediation, but also those actions started a gradual process of investing in our overall deploy pipeline that is continuing to yield benefits.
From another post-mortem, we overhauled our incident response plan and found much better ways to loop in our customer-facing teams. Furthermore, we clearly designated someone from those teams as a communications lead during incidents.
For a post-mortem for nasty, totally silent failures in our data ingest pipeline, we ended up building visibility and then alerting around the absence of expected data.
From one big project retro we did, we’ve ended up putting really strong, executive-level focus on the overall quarter-to-quarter sequencing for our most important, longest-running projects and on creating explicit options for early off-ramps as we went. That’s moved the conversation from “Is this 9-12 month estimate accurate?” to a much more collaborative, “How can we best create value for our customers, steering into risks all together as we do so?”
What's one thing you've done to improve your post-mortems over time? What were the results?
When we first started doing them, we developed the timeline in the post-mortem room, which was time-consuming and difficult, particularly if people couldn't remember the sequence of events after a few days. We shifted to building the timeline async but immediately after the event in a Google Doc. That’s been very useful and means we can spend more time during the post-mortem discussing deeper reasons for the failure and potential fixes.
We’ve also been working to expand the pool of people who can facilitate post-mortems, which takes some real practice. One nice fix we found was to split up the traditional roles of the “facilitator” (a person who leads the discussion, asking questions as they go) and add in a “scribe” (a person who records the conversation, the whys, the fixes, etc). Our most experienced facilitators can do both, but that was too much to ask of someone new to doing it. So, we’ve actually used that as a form of training and shadowing — someone experienced will serve as scribe, which makes it easy for them to backstop the facilitator, and occasionally offer another question or idea to pursue.
Healthgrades
Mahesh Balasubramanian
VICE PRESIDENT OF ENTERPRISE DATA SOLUTIONS
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you’re making the most of that time?
The Healthgrades health system engineering team conducts a post-mortem at the end of our two-week sprint following the sprint demonstration. The post-mortem is hosted by the technical project managers and includes all engineers and analysts who are part of the scrum team. It does not include any leaders or managers in order to provide an open forum for improvement.
The post-mortem is an important part of our process, second to the planning session, and helps calibrate the working process and improve efficiency and morale from within the core of the team. It promotes transparency and helps increase communication through shared experiences.
Our team’s post-mortems are structured around what we call “five Ls”: liked, learned, lacked, longed for and let’s make it happen. We use retros to review sprint and product goals, document constructive feedback on what went well or didn’t, provide closure on the sprint, decide on key action items or experiment with an approach based on the discussions. The team uses EasyRetro and Confluence to summarize the conversation for future review.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
It might sound cliche but the importance of positivity can never be overlooked. It’s invigorating for the team to take time to celebrate successes and small wins while reviewing areas for improvement. We boost efficiency by streamlining the post-mortem discussion to review the current process and methodology and document action items. This has helped increase participation from team members.
It’s invigorating for the team to take time to celebrate successes and small wins while reviewing areas for improvement.”
What’s one thing you’ve done to improve your post-mortems over time, and what were the results?
Based on discussion with the post-mortem participants, the technical project managers maintain a prioritized retro backlog to help track the progress of all tangible improvements the team members have agreed to achieve in the upcoming sprints. This feedback loop helps the team understand which areas are improving over time and which need greater focus. This also ensures overall team adaptability and prevents the recurrence of the same type of issues. The team’s cadence and shared responsibility have improved drastically based on this action-oriented approach.
Zipwhip
Michael Olshansky
SENIOR ENGINEER
What does a typical post-mortem look like for your team, and how do you structure those meetings to ensure you’re making the most of that time?
At Zipwhip, I look at a project post-mortem like a greatly enhanced version of a sprint retrospective. While engineers and a manager are always present, it’s sometimes prudent to include product owners, project managers, DevOps and platform engineers, managers, engineers and even directors. We ask that everyone comes prepared with a list of items that they think went well and could have been done better, with no subject area off-limits. We then go around the room and give everyone a chance to say what they think, aggregate the feedback and distill it into action items that are then used as process improvements on subsequent projects.
The goal is always the same: to be constructive and reflective, understand each other’s perspectives and learn what we can do better — individually, as a team and as an organization.
What’s one of the most valuable revelations or lessons that has come from a project post-mortem, and how has that helped your team grow?
Regardless of whether this is a new team or a well-established one that has done project post-mortems many times, the common themes are always communication, decision-making and process improvements. One valuable lesson we learned is the importance of getting off on the right foot. If requirements are ambiguous, it can result in analysis paralysis, which can be a drag on a team’s velocity and a net negative for their confidence. The solution is to call a meeting immediately, including the stakeholders needed to resolve the ambiguity, as well as a stakeholder who will act as the tie-breaker if a decision can’t be made by the group. This leads to another revelation — sometimes we simply need to disagree and move forward for the good of the project.
The goal is always the same: to be constructive and reflective, understand each other’s perspectives and learn what we can do better.”
What’s one thing you’ve done to improve your post-mortems over time?
It’s important to identify trends. Before starting a project post-mortem, it can be helpful to look back at ones from previous projects. Are we repeating past mistakes? Are people complaining about the same sort of issues? Addressing them by creating action items or process improvements can be a big plus, allowing the team to self-identify and self-correct rather than fall back into old patterns.