Have You Learned Your Lesson?
BY EVA KAPLAN-LEISERSON
Design and construction firms have long conducted post-project reviews to examine what went well and what might have been improved. These are sometimes called “lessons-learned” initiatives, and the processes can range from informal conversations resulting in circulated documents to a formal collection of vetted lessons saved in computer-based systems. Whatever their form, as the baby boomer generation ages out of the workplace, such efforts may be increasingly important to pass on knowledge and best practices to younger employees.
“If we have a lessons-learned system and actually use it and keep ourselves from making a mistake, it saves money.”
Edd Gibson, P.E.
Lessons-learned processes can provide important benefits, such as creating a culture of continuous improvement. But they can also come with risks, such as opening firms up to liability. Companies need a good strategy to balance the opportunities and potential pitfalls.
According to Timothy Corbett, founder and president of design and construction risk and performance consultancy SmartRisk, a project close-out process that incorporates lessons learned is an essential part of a company’s risk management strategy, and can improve both performance and client satisfaction.
However, he sees such processes only in about a third of companies. Engineers don’t learn about conducting lessons-learned programs in school or during the licensure process, he says. But, “firms end up really taking lessons learned more seriously after they’ve been burned”—through litigation, professional liability insurance increases, overall reputation, or impacted business opportunities. “Pain is quite an influencer,” Corbett notes.
As he explains, a project close-out or lessons-learned process should examine “what worked, what didn’t work, why didn’t some of those things work, what could we have done better, did we make money, and, if not, why?” and ensure that those lessons are incorporated into processes and practices for future projects.
After recent economic downturns led companies to reduce experienced staff, institutional knowledge got lost and technical claims and errors have been climbing, he explains. “Younger staff don’t know what they don’t know.”
The Construction Industry Institute (CII), based at the University of Texas at Austin, put together resources for lessons-learned programs in 2007, following up on a study examining such programs about 10 years previously. The research team conducted surveys and case study assessments, and released papers such as Effective Management Practices and Technologies for Lessons-Learned Programs and Implementation of Lessons-Learned Programs. NSPE member Edd Gibson, P.E., director of the School of Sustainable Engineering and the Built Environment at Arizona State University, was a co-principal investigator.
The team defined “lessons learned” as “knowledge gained from experience, successful or otherwise, for the purpose of improving future performance.” Examples could include a lesson incorporated into a work process, a tip to improve future performance, a solution to a problem, a lesson incorporated into a policy or guideline, or a situation to avoid. Lessons gained from positive experiences were deemed just as important as those generated from mistakes or failures.
According to the CII resources, respondent organizations recognized the inherent value of such programs, particularly in combination with other knowledge management efforts to make the company more of a learning organization. However, they didn’t find that firms successfully quantified programs’ direct value. “If we have a lessons-learned system and actually use it and keep ourselves from making a mistake, it saves money,” says Gibson. “Problem is, how do you determine how much money you saved?”
But other benefits are more clear. According to Implementation of Lessons-Learned Programs, such efforts can:
- ensure institutional memory;
- facilitate continuous improvement;
- increase project efficiency; and
- provide advantages in an increasingly competitive industry.
The CII resources spotlight a myriad of best practices and offer tools, including a jump-start guide to help a company develop, implement, and maintain a lessons-learned program; transactional workflow diagram for collection, analysis, and implementation; and maturity model matrix to enable an organization to self-assess its program.
Gibson highlights important elements of formal strategies, such as a collection system (often computer-based and custom-designed) and subject matter expert vetting of lessons. But he acknowledges that costs can add up for design and maintenance of these systems and notes that companies also employ low-tech solutions, such as simply discussing lessons at the end of a project and writing up an after-action report used for similar projects.
NSPE member Karen Moran, P.E., F.NSPE, senior vice president at multidisciplinary firm Whitman, Requardt & Associates, explains that her company has used these kinds of informal lessons-learned discussions, particularly when scope creep caused a project to go over budget.
Such a debriefing session might include discussion of strategies to use with clients when projects start to go beyond the previously agreed upon scope. According to Moran, debriefing sessions can both help employees become more attuned to such issues in the future and teach the younger staff ways to handle the issues or who to approach when they arise.
Notes taken might be shared internally but not generated into a formal work product, Moran says. And the discussions are often done sporadically, since they cost money by pulling staff off current projects.
While low-tech solutions can work, Gibson explains that they are less effective for large, multinational companies. “If somebody loses the paper document, it’s lost,” he says. “And over time people leave, and you lose that knowledge that you’ve learned in the past.”
In the nuclear power industry, such reviews are more formalized, says Steven Arndt, P.E., F.NSPE, senior technical advisor at the Nuclear Regulatory Commission. He explains that the process has evolved over more than 30 years. The industry incorporates internal peer review programs both before and after all projects and major meetings as well as external peer reviews done by peer and industry organizations, and others. “Because of the high cost of failures in the nuclear business—both from a safety and cost point of view—multiple formal peer reviews are common,” he says. Although this might be too costly for many industries, “it has resulted in very effective process improvement, both within organizations and across the industry.”
Arndt stresses the importance of not looking at lessons learned too narrowly. “If there was a problem with how the specifications were written, for example, the corrective action should not be just to spend more time on that specific specification in the future. It should be to look at the whole specification-development process and what resources (time, people, etc.) are provided for it, to ensure that the specification process is done effectively and efficiently on future projects.”
While the PE says lessons-learned or internal peer review processes are common in “high consequence industries” such as nuclear, he believes the strategies are “useful and appropriate in all engineering work.”
Whether a program is formal or informal, Edd Gibson emphasizes the need for leadership buy-in. “If leadership is committed and puts the resources into it, the right people into it,” he says, “they’ll get the value.”
Despite the benefits, some firms may not use lessons-learned programs due to fear of liability. Gibson says his research team members talked to several organizations that had lessons-learned programs but actually got rid of them because they were concerned about the legal implications. “I think that was pretty shortsighted,” he says. “Knowing those companies, they haven’t improved in the 10 years since.”
Still, it’s important to consider the risks. As Gibson explains, lessons-learned information could become discoverable in a legal case. For example, if the process identifies a practice not to engage in going forward, and employees do it anyway, “the other party can say you haven’t used your own systems.”
Capturing failures from projects that may end up in litigation could also be problematic.
And formal processes can face more liability concerns than informal ones. “If you have a system, [lawyers] can go right through the front door and start looking at things,” Gibson says. But with an informal system, more digging is required.
“[Lawyers] will have as much problem finding the information as the project teams are,” Gibson says. “And that’s probably the reason why they didn’t do the right thing in the first [place], because they couldn’t find the information either!”
NSPE member Skip Lewis, P.E., F.NSPE, chairman at H2L Consulting Engineers and a long-time member of NSPE’s Professional Liability Committee, says he favors lessons-learned reviews for quality improvement but he is “very sensitive to the discoverability of these reviews.” He recommends that companies engage in formal lessons-learned reviews only after consulting corporate counsel.
Lewis’s company conducts informal lunch-and-learn sessions after a project if “hiccups” occur, but he says they’ll remain “very seldom” until there’s legislation to protect the confidentiality of a traceable record.
“If what we’re doing is really assessing a problem area, an area that reflected what some might argue to be negligent design or an error in the plans or specs,” the company would not make it part of a project record, he says. The firm may discuss or even write about a problem, but wouldn’t tie that instance to a specific project.
As Karen Moran says, the engineer’s standard of care isn’t perfection—“nobody’s perfect; nobody’s had a perfect project.” But engineers who know that debriefings are going to be discoverable may avoid them. “You lose the feedback loop and lose the chance to make the next project better,” she says.
Moran isn’t sure how many engineers are aware of the liability issues, however; she wasn’t until she helped draft a position statement on peer review with NSPE’s Committee on Policy and Advocacy (see sidebar). Now she says conducting a lessons-learned debrief would “give me pause to stop and say, ‘How are we going to do this to make sure we’re protecting ourselves as much as possible while still learning lessons and making the next project better?’”
Fortunately, workarounds are available and additional assistance is underway. The resources from the Construction Industry Institute touch on liability concerns and strategies. In addition, guidance for managing the perceived risk of liability for lessons-learned programs is in the works from a group drawn from the American Council of Engineering Companies’ Risk Management Committee and NSPE’s Professional Liability Committee.
The project aims to identify and elaborate on techniques, including:
- Creating lessons learned that use generic descriptions or hypothetical situations;
- Submitting lessons learned to the firm’s lawyer for removal of legal conclusions or admissions against interest;
- Making the lessons-learned program informal; and
- Passing legislation that provides an evidentiary privilege for lessons-learned processes.
According to the project description, “Our hope is that illustrating a variety of potential techniques for managing these risks will empower design firms to proceed more confidently in instituting lessons-learned processes and reaping the enormous benefits that they can provide.”
Karen Erger is vice president and director of practice risk management at insurance brokerage Lockton Companies, as well as an ACEC Risk Management Committee member. She explains that the project aims to flesh out some of the liability advice in the “great guidance” from CII.
She says the work, currently in the planning stages, may culminate in a short paper and perhaps a webinar for ACEC and NSPE.
Erger believes that most firms understand the importance of lessons-learned programs, but are hindered by time, resources, or liability concerns. But young engineers are particularly interested, she says, because lessons-learned programs can help them “not have to pay their tuition in the school of hard knocks themselves” and instead learn from the firm’s past experiences.
Debriefing sessions can help employees become more attuned to scope creep and teach younger staff ways to handle it.
Karen Moran, P.E., F.NSPE
As SmartRisk’s Corbett reiterates, if lessons-learned programs assist in continuous improvement, they can not only reduce potential for claims and insurance costs but also ensure client satisfaction. That will create more business opportunities and, ultimately, improve a company’s bottom line.
Access the CII resources at www.construction-institute.org.