Friday 27 July 2012


Why aren't Lessons Learned working at NASA?


NASA Rocket Successfully Launched January 11 Here's an interesting document - an audit of lesson-learning at NASA conducted this year.

NASA spends 750,000 annually on their lessons learning approach, centred around a tool called LLIS (lessons Learned Information System).

Is this money well spent? Not according to the levels of use and application found by the auditors
"We found that NASA program and project managers rarely consult or contribute to LLIS even though they are directed to by NASA requirements and guidance. In fact, input to LLIS by most Centers has been minimal for several years. Specifically, other than the Jet Propulsion Laboratory (JPL), no NASA Center consistently contributed information to LLIS for the 6-year period from 2005 through 2010. For example, the Glenn Research Center and the Johnson Space Center contributed an average of one lesson per year compared to the nearly 12 per year contributed by JPL ..... Taken together, the lack of consistent input and usage has led to the marginalization of LLIS as a useful tool for project managers" 
So why is this?  The project managers that were surveyed offered a variety of reasons for not using or contributing to LLIS, including 

  • A belief that LLIS is outdated, and is not user friendly
  • A belief that LLIS does not contain information relevant to their project
  • Competing demands on their time in managing their respective projects.  
  • Policy Requirements have been weakened over time. 
  • Inconsistent Policy direction and implementation. 
  • Lack of Monitoring. 
Interesting that three out of these six reasons are directly related to governance. One wonders that, even if a spanking new LLIS were introduced, whether (without better governance) anyone would bother to use it. 

Here are the recommendations from the audit
  • develop and implement a comprehensive strategy for institutional knowledge management that includes collection and dissemination of lessons learned. 
  • determine if LLIS is worth the cost of continued operation, support, and sustainment.  
  • revise the Lessons Learned policy to establish program and project management requirements that align with NASA’s strategic vision for institutional knowledge management and sharing. 
  • revise the policy to align Center and program/project management practices and improve the collection and dissemination of lessons learned Agency-wide by establishing requirements that can be supported by available resources. 
All very sensible, and hopefully this will put lesson-learning back on a much firmer footing.

6 comments:

Christian DE NEEF said...

Interesting but not surprising. My experience with many organizations (including some clients) is that lessons are often learned (documented), but the learnings rarely reused... That is, people document as a project's outcome because it's mandatory, but they fail to reap the benefits when starting new projects! - @cdn

Stephen Duffield said...

Hi Nick, very interesting article when you consider the work of APQC best practice report has NASA JPL down as a case study for leading KM practices... I have recently completed a joint paper on PM lessons learned to be presented at PMOz in Australia (Link attached) where I have referenced some of your work and the NASA Office of Inspector General report. Enjoy the read... Regards, Stephen
http://www.invictaprojects.com.au/pmlessonslearnedblog/?p=608

Nick Milton said...

Hi Christian

At NASA, they have even stopped contributing and documenting, according to the audit

My argument is that a lesson is not learned when it is documented, it is learned when something has changed as a result. See http://www.nickmilton.com/2009/05/what-is-lesson-learned.html

Christian DE NEEF said...

Of course! - @cdn

Barbara Fillip said...

I don't think anyone within NASA was surprised by the report. In defense of NASA (I work there as a contractor in the KM office at Goddard), the audit focused entirely on the Lessons Learned system and didn't cover the broader range of knowledge management efforts. So, if knowledge management is synonymous with a database of lessons learned, then yes, the Lessons Learned system at NASA has not performed well. There is a lot that remains to be done in terms of getting NASA projects and programs to engage in meaningful capture of knowledge, but that needs to be embedded in project and program management processes. The most effective approaches to sharing lessons learned within NASA have been successfully implemented for years: these include knowledge sharing workshops where experienced project managers, engineers, scientists, etc... share their knowledge in informal settings with their peers; and case studies. And we are trying to improve the official lessons learned processes, though in my opinion, that won't happen (at least not in an effective manner) until we have done more foundation work on embedding knowledge management processes within the projects' planned/scheduled activities. In my experience working within NASA, I'd say there is a great deal of knowledge sharing going on and a great deal of learning. Clearly there's room for improvement and we're working on it, but to conclude based on the audit of the lessons learned that NASA doesn't learn is misleading.

If you really want to learn more about NASA's Knowledge Management approaches, I'd recommend going beyond the Lessons Learned audit.

Here are a couple of resources:
APPEL: a NASA Headquarters program led by Ed Hoffman, focused on training and knowledge sharing across NASA.
http://www.nasa.gov/offices/oce/appel/home/index.html

OCKO: Office of the Chief Knowledge Officer (Goddard Space Flight Center)
http://www.nasa.gov/centers/goddard/about/organizations/OCKO/index.html

Nick Milton said...

Thanks Barbara - very useful input, and I entirely agree about the need for embedding.

Blog Archive