Lesson Learning in NATO (video)

 Here's a great video from NATO about their lesson learned capability

Source here

The three levels of lesson-learning - don't get stuck at level 1!

Here is a reprised blog post presenting 3 potential levels of maturity for a lesson-learning system. Many or most organisations are stuck at level 1.

Lesson-learning is a common component of KM management systems, and there are three levels of rigour, or levels of maturity, regarding how Lesson-learning can be applied.

• The first is to reactively capture and document lessons for others to search for, find and read ("Lessons for Information");
• The second is to reactively capture lessons at the end of projects and after events, document them, and as a result make changes to company procedures, practices and training ("Lessons to Make Change");
• The third is to proactively hunt lessons from wherever they can be found, and make changes to company procedures and practices so that the lessons are embedded into practice. 

Lesson-learning can be a very powerful way for an organisation to learn, change and adapt, but only if it is approached in a mature way. Level 1, to be honest, will not add much value, and its only when you get to level 2 that Lesson learning really starts to make a difference.

Let's look at those levels in more detail.

Level 1

Level 1  is to reactively capture lessons at the end of projects, and document them so that others can learn. Lessons are stored somewhere, and people need to find and read the lesson in order to access the knowledge. There are sub-levels of maturity in level 1, which include

1a) Ad-hoc documentation of lessons by the project leader, documenting them and storing them in project files with no quality control or validation step. Lessons must therefore be sought by reading project reports, or browsing project file structures

1b)Structured capture of lessons, through scheduled lessons identification meetings such as retrospects, documenting and storing the lessons in project files with no quality control or validation step.

1c) Structured capture of lessons, through sheduled lessons identification meetings such as retrospects, documenting and storing the lessons in a company-wide system such as a lessons database or a wiki. This often includes a validation step.

1d) Structured capture of lessons, through lessons identification meetings such as retrospects, documenting and storing the lessons in a company-wide system with auto-notification, so that people can self-nominate to receive specific lessons.

Level 2

Level 2 is to reactively capture lessons at the end of projects, document them, and as a result make changes to company procedures and practices so that the lessons are embedded into practice, procedure, guidance, design and/or training. Here people do not need to read the lesson to access the knowledge, they just need to follow the updated and improved practice etc. Lesson learning has become integrated into, and drives, continuous improvement. Again, there are sub-levels of maturity in level 2, which include:

2a) Lessons are forwarded (ideally automatically, by a lesson management system) to the relevant expert for information, with the expectation that they will review them and incorporate them into practice, procedure guidance or training;

2b) Lessons include assigned actions for the relevant expert, and are auto-forwarded to the expert for action;

2c) As 2b, with the addition that the actions are tracked and reported.

Level 3  is to proactively hunt lessons from wherever they can be found, and make changes to company procedures and practices so that the lessons are embedded into practice.  There are not enough organisations at level 3 to recognise sub-levels, but there are some ways in which Level 3 can operate

3a) Senior managers can identify priority learning areas for the organisation. Some projects are given learning objectives; objectives for gathering knowledge and lessons on behalf of the organisation. These may be written into project knowledge management plans. 

3b) Learning teams may analyse lessons over a period of months or years to look for the common themes and the underlying trends - the weak signals that operational lessons may mask.

3c) Organisations may deploy specific learning resources (Learning Engineers, Project Historians, etc) into projects or activity, in order to pick up specific learning for the organisation. See this example from the Australian emergency services. 

I have only really come across level 3 in the military and the emergency services.  For example, see this quote from Lieutenant-General Paul Newton

"The key is to ‘hunt’ not ‘gather’ lessons, apply them rigorously—and only when you have made a change have you really learned a lesson. And it applies to everyone … It is Whole Army business". 

However even level 2 is quite rare. Many organisations have not gone beyond the "document and store" stages of Level 1, and often have been disappointed by the outcomes, and by the tendency to re-"learn" lessons over and over again. After all, in Level 1, nothing changes, and if nothing changes, nothing has been learned. 

If you aspire to be a learning organisation, set your sights at levels 2 or 3.

How escalating lessons can help burst knowledge bubbles

Any lesson learning system requires a method for escalating lessons. However escalated lessons may need to break through "knowledge bubbles". How do we reconcile these two issues?

Rainbow bubble by Lena on Flickr

As we have often argued on this blog, the purpose of lesson learning is to drive change and continuous improvement. If nothing changes, nothing has been learned. Each lesson therefore must lead to an action or a request for action, and the action must lead to change. Once the change is made, the lesson has done its job. 

But change can be hard when people do not want to change, and where they protect themselves by knowledge bubbles; hearing only what they want to hear. How can we address this issue?

The issue arises when the team that identifies the lesson does not have the authority to make the change, and has to escalate it to a higher level. Maybe the team has identified a company process that needs to be updated or written. Maybe an equipment redesign is needed. 

These are actions for either the company process owner or the person who holds the contract with the equipment supplier. Neither of these people is in the team, and the team leader doesn’t have authority to tell them what to do. The action has to be escalated to a level where the required authority lies - perhaps with the head of projects, the leader of a particular functional discipline, or a member of the senior level steering group.

Here we run into the issue of the Knowledge Bubble. 

The knowledge bubble is the unwillingness of people to believe new knowledge that contradicts their world-view, for example Stalin's refusal to believe that Hitler was about to invade Russia despite the huge amount of intelligence which proved otherwise. It's not just dictators that face this issue - all of us cherish our own established knowledge and views, and are reluctant to change them.

How then can we ensure that the head of projects, the functional leader and the senior level steering group are not protected by knowledge bubbles, and that they are open to new knowledge supplied through the lessons system? The political version of this is "speaking truth to power", and is not an easy thing. Likewise telling management that change is needed, based on lessons from the troops, can also be difficult at times. 

There are 8 ways in which we can help escalated lessons break through any knowledge bubbles./ 

1. The senior people need to be involved in, and have ownership of, the lesson learning system. Their roles need to be well defined, and they should have been consulted during the design of the system.

2. The lesson learning system needs to be applied rigorously, so that the senior people can expect to see a flow of escalated lessons as part of normal business.

3.   The lesson learning system needs to be quality-controlled and validated, perhaps by the technical experts, so that the senior people know that lessons represent real knowledge rather than opinion. This implies that the documented lessons should show the evidence and the root cause analysis behind the request for change.

4. The lessons must have some measure or description of impact, so the senior people can realise the importance of making the change

5. The lessons must be associated with an author, so the senior people know who to contact for more details

6. The senior people must not be able to reject a lesson without both registering that they are rejecting it, and giving a reason for rejection. 

7. The lesson learning system must be monitored and reported to a senior level, so that the organisation knows whether lessons are being consistently ignored or rejected by certain individuals or groups. 

8. Through their involvement, management must collectively accept accountability for the operation of the lesson learned system. 

The last component is key. It is this accountability, linked with monitoring and reporting, that identifies and maps out the knowledge bubbles, which the most senior management of the organisation (provided they buy into the concept of organisational learning) must then use their influence to dismantle. 

In a comment to an earlier version of this post. Lisandro Gaertner said that in his organisation:

"We work with a two step implementation process. First the technical experts assure the quality of the knowledge and, when it needs further actions to implement, it is passed on to the senior leaders who should tell how, when and by who it will be implemented or why it is not going to create changes on the company. The senior leaders are then accountable for the knowledge they didn't implement. It empowers the employees and create a system to fight the knowledge bubble".

This is a combination of many of the approaches above, with emphasis on the assurance role of the experts, and the accountability of management. 

Contact knoco for guidance on building a bubble-proof lesson-learning system

3 ways to estimate the value of lessons learned

Many organisations attempt to assign value to lessons in a lessons management system, and there are three ways you can do this. 

A screen sub-panel from the lessons management hub
showing value assigned to lessons

Assigning value to lesson-learning has three main advantages;

• It reassures the people using the system that there is value in lesson learning. A panel on the front page of your lessons management system, such as the one shown here, reassures people logging into the system that sharing and re-using lessons is a valuable thing to do.
• Lessons can be high-graded according to value, with the most valuable lessons getting highest priority.
• It reassures management that there is value in lesson learning, and makes them think twice before axing the lessons management team and closing down the system.

However there are also counter-arguments;

• Assigning value to lessons can be subjective (see below)
• Value is yet another piece of metadata that needs to be added when documenting a lesson

If you decide to go down the path of assigning value to lessons, you can estimate this value in three ways;

1. You estimate a projected value, where you look at the impact the lesson had on the project where the lesson was identified (measured through lost time, saved time, wasted cost, saved cost etc), and then you forecast  that forward by estimating the frequency of recurrence. Imagine a new way of working was developed which saved a project $100k on a particular activity. Imagine that activity is repeated in other projects a total of 10 times a year. If you document that new way of working as a lesson, and/or embed it into project procedures, then the projected value of that lesson is $1 million per year ($100k times 10, assuming all the future projects re-use the lesson). This of course is only an estimation - maybe the activity is repeated only 5 times, or 20 times. In some cases it might save $50k, in others $200k.

2. You record an actual value, where the lesson is re-used (either directly, or through embedding into process) and you can then measure the improvement that results. This is more accurate than the projected value, but it sometimes can be difficult to isolate the impact of one lesson when many lessons are applied together.  The reporting is often anecdotal - "We started our project by reviewing and adopting all the lessons from the past. The project was delivered in X time/cost which is  saving of Y compared to the benchmark". This approach was used by the Ford Best Practice replication system, where manufacturing-plant contact who received a lesson needed to report what had been done with this knowledge in their local plant and, if they applied the lesson, what value it has added.

3. You record an aggregate value, by looking at the improvement in results over time.  There are several examples on this blog of the aggregate improvement that comes through learning and re-use of knowledge, for example in Trinidad offshore platforms, Oil wells in Oman, Nuclear power stations in Korea, and jumping frogs in California.

If you can, assign value to lessons. This reassures both managers and workers that lesson-learning is a good investment of time and effort.

How to learn like an ant

Social and organisational learning is so easy that even ants can do it, and we can learn from the principles they apply.

Leafcutter ant trail by Jasper Nance on flickr

If you look at an ant trail from the nest to a source of food, it is pretty direct. The food may have been found by a random foraging ant, but pretty soon the whole colony has been alerted to the food, and developed a path that approximates to the quickest route between the food and the nest.

So how do ants collaborate to build the "best route"? Do they design it, or do they evolve it through continuous improvement? And if the latter, exactly how are these continuous improvements made? How do ants LEARN to make a straight path?

1. When ants move, they leave a scent trail behind. It's a fading trail - immediately after depositing, it starts to fade. Initially their trail is pretty random, and may zigzag all over the place. Other ants follow the trail, following the strongest scent, but not 100% faithfully. There's a little bit of variation built in.
2. Because of that variation, ants will sometimes find a shortcut, and cut out one of the zigzags. Wherever they cut the corner and get there faster, their scent is stronger, and becomes the dominant trail. Other ants follow them. If their new way takes longer (a longcut), their scent has faded, and others don't follow. For the ants - faster is better as it is more efficient.
3. So the improvements in the trail are reinforced, and the trail gets progressively better and straighter. Over time, the trail becomes straight.

For ants, the organisational memory lies in the trail itself, embedded in the scent. For organisations, the organisational memory lies in the processes. We can follow the principles of Ant-learning, if we replace the scent-trail with the operational procedures.

1. We record our knowledge in operational procedures, which represent our current best approach to doing a particular task. Initially the procedures may not be particularly effective, and people are allowed to vary from the procedures if they find a better way.
2. The variations are recorded as process improvements or lessons learned and embedded in improved procedures, like the new scent trail along the shortcut.  There needs to be some way to know whether the new variations are better. Perhaps they improve efficiency, or quality, or cost, or customer satisfaction. You need to be able to tell what "better" looks like.
3. Other teams follow the new better procedures (like the ants following the stronger scent) so that these become the dominant procedures - until the next improved variation is found. Over time the procedure becomes optimal. 

You can see this learning process at work in any continuous improvement process, whether it is a drilling crew improving their rig procedures, a lean manufacturing team improving their manufacturing procedures, or an Army improving their doctrine. They all learn like an ant - recording and embedding the improvements until the trail is straight.

6 benefits of reflective team learning

I was reflecting recently after a major lessons capture exercise from a multi-million Euro project of the benefits of this sort of reflective team learning.  It struck me that there are actually 6 areas of benefit from this practice.

Team reflection, by Loren Kernson Flickr

1. Firstly, the team members learn themselves, as individuals. By listening to the reflections and analysis of their colleagues, they gain new learning and new insight which will help them improve their personal repertoire of skills and understanding.
2. The team members are able to share their thoughts and feelings in a non-judgmental environment. They find this emotionally valuable, and it can be a healing process after a traumatic project.
3. The teams identify improvements they can make to their own team process. By reflecting on the past, they can improve their own ways of working together. 
4. The conversations and analysis can be recorded, and turned into valuable learning material for other teams.
5. This documented learning material is not only an asset in its own, right, it also allows other teams to know what lessons have been learned, and who to contact to learn more. Documented lessons are often of most value when they are the catalyst for conversations (and we know that conversations are approximately 14 times more effective than written material in transferring knowledge). 
6. Finally the lessons and experiences of the team can drive improvements in the way the whole organisation operates. In a recent exercise we identified 30 major learning points, and about half of these were associated with requests of recommendations for Headquarters, to embed the lessons into new process or new structures. 

KM and Hansei, where "no problem" becomes a problem

Effective learning within an organisation requires consistent and rigorous self-analysis, in order to pick up learning points and points of improvement. In Japan, this process is known as Hansei.

Hansei, by Jim O'Neil, on Flickr

Although alien to many in the west, Hansei is an important part of the Japanese culture. Han means “change” and Sei means “to review”, so the whole thing means “introspection” or “reflection for the purposes of change”.  This translates into a behaviour , instilled from childhood, of looking for mistakes, admitting responsibility, and implementing change.(When Japanese children do something wrong, for example, they are scolded and are told: hansei shinasai - Do hansei!).

Hansei can become very public, as the footage of the crying Toyota CEO shows. As a response to poor safety performance, the CEO admitted responsibility, apologised, promised change, and wept - behaviour lauded in Japan but deemed strange in the West. In the West we would probably avoid responsibility, deny the mistake as "fake news", and insult our detractors.

Hansei is at the heart of Kaizen - the "learning from experience" approach seen in Japanese industry. It may part of the reason why Japanese companies succeed so well at Kaizen as a core component of Knowledge Management, while other cultures struggle. Where a European company might see lesson-learning as a witch-hunt, for example, a Japanese company would see it as a way to put right the embarrassment of self-acknowledged failure. Where a US company might fear a blame culture, Hansei means that individuals already accept any blame and if they fear anything, they fear the lack of ability to make restitution.

How do we develop Hansei?

In non-Japanese cultures we have not been brought up with Hansei. Seeing our leaders accepting full responsibility for mistakes and sincerely, with emotion, promising change is something exceedingly rare (name me one example!).  However this is a behaviour we would dearly love to promote at work, so that mistakes are not hidden, but lead to learning and change.

So here are six things we can do to begin to develop Hansei behaviours.

1. We can understand the current culture, and recognise the barriers. One of the 10 cultural barriers is defensiveness - an unwillingness to take responsibility and examine your mistakes. Our cultural assessment service allows you to see whether this is a major barrier in your own organisation.

2. We can build reflection into the work process. After Action Review, for example, is a Kaizen activity that can be embedded into the working pattern, to trigger reflection and change on a regular basis.

3. We can adopt no-blame learning processes. The Retrospect is widely recognised as a no-blame lesson-learning process for use at the end of projects or project stages. The open questioning within the Retrospect gives people the opportunity to examine what went wrong, and to suggest how this might be improved.

4. We can ask the team leader to set the tone. If we are concerned about lack of openness in a Retrospect, we can work with the team leader before hand to identify an area where they can openly admit to making a mistake, and explore how to avoid this happening again. When the leader sets the tone, the rest of the team may follow.

5. We can ensure all learnings lead to action. We must make sure that everyone present in a Retrospect or After Action review can see that their admissions, introspections and lessons will lead to action. Lessons will not just rot away in overstuffed databases, but become embedded in changes to process. Knowing that your mistakes can be turned into successes for others can make Retrospects into something like group therapy. This is the positive outcome of Hansei.

6. We can become intolerant of complacency. Another of the 10 cultural elements, complacency is the feeling that "we did alright, there was no problem, we don't need to change anything".  Here is what the Toyota website says about "no problem":

"Even if a task is completed successfully, Toyota recognises the need for a hansei-kai, or reflection meeting; a process that helps to identify failures experienced along the way and create clear plans for future efforts. An inability to identify issues is usually seen as an indication that you did not stretch to meet or exceed expectations, that you were not sufficiently critical or objective in your analysis, or that you lack modesty and humility. Within the Hansei process, no problem is itself a problem".

This type of thinking - where "no problem" is seen as symptom of a lack of introspection and a lack of analysis, and something to challenge rather than to feel smug about - may be what separates the  true learning cultures from the also-rans.

Why a no-blame culture needs no-blame processes

We hear a lot about the importance of a "no-blame culture" in Lesson-learning, but a no-blame culture won't work unless you have no-blame processes as well. 

Image from wikimedia commons

Learning lessons in an organisation requires a culture of openness, so that people are willing to explore honestly and openly what happened, and what might be learned.  This is particularly important when learning from projects or incidents where things have gone wrong and where mistakes have been made.  If people feel they are likely to blamed for the mistakes, and that their career or reputation might suffer as a result, they are unlikely to discuss the issue openly.

When we discuss the topic of lesson-learning with clients, we often hear the concern that lesson-learning meetings could be seen as "witch-hunts" - in other words, a search for someone to blame. To reassure them, we talk them through the lesson-learning process we use; the Retrospect.

The Retrospect is an externally-facilitated team meeting, held after the end of a project or project stage, where the team identifies and analyses learning points through discussion and dialogue, in order to derive lessons and actions for the future. The facilitator leads an inclusive discussion to identify

• What went well and what did not go to plan in the project
• Why the successful elements succeeded, and why the failures and mistakes happened (looking for root cause)
• How future projects can repeat the success and avoid the failure. 

The key questions are therefore What, Why and How; very open questions which allow full exploration of the lessons learned.

Note that there is no Who question. 

We do not really care who was the hero or who was the villain. This is a no-blame process, as well as a no-praise process. We are searching neither for Witches or for Knights; only for the truth.

An open, no-blame culture requires open no-blame processes such as the Retrospect.

Why "Knowledge for action" is better than "knowledge for storage"

Knowledge has to lead to action in order to add value. 

As the blogger Bill Wilson says (in the context of root cause analysis) "Learning without action is mere mental trickery, while action without learning is simply useless physical exercise".  If knowledge management is to deliver more than mere mental trickery and to live up to its promise of adding value, then it must lead to action.

A few years ago we worked with a client who was developing a lesson learning system from projects. The collection of lessons has been going well, but the client had the firm view that lessons should be stored in a library that future projects could review if they wanted. For them, the knowledge would be stored "for future reference".

Of course, few people have time to read through the lessons, and there are now so many lessons that reading through them is becoming more and more daunting. 

We are now helping the client to move to a different philosophy, where lessons are forwarded to the owners of the organisational processes, so they can continue to update the processes, procedures and guidance in the light of the new learning. This is "knowledge for action", and if we assume that people follow the updated guidance, it should result is less "useless physical exercise" and to more efficient ways of working.

This philosophy is that wherever possible, every piece of new knowledge should lead to an action. The action might be;

• Fix a problem,
• Investigate further (especially if the learning is not yet clear),
• Document a new procedure, process or guidance document,
• Update an existing documented procedure, process or guidance document,
• Update a training course or other training or e-learning material,
• Circulate the lesson for others to decide on an action.

Communities of practice, as well, should focus on creating and managing actionable knowledge. Actionable knowledge can be stored on the community wiki, and includes

• Advice and guidance
• Good practices
• Improved practices
• Solutions to problems
• Answers to questions
• New approaches
• Recommendations
• Tips and Tricks

Non-actionable knowledge is

• Interesting articles
• Links to interesting articles
• Musings
• Quotes and aphorisms
• Descriptions of what you are doing (unless you analyse this to bring out actionable learning)
• Descriptions of what you have done (unless you analyse this to bring out actionable learning)
• Large document stores

Communities that circulate non-actionable knowledge, or "knowledge for interest" are classified as Communities of Interest rather than communities of practice, the clue being in the title. 

CoPs deliver more value when they focus on solving the problems of the members than when they circulate "interesting links and ideas". CoPs that operate through a Pull process - where members with problems or issues ask questions and receive recommendations and support from other members - know they are adding value.  Each answered question represents a solved problem; knowledge which the person who asked the question can immediately put into action.

So when you are sharing knowledge in a CoP, ask yourself whether you are sharing "something that others will find interesting" or "something that will help people do their job better" - something actionable.

And when you are designing lesson learning systems, make sure each lesson leads to action, rather than being retained "for interest".

We recommend "knowledge for action" rather than "knowledge for storage" as being a far more effective system.

How to calculate the transmission efficiency of lessons learned

The lesson learning system should be a supply chain for new knowledge. But how do you calculate its effectiveness?

We can look at lesson learning as a supply chain; identifying new pieces of knowledge and supplying them to other knowledge workers so they can improve their work. If it works well, knowledge is gathered, accumulated, synthesised into guidance, and used to inform future operations.

But how well does this supply chain work, and how can you calculate its efficiency?

This is something I tried to measure recently with a client, using a survey.

I divided the lesson supply chain into three steps, shown below

1. Capture and documentation of lessons from project experience;
2. Update of guidance documents based on new lessons;
3. Review of guidance documentation in future projects.

Now this particular client  did not have a clear supply chain for lessons, so many people accessed lessons through search.

I therefore asked survey respondents to estimate in what percentage of cases the following happened, giving them the options to selecting 0%, 20%, 40%, 60%, 80% and 100%. The average estimate from the survey responders is shown below.

• Any given project documents lessons (46%)
• Any given lesson is used to update guidance documents (32%)
• Any given project makes use of guidance documents (43%)
• Any given project seeks for documented lessons (44%)
• Any given search finds useful lessons (36%)

We can then use these figures to estimate the effectiveness of lesson transmission, as shown in the diagram above, and described below.

The effectiveness of transmitting lesson through guidance is 46% (capture) x 32% (update) x 43% (review) = 6%.

The effectiveness of transmitting lesson through search is 46% (capture) x 44% (search) x 36% (find) = 7%.

These are pretty low figures! Even if both routes were independent and there was an overall success rate of 13%, that still means that 87% of project knowledge was never transmitted other than via human memory.

Let's compare that with an organisation that treats lessons seriously. The diagram and numbers below are not from a survey, but from published statistics in another organisation. This organisation does not require users to search for lessons, but has a well-resourced supply chain to embed lessons into procedures and guidance.

• Any given project documents lessons (100% - lesson capture is mandatory part of every project, and supported by dedicated resources)
• Any given lesson is used to update guidance documents (93% - this figure was tracked on a quarterly basis and the "closure rate" for lessons varied between 88% and 95%)
• Any given project makes use of guidance documents (100% - all activity was directed by operational procedures which were reviewed and use to make the action plans)

The overall transmission efficiency is 93%

Please note that these figures only reflect the documentation of lessons and their transmission to future knowledge workers; they don't include the loss of knowledge in the documentation process, or the issues of transmission of understanding from writtenprocedires into the human braid - they only look at the effectiveness of the supply chain of written lessons.

This effectiveness can be measured, through surveys or through lesson tracking, and we can see that is can vary between very ineffective (as low as 6%) or extremely effective (as high as 93%).  If you are applying lessons learned, then aim for high effectiveness!

Why you should never leave your lessons in the project report

Probably the worst place to store project lessons is to leave them in the End of Project Report.

Tombstone created using http://tombgen.appspot.com/

I know this is still the default approach for many project organisations, and 19% organisations who run KM programs still use this approach (according to our KM survey). However this approach was linked to the second-lowest satisfaction score for lesson learning, second only to "we don't store our lessons anywhere". So leaving lessons in project files is only slightly better than not storing them anywhere.

But why is this the case?

It makes sense for the project team to leave the lessons in the report, because lessons are a project output, and surely all project outputs are documented in the report? Once the report is written, the job of the project team is done.

However although the job of the lesson writer may be over, the job of the lesson-seeker is only just starting.

People searching for lessons (whether they are people in future project teams, or subject matter experts maintaining process documentation) will be searching for lessons to help with a specific issue, risk, process or task. They would like to find all the lessons associated with that issue, risk or task in the same place. They almost certainly will not know which projects learned lessons on which topic. They don’t want to have to go back through 20 or 30 project reports, looking through the lessons learned section of each one, just in case there is something of relevance.

Therefore they don't bother.

The lessons remain unread; the project report becomes a tomb in the lessons graveyard. Learned once, and destined to be relearned in future.

Two of the basic principles for effective lesson learning are that you need to 1) capture lessons individually, and 2) store them centrally. As I pointed out in yesterday's blog post, lessons are a knowledge output from a project, and should be stored with the rest of the knowledge, not the rest of the project files. In order to make it easy for the lessons to be found, you need to store the individual lessons under themes or topics in a lessons management system until they make their way into updated practice and procedure, guidance and training.

 If lessons are being learned from many projects, for example from Retrospects and After Action reviews, then these lessons need to be stored somewhere where they can be compared, reconciled, and actions assigned, notified, tracked and managed. They need to be captured in a consistent format, and stored in a central system.

 This is often done in some sort of Lessons Management System, set up either for a major business stream such as Sales or Engineering or for the entire organisation, to store and track lessons from many projects.

An effective lessons management system should be structured according to the needs of the “knowledge user”. People who come to access lessons from the database should be able to find what they are looking for very easily. If they don't find relevant and useful knowledge within a few minutes, they will leave and never come back.

Think about the needs and interests of the knowledge user, think about how the knowledge will be re-used, and think about how you should structure the database to most easily give them access to what they need.  The knowledge seeker is  looking for lessons about procuring steel pipe for example, or for all the lessons to do with safety while working at height, or for all the lessons to do with partnering with a particular national authority; in other words, lessons to do with the particular issue that they face. The lessons therefore need to be grouped into a structure,  index or taxonomy, based on work activity or work process. Think about how supermarkets gather and present produce, and use ideas such as this to stock your knowledge supermarket.

In addition, the lessons management system should be able proactively to route lessons and associated actions to the relevant process owner, in order that the lessons can be embedded in improved process, and can be declared properly "learned" and the learning loop can be closed.

Leaving them in the project report graveyard destroys the learning loop at the first step.

Lesson learning at NASA - video

From the AFAC lesson management forum last week, this video below from David Oberhettinger, NASA's Jet Propulsion Lab, talking about lesson learning in their human and robotic space exploration program


David makes some interesting points, such as

• the need to dedicate individuals for lessons capture (although NASA allows engineers to submit lessons themselves, he says he has never seen this happen in 25 years)
• lessons need verification and quality control - particularly verifying the facts of what happened
• lessons recommendations need to be "infused" into procedures and training to ensure closed-loop learning. The key documents are the JPL design principles and the JPL flight project practices, which are built up over time from the combined and synthesised lessons
• weekly meetings of the lessons learned committee, for the past 35 years

Every lesson is a story

Capture your lessons as stories, that way they will be remembered more easily.

Native American Storytelling. Photo By: Johnny Saldivar

One of the way humans learn is through stories. Stories can Inform, Educate, and/or Entertain (transmit information, knowledge, and entertainment) and some of the best stories do all three.

A post from the Farnham Street blog entitled "The structure of a Story" tells us that a great story has structure, and includes Context, Action and Result; where Context includes Where, Who, What do they Want, and What's getting in the way. As the blog says

"It all hangs on the central Context, Action, Result (CAR) structure. It starts with the Subject, Treasure, and Obstacle, and concludes with the Right lesson and link back to why it’s being told".

When we transfer Knowledge, story is one of the best formats. That is equally true of lesson-learning as it is of the other elements of KM. Yet all too often, lessons fail to meet the test of a good story.

As the Farnham Street blog says, aspirant storytellers usually start with the Action, because that is the exciting part, and omit the context, and as a result the reader of the story or lesson cannot easily identify with nor internalise the learnings.

When we teach lesson-learning to organisations, we say that Every Lesson is a Story, and we recommend that the lesson has at least 6 components

1) Context. Here we explain what was expected to happen, but didn't. This is the "What did we want" step of the story. It is amazing how many project teams or facilitators skip this step. Because "what was expected to happen" is so obvious to them, they expect it to be obvious to the reader, but of course it seldom is. 

2) Outcome. Here we explain what actually happened - the Action step of the story. What went wrong? What actually happened? 

3) Root cause. Here we explain WHY it happened - what the Obstacles were, what was missing that should have been there, or (in a success story) what was done to achieve the result. 

4) Impact. Here we explain or try to quantify the result, either negative or positive. 

5) Lesson. Based on the root cause, what is the learning for future projects? This is the "Right Lesson" part of the story.

6) Action to Embed.What action do we take to embed that lesson into business process, so mistakes are never repeated, and successes always copied.

A lesson with a story-based structure such as this takes about a page to write down or a couple of minutes to tell, compared to typical bullet-point one-liner lessons - told in seconds, forgotten almost as quickly.

However if we treat every lesson as a story, then we realise that it must be presented as a Narrative, with Context, Action and Result, concluding with the Right Lesson.

This is the simple structure that conveys the message.

The four most costly words in KM - "this time it's different"

"This time it's different" can be the four most costly words in project knowledge management, if they are used as a reason not to learn from the past.

Albert Einstein's definition of insanity was "doing the same thing over and over again and expecting different results".

And yet, any analysis of a collection of corporate lessons will show the same mistakes being made time after time. So organisations obviously DO "do the same thing over and over again and expect different results".

Are organisations insane?  Or is there another factor at work?

The factor may well be what the Farnam Street blog calls the "This time it's different" fallacy. I quote from the blog -

“This time is different” could be the 4 most costly words ever spoken. 

It’s not the words that are costly so much as the conclusions they encourage us to draw. We incorrectly think that differences are more valuable than similarities. After all, anyone can see what’s the same but it takes true insight to see what’s different, right? We’re all so busy trying to find differences that we forget to pay attention to what is the same.

Different is exciting and new, same is old hat. People focus on the differences and neglect the similarities. In projects, this becomes the "my project is different" fallacy that I described here. People look at their projects, see the unique situations, find the differences, overlook the similarities to all similar projects on the past, and assume that "this time it will be different".

It never is.

The same old mistakes will creep up on you and bite you in the bottom, as they always do.

Instead of assuming "this project is different", perhaps we should start with the assumption that "this project is just like any project. It involves building and understanding client requirements, choosing and forming the team, selecting and managing sub contractors, balancing the innovation against the risk, communicating within the team and with the client, keeping the client requirements always in mind, managing quality, managing cost, managing time, managing expectations, managing risk, and so on".

Then look for the lessons that will help you with all those tasks, and will help you avoid all the old pitfalls. As the Farnam Street blog says,

If you catch yourself reasoning based on “this time is different” remember that you are probably speculating. While you may be right, odds are, this time is not different. You just haven’t looked for the similarities.

A great antidote to the "This time it's different" fallacy is that good old, tried and tested mainstay of Knowledge Management, the Peer Assist. Once a project team gets into a room with a bunch of people with experience, the conversation automatically focuses on the similarities. "Yes, we've seen that, we've been there, here's what we learned" and it becomes increasingly difficult to maintain that "This time it will be different".

Operationalising Lessons Learned in a small organisation

Here's a really great video on a small organisation operationalising a lessons learned process

The organisation is Boulder Associates, an Architect and Design firm with a couple of hundred staff working out of a handful of US locations. The video was recorded at the KA-connect conference in San Francisco in 2018; an annual knowledge-focused conference for the AEC community organised by Knowledge Architecture.

The video is of a 27 minute presentation given by Todd Henderson and James Lenhart of Boulder Associates, and thanks Todd for the namechecks!

They make the point that collecting lesson is not the purpose of lesson learning, and they drive lesson learning through just in time delivery, using checklists as the final destination for the learning. Their combination of spreadsheet, tracking dashboard and checklists is a very simple, very appropriate system for an organisation of this scale.

Why admitting mistakes is so hard, and what we can do to counter this

It is part of the human condition to deny our mistakes, but that makes it hard for us, and for our organisations, to learn.

make no mistake, a photo by Meshl on Flickr.

I can recommend a really interesting book called "Mistakes were made - but not by me - (why we justify foolish beliefs, bad decisions and hurtful acts)".

The book is about cognitive dissonance - how people square their self-image of being a good, competent and smart person, with the realisation that they can make mistakes - sometimes pretty big mistakes.

The way most people deal with this problem is to maintain the self-image and explain away the mistake.

"It wasn't really a mistake" - "Of course I didn't see the stop sign, it was in such a stupid place" - "The sun was in my eyes" - "Nobody told me it was wrong" - "He pushed me" - "I don't see why I should say sorry, he started it" - "I'm not wrong, aliens really DO exist; the government is covering it all up" and so on.

They didn't REALLY make a mistake; they were smart people all along. It wasn't their fault.

This cognitive dissonance works particularly strongly in cultures that are intolerant of mistakes, and to be honest, that's most cultures. As the playwright Lillian Hellman says about US culture, for example

"We are a people who do not want to keep much of the past in our heads. It is considered unhealthy in America to remember mistakes, neurotic to think about them, psychotic to dwell on them".

Knowledge Management, however, requires that organisations, and the people within them, learn from experience, and 50% of experience comes from Mistakes. KM, if it is balanced, must allow learning equally from failures and from successes. Somehow this very powerful driving force of cognitive dissonance, present in every culture and every human, must be allowed for, sidestepped and redressed.

How do we do this?

Firstly, there much be a very conscious top-down drive for a culture that allows learning from mistakes. Call it a no-blame culture, call it an organisational learning culture, call it openness - it needs to be a clear and conscious expectation. This sets up a counter-dissonance. "I am a smart person. But I made a mistake. But the company expects me, if I am smart, to admit mistakes. Hmmmm!"

We all know the story told about Tom Watson Sr, the first president of IBM.

A young worker had made a mistake that lost IBM $1 million in business. She was called in to the President’s office and as she walked in said, “Well, I guess you have called me here to fire me.” “Fire you?” Mr. Watson replied, “I just spent $1 million on your education!”

That is a very powerful story that sets up a counter-dissonance.

Or look at the NASA approach, of getting senior leaders to publish stories entitled "my best mistake".  I suggested this to an organisation recently, and the head of KM actually said "you will NEVER get leaders to publish their mistakes". And yet that's just what they did at NASA.

Or look at the Japanese attitude of Hansei. Although alien to many in the west, Hansei is an important part of the Japanese culture. Han means “change” and Sei means “to review”, so the whole thing means “introspection” or “reflection for the purposes of change”.  This translates into a behaviour , instilled from childhood, of looking for mistakes, admitting responsibility, and implementing change.(When Japanese children do something wrong, for example, they are told: "Hansei shinasai - Do hansei"!). Hansie also contains the thought that "to say there were no mistakes, it itself a mistake"

Secondly, we use objective facilitation.

Take lessons-identification meetings, for example. The temptation, when scheduling a meeting for a project team to identify lessons from experience, is for the project leader to lead the meeting. However the outcome, most of the time, is that the project team made No Mistakes. Sure, mistakes were made, but not by the project team!

You see this, for example, when the lessons coming from the team are like this ....

We ordered a set of number 6 widgets, and when they arrived, they were very poor quality. We had to send them all back, which delayed construction by a week.  The lesson is not to use that supplier again".

So the fault was with the supplier.

However a good objective facilitator would dig deeper. They would ask - what were your quality control procedures? What was your ordering philosophy? Did you wait for the widgets to arrive before doing quality control? If so, why did you leave it until the last minute, so that re-ordering caused delay? Did you just assume that everything would be top quality?" The lesson would be more about quality control procedures and mental assumptions, that it would be about suppliers and vendors.

Any good system of lessons identification and lesson-learning has to use objective external facilitation, if you are to overcome the tendency to say "mistakes were made, but not by my team." That's why an increasing number of companies are calling us in, for example, to provide that skilled external evaluation as part of their lesson learning system.

This is even more the case with high level review and learning. As the authors of the book say,

"Few organisations welcome outside supervision and correction. If those in power prefer to maintain their blind spots at all costs, then impartial review must improve their vision ....... If we as human beings are inevitably afflicted with tunnel vision, at least our errors are more likely to be reduced or corrected if the tunnel is made of glass".

This goes for teams and individuals as well as organisations - impartial assistance is vital.

So, set the high level expectation for openness, and use external facilitation to probe the blind spots.

We all make mistakes - mistakes have been made, often by us, and those mistakes are an opportunity to learn and improve, despite our best efforts to pretend that they never happened.

Lesson learning in the US Army - example from Haiti

Army learning is not just about fighting battles - here's an example from disaster response

Marines hand out rations in Haiti, by Marines Relief, on Flickr

In 2010, the US Army was called in to provide humanitarian aid, including food and shelter, after a category 7 earthquake in Haiti. This article by the US contracting commend, entitled Lessons learned and used during Haiti deployment described how lesson learning made this a more efficient and effective process.

The article describes how, at the height of the response, the US Army supplied more than 15 million meals  in a 10-day period to the Haitian population, as well as setting up distribution points for families to receive boxes and bags of rice, beans and cooking oil. All of this required a supply chain and contracting and purchasing activities, and by the end of the mission, the Expeditionary Contracting Command had created more than 380 contracting actions valued at almost $12 million. Of course this supply chain needed to be slick and well organised, and to have learned from the past.

"We took advantage of a lot of lessons learned from previous deployments. We didn't do these types of things early on in Operation Iraqi Freedom or Operation Enduring Freedom. However, we learned those lessons and brought these capabilities to Haiti early on," said Brig. Gen. Joe Bass, commander, Expeditionary Contracting Command. "We were very proactive from the beginning, deploying the right personnel mix needed to provide quality assurance, legal, policy and other areas where we could address issues on the front end rather than after they've been done.

Lessons included

• the personnel which needed to be deployed with the first troops, 
• the need to set up a support centre in the US, 
• the need for review and decision making boards in Haiti, 
• creation of "pre-positioned deployable equipment packages", and 
• the correct level of decision making authority for procurement orders of different value from $100 thousand to $1 million.

The Haiti organisation did not just learn lessons from the past, they created new lessons to help future operations.

"Learning from the past helped us deploy quicker and smarter," Bass said. "Just as we gathered lessons learned from previous deployments, we have gathered some from the Haiti deployment that should help us the next time we have to deploy. Moving forward means reviewing what we've done and how we have done it in the past, then reviewing it again and constantly using those lesson to better ourselves with each new challenge"

This is a great example of an organisation using lesson learning to continuously improve operations. 

Lessons Learned, or lessons lost?

Are you learning lessons, or losing lessons?

Kizar [Public domain], from Wikimedia Commons

A Lesson is an Investment

It might take a project team of 10 people, one day to create, through facilitated discussion, ten high-quality lessons. So each of these takes one man-day to create; say £500 - plus some write-up time - lets say £1000 per lesson to choose a nice round number.

However that lesson is encapsulated knowledge which, if re-used, may save man-months of wasted time and effort in the future, or tens of thousands of pounds of wasted cost. The project team have invested their time to deliver future ten-fold or hundred-fold return on that investment. That lesson may therefore be an investment worth £10,000 to £100, 000 when realized through application.

So what do we normally do with our investments?

Do we hide them in a hole and never look at them again? Do we file that share certificate in a drawer and never look at it again? Or do we track our investments until the time and situation is right to realise them? When it comes to money, we do the latter. But what about lessons?

Unfortunately, a common approach to lessons is to record them in a report, or on a spreadsheet, then put them into project files, and hide them in the online filing system. I heard a report a couple of weeks ago about a team that created some hugely valuable lessons, and stored them in security-controlled project files to which no other project was allowed access!

That's not Lessons Learned; that's Lessons Lost.

That's like the last scene in Indiana Jones, when the Ark of the Covenant is put in a crate, and hidden in a vast warehouse of identical crates.

What we should do with investments, is put them in a managed portfolio, and we track them, until we find the opportunity to realise their value.

What we should do with lessons, is put them in a Lessons Management System, and track them, until we find the opportunity to reuse them or to embed them into process and procedure, thus realising the investment of time and effort that was put into generating them in the first place.

That's Lessons Learned; rather than Lessons Lost.

How the coastguard seeks input to lesson learning

Public organisations can learn from the coastguard when it comes to getting wide scale input to lesson learning

US coastguard units train on Lake Ponchartrain by
  Coast Guard on Flickr

Any public organisation, especially one with an element of high priority service, needs a lesson-learning process to improve that service. The emergency response services in particular have well-developed lesson learning systems, but here is a wrinkle I had not seen before, from the US coastguard.

This article from 2017, entitled "Innovation Program seeks hurricane lessons learned from Coast Guard responders" describes how the US coastguard set up what they called the "Hurricane Lessons Learned challenge" on the Coast Guard’s ideas-capturing portal CG_Ideas@Work.

This portal was started as a way to preserve and institutionalize the wealth of lessons learned during hurricane response efforts, and all Coast Guard personnel who participated in any of the response efforts are encouraged to share their observations, issues and ideas.

This is a means of capturing ideas observations and insights which analysts later could convert into lessons (the sequence from Observations to Insights to Lessons is widely recognised in the Lesson learning community). Some direct lessons may also be captured.

As the article explains

 The Coast Guard routinely captures lessons learned as a way to improve its operations, but the CG_Ideas@Work challenge offers one distinct advantage: “Our crowdsourcing platform not only provides a place to submit ideas, but also to collaborate on them,” (Cmdr. Thomas “Andy”) Howell said. “Everyone from non-rates to admirals can discuss ideas.” Speed is also an advantage. “Catching the ideas when they’re fresh and raw preserves their integrity,” Howell said.

The US Coastguard are well aware that capturing lessons is not enough for them to be a learnign organisation. These lessons must also drive change.

“The Commandant’s Direction says we need to become an organization capable of continuous learning, so it’s important that the innovations and adaptations that made this response successful are institutionalized,” Howell said. Ideas shared through the Hurricane Lessons Learned challenge are immediately shared with the responsible program. Many will be considered as potential projects for next year’s Research, Development, Test and Evaluation Project Portfolio.

The portal has been very well received

“We’ve heard from pilots, inspectors, commanding officers, district command staffs, reservists, Auxiliary personnel – the entire gamut of responders,” Howell said. “It’s a very user-friendly way to collect information, and comes with the benefit of collaboration,” he said.

This is an approach other similar organisations can learn from.

What's the difference between a lesson-learned database and a lesson management system?

In this blog post I want to contrast two software systems, the Lessons Database, and the Lessons Management System.

Lessons Leaned. Sure, by Mike Licht on Flickr

There are two types of Lessons Learned approaches, which you could differentiate as "Lessons for Information" and "Lessons for Action".

These represent maturity levels 1 and 2 from my three level categorisation, and can be described as follows.

"Lessons for Information" is where lessons are captured and put in reports, or in a database, in that hope that people will look for them, read them, and assimilate them.

"Lessons for Action" is where lessons are used to drive change and improvement. Lessons are captured, reviewed, validated, and action is taken to embed the lessons in process, procedure, standards and/or training.

"Lessons for Information" is supported by a Lessons Database, "Lessons for Action" by a Lessons Management System. Let's contrast the two.

• In a Lessons Database, the database is the final home of the lessons. In a Lessons Management System, the final home of lessons is considered to be the compiled knowledge of the organisation, which may be procedures, doctrine, guidance, best practices, wikis, etc. The Lesson Management System manages lessons on the way to their final homes.

• In a Lessons Database, lessons reach their reader through search. In a Lessons Management System, lessons are pro-actively routed to those who need to see them and to take action.

• In a Lessons Database, lessons accumulate over time (this was the problem with my first Lessons system in the 90s - it got clogged up over time with thousands of lessons, until people stopped looking). In a Lessons Management System, lessons are archived once they have been embedded into process and procedure, and the only live content in the system is the set of lessons currently under review.

• In a Lesson Database there is only one type of lesson - the published lesson. In a Lesson Management system there are at least two types of lesson - the open lesson (where action has not yet been taken) and the closed lesson, which may then be archived. Some organisations recognize other types, such as the draft lesson (not yet validated) and the Parked lesson (where action cannot yet be taken, or where action is unclear, and where the lesson needs to be revisited in the future).

• In a Lessons Database, there may be duplicate lessons, out of date lessons, or contradictory lessons. Through the use of a Lessons Management System, these have all been resolved during the incorporation into guidance.

•  In a Lessons Database, there are limited options for metricating the process. You can measure how many lessons are in the system, but that's about it (unless you capture data on re-use). Through the use of a Lessons Management System, you can track lessons through to action, and can measure whether they are being embedded into process, you can see where they are being held up, and by whom, and you can see how long the process is taking and where it needs to be speeded up.

Lessons management is the way to go. Lesson databases really do not work in the long term, and usually become lesson graveyards, and the home for Lessons Lost.