Monday, 20 June 2022

The three levels of lesson-learning - don't get stuck at level 1!

Here is a reprised blog post presenting 3 potential levels of maturity for a lesson-learning system. Many or most organisations are stuck at level 1.

Lesson-learning is a common component of KM management systems, and there are three levels of rigour, or levels of maturity, regarding how Lesson-learning can be applied.

  • The first is to reactively capture and document lessons for others to search for, find and read ("Lessons for Information");
  • The second is to reactively capture lessons at the end of projects and after events, document them, and as a result make changes to company procedures, practices and training ("Lessons to Make Change");
  • The third is to proactively hunt lessons from wherever they can be found, and make changes to company procedures and practices so that the lessons are embedded into practice. 

Lesson-learning can be a very powerful way for an organisation to learn, change and adapt, but only if it is approached in a mature way. Level 1, to be honest, will not add much value, and its only when you get to level 2 that Lesson learning really starts to make a difference.

Let's look at those levels in more detail.

Level 1

Level 1  is to reactively capture lessons at the end of projects, and document them so that others can learn. Lessons are stored somewhere, and people need to find and read the lesson in order to access the knowledge. There are sub-levels of maturity in level 1, which include
1a) Ad-hoc documentation of lessons by the project leader, documenting them and storing them in project files with no quality control or validation step. Lessons must therefore be sought by reading project reports, or browsing project file structures
1b)Structured capture of lessons, through scheduled lessons identification meetings such as retrospects, documenting and storing the lessons in project files with no quality control or validation step.
1c) Structured capture of lessons, through sheduled lessons identification meetings such as retrospects, documenting and storing the lessons in a company-wide system such as a lessons database or a wiki. This often includes a validation step.
1d) Structured capture of lessons, through lessons identification meetings such as retrospects, documenting and storing the lessons in a company-wide system with auto-notification, so that people can self-nominate to receive specific lessons.

Level 2

Level 2 is to reactively capture lessons at the end of projects, document them, and as a result make changes to company procedures and practices so that the lessons are embedded into practice, procedure, guidance, design and/or training. Here people do not need to read the lesson to access the knowledge, they just need to follow the updated and improved practice etc. Lesson learning has become integrated into, and drives, continuous improvement. Again, there are sub-levels of maturity in level 2, which include:
2a) Lessons are forwarded (ideally automatically, by a lesson management system) to the relevant expert for information, with the expectation that they will review them and incorporate them into practice, procedure guidance or training;
2b) Lessons include assigned actions for the relevant expert, and are auto-forwarded to the expert for action;
2c) As 2b, with the addition that the actions are tracked and reported.

Level 3  is to proactively hunt lessons from wherever they can be found, and make changes to company procedures and practices so that the lessons are embedded into practice.  There are not enough organisations at level 3 to recognise sub-levels, but there are some ways in which Level 3 can operate
3a) Senior managers can identify priority learning areas for the organisation. Some projects are given learning objectives; objectives for gathering knowledge and lessons on behalf of the organisation. These may be written into project knowledge management plans. 
3b) Learning teams may analyse lessons over a period of months or years to look for the common themes and the underlying trends - the weak signals that operational lessons may mask.
3c) Organisations may deploy specific learning resources (Learning Engineers, Project Historians, etc) into projects or activity, in order to pick up specific learning for the organisation. See this example from the Australian emergency services. 

I have only really come across level 3 in the military and the emergency services.  For example, see this quote from Lieutenant-General Paul Newton
"The key is to ‘hunt’ not ‘gather’ lessons, apply them rigorously—and only when you have made a change have you really learned a lesson. And it applies to everyone … It is Whole Army business". 

However even level 2 is quite rare. Many organisations have not gone beyond the "document and store" stages of Level 1, and often have been disappointed by the outcomes, and by the tendency to re-"learn" lessons over and over again. After all, in Level 1, nothing changes, and if nothing changes, nothing has been learned. 

If you aspire to be a learning organisation, set your sights at levels 2 or 3.

Monday, 13 June 2022

How KM delivers value by solving knowledge problems

Much of the value delivered through Knowledge Management comes as a result of solving problems.

"How do you show the value of Knowledge Management?"  Yet another client was asking me the same question - how can you demonstrate the value?

I had just quoted to him Shell's 2007 claim that they delivered $200m per year though their knowledge-sharing communities (for an annual investment of $5m), and he just could not see how that value could be measured.

The answer is quite simple.

Shell's Communities of Practice are based on Pull - on Problem Solving. People in the Shell business units have a problem, they need to access knowledge of how to solve the problem, and they go to the Communities of Practice to find that knowledge. When they have found the knowledge and implemented the solution, they estimate how much time, money or risk that solution offered them.

Here's an example taken from the document "Managing knowledge though new ways of working".
You can see clearly how access to Community Knowledge allowed this guy to obtain a better deal with a vendor by accessing knowledge of data prices from the community. Demonstrable value added. Here's another example
One man-month of effort avoided. Demonstrable value. Many more examples are recorded in the document linked above. 

It is by adding up examples like these, that Shell came to the figure of $200m annual savings that they claim for knowledge management. 

Part of the challenge for the customer, was that until that point, he had not made the link between Knowledge Management and Problem Solving. To him, KM was all about blogs and case studies  - about Knowledge Push, and not Pull.

Once he could see the problem-solving link, he could more clearly see how KM could deliver value.

You can find many more value stories on this blog.

Monday, 6 June 2022

Knowledge management: formal or informal? Connect or Collect?

People are often instinctively drawn towards one component of Knowledge Management. Here's a way of looking at those components.

Choices about the approach to KM are often made implicitly, emotionally, or through assumption, so it’s worth taking time out to analyse these approaches intellectually, before starting work on your Knowledge Management Strategy or Framework.

 I am going to look at some of these philosophical choices under two headings;

  • Whether we will choose a formal system or an informal system;
  • Whether we will focus on connecting people, or on collecting documented knowledge.

Together these two choices form a Boston Square, shown here.

 Formal v informal

The choice between formal and informal is an important one. We can take a formal system as being one with defined roles, defined expectations, defined technology and defined workflows. The system operates within a defined framework, or set of rules. An informal system, on the other hand, has few if any defined roles or expectations, and operates in an adhoc manner.

The choice between formal and informal is often an ideological choice. Either we feel that learning and knowledge are organic processes that will be killed by any degree of formality, or whether we feel that learning is far too important to be left to adhoc informality and chance.

Connect v Collect

The system can also be driven through Connect, or through Collect. In other words, knowledge and lessons can stay largely undocumented in the heads of people, or we can try to transfer knowledge through the use of recorded or written material. In a Connect system, we look at building networks of people who seek and share knowledge through dialogue and conversation. In a Connect system, the transfer of knowledge requires it to be written down and stored, so others can find it and learn from it.

Again, this is often an ideological or philosophical choice. People feel that knowledge is inherently a human property that can only be transferred through human interaction. Or people feel that there needs to be a centrally accessible knowledge base that people can refer to and rely on.

As this picture shows, the interplay of connect/collect and formal/informal gives four quadrants, which can represent four end-member choices for a lesson learning system.

Formal Collection 

The formal collection quadrant is where a company has an organized and managed system for building a collection of documented knowledge. This quadrant is the home of lessons databases, knowledge bases, best practice collections etc.  Formal knowledge bases have the great advantage that it is easy to track, find, sort and group knowledge. The disadvantages with such a system are that often crucial knowledge is not documented, or that people can find it frustrating or difficult to update the knowledge base as knowledge changes.  The more formal the system, often the more challenging it becomes to enter content, though equally it becomes easier for retrieving and tracking content.

Informal Collection 

At the other end of the formality scale are the voluntary, ad-hoc and self-organising community tools such as wikis (although of course a degree of formality can be built around a wiki). The Wikipedia model has sometimes been suggested as a model for informal knowledge collecting in a large organization, allowing documented knowledge to "spontaneously emerge from crowds". The great advantage of wiki technology is that it is extremely easy to enter basic content, and a little bit of technical skill allows you to add a richness of multimedia content as well. If you are motivated to publish, Wikis such as Wikipedia offer an simple route, and the crowd can be expected to edit as well as to source material.

 However there are drawbacks with the informal Wikipedia model. The 90:9:1 rule tells us that voluntary wikis draw on only about 2% -3% of available knowledge, and all submissions in the Wikipedia model are voluntary and adhoc. So unless there is a huge user base and massive redundancy or overlap in knowledge, there is a real risk that crucial knowledge may never enter the system.

Formal Connect 

Formal connect-based systems are the formal networks, expert locators and the top-down Communities of Practice, which allow members use each other as a resource and repository of unwritten knowledge. Here knowledge exchange is through dialogue, accomplished within a formal network of people or at a formal meeting. Formal Connect systems are ideal for sharing lessons in areas of complex or context-specific need and for topics which are rapidly changing, and where new problems are being regularly identified. They are less appropriate where processes are becoming better defined and more standardized, and where lessons can be collected and incorporated into standards and guidelines.

Informal connect 

The fourth quadrant represents the informal Connect-based systems. Examples here can be found in self-organising social networks. Here is the extreme of informality, where discussion groups emerge from bottom-up interest, allowing questions to be asked, answers to be given and lessons to be exchanged in a loose and mobile network of contacts. The appeal of these systems is their extreme informality and ease of use, and the introduction of systems such as these can help to develop a more open discussion-oriented culture in an organisation. They also allow for serendipity; chance meetings with unusual sources of learning. The disadvantage is the great difficulty in ensuring the right questions are asked in the first place, and then in making sure they are answered by someone with valid lessons and experience to offer. Many online discussions can end up as an exchange of opinions among a random grouping, rather than an effective trawl for experience; more gossip than an exchange of lessons. Also the lack of rules can lead to a proliferation of networks covering the same content (see for example the 11,000 groups you find on Linked-In if you look for a Knowledge Management group. Ideally there should be just one! Or at the most, a couple of dozen to cover key sub-areas)

The blended and balanced approach 

As far as Connect and Collect are concerned, I firmly believe that the answer is “Both/And” rather than Either/Or. They are not alternatives, but complementary approaches. Any complete Knowledge Management Framework needs a blend of Connect and Collect, running both approaches in parallel. They need to be cross-linked of course, and the communities or networks can take accountability for some of the Collection, as well as the Connection.

As far as informal/formal is concerned, the answer is to find the right balance. Not a blend, but a balance. In any one company, for any one topic, to run formal and informal systems in parallel would be to confuse the user, and often to undermine one or other of the two - “I know that’s what it says in the official process, but what is the word on the street?”

There is no value to anyone if the word on the street and the official line diverge. Which lessons will you follow in that case? The formal/informal balance on the Connect side is found in the communities of practice, where a community will develop (or be given) a level of formality which suits their need and purpose.

The answer is to recognise these four end members, and to see them not as options to choose between, but as components of a complete, blended and balanced system. 

Monday, 30 May 2022

How much knowledge can one person hold? Introducing the Personbyte.

What are the limits to one person's knowledge? Other than being "one personbyte"?

The April 1, 2017 edition of New Scientist magazine had the theme of Knowledge, and contained a set of Knowledge-related articles, one of which dealt with the issue of how much one person can know.

The issue, it turns out, is not so much about the storage capacity of the human brain, but the appallingly low speed of uploading the knowledge. Compared to computers, we are very very slow at inputting new material, and the limits to knowledge are constrained by our learning rate rather than our storage capacity.

The article describes the idea of a "personbyte" - the amount of knowledge one person can reasonably learn in a lifetime. In the craftsman economy of 100 years ago, a personbyte was enough knowledge to create an impressive artefact - a steamboat, a canal, or a suspension bridge. Nowadays one personbyte is nowhere near enough to create modern products, or deliver modern services.

For example New Scientist suggests that
"to build an F-22 Raptor fighter jet, complete with on-board missile-guidance systems, you are going to need many thousands of personbytes."

Which means, of course, that you are going to need Knowledge Management, to ensure that the knowledge from thousands of people is integrated into a single design and product.

If all the necessary knowledge can be held by one person then there is no need for KM (Aristotle didn't need a knowledge manager), but when a job requires the knowledge of thousands of people, then there needs to be some sort of management system in place; not just to coordinate this knowledge, but to preserve it for the future. When the Raptor Jet needs to be serviced, for example, that knowledge needs to still be available.

For any job requiring more than one personbyte of knowledge, the knowledge work must be seen as a collective endeavour and not a personal endeavour, which means that the knowledge must be treated collectively and seen as a collective resource.  The role of KM is to ensure that the collective resource is well managed, accessible when needed, and of a reliable quality.

As the New Scientist article concludes;
"The barrier to progress lies not in the quantity of knowledge our brains can hold, but in its quality"
So its not so much about how much knowledge a worker can personally know, but whether they can access the correct, relevant, high quality, knowledge at the right time.

Ensuring the availability and quality of this knowledge is our task as knowledge managers.

Monday, 23 May 2022

Is that really tacit knowledge, or could it be cognitive bias?

Is that really Tacit Knowledge in your head, or is it just the Stories you like to tell yourself?

IMAGINATION by archanN on wikimedia commons
All Knowledge Managers know about the difference between tacit knowledge and explicit knowledge (or at least they think they do!), and they know the difference between the undocumented knowledge you hold in your head, and documented knowledge which can be stored.   We often assume that the "head knowledge" (whether tacit or explicit) is the Holy Grail of KM; richer, more nuanced, more contextual and more actionable than the documented knowledge.

However the more I read about (and experience) cognitive bias and the failures of memory, the more suspicious I become of what we hold in our heads.

These biases and failures are tendencies 1) to think in certain ways that can lead to systematic deviations from good judgement, and 2) to remember (and forget) selectively and not always in accordance with reality. We all create, to a greater or lesser extent, our own internal "subjective reality" from our selective and flawed perception and memory. Some of this might be real knowledge, some might not.

Cognitive and memory biases include:

  • Confirmation bias, which leads us to take on new "knowledge" only when it confirms what we already think;
  • Gamblers fallacy, which leads us to think that the most recently gained knowledge is more important;
  • Post-investment rationalisation, which leads us to think that any costly decisions we made in the past must have been correct ("we spent a lot to learn that, so the knowledge must be correct");
  • Observational selection bias, which leads us to think that things we notice are more common that they are (like when you buy a yellow car, and suddenly notice how common yellow cars are);
  • Attention bias, where there are some things we just don't notice (see the Gorilla Illusions);
  • Memory transience, which is the way we forget details very quickly, and then "fill in the gaps" based on what we think should have happened;
  • Misattribution, where we remember things that are wrong;
  • Suggestibility, which is where we create false memories.

So some of those things in your head that you "Know" may not be knowledge at all. Some may be opinions which you have reinforced selectively, or memories you have re-adjusted to fit what you would have liked to happen, or suggestions from elsewhere that feel like memories. Some of them may be more like a story you tell yourself, and less like actual knowledge.

Do these biases really affect tacit knowledge? 

Yes they really do, and they can affect the decisions we make on the basis of that knowledge.  Chapter 10 of the 2015 World development Report, for example, looks at cognitive biases among development professionals, and makes for interesting reading.

While you would expect experts in the World Bank to hold a reliable store of tacit knowledge about investment to alleviate poverty, in fact these experts are as prone to cognitive bias as the rest of us. Particularly telling, for me, was the graph that compared what the experts predicted poor people would think, against the actual views of the poor themselves. 

The report identifies and examines 4 "decision traps" that affect the development professionals and influence the judgements that they make:

  • the use of shortcuts (heuristics) in the face of complexity; 
  • confirmation bias and motivated reasoning; 
  • sunk cost bias; and 
  • the effects of context and the social environment on group decision making.

And if the professionals of the World Bank are subject to such traps and biases, then there is no guarantee that the rest of us are any different.

So what is the implication?

The implication of this study, and many others, is that one person's "tacit knowledge" may be unreliable, or at best a mish-mash of knowledge, opinion, bias and falsehood. There is a risk that knowledge from one person's is unreliable, unless tested somehow. As Knowledge Managers, there are a number of things we can do to counter this risk.

  1. We can test Individual Knowledge against the knowledge of the Community of Practice. The World Bank chapter suggests the following: "group deliberation among people who disagree but who have a common interest in the truth can harness confirmation bias to create an efficient division of cognitive labor. In these settings, people are motivated to produce the best argument for their own positions, as well as to critically evaluate the views of others. There is substantial laboratory evidence that groups make more consistent and rational decisions than individuals and are less likely to be influenced by biases, cognitive limitations, and social considerations. When asked to solve complex reasoning tasks, groups succeed 80 percent of the time, compared to 10 percent when individuals are asked to solve those tasks on their own. By contrast, efforts to debias people on an individual basis run up against several obstacles, and when individuals are asked to read studies whose conclusions go against their own views, they find so many flaws and counterarguments that their initial attitudes are sometimes strengthened, not weakened". Therefore community processes such as Knowledge ExchangePeer Assist and general community discussion can be ideal ways to counter individual biases.
  2. We can routinely test community knowledge against reality. Routine application of reflection processes such as After Action review and Retrospect require an organisation to continually ask the questions "What was expected to happen" vs "What actually happened".  With good enough facilitation, and then careful management of the lessons, reality can be a constant self-correction mechanism against group and individual bias.
  3. We can test the knowledge against other viewpoints. Peer Assist, for example, can be an excellent corrective to group-think in project teams, bringing in others with potentially very different views. 
  4. We can combine individual memories to create a team memory. Team reflection such as Retrospect is more powerful than individual reflection, as the team notices and remembers more things than any individual can.
  5. We can codify knowledge. Lean as codified knowledge is, it at least acts as an aide memoire, and counteracts the effects of memory transience, misattribution and suggestibility. 
But maybe the primary thing we can do is to stop seeing individual tacit knowledge as being safe and reliable, and instead start to concentrate on the shared knowledge held within communities of practice.  

Think of knowledge as Collective rather than Individual, and you will be on the right track.

Monday, 16 May 2022

How do pilots ensure knowledge is reused?

One of the biggest challenges is knowledge re-use. How does the aviation industry address this challenge?

Image from wikimedia commons
I often refer to aviation as a successful example of knowledge management, with lessons captured from every accident and incident and provided to pilots in the form of checklists, or shared through site such as the Skybrary.

But how does the aviation industry address the issue of knowledge re-use? Why don't experienced pilots skip the checklist?

We know that this is a big challenge in other industries, and that experienced doctors, engineers, programmers and consultants often do not re-use knowledge, but rely instead on the knowledge they already have. 

How do you make sure that experienced airline pilots, with thousands of hours under their belt, stay disciplined and use the checklists, even after flying has become routine?

  • Because if they skip (the checklist) and it was not OK they can be fired and lose their license
  • In four years in an airline cockpit I only encountered 1 person who didn't respect checklists. Perhaps not coincidentally he did not make it through his probationary year and was fired 
  • Because if they skip it and it was not OK they could DIE
  • On commercial flights, key checklist items are forced by using a procedure call a "cross check". The one pilot must "challenge" another for those specific check list items. 
  • The cockpit consists of two people, one reading/actioning the checklist, the other one monitoring and checking/cross-checking. If you say: "Nah, no checklist today", your copilot is bound to say: "Sorry, but we have to!" 
  • The importance of a proper preflight is drilled into you by your primary instructor from day one in light aircraft, and that mentality carries through all the way up to heavy transport-category aircraft: You want to find any problems you can while you're on the ground, because if you take a problem into the air with you it's a decision you can quickly come to regret.
  • If you do it often enough, it becomes a habit. It then feels wrong to not run the checklist. 
  • You stay vigilant by having seen things go wrong.
  • Everyone expects everyone else to do the checklists properly and if you don't do it you will get called out. 
  • In an airline environment you will have recurrent check-rides every 6-12 months and captains will have line checks every year and proper checklist usage is among the most basic requirement to pass these checks.
  • Every year we sit through a day of crew resource management training and part of that day involves looking at past accidents and understanding what the first thing was that set the accident events in motion (pilot error!). These often serve as vivid examples of how bad things can get if you start ignoring the checklists (among other things) 
  • There are a few tricks that are used to stop you falling in to that "Yeah, everything will be fine" mindset and just skipping the checks 
    • Not doing the checks from memory, but actually doing them in reference to a physical check list. 
    • A prescribed order of checks, starting at a point on the aircraft and moving around methodically 
    • The fear of missing something, such as engine oil levels, which gets very serious once airborne. 
    • Once carrying passengers, especially nervous ones, they tend to feel safer when they've seen you PHYSICALLY checking the aircraft before flying it. 

We can see several factors at work here, including

  • a logical and emotional case for learning (we might die, our passengers might die, better to fix things on the ground, fear of missing something), 
  • peer pressure from the copilot (you will get called out) and passengers
  • you might lose your job if you skip it
  • an awareness of what might go wrong (looking at past accidents)
  • training (from day one, and every year)
  • audit (checkrides)
  • physical lists (not relying on memory)
  • logical lists (prescribed order of checks)
  • habit

Many of these can be transferred from the aviation sector into other sectors. You could imagine a company where the re-use of existing knowledge (in checklists or procedures or other guidance) was mandatory, trained, supported, checked, believed-in (perhaps through regular analysis of failures), audited and habitual.

I agree this is a long way from where many of us are at the moment, but it is a vision for how an industry can support the re-use of knowledge.

To finish, here is a personal story from the Stack Exchange thread of how one person re-learned the importance of checklists
  •  In my case, I learned the discipline to use a checklist for every action on every flight, the one time I decided not to use a checklist while taxiing from the fuel pump back to the parking ramp. It was winter, and a snowplow pulled up behind me, so I decided not to use the checklist in the interest of expediency (ha!). I primed the engine, checked the fuel valve, engaged the electrical system, keyed the starter, and the engine responded by firing up and then immediately dying. Repeat about a half-dozen times, at which point, I finally decided to use the checklist because something obviously wasn't right. Once again, primed the engine, checked the fuel valve, move the mixture to the rich posi-- Oh...I had left the mixture in the idle-cut-off position. Oops. I've used a checklist religiously ever since then ;)

Monday, 9 May 2022

At what point does Learning and Development take over from Knowledge Management?

Knowledge Management and L&D are both part of the spectrum for Organisational Learning. But where does one take over from the other?

This topic has been a point of discussion ever since KM began.  Where does KM end, and Learning and Development take over?

We can look at this through the lens of 70:20:10, with KM facilitating the 70%, L&D the 10%, and both facilitating the 20%  in the middle. Or we can look at it through the lens of Push and Pull and knowledge flow. L&D is primarily a Push model with one-way flow of knowledge to the user, while in KM everyone is a teacher (or a supplier of content, if you prefer) as well as a learner. 

Another way to look at the question is to consider the characteristics of the knowledge topics themselves. 

You can use the matrix above to characterise knowledge topics using two dimensions; the maturity of the topic (see here for an explanation of knowledge maturity) and the number of people who need the knowledge.  L&D requires corporate investment in creating eLearning and training courses, and is best suited for knowledge which is mature and so not likely to change on a monthly basis, and which has a large enough user-base to merit the investment.  You can develop some marvellous on-line material to distribute this knowledge, and if the topic is mature, you don't need to worry that the flow of knowledge is one-way, as there is not much to share that is new.

If on the other hand the Knowledge is evolving, changing and developing, you need the multi-way flow processes of KM, using knowledge from the users to continually improve the reference material. If there are many users of the knowledge, you are in Community of Practice territory. Knowledge can be shared between the CoP members, who can both use the knowledge and add knowledge of their own. The reference resources can be co-created on the community wiki and continuously updated with new experience. Creating an eLearning syllabus would be a waste of money, as it would be out of date in the first month.

Where there are few users of the evolving knowledge, a CoP solution may be too large-scale. Perhaps we need action learning, or a small group of interconnected experts acting as a global practice group. The focus here will be on knowledge creation as much as knowledge distribution.

The quadrant which is less clear is the bottom left quadrant above, where knowledge is mature (and thus we can focus on one-way knowledge flow) but there are too few users to merit investment in eLearning or formal training. Perhaps the answer here, of the knowledge topic is important enough, is to build the knowledge assets which will act as reference material, and maybe here KM and L&D can work together.

Of course the real world does not divide into simple quadrants, and knowledge topics are often on the boundaries between one quadrant and another, or even move across the boundaries as time goes by. This is why L&D and KM need to cooperate - the corporate universities and the knowledge management teams working together to map out the landscape of knowledge topics, and deciding between them how best to keep that knowledge fresh, current, accurate, accessible and easy to assimilate and use. 

But in general terms, we can say that L&D takes over from KM when the knowledge topic is mature enough to be stable, and when the user population is large enough to merit the investment.

Blog Archive