Monday, 19 March 2018

How the Emergency Services are resourcing real-time learning

It is common practice to invest time and resources in learning after a project.  Here are some examples of investment during a project. 

I blogged lasy week about lesson learning in the Australian Emergency Services, and made passing reference to Real-Time Evaluation. It's worth spending a little more time on this topic, as this is a departure from the general practice of capturing lessons only in the aftermath of an event.

My colleague Ian sent me these two examples from the Australian emergency services putting resources on to the ground to collect lessons during an incident, rather than waiting until afterwards.

Learning from the Nov 2017 Heavy Rain Event
The Victorian State Emergency Service and Emergency Management Victoria teamed up over December 2017 and January 2018 to conduct a series of debriefs at the incident, region and state level relating to the heavy rain event that occurred at the end of November.
For the first time under the new arrangements a Real Time Monitoring and Evaluation (RTM&E) team was also deployed during the event to inform real time learning. The resulting report, together with the debrief outcomes, will be analysed for insights and lessons and included in EM-Share to support ongoing continuous improvement.

RTM&E Deployed into the State Control CentreOn 19 and 20 January 2018 a small Real Time Monitoring and Evaluation (RTM&E) team was deployed for the first time into the Victorian State Control Centre (SCC) to support the real time learning of SCC staff during the recent heat event.
It was a great opportunity to look at new arrangements and inform future continuous improvement activities across the Victorian Emergency Management sector. All outcomes will be also included in EM-Share.

These are examples of what I call "Level 3" lesson learning; the proactive hunting for lessons rather than reactive capture of lessons after the event.  Please note that Real Time Evaluation is not an alternative to Post-Event Evaluation - both are needed. However the benefits of Real Time Evaluation, and the Proactive capture of lessons, are as follows:

  • The level of resourcing is often greater, rather that trying to squeeze in evaluation time after the activity is over
  • Lesson can be acted on, and problems corrected, while the activity is in progress
  • The RTM&E team can look out for early signs of things happening, and can specifically watch out for lessons on specific topics
  • The RTM&E team can capture lessons while memories are still fresh, before people start to forget.
The main reason why RTM&E needs to be partnered with Post-Event Evaluation such as a Retrospect or After Action review is that until the event is complete, you don't yet know the outworkings of the decisions you made earlier. For example, you may take a course of action that speeds things up, record that as a successful lesson through RTM&E, and then after the event find that there were a whole series of unintended consequences which meant that the course of action was, with hindsight, unwise. 

However, given that caveat, Real Time Evaluation, and the capture of lessons as an event unfolds, can be a really valuable partner to more traditional Post-Event Review.

Friday, 16 March 2018

How the Australian Emergency Services manage lessons

Taken from this document, here is a great insight into lesson management from Emergency Management Victoria. 

 Emergency Management Victoria coosrinates support for the state of Victoria, Australia during emergencies such as floods, bush fires, earthquakes, pendemics and so on. Core to their success is the effective learning of lessons from carious emergencies.

The diagram above summarises their approach to lesson learning, and you can read more in the review document itself, including summaries of the main lessons under 11 themes.

  • They collect Observations from individuals (sometimes submitted online), and from Monitoring, Formal debriefs, After ActionReviews and major reviews.
  • These observations are analysed by local teams and governance groups to identify locally relevant insights, lessons and actions required to contribute to continuous improvement. These actions are locally coordinated, implemented, monitored and reported. 
  • The State review team also take the observations from all tiers of emergency management, and analyse these for insights, trends, lessons and suggested actions. they then consult with subject matter experts to develop an action plan which will be presented to the Emergency Management Commissioner and Agency Chiefs for approval.
  • The State review team supports the action plan by developing and disseminating supporting materials and implementation products, and will monitor the progress of the action plan.

This approach sees lessons taken through to action both at local level and at State level, and is a very good example of Level 2 lesson learning.

Thursday, 15 March 2018

What Google Trends really tells us about KM popularity

Again yesterday I was corresponding with someone who used Google Trends as an argument that KM was dying.

Taken at face value this view is understandable. The google trends plot for KM decreases over time as shown below, showing a steady reduction in relative searches for the term "knowledge management" over the past 8 years.  At first sight this could suggest that the popularity of KM is on the wane, and that fewer and fewer people are searching for the term. However if you dig a little deeper this plot is misleading, and the conclusion that interest in KM is dying is actually a fallacy.

Let me explain why.

Google trends is not an absolute indicator of the popularity of a topic.

That is because Google trends measures "how often a term is searched for relative to the total number of searches, globally", and the total number of searches, everywhere in the world, has rocketed (screengrab from this site below).

Any decrease in the relative percentage, as in the first graph, has to be normalised against the increase in the total number of searches in the second graph.  If the top graph is a measure of the percentage and the bottom graph is the total, then all we need to do is multiply them together to get a measure of the total number of KM searches, and then we will be able to say something meaningful.

That is exactly what I have done in the plot below. The numbers are inexact, as I have just read points visually from the first plot (see table at the base of the post for figures) but the conclusion is obvious.


Google trends is a meaningless indicator unless normalised against the total number of searches. If you do this, then far from KM being in a decline ...

... the total number of Google searches for Knowledge Management has actually increased steadily from 2004 to 2012.  

Raw data for the 3rd graph
year total searches (billion) Googletrends measure of KM share measure of total number of searches for KM
2004 86 100 8600
2005 141 70 9870
2006 230 50 11500
2007 372 40 14880
2008 584 30 17520
2009 792 25 19800
2010 998 21 20958
2011 1109 20 22180
2012 1216 19 23104

Wednesday, 14 March 2018

Using a "pretend customer" in knowledge capture reviews.

When capturing knowledge, sometimes its useful to have a pretend customer you can introduce. 

Image from wikimedia commons
I have blogged about the need to write knowledge as if it were for a "knowledge customer", and to "document for the customer" when capturing knowledge.  But what do you do if nobody knows who the customer is?

A friend of mine recently came up with an interesting solution to this, but introducing a fictional customer.

He was running a Retrospect, and was discussing the learning points from one particularly tricky set of events. The group were struggling to express the lessons in useful terms for future projects, so he reached into his bag and pulled out a little blue toy dog which he was taking home to his son.

"This is Blue Dog" he said. "Blue Dog is running the next project. What advice would you give to Blue Dog, to help avoid the problems you had this time around?"

OK, a bit corny, but it gave the discussion a focus, and the project team were able to come up with some specific and actionable recommendations for Blue Dog. Blue Dog became a very visible stand-in for the Unknown Knowledge User.

Another friend, Lisandro Gaertner fron Brazil, used a similar approach with a three months long training/best practices sharing online program with Sheriffs.

They were challenged to give advice to a new Sheriff called Sherlock Silva about the most common problems they faced it. That worked very well and Sherlock Silva turned in a kind of a meme in the community. "What should Sherlock Silva do?" 

So if your Retrospect is struggling, maybe its worth having a blue dog in your bag, or introducing Sherlock Silva!

Tuesday, 13 March 2018

Example KM policy - AVANGRID

Here's a great principles-based KM Policy

Avangrid is a US-based service company in the Energy Market. Their website says "Our 6,800 employees collaborate to deliver projects that power America’s future, provide clean energy and improve customers’ lives and communities". To support this collaboration, they recently published this Knowledge Management Policy, most of which I reproduce below.

I really like the vision of making knowledge a common asset, and aligning it with strategic competencies.

The Board of Directors of Avangrid, Inc. (“Avangrid”) oversees the management of Avangrid and its business with a view to enhance the long-term value of Avangrid for its shareholders. The Board of Directors of Avangrid (the “Board of Directors”) has adopted this Knowledge Management Policy (this “Policy”) to assist in exercising its responsibilities to Avangrid and its shareholders .....
1. Purpose 
The effective development, dissemination, sharing, and protection of Avangrid’s intellectual capital enhances operational efficiency and is a key element in creating sustainable value for Avangrid’s shareholders. As part of Avangrid’s efforts to implement best practices in knowledge management, this Policy sets forth the main principles that will guide the Avangrid Group in the appropriate dissemination, sharing, and protection of existing knowledge and the implementation of initiatives, procedures, and tools that enable its directors, officers, and employees to benefit from the continuous learning and cultural exchange opportunities. 
2. Principles 
To achieve these goals, Avangrid will endeavor to: 
a) Identify the existing knowledge held by each person and working group within the Avangrid Group and promote the further development of such knowledge. To the extent strategically beneficial and permitted by applicable law, the Avangrid Group will make existing and newly developed knowledge accessible to all other members of the Avangrid Group in order to maximize operational efficiency. 
b) To the extent strategically beneficial and permitted by applicable law, integrate the Avangrid Group’s tangible and intangible assets in order to create an intelligent organizational structure that rewards continuous learning and innovation. 
c) Align knowledge management with the competences and requirements set out in the Avangrid Group’s strategy. 
d) Develop standard systems of knowledge management, identification, and protection across the Avangrid Group that streamlines the proper dissemination and sharing of knowledge within Avangrid Group and enhances operational efficiencies. This will include identifying, developing and putting into places the resources necessary to foster knowledge sharing to the great extent possible through efficient internal dissemination and training; where appropriate, creating and enhancing organizational networks throughout the Avangrid Group; and enhancing the cohesion of existing working groups and teams. 
e) Evaluate the existing knowledge within the Avangrid Group in a consistent manner so that the Board of Directors and, where appropriate, management can assess the effectiveness of the initiatives implemented under this Policy, make changes and improvements where necessary, and promote new innovations in knowledge management. 
f) Respect the intellectual and intangible property rights of third parties in the knowledge management of the Avangrid Group.

Monday, 12 March 2018

Is KM dead? Further evidence of life

We often hear claims that KM is dead or dying, but what does the hard data say?

The "KM is Dead" meme is one with a long history; see articles from 2004, 200820112012, 2015, 2016 to choose but a few. It still seems to resurface several times a year; usually when a software vendor has something to sell (example).

Very seldom are these assertions of the demise of KM accompanied by any data or analysis of trends, other than the Googletrends plot, which as we have seen, is based on searches as a proportion of the total, and would also  point to the demise of project management, risk management, financial management, and so on.

I showed some data from our global KM survey last year which suggested that the uptake of KM may actually be increasing, and here is some new data from the academic world.  The authors of this new study looked at academic KM publications since 1974, when the term was first used, and one of the tables in the text of their article is a list of the number of academic KM publications per year. I used this table to create the graph above.

I don't think you could look at this plot and say KM is dead. You might say it has slowed down a little since a peak in 2010, and that the current number of publications is at about 80% of peak levels, but that's a long way from being dead or dying.

Is KM dead? According to the number of academic publications - No!

Friday, 9 March 2018

Why good Titles are important in KM

If you want knowledge in a lesson, post or knowledge article to be found, give it a good title.

One of the occasional recurring themes of this blog is the importance of Knowledge findability. Knowledge needs to be used in order to add value, and before it can be used it needs to be found. This includes the ability to find knowledge in lessons within a database, stories within a story folder, relevant posts within a community blog, or experience within the head of an expert.

One of the key enablers of findability when it comes to documented stories, lessons and knowledge articles is a Good Title. The Title is the most prominent item in any browsing system or set of search results. The purpose of the Title is to enable the reader to understand whether the item is likely to be relevant to them. Based on the Title, they decide whether to open and read the item. If the title makes no sense, then the seeker may not even realise they have found the knowledge, and may pass over it unknowingly.

So part of the role of the publisher of knowledge, in ensuring findability and reusability, is to give a knowledge item a good and relevant title - not a lazy title, or a "clever" title, or an artistic title, but a title that tells the reader what's inside.

Bad titles

Would you know what the lessons listed were about, before opening them? Would the titles help you find relevant content? Would you even bother to open them? (although I could see you might be intrigued, in some cases). Apologies to any of you who wrote any of these, by the way.
  • Duplicate
  • Learning 1 of 3
  • Public Lessons Learned Entry: 0406
  • Additional learning from (Incident X)
  • Spurious event on (Project Y)
  • Z Project - After Action Review (Lesson Learned)
  • When you sweep the stairs, always start from the top (this one was not about stair sweeping by the way)
  • From take-off to landing (and it's not about flying a plane)
  • Problem

Good titles.

If you want to see good practice in using titles, browse the NASA lessons database where you can find titles like these:

These titles clearly tell you what the lesson is about, and the reader instantly knows whether the lesson is relevant to their context. In almost every case the lesson is related to a process - handling of panels, mapping PC boards, fabrication of cable - or to a component such as Hypergol checkout panels. Someone planning a similar process, or designing a similar component, can find the lesson based in the title.

If you want knowledge to be found and used - pay attention to the title!

Blog Archive