How often do you audit your KM framework?

Any type of management system requires a regular audit. How often do you audit your KM management system?

audit checklist by Boris Dzhingarov
on Flickr

If you look at the ISO standards for management systems, all of them include a clause about audit.

The quality systems standard ISO9001:2015, for example, contains the following clauses:

• 9.2.1 the organization shall conduct internal audits at planned intervals to provide information on whether the quality management system 
• a) conforms to 
• 1) the organizations own requirements for a quality management system,
• 2) the requirements of this International Standard
• b) is effectively implemented and maintained.
• 9.2.2 the organization shall plan, establish and maintain an audit program including the frequency, methods, responsibilities, planning requirements and reporting ... (the text continues for several sub-bullets).

In the absence of a completed ISO standard for Knowledge Management (due out 2017 or 2018), we need to look at these other management system standards to see what good practice looks like, and ISO good practice for every management system includes regular audit and review.

KM Framework audits

In a recent post, I suggested that there are two types of KM audits; the scan and the assessment. I said the first was like counting the apples in your orchard, the second was like reviewing your farming methods.

It is the assessment - the methods review - that has the most value, in that it tells the health of your Knowledge Management Framework (and, to continue the metaphor, if the farm is healthy the apples will grow).  The assessment will look at every element of your framework, and identify those areas where attention and improvement is needed, be they roles and accountabilities, technologies, processes, or governance.

The assessment is therefore closest to the type of audit mentioned in the ISO standard above, and really should be conducted on a regular basis.

However the audit is also valuable, so long as you also run it regularly. It doesnt idtenfity what you are doing well or badly in KM terms, but it tells you how well managed your knowledge is.

The audit measures the current management status of various knowledge topics - are they documented, are they covered by communities, are they held only in the heads of a few people approaching retirement? - and outputs these as metrics. When you introduce your improved KM approaches, the audit should show improvement over a range of numbers.

Our favoured approach to run the audit on an annual basis. It works like this

• The KM team works with the knowledge owner, or process owner, to audit their own knowledge area.
• Based on the results of the audit, they identify actions for the following year
• The next year, the audit should show improvement
• The KM team can use the audit results, and the change in results over the year, to construct a KM dashboard for reporting to senior management. If KM is working, all audit scores should show an increase over the previous year.

Contact us if you want help auditing your KM Framework.

KM audit - counting apples, or improving the farm?

There are two types of Knowledge Management Assessment or Audit you may need when starting up a KM program.  This is how they differ.

The two types go by different names for different people - survey, scan, assessment, audit or maturity measure, and I explain the difference between the two types with an analogy to assessing the business state of an apple farm.

If you were looking at improving your farming methods in an apple orchard, you could do two assessments; the first would be counting the apples, the second would be reviewing the farming approach.

• If you count the apples but don't review your farming approach, then you might know how many apples you have, but you don't know how to increase the yield.
• If you review the farming approach but don't count the apples, you can't track or demonstrate the increase in yield, and you don't know which trees need the most attention.

Obviously you would do both assessments.

The first assessment, the KMN equivalent of  which we in Knoco call a Knowledge Management Scan, reviews the knowledge topics or knowledge areas within this organisation, and looks at the status of each one in terms of Knowledge Management. Is the knowledge widespread, or local? Is it held by one person, or by a community? Is it purely tactit, or is it fully documented? Does the organisation have the knowledge they need, or do they need to learn more?

The second, which we in Knoco call a Knowledge Management Assessment, looks at the status of the Knowledge Management framework. Are communities of practice in place? Is the technology infrastructure deployed? Are there roles and accountabilities for KM? Is there a closed learning loop? Are there KM expectations or policies? Is there a process for "learning before, during and after"? Is Knowledge Management Governance in place?

If I were to focus on one, however, I would focus on the assessment. If I had to choose just one, I would rather improve the farming method and let the apples take care of themselves. However why choose one?

Why not do both?

Contact us for help in reviewing the status of knowledge management in your organisation.

How effective is government learning? DFID case study

---

Yesterday I blogged about why governments screw up, and made some conclusions about the learning culture in the government system.

Here is a deeper look into the issue of organisational learning in one government department, from an April 2014 document written by The Independent Commission for Aid Impact (ICAI) entitled How DFID learns


DFID is the Department for International Development, responsible for handing the UK's overseas aid budget. ICAI reviewed the extent to which DFID uses information and experience (ie knowledge) to influence its decisions.

It's an excellent report, very well researched and written, which gives DFID a "Mixed" rating for organisational learning. Some very good things are in place, for example a Corporate Learning Strategy, endorsed by the Permanent Secretary in the text that follows -

‘We have committed ourselves to developing a culture of innovation and continuous improvement, being more open and honest about our successes and failures and learning
from the successes and failures of others. As a learning organisation, we aim to encourage constructive challenge to our established practices.’ 

That's a great mission statement that many CEOs could copy, and the Learning Strategy is a highly commendable first step - a sort of KM Policy for DFID. I know from my own experience with DFID that many other elements of KM are in place as well.

However the report shows that improvement in implementing this policy and these elements is still needed, and makes 5 recommendations

1. DFID needs to focus on consistent and continuous organisational learning based on the experience of DFID, its partners and contractors and the measurement of its impact, in particular during the implementation phase of its activities
2. All DFID managers should be held accountable for conducting continuous reviews from which lessons are drawn about what works and where impact is actually being achieved for intended beneficiaries
3. All information commissioned and collected (such as annual reviews and evaluations) should be synthesised so that the relevant lessons are accessible and readily re-usable across the organisation. The focus must be on practical and easy to use information. Know-how should be valued as much as knowledge. (that's an interesting sentence, that last one. I suspect what they call know-how I would call Knowledge, and what they call knowledge I would call Information) 
4. Staff need to be given more time to acquire experience in the field and share lessons about what works and does not work on the ground. 
5. DFID needs to continue to encourage a culture of free and full communication about what does and does not work. Staff should be encouraged always to base their decisions on evidence, without any bias to the positive

All the links above have been added by  me, as the recommendations are all for elements of a full Knowledge management Framework - elements of process, of accountabilities, of governance and of culture. Many organisations I know, private as well as public sector, require similar changes.If DFID can implement these recommendations, then they can become a leading example of KM in the UK government sector.

The most heartening and pleasing thing about the whole report, however, is that it was commissioned in the first place. 

This is evidence that the government wants to learn, is trying to learn, and is willing to be assessed and audited on its learning capability.

10 corridor conversations that say "KM needs a tune-up"

---

If you hear any of these statements in the corridors around your organisation, then your Knowledge Management is not working, and needs a thorough investigation!

1. Why do we keep having to re-learn this lesson?
2. I’m sure I heard someone mention that to me the other day, now who was it?
3. I am sure someone must have done this before somewhere - if only we knew who.
4. You know what - I have just found out the Australia office is doing exactly the same piece of work I am! What a waste.
5. Nobody ever asks me what I think.
6. It's been a total mess since Susie left - she was the only one who knew what to do.
7. It was pure luck that I met Freddy – he had just the answer I was looking for.
8. That project was just deja vu all over again - same old mistakes every time.
9. I could have told you that wouldn't work!
10. I just found this in the archives - if only we had had this when we started the job.

What does a Knowledge Auditor do?

---

Every so often, a company may hold a knowledge audit. The objective of the audit is to look at the knowledge within the company, and see whether it is being properly managed.  The purpose of the audit, and the job of the auditor, is to reassure the company and the shareholders that the intangible asset that is Knowledge is being properly looked after, much as a financial audit provides reassurance that money is being properly managed.

A Knowledge Auditor has a much tougher job than a financial auditor. The invisible nature of knowledge, and the lack of a paper trail for knowledge transfer, compared to the fully documented and counter signed world of money, makes knowledge auditing less of a science and more of an art. However a structured and evidence based approach works  wonders.

The knowledge auditor will look at the state of management of each of the main knowledge domain areas. They will seek for evidence that

1. the explicit knowledge is well documented, complete and up to date
2. the knowledge is owned
3. the knowledge is sufficiently well spread round the organisation
4. there are sufficient subject matter experts 
5. the risk of loss of these experts is manageable
6. there is a sufficiently effective community of practice in place (the auditor may also assess the maturity of the CoP)

Knowledge Management - what's most commonly missing?

---

One of the services we offer at Knoco is an assessment and benchmarking of your Knowledge Management Framework for completeness and maturity.

We do this in two ways, through a high level online self-assessment, and through a detailed diagnostic study where we conduct in-depth interviews and bring an expert eye to bear on your KM approach.

And you know what? There are some interesting patterns emerging from the results. Let's look at the average results from the online survey results first (top right; components are scored from 1 to 5) and look at the factors that, on average, receive the lowest score. These are:

• Roles and accountabilities (a lack of clear KM roles and accountabilities in the organisation)
• Business alignment (KM not aligned with business goals and objectives)
• Governance (no governance, for example no clear expectations, no performance management, no support)

On the other hand, the highest scores are Technology, and Behaviours and Cultures.

Many organisations think that the way to address KM is to address behaviours, and buy technology. The results of the survey suggest that these elements are relatively well covered, and that instead you should introduce some accountable roles, align KM with the business objectives, and get some governance in place, in order that you can deliver value from the technology and behaviours you already have.

If we extract the average scores for the elements of People, Processes, Technologies and Governance as measured by the detailed diagnostic survey (which uses a more detailed analysis, and different scoring levels but the same range), we see a similar pattern. Technology scores highest, Governance scores lowest, and People (roles and accountabilities) scores second lowest.

Technology is not the biggest problem! Governance is the problem, and the lack of Roles.

Finally, from the detailed diagnostic, let's extract the average scores for the four Nonaka elements of Socialisation, Externalisation, Combination, and Internalisation. 

Socialisation (the transfer of knowledge through discussion and conversation, either face to face or through social networks) scores highest, followed by Combination (working with explicit knowledge, combining and storing it).

The lowest score, and significantly lower, goes to Internalisation (the interaction with, and re-use of, explicit knowledge).

Socialising and Sharing, and building knowledge bases, are not the biggest problem! Re-use is the problem.

So what is the learning? The learning is that if you are seeking to improve the effectiveness of your Knowledge management approach, the knee-jerk reactions of "buy more technology" and "share more" may not address where the weaknesses traditionally are. You may need to think more about roles, about business alignment, about governance and about re-use of knowledge. 

Your first step always should be to assess your current state, and identify the gaps and weaknesses that need to be addressed. 

Can you "self-assess" your knowledge management maturity?

---

A simple self-assessment for knowledge management maturity is an attractive proposition for many clients. They come to us asking for our free self-assessment tool, or looking to use our online KM maturity benchmarking survey.

These simple tools give a first pass awareness of how far people have got with Knowledge Management, but there is a real limit with this sort of self-assessment, which is that the relative beginner in KM terms does not know what they do not know, and often will not see what is missing.

We saw that in one company, where we ran a number of surveys, and initially people scored themselves quite high, as they thought "surely we do a lot of KM stuff already". After a while, when they realised what Good KM really could look like, they realised they had a very long way to go, and their self- assessment scores dropped. At first sight the fall in scores might be interpreted as meaning they were moving backwards, whereas in fact they were just beginning to understand how far they had to go.

The self-assessment is a start, but if you are to take KM seriously, you also need an expert assessment.

• If the self-assessment is a Readers Digest article on "How fit are you", then the expert assessment is the visit to the personal trainer.
• If the self-assessment is taking your pulse and temperature, then the expert assessment is the visit to the doctor.
• If the self-assessment is  filling in a cosmo Relationship survey, then the expert assessment is the visit to the Relationship Counsellor.

By all means try our self-assessments as a first step, then when you are ready to really make a start, contact us for a full expert assessment. 

Measuring knowledge

---

In knowledge audits, we seek to measure the status of knowledge within an organisation, to see whether it is at risk, and if so, to what degree of risk, so that we can prioritise our KM efforts.

The problem begins with the “measurement”. How can you “measure” the state of knowledge? Knowledge is not something that can be counted, or weighed, or measured.

All our efforts so far have had to fall back on a qualified analysis of knowledge. We ask for marks out of 10, or marks out of 5, for things like the in-house level of knowledge, the level of documentation, the spread of knowledge within the company, the maturity of the topic and so on.

  It is all very subjective, even when you give guidance in terms of descriptions of what the levels 1 to 5 “look like”, but at this stage I think that this is the best we can do. We bolster this by using the same person to run multiple audits, and to sense-check the results, but there is that underlying subjectivity.

 You can argue that if all we are looking to do is to rank knowledge topics for attention, and to provide a relative baseline to show improvement and to create a dashboard, then subjectivity is OK so long as it is consistent. 

But does anyone else have a more objective approach to the measurement of knowledge, as part of an audit? Suggestions welcome.

KM, the regular health-check

---

There is a lot of value in a KM audit, so long as you do it regularly.

In this post, I suggested that there are two types of KM surveys; the audit and the assessment. I said the first was like counting the apples in your orchard, the second was like reviewing your farming methods.

It is the assessment - the methods review - that has the most value, in that it tells you how to improve your knowledge management approach. However the audit is also valuable, so long as you run it on a regular basis.

The audit measures the current management status of various knowledge topics - are they documented, are they covered by communities, are they held only in the heads of a few people approaching retirement? When you introduce your improved KM approaches, the audit should show improvement.

Our favoured approach to run the audit on an annual basis. It works like this

• The KM team works with the knowledge owner, or process owner, to audit their own knowledge area.
• Based on the results of the audit, they identify actions for the following year
• The next year, the audit should show improvement
• The KM team can use the audit results, and the change in results over the year, to construct a KM dashboard for reporting to senior management. If KM is working, all audit scores should show an increase over the previous year.

KM - counting the apples or improving the farm?

---

There are two types of Knowledge Management Assessment you may need when starting up a KM program. They go by different names for different people - a survey, a scan, an audit.

The first, which we in Knoco call a Knowledge Management Audit, reviews the knowledge topics or knowledge areas within this organisation, and looks at the status of each one in terms of Knowledge Management. Is the knowledge widespread, or local? Is it held by one person, or by a community? Is it purely tactit, or is it fully documented? Does the organisation have the knowledge they need, or do they need to learn more?

The second, which we in Knoco call a Knowledge Management Assessment, looks at the status of the Knowledge Management framework. Are communities of practice in place? Is the technology infrastructure deployed? Are there roles and accountabilities for KM? Is there a closed learning loop? Are there KM expectations or policies? Is there a process for "learning before, during and after"? Is Knowledge Management Governance in place?

If you were looking at improving your farming methods in an apple orchard, the first would be counting the apples, the second would be reviewing your farming approach.
Both are needed.

If you don't review your farming approach, then you might know how many apples you have, but you don't know how to increase the yield.
If you don't count the apples, you can't track or demonstrate the increase in yield, and you don't know which trees need the most attention.

If I were to focus on one, however, I would focus on the assessment. If I had to choose just one, I would rather improve the farming method and let the apples take care of themselves.