Vanity metrics aren’t actionable. Course completion, login stamps, course progression - they make pretty graphs, tables and charts, but when you sit down to make decisions from the data, you can’t. The metrics aren’t actionable.
Here’s the thing: a metric is only useful when you can use it.
Actionable metrics give you valuable insights that help you make informed business decisions. They’re the gold we mine for. Good metrics will let you know if your course has achieved the desired business objective.
Here are 3 questions to consider when choosing you course metrics:
1. What’s the the course objective?
The most important question you can ask here is:
What do you want your participants to do by the end of the course?
This give you your course objective. A clear objective gives you the framework to find the metrics that will prove whether or not your learners can in fact perform the desired objective. You will see that vanity metrics have a hard time proving the objective, that’s how you know you’re on the right track. Metrics that prove an objective often refer to being able to do something new, doing something faster, or doing something at better quality. The challenge is setting up the mechanisms to measure these.
2. What level of learning are you pitching at?
A course will fit into one of the following MUD levels:
- Memorising is the ability to store the information in their memory.
- Understanding is the ability to teach it or explain it.
- Doing is the ability to action it.
The question to ask is:
“How will I measure whether my participants have achieved the MUD level outcomes that I want?”
You’ll measure your outcomes through an assessment process. For instance, if you want participants to be able to do something by the end of the course, then they’ll have to demonstrate that they can perform the desired action by the end of the course.
3. How much work is your metric?
Some metrics take a lot of time and resources to gather effectively. Measuring how many employees took the course is easy to do. Measuring how well those employees can explain and train new employees is more difficult.
Automatic reports can go a long way in collecting quantitative data around completion and assessment results. But, it’s beneficial to have bigger evaluations that are more qualitative in nature, like team feedback sessions with groups of learners, or tracking the learning progression of an individual or a team.
Consider how you would go about measuring your metric. The questions to ask here are:
How much will it cost you to get that data? How reliable will it be? How labour-intensive will it be? Can you record it on your own or do you need expert help?
A good metric is worth the work
How you define success will determine how you measure it. When you’re setting up an e-learning course, it’s worth taking the time to consider both of those things. What do I want my participants to be able to do? And how am I going to measure that?
Finding actionable metrics might take some work initially, but when you are called on to make real business decisions from those metrics, you’ll be able to make informed decisions that may have a significant, positive impact on your learners, your team, or even your company as a whole.
The original Medium post can be found here