Creating effective learning is a continuous challenge for corporate businesses. The introduction of new technologies has eased this by opening up more ways for employees to take part in workplace training, such as e-learning platforms, video learning, and virtual reality. Formal training has vastly progressed over the years, but there is room for improvement through assessments, and that’s where Certainty Based Marking (CBM) can have a huge impact on training and development.
Multiple-choice assessments remain the most common and convenient method of evaluation; scoring is quick and simple, multiple versions can be created and simple metrics highlight who knows the subject matter and who doesn’t. However, these traditional metrics aren’t completely reliable. They fail to demonstrate how training impacts the employees’ performance, their confidence, or how effectively knowledge is retained over time. Despite the convenience of multiple-choice assessments in terms of distribution and the metrics it provides, they are problematic. Multiple-choice assessments make it easier for the learner to guess the answer, rather than demonstrate that they know specific information.
Multiple-choice also tests the learner’s ability to recall information, as opposed to showing that the learner understands the answer’s application in real-life situations. For example, a salesperson might be taught their organization’s sales processes and be able to identify them correctly in a multiple-choice question, but that doesn’t necessarily mean that they’ll be able to remember that information independently and put it into practice in the workplace. Assessments help organizations understand their employees’ knowledge levels. By tracking the correlation between performance and training results, organizations can clearly see the impact training has had on their workforce. Additionally, there is a third metric that can also contribute to how effective training is: employee confidence. We have found that measuring a learner’s confidence in their knowledge is not only a good way to test how solid their knowledge really is, but it also provides a strong indication of how well a learner can put that knowledge into practice.
What is Certainty Based Marking?
Certainty Based Marking (CBM) can be utilized to evaluate assessments. Not only must learners indicate what they think the correct answer is, they must also rate how certain they are of it (from ‘High’ to ‘Low’). For example, if a learner gets the answer right with high certainty, they are sufficiently knowledgeable. However, if a learner gets the answer wrong with high certainty, the unique scoring system alerts organizations to any ‘confidently incompetent’ employees. CBM was theorized by Tony Gardner-Medwin, a professor of physiology at University College London in 1994. According to him, past experiments show that asking learners to indicate their confidence levels stimulate a higher level of thinking, which assists learning and recall:
“When properly set up, it motivates students to think carefully about whether they can see reasons for either sound justification or uncertainty about the answers they give in tests. It is particularly helpful as a study aid in self-tests, providing a stimulus and challenge, and a warning signal when misconceptions have arisen. But it has also been clearly shown to improve the statistical reliability and validity of assessment data.” CBM enhances traditional assessment approaches by measuring both competence and confidence. It also avoids false indications of accuracy, as learners are unable to simply guess the right answer. To encourage honesty from learners, the CBM assessment scoring system does not penalize for incorrect answers with low confidence. Therefore, organizations can be more confident that they’re getting an accurate reflection of a learner’s knowledge. In addition, unsure but careful thinkers are not deflated by receiving a minus score – they are actually encouraged because more considered responses receive higher marks.
Using Certainty Based Marking with Assessments
Employee competence is invaluable to an organization, but only as long as it lasts for the long term, otherwise, the organization could face negative consequences. Employee evaluation and assessments are commonly implemented as a one-off event when in reality they should be implemented consistently in order to successfully track employee’s knowledge levels. Assessments also help inform employers’ training decisions as it highlights the subject areas employees are struggling with. This allows training to be refined and for relevant modules to be delivered in a cost-effective way. To do this, Wranx uses pre and post-assessments in conjunction with CBM.
A pre-assessment determines the learner’s existing knowledge to act as a benchmark. Spaced Repetition training is then delivered over a given period and upon conclusion, the learner is assessed again to determine the impact of training on their knowledge. This is particularly useful for those falling below the pass-rate, as their progression becomes more clear. On average, using the AMA approach, we’ve found that learners improve knowledge levels by 25 points from pre- to post-assessment. By adding CBM into the mix means we see increased confidence, with high-engagement learners improving by 38% and even low-engagement learners improving by 20%.
So, as opposed to making assessments a one-off event, they can be delivered at certain time periods to continuously refresh and assess employees’ knowledge. Assessments also help determine the type and frequency of training delivered for each individual on the Wranx platform. Using analytics with assessments Another advantage of using CBM assessments is that they safeguard organizations by exposing employee knowledge gaps that could potentially be harmful. Wranx’s performance analytics can visually demonstrate to organizations a learner’s accuracy against their confidence. We call this the ‘knowledge quadrant’ – this places each learner into a specific category, depending on the two variables: Aligned – Learners who are accurately answering and doing so with high certainty and will likely be applied correctly in practice. These individuals would be considered masters of that knowledge domain, resulting in higher performance and productivity.
Misaligned – Learners who are somewhat confident in their answers. The individual believes their answer to be correct, but an element of doubt exists that may cause them to not act on that knowledge. Uninformed – Learners who have gaps in their knowledge but are aware of it. They are uninformed and are unlikely to act. Misinformed – A ‘confidently incompetent’ employee. This individual has knowledge that they confidently believe to be correct, but which is actually incorrect. Those who have confidence in wrong information will very likely make mistakes on the job, which could potentially put the organization at risk. Being able to see what category employees fit into will help inform future training decisions for an organization. More focused, relevant learning content can be delivered and close any previous knowledge gaps employees may have – which results in more highly and broadly skilled workforces.
Success in the Workplace with Certainty Based Marking
While new methods of learning and extending knowledge retention are being developed all the time, knowledge assessments remain one of the key areas that can be improved. Naturally, it’s important to guarantee employees learn effectively, but the next step should be to assess how well they can apply their newfound knowledge in the workplace – both effectively and with confidence. Wranx offers a unique approach to employee assessment through its use of Certainty Based Marking. As explored above, this method requires those being assessed to indicate how certain they are of an answer before proceeding. Not only does this highlight the learner’s ability to recall specific information, but it also provides employers with an understanding of how likely their staff are to implement that knowledge in a workplace scenario. One of our clients who has successfully implemented CBM is The North Face®; they used this technique to assess their employees’ knowledge levels and see the measurable benefits of training with Wranx.
Following a month-long module, The North Face® saw 80% of their learners improve their CBM score, meaning they were answering more questions correctly and with greater confidence. Being a retail organization, The North Face® focus was primarily on delivering product knowledge training. With such a high volume of information on a wide range of products, it’s crucial that their employees are able to successfully recall key information when speaking with customers. Through the addition of CBM assessments, The North Face® gained a deeper insight into how confident their staff were and which areas needed improvement. In a retail environment, this confidence level can be the key factor in guaranteeing sales.
The Wranx Approach to CBM
For a learner to be truly confident in their knowledge, they must be able to use that knowledge in a real-life situation – where it counts most. There is empirical evidence to prove that effective training, which supports knowledge retention and boosts confidence, has a measurable impact on performance and can vastly improve an organization’s success in the long term. The inclusion of Certainty Based Marking assessments fits perfectly into a Wranx training program. By delivering a month-long module between a pre and post-assessment, employers are provided with detailed insight into their learners’ abilities to perform their duties correctly in the workplace. This is a powerful metric – one that helps to safeguard against unnecessary and easily avoidable mistakes. Most importantly, it demonstrates the true ROI of Wranx training.
Interested in learning more about Certainty Based Marking (CBM) and how it can enhance your training initiatives?
