When Benjamin Bloom conducted his ‘2 Sigma’ study in 1984, he evidenced what many had suspected; people learn much more effectively in a personalized environment, such as one-to-one tutoring with an expert than in the classroom or in a group setting.
Ever since this study was published, trying to replicate the positive impacts of one-to-one tutoring has been a mission for many involved in education and organizational development, including here at Learning Pool, and a lot of progress has been made in terms of adaptive pathways and tailored content. But the real value in one-to-one tutoring is personalized feedback which has been really hard to replicate at scale – until now!
The new wave of artificial intelligence models – known as generative AI due to their ability to generate content, whether textual, voice, or visual – opens up a whole world of possibilities.
These AI Large Language Models are incredibly clever. For example, you could ask an LLM to write you a bedtime story personalized to your child. It could include their interests, it could be done in the style of their favorite author and it could even create some amazing illustrations to go alongside it. But there are also some challenges. Generative AI can’t be entirely trusted to always give quite the right response or even to be factually correct. So if you’re using it for learning or sharing information the output needs “to be carefully double-checked as the software seems to generate inaccurate content, based on inaccurately reported sources of ideas.” (International Journal of Information Management Volume 71, August 2023, 102642)
And this fact-checking is hard, if not impossible, to do at scale.
This is why the Learning Pool and Mind Tools teams decided to collaborate to look at how we can use the power of generative AI combined with highly trusted content to create unique, trustable, scalable and effective learning interactions that not only impact the learning experience but also improve learning outcomes.
And for our first foray into this, we decided to focus on feedback so the AI initially isn’t generating content or training but instead analyzing the inputs of the learners against a trusted framework.
Why is feedback so critical to learning?
Feedback is a critical component of the learning process, as it provides learners with valuable information about their performance, progress, and areas for improvement. Academic research has shown that effective feedback can enhance learning outcomes, motivate learners, and improve their self-regulation and metacognitive skills.
One study by Hattie and Timperley (2007) examined the effects of feedback on learning and found that task-specific feedback that focused on how to conduct the task more effectively had one of the largest effect sizes on student achievement. They identified three key elements of effective feedback:
- Clarity
- Specificity
- Goal orientation
The study also emphasized the importance of providing timely feedback, as learners tend to benefit more from feedback when it is received immediately after the task is completed.
From the self-regulation model and the research literature on formative assessment it is possible to identify some additional principles of good feedback practice.
- Promotes reflection
- Encourages dialogue
- Improves self-esteem
- Provides opportunities to close the skills gap
Additionally, research has shown that learners who receive feedback that is personalized and aligned with their individual goals and needs tend to be more motivated and engaged in the learning process (Nicol & Macfarlane-Dick, 2006).
Achieving this type of specific, timely and personalized feedback at scale has long been a challenge since it is so dependent on human interactions. It is exactly this type of feedback achievable at scale that we set out to provide when we designed Converse.
What is Converse?
Converse is a prototype that uses AI to allow managers to build their communication and management skills by practicing the types of difficult conversations they may need to have with people that report to them. They not only get feedback from the AI in the form of realistic, conversational responses, but they also receive detailed personalized feedback on how they performed down to the specifics of the exact words they used and how they met certain pre-set criteria.
