The response from leaders was phenomenal; more than 30,000 responses (scoring 100,000 ‘experience points’) were generated in response to the curated material from IHG. More than 1,500 videos were added back to the platform, with colleagues in 42 countries contributing back to the platform.
“An hour after the MOOC, I was already faced with an opportunity to apply the SBI Model”.
Later analysis has shown clear progression of thought over the MOOC, with increasing numbers of learners committing to implementing changes in their behavior, and giving evidence of how they have tried new techniques in the workplace. In complete contrast to most MOOCs, leaders on IHG’s ‘Brilliant Conversations’ actually increased the amount of time they spent on the platform week-on-week.
“It was great opportunity to develop my ability on making Brilliant Conversation with my colleagues”.
Measuring the Value of Social with xAPI
IHG wanted to try and measure the actual impact on behavior of running a MOOC. Without evidence as to the quality of the learning intervention, it would be hard to assess the actual impact the MOOC had on the organization, beyond a range of ‘feel good’ engagement metrics. IHG felt the MOOC had been successful, but they lacked real hard evidence that it had made a difference to the organization.
Whilst the MOOC was going on, HT2 Labs was collecting evidence across systems using Learning Locker and the xAPI. With this source of data HT2 Labs’ data scientists were able to do something really clever: They were able to analyze conversations retrospectively to see how many people were actually doing something different in the workplace by the end of the 5-week experience. The results were ground-breaking. Using an approach to measure the quality of online conversation in a learning environment known as Cognitive Presence, HT2 Labs were able to map the submissions at the ‘end of week assignment’ to assess the quality of the reflective conversations occurring on a 3-point scale.
Within 4 weeks of the social experience starting, IHG could show that 50% of participants had seen how they could change and 12% of participants had actually tried a new behavior. A remarkable insight into the impact a social learning experience has had on individuals and, collectively, the organization as a whole.
At level 1 learners were simply expressing what they had watched, read or understood during the weeks learning activities.
At level 2 they were expressing a desire to change; that they might try to apply some of their new insights at some future point.
And at level 3 they were reporting back on how they tried to do something new, reflecting on how it went and on how they might do things next time as they try to form a new habit of better conversations.