Measuring the value and return on investment (ROI) of training and development is consistently one of the most common challenges faced by businesses – perhaps even more so when it comes to Social Learning – and the C-Suite is rightly challenging the L&D department to prove its value to the business.
All too common is that Social Learning gets overlooked because L&D aren’t quite sure how they will show that their investment has been worthwhile and actually delivered the results that it set out to achieve. So, just how can you measure the value of online, social learning activities in your workplace…?
1. Measure Engagement
The most common report you will see about the success of a social learning program is how much was viewed/contributed/commented during the learning process. This isn’t a bad judgement; in terms of sheer numbers these figures can tell a story about how many people were reached by your learning experience.
Quite often we’re looking to create a spark of inspiration with Social Learning: to pass on a Tweet that makes people think; to hold a mentoring session that gets people reflecting on how they work; to share a video that gets people talking. And so measuring the initial impact of that moment of inspiration probably is about counting views or comments. Social Learning experiences benefit from network effects – the more people being social, the greater the value generated.
But be warned, the figures can be misleading. In one experiment where we required everyone to participate in a social learning experiences we generated more than 3,000 responses from just 30 learners over a 6-week period. But when we came to our next measure, quality, we found that more than 95% of those contributions weren’t really valuable in terms of adding to the learning on offer. That’s a lot of wasted engagement…
2. Measure Quality
Whilst measuring engagement is easy to do it is also easy to criticise. When you only measure engagement you only know that people talked, you don’t know what they were talking about. Whilst that might have sparked some ideas and further work, we’ve got no real idea that conversation equalled learning. Or do we..?
Tools like Google’s Natural Language API are starting to put the power of machine learning into our hands to analyse not just how much people use social tools, but what they are saying whilst they are doing it.
We’ve taken this approach to analyse the conversation in a range of social learning experiences. Working for InterContinental Hotels Group (IHG), we measured the intentional language of contributors; who was just talking, who had identified they could do something differently and who was actually trying new behaviours in the workplace and reflecting back the results:
Knowing what ‘quality’ actually represents in a learning opportunity isn’t straightforward. For an educational experience we might equate quality with critical thought. But for a workplace experience, we’re more likely to equate quality with a change in behaviour (e.g. someone has taken a piece of learning and changed behaviour as a result). How that mounts up to business results, is the holy grail.
3. Measure Change
Attributing changes in the business environment to a social learning experience is difficult because these experiences don’t happen in isolation. Seasonal variations, the fortunes of the market, the activities going on elsewhere in the business; there are a hundred reasons that a business’s bottom-line might change.
However, the case for Social Learning having made an impact on the bottom line is a lot greater when you follow the value chain back down through quality and engagement. If you have evidence that people were engaging on a topic, and you have further evidence that they suggested they were going to do something different – or indeed have done something different – you begin to make a compelling case for your social experience to have impacted the company.
Post-rationalising this is very hard. In order to measure change in the business we first of all have to establish a baseline from which to work. Villeroy & Boch (V&B) showed exactly this on their way to winning last years ‘Best Social & Collaborative Learning’ award from the Learning & Performance Institute.
By comparing like-for-like sales results in a variety of global regions and using an A/B approach (some regions took part in the social experience, others didn’t), V&B were able to demonstrate the impact of social learning on bottom-line performance, taking out seasonal variations in-demand:
Getting this data took discipline and planning. Social Learning experiences were heavily targeted towards specific strategic outcomes; this wasn’t a case of just letting people go nuts on a forum, but of encouraging staff discussion on particular content topics. And because V&B were also measuring the quantity and quality of the conversation as they went, the case for social learning having truly made a bottom-line difference was well and truly made.
Get The Tools to Make the Change Today
There are literally hundreds of technologies that can help you with your measurement activities. Something as simple as Google Analytics can do a huge amount of heavy lifting for engagement metrics with just a few clicks to install.
It is entirely possible to generate credible metrics for every stage of a social learning experience – so don’t let anyone tell you social isn’t measurable!
Download our free Learning Technology Manager’s Guide to xAPI to find out more about this new approach for tracking experience data in learning activities, and a key part in today’s L&D Toolkit.
Ben serves as CEO for Learning Pool LTD, with responsibility for the commercial, product and people functions based mostly in the UK, reporting to the Group CEO.
Previously, Ben served as Chief Product Officer for Learning Pool where he worked to help define and develop Learning Pool’s next generation of workplace digital learning platforms, with a focus on Learning Experience Platforms and the Learning Analytics space.
Before Learning Pool, Ben helped to build HT2 Labs from humble beginnings into a globally recognized innovator in workplace digital learning. Learning Pool completed an acquisition of HT2 Labs in June 2019.
Ben’s expertise is based in research, having previously completed his PhD researching the impact of gamification on adult social learning, Ben has authored and contributed chapters for many books, has two peer-reviewed academic papers and has presented at conferences around the world, including TEDx.
Get started by telling us what you need and one of our team will be in touch very soon.
+44 207 101 9383
US +1 857 284 1420
+44 345 074 4114*
US +1 844 238 5577
* call charges vary depending on your provider