Why Are We Doing This? Asking the Right Questions of Your Learning Data

• Home
• Blog
• Learning & Development

Chris Dunne, Digital Marketing Manager

One day you decide, or your boss decides for you, that it’s time put your Learning Data to work.

Maybe it’s course data from your learning management system (LMS).  Or maybe you’re using, or thinking about using, a learning record store (LRS) such as Learning Locker.

No matter where your data is, you start with the basic question: What are we doing this for anyway?

Data-Driven Decision Making

When we collect and analyze data, we’re generally doing so to gain understanding in order to make better decisions about future actions.   Which means that even this most basic question is likely have a variety of answers, depending on who is asking it.

Maybe L&D wants to find out if the design changes to a course have made it easier for people to find the elements they need.  Or the HR department wants to know the outcomes from that sales training last quarter.  Perhaps the leadership team is asking about the participation and completions rates, and want to know if those rates have any correlation with meeting performance goals.

In one sense, most data analysis requests boil down to the same request:  “Did it work?”  But the answers to that question and the data we need to reach those answers are based on what the stakeholder is looking for.

Defining Your Data Needs

If I have a request for some participation data for a course, the first step would be to clarify how we want to define “participation”: is it logging in at least once week, accessing a certain percentage of course objects, completing the course?   Additionally, I would want to look into what decisions or actions would be based on this information; looking into the intentions behind the request makes it much more likely that I’ll collect a useful data set on the first pass.

In a broad sense, data needs can be defined by looking at two elements: the Scale and the Categories of information needs.

For scale our information needs are going to focus on one or more levels, for example, commonly I’m asked questions about the performance of Participants, Course Components, or the Course itself (although other scales are possible, obviously).

Participants

Data at the learner level can be used to examine learning paths, create individual recommendations based on similar users or based on performance (e.g. novice > expert mapping), or to create more meaningful branched learning or scenario based learning.

Course Components

Looking at course  context can provide insight the what’s working and what isn’t; we can get a sense of what provides value:

• What did people struggle with?
• What did people ignore?   Did it matter?
• How long will people pay attention to different media types?
• What generates interaction, conversation or reflection?
• Do different roles or regions respond differently to different types of activities?
• Were people who completed A measurably more successful at B?

This sort of data can provide valuable insight for future course design or modifications

Courses

Similarly we can look at course data in context.  With carefully collected data, we can look at much more than what percentage of those enrolled completed the course or passed their assessment.

We can look at how the course is performing: if everyone is struggling or there is a high proportion of non-completion, we might want to dig deeper, to find if there are places the course lack clarity or is too advanced or too simple, or actually not relevant or necessary. We might investigate if is there a tipping point where people either continue or quit?   If so, can we determine why?

And of course a key goal of any data analysis is to look for evidence of whether the course is having the desired effect.

What Questions Can We Ask?

Regardless of scale, the questions we want to answer will generally fall under one of two categories – Participation Related, or Performance Related.  We’ll now go on to look  at some common questions and some of the data we might explore to answer them.

So, What Are You Doing It For?

The key thing to remember is that learning data’s greatest power is the insight that it gives us about our courses and the way participants engage and learn from them.

There are of course many other questions that you may be looking at your data to provide answers to from those explored above; but it’s also important to remember that the specific insights you wish to gain will likely change over time.  Ensuring that you return to the ‘why are we doing this? question should ultimately result in you having courses that are designed as best as they can be: providing an engaging learning experience for your learners that delivers measurable change.

Is your LMS/LRS providing the data that you need?  To find out more about getting started with big data and learning analytics, get in touch – we’d love to hear from you.