Diagram of the Community of Inquiry Framework
Earlier this year Learning Pool (formerly HT2 Labs) introduced the award-winning Social Intelligence Dashboard, SID, as part of its growing army of tools to help organizations all over the world make data-driven decisions about their learning practices.
SID has been inspired by the Community of Inquiry (CoI) framework to reveal insight about a learners progression through analyzing the text they contribute in a social learning environment. This work has become a key differentiator for Stream LXP (formerly Curatr), our Learning Experience Platform.
SID is predicated on Machine Learning techniques that attempt to label comments according to a marking schema that we invented (based on CoI). The idea is to give a percentage certainty that a new comment has been marked as being of high quality, in that it demonstrates an individual’s stage in the learning process. But whilst we were developing our algorithms we found ourselves constantly tweaking to eek out every last drop of accuracy possible. As you iterate towards the best solution and fine tune hyperparameters (human-defined settings, as opposed to the ‘black box’ machine learning stuff), you find yourself in a long and arduous process. And it was becoming clear from early customer feedback that a single generic classification algorithm would not suffice – we would need to build multiple models, deploying different marking schemas. Not everybody would want to assess ‘quality’ in the same way we do.
Enter xAI – our Model Manager
Our xAI Model Manager allows us to both edit existing models and new models quickly and reproducibly using a simple graphical interface, allowing us to load up a high performing model and quickly make changes that help adapt it to new data or to a new use case.
Screenshot of the xAI Model Manager
We can visualize exactly how a model was trained, and on what data, as well as earmark favourite models for production or sort models based on characteristics such as language or any other number of data or model specific attributes.
Using a model pipeline it is possible to feed filtered comments from one model into another or return multiple scores from multiple models reflecting a variety of criteria demonstrated in comments.
Using tools like Live inference we can visually see how the new model reacts to specific sentences and words, look at its training summary as a visualization and assess its accuracy and precision on unseen data with a comprehensive breakdown demonstrating its decisions against the human classification for each line of input text.
Live Inference Testing Tool
Our CSV manager allows easy import of CSVs from Learning Locker® or any CSV of comments to add to the Marking tools database while importing useful metadata and the ability to aggregate or subset data for different models.
This flexibility allows you, for example, to quickly identify your highest performing model for Language x trained on criteria list y.
CSV Import Tool
Learning Pool (formerly HT2 Labs) Marking Tool allows you to adjust descriptive criteria that comments are human marked on allowing the training data to adapt a model to your specific use case or corporate culture. This labelling can be shared via URL with as many markers as you need making supervised learning accessible and scalable.
The speed of human marking can be dramatically improved and performed internally maximizing GDPR compliance. The layout of an intuitive web page layout helps to imprint on your markers a strict set of marking criteria, helping them to mark comments correctly and enabling them to take breaks and mark at their own time to help avoid fatigue and its effect on your resulting model.
The features and tools provided by xAI allow you to train and create models with only a very high level understanding of the complexity behind machine learning and do so in a way that offers you reproducible plug and play functionality that makes training, deploying, comparing and editing models as easy as clicking a button, while offering you the world’s state of the art machine learning capabilities.
xAI represents a great leap forward in helping you connect with and understand your users, employees and learners…
Model Training Summary Diagram
Further cementing the analogy that when it comes to workplace learning…
Learning Pool (formerly HT2 Labs) is a place of firsts.