The report explores the use of Machine Learning to analyse higher-order thinking in student responses; with a goal of evaluating and improving Instructional Design.
In June 2017, UK based Teaching publication Tes released their ‘Growth Mindsets’ CPD course; designed to provide guidance to Teachers looking to implement a ‘Growth Mindset’ in the classroom. The practice encourages students to embrace challenges and practice resilience in order to achieve their learning goals.
In addition to discussion enabling social features, the course details over 120 practical strategies, activities and techniques covering both theory and classroom practice to help Teachers make the Growth Mindset a reality.
Method + Results
The study was performed using supervised text analysis, based on a training set of hand coded data, as well as by observing the presence [or absence] of keywords as deemed relevant in the assessment of higher order thinking.
The results indicate that the Multi Class Model did not perform as well as the Binary Model, which proved valuable in predicting higher and lower levels of thought. This is useful from an Instructional Design perspective as it makes it possible to efficiently examine the comment quality and, to explore the reasons behind the performance.
Predicted Comment Quality Graph
When analysing the data, Ingrid & Janet found that it is possible through Machine Learning to determine comment quality with reasonable accuracy. Ultimately, this provides the opportunity to accurately assess which objects in the course lead to proportionally more comments which exhibit higher order thinking.
The full report is available here.
