TALE 2022: Predicting Guesses and Slips Through Question Encoding with Complexity Hints

My buddy from my master's and I started on this work in 2019. With the pace at machine learning research is moving nowadays, we were feeling a little bit not good about this anymore since some parts of this work already sound dated. In any case, we decided it is the takeaway that is important, hence we decided to push through submitting to the TALE conference. This is the third year I am joining TALE, and this time in person!

For a copy of relevant materials (e.g., presentation, paper) or any questions you may have, please feel free to reach out to me through the Contact Me gadget on this blog's sidebar.

Details

Title: Predicting Guesses and Slips Through Question Encoding with Complexity Hints

Authors: May Kristine Jonson Carlon*, Sangwhan Moon*, Naoaki Okazaki, Jeffrey S. Cross. (* Equal first author contribution)

Venue: IEEE International Conference on Engineering, Technology, and Education TALE 2022

Date: December 4 to 7, 2022

DOI: TBD

Abstract

One commonly researched adaptive learning method, knowledge tracing, supposes that a learner answering an exercise correctly does not immediately imply that they have mastered the associated knowledge component, and vice-versa for answering incorrectly. Alternative explanations for learner performance such as guesses and slips reduce the accuracy of assessment items in determining learning achievement. One possible contributor to the learner's tendency to make guesses or commit slips is the unnecessary complexity of a problem. In this paper, the potential usefulness of complexity hints in predicting mastery of algebra problems is explored. Natural language processing methods were used to derive a complexity estimate of each assessment problem and applied to estimate guess and slip probabilities using individualized neural network models. An added important advantage of complexity estimation over previous methods for knowledge tracing is its potential contribution to assessment design by revealing which problems tend to be more confounding than helpful to learner progress. This potentially may lead to new lines of research in natural language processing applications, knowledge tracing, and curriculum quality assurance.