C. Piech, M. Sahami, J. Huang, and L. Guibas, Autonomously Generating Hints by Inferring Problem Solving Policies, Learning at Scale 2 (2015), 195-204.


Exploring the whole sequence of steps a student takes to produce work, and the patterns that emerge from thousands of such sequences is fertile ground for a richer understanding of learning. In this paper we autonomously generate hints for the Code.org ‘Hour of Code,’ (which is to the best of our knowledge the largest online course to date) using historical student data. We first develop a family of algorithms that can predict the way an expert teacher would encourage a student to make forward progress. Such predictions can form the basis for effective hint generation systems. The algorithms are more accurate than current state-of-the-art methods at recreating expert suggestions, are easy to implement and scale well. We then show that the same framework which motivated the hint generating algorithms suggests a sequence-based statistic that can be measured for each learner. We discover that this statistic is highly predictive of a student’s future success.


 author = {Piech, Chris and Sahami, Mehran and Huang, Jonathan and Guibas, Leonidas},
 title = {Autonomously Generating Hints by Inferring Problem Solving Policies},
 booktitle = {Proceedings of the Second (2015) ACM Conference on Learning @ Scale},
 series = {L@S '15},
 year = {2015},
 isbn = {978-1-4503-3411-2},
 identifier = {pmhg-aghipss-15},
 location = {Vancouver, BC, Canada},
 pages = {195--204},
 numpages = {10},
 url = {http://doi.acm.org/10.1145/2724660.2724668},
 doi = {10.1145/2724660.2724668},
 acmid = {2724668},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {educational datamining., hint generation, problem solving policy},