Improving Construct Validity Yields Better Models of Systematic Inquiry, Even with Less Information

Abstract

Data-mined models often achieve good predictive power, but sometimes at the cost of interpretability. We investigate here if selecting features to increase a model’s construct validity and interpretability also can improve the model’s ability to predict the desired constructs. We do this by taking existing models and reducing the feature set to increase construct validity. We then compare the existing and new models on their predictive capabilities within a held-out test set in two ways. First, we analyze the models’ overall predictive performance. Second, we determine how much student interaction data is necessary to make accurate predictions. We find that these reduced models with higher construct validity not only achieve better agreement overall, but also achieve better prediction with less data. This work is conducted in the context of developing models to assess students’ inquiry skill at designing controlled experiments and testing stated hypotheses within a science inquiry microworld.