An Epistemological Approach to Explainable Automated Assessment of Open Concept Map Propositions Using SHAP

Abstract

Concept mapping is widely recognized as an effective method for supporting meaningful learning and critical thinking because it allows teachers to assess students’ underlying knowledge structures. However, evaluating concept maps and providing feedback remain challenging, as these processes are time-consuming, increase teachers’ workload, and can reduce instructional efficiency. To address this issue, this study applies Transformer-based architectures, which rely on large-scale pre-training and task-specific fine-tuning, to develop an automated assessment system for concept maps. In addition, Explainable Artificial Intelligence (XAI) is integrated through the SHAP (SHapley Additive exPlanations) framework to generate interpretable explanations of the model’s scoring decisions. Using Transformer models such as BERT and DeBERTa, SHAP values are computed at the token level to show how individual words within each proposition contribute to the final score. The results indicate that tokens with positive SHAP values increase scores in line with correct rubric indicators, whereas negative values reduce them. Tokens that consistently show positive contributions in high-scoring outputs reflect stable and faithful model reasoning. Overall, the findings demonstrate that combining Transformer-based assessment with SHAP explanations improves epistemic transparency by aligning the model’s internal reasoning with expert evaluation criteria, thereby supporting more reliable, interpretable, and trustworthy automated feedback in concept mapping-based learning.