A gated multi-scale attention framework for automated short answer grading in Turkish
Information Processing and Management, vol.63, no.7, 2026 (SCI-Expanded, SSCI, Scopus)
- Publication Type: Article / Article
- Volume: 63 Issue: 7
- Publication Date: 2026
- Doi Number: 10.1016/j.ipm.2026.104791
- Journal Name: Information Processing and Management
- Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Social Sciences Citation Index (SSCI), Scopus, Periodicals Index Online, ABI/INFORM, Aerospace Database, Applied Science & Technology Source, Communication Abstracts, Compendex, EBSCO Education Source, Education Abstracts, Information Science and Technology Abstracts, INSPEC, Library Literature and Information Science, Library, Information Science & Technology Abstracts (LISTA), MLA - Modern Language Association Database, zbMATH, Information Science & Technology Abstracts (LISTA), MLA International Bibliography, Social Science Premium Collection (ProQuest), Business Source Ultimate (EBSCO), Education Source Ultimate (EBSCO), Engineering Source (EBSCO), Library & Information Science Collection (ProQuest), Technology Collection (ProQuest)
- Keywords: Automatic short answer grading, BiLSTM, CNN, Gated multi-scale head attention, Transformer models
- Ondokuz Mayıs University Affiliated: Yes
Abstract
This study proposes a new deep learning architecture for automated short-answer grading (ASAG) in Turkish. The model integrates transformer-based encoders (BERTurk, RoBERTa-Turkish (RoBERTa), ELECTRA-Turkish (ELECTRA)) with a Gated Multi-Scale Head Attention (GMHA) layer. Additionally, a BiLSTM network and parallel CNN layers are employed to capture both global dependencies and local linguistic patterns.The model was evaluated on a newly created Turkish dataset containing 2200 student responses from 220 learners, each graded by four experts. Among the pretrained Turkish encoders, the BERT-based model with GMHA achieved the best performance (QWK = 0.839, Pearson = 0.856, RMSE = 0.918). The model was also compared with traditional machine learning approaches and large language models (LLMs) using the Leave-One-Question-Out (LOQO) validation protocol. Under this setting, it achieved QWK = 0.337, Pearson = 0.468, and RMSE = 1.213, outperforming all baselines.Ablation studies confirmed the effectiveness of the GMHA and parallel CNN components. These results highlight both the theoretical contribution of multi-scale gated attention and the practical potential of developing reliable automated grading systems for morphologically rich, low-resource languages like Turkish.