Monte Carlo Simulation of Critical-Thinking Assessment Categories from University Presentation Records
DOI:
https://doi.org/10.57255/intellect.v5i2.1825Keywords:
Monte Carlo simulation, critical-thinking assessment, presentation records, , learning analytics, historical distributionAbstract
Critical-thinking performance is difficult to infer directly from presentation grades, but historical grade categories can be modelled as an aggregate distribution. This study applied a quantitative descriptive-analytical Monte Carlo approach to 474 eligible presentation records from a population of 537 Informatics and Computer Engineering Education students in the 2023 and 2024 cohorts at UIN Sjech M. Djamil Djambek Bukittinggi. Observed counts were 241 high, 207 moderate, and 26 low. The reported 474-draw simulation produced 241, 213, and 20, respectively. Absolute count differences were 0, 6, and 6, giving MAE = 4. The aggregate agreement calculated from the manuscript’s stated formula was 97.47%, while the previously reported 91.34% is the unweighted mean of category-specific agreements. Retrospective reconstruction of the sample pseudorandom sequence suggested a candidate linear congruential generator with a 127-value period; its actual parameters require confirmation. Because the same records informed both simulation probabilities and evaluation, these figures describe internal distributional agreement rather than accuracy on new cohorts. The study contributes a context-specific demonstration of category simulation and highlights the importance of instrument validity, generator transparency, and independent validation.
Abstrak
Kinerja berpikir kritis sulit disimpulkan secara langsung dari nilai presentasi, namun kategori nilai historis dapat dimodelkan sebagai distribusi agregat. Studi ini menerapkan pendekatan kuantitatif deskriptif-analitis berbasis metode Monte Carlo terhadap 474 catatan presentasi yang memenuhi syarat, yang diambil dari populasi 537 mahasiswa Pendidikan Teknik Informatika dan Komputer angkatan 2023 dan 2024 di UIN Sjech M. Djamil Djambek Bukittinggi. Hasil observasi menunjukkan 241 data berkategori tinggi, 207 sedang, dan 26 rendah. Simulasi dengan 474 kali pengambilan sampel menghasilkan masing-masing 241, 213, dan 20 data untuk kategori tersebut. Selisih jumlah absolut adalah 0, 6, dan 6, yang menghasilkan nilai MAE sebesar 4. Tingkat kesesuaian agregat yang dihitung berdasarkan rumus dalam naskah adalah 97,47%, sementara angka 91,34% yang dilaporkan sebelumnya merupakan rerata tidak berbobot dari tingkat kesesuaian untuk setiap kategori. Rekonstruksi retrospektif terhadap urutan semu acak (pseudorandom) dari sampel tersebut mengindikasikan penggunaan *linear congruential generator* dengan periode 127 nilai; parameter aktualnya masih memerlukan konfirmasi. Karena data yang sama digunakan baik untuk menentukan probabilitas simulasi maupun evaluasi, angka-angka ini lebih mencerminkan kesesuaian distribusi internal daripada akurasi untuk kelompok mahasiswa yang baru. Studi ini memberikan demonstrasi simulasi kategori dalam konteks tertentu serta menekankan pentingnya validitas instrumen, transparansi generator, dan validasi independen.
Downloads
References
[1] H. Hyytinen, M. Jämsä, T. Tuononen, and K. Kleemola, “A systematic-narrative review of performance-based assessments of critical thinking in higher education,” Assessment & Evaluation in Higher Educa-tion, vol. 50, no. 8, pp. 1293–1310, 2025, doi: 10.1080/02602938.2025.2553341.
[2] P. Andreucci-Annunziata et al., “Conceptualizations and instructional strategies on critical thinking in higher education: A systematic review of systematic reviews,” Frontiers in Education, vol. 8, art. no. 1141686, 2023, doi: 10.3389/feduc.2023.1141686.
[3] D. Van Damme, D. Zahner, O. Cortellini, T. Dawber, and K. Rotholz, “Assessing and developing critical-thinking skills in higher education,” European Journal of Education, vol. 58, no. 3, pp. 369–386, 2023, doi: 10.1111/ejed.12563.
[4] K. Kleemola, H. Hyytinen, and A. Toom, “Exploring internal structure of a performance-based critical thinking assessment for new students in higher education,” Assessment & Evaluation in Higher Educa-tion, vol. 47, no. 4, pp. 556–569, 2022, doi: 10.1080/02602938.2021.1946482.
[5] H. Hyytinen, K. Nissinen, K. Kleemola, J. Ursin, and A. Toom, “How do self-regulation and effort in test-taking contribute to undergraduate students’ critical thinking performance?,” Studies in Higher Ed-ucation, vol. 49, no. 1, pp. 192–205, 2024, doi: 10.1080/03075079.2023.2227207.
[6] H. Galindo-Domínguez, M.-J. Bezanilla, L. Campo, D. Fernández-Nogueira, and M. Poblete, “A teachers’ based approach to assessing the perception of critical thinking in Education university students based on their age and gender,” Frontiers in Education, vol. 8, art. no. 1127705, 2023, doi: 10.3389/feduc.2023.1127705.
[7] A. Ostendorf and E. Kaczk, “Critical thinking in the community of inquiry framework: An analysis of the theoretical model and cognitive presence coding schemes,” Computers & Education, vol. 193, art. no. 104662, 2023, doi: 10.1016/j.compedu.2022.104662.
[8] D. Vlachopoulos and A. Makri, “A systematic literature review on authentic assessment in higher educa-tion: Best practices for the development of 21st century skills, and policy considerations,” Studies in Edu-cational Evaluation, vol. 83, art. no. 101425, 2024, doi: 10.1016/j.stueduc.2024.101425.
[9] O. Jabali, B. Hamamra, and A. Ayyoub, “Critical thinking, assessment, and educational policy in Pales-tinian universities,” International Journal for Educational Integrity, vol. 20, art. no. 12, 2024, doi: 10.1007/s40979-024-00160-9.
[10] T. Li, Z. Zhan, Z. Lin, and X. Zhang, “Assessment of higher-order thinking in education—A systematic review of empirical studies,” Studies in Educational Evaluation, vol. 90, art. no. 101655, 2026, doi: 10.1016/j.stueduc.2026.101655.
[11] A. Stojanov and B. K. Daniel, “A decade of research into the application of big data and analytics in high-er education: A systematic review of the literature,” Education and Information Technologies, vol. 29, pp. 5807–5831, 2024, doi: 10.1007/s10639-023-12033-8.
[12] L. Paulsen and E. Lindsay, “Learning analytics dashboards are increasingly becoming about learning and not just analytics—A systematic review,” Education and Information Technologies, vol. 29, pp. 14279–14308, 2024, doi: 10.1007/s10639-023-12401-4.
[13] L. Márquez, V. Henríquez, H. Chevreux, E. Scheihing, and J. Guerra, “Adoption of learning analytics in higher education institutions: A systematic literature review,” British Journal of Educational Technolo-gy, vol. 55, no. 2, pp. 439–459, 2024, doi: 10.1111/bjet.13385.
[14] E. Drugova, I. Zhuravleva, U. Zakharova, and A. Latipov, “Learning analytics driven improvements in learning design in higher education: A systematic literature review,” Journal of Computer Assisted Learning, vol. 40, no. 2, pp. 510–524, 2024, doi: 10.1111/jcal.12894.
[15] M. Afzaal and J. Nouri, “A systematic review of software for learning analytics in higher education,” In-ternational Journal of Emerging Technologies in Learning, vol. 19, no. 7, pp. 17–43, 2024, doi: 10.3991/ijet.v19i07.50313.
[16] N. Bergdahl, M. Bond, J. Sjöberg, M. Dougherty, and E. Oxley, “Unpacking student engagement in higher education learning analytics: A systematic review,” International Journal of Educational Technology in Higher Education, vol. 21, art. no. 63, 2024, doi: 10.1186/s41239-024-00493-y.
[17] A. Palanci, R. M. Yılmaz, and Z. Turan, “Learning analytics in distance education: A systematic review study,” Education and Information Technologies, vol. 29, pp. 22629–22650, 2024, doi: 10.1007/s10639-024-12737-5.
[18] R. Kleimola, L. Hirsto, and H. Ruokamo, “Promoting higher education students’ self-regulated learning through learning analytics: A qualitative study,” Education and Information Technologies, vol. 30, pp. 4959–4986, 2025, doi: 10.1007/s10639-024-12978-4.
[19] P. G. de Barba, E. A. Oliveira, and N. English, “Development and validation of a learning analytics rubric for self-regulated learning,” Educational Technology Research and Development, vol. 73, pp. 3223–3245, 2025, doi: 10.1007/s11423-025-10521-x.
[20] K. Alalawi, R. Athauda, R. Chiong, and I. Renner, “Evaluating the student performance prediction and action framework through a learning analytics intervention study,” Education and Information Tech-nologies, vol. 30, pp. 2887–2916, 2025, doi: 10.1007/s10639-024-12923-5.
[21] K. Alalawi, R. Athauda, and R. Chiong, “An extended learning analytics framework integrating machine learning and pedagogical approaches for student performance prediction and intervention,” Interna-tional Journal of Artificial Intelligence in Education, vol. 35, pp. 1239–1287, 2025, doi: 10.1007/s40593-024-00429-7.
[22] A. Angeioplastis, J. Aliprantis, M. Konstantakis, and A. Tsimpiris, “Predicting student performance and enhancing learning outcomes: A data-driven approach using educational data mining techniques,” Computers, vol. 14, no. 3, art. no. 83, 2025, doi: 10.3390/computers14030083.
[23] Z. Alamgir, H. Akram, S. Karim, and A. Wali, “Enhancing student performance prediction via educa-tional data mining on academic data,” Informatics in Education, vol. 23, no. 1, pp. 1–24, 2024, doi: 10.15388/infedu.2024.04.
[24] D. Khairy et al., “Prediction of student exam performance using data mining classification algorithms,” Education and Information Technologies, vol. 29, pp. 21621–21645, 2024, doi: 10.1007/s10639-024-12619-w.
[25] S. Malik et al., “Advancing educational data mining for enhanced student performance prediction: A fu-sion of feature selection algorithms and classification techniques with dynamic feature ensemble evolu-tion,” Scientific Reports, vol. 15, art. no. 8738, 2025, doi: 10.1038/s41598-025-92324-x.
[26] V. Balachandar and K. Venkatesh, “A multi-dimensional student performance prediction model (MSPP): An advanced framework for accurate academic classification and analysis,” MethodsX, vol. 14, art. no. 103148, 2025, doi: 10.1016/j.mex.2024.103148.
[27] F. J. García-García, M. I. Gómez-Núñez, and C. Molla-Esparza, “Applications of learning analytics in the study of academic performance in higher education: A meta-review with an educational perspective,” Higher Education, published online June 2026, doi: 10.1007/s10734-026-01709-y.
[28] Q. Li, Y. Jung, and A. F. Wise, “How instructors use learning analytics: The pivotal role of pedagogy,” Journal of Computing in Higher Education, vol. 38, pp. 227–255, 2026, doi: 10.1007/s12528-025-09432-w.
[29] D. Šimić et al., “HELA-CMM: Capability maturity model for adoption of learning analytics in higher education,” International Journal of Educational Technology in Higher Education, vol. 22, art. no. 25, 2025, doi: 10.1186/s41239-025-00509-1.
[30] E. Caro, C. González, and J. M. Mira, “Student academic performance stochastic simulator based on the Monte Carlo method,” Computers & Education, vol. 76, pp. 42–54, 2014, doi: 10.1016/j.compedu.2014.03.007.
[31] A. Pritchard, D. Taylor, and M. Belford, “Teaching data-driven decision making for inventory analysis with Monte Carlo simulation,” Decision Sciences Journal of Innovative Education, vol. 23, art. no. e12328, 2025, doi: 10.1111/dsji.12328.
[32] M. Bieżuński, M. Dulek, M. Jessa, and Ł. Matuszewski, “Statistical quality of number sequences produced by pseudorandom number generators built into contemporary programming languages,” Applied Sci-ences, vol. 16, no. 20, art. no. 9979, 2026, doi: 10.3390/app16209979.
[33] X. Wu et al., “Pseudorandom number generators based on neural networks: A review,” Journal of King Saud University—Computer and Information Sciences, vol. 37, art. no. 18, 2025, doi: 10.1007/s44443-025-00007-4.
[34] L. G. Suhaidir, Sumijan, and Y. Yunus, “Prediksi tingkat pemahaman siswa terhadap data nominatif,” Jurnal Sistim Informasi dan Teknologi, vol. 2, 2020, doi: 10.37034/jsisfotek.v2i3.28.
[35] A. Y. Vladova and K. M. Borchyk, “Predictive analytics of student performance: Multi-method and code,” JRAMathEdu, vol. 9, no. 4, 2024, doi: 10.23917/jramathedu.v9i4.4643.
[36] Y. Qi, “A multi-dimensional prediction system for students’ academic performance driven by deep learning,” Discover Artificial Intelligence, vol. 6, art. no. 69, 2026, doi: 10.1007/s44163-025-00744-5.
[37] J. Gimenez et al., “Critical thinking on a 10-week EAP course: Teachers’ and international pre-university students’ views and experiences,” Journal of English for Academic Purposes, vol. 76, art. no. 101534, 2025, doi: 10.1016/j.jeap.2025.101534.
[38] C. Sellberg and A. Sharma, “Toward multimodal learning analytics in simulation-based collaborative learning: A design ethnography of maritime training,” International Journal of Computer-Supported Collaborative Learning, vol. 20, pp. 201–221, 2025, doi: 10.1007/s11412-024-09435-2.
[39] X. Yin and K. Dou, “An AI-assisted critical thinking intervention to enhance undergraduate EFL learners’ writing proficiency,” Studies in Educational Evaluation, vol. 86, art. no. 101480, 2025, doi: 10.1016/j.stueduc.2025.101480.
[40] B. Flanagan, R. Majumdar, and H. Ogata, “Early-warning prediction of student performance and engage-ment in open book assessment by reading behavior analysis,” International Journal of Educational Technology in Higher Education, vol. 19, 2022, doi: 10.1186/s41239-022-00348-4.
Downloads
Submitted
Accepted
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Diva Amanda Putri, Firdaus Annas
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
