Comparing Learning Analytics-Supported and Traditional Mathematics Instruction across Higher Education
DOI:
https://doi.org/10.57255/jemast.v5i1.1862Keywords:
Learning analytics, technology-based teaching, mathematics achievement, traditional teaching, Afghanistan higher educationAbstract
Learning analytics (LA)-supported instruction has shown promise for improving student engagement, yet empirical evidence from infrastructure-constrained higher education systems remains limited. This study addresses that gap by pursuing two objectives: first, to compare mathematics achievement between students taught using LA-supported technology-based instruction and students taught using traditional lecture-based instruction at Samangan University, Afghanistan; and second, to examine the extent to which learning analytics engagement indicators login frequency, video time-on-task, quiz attempts, and forum participation predict students' mathematics achievement. A quasi-experimental, nonequivalent control-group, pretest–posttest design was employed with 96 second-year mathematics education students across two intact sections (experimental n = 48; control n = 48) over a 14-week term. Data were analyzed using independent-samples t-tests, analysis of covariance (ANCOVA), Pearson correlations, and multiple regression. Students in the LA-supported condition achieved significantly higher posttest scores (M = 78.6, SD = 7.4) than those in the traditional condition (M = 65.2, SD = 9.0), t(94) = 7.42, p < .001, d = 1.52, an advantage that remained significant after adjusting for pretest differences, F(1, 93) = 54.87, p < .001, partial η² = .37. Among the engagement indicators, video time-on-task (β = .34, p = .004) and quiz attempts (β = .27, p = .024) emerged as significant independent predictors, with the full set of indicators jointly accounting for 46% of the variance in posttest scores (adjusted R² = .41). Although the nonrandomized design limits causal inference, these findings extend empirical evidence on LA-supported mathematics instruction to a resource-constrained, fragile-state higher education context and indicate that behaviorally substantive engagement is a more informative predictor of achievement than platform access alone.
References
Abou-Khalil, V., Helou, S., Khalifé, E., Chen, M. A., Majumdar, R., & Ogata, H. (2021). Emergency online learning in low-resource settings: Effective student engagement strategies. Education Sciences, 11(1), 24.
Alam, M. I., Malone, L., Nadolny, L., Brown, M., & Cervato, C. (2023). Investigating the impact of a gamified learning analytics dashboard: Student experiences and academic achievement. Journal of Computer Assisted Learning, 39(5), 1436–1449.
Al-Azawei, A., Parslow, P., & Lundqvist, K. (2017). Investigating the effect of learning styles in a blended e-learning system: An extension of the technology acceptance model (TAM). Australasian Journal of Educational Technology, 33(2), 1–23.
Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In J. A. Larusson & B. White (Eds.), Learning analytics: From research to practice (pp. 61–75). Springer.
Bond, M., Zawacki-Richter, O., & Nichols, M. (2019). Revisiting five decades of educational technology research: A content and authorship analysis of the British Journal of Educational Technology. British Journal of Educational Technology, 50(1), 12–63.
Bray, A., & Tangney, B. (2017). Technology usage in mathematics education research: A systematic review of recent trends. Computers & Education, 114, 255–273.
Chen, L., Geng, X., Lu, M., Shimada, A., & Yamada, M. (2023). How students use learning analytics dashboards in higher education: A learning performance perspective. Sage Open, 13(3).
Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K–12 classrooms: A meta-analysis. Educational Research Review, 9, 88–113.
Clark, R. C., & Mayer, R. E. (2016). E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning (4th ed.). Wiley.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Rand McNally.
Corrin, L., & de Barba, P. (2014). Exploring students' interpretation of feedback delivered through learning analytics dashboards. In B. Hegarty, J. McDonald, & S.-K. Loke (Eds.), Rhetoric and reality: Critical perspectives on educational technology. Proceedings of ascilite 2014 (pp. 629–633).
Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of Research on Technology in Education, 42(3), 255–284.
Fishman, B., Riconscente, M., Snider, R., Tsai, T., & Plass, J. (2014). Empowering educators: Supporting student progress in the classroom with digital games. University of Michigan.
Gasevic, D., Dawson, S., & Siemens, G. (2015). Let's not forget: Learning analytics are about learning. TechTrends, 59(1), 64–71.
Heikkinen, S., Saqr, M., Malmberg, J., & Tedre, M. (2023). Supporting self-regulated learning with learning analytics interventions – a systematic literature review. Education and Information Technologies, 28(3), 3059–3088.
Higgins, S., Xiao, Z., & Katsipataki, M. (2012). The impact of digital technology on learning: A summary for the Education Endowment Foundation. Durham University.
Ifenthaler, D., & Yau, J. Y.-K. (2020). Utilising learning analytics to support study success in higher education: A systematic review. Educational Technology Research and Development, 68(4), 1961–1990.
Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017). Awareness is not enough: Pitfalls of learning analytics dashboards in the educational practice. In É. Lavoué, H. Drachsler, K. Verbert, J. Broisin, & M. Pérez-Sanagustín (Eds.), Data driven approaches in digital education (pp. 82–96). Springer.
Kaliisa, R., Jivet, I., & Prinsloo, P. (2023). A checklist to guide the planning, designing, implementation, and evaluation of learning analytics dashboards. International Journal of Educational Technology in Higher Education, 20(1), 28.
Kaliisa, R., Misiejuk, K., López-Pernas, S., Khalil, M., & Saqr, M. (2024). Have learning analytics dashboards lived up to the hype? A systematic review of impact on students' achievement, motivation, participation and attitude. In Proceedings of the 14th Learning Analytics and Knowledge Conference (LAK24) (pp. 295–304). ACM.
Katawazai, R. (2021). Implementing outcome-based education and student-centered learning in Afghan public universities: The current practices and challenges. Heliyon, 7(5), e07076.
Kebritchi, M., Hirumi, A., & Bai, H. (2010). The effects of modern mathematics computer games on mathematics achievement and class motivation. Computers & Education, 55(2), 427–443.
Khalil, M., & Ebner, M. (2016). Learning analytics: Principles and constraints. In Proceedings of EdMedia + Innovate Learning 2016 (pp. 1789–1799). Association for the Advancement of Computing in Education.
Li, Q., & Ma, X. (2010). A meta-analysis of the effects of computer technology on school students' mathematics learning. Educational Psychology Review, 22(3), 215–243.
Long, P., & Siemens, G. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5), 30–32.
Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2010). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. U.S. Department of Education, Office of Planning, Evaluation, and Policy Development.
Moon, J., Yeo, S., Banihashem, S. K., & Noroozi, O. (2024). Using multimodal learning analytics as a formative assessment tool: Exploring collaborative dynamics in mathematics teacher education. Journal of Computer Assisted Learning, 40(6), 2753–2771.
Musawi, S. Z., & Baktash, J. A. (2021). Identification and ranking of cloud-based applications in e-learning of Afghanistan: A case of public universities. Elsya: Journal of English Language Studies, 3(2), 78–85.
Noori, A. Q., Orfan, S. N., Akramy, S. A., & Hashemi, A. (2022). The use of social media in EFL learning and teaching in higher education of Afghanistan. Cogent Social Sciences, 8(1), Article 2027613.
Oryakhail, M. S., Saay, S., & Nasery, H. (2021). Challenges in the implementation of e-learning in Afghanistan higher education. In 2021 International Conference on Advancement in Data Science, E-Learning and Information Systems (ICADEIS) (pp. 1–6). IEEE.
Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64.
Rahim, M. N., & Sandaran, S. C. (2020). EFL teachers' perceptions of the barriers and opportunities for implementing eLearning at Afghanistan universities. Universal Journal of Educational Research, 8(11C), 97–104.
Rienties, B., & Toetenel, L. (2016). The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules. Computers in Human Behavior, 60, 333–341.
Roll, I., & Winne, P. H. (2015). Understanding, evaluating, and supporting self-regulated learning using learning analytics. Journal of Learning Analytics, 2(1), 7–12.
Romero, C., & Ventura, S. (2020). Educational data mining and learning analytics: An updated survey. WIREs Data Mining and Knowledge Discovery, 10(3), Article e1355.
Sclater, N. (2017). Learning analytics explained. Routledge.
Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2020). Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior, 107, Article 105512.
Selwyn, N. (2016). Is technology good for education? Polity Press.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.
Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400.
Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47, 157–167.
Tondeur, J., van Braak, J., Ertmer, P. A., & Ottenbreit-Leftwich, A. (2017). Understanding the relationship between teachers' pedagogical beliefs and technology use in education: A systematic review of qualitative evidence. Educational Technology Research and Development, 65(3), 555–575.
Trucano, M. (2005). Knowledge maps: ICT in education. infoDev / World Bank.
Trust, T., & Whalen, J. (2020). Should teachers be trained in emergency remote teaching? Lessons learned from the COVID-19 pandemic. Journal of Technology and Teacher Education, 28(2), 189–199.
Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1500–1509.
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110.
Voogt, J., & Roblin, N. P. (2012). A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies, 44(3), 299–321.
West, D., Huijser, H., & Heath, D. (2016). Putting an ethical lens on learning analytics. Educational Technology Research and Development, 64(5), 903–922.
Wise, A. F., & Shaffer, D. W. (2015). Why theory matters more than ever in the age of big data. Journal of Learning Analytics, 2(2), 5–13.
Downloads
Published
How to Cite
Issue
Section
Citation Check
License
Copyright (c) 2026 Abdul Basir Deljoy, Musawer Hakimi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
