Monitoring of Student Learning in Learning Management Systems: An Application of Educational Data Mining Techniques
Abstract
:Featured Application
Abstract
1. Introduction
1.1. Learning Analytics Procedures
1.2. Educational Data Mining (EDM) Procedures
- Apply an external logs analysis tool in Moodle to detect students at risk over the course of a semester in different phases (initial, intermediate, and final);
- Detect in the sample, with an external log analysis tool in Moodle, the clusters in different phases (initial, intermediate, and final) differentiating by type of algorithm (k-means ++, fuzzy k-means, DBSCAN);
- Check if there were differences in the clusters found depending on the type of algorithm (k-means ++, fuzzy k-means, DBSCAN);
- To check students’ satisfaction with the teaching process and the monitoring of learning.
- RQ1:
- There will be different activity clusters in the Moodle platform depending on the log collection phases (initial, intermediate, and final);
- RQ2:
- There will be differences in the groupings obtained in the clusters in the log collection phases (initial, intermediate, and final) depending on the algorithm applied (k-means ++, fuzzy k-means, DBSCAN);
- RQ3:
- There will be differences in the clustering obtained in the log collection phases (initial, intermediate, and final) depending on the applied algorithm (k-means ++, fuzzy k-means, DBSCAN) providing a better fit in the DBSCAN algorithm;
- RQ4:
- Students will perceive the monitoring of their learning performed with the UBUMonitor application as reflected in high levels of satisfaction in the Questionnaire of Student Opinion on Quality of Teaching (QSOQT).
2. Materials and Methods
2.1. Participants
2.2. Instruments
2.2.1. UBUMonitor Tool
2.2.2. Teaching Methodology
2.2.3. Questionnaire of Student Opinion on Quality of Teaching—QSOQT—by Bol, Sáiz and Pérez-Mateos (2012)
2.3. Procedure
2.4. Statistical Analysis
3. Results
4. Discussion
5. Conclusions
6. Patents
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Acknowledgments
Conflicts of Interest
References
- Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; Ochoa-Orihuel, J. Effectiveness of using voice assistants in learning: A study at the time of covid-19. Int. J. Environ. Res. Public Health 2020, 17, 5618. [Google Scholar] [CrossRef] [PubMed]
- Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; García-Osorio, C.I. Monitoring students at the university: Design and application of a moodle plugin. Appl. Sci. 2020, 10, 3469. [Google Scholar] [CrossRef]
- Carbonero, M.Á.; Martín-Antón, L.J.; Flores, V.; Freitas, A.R. Estudio comparado de los estilos de enseñanza del profesorado universitario de ciencias sociales en España y Brasil [Comparative study of the teaching styles of university social science teachers in Spain and Brazil]. Rev. Complut. Educ. 2017, 28, 631–647. [Google Scholar] [CrossRef] [Green Version]
- Cantabella, M.; Martínez-España, R.; Ayuso, B.; Yáñez, J.A.; Muñoz, A. Analysis of student behavior in learning management systems through a Big Data framework. Futur. Gener. Comput. Syst. 2019, 90, 262–272. [Google Scholar] [CrossRef]
- Slater, S.; Joksimović, S.; Kovanovic, V.; Baker, R.S.; Gasevic, D. Tools for Educational Data Mining: A Review. J. Educ. Behav. Stat. 2017, 42, 85–106. [Google Scholar] [CrossRef]
- Luna, J.M.; Castro, C.; Romero, C. MDM tool: A data mining framework integrated into Moodle. Comput. Appl. Eng. Educ. 2017, 25, 90–102. [Google Scholar] [CrossRef]
- Ventura, S.; Luna, J.M. Supervised Descriptive Pattern Mining. Supervised Descriptive Pattern Mining, 1st ed.; Springer: Cham, Switzerland, 2018. [Google Scholar] [CrossRef]
- Ahmed, S.A.; Khan, S.I. A Machine Learning Approach to Predict the Engineering Students at Risk of Dropout and Factors Behind: Bangladesh Perspective. In Proceedings of the 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kanpur, India, 6–8 July 2019; pp. 1–6. [Google Scholar] [CrossRef]
- Romero, C.; Ventura, S. Educational data mining: A review of the state of the art. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 2010, 40, 601–618. [Google Scholar] [CrossRef]
- Angeli, C.; Howard, S.K.; Ma, J.; Yang, J.; Kirschner, P.A. Data mining in educational technology classroom research: Can it make a contribution? Comput. Educ. 2017, 113, 226–242. [Google Scholar] [CrossRef] [Green Version]
- Sáiz-Manzanares, M.C.; Osorio, C.I.G.; Díez-Pastor, J.F.; Antón, L.J.M. Will personalized e-Learning increase deep learning in higher education? Inf. Discov. Deliv. 2019, 47, 53–63. [Google Scholar] [CrossRef]
- Jommanop, T.; Mekruksavanich, S. e-Learning Recommendation Model Based on Multiple Intelligence. In Proceedings of the 14th International Joint Symposium on Artificial Intelligence and Natural Language Processing iSAI-NLP, Chiang Mai, Thailand, 30 October–1 November 2019; pp. 1–6. [Google Scholar] [CrossRef]
- Benner, J.; McArthur, J.J. Data-driven design as a vehicle for BIM and sustainability education. Buildings 2019, 9, 103. [Google Scholar] [CrossRef] [Green Version]
- Li, X.; Jiang, Z.; Guan, Y.; Li, G.; Wang, F. Fostering the transfer of empirical engineering knowledge under technological paradigm shift: An experimental study in conceptual design. Adv. Eng. Inform. 2019, 41, 100927. [Google Scholar] [CrossRef]
- Sáiz-Manzanares, M.C.; Báez Sánchez, M.Á.; Ortega-López, V.; Manso-Villalaín, J.M. Self-Regulation and Rubrics Assessment in Structural Engineering Subjects. Educ. Res. Int. 2015, 2015, 340521. [Google Scholar] [CrossRef] [Green Version]
- Sáiz-Manzanares, M.C.; García-Osorio, C.I.; Díez-Pastor, J.F. Differential efficacy of the resources used in B-learning environments. Psicothema 2019, 31, 170–178. [Google Scholar] [CrossRef] [PubMed]
- Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; García-Osorio, C.I.; Díez-Pastor, J.F. How do B-learning and learning patterns influence learning outcomes? Front. Psychol. 2017, 8, 1–13. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Schneider, K.; Berens, J.; Burghoff, J. Early detection of student dropout: What is relevant information? J. Educ. 2019, 22, 1121–1146. [Google Scholar] [CrossRef]
- Kosheleva, O.; Villaverde, K. Studies in Computational Intelligence 750 How Interval and Fuzzy Techniques Can Improve Teaching Processing Educational Data: From Traditional Statistical Techniques to an Appropriate Combination of Probabilistic, Interval, and Fuzzy Approaches, 1st ed.; Springer: El Paso, TX, USA, 2018. [Google Scholar] [CrossRef]
- Cerezo, R.; Sánchez-Santillán, M.; Paule-Ruiz, M.P.; Núñez, J.C. Students’ LMS interaction patterns and their relationship with achievement: A case study in higher education. Comput. Educ. 2016, 96, 42–54. [Google Scholar] [CrossRef]
- Dimić, G.; Rančić, D.; Pronić-Rančić, O.; Spalević, P. Descriptive Statistical Analysis in the Process of Educational Data Mining. In Proceedings of the 14th International Conference on Advanced Technologies, Systems and Services in Telecommunications TELSIKS, Nis, Serbia, 23–25 October 2019; pp. 388–391. [Google Scholar] [CrossRef]
- Ilieva, G.; Yankova, T. Early multi-criteria detection of students at risk of failure. TEM J. 2020, 9, 344–350. [Google Scholar] [CrossRef]
- Fakhrusy, M.R.; Widyani, Y. Moodle Plugins for Quiz Generation Using Genetic Algorithm. In Proceedings of the International Conference on Data and Software Engineering (ICoDSE), Palembang, Indonesia, 1–2 November 2017; pp. 1–6. [Google Scholar] [CrossRef]
- Brito, M.; Medeiros, F.; Bezerra, E. A Report-Type Plugin to Indicate Dropout Risk in the Virtual Learning Environment Moodle. In Proceedings of the 19th International Conference on Advanced Learning Technologies (ICALT), Macei, Brazil, 15–18 July 2019; pp. 127–128. [Google Scholar] [CrossRef]
- Peramunugamage, A.; Usoof, H.; Hapuarachchi, J. Moodle mobile plugin for problem-based learning (PBL) in engineering education. In Proceedings of the 2019 IEEE Global Engineering Education Conference (EDUCON), Dubai, United Arab Emirates, 8–11 April 2019; IEEE: Dubai, United Arab Emirates, 2019; pp. 827–835. [Google Scholar] [CrossRef]
- Kadoić, N.; Oreški, O. Analysis of Student Behavior and Success Based on Logs in Moodle. In Proceedings of the 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croacia, 21–25 May 2018; pp. 654–659. [Google Scholar] [CrossRef]
- Badea, G.; Popescu, E.; Sterbini, A.; Temperini, M. Integrating Enhanced Peer Assessment Features in Moodle Learning Management System. In Foundations and Trends in Smart Learning, Proceedings of the International Conference on Smart Learning Environments; Lecture Notes in Educational, Technology; Denton, TX, USA, 18–20 March 2019, Chang, M., Popescu, E., Kinshuk, C.N., Jemni, M., Huang, R., Spector, J.M., Sampson, D.G., Eds.; Springer: Singapore, 2019; pp. 135–144. [Google Scholar] [CrossRef]
- Reynaldo, V.; Wicaksana, A.; Hansun, S. Brotli data compression on moodle-based E-learning server. ICIC Int. Express Lett. Part B Appl. 2019, 10. [Google Scholar] [CrossRef]
- Gueye, A.D.; Faye, P.M.D.; Gueye, B.; Lishou, C. Scheduling Synchronous Tutoring Sessions in Learning Activities. In The Challenges of the Digital Transformation in Education. ICL 2018. Advances in Intelligent Systems and Computing; Auer, M., Tsiatsos, T., Eds.; Springer: Cham, Switzerland, 2018; Volume 916, pp. 344–352. [Google Scholar] [CrossRef]
- Badea, G.; Popescu, E.; Sterbini, A.; Temperini, M. Exploring the Peer Assessment Process Supported by the Enhanced Moodle Workshop in a Computer Programming Course. In Proceedings of the 9th International Conference, Workshops (MIS4TEL 2019), Advances in Intelligent Systems and Computing, L’Aquila, Italy, 17–19 June 2019; Popescu, E., Gil, A.B., Lancia, L., Sica, L.S., Mavroudi, A., Eds.; Springer: Cham, Switzerland; pp. 124–131. [Google Scholar] [CrossRef]
- Dobudko, T.V.; Ochepovsky, A.V.; Gorbatov, S.V.; Hashim, W.; Maseleno, A. Functional monitoring and control in electronic information and educational environment. Int. J. Recent Technol. Eng. 2019, 8, 1383–1386. [Google Scholar] [CrossRef]
- Romero, C.; Ventura, S.; García, E. Data mining in course management systems: Moodle case study and tutorial. Comput. Educ. 2018, 51, 368–384. [Google Scholar] [CrossRef]
- Hooshyar, D.; Yang, Y.; Pedaste, M.; Huang, Y.M. Clustering Algorithms in an Educational Context: An Automatic Comparative Approach. IEEE Access 2020, 8, 146994–147014. [Google Scholar] [CrossRef]
- Dobashi, K.; Ho, C.P.; Fulford, C.P.; Lin, M.F.G. A Heat Map Generation to Visualize Engagement in Classes Using Moodle Learning Logs. In Proceedings of the Name of the 2019 4th International Conference on Information Technology (InCIT), Bangkok, Thailand, 24–25 October 2019; pp. 138–143. [Google Scholar] [CrossRef]
- Ji, Y.P.; Marticorena-Sánchez, R.; Pardo-Aguilar, C. UBU Monitor: Monitoring of Students on the Moodle Platform. 2018. Available online: https://github.com/yjx0003/UBUMonitor (accessed on 24 December 2020).
- Félix, I.M.; Ambrósio, A.P.; Neves, P.S.; Siqueira, J.; Brancher, J.D. Moodle Predicta: A Data Mining Tool for Student Follow Up. In Proceedings of the 9th International Conference on Computer Supported Education (CSEDU), Porto, Portugal, 21–23 April 2017; pp. 339–346. [Google Scholar] [CrossRef]
- Saqr, M.; Fors, U.; Tedre, M. How learning analytics can early predict under-achieving students in a blended medical education course. Med. Teach. 2017, 39, 757–767. [Google Scholar] [CrossRef] [PubMed]
- García, S.; Luengo, J.; Herrera, J. Data Preprocessing in Data Mining. In Intelligent Systems Reference Library; Springer: New York, NY, USA, 2015; Volume 72. [Google Scholar] [CrossRef]
- Arthur, D.; Vassilvitskii, S. k-means ++: The Advantages of Careful Seeding. In Proceedings of the Eighteenth annual ACM-SIAM Symposium on Discrete Algorithms 2006, Miami, FL, USA, 22–26 January 2006; pp. 1027–1035. [Google Scholar]
- Fernández, S.P. Unidad de Epidemiología Clínica y Bioestadística. Complexo Hospitalario Universitario de A Coruña [Clinical Epidemiology and Biostatistics Unit University Hospital Complex of A Coruña]. Cad. Aten. Primaria 1993, 3, 138–141. Available online: https://www.fisterra.com/formacion/metodologia-investigacion/determinacion-tamano-muestral (accessed on 5 January 2021).
- Schneier, B. Description of a New Variable-Length Key, 64-bit Block Cipher (Blowfish). In Fast Software Encryption; Anderson, R., Ed.; Springer: Berlin/Heidelberg, Germany, 1993; Volume 809, pp. 191–204. [Google Scholar] [CrossRef] [Green Version]
- The Apache Software Foundation. No TitlMath-Commons Math: The Apache Commons Mathematics Librarye. 2016. Available online: https://commons.apache.org/proper/commons-math/ (accessed on 15 March 2021).
- Li, H. Smile Statistical Machine Intelligence and Learning Engine. 2020. Available online: https://haifengl.github.io/ (accessed on 27 December 2020).
- Sáiz-Manzanares, M.C. E-Project Based Learning en Terapia Ocupacional: Una Aplicación en la Asignatura “Estimulación Temprana” [E-Project Based Learning in Occupational Therapy: An Application in the Subject “Early Stimulation”], 1st ed.; Servicio de Publicaciones de la Universidad de Burgos: Burgos, Spain, 2018. [Google Scholar]
- Bol-Arreba, A.; Sáiz-Manzanares, M.C.; Pérez-Mateos, M. Validation of Test Teaching Activity in Higher Education Open Classroom. Open Classr. 2013, 41, 45–54. Available online: https://cutt.ly/zjiSwLN (accessed on 2 January 2021).
- Marsh, H.W. Students’ Evaluations of University Teaching: Dimensionality, Reliability, Validity, Potential Biases and Usefulness. In The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective; Perry, R.P., Smart, J.C., Eds.; Springer: Dordrecht, The Netherlands, 2007; pp. 319–384. [Google Scholar] [CrossRef]
- University of Burgos. Student Opinion Survey on the Quality of Teaching (QSOQT). 2012. Available online: https://www.ubu.es/sites/default/files/portal_page/files/encuesta_revisada_v2014.pdf (accessed on 29 December 2020).
- IBM Corporation. SPSS Statistical Package for the Social Sciences (SPSS); Version 24; IBM: Madrid, Spain, 2016. [Google Scholar]
- IBM Corporation. AMOS SPSS Statistical Package for the Social Sciences (SPSS); Version 24; IBM: Madrid, Spain, 2016. [Google Scholar]
- Atlas.ti. Software Package Qualitative Data Analysis; Version 8; Atlas.ti Scientific Software Development; GmbH: Berlin, Germany, 2020; Available online: https://atlasti.com/es/ (accessed on 31 December 2020).
- Bandalos, D.L.; Finney, S.J. Item parceling issues in structural equation modeling. In New Development and Techniques in Structural Equation Modeling; Marcoulides, G.A., Schumacker, R.E., Eds.; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 2001; pp. 269–296. [Google Scholar]
n (Students) | Students Gender | |||||||
---|---|---|---|---|---|---|---|---|
Women | Men | |||||||
n | % | Mage | SDage | n | % | Mage | SDage | |
49 | 41 | 83.67 | 22.37 | 2.19 | 8 | 16.32 | 21.63 | 1.77 |
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | |
---|---|---|---|---|---|---|---|---|---|
1 | 1 | 0.14 | 0.30 | 0.45 | 0.14 | 0.22 | 0.008 | −0.13 | −0.02 |
2 | 0.14 | 1 | 0.17 | 0.09 | 0.34 | 0.12 | −0.05 | −0.18 | 0.02 |
3 | 0.30 | 0.17 | 1 | 0.09 | 0.03 | 0.62 | −0.04 | −0.12 | −0.03 |
4 | 0.45 | 0.09 | 0.09 | 1 | 0.14 | 0.13 | 0.28 | −0.09 | −0.03 |
5 | 0.14 | 0.34 | 0.03 | 0.14 | 1 | 0.09 | 0.02 | −0.06 | 0.02 |
6 | 0.22 | 0.12 | 0.62 | 0.13 | 0.09 | 1 | −0.04 | −0.11 | 0.05 |
7 | 0.008 | −0.05 | −0.04 | 0.28 | 0.02 | −0.04 | 1 | −0.03 | 0.11 |
8 | −0.13 | −0.18 | −0.12 | −0.09 | −0.06 | −0.11 | −0.03 | 1 | 0.04 |
9 | −0.02 | 0.02 | −0.03 | −0.03 | 0.02 | 0.05 | 0.11 | 0.04 | 1 |
Goodness-of-Fit Index | k-Means ++ | Fuzzy k-Means | DBSCAN | Accepted Value |
---|---|---|---|---|
df | 24 | 24 | 24 | |
χ2 | 102.47 * | 96.750 * | 91.588 * | * p > 0.05 α = 0.05 |
RMSEA | 0.261 | 0.251 | 0.242 | >0.05–0.08 |
RMSEA Interval (90%) | 0.210–0.341 | 0.200–0.305 | 0.191–0.296 | |
SRMR | 0.196 | 0.205 | 0.00 | >0.05–0.08 |
TLI | 0.704 | 0.628 | 0.704 | 0.85–0.90< |
IFC | 0.803 | 0.752 | 0.803 | 0.95–0.97< |
AIC | 162.47 | 156.750 | 151.588 | The lowest value |
ECVI | 3.385 | 3.266 | 3.158 | The lowest value |
ECVI interval (90%) | 2.810–4.117 | 2.712–3.977 | 2.624–3.850 | The lowest value |
Cluster | Measurement | Mean | SD | Max | Min | MRange | χ2 | gl | p |
---|---|---|---|---|---|---|---|---|---|
Initial | |||||||||
k-means ++ | 13.00 | 13.46 | 2 | 28 | 1.83 | 0.55 | 2 | 0.76 | |
Fuzzy k-means | 9.45 | 9 | 27 | 2.33 | |||||
DBSCAN | 12.67 | 17.62 | 2 | 33 | 1.83 | ||||
Intermediate | |||||||||
k-means ++ | 16.33 | 17.21 | 4 | 36 | 2.17 | 0.55 | 2 | 0.76 | |
Fuzzy k-means | 16.33 | 15.70 | 4 | 34 | 2.17 | ||||
DBSCAN | 14.67 | 21.08 | 2 | 39 | 1.67 | ||||
Final | |||||||||
k-means ++ | 16.33 | 23.18 | 1 | 43 | 2.33 | 2.00 | 2 | 0.37 | |
Fuzzy k-means | 16.33 | 11.06 | 6 | 28 | 2.33 | ||||
DBSCAN | 13.33 | 21.39 | 0 | 38 | 1.33 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Sáiz-Manzanares, M.C.; Rodríguez-Díez, J.J.; Díez-Pastor, J.F.; Rodríguez-Arribas, S.; Marticorena-Sánchez, R.; Ji, Y.P. Monitoring of Student Learning in Learning Management Systems: An Application of Educational Data Mining Techniques. Appl. Sci. 2021, 11, 2677. https://doi.org/10.3390/app11062677
Sáiz-Manzanares MC, Rodríguez-Díez JJ, Díez-Pastor JF, Rodríguez-Arribas S, Marticorena-Sánchez R, Ji YP. Monitoring of Student Learning in Learning Management Systems: An Application of Educational Data Mining Techniques. Applied Sciences. 2021; 11(6):2677. https://doi.org/10.3390/app11062677
Chicago/Turabian StyleSáiz-Manzanares, María Consuelo, Juan José Rodríguez-Díez, José Francisco Díez-Pastor, Sandra Rodríguez-Arribas, Raúl Marticorena-Sánchez, and Yi Peng Ji. 2021. "Monitoring of Student Learning in Learning Management Systems: An Application of Educational Data Mining Techniques" Applied Sciences 11, no. 6: 2677. https://doi.org/10.3390/app11062677
APA StyleSáiz-Manzanares, M. C., Rodríguez-Díez, J. J., Díez-Pastor, J. F., Rodríguez-Arribas, S., Marticorena-Sánchez, R., & Ji, Y. P. (2021). Monitoring of Student Learning in Learning Management Systems: An Application of Educational Data Mining Techniques. Applied Sciences, 11(6), 2677. https://doi.org/10.3390/app11062677