An Intelligent Model for Parametric Cognitive Assessment of E-Learning-Based Students
Abstract
:1. Introduction
- Remembering: The recall of the basic facts and information.
- Understanding: The explanation of ideas or models.
- Application: Real-world usage along with associated knowledge in the latest situations.
- Analysis: The splitting of information into parts to understand relationships or patterns.
- Evaluation: Judgments based on criteria with evidence.
- Creation: The combination of parts to form a new whole or propose new ideas.
Research Objectives
- ✓
- Cognitive skills need to be assessed in an online environment to evaluate students.
- ✓
- Student performance and cognitive engagement relate to one another in online education.
- ✓
- Levels of Bloom’s Taxonomy will be used to evaluate student learning states.
- ✓
- The machine-learning model is used for the assessment of cognitive abilities through textual answers.
- ✓
- To contribute to academia in a way that ensures distance-learning education promotion with guaranteed student presence.
- ✓
- To inculcate effective modeling, which includes Knowledge units, Communication units, and Response units, these units work together to efficiently measure student assessment and effectively measure the responses during action.
- ✓
- To provide possibly powerful scoring and reporting as real-time feedback mechanisms. Cognitive assessment will be at the student end for measuring student performance based on delivered content.
- ✓
- The suggested research enhances the cognitive-evaluation process for an e-learning platform, resulting in transformed e-learning.
- (1)
- A dataset pertaining to 20 questions at 6 cognitive levels for 300 students, which makes a sum of 36,000 entries in a single episode.
- (2)
- A custom-built LSTM classification model to ensure BT on examination questions.
- (3)
- A parametric cognitive-based assessment, in an episodic manner, is a unique idea to diagnose and evaluate the learning using a pretrained question-pool bank.
2. Literature Review
Relationship Between BT and Cognitive Domain
3. Materials and Methods
- (1)
- Outline the span and standing of artificial intelligence in everyday life applications.
- (2)
- Describe the elementary principles and concepts related to heuristics-based problems.
- (1)
- “How do you define transferable skills?”
- (2)
- “Define and elaborate the use of the Genetic Algorithm.”
3.1. Data Acquisition
3.2. Domain Understanding/Question Pool Development and Selection
- (1)
- It is considered to be a crucial activity not only for the assessment of course quality but also to ensure the quality of the examination paper, which is needed to measure student learning based on outcome and vice versa.
- (2)
- CLO-based question statement holds Bloom’s keyword/action verb that overlaps different levels from neighboring words. These are examined at diverse levels. It is the words that are recognized through the family of words and for which the word-embedding library of Python is deployed to ensure the integrity of action verbs. Analyzing the results of 36,000 records from an RNN (recurrent neural network)-based assessment involves several steps, as follows:
- Data Preparation: This includes preprocessing the data, which may involve cleaning, normalizing, and encoding it into a format appropriate for input into the RNN.
- Model Training: Training the RNN model applied on a fragment of the data by creating epochs. Epochs involve feeding the input sequences into the network for computing the output predictions and comparing them with the actual targets by adjusting the network weights using procedures like backpropagation and gradient descent.
- Validation: Evaluating the trained model on a split fragment of the data. The split of training to test data on the ratios of 70% to 30% to generalize the model performance and to detect the overfitting.
- Assessment Metrics: Calculating various metrics for assessing the performance of the model at both episode levels, such as Accuracy, Precision, Recall, F1-Score (a predictive performance measure based on true positive, false positive, true negative, and false negative values), etc., depending on the assessment nature (classification, regression, etc.).
- Analysis of Results: This involves interpreting assessment metrics to draw conclusions about the effectiveness of the model. For example, if it is a classification task, one might analyze confusion matrices to see which classes of students/groups are struggling to achieve the parametric threshold of CLO, which was and will be set at 70–90% approx.
- Iterative Improvement: Based on the analysis, one might iterate on the model architecture (hyperparameters like learning rate with batch size and number of epochs to optimize training for your dataset), or data-preprocessing steps are taken to expand the performance with respect to precision and accuracy of the designed system.
- (3)
- Question-pool training holds the task of a question pool established for students to be tested. It is prepared through the division of training and testing data operating on the layers of LSTM for batch normalization for testing different cognitive levels of students, which are refined and then projected to students for ultimate assessment in an iterative fashion. The long short-term memory network is basically a kind of recurrent neural network, which is usually cast off for performing a sequence of modeling tasks. In our research case study, we are exercising episodic-based assessment to chain the difference between the previous and the new findings, which are used as batch normalization for feedback. This can be adapted to LSTMs to stabilize and accelerate the training process. Detailed below is the explanation of how batch normalization works in the context of LSTM networks. An LSTM cell has several gates (input, forget, output) with a memory cell that helps in maintaining information over long sequences. The basic equations governing an LSTM cell are mentioned in Figure 1, generated through Spyder IDE in Python. The flowchart of a neural network architecture is shown in Figure 2, where LSTM layers are combined with batch normalization. Here is a breakdown of how batch normalization is applied in this LSTM-based model, detailed by architectural overview.
3.3. Architecture Overview
- Input Layer: The input model has a shape of (None, 33), where None represents the batch size and 33 is the number of features.
- Embedding Layer: This layer transforms the input into a higher-dimensional space, with the output shape (None, 33, 300) indicating 300-dimensional embedding for each input feature.
- First LSTM Layer: The LSTM processes the embedded input, producing an output with the shape (None, 33, 128), meaning 128-dimensional output vectors for each time step.
- Batch Normalization: After the first LSTM, batch normalization is applied to the LSTM output. This operation normalizes the output of LSTM across the batch, helping to stabilize and speed up training. The output shape remains (None, 33, 128).
- Second LSTM Layer: The normalized output is passed through a second LSTM layer, reducing the dimensionality to (None, 64).
- Dense Layer: A dense layer is applied, producing an output of shape (None, 64).
- Second Batch Normalization: Batch normalization is applied again after the dense layer, normalizing the output across the whole batch.
- Dropout Layer: A dropout layer develops with the same output shape (None, 64). Dropout is alleviated by preventing and overfitting in randomly setting a fraction of input units to value “0” through the training process.
- Third Batch Normalization: Another batch-normalization layer is applied, further stabilizing the output.
- Final Dense Layer: Finally, the model inputs a layer with 3 units, typically representing a classification into classes, as follows:
- (1)
- Word-based vectors by means of skip-gram-built word embedding for representing text in the form of numeric features after processing.
- (2)
- Lastly, the use of a BT-based level classifier is deployed for category-based questions into one of the predefined types.
3.4. Proposed System Overview
- (1)
- The initial step is based on text preprocessing with data cleaning, which takes an input in the form of text and then preprocesses it by altering it into baseline lower-case letters, removing stop words with punctuation marks, and then converting those words to their root words by means of lemmatization.
- (2)
- The next step, after text preprocessing, is to compute and calculate numeric-based word vectors on skip-gram word embedding to represent text into numeric-based feature selection.
- (3)
- Lastly, the BT level classifies the text into pre-distinct categories.
- (4)
- Text preprocessing with cleaning is the initial step, which receives the input text and then preprocesses it by adapting that given text into lower-case form by removing those stop words with the punctuation marks and then by adapting all the words in the mentioned root words in the form of lemmatization.
- (5)
- After the completion of text preprocessing, the succeeding step is to calculate numeric word-based vectors by means of word embedding based on skip-gram to represent text in the form of numeric features after processing.
- (6)
- Lastly, the BT classifier is deployed to categorize questions into predefined types.
3.5. Construction of the Proposed System
3.5.1. Data Preprocessing and Cleaning
Algorithm 1. Pseudocode of Data Preprocessing, Cleaning Algorithm |
Input Data: Sample Query Text Output Data: CLO-Based Preprocessed form of Data Sentences extract Sentences as (INPUT) Prep Input = Vacant while sent sentence structure do stSentence. remove StopWords(sent) PSentenceremove Punctuation(stSentencebased) words extract-Words-from(pSentence) PrepSentence structure Empty (default empty bucket) while words do l for word lower(word) Lemma Word Word.NetLemmatizer(lword) prep Sentence lemma word prep Sentence-space End while Prep Input. append state (prep Sentence) end while Output Result ← prep Input (by tokenizing keywords and then formulating them into meaningful/useful sentences) |
3.5.2. Data Preparation with Splitting
Algorithm 2. Data Splitting with Data Preparation |
INPUT: Preprocessed Input Text based purely on the Class Labels OUTPUT: Training based on Test Data Partition Unique Words ← unique Words ← max Length ← max Len-Test Size ← Test-Size Text ← Based sequencing (Preprocessed Input Text) Tags Mark ← Basic coding (Class Mark Marker) Data Markers ← shuffle (Text, Markers) Train Data = Partition (Data, Relation Ratio = 1 − Test-Size) Test Data = Partition (Data, Relation Ratio = Test-Size) |
3.5.3. Development of BT Level Classifier
3.5.4. Word Representation with Word-Embedding-Based Learning
- ✓
- Word-based embedding is prepared for training on Wikipedia-provided text utilizing techniques for word representation generation relying on its neighboring words.
- ✓
- The dataset consists of the maximum number of words from the Wikipedia-provided corpus.
4. Experimental Setup
- Preprocessing:
- ▪
- Remove stop words using NLTK or SpaCy.
- ▪
- Tokenize the text data and convert it into sequences.
- Word Embedding with RNN:
- ▪
- Use deep-learning libraries like TensorFlow or PyTorch to create an RNN model.
- ▪
- Implement an embedding layer to convert words into numeric vectors.
- ▪
- Construct an RNN model (LSTM or GRU) that takes these embeddings as input.
- Identifying Cognitive Levels:
- ▪
- Train the RNN model on a dataset labeled with different cognitive levels.
- ▪
- Use the trained model to predict the cognitive level of new text inputs.
- ▪
- The process mentioned above is considered to be a single episode followed up by Episode 2, repeating the same procedure to effectively access 300 students in the same defined parameters associated with Bloom’s Taxonomy assigned with a pre-allocated percentile of assessment. The student’s cognitive assessment model is based on the text-input feed with rule-based text processing based on completeness and correctness. The results are classified using a Random Forest classifier for predicting the level of correctness achieved by the students based on a defined assessment threshold as follows: Below Average <50%, Average 50%–60%, Good >60%.
- ▪
- By tuning the hyperparameters (hyperparameters like learning rate with batch size and the number of epochs to optimize training for your dataset), LSTM in RNN obtains the optimized and refined assessment results with over 90% accuracy as compared to previous models on a standard dataset.
5. Results and Discussion
5.1. Comparison of Episode 1 and Episode 2
5.2. Random Forest Classifier Prediction
5.3. Performance Comparison Analysis of Different Classification Algorithms on Same Dataset
5.4. Performance Comparison Analysis of Metric Base of Algorithms on Dataset
5.5. Comparison of RNN Classifier
5.6. Experimental Results of Random Forest Classifier
6. Performance Results at Each Level of Bloom’s Taxonomy—Episode 1
7. Performance Results at Each Level of Bloom’s Taxonomy—Episode 2
Summarized Result of Episode 1 and Episode 2
8. Conclusions
9. Future Work
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
AI | Artificial Intelligence |
CogAT | Cognitive Ability Testing |
LSTM | Long Short-Term Memory |
GENAI | Generative artificial intelligence |
RF | Random Forest |
ML | Machine Learning |
NLP | Natural-Language Processing |
BT | Blooms Taxonomy |
COG | Cognition |
KNN | K Nearest Neighbor |
RNN | Recurrent Neural Network |
CNN | Convolutional Neural Network |
MOOCS | Massively Open Online Courses |
IDF | Inverse Document Frequency |
NLTK | Natural-Language Toolkit |
DL | Deep Learning |
LO | Learning Objective |
CLO | Course Learning Objective |
References
- Omar, N.; Haris, S.S.; Hassan, R.; Arshad, H.; Rahmat, M.; Zainal, N.F.A.; Zulkifli, R. Automated analysis of exam questions according to BT. Procedia Social Behav. Sci. 2012, 59, 297–303. [Google Scholar] [CrossRef]
- Chang, W.-C.; Chung, M.-S. Automatic applying BT to classify and analysis the cognition level of english question items. In Proceedings of the Joint Conferences on Pervasive Computing (JCPC), Taipei, Taiwan, 3–5 December 2009; pp. 727–734. [Google Scholar]
- OSMAN; Yahya, A.A. Classifications of exam questions using natural language syntatic features: A case study based on Bloom’s taxonomy. In Proceedings of the 3rd International Arab Conference on Quality Assurance in Higher Education (IACQA’2016), Zarqa, Jordan, 9–11 February 2016; pp. 1–8. [Google Scholar]
- Monrad, S.U.; Bibler Zaidi, N.L.; Grob, K.L.; Kurtz, J.B.; Tai, A.W.; Hortsch, M.; Gruppen, L.D.; Santen, S.A. What faculty write versus what students see? Perspectives on multiple-choice questions using Bloom’s taxonomy. Med. Teach. 2021, 43, 575–582. [Google Scholar] [CrossRef] [PubMed]
- Krathwohl, D.R.; Anderson, L.W. Merlin C. Wittrock and the revision of BT. Educ. Psychol. 2010, 45, 64–65. [Google Scholar] [CrossRef]
- Bloom, B.S. Taxonomy of educational objectives: The classification of educational goals. Cognit. Domain 1981, 51, 441–453. [Google Scholar]
- Chyung, S.-Y.; Stepich, D. Applying the ‘congruence’ principle of BT to designing online instruction. Q. Rev. Distance Educ. 2003, 4, 317–330. [Google Scholar]
- Kowsari, K.; Meimandi, K.J.; Heidarysafa, M.; Mendu, S.; Barnes, L.; Brown, D. Text classification algorithms: A survey. Information 2019, 10, 150. [Google Scholar] [CrossRef]
- Haris, S.S.; Omar, N. A rule-based approach in BT question classification through natural language processing. In Proceedings of the 2012 7th International Conference on Computing and Convergence Technology (ICCCT), Seoul, Republic of Korea, 3–5 December 2012; pp. 410–414. [Google Scholar]
- Dwivedi, S.K.; Arya, C. Automatic text classification in information retrieval: A survey. In Proceedings of the 2nd International Conference on Information and Communication Technology for Competitive Strategies (ICTCS), Udaipur India, 4–5 March 2016; p. 131. [Google Scholar]
- Das, S.; Mandal, S.K.D.; Basu, A. Classification of action verbs of BT cognitive domain: An empirical study. J. Educ. 2021, 201, 002205742110021. [Google Scholar]
- Young, T.; Hazarika, D.; Poria, S.; Cambria, E. Recent trends in deep learning based natural language processing. IEEE Comput. Intell. Mag. 2018, 13, 55–75. [Google Scholar] [CrossRef]
- Bengio, Y.; Senecal, J.-S. Adaptive importance sampling to accelerate training of a neural probabilistic language model. IEEE Trans. Neural Netw. 2008, 19, 713–722. [Google Scholar] [CrossRef]
- Khairuddin, N.N.; Hashim, K. Application of BT in software engineering assessments. In Proceedings of the 8th WSEAS International Conference on Applied Computer Science (ACS’08), Venice, Italy, 21–23 November 2008; pp. 66–69. [Google Scholar]
- Zhang, J.; Wong, C.; Giacaman, N.; Luxton-Reilly, A. Automated classification of computing education questions using BT. In Proceedings of the 23rd Australasian Computing Education Conference, Virtual, 2–4 February 2021; pp. 58–65. [Google Scholar]
- Abduljabbar, D.A.; Omar, N. Exam questions classification based on BT cognitive level using classifiers combination. J. Theor. Appl. Inf. Technol. 2015, 78, 447. [Google Scholar]
- Osadi, K.; Fernando, M.G.N.A.S.; Welgama, W. Ensemble classifier-based approach for classification of examination questions into BT cognitive levels. Int. J. Comput. Appl. 2017, 975, 8887. [Google Scholar]
- Shahzad, R.; Aslam, M.; Al-Otaibi, S.; Javed, M.S.; Khan, A.R.; Bahaj, S.A.; Saba, T. Multi-Agent System for Students Cognitive Assessment in E-Learning Environment. IEEE Access 2024, 12, 15458–15467. [Google Scholar] [CrossRef]
- Badjatiya, P.; Gupta, S.; Gupta, M.; Varma, V. Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web Companion, Perth, Australia, 3–7 April 2017; pp. 759–760. [Google Scholar]
- Swart, J.; Daneti, M. Analyzing learning outcomes for electronic fundamentals using BT. In Proceedings of the 2019 IEEE Global Engineering Education Conference (EDUCON), Dubai, United Arab Emirates, 9–11 April 2019; pp. 39–44. [Google Scholar]
- Rahmatih, A.N.; Indraswati, D.; Gunawan, G.; Widodo, A.; Maulyda, M.A.; Erfan, M. An analysis of questioning skill in elementary school pre-service teachers based on BT. J. Phys. Conf. Ser. 2021, 1779, 012073. [Google Scholar] [CrossRef]
- Hoogeveen, D.; Wang, L.; Baldwin, T.; Verspoor, K.M. Web forum retrieval and text analytics: A survey. Found. Trends Inf. Retr. 2018, 12, 1–163. [Google Scholar] [CrossRef]
- Manning, C.D.; Raghavan, P.; Schutze, H. Introduction to Information Retrieval; Cambridge University Press: Cambridge, UK, 2008; Chapter 20; pp. 405–416. [Google Scholar]
- Buckley, C. Implementation of the Smart Information Retrieval System; Tech. Rep. TR85-686; Cornell University: New York, NY, USA, 1985; Available online: https://ecommons.cornell.edu/server/api/core/bitstreams/9f55bb4c-70a1-49b8-b78d-8914a8f51e62/content (accessed on 24 November 2024).
- O’Riordan, C.; Sorensen, H. Information filtering and retrieval: An overview. In Proceedings of the 16th Annual International Conference of the IEEE, Atlanta, GA, USA, 4–7 May 1997; pp. 28–31. [Google Scholar]
- van Hoeij, M.J.W.; Haarhuis, J.C.M.; Wierstra, R.F.A.; van Beukelen, P. Developing a classification tool based on Bloom’s taxonomy to assess the cognitive level of short essay questions. J. Vet. Med. Educ. 2004, 31, 261–267. [Google Scholar] [CrossRef]
- Tariq, M.U.; Sergio, R.P. Innovative Assessment Techniques in Physical Education: Exploring Technology-Enhanced and Student-Centered Models for Holistic Student Development. In Global Innovations in Physical Education and Health; IGI Global: Hershey, PA, USA, 2025; pp. 85–112. [Google Scholar]
- Ozen, Z.; Pereira, N.; Karatas, T.; Castillo-Hermosilla, H.; Maeda, Y. A Meta-Analytic Evaluation: Investigating Evidence for the Validity of the Cognitive Abilities Test. Gift. Child Q. 2025, 69, 3–15. [Google Scholar] [CrossRef]
- Almutawa, S.S.; Alshehri, N.A.; AlNoshan, A.A.; AbuDujain, N.M.; Almutawa, K.S.; Almutawa, A.S. The influence of cognitive flexibility on research abilities among medical students: Cross-section study. BMC Med. Educ. 2025, 25, 7. [Google Scholar] [CrossRef]
- Jayakodi, K.; Bandara, M.; Perera, I. An automatic classifier for exam questions in engineering: A process for BT. In Proceedings of the 2015 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), Zhuhai, China, 10–12 December 2015; pp. 195–202. [Google Scholar]
- Yamasari, Y.; Rochmawati, N.; Putra, R.E.; Qoiriah, A.; Yustanti, W. Predicting the students’ performance using regularization-based linear regression. In Proceedings of the 2021 Fourth International Conference on Vocational Education and Electrical Engineering (ICVEE), Surabaya, Indonesia, 2–3 October 2021; pp. 1–5. [Google Scholar]
- Singla, A. Roberta and BERT: Revolutionizing Mental Healthcare through Natural Language. Shodh Sagar J. Artif. Intell. Mach. Learn. 2024, 1, 10–27. [Google Scholar] [CrossRef]
- Stringer, J.K.; Santen, S.A.; Lee, E.; Rawls, M.; Bailey, J.; Richards, A.; Perera, R.A.; Biskobing, D. Examining Bloom’s taxonomy in multiple choice questions: Students’ approach to questions. Med. Sci. Educ. 2021, 31, 1311–1317. [Google Scholar] [CrossRef]
- Yusof, N.; Hui, C.J. Determination of Bloom’s cognitive level of question items using artificial neural network. In Proceedings of the 2010 10th International Conference on Intelligent Systems Design and Applications, Cairo, Egypt, 29 November–1 December 2010; pp. 866–870. [Google Scholar]
- Chanaa, A. A cognitive level evaluation method based on machine learning approach and Bloom of taxonomy for online assessments. J. Educ. Learn. (EduLearn) 2024, 18, 553–560. [Google Scholar] [CrossRef]
- KJayakodi; Bandara, M.; Perera, I.; Meedeniya, D. Wordnet and cosine similarity-based classifier of exam questions using BT. Int. J. Emerg. Technol. Learn. 2016, 11, 142–149. [Google Scholar] [CrossRef]
- Hong, S.; Kim, J.; Yang, E. Automated text classification of maintenance data of higher education buildings using text mining and machine learning techniques. J. Archit. Eng. 2022, 28, 04021045. [Google Scholar] [CrossRef]
- Botelho, A.; Baral, S.; Erickson, J.A.; Benachamardi, P.; Heffernan, N.T. Leveraging natural language processing to support automated assessment and feedback for student open responses in mathematics. J. Comput. Assist. Learn. 2023, 39, 823–840. [Google Scholar] [CrossRef]
- Gupta, S.; Gulia, P. Comparative Analysis of Predictive Algorithms for Performance Measurement. IEEE Access 2024, 12, 33949–33958. [Google Scholar] [CrossRef]
- Mohammed, M.; Omar, N. Question classification based on BT cognitive domain using modified TF-IDF and word2vec. PLoS ONE 2020, 15, e0230442. [Google Scholar] [CrossRef]
- Mikolov, T.; Karafiát, M.; Burget, L.; Černocky, J.; Khudanpur, S. Recurrent neural network-based language model. In Proceedings of the 11th Annual Conference of the International Speech Communication Association, Chiba, Japan, 26–30 September 2010; pp. 1–24. [Google Scholar]
- Sharma, S.; Saxena, D.K. Understanding the Cognitive Constituents of E-Learning. In Best Practices and Strategies for Online Instructors: Insights from Higher Education Online Faculty; IGI Global Scientific Publishing: Hershey, PA, USA, 2025; pp. 277–312. [Google Scholar]
- Wijanarko, B.D.; Heryadi, Y.; Toba, H.; Budiharto, W. Question generation model based on key-phrase, context-free grammar, and BT. Educ. Inf. Technol. 2021, 26, 2207–2223. [Google Scholar] [CrossRef]
- El Msayer, M.; Aoula, E.S.; Bouihi, B. Artificial intelligence in computerized adaptive testing to assess the cognitive performance of students: A Systematic Review. In Proceedings of the 2024 International Conference on Intelligent Systems and Computer Vision (ISCV), Fez, Morocco, 8–10 May 2024; pp. 1–8. [Google Scholar]
- Shaikh, S.; Daudpotta, S.M.; Imran, A.S. Bloom’s learning outcomes’ automatic classification using lstm and pretrained word embeddings. IEEE Access 2021, 9, 117887–117909. [Google Scholar] [CrossRef]
- Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G.S.; Dean, J. Distributed representations of words and phrases and their compositionality. In Proceedings of the Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013, Lake Tahoe, NV, USA, 5–8 December 2013; pp. 3111–3119. [Google Scholar]
- Socher, R.; Perelygin, A.; Wu, J.; Chuang, J.; Manning, C.D.; Ng, A.; Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA, 18–21 October 2013; pp. 1631–1642. [Google Scholar]
- Vasilomanolakis, E.; Karuppayah, S.; Mühlhäuser, M.; Fischer, M. Taxonomy and survey of collaborative intrusion detection. ACM Comput. Surv. 2015, 47, 55:1–55:33. [Google Scholar] [CrossRef]
- Song, D.; Rice, M.; Oh, E.Y. Participation in online courses and interaction with a virtual agent. Int. Rev. Res. Open Distrib. Learn. 2019, 20. [Google Scholar] [CrossRef]
- Swiecki, Z.; Khosravi, H.; Chen, G.; Martinez-Maldonado, R.; Lodge, J.M.; Milligan, S.; Selwyn, N.; Gašević, D. Assessment in the age of artificial intelligence. Comput. Educ. Artif. Intell. 2022, 3, 100075. [Google Scholar] [CrossRef]
- Wu, Q.; Bansal, G.; Zhang, J.; Wu, Y.; Zhang, S.; Zhu, E.; Li, B.; Jiang, L.; Zhang, X.; Wang, C. Autogen: Enabling nextgen llm applications via multi-agent conversation framework. arXiv 2023, arXiv:230808155. [Google Scholar]
- Atiullah, K.; Fitriati, S.W.; Rukmini, D. Using revised BT to evaluate higher order thinking skills (hots) in reading comprehension questions of english textbook for year X of high school. English Educ. J. 2019, 9, 428–436. [Google Scholar] [CrossRef]
- Laddha, M.D.; Lokare, V.T.; Kiwelekar, A.W.; Netak, L.D. Classifications of the summative assessment for revised BT by using deep learning. arXiv 2021, arXiv:2104.08819. [Google Scholar]
- Febri, A. Analysis of Students’ Critical Thinking Skills at Junior High School in Science Learning. J. Phys. Conf. Ser. 2019, 1397, 012018. [Google Scholar] [CrossRef]
- Zhang, D.; Lee, W.S. Question classification using support vector machines. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Toronto, ON, Canada, 28 July–1 August 2003; pp. 26–32. [Google Scholar]
- Yan, L.; Greiff, S.; Teuber, Z.; Gašević, D. Generative Artificial Intelligence and Human Learning. arXiv 2024, arXiv:2408.12143. [Google Scholar]
- Musa, M.; Ahmadu, A.S.; Williams, C. Comparative Analysis of K-Means and Naïve Bayes Algorithms for Predicting Students’ Academic Performance. Int. J. Dev. Math. (IJDM) 2024, 1, 196–208. [Google Scholar] [CrossRef]
- Servin, C.; Tang, C.; Geissler, M.; Stange, M.; Tucker, C. Enhanced verbs for BT with focus on computing and technical areas. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education, Virtual, 13–20 March 2021; p. 1270. [Google Scholar]
- Yahya, A.; Osman, A. Automatic Classification of Questions into Bloom’s Cognitive Levels Using Support Vector Machines; Tech. Rep. 2914; Naif Arab University for Security Sciences: Riyadh, Saudi Arabia, 2011. [Google Scholar]
Year—Ref. No. | Methodology | Participants | Results | Research Gap |
---|---|---|---|---|
2016—[3] | NLP technique to assess student response | Online students’ | Identification of online learning challenges | Required to consider student mental state |
2021—[4] | ML-based text analysis for measuring conceptual knowledge | Course—Computer Signals | Automatic evaluation of learners | It needs to be tested on a higher amount of dataset |
2024—[18] | Multiagent-based cognitive assessment | Undergrad students’ data—SE | Accuracy of 91% | Small number of questions |
2021—[21] | Intelligent agent-based system for tracking students’ performance | MOODLE Platform | Innovative way to measure students’ involvement | Need to determine cognitive level |
2025—[27] | Timeline based on student’s assessment | 60 | Skill-set-based results | Competency Testing Lacking |
2025—[28] | CogAt Systematic Assessment | 24 | Effect size-based estimations | Nonverbal assessment scale |
2025—[29] | Teaching-Research Correlation | Undefined | 95% Confidence Level | Just medical level students tested based on questionnaires |
2015—[30] | An automated approach to determine the message’s cognitive level | Discussion forum messages | Enhanced the model of assessment | Need to evaluate performance |
2021—[31] | Prediction model of students’ performance | Random Volunteer Student Dataset | Linear regression based on Regularization | Need to focus on qualitative exams and deep-learning technique optimization. |
2024—[32] | Students’ mental state assessment using Roberta, and BERT model | Text data from MOOC | Overall accuracy more than 92% but on biased data | It needs to be tested on biased free higher volume of dataset |
2021—[33] | Examining Blooms taxonomy using multiple choice questions | 137 students tested | Achieved 74.9% Accuracy | Text-based responses of students’ need to be tested |
2010—[34] | Use of the KNN algorithm and RBT to classify cognitive states | 100 CS Students | Accuracy is 84% | Need to compare neural network-based models |
2024—[35] | ML-based Cognitive Analysis | The Dataset of the assessment data of school | Accuracy of 82.2% | Model integrator fixed parameters-based, online recommendation system |
2021—[36] | WordNet similarity algorithms with NLTK and cosine similarity algorithm | Wayamba University, Sri Lanka—Random | 70% Accuracy | Fixed Dataset |
2022—[37] | Questions Classification based on Text-Mining Method | Exam Questions | Analyze Question Paper Using BT | Required to map cognitive level with responses |
2023—[38] | Regression model for categorizing subjective questions | Computer Networks Course—Item Bank | BT-Based Level Prediction | Testing is required using course questions |
2024—[39] | Use of SVM model for predictive analysis and K-Mean Clustering for descriptive analysis | University Students | Accuracy is 87% | Need to improve performance on defined dataset |
Data Description | Information Source | Instances | Levels | Class Instances |
---|---|---|---|---|
Dataset for Episode 1 (300 students on test Bed) | Graduate-Level University Students (Virtual)-AI Domain | 300 students × 20 Questions each level = 6000 | 6 | Knowledge: 5000 Assigned Weight: 15% Comprehension: 5000 Assigned Weight: 10% Application: 5000 Assigned Weight: 20% Analysis: 5000 Assigned Weight: 25% Synthesis: 5000 Assigned Weight: 10% Evaluation: 5000 Assigned Weight: 20% |
Dataset for Episode 2 (300 students on test Bed) | Graduate-Level University Students (Virtual)-AI Domain | 300 students × 20 Questions each level = 6000 | 6 | Knowledge: 5000 Assigned Weight: 15% Comprehension: 5000 Assigned Weight: 10% Application: 5000 Assigned Weight: 20% Analysis: 5000 Assigned Weight: 25% Synthesis: 5000 Assigned Weight: 10% Evaluation: 5000 Assigned Weight: 20% |
Sr# | Knowledge | Comprehension | Application | Analysis | Synthesis | Evaluation |
---|---|---|---|---|---|---|
1 | 71 | 34 | 50 | 48 | 45 | 43 |
2 | 68 | 54 | 74 | 68 | 56 | 56 |
3 | 67 | 41 | 70 | 48 | 67 | 44 |
4 | 62 | 52 | 60 | 75 | 76 | 29 |
5 | 58 | 74 | 61 | 63 | 45 | 65 |
6 | 59 | 71 | 62 | 48 | 34 | 56 |
7 | 76 | 72 | 49 | 38 | 74 | 67 |
8 | 71 | 74 | 67 | 48 | 35 | 74 |
9 | 74 | 35 | 75 | 68 | 31 | 48 |
10 | 49 | 75 | 75 | 76 | 44 | 73 |
11 | 69 | 75 | 52 | 68 | 46 | 74 |
12 | 58 | 38 | 54 | 49 | 47 | 47 |
13 | 65 | 76 | 74 | 38 | 39 | 65 |
14 | 75 | 56 | 68 | 48 | 41 | 51 |
15 | 44 | 46 | 74 | 68 | 42 | 48 |
16 | 67 | 69 | 71 | 75 | 44 | 71 |
17 | 71 | 49 | 72 | 68 | 76 | 54 |
18 | 74 | 74 | 56 | 48 | 49 | 51 |
19 | 75 | 67 | 76 | 74 | 74 | 61 |
20 | 64 | 48 | 58 | 67 | 46 | 56 |
21 | 54 | 72 | 28 | 49 | 45 | 65 |
22 | 75 | 75 | 29 | 71 | 59 | 71 |
23 | 38 | 73 | 74 | 64 | 73 | 72 |
24 | 41 | 74 | 46 | 76 | 14 | 49 |
25 | 48 | 74 | 64 | 48 | 67 | 69 |
⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ | ⋮ |
275 | 54 | 45 | 68 | 76 | 19 | 67 |
276 | 76 | 45 | 58 | 68 | 56 | 46 |
277 | 65 | 75 | 77 | 68 | 57 | 76 |
278 | 65 | 64 | 46 | 76 | 66 | 66 |
279 | 49 | 54 | 45 | 48 | 55 | 70 |
280 | 73 | 74 | 19 | 74 | 56 | 45 |
281 | 29 | 39 | 73 | 76 | 74 | 67 |
282 | 74 | 72 | 72 | 76 | 76 | 76 |
283 | 46 | 75 | 69 | 74 | 46 | 45 |
284 | 65 | 68 | 71 | 19 | 45 | 67 |
285 | 47 | 49 | 75 | 75 | 47 | 74 |
286 | 75 | 74 | 76 | 71 | 57 | 54 |
287 | 64 | 68 | 73 | 68 | 46 | 67 |
288 | 59 | 69 | 73 | 46 | 76 | 76 |
289 | 75 | 71 | 19 | 71 | 58 | 67 |
290 | 41 | 29 | 49 | 74 | 41 | 73 |
291 | 28 | 68 | 69 | 72 | 28 | 65 |
292 | 37 | 69 | 71 | 68 | 39 | 45 |
293 | 39 | 49 | 74 | 75 | 56 | 68 |
294 | 44 | 67 | 69 | 64 | 44 | 94 |
295 | 48 | 49 | 58 | 64 | 49 | 57 |
296 | 49 | 58 | 58 | 54 | 53 | 76 |
297 | 52 | 68 | 71 | 68 | 51 | 71 |
298 | 57 | 74 | 75 | 67 | 54 | 72 |
299 | 61 | 67 | 76 | 74 | 65 | 54 |
300 | 74 | 61 | 77 | 71 | 55 | 71 |
Student Ident | Knowledge | Comprehension | Application | Analysis | Synthesis | Evaluation |
---|---|---|---|---|---|---|
1 | 71 | 34 | 50 | 48 | 45 | 43 |
50 | 39 | 48 | 58 | 71 | 75 | 64 |
Ranking | Knowledge | Comprehension | Application | Analysis | Synthesis | Evaluation |
---|---|---|---|---|---|---|
Below Average | 105 | 71 | 48 | 66 | 126 | 39 |
Average | 87 | 111 | 117 | 117 | 93 | 141 |
Good | 108 | 117 | 135 | 117 | 81 | 120 |
Id | Algorithm | Accuracy | Precision | Recall | F1 Score | Execution Time (ms) | Complexity |
---|---|---|---|---|---|---|---|
1 | Graph Cosine | 75 | 72 | 78 | 75 | 15 | Moderate |
2 | K-means | 55 | 50 | 60 | 54 | 10 | Low |
3 | Naïve Bayes | 90 | 88 | 92 | 90 | 5 | Low |
4 | Vector Lookup Table | 80 | 82 | 78 | 80 | 12 | Moderate |
5 | Recurrent Neural Network (RNN) | 98 | 97 | 99 | 98 | 200 | High |
Metric | RNN | Baseline Model | ML Model |
---|---|---|---|
Accuracy % | 85.3 | 70.2 | 83.5 |
Training Time Sec | 300 | 15 | 120 |
Precision | 0.89 | 0.72 | 0.87 |
Recall | 0.91 | 0.70 | 0.88 |
Interoperability | Low | High | Medium |
Model | Indicators | Taxonomy Levels Based Cognitive Scales | |||||
---|---|---|---|---|---|---|---|
Random Forest | Level#1 | Level#2 | Level#3 | Level#4 | Level#5 | Level#6 | |
Precision | 89.8% | 89.2% | 93.3% | 93% | 94.16% | 94.7% | |
Recall | 92.0% | 91.5% | 92.1% | 94.7% | 92.95% | 95.7% | |
F1 | 90.7% | 90.3% | 92.7% | 93.8% | 93.55% | 95.2% | |
Accuracy | 92.16% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Javed, M.S.; Aslam, M.; Khurshid, S.K. An Intelligent Model for Parametric Cognitive Assessment of E-Learning-Based Students. Information 2025, 16, 93. https://doi.org/10.3390/info16020093
Javed MS, Aslam M, Khurshid SK. An Intelligent Model for Parametric Cognitive Assessment of E-Learning-Based Students. Information. 2025; 16(2):93. https://doi.org/10.3390/info16020093
Chicago/Turabian StyleJaved, Muhammad Saqib, Muhammad Aslam, and Syed Khaldoon Khurshid. 2025. "An Intelligent Model for Parametric Cognitive Assessment of E-Learning-Based Students" Information 16, no. 2: 93. https://doi.org/10.3390/info16020093
APA StyleJaved, M. S., Aslam, M., & Khurshid, S. K. (2025). An Intelligent Model for Parametric Cognitive Assessment of E-Learning-Based Students. Information, 16(2), 93. https://doi.org/10.3390/info16020093