Next Article in Journal
A Survey of Security Challenges in Cloud-Based SCADA Systems
Next Article in Special Issue
A Holistic Approach to Use Educational Robots for Supporting Computer Science Courses
Previous Article in Journal
Voltage and Reactive Power-Optimization Model for Active Distribution Networks Based on Second-Order Cone Algorithm
Previous Article in Special Issue
Assessing the Acceptance of Cyborg Technology with a Hedonic Technology Acceptance Model
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Computers
Volume 13
Issue 4
10.3390/computers13040096
computers-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Systematic Review

The Use of Integrated Multichannel Records in Learning Studies in Higher Education: A Systematic Review of the Last 10 Years

by
Irene González-Díez
1,2,*,
Carmen Varela
1,3 and
María Consuelo Sáiz-Manzanares
1,2,*
1
Departament of Health Sciences, Faculty of Health Sciences, University of Burgos, Paseo Comendadores s/n Burgos, 09001 Burgos, Spain
2
DATAHES Research Group, Consolidated Research Unit Nº. 348, Universidad de Burgos, 09001 Burgos, Spain
3
Departament of Psychobiology and Clinical Psychology, Faculty of Psychology, University of Barcelona, 08035 Barcelona, Spain
*
Authors to whom correspondence should be addressed.
Computers 2024, 13(4), 96; https://doi.org/10.3390/computers13040096
Submission received: 2 February 2024 / Revised: 8 March 2024 / Accepted: 26 March 2024 / Published: 10 April 2024
(This article belongs to the Special Issue Feature Papers in Computers 2024)
Browse Figure
Versions Notes

Abstract

:
Neurophysiological measures have been used in the field of education to improve our knowledge about the cognitive processes underlying learning. Furthermore, the combined use of different neuropsychological measures has deepened our understanding of these processes. The main objective of this systematic review is to provide a comprehensive picture of the use of integrated multichannel records in higher education. The bibliographic sources for the review were Web of Science, PsycINFO, Scopus, and Psicodoc databases. After a screening process by two independent reviewers, 10 articles were included according to prespecified inclusion criteria. In general, integrated recording of eye tracking and electroencephalograms were the most commonly used metrics, followed by integrated recording of eye tracking and electrodermal activity. Cognitive load was the most widely investigated learning-related cognitive process using integrated multichannel records. To date, most research has focused only on one neurophysiological measure. Furthermore, to our knowledge, no study has systematically investigated the use of integrated multichannel records in higher education. This systematic review provides a comprehensive picture of the current use of integrated multichannel records in higher education. Its findings may help design innovative educational programs, particularly in the online context. The findings provide a basis for future research and decision making regarding the use of integrated multichannel records in higher education.

1. Introduction

Nowadays, new technologies play a fundamental role in optimizing the teaching-learning process [1,2]. Neurotechnology has emerged as a new technology with great potential in education, especially in remote learning environments [2]. Neurotechnology is used to record the physiological signals underlying students’ cognitive processes when interacting with learning activities [3,4,5,6]. The most widely used neurophysiological measures have been cognitive load, attention, and emotion because of their important role in student learning [6]. Various techniques have been used to collect and analyze these data from the Central Nervous System (CNS) and the Automatic Nervous System (ANS), such as electroencephalography (EEG), eye tracking, and galvanic skin response (GSR) [6,7,8].
Eye tracking is a method of measuring and recording the movements of the eyes in relation to an external stimulus [2,9], allowing us to observe the initial steps of human cognitive processing [10,11,12]. Furthermore, it is the most widely used technique for automated attention monitoring [7]. Eye tracking offers opportunities to better understand the role of visual processes in educational practice [10,13,14]. Applying it in virtual educational environments improves task performance and learning [9,10,14] and produces engagement [2]. In addition, eye tracking has been classified as a relatively unintrusive method [6].
In contrast, EEG monitors the electrical activity of the brain by recording waves that provide information about a person’s mental state [7,11,15,16,17]. Darvishi et al. [6] classified EEG as a moderately intrusive method because it can be used in two ways: by placing electrodes on the scalp [2,15,16] or using a headband, which has already been used in face-to-face training [2,4,18,19,20]. It is the most commonly used technique for evaluating brain activity because of its lower cost [2,16,21], non-invasiveness [22], and suitability for education [7,23]. Other advantages are its portability [3,15,18], ease of use [15], silence, absence of claustrophobia, non-ionization [16], and high temporal resolution [4,18]. Furthermore, it has been used to assess cognitive load [4,5,20,24,25], test the effectiveness of different learning strategies [5], evaluate effectiveness in different learning modalities [4,23], and measure cognitive load in both traditional face-to-face learning and online education [20]. In addition, each EEG is individual [26,27] because brain activity can be influenced by many genetic and non-genetic factors, such as stress [15]. However, it has not yet been used in remote learning [2].
Electrodermal activity is recorded by GSR. This is a biometric measure for collecting and monitoring a participant’s emotional intensity or arousal [28,29]. GSR evaluates the electrical characteristics of the skin and its changes in relation to the ANS [3,6,30]. Darvishi et al. [6] classified it as a highly intrusive measure due to its placement on a specific area of the participants’ bodies.
The use of these types of measures in integrated multichannel records has increased over recent years as researchers adopt a multimodal approach [13,25,30,31,32]. This increase is related to improved observations providing a deeper understanding of subconscious processes [24,33] and recent technical advances that make these instruments increasingly reliable, portable, and affordable [6,13,21,30]. Despite that, one of the biggest disadvantages of these techniques is the huge amount of data that they collect, which must be processed and analyzed [15,21,24,27,30,34]. That means that supervised (prediction and classification) and unsupervised (clustering) machine learning techniques must be used for processing [14,23,35], and analysis requires the use of data fusion techniques [35,36].
The Brain–Computer Interface (BCI) is also an emerging multidisciplinary technology, where the brain is directly connected to devices [26,37] to measure brain activity [7,24,27,36]. In combination with new sensor technologies, it offers innovative ways for measuring and monitoring student performance, even allowing us to provide neurofeedback to students [2,38].
Other disadvantages of neurophysiological measures are related to the specific characteristics of the novel sensor devices, such as intrusiveness [6,15,22], interference in the classroom [2,15], difficulties of remote application in e-learning students [2], and the high cost of newer technologies [2,16,21]. Technological advances are resolving many of these limitations, but they must still be considered when choosing the most appropriate technique according to the learning conditions and the study objective.
In conclusion, multichannel integration of different neurophysiological records is a challenge and an opportunity for psychology [39], especially in learning contexts [21,38,40]. The simultaneous use of a variety of physiological signals allows us to improve our understanding and knowledge of learning processes with the aim of optimizing results [4,18]. Their advantages include increased reliability of the data obtained on information processing, learning strategies, and student behavior during the performance of a given activity [25,30,31]. This technology is also a great opportunity to provide individualized solutions for each student [2,9,13,14,33].
To the best of our knowledge, this is one of the first studies combining “integrated multichannel records” and “higher education” through a systematic literature review to understand recent developments and the progression of research in the educational field. Therefore, the main objective of this systematic review is to analyze the existing literature on the use of integrated multichannel records in learning studies in higher education during the last 10 years. In this regard, the following research questions are addressed:
  • What is the general state of scientific research on the use of multichannel records in learning studies in higher education?
  • How have these technologies been used over the last 10 years in higher education?
The remainder of the article is structured as follows. Section 2 presents the materials and methodology applied to perform the systematic review. Section 3 presents the results of this study. Section 4 discusses this study, in terms of findings, implications for future research, and limitations of the systematic review. Section 5 provides the conclusions of this study and offers suggestions for further research.

2. Materials and Methods

A systematic review of the literature was performed following an explicit, systematic search strategy with inclusion and exclusion criteria. The review process followed the recommendations of the PRISMA Statement [41].

2.1. Search Strategy

The selected databases were Web of Science (WOS), Scopus, PsycINFO, and Psicodoc. The research string was applied to the four international databases in the fields title, abstract, and keywords (see Table 1). The inquiry process included the keywords presented in Table 1 in the search terms column using “AND” between the terms of the different topics and “OR” between the terms of the same topic. The literature search was conducted between January 2013 and November 2023, with an initial identification of 90 records.
In the second stage, the 90 identified documents were exported to Rayyan meta-analysis software. Rayyan is a free application that helps in the first stages of a systematic review by detecting duplicates in the different databases and assisting in selection, as well as allowing collaboration [42]. The search for duplicates led to the elimination of 18 articles.

2.2. Inclusion and Exclusion Criteria

After eliminating duplicate studies (n = 20), two independent reviewers screened titles and abstracts according to the prespecified inclusion and exclusion criteria (see Table 2). The selected articles were fully evaluated by the same independent reviewers to ensure objectivity and minimize bias during the selection process. It was decided to limit the search to articles published in the last 10 years, from January 2013 to November 2023.

2.3. Methodological Quality Assessment

Quality assessment is an integral part of any systematic review to identify and evaluate all available research evidence relating to the objective of the review [43]. The 10 articles finally included were assessed in terms of methodological quality in order to avoid possible bias. The included studies were critically and independently examined by two reviewers using an eleven-item checklist developed by Aromataris and Munn [44]. The Johanna Briggs Checklist (JBI) is a tool used to evaluate the methodological quality of research studies that consists of a set of criteria used to determine the rigor and validity of a study [45]. The checklist encompassed the following evaluation criteria:
-
The objective of the research is clearly specified;
-
Addresses the use of integrated multichannel records in learning studies;
-
The results are useful for the research community;
-
The authors’ conclusions are supported by the data;
-
Recommendations are made for future research.
No studies were excluded due to quality issues based on the JBI checklist.

2.4. Selection of Studies

A total of 90 records were identified across the 4 electronic databases (WOS, Scopus, PsycINFO, and Psicodoc). After deleting duplicate records (n = 20), the studies were screened by two independent reviewers in the title and abstract phase to evaluate eligibility (n = 70). A total of 59 studies were excluded for not meeting the prespecified inclusion criteria. The same independent reviewers assessed by full text reading the remaining 11 articles. Finally, a total of 10 studies were eligible for inclusion in the review. Figure 1 represents a diagram of the record selection process followed based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [41].

2.5. Data Extraction and Analysis

In order to address the proposed research questions, we carried out content analysis combining qualitative and quantitative approaches to the 10 search studies obtained. Data related to the year and country of publication, progression over time, and participants were analyzed using quantitative methods, such as means and percentages. The quantitative analysis allowed us to visualize and understand general aspects of the topic through descriptive graphs. For the qualitative analysis, we analyzed the contents of the documents considering the objectives, population, neurotechnologies used, metrics used, and main conclusions of each study.

3. Results

Once the studies related to the use of integrated multichannel records in learning studies in higher education were collected, a total of 10 relevant articles were selected for this systematic review. The main characteristics of the studies included in this review are provided in Table 3.
The review shows that 50% of the studies were published in 2023, which is the highest percentage. At the beginning of the period reviewed, in 2013, no articles were published. The oldest article is from 2017, and the number of publications grew between 2017 and 2023.
Half of the studies were published in China [4,46,48,51,53]. Asia is the continent with the largest proportion of publications, with five studies published in China [4,46,49,51,53] and one in Turkey [50], representing 60% of the articles selected in the sample. The second-highest number of publications was in the USA (20%) [47,52]. Finally, two articles in the last 10 years were published in Europe (20%): one in Spain in 2023 [3] and another in Denmark in 2019 [49].
The largest sample in the studies was 78 subjects [49], while the smallest sample had 10 participants [47]. The total number of participants in the selected studies was 416 subjects (M = 41.6, SD = 24.4). The percentage of women in the sample ranged between 20% [47] and 80% [52]. However, half of the studies [3,48,50,51,53] had samples with 50% women.
Table 4 presents the results of this review in terms of objectives, biometric techniques used, cognitive processes analyzed, and main conclusions.
Looking at the methodology used in the selected articles, there were five types of sensor devices used: eye tracking, EEG, GSR, HRV, and fNIR. All of the studies used eye tracking. The second most commonly used measurement technique was EEG [3,4,46,47,48,49,50,51,53], followed by GSR [3,52], HRV [47,52], and, finally, fNIR, which was only used in one study [52]. In terms of multichannel integration of measurements, the reviewed studies used between two and four techniques simultaneously. However, the most frequent combination was using two measurement techniques simultaneously (70%) [3,46,48,49,50,51,53] followed by three measurement techniques (20%) [3,47], and, finally, four techniques simultaneously (10%) [52].
In response to the main objective of this systematic review, Table 5 summarizes the data relating to the simultaneous use of the various neurophysiological records. The most common combination of sensor devices used together was eye tracking and EEG (n = 9). One of the studies used eye tracking, GSR, and EEG [3], another used eye tracking, EEG, and HRV [47], and another used eye tracking, GSR, fNIR, and HRV [52].
In addition, the 10 studies selected in the review examined a total of 11 cognitive processes, which were attention, cognitive load, interest, stress, engagement, cognitive processing, recognition, visual search, reading comprehension, affective responses, and attitudinal and behavioral intention responses. Of the ten selected studies, four had a single cognitive process as the object of study (40%) [4,46,48,53], another four had two cognitive processes as the object of study (40%) [4,46,49,53], one study examined three cognitive processes (10%) [52], and another study simultaneously examined four cognitive processes (10%) [3]. Table 6 summarizes the cognitive processes studied simultaneously in the studies selected for the review.
We performed a qualitative analysis considering the main objectives identified after reading the selected articles to understand current research trends in integrated multichannel records in higher education. The most common objective was to investigate how cognitive processes occur when subjects are faced with certain activities or tasks (60%) [4,48,49,51,52,53]. One study aimed to investigate the integration of EEG and eye tracking [48], another investigated brain interaction patterns [51], and two studies examined cognitive load in multitasking requirements [47] and reading comprehension [53]. Half of the studies focused on online or remote learning through video conferencing [46], multimedia environments [4,49,50], or virtual reality [52]. Additionally, only one study compared online and in-person learning processes [3]. Moreover, none of the selected studies sought to replicate other studies or apply the conclusions of previous studies.
Finally, the study conclusions focused on the specific tasks proposed in this study. Liu et al. [4] concluded that color coding has benefits in multimedia learning, while Zhang and Liu [53] noted a positive effect of multi-screen presentations on students’ comprehension and attention. Cao et al. [46] concluded that the presence of teachers in material designed for remote learning influenced student concentration and attention, while students’ perceived satisfaction was related to their learning. Considering biometric measures, Lim et al. [47] found differences in multitasking requirements and task difficulty in EEG and eye tracking, while HRV only exhibited differences in multitasking requirements, and Slevitch et al. [52] found differences in arousal, immersion, and engagement. Despite all of that, Juárez-Varón et al. [3] concluded that online learning was less effective in terms of brain signals. Furthermore, Makransky et al. [49] noted that subjective and objective measures can provide a better understanding of the mechanisms underlying learning, while Mutlu-Bayraktar et al. [50] confirmed the relationship between fixations and frequency bands but not between self-reported measures and biometric measures. This is consistent with Slevitch et al. [52], who only found differences in self-reporting in the temporal dimension. Other studies on visual tasks found gender differences [51] and a higher level of processing in materials with text than with images [48], although Zhang and Liu [53] concluded that subjects showed greater interest in materials with text or text and images than in just images. In sum, the conclusions of the different studies have direct applications to educational practice.

4. Discussion

We performed a systematic review of integrated multichannel records in learning studies in higher education over the last 10 years. In response to the first research question about the general state of scientific research, half of the studies reviewed were published in the previous year (2023). This review shows an increase in studies over the last 10 years, especially since the COVID-19 pandemic, although Jamil et al. (2021) have already noted a growth in the number of publications from 2011 to 2018. The reasons behind this include improvements in neurophysiological sensors and increased interest in online and hybrid training following the COVID-19 pandemic [2,18,20].
According to Darvishi et al. [6], the most common objective in studies between 2014 and 2018 was to monitor participants. This may have evolved with the incorporation of different neurophysiological metrics seeking deeper knowledge of students’ cognitive processes during the learning process. Our results between 2013 and 2023 in integrated multichannel records indicate that the most common objective is to understand how cognitive processes occur when subjects are faced with certain activities or tasks, regardless of students’ areas of knowledge. Furthermore, Jamil et al. (2021) found an increase in studies with university students since 2012 contributing to the development of BCI and education. That progress has not stopped, as the use of neurophysiological measurement in the context of education is increasing. However, it is still in the early stages of development in terms of its application to higher education and the reproducibility of experiments [6]. In this regard, our review did not find any studies reproducing or applying conclusions from previous research in terms of verifying improvements in learning.
The samples in the studies we reviewed ranged between 10 and 78 subjects, although half of the studies had between 10 and 34. Many authors noted the small number of participants as a limitation in studies with biometric measures [6,22,30,40]. This is also related to other previously noted limitations, such as the majority of subjects not having disabilities [7,16,37] and university students being the largest group being studied [3]. In terms of gender balance in the samples, most had at least half of their sample made up of women [3,46,48,49,50,51,52,53]. Despite this, Quian et al. [51] noted the existence of potential gender differences in visual search tasks. Future studies need to explore this more deeply by differentiating data collected from men and women [11,34].
Regarding the second research question about the use of these technologies over the last 10 years in higher education, the results of our review are consistent with Darvishi et al. [6]. EEG and eye tracking were the most common combination of measurement with neurotechnology devices. This may be because both devices are suitable for face-to-face and online education as they are portable and non-invasive. In contrast, sensors such as the fNRI mean higher costs and are more invasive, which may explain their lower levels of use in educational environments. According to Darvishi et al. [6], the constructs that were researched least between 2014 and 2018 were meta-cognitive related. On the other hand, if we talk about the use of neurophysiological measurements, the most often studied cognitive process was attention [7,12], and regarding affective factors, motivation and interest [7]. In the present review, we also found that cognitive constructs, such as cognitive load and attention, were studied more. Looking at cognitive processes being examined simultaneously, we found only one study in which attention and cognitive load were investigated together [46], although these were the main processes investigated with eye tracking and EEG. Following these indications, both attention and cognitive load will be important factors as online education becomes the norm [7,14,18,20].
Finally, research objectives have evolved. Since COVID-19, there has been a greater specialization of research seeking to understand very specific aspects of learning, with objectives such as recording and analyzing [3], understanding requirements [47], and comparing [50]. This is in contrast to studies before 2020, which had more general objectives, such as relatively superficial examinations [46]. This is reflected in the cognitive processes being studied, with a pattern over recent years of increased attention to cognitive processes related to motivational aspects [3,52].
The review has highlighted some of the future challenges in integrating multichannel records in higher education. Chief among them are the difficulties in analyzing and interpreting the data. In this regard, Darvishi et al. [6] emphasized challenges and concerns related to the accuracy and validity of the constructs captured. Another keenly felt need is research on the intrusiveness of measurement instruments [6]. This requires studies that combine these technologies with qualitative surveys to understand user perceptions and record future improvements to each technology [3]. Lastly, more research in real classrooms is needed to understand the role of joint attention during teaching and learning [3,4,10,13,17,18,20], and more research is needed to detect possible differences between laboratory studies and studies in in-person settings [21] in order to provide evidence on the viability of these metrics.
Following these lines, when it comes to online training, this study provides a basis for future research both in virtual or hybrid educational environments and in the design of teaching materials. The articles included in the review reflect specific aspects that can improve learning in higher education environments, such as color coding [4], the type of teaching materials provided to students [45,52], and the evaluation and planning of the workload in multitasking requirements [47]. Future studies should follow this line of applied research that allows the resulting knowledge and improvements to be transferred to online and in-person pedagogical practices.
In this regard, these instruments are expected to improve through research and technological advances. Their development over recent years and implementation in the educational field demonstrate both their usability and their research possibilities. Furthermore, we expect the development of neurophysiological measurement instruments to continue to improve in terms of suitability for in-person and online use in higher education. In addition, advances in BCI and data mining techniques [12,32] will potentially improve data extraction and analysis of results in the coming years.
It is important to consider this review in light of its limitations. It is possible that potential studies may have been left out of the selection, although using other libraries may only result in duplicate studies. Another potential limitation is the inclusion criteria for this review, which focused only on students in higher education, leaving out studies that included other types of participants, such as school students. Nevertheless, the hybrid method should be used at other educational levels. Additionally, the articles included in this systematic review covered only the last 10 years. Future studies should consider other periods of time or compare the development of the use of integrated multichannel records before and after the COVID-19 pandemic.

5. Conclusions

This systematic review has shown that research on the use of integrated multichannel records in learning studies in higher education is still scarce. This may be due to the notable requirements in terms of data analysis. In any case, the combined use of different neurophysiological devices in education has emerged as an innovative and promising methodology that seeks to improve knowledge about cognitive processes during learning.
In general, integrated recording of eye tracking and electroencephalograms are the most widely used metrics, followed by integrated recording of eye tracking and electrodermal activity. These techniques are minimally intrusive and that advantage, along with recent advances in portability, make them extremely well suited to application in online and in-person classes. Moreover, they offer information on important aspects of virtual platforms, such as visual attention, interest, and motivation.
On the other hand, cognitive load is the learning-related cognitive process that is most commonly studied using integrated multichannel records. Understanding different educational materials’ cognitive loads for each student allows the teacher to improve the design of the materials and personalize them according to each student’s individual learning characteristics. In addition, cognitive load and attention—which was the second most studied process in this review—are key cognitive processes in remote learning.
In this regard, although neurophysiological sensors are not new, recording neurophysiological information through multichannel recordings provides an opportunity to improve knowledge about cognitive processes during learning in order to design innovative educational programs, especially in the online context. Therefore, it is important to continue researching the potential of integrated multichannel records and their application in higher education, both in face-to-face contexts and, especially, in online learning.
This systematic review is a brief update on the use of integrated multichannel records in higher education. It aims to offer a thorough picture of the current situation that can serve as a basis for future studies on learning in higher education. It also provides a summary of practical evidence to aid the design of innovative educational programs, especially in online or hybrid training.

Author Contributions

Conceptualization, I.G.-D., C.V. and M.C.S.-M.; methodology, I.G.-D., C.V. and M.C.S.-M.; software, I.G.-D., C.V. and M.C.S.-M.; validation, I.G.-D., C.V. and M.C.S.-M.; formal analysis, I.G.-D., C.V. and M.C.S.-M.; investigation, I.G.-D., C.V. and M.C.S.-M.; resources, I.G.-D., C.V. and M.C.S.-M.; data curation, I.G.-D., C.V. and M.C.S.-M.; writing—original draft preparation, I.G.-D., C.V. and M.C.S.-M.; writing—review and editing, I.G.-D., C.V. and M.C.S.-M.; visualization, I.G.-D., C.V. and M.C.S.-M.; supervision, I.G.-D., C.V. and M.C.S.-M.; project administration, M.C.S.-M.; funding acquisition, M.C.S.-M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Ministry of Science and Innovation (Spain) [POD2020-117111RB-I00].

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Montenegro-Rueda, M.; Fernández-Cerero, J.; Fernández-Batanero, J.M.; López-Meneses, E. Impact of the implementation of ChatGPT in education: A systematic review. Computers 2023, 12, 153. [Google Scholar] [CrossRef]
  2. Jamil, N.; Belkacem, A.N.; Lakas, A. On enhancing students’ cognitive abilities in online learning using brain activity and eye movements. Educ. Inf. Technol. 2023, 28, 4363–4397. [Google Scholar] [CrossRef] [PubMed]
  3. Juárez-Varón, D.; Bellido-García, I.; Gupta, B.B. Análisis del estrés, atención, interés y conexión emocional en la enseñanza superior presencial y online: Un estudio neurotecnológico. Comun. Rev. Científica Iberoam. Comun. Educ. 2023, 76, 21–34. [Google Scholar] [CrossRef]
  4. Liu, Y.; Ma, W.; Guo, X.; Lin, X.; Wu, C.; Zhu, T. Impacts of color coding on programming learning in multimedia learning: Moving toward a multimodal methodology. Front. Psychol. 2021, 12, 773328. [Google Scholar] [CrossRef] [PubMed]
  5. Pi, Z.; Zhang, Y.; Zhou, W.; Xu, K.; Chen, Y.; Yang, J.; Zhao, Q. Learning by explaining to oneself and a peer enhances learners’ theta and alpha oscillations while watching video lectures. Br. J. Educ. Technol. 2021, 52, 659–679. [Google Scholar] [CrossRef]
  6. Darvishi, A.; Khosravi, H.; Sadiq, S.; Weber, B. Neurophysiological measurements in higher education: A systematic literature review. Int. J. Artif. Intell. Educ. 2022, 32, 413–453. [Google Scholar] [CrossRef]
  7. Jamil, N.; Belkacem, A.N.; Ouhbi, S.; Guger, C. Cognitive and affective brain–computer interfaces for improving learning strategies and enhancing student capabilities: A systematic literature review. IEEE Access 2021, 9, 134122–134147. [Google Scholar] [CrossRef]
  8. Weber, B.; Fischer, T.; Riedl, R. Brain and automatic nervous system activity measurement in software engineering: A systematic literature revier. J. Syst. Softw. 2021, 178, 110946. [Google Scholar] [CrossRef]
  9. Alemdag, E.; Cagiltay, K. A systematic review of eye tracking research on multimedia learning. Comput. Educ. 2018, 125, 413–428. [Google Scholar] [CrossRef]
  10. Jarodzka, H.; Skuballa, I.; Gruber, H. Eye-tracking in educational practice: Investigating visual perception underlying teaching and learning in the classroom. Educ. Psychol. Rev. 2021, 33, 1–10. [Google Scholar] [CrossRef]
  11. Cuesta-Cambra, U.; Niño-González, J.I.; Rodríguez-Terceño, J.M. El procesamiento cognitivo en una app educativa con electroencefalograma y Eye Tracking. Comunicar 2017, 52, 41–50. [Google Scholar] [CrossRef]
  12. González-Diez, I.; Varela, C.; Sáiz-Manzanares, M.C. Use of Eye-Tracking Methodology for Learning in College Students: Systematic Review of Underlying Cognitive Processes. In Lecture Notes in Networks and Systems (LNNS); García Bringas, P., Pérez García, H., Martínez de Pisón, F.J., Martínez Álvarez, F., Troncoso Lora, A., Herrero, A., Calvo Rolle, J.L., Quintián, H., Corchado, E., Eds.; Springer: Cham, Switzerland, 2023; Volume 748, pp. 1–15. [Google Scholar] [CrossRef]
  13. Francisti, J.; Balogh, Z.; Reichel, J.; Magdin, M.; Koprda, Š.; Molnár, G. Application experiences using IoT devices in education. Appl. Sci. 2020, 10, 7286. [Google Scholar] [CrossRef]
  14. Sáiz-Manzanares, M.C.; Marticorena-Sánchez, R.; Martín-Antón, L.J.; González-Díez, I.; Carbonero-Martín, M.Á. Using Eye Tracking Technology to Analyse Cognitive Load in Multichannel Activities in University Students. Int. J. Hum.-Comput. Interact. 2023, 2023, 1–19. [Google Scholar] [CrossRef]
  15. Gui, Q.; Ruiz-Blondet, M.V.; Laszlo, S.; Jin, Z. A survey on brain biometrics. ACM Comput. Surv. 2019, 51, 1–38. [Google Scholar] [CrossRef]
  16. Tandle, A.L.; Joshi, M.S.; Dharmadhikari, A.S.; Jaizwall, S.W. Mental state and emotion detection from musically stimulated EEG. Brain Inf. 2018, 5, 14. [Google Scholar] [CrossRef]
  17. Kim, H.; Chae, Y.; Kim, S.; Im, C.H. Development of a Computer-Aided Education System Inspired by Face-to-Face Learning by Incorporating EEG-Based Neurofeedback Into Online Video Lectures. IEEE Trans. Learn. Technol. 2022, 16, 78–91. [Google Scholar] [CrossRef]
  18. Ramírez-Moreno, M.A.; Díaz-Padilla, M.; Valenzuela-Gómez, K.D.; Vargas-Martínez, A.; Tudón-Martínez, J.C.; Morales-Menendez, R.; Ramírez-Mendoza, R.A.; Pérez-Henríquez, B.L.; Lozoya-Santos, J.J. Eeg-based tool for prediction of university students’ cognitive performance in the classroom. Brain Sci. 2021, 11, 698. [Google Scholar] [CrossRef] [PubMed]
  19. Ko, L.W.; Komarov, O.; Hairston, W.D.; Jung, T.P.; Lin, C.T. Sustained attention in real classroom settings: An EEG study. Front. Hum. Neurosci. 2017, 11, 388. [Google Scholar] [CrossRef] [PubMed]
  20. Hsu, L. A tale of two classes: Tourism students’ cognitive loads and learning outcomes in face-to-face and online classes. J. Hosp. Leis. Sport Tour. Educ. 2021, 29, 100342. [Google Scholar] [CrossRef]
  21. Dahlstrom-Hakki, I.; Asbell-Clarke, J.; Rowe, E. Showing is knowing: The potential and challenges of using neurocognitive measures of implicit learning in the classroom. Mind Brain Educ. 2019, 13, 30–40. [Google Scholar] [CrossRef]
  22. Örün, Ö.; Akbulut, Y. Effect of multitasking, physical environment and electroencephalography use on cognitive load and retention. Comput. Hum. Behav. 2019, 92, 216–229. [Google Scholar] [CrossRef]
  23. Lin, F.R.; Kao, C.M. Mental effort detection using EEG data in E-learning contexts. Comput. Educ. 2018, 122, 63–79. [Google Scholar] [CrossRef]
  24. Liu, Y.; Ayaz, H.; Shewokis, P.A. Multisubject “learning” for mental workload classification using concurrent EEG, fNIRS, and physiological measures. Front. Hum. Neurosci. 2017, 11, 389. [Google Scholar] [CrossRef]
  25. Zhou, T.; Cha, J.S.; Gonzalez, G.; Wachs, J.P.; Sundaram, C.P.; Yu, D. Multimodal physiological signals for workload prediction in robot-assisted surgery. ACM Trans. Hum.-Robot. Interact. 2020, 9, 1–26. [Google Scholar] [CrossRef]
  26. Xia, K.; Duch, W.; Sun, Y.; Xu, K.; Fang, W.; Luo, H.; Zhang, Y.; Sang, D.; Xu, X.; Wang, F.; et al. Privacy-preserving brain–computer interfaces: A systematic review. IEEE Trans. Comput. Soc. Syst. 2022, 10, 2312–2324. [Google Scholar] [CrossRef]
  27. Khademi, Z.; Ebrahimi, F.; Kordy, H.M. A review of critical challenges in MI-BCI: From conventional to deep learning methods. J. Neurosci. Methods 2023, 383, 109736. [Google Scholar] [CrossRef] [PubMed]
  28. Bolinski, F.; Etzelmüller, A.; De Witte, N.A.; van Beurden, C.; Debard, G.; Bonroy, B.; Cuijpers, P.; Riper, H.; Kleiboer, A. Physiological and self-reported arousal in virtual reality versus face-to-face emotional activation and cognitive restructuring in university students: A crossover experimental study using wearable monitoring. Behav. Res. Ther. 2021, 142, 103877. [Google Scholar] [CrossRef] [PubMed]
  29. Yu, H.; Xu, M.; Xiao, X.; Xu, F.; Ming, D. Detection of dynamic changes of electrodermal activity to predict the classroom performance of college students. Cogn. Neurodynam. 2023, 18, 173–184. [Google Scholar] [CrossRef] [PubMed]
  30. Vanneste, P.; Raes, A.; Morton, J.; Bombeke, K.; Van Acker, B.B.; Larmuseau, C.; Depaepe, F.; Van den Noortgate, W. Towards measuring cognitive load through multimodal physiological data. Cogn. Tech. Work 2021, 23, 567–585. [Google Scholar] [CrossRef]
  31. Jimenez-Molina, A.; Retamal, C.; Lira, H. Using psychophysiological sensors to assess mental workload during web browsing. Sensors 2018, 18, 458. [Google Scholar] [CrossRef] [PubMed]
  32. Sáiz-Manzanares, M.C.; Rodríguez-Díez, J.J.; Marticorena, R.; Zaparaín, M.J.; Cerezo, R. Lifelong Learning from Sustainable Education: An Analysis with Eye Tracking and Data Mining Techniques. Sustainability 2020, 12, 1970. [Google Scholar] [CrossRef]
  33. Sáiz-Manzanares, M.C.; Payo-Hernanz, R.; Zaparaín-Yáñez, M.J.; Andres-López, G.; Marticorena-Sánchez, R.; Calvo-Rodríguez, A.; Martín, C.; Rodríguez-Arribas, S. Eye-tracking Technology and Data-mining Techniques used for a Behavioral Analysis of Adults engaged in Learning Processes. J. Vis. Exp. 2021, 172, e62103. [Google Scholar] [CrossRef]
  34. Gemein, L.A.; Schirrmeister, R.T.; Chrabąszcz, P.; Wilson, D.; Boedecker, J.; Schulze-Bonhage, A.; Hutter, F.; Ball, T. Machine-learning-based diagnostics of EEG pathology. NeuroImage 2020, 220, 117021. [Google Scholar] [CrossRef] [PubMed]
  35. Sáiz-Manzanares, M.C.; Marticorena, R.; Arnáiz González, Á. Improvements for therapeutic intervention from the use of web applications and machine learning techniques in different affectations in children aged 0–6 years. Int. J. Environ. Res. Public Health 2022, 19, 6558. [Google Scholar] [CrossRef] [PubMed]
  36. Cheng, S.; Wang, J.; Zhang, L.; Wei, Q. Motion imagery-BCI based on EEG and eye movement data fusion. IEEE Trans. Neural Syst. Rehabil. Eng. 2020, 28, 2783–2793. [Google Scholar] [CrossRef] [PubMed]
  37. Maslova, O.; Komarova, Y.; Shusharina, N.; Kolsanov, A.; Zakharov, A.; Garina, E.; Pyatin, V. Non-invasive EEG-based BCI spellers from the beginning to today: A mini-review. Front. Hum. Neurosci. 2023, 17, 1216648. [Google Scholar] [CrossRef] [PubMed]
  38. Xia, Q.; Chiu, T.K.; Li, X. A scoping review of BCIs for learning regulation in mainstream educational contexts. Behav. Inf. Technol. 2023, 2023, 1–22. [Google Scholar] [CrossRef]
  39. Hallett, M.; de Haan, W.; Deco, G.; Dengler, R.; Di Iorio, R.; Gallea, C.; Gerloff, C.; Grefkes, C.; Helmich, R.C.; Kringelbach, M.L.; et al. Human brain connectivity: Clinical applications for clinical neurophysiology. Clin. Neurophysiol. 2020, 131, 1621–1651. [Google Scholar] [CrossRef]
  40. Xu, J.; Zhong, B. Review on portable EEG technology in educational research. Comput. Hum. Behav. 2018, 81, 340–349. [Google Scholar] [CrossRef]
  41. Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Int. J. Surg. 2021, 88, 105906. [Google Scholar] [CrossRef] [PubMed]
  42. Ouzzani, M.; Hammady, H.; Fedorowicz, Z.; Elmagarmid, A. Rayyan-a web and mobile app for systematic reviews. Syst. Rev. 2016, 5, 210. [Google Scholar] [CrossRef] [PubMed]
  43. Whiting, P.; Rutjes, A.W.; Reitsma, J.B.; Bossuyt, P.M.; Kleijnen, J. The development of QUADAS: A tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med. Res. Methodol. 2003, 3, 25. [Google Scholar] [CrossRef] [PubMed]
  44. Aromataris, E.; Munn, Z. JBI Manual of Evidence Synthesis; Joanna Briggs Institute: Adelaide, Australia, 2020. [Google Scholar]
  45. Munn, Z.; Tufanaru, C.; Aromataris, E. JBI’s systematic reviews: Data extraction and synthesis. AJN Am. J. Nurs. 2014, 114, 49–54. [Google Scholar] [CrossRef] [PubMed]
  46. Cao, X.; Cheng, M.; Xue, X.; Zhu, S. Effects of lecture video types on student learning: An analysis of eye-tracking and electroencephalography data. In Advances in Intelligent, Interactive Systems and Applications, Proceedings of the 3rd International Conference on Intelligent, Interactive Systems and Applications (IISA2018), HongKong, China, 29–30 June 2018; Xhafa, F., Patnaik, S., Tavana, M., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 498–505. [Google Scholar] [CrossRef]
  47. Lim, C.; Barragan, J.A.; Farrow, J.M.; Wachs, J.P.; Sundaram, C.P.; Yu, D. Physiological Metrics of Surgical Difficulty and Multi-Task Requirement during Robotic Surgery Skills. Sensors 2023, 23, 4354. [Google Scholar] [CrossRef] [PubMed]
  48. Luo, X.; Lin, Y.; Guo, R.; Gao, X.; Zhang, S. ERP and Pupillometry Synchronization Analysis on Rapid Serial Visual Presentation of Words, Numbers, Pictures. IEEE Trans. Neural Syst. Rehabil. Eng. 2023, 31, 1933–1942. [Google Scholar] [CrossRef] [PubMed]
  49. Makransky, G.; Terkildsen, T.S.; Mayer, R.E. Role of subjective and objective measures of cognitive processing during learning in explaining the spatial contiguity effect. Learn. Instr. 2019, 61, 23–34. [Google Scholar] [CrossRef]
  50. Mutlu-Bayraktar, D.; Ozel, P.; Altindis, F.; Yilmaz, B. Relationship between objective and subjective cognitive load measurements in multimedia learning. Interact. Learn. Environ. 2023, 31, 1322–1334. [Google Scholar] [CrossRef]
  51. Qian, L.; Ge, X.; Feng, Z.; Wang, S.; Yuan, J.; Pan, Y.; Shi, H.; Xu, J.; Sun, Y. Brain Network Reorganization During Visual Search Task Revealed by a Network Analysis of Fixation-Related Potential. IEEE Trans. Neural Syst. Rehabil. Eng. 2023, 31, 1219–1229. [Google Scholar] [CrossRef] [PubMed]
  52. Slevitch, L.; Chandrasekera, T.; Mejia-Puig, L.; Korneva, K.; Akosa, J.S. Virtual Reality images’ impact on cognition and affect in hotel promotions: Application of self-reported and psycho-physiological measures. J. Hosp. Tour. Manag. 2022, 53, 176–187. [Google Scholar] [CrossRef]
  53. Zhang, X.; Liu, S. Understanding Reading Comprehension in Multi-display Presenting System: Visual Distribution and Cognitive Effect. In Communications in Computer and Information Science; Stephanidis, C., Ed.; Springer: Cham, Switzerland, 2017; Volume 714, pp. 207–214. [Google Scholar] [CrossRef]
Computers 13 00096 g001
Figure 1. Flowchart of the study selection process.
Figure 1. Flowchart of the study selection process.
Computers 13 00096 g001
Table 1. Search strategy.
Table 1. Search strategy.
TopicSearch Terms
Eye tracking“eye tracking” OR “eye-tracking” OR “seguimiento ocular”
Galvanic skin response“GSR” OR “respuesta galvánica de la piel” OR “galvanic skin response” OR “actividad electrodérmica” OR “electrodermal activity” OR “conductancia de la piel” OR “skin conductance”
Electroencephalogram“EEG” OR “electroencefalograma” OR “electroencephalogram” OR “electroencefalografía” OR “electroencephalography”
Higher education“educación superior” OR “higher education” OR “college student*” OR “college” OR “university”
Table 2. Inclusion and exclusion criteria.
Table 2. Inclusion and exclusion criteria.
Inclusion CriteriaExclusion Criteria
Publication periodPublished between 2013 and the present (November 2023)Published before 2013
PopulationHigher education studentsPopulation other than higher education students
MethodologyUse of at least two neurotechnological instruments to extract dataUse of other instruments or just one neurotechnological device
Research topicEducational contextFields other than education
Table 3. Characteristics of the studies included in the review: country and sample.
Table 3. Characteristics of the studies included in the review: country and sample.
Authors/YearCountryNWomen (%)Age
(M, SD/Range)
(Years)
Cao et al. (2019) [46]China6259.7%-
Juárez-Varón et al. (2023) [3]Spain2050%22–25
Lim et al. (2023) [47]United States of America (USA)1020%25 (1.2)
Liu et al. (2021) [4]China4238.1%20.81 (1.13)
Luo et al. (2023) [48]China2050%19–24
Makransky et al. (2019) [49]Denmark7860.3%23.59 (3.46)
19–45
Mutlu-Bayraktar et al. (2023) [50]Turkey2050%20.5 (3.45)
19–34
Quian et al. (2023) [51]China7050%22.4 (2.3)
Slevitch et al. (2022) [52]USA6080%-
Zhang and Liu (2017) [53]China3450%-
M = mean age; SD = standard deviation.
Table 4. Characteristics of the studies included in the review: objective, biometric techniques used, cognitive processes analyzed, and main conclusion.
Table 4. Characteristics of the studies included in the review: objective, biometric techniques used, cognitive processes analyzed, and main conclusion.
Authors/YearGeneral ObjectiveBiometric Techniques UsedCognitive Processes AnalyzedMain Conclusion
Cao et al. (2019) [46]Examining the effect of different lecture video types on student learningEye tracking
EEG		Attention
Cognitive load		The presence of teachers influences student concentration and attention, and perceived satisfaction is related to student learning
Juárez-Varón et al. (2023) [3]Record and analyze the effect of relevant variables in the learning process in in-person and online contextsEye tracking
GSR
EEG		Attention
Interest
Stress
Engagement		Less effectiveness of online learning compared to in-person learning in terms of brain signals
Lim et al. (2023) [47]Understand how multitasking requirements contribute to the prediction of cognitive load in robot-assisted surgery under different task difficultiesEye tracking
EEG
Heart Rate Variability (HVR)		Cognitive loadEEG and eye tracking measures differ in different multitasking difficulties and requirements, but HRV only provides significant differences in multitasking requirements
Liu et al. (2021) [4]Determine the effect of color coding on learning programming in multimedia learningEye tracking
EEG		Cognitive load
Cognitive processing		There are benefits to using color coding when learning programming during multimedia learning
Luo et al. (2023) [48]Investigate the fusion methods between EEG and eye tracking in Rapid Serial Visual Presentations (RSVPs)Eye tracking
EEG		Recognition
Cognitive load		The higher complexity of pictures (words and numbers vs. pictures) required a higher level of cognitive process
Makransky et al. (2019) [49]Investigate the potential of combining subjective and objective measures of learning processes in multimedia learningEye tracking
EEG		Cognitive load
Cognitive processing		Subjective and objective measures of cognitive load can provide different information to test the theoretical mechanisms involved in multimedia learning
Mutlu-Bayraktar et al. (2023) [50]Compare subjective and objective cognitive load measurements in a multimedia learning environmentEye tracking
EEG		Cognitive loadA relationship was found between fixations and EEG frequency bands, but not between self-reported measures and biometric measures
Quian et al. (2023) [51]Investigate brain interaction patterns during the visual search processEye tracking
EEG		Visual search (perception)Potential gender differences in visual search tasks
Slevitch et al. (2022) [52]Provide more empirical evidence and investigate whether more immersive and engaging 360° virtual reality (VR) images would be more effective than static VR images in hotel promotionsEye tracking
Functional Near-Infrared Spectroscopy (fNIR)
GSR
HRV		Cognitive load
Affective responses
Attitudinal and behavioral intention responses		Differences in arousal reflect greater immersion and engagement in 360° VR images, but no differences were found using self-report measures, except in the temporal dimension of cognitive load
Zhang and Liu (2017) [53]Investigate students’ reading comprehension and changes in cognitive load with a multi-screen presentation systemEye tracking
EEG		Reading comprehension
Cognitive load		The multi-screen presentation system has a positive effect on comprehension and attention levels. The text-only and text-image formats attracted more attention and took more time than the image-only format in both presentations
Table 5. Simultaneous use of integrated multichannel records in the selection.
Table 5. Simultaneous use of integrated multichannel records in the selection.
EEG9
GSR22
HRV211
fNRI1011
Eye trackingEEGGSRHRV
Table 6. Cognitive processes investigated simultaneously in the selected studies.
Table 6. Cognitive processes investigated simultaneously in the selected studies.
Cognitive ProcessesAttentionCognitive Load
Cognitive load1-
Stress1
Interest1
Engagement1
Cognitive processing 1
Recognition 1
Visual search--
Reading comprehension 1
Affective responses 1
Attitudinal and behavioral intention responses 1
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

González-Díez, I.; Varela, C.; Sáiz-Manzanares, M.C. The Use of Integrated Multichannel Records in Learning Studies in Higher Education: A Systematic Review of the Last 10 Years. Computers 2024, 13, 96. https://doi.org/10.3390/computers13040096

AMA Style

González-Díez I, Varela C, Sáiz-Manzanares MC. The Use of Integrated Multichannel Records in Learning Studies in Higher Education: A Systematic Review of the Last 10 Years. Computers. 2024; 13(4):96. https://doi.org/10.3390/computers13040096

Chicago/Turabian Style

González-Díez, Irene, Carmen Varela, and María Consuelo Sáiz-Manzanares. 2024. "The Use of Integrated Multichannel Records in Learning Studies in Higher Education: A Systematic Review of the Last 10 Years" Computers 13, no. 4: 96. https://doi.org/10.3390/computers13040096

APA Style

González-Díez, I., Varela, C., & Sáiz-Manzanares, M. C. (2024). The Use of Integrated Multichannel Records in Learning Studies in Higher Education: A Systematic Review of the Last 10 Years. Computers, 13(4), 96. https://doi.org/10.3390/computers13040096

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop