Virtual Reality Applied to Heritage in Higher Education—Validation of a Questionnaire to Evaluate Usability, Learning, and Emotions
1
Facultad de Formación del Profesorado, Universidad de Extremadura, 10004 Cáceres, Spain
2
Escuela de Ingenierías Industriales, Universidad de Extremadura, 06006 Badajoz, Spain
3
Facultad de Educación y Psicología, Universidad de Extremadura, 06006 Badajoz, Spain
*
Author to whom correspondence should be addressed.
Heritage 2024, 7(6), 2792-2810; https://doi.org/10.3390/heritage7060132
Submission received: 8 April 2024
/
Revised: 15 May 2024
/
Accepted: 24 May 2024
/
Published: 28 May 2024
Abstract
:Cultural heritage is one of the areas where Extended Reality is having a significant impact nowadays. Although often associated with entertainment, this technology has enormous educational potential when applied to heritage. Therefore, it is essential to implement monitoring tools in educational practice to assess its actual effectiveness. This article presents the process of generating and validating a statistical data collection instrument developed to evaluate a virtual reality experience created using the archaeological heritage of the ancient Roman city of Augusta Emerita (Mérida, Spain). It can be easily adapted to evaluate similar experiences. The aim is to gauge the effectiveness of these experiences as a didactic resource. The questionnaire was subjected to an evaluation of its three dimensions. Content validity was analyzed through expert judgments, while applicability was tested by students. Finally, a series of statistical tests were conducted to verify construct reliability and internal consistency. Based on the results obtained and cross-referenced with the data provided by the literature, the suitability of this tool for collecting data on usability, learning, and emotions in virtual reality experiences is confirmed.
1. Introduction
The dizzying technological advances of recent years need to be accompanied by practical applications that improve people’s quality of life. In the case of digitization, modeling, and 3D visualization (understood as extended reality), the dissemination of cultural heritage is one of the areas where they are having the greatest impact, especially when applied to historic buildings and sites. But beyond a more playful aspect, the educational potential of extended reality experiences applied to heritage is enormous.
Heritage-related educational programs have experienced a rapid increase worldwide since the 1990s [1,2], although it was not until the beginning of this century that their didactic potential was transferred to the classroom [3]. Thus, museum pieces, historical buildings, traditions, customs, archaeological remains, or popular stories become highly effective instruments in the teaching–learning process [4]. European policies have also helped to expand this concept, especially following the framework agreement signed in Faro (Portugal) in 2005 [5] and ratified in 2021. In harmony with these lines of action, heritage education is gaining importance within the Spanish educational system [6] in such a way that knowledge, respect, valuation, and conservation of heritage are part of the exit profiles of each stage under the new educational law (LOMLOE).
Therefore, it is essential to implement the use of heritage elements, in general, and archaeological heritage, in particular, in subjects like social sciences and history in the degrees of Early Childhood and Primary Education due to its crucial role as one of the most significant pedagogical resources within these disciplines [7] and its contribution to the construction of social and cultural identities [8]. But, following Gómez-Carrasco et al. [9], even today, master classes, textbooks, and rote exams predominate in training future university teachers. In the constant search for improving the teaching–learning process, in the past few decades, various alternatives have been imposed to complement the traditional merely expository classes. It is about adopting an active didactic methodology in which students participate in their self-learning [10]. In this context, information and communication technologies (ICT) become relevant. Their full incorporation into the classroom is one of the most urgent challenges facing the education system [11] to acquire the digital skills demanded in the society of the 21st century [12]. Recently, virtual and augmented reality (VR and AR, respectively) have also begun to be deployed for their possibilities in education [13,14,15,16].
Their use in the field of historical and archaeological heritage, as well as in museums, has become increasingly widespread in recent years due to their well-known capabilities for dissemination. However, the application of these VR experiences in formal educational environments remains limited [17], either because of the cost of the necessary devices or because of the lack of teacher training when implementing these kinds of experiences [18]. Even so, we can find some examples, such as Yildirim et al. [19], who facilitated a VR experience with Primary Education degree students for learning about Islamic culture, or Arias, Egea, and García [20], who presented an immersive virtual reality experience allowing secondary school students to visit the Roman theatre of Cartagena, demonstrating VR’s utility in two distinct educational areas.
Nevertheless, in the design of activities of this type, the didactic approach is sometimes lost, turning these active and innovative methodologies into a mere diversion. So much so that some studies consider the suitability of certain experiences when it comes to producing coherent and structured learning [21]. Similarly, although our literature review on the subject shows that virtual reality experiences are motivating students [16,22], it is necessary to know the type of emotions generated by these immersive experiences [23] and their impact on the teaching–learning process [24].
Therefore, it is essential to implement monitoring tools in our educational practice that allow us to assess from different points of view (physical/psychological effects, usability, learning, emotions...) the real impact that the proposed activities have on students, as we see in Baxter and Hainey ([25]), Lin and Mawela [26], Liritzis et al. [17], Bazargani et al. [27], Paolanti et al. [28], or Yildirim et al. [19].
Based on these premises, the questionnaire, the validation of which is presented in this article, was developed. Its objective is to evaluate the effectiveness of the VR experience that was created specifically by this research group to use the archaeological heritage of the ancient city of Augusta Emerita (Mérida, Spain) as a didactic resource for higher education students. For this purpose, a 3D reconstruction of the Aeneas group, a sculptural group from Roman times whose remains are currently in the National Museum of Roman Art, was used. The 3D Co-ViM research group of the University of Extremadura developed a virtual reality experience based on the digitization of its pieces [29] and the space in which this sculptural group was originally located. It consists of a visit to a virtual museum in which “visitors” can move through its rooms and interact. The first room contains the fragments still preserved of the various statues that made up the group (Aeneas, the main character, a mythical hero who flees Troy accompanied by his father Anchises, who is carried on his shoulders, and his son Ascanius, who is held by his hand). In the second room, it is possible to see its full-size reconstruction and to perceive the sensation of the real height of the group (more than 5 m) when standing next to it. The visit ends with a tour around the place where the three statues would originally be located, a Roman space in the city of Mérida itself, the so-called “marble forum” or forum adiectum, currently not accessible to the public, to contemplate the Aeneas group exposed on what would have been its pedestal [30]
To present the work done, the rest of the article is structured as follows: In Section 2, both the instrument created for data collection, in the form of a questionnaire, and the methodology to be followed to evaluate its reliability and validity through various qualitative and quantitative tests are presented. Subsequently, the results obtained are analyzed to establish the relevance of the questionnaire. The following section will present a discussion of the existing literature on the subject, and some conclusions will be drawn.
2. Materials and Methods
Taking into account the context described above, there is a clear need to develop a series of instruments to measure the effects of the use of virtual reality technology applied in teaching–learning processes of competencies related to history and artistic heritage. In response to this need, a tool has been designed and subjected to a validation process consisting of a questionnaire prior to and a questionnaire after the virtual reality experience briefly described above. The parameters for the design and validation of the questionnaire are described below, following the lines of work set by Flores-Camacho et al. [31] or Pérez-Escoda et al. [32].
2.1. Instrument
2.1.1. Design
The questionnaire has been designed to collect information on three fundamental aspects in relation to the didactic use of virtual reality for the teaching of history: usability of the VR application used, learning of content related to Roman art acquired through the use of VR, and emotions experienced by the participants. Previously, the sociodemographic information necessary for the study was collected. The design of the tool followed the steps described in Figure 1.
The tool was developed ad hoc, based on some previous work that has been used for the analysis of virtual reality experiences applied to the teaching–learning processes of history and historical heritage [33,34,35]. Looking at each of the three main blocks of information collected through this tool, the following references have been particularly considered:
In relation to the usability and the degree of acceptance of the developed VR experience, this research builds upon a series of previous studies dedicated to the analysis of applications developed to support the learning process in different educational areas [19,36,37,38,39,40].
Regarding the learning of history and art skills through the use of VR, key studies include those of Arias Ferrer and collaborators [41], Cózar Gutiérrez and collaborators [42], Velasteguí López [14], and Zapatero [15]. These studies delve into the importance of the use of VR technologies for the teaching of historical and heritage skills.
With regard to emotional performance, numerous studies have analyzed the emotional impact on learning [43,44,45,46,47].
Taking all these previous studies as references, we have developed the data collection instrument presented here, which aims to measure the usefulness of a didactic experience that includes virtual reality for the teaching of historical heritage in the three areas mentioned above: usability of the VR tool, learning performance, and emotional performance.
This process has produced a questionnaire composed of four main blocks that are presented below.
Table 1 shows the sociodemographic information based on the description of questions and response variables.
Below are the items linked to each of the blocks of information that are collected through the tool designed. The responses to these items have been set on a Likert scale of 1–5, with 1 being “I strongly disagree” and 5 “I strongly agree”.
Table 2 presents the items related to the usability of the virtual reality tool used throughout the experience.
Table 3 shows the items through which the learning of historical competencies related to the virtual reality experience has been assessed.
Table 4 assesses the incidence of emotions during the development of the virtual reality didactic experience, as well as the possible causes that provoked students to experience these emotions.
2.1.2. Validation Process
Throughout the process, the tool underwent three main phases of validation:
2.1.3. Sample for the Three Validation Phases
In each of the phases described above, a sample that allows the objectives to be met was used.
Regarding the analysis of the validity of the content through expert judgments, a group of 8 expert judges was selected, in accordance with the needs of the tool designed, and seeking compliance with at least 3 of the following criteria:
- They teach in the educational stage in which the tool is going to be applied, i.e., at the university stage or at the stage immediately prior to that.
- They teach in the areas of “Didactics of Social Sciences, Language and Literature”, or “Systems Engineering and Automation”.
- They have collaborated in scientific publications in the educational field related to the teaching of the areas described above.
- They have carried out didactic experiences linked to the use of new technologies applied to teaching and learning.
In relation to these parameters, the group of expert judges who analyzed the tool was configured as presented in Table 5.
In relation to the testing phase by students, the selected sample was composed of four students belonging to the group with which the tool was tested and who were randomly selected. fulfilling the following criteria:
- They are Early Childhood Education degree students.
- They have used a designed tool.
- They have been selected by a non-probabilistic procedure.
Finally, in relation to the application of statistical tests to check the reliability and internal consistency of the construct, the tool was applied to a sample of 136 students, selected for convenience in a non-probabilistic way, following these criteria:
- They are in the third year of the degree in Early Childhood Education.
- They are studying the subject of Didactics in Social Sciences.
The sample selection and data collection procedure followed the guidelines of the ethics committee of the University of Extremadura, informing the participants of the purposes of the research and treating the information collected anonymously. In addition, a pilot test experience was carried out with 51 teachers in training. The results of this test can be found in Corrales et al. [57]. In the process of collecting information, the channels prescribed by the ethics committee of the University of Extremadura were followed, using informed consent. This process was approved by the ethics committee of this university, with registration number 56//2023.
2.2. Reliability
To verify the reliability of the instrument, Cronbach’s alpha coefficient has been used, which determines the significance of the items.
2.3. Construct Validity
To verify the validity of the construct, once the tool was applied to the experimental group, the data were analyzed with the statistical package “Statistical Package for the Social Sciences” (SPSS) version 27, applying the following tests:
For the relationship between variables and multidimensional value, sample adequacy of Kaiser–Meyer–Olkin (KMO) was analyzed, which allows verifying whether a factor analysis is possible.
Bartlett sphericity to verify the correlation between variables.
3. Results
3.1. Content Reliability Analysis
3.1.1. Expert Judgment
The information collected in the reports of the expert judges, which assesses the content of the items of the questionnaire, is shown in Table 6. In it, the comments of the expert judges in relation to the blocks and items of the questionnaire are ordered, as well as the decisions taken in relation to possible changes in the questionnaire.
To complete the information collected from the expert judges, Aiken’s V statistical test has been performed, and the results are presented in Table 7.
3.1.2. User Testing
In a second phase of the validation process, and with the aim of assessing to what extent the format and language of the tool are adequate for its being understood by the students for whom it is intended, a test was carried out by four students. This sample was selected by a non-probabilistic convenience procedure to meet the characteristics necessary to test the questionnaire, namely, Early Childhood Education degree students, of both genders (two belonging to the male gender and two to the female), of the age corresponding to the academic year (21 years) who did not previously know the experience.
To carry out this process, the selected students were asked to answer the questionnaire and then evaluate it according to the parameters of comprehensibility, adequacy, and approximate response time. The results of this process are shown in Table 8.
3.2. Reliability and Validity of the Construct
3.2.1. Cronbach’s Alpha Test
Table 9 and Table 10 show a very high reliability index after the application of Cronbach’s alpha test (α = 0.868) for the total of the questionnaire items (34 items). In order to assess to what extent each of the items affects this reliability index, the possibility of excluding items that reduce reliability was analyzed. The deletion of items QA10 and QE9 raises the reliability index of the construct to α = 0.88.
3.2.2. Factor Analysis
The KMO test yields a result of KMO = 0.869, as shown in Table 11.
This result, greater than 0.5, provides a value that shows high significance for performing factor analysis. The KMO index takes values between 0 and 1. It presents the following scale as an interpretation guide: <0.5 is unacceptable; 0.5–0.6, bad; 0.6–0.7, moderate; 0.7–0.8, good; and >0.8, excellent [52,53]. The index obtained, KMO = 0.869, allows the performance of factor analysis to check the sphericity of the results.
On the other hand, the significance of Bartlett B = 0.00 allows us to reject the null hypothesis of this test (the variables analyzed are not correlated in the sample), and therefore it can be affirmed that the different variables are sufficiently related.
The results of the factor analysis are shown in Table 12.
4. Discussion
The whole process carried out has allowed us to obtain a series of conclusions about the validity of the content of the tool designed, as well as the reliability and validity of the construct.
The process applied for the validation of the questionnaire coincides with similar processes, such as in the study by Roblero [58] to validate a questionnaire on time management among Mexican students, or the study by Flores-Camacho and collaborators [31] to validate a questionnaire on representations in physics teaching. Pérez-Escoda et al. [32] apply in the validation process a review by 11 experts and the measurement of reliability through Cronbach’s alpha, in addition to other tests, such as factor analysis. The same validation procedures of instruments have been implemented in the Primary Education stage [59], as well as in secondary education [60].
With regard to the validity of the content, the analysis conducted by the expert judges makes it possible to affirm that the content responds to the design intention of the tool. Bearing in mind that the overall assessment made by the expert judges is positive, and once the proposed corrections have been incorporated, it can be affirmed that the content responds to the research objectives. In this sense, several studies support the method of review by expert judges as a method of validation of tools similar to the one presented [58,61,62]. To complete this block, the result of Aiken’s V test (0.81) confirms the high degree of agreement of the judges on the relevance of the content of the questionnaire. This content validity index is considered a useful tool for completing the information of the qualitative analysis of the judges’ ratings, as some studies state [58,63,64].
On the other hand, the tests of validity and reliability of the construct report a high degree of reliability, as indicated by the index α = 0.88. In this sense, some studies, such as that by Barrios and Cosculluela [65] or Herrán Gascón et al. [66], indicate a range between 0.7 and 0.9–0.95 as the optimal level of reliability.
Regarding construct validity, the results of KMO = 0.869 should be interpreted similarly; being greater than 0.5, which is the standard commonly used as a reference [67] and the application of Bartlett’s sphericity test allows us to conclude that the designed questionnaire presents a high degree of validity for the construct.
5. Conclusions
The main objective of this research was to design and validate an instrument for evaluating the effectiveness of the VR experience that was created expressly by this research group to use the archaeological heritage of Mérida as a didactic resource for higher education students. The design process has been satisfactory, obtaining a tool that allows measuring the usability, learning, and emotions of the students participating in the VR experience. The validation process, in turn, has verified the adequacy of the content, and the statistical tests carried out guarantee the reliability and validity of the construct. Among the limitations of this work, it should be mentioned that it was restricted to a pilot test experience. Replication of the experience will be necessary to consolidate the results obtained.
Author Contributions
Conceptualization, M.C.-S., P.M., M.J.M. and E.P.; methodology, M.C.-S.; software, E.P. and P.M.; validation, M.C.-S. and E.P.; formal analysis, M.C.-S. and M.J.M.; investigation, M.C.-S. and M.J.M.; resources, P.M. and E.P.; writing—original draft preparation, M.J.M. and M.C.-S.; writing—review and editing, P.M. and E.P.; visualization, M.J.M. and E.P.; supervision, M.J.M.; project administration, P.M.; funding acquisition, P.M. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by the Consejería de Economía, Ciencia y Agenda Digital (Junta de Extremadura) and the European Regional Development Fund (ERDF) “A way to make Europe”, grant number IB20172, and the Agencia Estatal de Investigación (Ministerio de Ciencia, Innovación y Universidades), grant number PID2020-114954GB-I00.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Data are contained within the article.
Conflicts of Interest
The authors declare no conflicts of interest.
References
- Fontal, O.; Ibañez-Etxeberria, A. Research on Heritage Education. Evolution and Current State through Analysis of High Impact Indicators. Rev. Educ. 2017, 2017, 184–205. [Google Scholar] [CrossRef]
- Simşek, G.; Elitok Kesici, A. Heritage Education for Primary School Children Through Drama: The Case of Aydın, Turkey. Procedia-Soc. Behav. Sci. 2012, 46, 3817–3824. [Google Scholar] [CrossRef]
- Lucas-Palacios, L.; Trabajo-Rite, M.; Delgado-Algarra, E.J. Heritage Education in Initial Teacher Training from a Feminist and Animal Ethics Perspective. A Study on Critical-Empathic Thinking for Social Change. Teach. Teach. Educ. 2023, 129, 104153. [Google Scholar] [CrossRef]
- van Boxtel, C.; Grever, M.; Klein, S. Introduction: The Appeal of Heritage in Education. In Sensitive Pasts: Questioning Heritage in Education; Berghahn Books: New York, NY, USA; Oxford, UK, 2016; pp. 1–18. [Google Scholar] [CrossRef]
- Branchesi, L. Heritage Education for Europe: Outcome and Perspective; Armando Editore: Rome, Italy, 2007. [Google Scholar]
- Castro-Calviño, L.; Rodríguez-Medina, J.; Gómez-Carrasco, C.J.; López-Facal, R. Patrimonializarte: A Heritage Education Program Based on New Technologies and Local Heritage. Educ. Sci. 2020, 10, 176. [Google Scholar] [CrossRef]
- Cuenca-López, J.M.; Martín-Cáceres, M.J.; Estepa-Giménez, J. Teacher Training in Heritage Education: Good Practices for Citizenship Education. Humanit. Soc. Sci. Commun. 2021, 8, 62. [Google Scholar] [CrossRef]
- Marín-Cepeda, S. Modelos de Educación Artística, Inclusiva y Patrimonial. Enlazando El Museo y La Universidad a Través de Los Vínculos. In Espacios Compartidos para la Educación, la Inclusión y el Conocimiento; Ediciones Trea: Gijón, Spain, 2020; pp. 129–146. [Google Scholar]
- Gómez-Carrasco, C.J.; Miralles-Martinez, P.; Fontal, O.; Ibañez-Etxeberria, A. Cultural Heritage and Methodological Approaches—An Analysis through Initial Training of History Teachers (Spain–England). Sustainability 2020, 12, 933. [Google Scholar] [CrossRef]
- Farrujia de la Rosa, A.J.; Martínez-Gil, T.; Gómez, C.M.H.; Sáez-Rosenkranz, I. Designing Heritage Itineraries in Trainee Teachers through Virtual Inter-University and Collaboration Groups: The Examples of Barcelona and La Laguna in Social Sciences Teaching. Front. Educ. 2022, 7, 834373. [Google Scholar] [CrossRef]
- Mohajerzad, H.; Schrader, J. Transfer from Research to Practice—A Scoping Review about Transfer Strategies in the Field of Research on Digital Media. Comput. Educ. Open 2022, 3, 100111. [Google Scholar] [CrossRef]
- Kaufman, K. Information Communication Technology: Challenges & Some Prospects from Pre-Service Education to the Classroom. Mid-Atl. Educ. Rev. 2014, 2, 1. [Google Scholar]
- Nesenbergs, K.; Abolins, V.; Ormanis, J.; Mednis, A. Use of Augmented and Virtual Reality in Remote Higher Education: A Systematic Umbrella Review. Educ. Sci. 2021, 11, 8. [Google Scholar] [CrossRef]
- Velasteguí López, E. El Proceso de Enseñanza Aprendizaje Mediante El Uso de Realidad Virtual. Explor. Digit. 2019, 2, 5–15. [Google Scholar] [CrossRef]
- Zapatero, D. La realidad virtual como recurso y herramienta útil para la docencia y la investigación. TE ET Rev. Iberoam. Tecnol. Educ. Educ. Tecnol. 2011, 6, 17–23. [Google Scholar]
- Kavanagh, S.; Luxton-Reilly, A.; Wuensche, B.; Plimmer, B. A Systematic Review of Virtual Reality in Education. Themes Sci. Technol. Educ. 2017, 10, 85–119. [Google Scholar]
- Liritzis, I.; Volonakis, P.; Vosinakis, S. 3D Reconstruction of Cultural Heritage Sites as an Educational Approach. The Sanctuary of Delphi. Appl. Sci. 2021, 11, 3635. [Google Scholar] [CrossRef]
- Abel, V.R.; Tondeur, J.; Sang, G. Teacher Perceptions about ICT Integration into Classroom Instruction. Educ. Sci. 2022, 12, 609. [Google Scholar] [CrossRef]
- Yildirim, G.; Elban, M.; Yildirim, S. Analysis of Use of Virtual Reality Technologies in History Education: A Case Study. Asian J. Educ. Train. 2018, 4, 62–69. [Google Scholar] [CrossRef]
- Egea-Vivancos, A.; Arias-Ferrer, L. Principles for the Design of a History and Heritage Game Based on the Evaluation of Immersive Virtual Reality Video Games. E-Learn. Digit. Media 2020, 18, 383–402. [Google Scholar] [CrossRef]
- Murillo-Zamorano, L.R.; López Sánchez, J.Á.; Godoy-Caballero, A.L.; Bueno Muñoz, C. Gamification and Active Learning in Higher Education: Is It Possible to Match Digital Society, Academia and Students’ Interests? Int. J. Educ. Technol. High. Educ. 2021, 18, 15. [Google Scholar] [CrossRef]
- Makransky, G.; Borre-Gude, S.; Mayer, R.E. Motivational and Cognitive Benefits of Training in Immersive Virtual Reality Based on Multiple Assessments. J. Comput. Assist. Learn. 2019, 35, 691–707. [Google Scholar] [CrossRef]
- Makransky, G.; Lilleholt, L. A Structural Equation Modeling Investigation of the Emotional Value of Immersive Virtual Reality in Education. Educ. Technol. Res. Dev. 2018, 66, 1141–1164. [Google Scholar] [CrossRef]
- Jiménez-Palacios, R.; López, J.M.C. La enseñanza y aprendizaje de las ciencias sociales a través del patrimonio, videojuegos y emociones. Estudio de caso en un IES de Huelva (España). Panta Rei. Rev. Digit. Hist. Didáctica Hist. 2021, 15, 103–133. [Google Scholar] [CrossRef]
- Baxter, G.; Hainey, T. Student Perceptions of Virtual Reality Use in Higher Education. J. Appl. Res. High. Educ. 2020, 12, 413–424. [Google Scholar] [CrossRef]
- Lin, A.; Mawela, T. Virtual Reality, Augmented Reality and Mixed Reality for Teaching and Learning in Higher Education. In Innovations in Bio-Inspired Computing and Applications; IBICA 2022. Lecture Notes in Networks and Systems, Seattle; Springer: Cham, Switzerland, 2023; Volume 649. [Google Scholar] [CrossRef]
- Bazargani, J.S.; Sadeghi-Niaraki, A.; Choi, S.M. Design, Implementation, and Evaluation of an Immersive Virtual Reality-Based Educational Game for Learning Topology Relations at Schools: A Case Study. Sustainability 2021, 13, 13066. [Google Scholar] [CrossRef]
- Paolanti, M.; Puggioni, M.; Frontoni, E.; Giannandrea, L.; Pierdicca, R. Evaluating Learning Outcomes of Virtual Reality Applications in Education: A Proposal for Digital Cultural Heritage. J. Comput. Cult. Herit. 2023, 16, 36. [Google Scholar] [CrossRef]
- Merchán, P.; Salamanca, S.; Pérez, E.; Adán, A.; Chacón, R.; Domínguez, V. Synthesizing Large and Incomplete Roman Sculptures. Working beyond the Models. In Proceedings of the 15th International Conference on Virtual Systems and Multimedia, Vienna, Austria, 9–12 September 2009. [Google Scholar]
- Pérez, E.; Merchán, M.J.; Moreno, M.D.; Merchán, P.; Salamanca, S. Touring the Forum Adiectum of Augusta Emerita in a Virtual Reality Experience. In Digital Heritage. Progress in Cultural Heritage: Documentation, Preservation, and Protection, Proceedings of the 7th International Conference, EuroMed 2018, Nicosia, Cyprus, 29 October–3 November 2018; Lecture Notes in Computer Science; Ioannides, M., Fink, E., Brumana, R., Patias, P., Doulamis, A., Martins, J., Wallace, M., Eds.; Springer: Cham, Switzerland, 2018; Volume 11196. [Google Scholar] [CrossRef]
- Flores-Camacho, F.; Gallegos-Cázares, L.; Lima González, C. Representaciones En Física: Construcción y Validación de Un Cuestionario Para La Enseñanza Media Superior. Rev. Electrónica Investig. Educ. 2020, 22, e14. [Google Scholar] [CrossRef]
- Pérez-Escoda, N.; Bisquerra, R.; Filella, G.R.; Soldevila, A. Construcción Del Cuestionario de Desarrollo Emocional de Adultos (QDE-A). Rev. Española Orientación Psicopedag. 2010, 21, 367–379. [Google Scholar] [CrossRef]
- Domínguez, P.Á. Museos virtuales de pedagogía, enseñanza y educación: Hacia una didáctica del patrimonio histórico-educativo. Educ. Artística Rev. Investig. (EARI) 2011, 2, 23–27. [Google Scholar]
- Rodriguez Ramirez, M.I. Visita Virtual Interactiva en 360 del Museo de Cera de Bayamo. Master’s Thesis, Universidad de las Tunas, Las Tunas, Cuba, 2020. [Google Scholar]
- Salinas, M.I. Entornos Virtuales De Aprendizaje En La Escuela: Tipos, Modelo Didáctico Y Rol Del Docente. Master’s Thesis, Universidad Católica de Argentina, Buenos Aires, Argentina, 2011. [Google Scholar]
- Alamiri, H.F.H.; Zaid, M.S.A. The Effect of Special Exercises Using Virtual Reality Glasses (3D) in Learning the Stop Attack with the Epee of the Students. Indian J. Public Health Res. Dev. 2019, 10, 430–434. [Google Scholar] [CrossRef]
- Andrade, E.A.O.; De Assis de Oliveira Andrade, F.; Gregório Antunes de Medeiros, I. Use of smartphone and virtual reality and augmented reality glasses in mathematics classes. J. Interdiscip. Debates 2021, 2, 23–27. [Google Scholar] [CrossRef]
- Kamińska, D.; Zwoliński, G.; Laska-Leśniewicz, A. Usability Testing of Virtual Reality Applications—The Pilot Study. Sensors 2022, 22, 1342. [Google Scholar] [CrossRef]
- Özgen, D.S.; Afacan, Y.; Sürer, E. Usability of Virtual Reality for Basic Design Education: A Comparative Study with Paper-Based Design. Int. J. Technol. Des. Educ. 2021, 31, 357–377. [Google Scholar] [CrossRef]
- Parlangeli, O.; Palmieri, S.; Mariani, M.; Tartaglia, R.; Arcangeli, G.; Bagnara, S. Usability Issues When Applying Virtual Reality in Education and Training: A Case Study. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 2000, 44, 5-628–5-631. [Google Scholar] [CrossRef]
- Arias-Ferrer, L.; Egea-Vivancos, A.; Monroy-Hernández, F. Evaluación de recursos audiovisuales para la enseñanza de las Ciencias Sociales en Educación Secundaria. Rev. Fuentes 2019, 1, 25–38. [Google Scholar] [CrossRef]
- Cozar-Gutiérrez, R.; Sáez-López, J.-M. Realidad Aumentada, proyectos en el aula de Primaria: Experiencias y casos en Ciencias Sociales. Edmetic 2017, 6, 165–180. [Google Scholar] [CrossRef]
- Brígido Mero, M.; Bermejo García, M.L.; Conde Núñez, M.C.; Borrachero Cortés, A.B.; Mellado Jiménez, V. Estudio longitudinal de las emociones en ciencias de estudiantes de maestro. Rev. Galego-Port. Psicoloxía Educ. Rev. Estud. Investig. Psicol. Educ. 2010, 18, 161–180. [Google Scholar]
- Mellado, V.; Blanco, J.L.; Borrachero, A.B.; Cárdenas, J.A. Las Emociones en la Enseñanza y el Aprendizaje de las Ciencias y las Matemáticas; Grupo de investigación DEPROFE: Valencia, Spain, 2013. [Google Scholar]
- Jiménez-Palacios, R.; López, J.M.C. The Teaching and Learning of Social Sciences through Heritage, Video Games and Emotions. Case Study at a Secondary School in Huelva (Spain). Panta Rei 2021, 15, 103–133. [Google Scholar] [CrossRef]
- Núñez, M.M.; Borreguero, M.G.M.; Correa, F.L.N. Comparación de Las Emociones, Actitudes y Niveles de Autoeficacia Ante Áreas STEM Entre Diferentes Etapas Educativas. Eur. J. Educ. Psychol. 2020, 13, 251–267. [Google Scholar] [CrossRef]
- Yáñez, R.O.; Valdés-León, G. Emociones, Motivación y Rendimiento Académico: Una Propuesta Para El Desarrollo de Habilidades Orales En Ingeniería Desde La Neuroeducación. Cent. Sur 2020, 4, 252–265. [Google Scholar] [CrossRef]
- Escobar-Pérez, J.; Cuervo-Martínez, Á. Validez de Contenido Y Juicio de Expertos: Una Aproximación a su Utilización. Av. Medición 2008, 6, 27–36. [Google Scholar]
- Galicia Alarcón, L.A.; Balderrama Trápaga, J.A.; Edel Navarro, R. Content Validity by Experts Judgment: Proposal for a Virtual Tool. Apertura 2017, 9, 42–53. [Google Scholar] [CrossRef]
- Robles, P.; del Carmen, M. La Validación por Juicio de Expertos: Dos Investigaciones Cualitativas en Lingüística Aplicada. Rev. Nebrija Lingüística Appl. Enseñanza Leng. 2015, 9, 124–139. [Google Scholar] [CrossRef]
- Laudadío, J.; Mazzitelli, C. Adaptación y Validación del Cuestionario de Relación Docente en el Nivel Superior* Adaptation and Validation of the Questionnaire on Teacher Interaction in Higher Education. Interdiscip. Rev. Psicol. Cienc. Afines = J. Psychol. Relat. Sci. 2018, 35, 153–170. [Google Scholar] [CrossRef]
- Salcines Talledo, I.; González Fernández, N.; Ramírez García, A.; Martínez Mínguez, L. Validación de La Escala de Autopercepción de Competencias Transversales y Profesionales de Estudiantes de Educación Superior. Profr. Rev. Currículum Form. Profr. 2018, 22, 31–51. [Google Scholar] [CrossRef]
- Soler, S.; Soler, L. Usos Del Coeficiente Alfa de Cronbach En El Análisis de Instrumentos Escritos. Rev. Médica Electrónica 2012, 34, 1–6. [Google Scholar]
- Rodríguez-Rodríguez, J.; Reguant-Álvarez, M. Calcular la fiabilidad de un cuestionario o escala mediante el SPSS: El coeficiente alfa de Cronbach. REIRE Rev. D’innovació Recer. Educ. 2020, 13, 1–13. [Google Scholar] [CrossRef]
- Gallego, L.; Araque, O. Variables of Influence in the Learning Capability. An Analysis by Clusters and Main Components. Inf. Tecnol. 2019, 30, 257–264. [Google Scholar] [CrossRef]
- Martínez Castro, J.; Velásquez Rodríguez, C.A. La Validez de Escalas NEP-R y ECS Para Medir Comportamiento y Preocupación Ambiental En Estudiantes Del CUNSURORI, USAC. Rev. Nat. Soc. Ambiente 2020, 7, 37–48. [Google Scholar] [CrossRef]
- Corrales, M.; Rodríguez, F.; Merchán, M.J.; Merchán, P.; Pérez, E. Comparative Analysis between Virtual Visits and Pedagogical Outings to Heritage Sites: An Application in the Teaching of History. Heritage 2024, 7, 366–379. [Google Scholar] [CrossRef]
- Roblero, G. Validation of Time Management Questionnaire in Mexican College Students. Rev. Electron. Investig. Educ. 2020, 22, e01. [Google Scholar] [CrossRef]
- Galafat-Hurtado, Á.; Rute-Vega, N.; Aguaded-Ramírez, E.; Carrillo-Rosúa, J. Validación de Un Registro de Observación, Para Analizar Interacciones de Éxito, En Un Grupo Interactivo En Comunidades de Aprendizaje. Rev. Española Orientación Psicopedag. 2016, 27, 83–98. [Google Scholar] [CrossRef]
- Eaton, I.G.A.; Caso-López, A.A.C.; López, C.D.D. Cuestionario Para Valorar Las Prácticas de Enseñanza En Secundaria Desde La Percepción de Los Estudiantes. Rev. Española Orientación Psicopedag. 2018, 29, 55–70. [Google Scholar] [CrossRef]
- López Dórame, D.; Norzagaray Benítez, C.C. Construcción y Validación de Un Instrumento Para Evaluar Factores Disposicionales Asociados a La Elección de Carrera. Enseñanza Investig. Psicol. 2016, 21, 120–130. [Google Scholar]
- Robles Garrote, P.; Rojas, M. Validation by Expert Judgements: Two Cases of Qualitative Research in Applied Linguistics. Rev. Nebrija Lingüística Apl. 2015, 18, 124–139. [Google Scholar]
- Merino Soto, C.; Livia Segovia, J. Intervalos de Confianza Asimétricos Para El Índice La Validez de Contenido: Un Programa Visual Basic Para La V de Aiken. An. Psicol./Ann. Psychol. 2009, 25, 169–171. [Google Scholar] [CrossRef]
- Robles Pastor, B.F. Índice de Validez de Contenido: Coeficiente V de Aiken. Pueblo Cont. 2018, 29, 193–197. [Google Scholar]
- Barrios, M.C.A. Reliability. In Psychometry; Meneses, J., Ed.; UOC: Barcelona, Italy, 2013; pp. 75–140. [Google Scholar]
- de la Herrán Gascón, A.; Paredes Labra, J.; Monsalve Treskow, D.V. Cuestionario Para La Evaluación de La Educación Inclusiva Universitaria (CEEIU). Rev. Complut. Educ. 2017, 28, 913–928. [Google Scholar] [CrossRef]
- Barinas, G.; Cañada, F.; Costillo, E.; Amórtegui, E. Validación de Un Instrumento de Creencias Sobre Las Ciencias Naturales Escolares En Educación Primaria. Prax. Saber 2022, 13, e14147. [Google Scholar] [CrossRef]
Figure 1.
Tool design procedure. Own elaboration.
Table 1.
Sociodemographic information from the questionnaire. Own elaboration.
| Item | Variable |
|---|---|
| Age | 18–25 years |
| Gender | Woman Man Unspecified |
| Modality of studies completed in baccalaureate | Sciences Social Sciences Humanities Arts |
| University degree | Early Childhood Education Primary Education Other (Specify) |
| Highest course you are enrolled in | 1st 2nd 3rd 4th |
| Subject from which the didactic experience is accessed | Didactics of Social Sciences (Early Childhood Education) Didactics of Social Sciences/of History (Primary Education) |
Table 2.
QU (usability questions) items related to usability. Own elaboration.
| Q | Item | Scale |
|---|---|---|
| QU1 | It’s easy for me to use VR | Likert 1–5 |
| QU2 | I am a regular user of VR tools | Likert 1–5 |
| QU3 | I adapt quickly to the use of VR headsets | Likert 1–5 |
| QU4 | I learn how to use the controllers easily | Likert 1–5 |
| QU5 | I need help from those responsible to carry out the activity | Likert 1–5 |
| QU6 | The explanations of those responsible are clear and precise | Likert 1–5 |
| QU7 | I have no trouble interacting with the sculpture fragments | Likert 1–5 |
| QU8 | I can move between the different rooms and move through them | Likert 1–5 |
| QU9 | I am used to using tools of this type in the classroom | Likert 1–5 |
| QU10 | I have experience using VR technical media to learn history content | Likert 1–5 |
| QU11 | It makes me insecure to work in class with VR tools | Likert 1–5 |
| QU12 | I think I lack technical knowledge to use VR in class | Likert 1–5 |
Table 3.
QL items (learning questions) on the learning achieved in the intervention. Own elaboration.
Table 3.
QL items (learning questions) on the learning achieved in the intervention. Own elaboration.
| Q | Item | Scale |
|---|---|---|
| QL1 | I am very interested in history content | Likert 1–5 |
| QL2 | The contents of Art interest me a lot for my preparation as a teacher | Likert 1–5 |
| QL3 | I think virtual reality can be a useful tool for learning history | Likert 1–5 |
| QL4 | I regularly use VR resources to learn history | Likert 1–5 |
| QL5 | I think this VR experience helps me better understand concepts linked to history. | Likert 1–5 |
| QL6 | I think this VR experience helps me better understand concepts related to art from different periods of history. | Likert 1–5 |
| QL7 | I think this VR experience makes me more motivated to learn history content. | Likert 1–5 |
| QL8 | I think this VR experience makes me like the class more. | Likert 1–5 |
| QL9 | After the explanation about the reconstruction of the group, I got an idea of how big it could be | Likert 1–5 |
| QL10 | I was surprised by the size of the sculptures when I saw them in the VR experience. | Likert 1–5 |
| QL11 | I think that being able to pick up, rotate, and observe the different fragments in this way has made me pay more attention to the pieces than if I saw them in a museum. | Likert 1–5 |
| QL12 | I think that the fact of seeing in VR the sculptural group inside the forum adiectum may make me associate it with the Aeneas group and its location when I visit Merida. | Likert 1–5 |
| QL13 | Do you know where the masterpiece that copies the Aeneas group you have seen was originally located? | Likert 1–5 |
Table 4.
QE items (emotion questions) about the emotions experienced in the didactic intervention. Own elaboration.
Table 4.
QE items (emotion questions) about the emotions experienced in the didactic intervention. Own elaboration.
| Q | Item | Variable |
|---|---|---|
| QE1 | Nervousness | Likert 1–5 |
| QE2 | Fear | Likert 1–5 |
| QE3 | Joy | Likert 1–5 |
| QE4 | Frustration | Likert 1–5 |
| QE5 | Curiosity | Likert 1–5 |
| QE6 | Concern | Likert 1–5 |
| QE7 | Surprise | Likert 1–5 |
| QE8 | Enthusiasm | Likert 1–5 |
| QE9 | Boredom | Likert 1–5 |
| QE10 | Anxiety | Likert 1–5 |
| QE11 | Causes Positive emotions | Being able to put on virtual reality glasses. Being able to move the pieces. Being able to see the full-scale reconstruction of the Aeneas group. The recreation of the forum adiectum. The location of the Aeneas group in the forum adiectum. I love History. I love Technology. I love Art. |
| QE12 | Causes negative emotions | Being able to put on virtual reality glasses. Being able to move the pieces. Being able to see the full-scale reconstruction of the Aeneas group. The recreation of the forum adiectum. The location of the Aeneas group in the forum adiectum. I love History. I love Technology. I love Art. |
Table 5.
Compliance with selection criteria of expert judges for content validation. Own elaboration.
Table 5.
Compliance with selection criteria of expert judges for content validation. Own elaboration.
| Judge | Criterion 1 | Criterion 2 | Criterion 3 | Criterion 4 |
|---|---|---|---|---|
| Judge 1 | University teaching | Engineering | Yes | No |
| Judge 2 | University teaching | Engineering | Yes | No |
| Judge 3 | University teaching | Engineering | Yes | Yes |
| Judge 4 | University teaching | Social Sciences | Yes | Yes |
| Judge 5 | University teaching | Social Sciences | Yes | Yes |
| Judge 6 | University teaching | Engineering | Yes | Yes |
| Judge 7 | Pre-university teaching | Social sciences | Yes | Yes |
| Judge 8 | Pre-university teaching | Experimental Sciences | Yes | Yes |
Table 6.
Synthesis of proposals by expert judges and action taken. Own elaboration.
| Block | Item | Judge | Suggestion | Action Taken |
|---|---|---|---|---|
| Sociodemographic information | Age | 5 | Add the option “under 18 years” | Proposal accepted |
| Gender | 7 | Proposes to state as “male/female/-” | No | |
| Usability (QU) | QU2 and QU3 | 1 | It is proposed to change the order QU2 and QU3 | Order is maintained because it is understood that the current order is more progressive |
| QU9 and Q10 | 3 and 8 | The similarity between the two items is detected | They are maintained since QU10 introduces the nuance of the History subjects, absent in QU9 | |
| Learning (QL) | QL1 | 1 and 8 | In the item “The contents of Art interest me a lot for my preparation as a teacher”. It is possible to delete the word “a lot” | Proposal accepted |
| QL8 | 6 | In the item “I think this VR experience makes me like the class more”. “The Classes” is proposed | Proposal accepted | |
| QL 11 | 2 | Better ordering actions: instead of ‘pick up, rotate and observe’ it represents a more logical sequence ‘observe, pick up and rotate’. | Not accepted because it is the logical order. A piece can be seen better because you can pick it up and rotate it. | |
| Emotions | QE Block | 6 | Missing an emotion that is usually put as basic: rejection | This emotion is added to the whole block |
| QE12 | 3 and 4 | Restructuring of the presentation of negative emotions is proposed: - ‘Being able to move the pieces’ → Not being able to move the pieces as I would like - Being able to wear VR headsets → Put on VR headsets. “To be able to see the location of the Aeneas’ group in the forum adiectum”, instead of “To be able to see the full-scale reconstruction of the Aeneas’ group” | Proposal accepted | |
| QE12 | 5 and 8 | It is proposed not to make the response to negative emotions mandatory | It is accepted, to facilitate the completion of the information by those participants who do not experience any emotion of this type | |
| QE11 and QE12 | 2 | It is proposed not to repeat all the options in the possible causes of positive and/or negative emotions | It remains as it was, after the modifications previously accepted. | |
| QE12 | 7 | To facilitate the task of differentiating the positive from the negative by the participant, it is proposed to detail this in the statement of the question. Example: If you have felt such, such and such... What do you think the reasons are? | Guidelines are introduced about the distinction between both types of emotions in the explanations of Items QE11 and QE12 | |
| Questionnaire in general | 4 | It is proposed to improve the didactic approach: to include a section so that they themselves have the option to reflect on the didactic usefulness of this experience, what they would change, and what other approach they would take. | It is positively valued for future uses of the questionnaire, but it is not taken into account for this study, as it is not within the research objectives. | |
| Other aspects | QU, QA, and QE blocks | 4 | Clarify acronyms such as RV. | Proposal accepted |
| 6 | It is proposed to specify the value of the items in the Likert scale: 1. I strongly disagree (as it was) 2. I somewhat agree. 3. I agree 4. I pretty much agree 5. I strongly agree (as it was) | Proposal accepted |
Table 7.
Aiken’s V test results data. Own elaboration.
| Item | CI 95% Inf | CI 95% Sup | V | |
|---|---|---|---|---|
| QU1 | 0.77 | 0.91 | 0.86 | |
| QU2 | 0.79 | 0.93 | 0.88 | |
| QU3 | 0.76 | 0.91 | 0.84 | |
| QU4 | 0.78 | 0.92 | 0.87 | |
| QU5 | 0.72 | 0.88 | 0.81 | |
| QU6 | 0.81 | 0.94 | 0.89 | |
| QU7 | 0.79 | 0.93 | 0.88 | |
| QU8 | 0.65 | 0.82 | 0.76 | |
| QU9 | 0.69 | 0.86 | 0.79 | |
| QU10 | 0.79 | 0.93 | 0.88 | |
| QU11 | 0.78 | 0.92 | 0.87 | |
| QU12 | 0.79 | 0.93 | 0.88 | |
| QA1 | 0.77 | 0.91 | 0.86 | |
| QA2 | 0.79 | 0.93 | 0.88 | |
| QA3 | 0.79 | 0.93 | 0.88 | |
| QA4 | 0.65 | 0.82 | 0.76 | |
| QA5 | 0.79 | 0.93 | 0.88 | |
| QA6 | 0.79 | 0.93 | 0.88 | |
| QA7 | 0.78 | 0.92 | 0.87 | |
| QA8 | 0.77 | 0.91 | 0.86 | |
| QA9 | 0.79 | 0.93 | 0.88 | |
| QA10 | 0.79 | 0.93 | 0.88 | |
| QA11 | 0.78 | 0.92 | 0.87 | |
| QA12 | 0.77 | 0.91 | 0.86 | |
| QA13 | 0.65 | 0.82 | 0.76 | |
| QE1 | 0.78 | 0.92 | 0.87 | |
| QE2 | 0.79 | 0.93 | 0.88 | |
| QE3 | 0.79 | 0.93 | 0.88 | |
| QE4 | 0.65 | 0.82 | 0.76 | |
| QE5 | 0.77 | 0.91 | 0.86 | |
| QE6 | 0.77 | 0.91 | 0.86 | |
| QE7 | 0.79 | 0.93 | 0.88 | |
| QE8 | 0.78 | 0.92 | 0.87 | |
| QE9 | 0.79 | 0.93 | 0.88 | |
| QE10 | 0.81 | 0.94 | 0.89 | |
| QE11 | 0.78 | 0.92 | 0.87 | |
| QE12 | 0.81 | 0.94 | 0.89 | |
| TOTAL | 0.81 |
Table 8.
Information collected through student testing. Own elaboration.
| Question | Student | Contribution | Action Taken |
|---|---|---|---|
| Overall rating | Student 1 | I think the questionnaire is very complete and serves to assess the usefulness of the VR experience in several dimensions. | None |
| Student 2 | I found the questionnaire very useful, because it takes into account not only whether we have learned from the experience of the “Aeneas group”, but also about how the VR headsets are used, and the emotions we have had. | None | |
| Student 3 | I found both the experience and the questionnaire interesting and innovative. | None | |
| Student 4 | I found it a very complete questionnaire, to know if we handle VR tools well and if we learn with them | None | |
| Do you think it can be used to know information about VR usability? | Student 1 | Yes | None |
| Student 2 | Yes | None | |
| Student 3 | Yes, because by asking before and after, you can see if our perception of VR has changed. | Take into account the importance of performing pre-test/post-test | |
| Student 4 | I think it can be useful, but it is important the initial explanation to understand everything well | Keep in mind that contextualization is necessary for the experience and data collection to be successful. | |
| Do you think it can be used to know information about learning about CC SS VR? | Student 1 | Yes, I think it is appropriate | None |
| Student 2 | I think it can be useful, especially with the previous explanation | Give importance to contextualization | |
| Student 3 | It seems to me that it can allow us to better understand what Roman sculpture was like. | None | |
| Student 4 | Yes, especially to learn content related to the sculpture of the ancient era | None | |
| Do you think it can be used to know information about emotions experienced during a VR activity? | Student 1 | Yes. I find this aspect very interesting in relation to the subsequent questionnaire. | Take into account this contribution in the data analysis |
| Student 2 | Yes, because during the experience you have contradictory emotions | None | |
| Student 3 | |||
| Student 4 | |||
| Have you understood the scale of 1 to 5 in the questions? | Unanimous response | Yes, it is well understood | None |
| Are there any questions or expressions you don’t understand? | Unanimous response | No | |
| Would you delete any items? | Unanimous response | No | None |
| Would you add any items? | Unanimous response | No | None |
| Was it easy for you to answer the questionnaire through the Google Form tool? | Student 3 | I had some difficulty reading the QR code with my mobile | Have prepared some alternative methods of access to the questionnaire |
| Other students | It’s been easy. | None |
Table 9.
Application of Cronbach’s alpha test on the sample. Own elaboration.
| N | % | ||
|---|---|---|---|
| Cases | Valid | 136 | 100.0 |
| Excluded | 0 | 0.0 | |
| Total | 136 | 100.0 |
Table 10.
Results of α with all the items of the questionnaire, and after the elimination of QA10 and QE9. Own elaboration.
Table 10.
Results of α with all the items of the questionnaire, and after the elimination of QA10 and QE9. Own elaboration.
| Reliability Statistics | |
|---|---|
| Cronbach’s Alpha | N of Elements |
| 0.868 | 34 |
| 0.880 | 32 |
Table 11.
Result of KMO and Bartlett. Own elaboration.
| KMO and Bartlett Test | ||
|---|---|---|
| Kaiser–Meyer–Olkin measurement of sampling adequacy | 0.869 | |
| Bartlett’s sphericity test | Approx. Chi-square | 2297.245 |
| Gl | 528 | |
| Gis. | 0.000 | |
Table 12.
Results of factor analysis. Own elaboration.
| Total Variance Explained | ||||||
|---|---|---|---|---|---|---|
| Component | Initial Eigenvalues | Sums of Loads Squared from Extraction | ||||
| Total | % Variance | Cumulative % | Total | % Variance | Cumulative % | |
| 1 | 8.497 | 25.749 | 25.749 | 8.497 | 25.749 | 25.749 |
| 2 | 3.519 | 1.663 | 36.412 | 3.519 | 10.663 | 36.412 |
| 3 | 2.978 | 9.025 | 45.437 | 2.978 | 9.025 | 45.437 |
| 4 | 1.804 | 5.465 | 50.902 | 1.804 | 5.465 | 50.902 |
| 5 | 1.484 | 4.496 | 55.398 | 1.484 | 4.496 | 55.398 |
| 6 | 1.371 | 4.154 | 59.552 | 1.371 | 4.154 | 59.552 |
| 7 | 1.196 | 3.626 | 63.178 | 1.196 | 3.626 | 63.178 |
| 8 | 1.143 | 3.463 | 66.641 | 1.143 | 3.463 | 66.641 |
| 9 | 0.936 | 2.836 | 69.477 | |||
| 10 | 0.872 | 2.644 | 72.120 | |||
| 11 | 0.811 | 2.459 | 74.579 | |||
| 12 | 0.801 | 2.428 | 77.007 | |||
| 13 | 0.741 | 2.245 | 79.252 | |||
| 14 | 0.681 | 2.063 | 81.315 | |||
| 15 | 0.586 | 1.775 | 83.091 | |||
| 16 | 0.573 | 1.735 | 84.826 | |||
| 17 | 0.494 | 1.498 | 86.324 | |||
| 18 | 0.470 | 1.425 | 87.749 | |||
| 19 | 0.441 | 1.337 | 89.086 | |||
| 20 | 0.399 | 1.209 | 90.295 | |||
| 21 | 0.378 | 1.144 | 91.439 | |||
| 22 | 0.361 | 1.095 | 92.535 | |||
| 23 | 0.353 | 1.071 | 93.606 | |||
| 24 | 0.332 | 1.006 | 94.611 | |||
| 25 | 0.290 | 0.878 | 95.490 | |||
| 26 | 0.281 | 0.851 | 96.341 | |||
| 27 | 0.234 | 0.710 | 97.051 | |||
| 28 | 0.216 | 0.653 | 97.704 | |||
| 29 | 0.199 | 0.603 | 98.307 | |||
| 30 | 0.163 | 0.494 | 98.801 | |||
| 31 | 0.148 | 0.449 | 99.249 | |||
| 32 | 0.130 | 0.393 | 99.643 | |||
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Corrales-Serrano, M.; Merchán, P.; Merchán, M.J.; Pérez, E. Virtual Reality Applied to Heritage in Higher Education—Validation of a Questionnaire to Evaluate Usability, Learning, and Emotions. Heritage 2024, 7, 2792-2810. https://doi.org/10.3390/heritage7060132
AMA Style
Corrales-Serrano M, Merchán P, Merchán MJ, Pérez E. Virtual Reality Applied to Heritage in Higher Education—Validation of a Questionnaire to Evaluate Usability, Learning, and Emotions. Heritage. 2024; 7(6):2792-2810. https://doi.org/10.3390/heritage7060132
Chicago/Turabian StyleCorrales-Serrano, Mario, Pilar Merchán, María José Merchán, and Emiliano Pérez. 2024. "Virtual Reality Applied to Heritage in Higher Education—Validation of a Questionnaire to Evaluate Usability, Learning, and Emotions" Heritage 7, no. 6: 2792-2810. https://doi.org/10.3390/heritage7060132
APA StyleCorrales-Serrano, M., Merchán, P., Merchán, M. J., & Pérez, E. (2024). Virtual Reality Applied to Heritage in Higher Education—Validation of a Questionnaire to Evaluate Usability, Learning, and Emotions. Heritage, 7(6), 2792-2810. https://doi.org/10.3390/heritage7060132
