1. Introduction
Technologies such as virtual reality (VR) and augmented reality (AR) have a great potential for museums and cultural heritage destinations. In fact, in recent years they are increasingly being applied to enhance the visit experience in different tourism and cultural destinations [
1], including museums [
2,
3,
4,
5,
6], art galleries [
7], cultural heritage destinations [
8,
9,
10], and urban heritage sites [
11,
12]. VR/AR is mainly applied in these sectors for marketing purposes, followed by educational and tourism experience enhancement goals [
13]. The use of VR and AR technologies in these contexts has several positive consequences for the visitors, such as enhanced learning outcomes (better understanding and knowledge, skills, creativity…), global satisfaction with the experience, and positive emotional responses, such as pleasure and enjoyment [
1].
1.1. Museums’ Didactic Tasks
The two main goals of museums are to acquire and conserve cultural heritage, and to perform pedagogical work, which has a positive influence on the evolution of society [
14]. This didactic perspective is highlighted by the museum definition from the International Council of Museums (ICOM) [
15], which states that museums “operate and communicate ethically, professionally and with the participation of communities, offering varied experiences for education, enjoyment, reflection and knowledge sharing”.
Both VR and AR can be used to provide visitors with an interactive virtual experience that can help to improve the learning outcomes of a museum visit. They can be used to access artifacts that are not physically present in the museum or to provide additional data and explanations about those that are exhibited [
16]. Additionally, these technologies can be used to provide access to protected or inaccessible places or artifacts [
17].
In most cases, just one of the technologies is incorporated into the museum experience. However, there are some cases that made use of both technologies, taking advantage of the specific affordances of each technology to generate a globally improved experience (i.e., [
18]). In the next paragraphs, the kinds of experiences for museums that can be generated with each of the technologies are described.
1.2. VR-Based Didactic Experiences in Museums
VR is a technology that immerses participants in a computer-generated virtual world with which they can interact in real time [
19]. The system does not allow users to visualize the real world during the experience. Consequently, this technology can be useful for experiences where the goal is to virtually transport the participant to another location different from the real one.
VR technology has applications in different contexts. One of the more relevant for the purposes of the current study is in education. Virtual worlds have a great pedagogical potential and can technologically provide an immersive learning experience [
20]. Immersion and presence in virtual environments seem to have positive effects on learning outcomes [
21] and there is a growing interest in applying VR in educational settings. However, the technology is not yet integrated in the academic context and it is not applied regularly in the classroom. Most of the previous studies about its application in higher education were conducted in controlled settings as part of an experimental work [
22].
This pedagogical role of VR is also relevant for its use in museum experiences. In some cases, VR provides a virtual visit to parts of the museum that are not easily accessible, especially for specific users, such as the elderly. This approach was followed in the underground mines of the Geevor Tin Mine Museum [
23].
In other cases, VR is used to provide new experiences that complement the museum experience. For example, The Dalí Museum visit includes a virtual reality experience (Dreams of Dalí) that shows visitors a virtual environment that makes them believe that they are inside one of the paintings and interact with parts of it.
Other museums offer users a virtual museum experience at their home, making the access to culture easier for everyone [
24]. A well-known initiative is Google Arts & Culture, which allows visitors to have virtual tours in many museums based on 360° photographs. However, other projects used other technological approaches, such as the digital 3D reconstruction of the original settings and artifacts, which allows more immersive and interactive experiences. This is the case of the Regolini Galassi tomb and their associated funerary goods [
25], in which the user experience is improved by the use of gesture-based interaction [
26]. These virtual museums act as immaterial containers that continuously exhibit their collection, without any temporal or location restrictions [
14].
1.3. AR-Based Didactic Experiences in Museums
AR is a technology that integrates virtual elements over real scenes in real time [
27]. In this case, the real world is visible during the experience, but combined with virtual elements. AR does not replace the real world as VR does. Instead, it supplements reality with additional information that is integrated in the 3D real world. In this case, the technology can help to enhance a real-world experience with added contents, which can range from verbal descriptions of exhibits to images, 3D models, or visual animations [
28].
AR experiences usually are based on portable devices, such as smartphones or tablets, in which the application is installed and the 3D content is shown on the device screen superimposed onto the real world. The high number of smartphone users makes it possible to reach more participants for AR visits and to personalize the experience according to each visitors’ needs [
29]. In fact, the most commonly used types of devices in museum experiences are mobile devices, such as smartphones and tablets [
30]. However, there are some experiences that used AR head-mounted displays, such as the Hololens (Microsoft, Redmond, DC, USA), achieving a more immersive experience [
31].
As happens with VR, AR has applications in many different fields. The application field most closely related to the purposes of the present work is education. It was observed that the use of AR in education is associated with several positive factors, such as learning improvements, more motivation, and an easier comprehension of abstract concepts [
32], specially in technologically related fields [
33]. It is important to highlight that AR technology can benefit from advances in other fields, such as user interfaces and gestural interactions [
34]. For example, user’s in-air gestures could be used to interact with the AR content in a natural way in different settings, including museums.
Museum professionals, researchers, and educators are paying increased attention to AR. In this field, the technology is mainly applied to deliver additional information about the exhibition to the visitors, thus resulting in an enhanced experience [
35,
36,
37]. It has a great capacity to increase engagement, especially in children [
37], adding value to the learning experience.
The added information can be just a 3D model, as happens in Augusta Romanica, a Roman archaeological site in an open-air museum, where AR is used to show a 3D model of the site superimposed onto information panels [
38]. In other cases, the augmented content includes animations, such as moving Pleistocen Cretan animals, which are shown during the visit to a museum fossil room [
39]. Animations and texts can also be combined, as happened in an AR prototype developed to enhance the visit to a famous Chinese art piece [
40]. In general, museums combined different kinds of content (text, video, audio, 3D animations, and avatars), either to better explain the museum and its history [
36] or to give additional information about the artifacts that are exhibited [
41]. It was observed that, in this kind of experience, receiving new information using AR improves the visitors’ learning process [
42].
AR was also used to show the original state of the objects of an exhibition or a building, combining it with the current state that is directly visualized in the real world during the visit. This kind of experience is available in the Casa Batlló, where both the current and the original state of the building with its old furniture can be observed by using an AR application [
43]. The Cathedral of Valencia also has an AR application that allows visitors to visualize the Baroque vault that was removed in 2006, and which was digitized with a 3D ground laser scanner before removal [
44].
AR can also be used to guide museum visitors during the experience. It was observed that a mobile AR guide system enhances participants’ learning effectiveness in comparison with other participants who followed audio instructions or no instructions at all [
45]. AR-guided participants spend more time focusing on the paintings, with positive responses and acceptance attitudes.
AR can also be used for visits from home. The Google Arts & Culture initiative allows users to use AR to place artworks at scale in their own homes.
VR and AR museum experiences were evaluated from different dimensions [
36]. Some studies focused mainly on experiential aspects [
1], such as the perceived enjoyment [
5], the emotional involvement [
10], and the engagement [
38]. Other focused more on the learning outcomes after the experience [
12]. In fact, education was identified as one of the main variables that represent the museum experience [
18].
Most of these previous works were conducted in controlled or laboratory conditions with a limited number of participants without comparing the technologically enhanced experience with the traditional one [
1].
1.4. University Museums
University museums are part of higher education centers and the educational factor of the visitor experience is even more important than in other kinds of museums, as they are part of institutions with a special focus on research and education. Universities can introduce museums to higher education and high school students. Their collections are specially selected to improve the learning process of students in specific fields of study [
46].
The University Museums and Collections Committee (UMAC), part of the International Council of Museums (ICOM), highlighted in its reports the important role of digital and emergent technologies in making museum collections available to a wider range of people. AR was already adopted in some university museums, such as the Tito Rossini permanent exhibition located at the Roma Tre University Department of Education [
47], with a positive overall evaluation with a sample of 14 students that tested the system through a web application for Android. However, the research on the use of VR/AR technologies in university museums is still scarce and focused on artistic exhibitions. The role of the technology to improve the learning experience for different kinds museum contents and artifacts was not evaluated in previous studies.
1.5. Goals
The present work evaluates if VR and AR technologies can be used to enhance the didactic experience of technological heritage university museums. There are two issues that are distinctive in these kinds of museums. First, they are university museums in which the pedagogical role is even more relevant than in other kinds of museums. Second, as technological museums, they show different kinds of devices and technological artifacts. In many cases, it is not possible to demonstrate how these artifacts worked without damaging them. However, using VR/AR technology it is possible to provide additional information in the form of images, videos, and animations about how the artifacts were used. VR/AR technology has great potential to contribute to the pedagogical experience while preserving the artifacts undamaged.
In order to assess the contribution of AR-based technological enhancements during these visits, it is necessary to compare the user experience during the AR-based visit with the traditional one. However, previous research that compared traditional visits with AR-based visits is scarce and only a few studies report these kinds of comparisons [
12,
35,
40]. The present work will compare the traditional visit with the AR-based one, which will contribute to a better understanding of the relevance of using AR in a museum visit.
Taking all these factors into account, the research questions (RQ) of the present study were determined:
RQ1: Do the visitors to a technological heritage university museum have a better learning experience if AR is used to enhance the visit?
RQ2: Do the visitors to a technological heritage university museum have a higher level of satisfaction during the visit if AR is used to enhance it?
In order to answer those questions, different goals must be achieved in the present study.
The first goal is to design and develop an AR-based system that can be used during the visit to this technological university museum and provide additional information about the technological artifacts that are shown in the exhibition, using different formats such as images, videos, or animations.
The second goal is to design and develop a VR-based system that allows for a virtual visit to the museum. This application will not be included in the physical visit, because it was conceived as an additional way to visit the museum exhibition from other places, and not as a substitute or an improvement of the traditional visit. It can be used to give more diffusion to the contents of the museum and to reinforce its didactic task, as any person can have this experience from their home or revisit it after having experienced the physical visit. However, it will not be evaluated in the current study because only the effects of the technological enhancements in the physical visit will be evaluated. VR and AR applications are independent, but VR standalone application includes all the AR content. So during the virtual tour, users can execute the AR in a similar way that the user can in the real world visit.
Finally, the third goal is to evaluate the visitors’ experience using the AR-based system in comparison with the visitors’ experience during the traditional visit, focusing especially on their satisfaction during the visit and their learning experience. This evaluation will be performed in a group of high school students, as long as this group is the most common kind of visitor that comes to this technological museum.
2. Materials and Methods
2.1. The Telecommunications History Museum Vicente Miralles Segarra
The research will be conducted in the Telecommunications History Museum Vicente Miralles Segarra, which is a university museum that exhibits technological artifacts related to the historical evolution of different technologies and devices. It is a university museum located in the Telecommunications Engineering School of the Universitat Politècnica de València, Spain. Its main goal is to illustrate the history of telecommunications, making it accessible to telecommunication engineering students and also to future students that come to the university in organized visits from their high school to gain knowledge about the kind of degrees they can study in the university.
The museum’s collection has more than 750 artifacts, including different kinds of telecommunication devices (telegraphy, telephony, radiocommunication, audiovisual equipment, and laboratory instrumentation) and also books and treatises from the latest centuries (19th to 21st centuries).
The artifacts in this collection also have an ethnographic heritage role. Many of the telecommunication devices present in the exhibition had a widespread social and cultural use, and represent in a way the aesthetics of their time by means of their design, typography, or materials.
Part of the collection is on display and accessible to the public. In fact, it is embedded in the common spaces of the school, even in its surroundings, as a thematic and chronological tour. Telecommunication students perceive it as an inseparable part of their school that permeates their daily life.
The didactic role of this museum is clear and was present since its beginnings. The objects that constitute the collection are conserved and exhibited so that the students can learn about their future profession and understand the evolution of telecommunication technologies since their origins.
Research and conservation are also relevant issues that are covered in this museum and that appeared as a logical evolution from its didactic function. The artifacts that are included in the museum have a value that transcended its didactic role. These artifacts constitute a clear case of complex and ephemeral technological heritage. Telecommunication devices soon become obsolete and are replaced by new devices or evolutions of the original devices with better characteristics. Consequently, one of the functions of the museum is to preserve them. Difficult decisions about the best way to conserve and restore this complex heritage have to be taken by restoration professionals, considering both the structural and functional aspects of the telecommunication devices.
2.2. The AR Application
The museum AR Android application, which can be downloaded from the museum website on the visitors’ mobiles or tablets [
48], enhances the visitor experience. The application was developed as a result of three bachelor theses produced by A. Castellanos in 2018 [
49], A. Martinez in 2019 [
50], and M. Martinez Simo in 2021 [
51]. Using the application, the visitors do not just examine the museum pieces as objects; they know more about their history and understand how the items work and were used. AR produces curiosity, and the visit becomes more pleasant and didactic.
The application uses AR reference targets that are on the museum showcases. The target should be in the proper position in relation to the object to avoid obfuscating it and to allow for presenting the multimedia content in the correct position. The target size is selected considering that the visitor device should detect it at an adequate distance from the observed object. The multimedia content will be shown on the user device display around or over the museum item.
Figure 1 presents an example of a target inside the museum.
Before developing the application, an analysis of the different needed multimedia contents was performed. First, a list of the museum objects to be enhanced with AR multimedia material was created. Various aspects were considered in selecting these items: the list should include objects from all the museum sections, the selected items should be very representative in their sections or have a non-obvious functionality or a fascinating history that could be shown. Second, the multimedia content and how it would be displayed for each item was decided and produced. The content can be audio, video, or interactive images and can be displayed next to or over the object.
The application was developed using the cross-platform 3D game engine Unity 2020.3 together with the AR software development kit Vuforia 8. Unity is an integrated design environment that can be used to create 3D graphic mobile applications, and Vuforia adds to Unity several tools for developing AR mobile applications. Vuforia includes several 2D image and 3D object recognition tools. It also allows the creation of your own AR targets that can be identified more precisely. Vuforia tools, using the device camera-captured image, sends real-time target detection, identification, and tracking (positioning in space and orientation) to Unity. Using this position and orientation, Unity shows multimedia content oriented in space in real-time on the device display. Unity also allows for programming different interactions with the displayed multimedia content. So, this content can be modified through tactile buttons represented on the user display.
Figure 2 and
Figure 3 show two different interactive content examples. The first content is shown over an original source of a Lenna standard test image of the image processing field. Users can modify the image by showing different image processing methods. The second content is shown over an original 1930s telephone switchboard, where users can select operators from ancient times.
At the museum, there are old artifacts not in use, AR allows us to understand how they work. As an example,
Figure 4 shows a video describing how an 1888 gramophone works next to the historical item.
The application works on any device with Android version 4.0 or higher.
2.3. The VR Application
The museum has two VR applications. Both were developed as a result of three bachelor theses produced by L. Perez in 2015 [
52], A. Vecina Dolz in 2019 [
53] and M. Martinez Simo in 2021 [
51]. The 2015 and 2019 VR application is an Oculus Rift experience. The 2021 VR application is an online VR experience that can be acceded through the museum website [
54].
The 2015 and 2019 applications replicate the entire museum in a virtual environment. Using the Oculus Rift VR headset, users can navigate without any restrictions through the two museum floors. All the museum showcases and boards were modelled and included in the VR application. The idea is to virtualize the real experience to generate a presence sensation in the user. For this reason, in the VR environment, the AR application multimedia content is also included at the corresponding showcases. The AR targets were added on these showcases. Users, by touching the AR target with their index finger, can execute the multimedia content and interact with it in the same way they can in the real museum with the AR application (
Figure 5).
The VR application was developed using Unity 2020.3 together with Oculus Integration tools v1.39. The Oculus integration tools allow Unity to track the user within the VR environment, their hand gestures and gaze direction in a simple way to simplify the programming interaction with the defined environment.
The museum environment was developed using Blender 2.72, an open-source 3D graphics modelling and animation program. The building plans were used to describe the environment without textures and main building objects. Stairs, doors, windows, the museum showcases, and other furniture were also modelled. Textures were mainly obtained from performed photographs and open-source textures.
The 2021 VR application was developed to update the VR museum content not present in the previous versions and to improve access to the application. One drawback of the 2019 application is that an Oculus Rift VR headset and a personal computer with a powerful video cardboard are needed to execute the experience. Nowadays, this needed setup is not very common in users’ homes. For this reason, the 2021 VR application was developed to provide access to the virtual museum through the museum website without a VR headset.
WebGL is a JavaScript API used to develop 3D graphics applications that can be executed online using compatible web browsers. Unity has a WebGL build support module that allows for building the application in WebGL format. The generated files can be published in an online hosting platform, so finals users can navigate through the VR environment using their web browser without requiring any local software installation (
Figure 6). To improve the execution performance through the web browser, all multimedia content from the AR application was removed. Using the keyboard and the mouse, users can navigate through the museum.
2.4. Validation of the AR Application
The AR application was validated in the context of a visit of high school students. The Telecommunication Engineering School organizes some visits every year for high school students. The goal of these visits is to give these students information about the degrees they can study in the school in a practical way. During the visit, they participate in short seminars and practical sessions related to some of the contents that are studied in the university school and adapted for their age. Additionally, the students visit the Telecommunications History Museum, which is located in the same building where the seminars take place. The visit to the museum is guided by the curator.
2.4.1. Experimental Protocol
A between-subject design was applied in the present study. In the context of the visit to the Telecommunication Engineering School, the participants performed the visit to the Telecommunication History Museum. Some of them visited the museum in the traditional way, without any technological enhancement, just visualizing the artifacts and reading the associated labels. The others used the AR application previously described to perform the visit, so they visualized additional information shown as images, videos, and interactive objects.
After the visit, they had to answer a short questionnaire to evaluate their experience in the museum. The questionnaire included the following three questions (a Likert-scale question, a multiple-option question, and a yes/no question):
What do you think about the visit? 1 (poorly satisfied)–5 (very satisfied)
What section of the Telecommunication Museum do you like most? Radiocommunication—telephony—audiovisual equipment—laboratory instrumentation—telegraphy
Has this visit helped you to better understand the telecommunication history in the Valencian community? Yes—No
Additionally, those students that had performed the visit enhanced with AR also had to answer the following two questions about their experience:
2.4.2. Participants
The participants in this study were high school students (14–18 years old) that enrolled in one of the visits that the Telecommunication Engineering School organized during the academic year 2021/2022. A total number of 245 students agreed to participate in the study. The gender distribution was as follows: 149 were male, 92 were female, 1 was non-binary, and 13 preferred to not indicate their gender.
2.5. Statistical Analysis
All the statistical analyses were conducted using SPSS Statistics for Windows, version 16.0 (SPSS Inc., Chicago, IL, USA) with a 0.05 significance level. The answer to the Likert-scale questions were binarized, considering a positive response if values were greater or equal to 3, and a negative response if values were smaller than 3.
The chi-square test was applied to determine whether or not there is a significant association between the kind of museum visit (traditional/AR) and binary/nominal variables. In cases when more than 20% of cells in the contingency table had frequencies less than 5, the Fisher’s exact test was applied instead of the chi-square test.
4. Discussion
In the present work, two different technological solutions based on AR and VR were designed and developed in order to enhance the visit to a technological heritage museum and improve its didactic contributions. Furthermore, a study was conducted to evaluate the participant’s experience associated with the AR-based visit.
Technology should not be an end in itself; instead, it should be considered as a way to generate a better experience for the visitors, also contributing to their understanding and knowledge acquisition [
16]. In the present work, the AR application was designed based on a detailed analysis of the museum artifacts to detect those that would be more suitable to be enhanced with multimedia content, including video, audio, or interactive images.
In some cases, the multimedia content was used to provide additional information that could not be obtained from the traditional visit, such as videos that described how old artifacts work, or images to demonstrate different image processing methods over an image included in the exhibit.
Using AR contents to show how an old device worked also contributes to the preservation of the original artifact without damage, which is an important issue for museums [
55]. If the exhibit put the actual artifacts in operation, they would deteriorate quickly. However, if their working behaviour is shown using AR videos or animations, the artifact will not deteriorate and it will be perfectly conserved for future generations.
In other cases, the multimedia content provides some contextual information, such as images from operators from different time periods that are shown close to a switchboard. This kind of additional content can also contribute to making the experience more enjoyable for the visitors.
Most of the previous studies that validate AR experiences in museums are focused on the description and evaluation of the technologically enhanced experience without comparing it with the traditional one [
1]. Only a few studies report comparing the AR-based experience with the traditional visit and with a reduced number of participants [
12,
35,
40]. In the present work, 245 volunteers participated in the study and they were assigned to two different experimental conditions to allow the comparison between the traditional visit and the AR-based one. The participants’ experience was evaluated by means of a questionnaire that they answered after having finished the visit, including Likert-like questions and open-answer questions.
Globally, a significant association was found in the present work between the visit characteristics (traditional/AR-based) and the visit outcomes, both regarding satisfaction and learning acquisition. Results show that it was more probable that participants reported a high level of satisfaction after the experience if they performed the AR-based visit. A similar pattern was found regarding the learning acquisition during the experience, which was more likely to be positive if the visit was performed with the AR application. These results support the use of AR technology for this kind of experience.
No signification association was found between the kind of visit and other evaluated factors, such as the preferred museum section.
Focusing only on the evaluation of the AR experience by those participants that performed this kind of visit, it can be highlighted that most of the participants (more than 90%) reported positive satisfaction levels after the experience. Their comments in the open-answer question also show a positive evaluation of the application. They indicate some issues that could be improved in future versions, which are specially related to issues such as image quality, audio volume, interactivity, and hardware compatibility. These issues will be considered in future versions of the application, where more interactivity will be included and compatibility for other kinds of mobile operating systems will be added. Regarding image quality, it has to be highlighted that the image resolution was adjusted as a compromise between image size and quality. In any case, it will be further evaluated in future versions of the application to increase image quality. Regarding audio volume, this is an issue that can be improved by modifying the characteristics of the visit (including the use of headphones, for example) instead of making changes in the AR application. The results from this study support conclusions from previous studies that indicate that AR is a technology that can provide a new and authentic learning experience in museums [
37]. The technology is especially attractive to younger visitors [
56,
57,
58], such as the high school students that participated in the present experience. Other studies also remarked on the additional advantages of AR experience, as it stimulates learning motivation and makes the participants more prone to visit other museums with AR [
59]. It also generates more engagement during the visit [
60,
61]
Regarding the VR application, it is thought of as a complement to the AR experience. It was developed as a virtual reproduction of the museum visit that can be experienced from any location, thus contributing to a higher diffusion of the museum contents.
The main limitation of the present study is that the validation of the AR system and visit was performed only with high school students. Future validations should be performed with other kinds of visitors to evaluate differences and allow the personalization of the application depending on the user’s characteristics. The user experience during the VR visit should also be evaluated in future studies.
Furthermore, it can be hypothesized that the users’ experience associated to the AR-based visit will differ between different cultures [
1], influenced by different factors, such as the sensitivity to time and space, which is affected by the cultural context. Future studies will have to evaluate if the AR-based visit generates a different experience in different cultures. This study can be easily organized in university settings, taking advantage of the mobility programs that allow the presence of different cultures in the university campus.
Independently of these future studies about the cultural issues, the use of AR-based visits can be easily extrapolated to other types of technological museums, and also to ethnographic ones. AR can show additional information about the artifacts and their functionality while preserving the original artifacts, and this is especially relevant in this kind of experience.
Another aspect that should be considered in future validations will be the point of view of the museum professionals. Although the museum curator participated in all the stages of the design, development, and validation of the developed applications, the opinion of independent museum professionals should also be evaluated and their recommendations should be considered when designing future applications or improving the current ones [
62].