Interaction in eXtended Reality Applications for Cultural Heritage

Abstract

Digital technologies in the modern era are almost mandatory for the presentation of all types of cultural heritage. Virtual depictions of crafts and traditions offer the users the possibility of time travel, taking them to the past through the use of 3D reconstructions of cultural monuments and sites. However, digital resources alone are not enough to adequately present cultural heritage. Additional information on the historical context in the form of stories, virtual reconstructions, and digitized objects is needed. All of this can be implemented using a digital multimedia presentation technique called digital storytelling. Nowadays, an integral part of many museum exhibitions is interactive digital storytelling. This paper gives an overview of the techniques and discusses different means of facilitating interaction on digital storytelling applications for virtual cultural heritage presentations. We describe the ways in which natural interaction and interaction via eXtended Reality (Virtual and Augmented Reality) applications for cultural heritage are made possible. Users will find the stories told through these applications educational and entertaining at the same time. Through user-experience studies, we measure the user edutainment level and present how users react to implemented interactions.

1. Introduction

Bosnia and Herzegovina has a rich history, as a country of different cultures, nations, and religions. In recent times, the process of preservation and presentation of the cultural heritage of this country has been implemented using digital technologies. They are an efficient tool for the visualization and presentation of cultural and historical heritage, especially in situations where budget and interested professionals are scarce.

Digital storytelling is an evolving methodology where the content is multimedia. It is considered to be the media of the future, adding to present storytelling media, such as film and theater, another dimension—interaction. Historical information can be divided into hyperlinked structures, giving the users an overview, at first sight, of what they can find inside. Online applications can consist of multiple short stories, enabling the viewer to choose his/her path to learn and explore through interaction.

The Internet media is changing how writers write their stories. They now have a rich set of tools, methods, and ways of delivering a story at their disposal. This type of storytelling demands the collaboration of multiple team members and different professions. The director decides on the narrative method, characters, and user interaction to raise the attractiveness for the users.

Technological development is introducing new ways of interactive communication in the domain of multimedia cultural heritage applications. Virtual and Augmented Reality technologies are facilitating access and increasing the value and public awareness of cultural and natural heritage [1,2]. Assessing user experience is a complex process that includes an evaluation of the ease of use and effectiveness of the media as well as the entertainment it provides.

The importance of entertainment in digital applications is its capacity to create a motivating and successful environment for learning [3], which introduces a new combined factor: edutainment (education + entertainment). Another important factor is immersion, a measure of absorption and engagement with the virtual multimedia environment. Immersion is important to entertainment, and it is difficult to evaluate with the standard user testing methods [3].

The use of interactive digital storytelling is expanding to different fields. In our research, we focus on the application of interactive digital storytelling in eXtended Reality (XR) cultural heritage presentations, with XR referring to both the virtual and the combined real and virtual environments and interactions (Virtual and Augmented Reality). We study the means of conveying rich historic information in an engaging and immersive way, not just to visitors of exhibitions but also to the general audience online.

In this paper, we present our experience in implementing interaction on eXtended Reality applications for cultural heritage presentations. We describe the ways in which the users communicate with our applications, how they move in virtual environments and interact with augmented reality content.

The paper is organized into the following sections. Section 2 gives an introduction to digital storytelling and its definitions. An overview of relevant papers is presented in Section 3. Section 4 explains problems and solutions in the implementation of XR interactive storytelling applications with different technologies in several case studies. Section 5 describes the user evaluation process, and in Section 6, conclusions and results are discussed.

2. Interactive Digital Storytelling

Storytelling has been the main means of communication since the beginning of humankind. Throughout history, the concept has remained the same, but the tools and methods have changed with time. Presenting stories has been enhanced by digital technologies, thus giving birth to digital storytelling. In the last few years, interaction has been added. Interactive storytelling encompasses many different fields that fall under the term of human–computer interaction [4]. Handler Miller defines digital storytelling as digital media platforms and interactivity for narrative purposes, either for fictional or non-fiction stories [5]. Another definition of digital storytelling is narrative entertainment that reaches the audience via digital technology and media [5].

Interactive digital storytelling gives the user an active role. The user can influence the flow and sometimes even the content of the story. Various interactive digital storytelling methods compete at the level of user immersion and aim to teach the viewers about the topic in an engaging and attractive manner. The quality of user experience is the main success factor of interactive digital storytelling methods applications. One measure is the edutainment value, which is a combination of education and entertainment. The use of interactive digital storytelling methods for cultural heritage presentation is becoming very popular. Virtual Reality, Augmented Reality, Mixed Reality, eXtended Reality, and all kinds of combinations of digital technologies are more effective in conveying heritage information if accompanied by storytelling [6,7].

We have been researching and developing a new method of carrying out interactive digital storytelling—hyper-storytelling. In the process of developing the hyper-storytelling method, we engaged experts from computer science, visual arts, film directing, psychology, communicology, and human–computer interaction. They analyzed a sample interactive digital storytelling application and offered their insights and recommendations on what could be embedded in the new methodology [8].

3. Related Work

In order to efficiently and comprehensively preserve and present cultural heritage, many technical and technological innovations have been used. The use of technologies such as Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR), eXtended Reality (XR), web-based 3D scenes, web applications, mobile applications, and desktop applications is widespread. For most of these technologies, the problem of user interaction and the narrative paradox challenge [9] are still in some sense open.

3.1. VR in Cultural Heritage

Virtual Reality is a technology where the user should be completely immersed in a virtual environment without the visibility of a real environment, except through computer-generated representations [10]. A significant part of VR applications in cultural heritage has been accomplished with the use of 360° video [11,12,13,14].

In [11], authors proposed motivational factor as a solution to the narrative paradox in presenting Sarajevo bridges. Using 360° video can be inexpensive, as shown in [12]. The application is displayed on a mobile phone with the use of Google Cardboard. The authors used time-pressure techniques and gamification to motivate users. In [13], the author described the framework for 360° video immersive applications. This framework was demonstrated in two case studies which showed that 360° VR applications can be effective in promoting areas of historical interest.

The effectiveness of VR applications is shown in [14], where authors used electroencephalography (EEG) to evaluate the user’s feeling of presence, immersion, and engagement with the experience of 360-video storytelling. The results showed high presence, engagement, and immersion in the participants.

Strategies to overcome problems with the coordination and management of people involved in the process of creating a 3D reconstruction and its adaptation to Virtual Reality applications are described in [15]. The authors showed that one VR application can be made in four months, which includes data collecting, 3D scanning, 3D reconstruction and texture mapping, software development, and hardware design.

However, there are a few very important things we must consider when VR is being used for storytelling. The VR experience comprises a lot of elements, and although we work with cultural heritage experts who can help us to shape the story and facts, we often find it hard to deliver content that is “interesting” for the regular visitor since their knowledge is vast, and it is hard for them to use only small pieces of it [16]. This can sometimes lead to the creation of a complex VR experience, with objects which do not have an actual story that users can follow. Crawford [17] actually suggested that “Stories are about people, not things”, which should always be first priority when creating for the VR experience. This actually gave the research the fuel needed to strive forward. VR storytelling requires interdisciplinary groups to work together in creating a high-quality experience [18,19,20].

VR experience can be enhanced with the inclusion of characters and storytellers inside the VR space. They can be virtual humans or real-life actors. How to create quality virtual humans and their contribution to immersion are explained in [21]. On the other hand, real actors give a different level of immersion and add life to VR narration [22].

There are many feature creators who are trying techniques such as adding the footprints in this new media [23] to enhance immersion. Others aim for high-level precision when creating VR applications; for example, Walmsley and his team combined different techniques such as “media—real-time 3D visualization and HDRI panoramic photography—allows the interactive and immersive potential of the former to complement the high-fidelity and photorealism of the latter” [24]. Through this, they explored how these different techniques could contribute to the VR experience. On the other hand, in a similar paper, Kersten’s team recreated the Al Zubarah Fortress in Qatar, which was then used to measure how demanding these models were for hardware, and how to smoothly integrate them without a drop in frame rate, etc. [25].

Some VR experiences do not require a story and can still provide an immersive feeling to a user, even if it is only an object from real life, scanned and imported as a 3D model [26]. Mafkereseb Bekele pointed out that there is a general agreement that visual CH tools suitable for users unskilled in multimedia technologies are important for CH dissemination, and that museums and museum installations that do not introduce new interactive technologies are regarded as less interesting and attract fewer visitors [27]. Experience can be delightful, but attention must be paid to interaction and the ease of use.

The useful experience from computer games was pointed out by Edler: “VR environments can be supported by new tools of user navigation and locomotion which have been adopted from video and computer gaming, such as mini-maps, signifying markers and teleporting techniques” [23]. The usage of symbols in games and how they are perceived by non-gamers is shown in Horbiński’s paper, where he analyzed map symbols in the game “Valheim” [28]. The findings can surely be applied to VR applications since they are also a form of a game, but we must understand that some of the new VR users are not gamers and so we must adjust accordingly. All VR applications have one thing in common—user interaction with the app that must be designed and implemented because: “While we can feel the controllers in our hands, they do not represent the objects that we might pick or touch” [29]. How to achieve the highest possible immersion and the most comfortable user interaction is still highly debatable and constitutes a crucial part of the research conducted in the VR domain.

3.2. AR and MR in Cultural Heritage

Augmented reality offers a reality enriched with additional computer-generated information. Mixed reality gives the control of real and virtual objects to the user. Most applications that use AR in presenting cultural heritage are mobile-based [30,31,32]. Mobile-based applications are usually widely available, easy to deploy, and effective to adopt. AR applications are adopted in many fields such as tourism [31,33,34], education [35], medicine [36], engineering and construction [37], and many others.

As stated in [27], the applications surveyed can be divided into two groups—those for indoor and those for outdoor use. The use of AR applications in the cultural heritage domain can have multiple purposes:

• Exhibition enhancement—improving visitor experience, usually through guided tours;
• Reconstruction—an interactive presentation of reconstructed sites;
• Exploration—supports learning and motivates the user to explore and gain new insights.

Consequently, AR provides us with a whole new level of exploration while the user is still present in the real world. “Inaccessibility therefore becomes an opportunity to experiment with new forms of communication, from storytelling to gaming, through the use of VR and AR, optimized in relation to the characteristics of the user groups” [38]. In the same paper, we could again see the inclusion of narration as well as the use of actors and 3D models.

User interaction that plays a huge part in the AR and MR experience, in the same way as in VR, still needs to be developed and standardized. In [39], the authors presented “the web-based Narralive Storyboard Editor (NSE) and the Narralive Mobile Player (NMP) app, developed with the objective to assist the creative process and promote research on different aspects of the application of mobile digital storytelling in cultural heritage settings”. They used it for the gathering of data from users regarding interaction, content, and ease of use. Many new approaches have been developed in user interaction, although some results [16,40] still show a lack of user interaction that would be satisfactory for the users.

3.3. Web-Based Cultural Heritage Applications

In the widespread adoption of web-based 3D applications, the emergence of WebGL was crucial. Before WebGL, applications were built for proprietary 3D web plugins, which often had a short lifetime and high maintenance cost. Today, most modern game engines support WebGL builds. Having this technology at their disposal allowed developers to offer rich 3D applications to users without the need for installation, configuration, or some additional hardware. This property is essential for cultural heritage applications because it allows more people to have access to the heritage, not just the museums visitors, but also internet users. In the COVID-19 pandemic, this was particularly useful during the lockdown, enabling people to visit cultural heritage sites from their homes.

Even though WebGL solved some problems, there are still issues with the limited power of web-based 3D rendering. Authors in [41] described a method of reducing geometry complexion to accelerate loading and rendering speed.

Web-based solutions can be practical because they inherently support multimedia through HTML5. Including and combining different types of content simultaneously is easy. One solution that combines videos, web pages, and 3D scenes on the web-based application is presented in [42].

Web technologies in combination with AR or XR applications to create more accessible solutions are described in many papers. Some of these papers are listed below.

Unity3D was used in [43] to create a web application in mixed reality. The authors concluded that using web languages (HTML, CSS, and JavaScript) lowered the bar for content creators. The problems that persist are for hardware makers to solve by loosening some restrictions on hardware configurability and adaptability to different software technologies.

The development of some VR features on a web-based application for an online virtual reality control laboratory is presented in [44]. In the laboratory, students and teachers had immersive experiences using Head-Mounted Displays. This increased the students’ engagement, motivation, and sense of presence. The application is based on the Tree.js library, which uses WebGL.

Virtual Reality in web technologies is explored in clinical imaging and 3D data visualization in [45]. Here, the authors described the transformation of the QuickTime VR format, which is a proprietary format in 3D models that can be displayed in HTML5 but is not supported by all devices. The authors used the Object2VR software to successfully convert objects into a format more suitable for HTML5.

4. Interaction in Digital Storytelling Applications

4.1. Web-Based Applications

The use of web technologies in digital storytelling for cultural heritage preservation and presentation is often obligatory. Web-based content is available for a wider audience, not just for people who are at the cultural heritage site, museum, or a similar establishment. In most cases, users of these types of applications are very diverse. They can be of different ages and have different education and level of computer literacy, therefore user interaction on these applications needs to be simple and intuitive (which is often not easy).

The content of these applications is also diverse. In one web application, users can see textual content, video content, interactive video content, images, 3D objects, 3D scenes, 360° video, interactive quizzes and games, and many variations and combinations of these. Controls and interactions for different content can vary; for some of the content, only a computer mouse is sufficient, and these controls can be familiar for most users because they are very similar to interactions with usual web pages. For some content such as 3D scenes, a user needs to use more controls to move the avatar through the 3D scene, which can be complicated for users who are not familiar with video games. The transition from simple web page content to fully interactive 3D scenes can be confusing for users. In the transition, users often do not see the same mouse cursor, and the scene is displayed in full screen, so no familiar web browser controls will be visible. It can confuse the users and induce dilemmas such as “where is the back button?”, “how to exit from this and return to the web page?”, “where is the mouse cursor?”, “what are the controls?”. At this point, it is very important to give users enough information, that which is visible and hard to miss, while not annoying experienced users. User expectations can also vary. Younger users and gamers might expect a game-like experience with challenges and quests, but for users with no gaming experience, even moving through the 3D scene can be challenging. Scripted 3D camera movement or limited and focused camera movement could help inexperienced users. However, this should be completed with a balance of immersion, control, and usability in mind.

4.2. Natural Interaction with Gestures (V-Must.net)

Keys to Rome was an international museum exhibition developed through the V-Must.net project, which took place in four cities in 2014 (Rome, Amsterdam, Alexandria, and Sarajevo) [46]. Its goal was to depict the differences and uniqueness of the Roman empire. The museums’ collections were available in a virtual journey through computer animations, natural interaction installations, as well as multimedia and mobile applications. Four cities were connected in the story, which gave visitors the ability to explore Roman culture in the characters of the old merchant Gaius and his nephew Marcus. Each historical site had a virtual reconstruction of the site and the objects in it. Each object had a story that was narrated by Gaius and Marcus.

Part of this exhibition was the Admotum installation (Figure 1). Users were able to walk through virtual environments, collect objects, and hear the stories. After all objects from one location were collected, other locations were unlocked for exploration. The Admotum installation consisted of a computer, projector, projector screen, and Kinect sensor. The interaction was gesture-based. The main problem with this type of interaction was that users had to perform gestures precisely, and they were required not to move from the static position where the Kinect sensor had the best accuracy. In the exhibition, the users often moved and Kinect lost traction. This was often confusing for a user and may have caused the loss of focus on the storytelling.

Another part of the Admotum installation was Holobox (Figure 2). This was a reinterpretation of an immersive theater technique called “Pepper’s ghost effect”, which gives an illusion of a 3D object floating in space. The user was able to transfer the object by using a gesture to Holobox where the object was controlled by Leap sensor controls. Leap sensor is a small device that detects a user’s hand gestures through a system of small infrared lights and cameras. This type of interaction was much more precise, but it lacked tangible feedback.

4.3. Interaction in VR Applications

4.3.1. Player’s Movement

The main challenge for player movement interaction in Virtual Reality is the creation of a user-friendly and intuitive way of moving which will not affect immersion in a bad way. Moreover, movement can easily produce a phenomenon called Virtual Reality sickness if not implemented right. In our previous virtual applications, we experimented with a few ways of implementing player movement to get to the one that users would find the easiest to understand and use.

In the Sarajevo War Tunnel VR [47] and the Mostar Cliff Diving VR [48,49], we used controller movement imitating hand movements while walking. Thus, in order to move, users had to hold the controller triggers and shake hands as they would when walking or running. This kind of movement appeared strange and was hard to understand for some users, so in future applications, we turned to more common ways of moving.

The Battle on Neretva VR Application and Crvena stijena VR Application enabled the users to experience events from historical periods presented in museum exhibitions. They executed the users’ movement through joysticks of the VR headset controllers. The users had to move one of their thumbs in the direction they wanted to go. Using the thumbs is very intuitive as most users who have used joysticks before know how they work. To avoid motion sickness, diagonal movement was disabled so that the users could only move forward, backward, and sideways. With only forward and backward movements, the users were always looking in the direction of the movement, which proved to produce less sickness. Sideways movement was added after initial user evaluation; users who were used to playing games agreed that the sideways movement was missing, and that it would make the movement of the player easier and more efficient. Even though moving with joysticks or moving thumbs on the touchpad did not imitate natural human movement, it appeared to be more user-friendly in Virtual Reality applications and less likely to decrease immersion as opposed to the hand movements used in the Sarajevo War Tunnel VR [47] and the Mostar Cliff Diving VR [48,49].

In the Old Crafts VR Application and Roman Heritage in the Balkans VR, we opted for the static player, with only an option to move between different scenes with the use of command clicks. This way, user placement was strongly controlled, and a view of the scene was centralized based on its position. This was appropriate in such an application since there was no gameplay, and the exhibition itself was static with regard to the places where the events occurred.

4.3.2. Interaction with User Interface

The user interface (UI) is an important part of each Virtual Reality application, and interaction should be intuitive and easy to understand. In most of our Virtual Reality applications, we used the triggers of the controllers to interact with the UI. For the user to see which part of the UI they were selecting, a laser pointer was used, as can be seen in Figure 3. The laser pointer creates a line that extends from the controller, and if the user presses the trigger when the line of the laser pointer touches one of the UI buttons, the UI button will be activated. As a way of preventing a wrong-button click, we also tried highlighting the button which was being pointed at so that the user could see what he was trying to click on.

We also experimented with the use of controller movement for button clicks. In the Sarajevo War Tunnel VR, we tried to implement the user interface clicks in a way that the user needed to put the controller through the button he wanted to click. Even though this kind of interaction was easy to understand and use, it produced some problems when the place where the Virtual Reality was set up was small, so the buttons were sometimes unreachable.

4.3.3. Holding Objects in Virtual Reality

In some of our Virtual Reality applications, such as Crvena stijena VR and Roman Heritage in the Balkans, we enabled the user to touch and explore reconstructed 3D objects. We used the grip buttons of the controller to pick up exhibit objects. The user needed to hover their hand over the exhibit object, and then press and hold the grip button of the corresponding controller so that the item would appear in the user’s hand. While the grip button was pressed, the item could not be dropped and would keep following the hand. In Crvena stijena VR, when the user releases the grip button, the item falls to the ground, and in Roman Heritage in the Balkans, we decided to return the object to its original position when it is released. This method of interacting with the exhibits mimics a natural feeling of holding objects in your hand, which is much more realistic than simply pressing a button to pick up an object or picking the object with the laser pointer. A problem with this approach can occur when the VR setup is in a smaller room, so the user is not able to reach some of the objects. This could be solved by allowing the user to teleport between different points in the scene.

4.3.4. Interaction in Virtual Reality Gameplays

Some Virtual Reality applications use gameplay in order to make history more interesting and appealing to wider groups of people. Thus, Virtual Reality is used to present cultural and historical heritage and to engage users actively in exploration.

In the Battle on Neretva VR Application [50], gameplay is used to enable the users to experience a famous battle from the WWII. The user has to help his comrades to win the battle and save the wounded. To do so, the user needs to detonate an explosive and destroy the bridge in order to deceive enemy troops. To complete the first assignment, the user has to pick up the reel with a cable and move it to the detonator. Picking up and holding the cable was implemented through a combination of the grab button and controller movement near the reel, but this way of interacting appeared to confuse players as they were unable to pick it up in the first few attempts. To make it easier for users who are inexperienced in Virtual Reality, we simplified the picking up by just moving the controller near the reel.

The second task on the Battle on Neretva VR Application is the destruction of enemy air crafts while the comrades are trying to build the improvised bridge. Users need to target air crafts with a head movement since the target is fixed in the center of their view, as can be seen in Figure 4; subsequently, the users have to shoot them with a trigger click on the controller. While most users did not find any difficulties in targeting and shooting, some users found it strange that the target was moving, unrelated to the hand and controller movements. Hence, one of the ideas for improvement was to bind the target to the controller movement instead of the head movement.

The gameplay on the Crvena Stijena VR features a deer hunt in prehistory. To catch the deer, it is necessary to pick up and throw a spear in front of it. The action of picking up the spear is the same as picking up the exhibit objects. More specifically, the user needs to hover their hand above the spear and press and hold the corresponding grip button. There is one difference in the mechanics—the throwing of the spear. When the user releases the grip button, the spear does not just drop down to the ground. Instead, it calculates the velocity of the movement of the controller in the hand of the user and adds force to the spear. The amount of force added depends on the velocity of the controller. The force is added to the direction in which the spear is being thrown.

4.4. Interaction in AR Applications

Creating intuitive and easily understandable interaction in Augmented Reality is challenging and depends on the device used. If using a hand-held device for Augmented Reality applications, user interaction must either rely on the user’s one hand, since the other hand will be holding the device, or on the movement of the device itself. Moreover, it is expected that the interaction will be somewhat familiar to the user who uses a mobile device or tablet regularly.

In our application, the Sarajevo 5D Tour Guide, which is a tracker-based application, additional information is presented to users on-demand. When the user scans the image mark, apart from the cultural heritage object bound to that target, the interactive pin is shown, as can be seen in Figure 5. When the user touches the screen in the place of the pin, additional content about the object is shown. This interaction is standard for all mobile phones and tablets, which is why it is easily understandable for the user. In addition, pins are commonly used to present points of interest on maps. A possible problem with this approach occurs if the user is holding a map on the other hand since it can then be inconvenient to click on the pin while holding a phone with the same hand.

On our second Augmented Reality application, Roman Heritage of Sarajevo, similar interaction principles are used. On this application, the user scans a map with tracker images, and 3D reconstructions of objects are then shown, together with different types of content. These include an HD video, 3D models, 360° video, text, and audio, as shown in Figure 6. HD video and audio content are started by clicking on the play icon that is commonly used, so the users did not have any problems with it. Informative text is shown on scroll-able panels to make it more visually appealing while not overwhelming the scene with a lot of text. To scroll through the text, a swipe motion is used, which is again widely used on mobile applications, only this time it is placed in 3D space. For movement in 360° video, the swipe motion in all directions is used. As an alternative to this, we could also use the phone movement.

5. User Experience Evaluation

Usability evaluation goals encompass both the exploration of novel interaction approaches and success in the exploitation of the technology in use. A common benefit of these different usability evaluation goals is the provision of informative feedback to designers and developers concerning users’ attitude towards the adoption of the XR technologies. The development of our XR applications is accompanied by relevant usability studies on genuine and overall user experience with interactive digital storytelling. An additional objective is to focus on measuring success in achieving specific features such as edutainment and immersion. In our previous UX studies, we employed the qualitative approach for the initial evaluations of the Mostar Cliff Diving VR [48] and the Sarajevo War Tunnel [47], and the quantitative evaluations for the Kyrenia [51] and Baiae [22] interactive digital stories, and for the follow-up studies on the Mostar Cliff Diving VR [49]. The quantitative studies enable our team to assess the level of development of the application and compare different solutions. This is the reason why we design our quantitative studies according to the specific benchmarking framework [52], which is to enable comparison between the studies and to identify patterns in user preferences. The most recent example of a quantitative usability evaluation study is that of the “Crvene stijena” VR application developed for the Nikšić Museum. We will present the overview of the evaluation results for this study as an example of our studies.

The framework for the usability studies we developed is built upon the constructs adopted from the UTAUT model [53]: Performance Expectancy (PE) and Effort Expectancy (EE). The UX questionnaire measures the following sub-scales: immersion and edutainment, both linked to the Performance Expectancy (PE); and perceived ease of use, linked to the Effort Expectancy (EE). The questionnaire was organized into three sections:

• Introductory part with demographic data;
• Quantitative UX evaluation comprising three sub-scales addressing immersion, edutainment, and ease of use;
• Open questions part where users can express their opinion and indicate the most favorable and the most problematic parts.

The evaluation of the sub-scales was performed using a 5-point Likert scale.

In our previous studies, participants were recruited by invitation and demographic data were used to reflect a balanced composition regarding the age, gender, education, professional background, etc. The novelty of this study was that the participants were visitors during the opening exhibition at the Nikšić Museum, and the demographic data reflect their composition: balanced regarding the age, with half of the participants being secondary school students.

The questionnaire structure and overview of measured statistics as well as the mean and the standard deviation are presented in

Table 1. Questions are split according to evaluation construct—EDx for Edutainment, IMx for Immersion, and EEx for Effort Expectancy. Responses were delivered on a 5-point Likert scale, with 1 = strongly disagree and 5 = strongly agree. Respective summary statistics are shown: Mean and Standard Deviation (SD). *: Questions EE2 and EE6 represent negative attitudes, hence the responses were complemented prior to calculation.

Item	Mean	SD
ED1 I learned a lot from the stories	4.44	0.61
ED2 I became interested in the history of that period	4.63	0.48
ED3 I would like to learn more about the historical events from that period	4.56	0.50
ED4 I was amazed by the creativity of the tools from that period	4.88	0.33
ED5 I was surprised by how the tools were used	4.88	0.33
ED6 I would love to see more stories about historical events	4.69	0.58
IM1 I liked the implementation of the VR environment	4.88	0.33
IM2 The voice of the actress aroused interest in the topic	4.56	0.70
IM3 My time passed quickly	4.94	0.24
IM4 The games were interesting for me	5.00	0.00
IM5 I almost felt like I was on the field	4.81	0.39
IM6 I could imagine prehistoric settlers in the environment	4.63	0.60
IM7 I liked the VR reconstruction of the prehistoric tools	4.81	0.39
IM8 The environment of the prehistoric settlement is realistic	4.81	0.39
IM9 My time passed quickly as I moved	4.81	0.39
IM10 Hunting simulation was very interesting	4.31	1.04
EE1 I recognized the commands for moving and interacting with objects	4.31	0.85
EE2 * I felt dizzy from moving	2.88	1.41
EE3 Navigating was easy	4.69	0.46
EE4 Interacting with objects was simple	4.81	0.39
EE5 It was clear to me how to navigate through the VR app	3.69	0.46
EE6 * I had a hard time controlling movement through the model	3.50	1.37
EE5 It was clear to me how to navigate through the VR app	3.69	0.46

. Mean values for Performance Expectancy, more specific for the items measuring the immersion, are in the range of 4.31–5.00, while the edutainment items are in the range of 4.44–4.88. Mean values for the Effort Expectancy items reflecting ease of use are in the range of 2.88–4.81.

The evaluation results are in line with our previous conclusions, that the quality of the narrative has a positive effect on immersion and edutainment, even in the presence of challenges regarding the ease of use. Mean values for the overall rating of the sub-scales are: 4.76 for the immersion, 4.68 for the edutainment, as compared to 3.94 for the ease of use. We are aware that many users at the sites of the museums had an opportunity to try completely new technology such as VR, and this produced more positive results compared to very experienced VR users. However, the benefits of bringing the broader population to the historical museums, of making visits to cultural heritage sites more popular, and of teaching visitors about intangible heritage such as hunting activities are clearly demonstrated by the evaluation results.

6. Conclusions

We presented our experience in enabling interaction with digital cultural heritage applications. We found that natural interaction using gestures recognized by the Kinect motion sensor was too demanding for the users and distracted them from the storytelling. In Virtual Reality applications, we facilitated the interaction with the UI through the use of controllers, interaction to simulate motion, interaction with digitized exhibits, and gameplay interaction. As the applications were designed for all target audiences, we tried to keep them simple and intuitive.

In the described digital cultural heritage applications, we implemented our research on interactive digital storytelling that resulted in the development of a hyper-storytelling methodology. The use of actors in digital storytelling added a human empathy dimension to our 3D reconstructions. We designed and developed virtual presentations of more than thirty cultural heritage objects and sites in Bosnia and Herzegovina, Italy, Malta, Albania, Serbia, and Montenegro. We digitally presented intangible heritage from several European and Asian countries.

User experience evaluation studies showed which of them were successful and what drawbacks users recognized. We also encountered the Virtual Reality sickness problem and overcame it by introducing static reference objects in the user’s field of view. Moreover, we used the teleport function instead of movement wherever possible in order to avoid the potential risk of VR sickness as we noticed that some users were not comfortable with the walking simulation. This did not reduce the perception of immersive virtual environments nor of their cognitive representations because our environments were not complex and the movement was not a significant part of the experience.

In Augmented Reality applications, the interaction was kept simple, and standard methods for mobile devices were used. The users were excited to navigate through the 360 videos by moving their devices around them.

We can conclude that interaction on XR digital heritage applications needs to be simple and intuitive, as they are installed in museums or online and are available to all visitors regardless of their age and computer literacy. In museums, the staff needs to be ready to assist the users who are not familiar with VR headsets. We can see similar findings in Bekele’s closing words [27]. We believe that in the future, the price of VR devices will decrease, and they will become available to more users. Therefore, we need to continue exploring the most suitable interaction techniques that facilitate interactive digital storytelling, one of the most immersive techniques for experiencing cultural heritage.