Multimodal Technologies and Interaction

22 pages, 29268 KiB

Open AccessArticle

NariTan: Enhancing Second Language Vocabulary Learning Through Non-Human Avatar Embodiment in Immersive Virtual Reality

by Shogo Fukushima, Keigo Sakamoto and Yugo Nakamura

Multimodal Technol. Interact. 2024, 8(10), 93; https://doi.org/10.3390/mti8100093 - 18 Oct 2024

Viewed by 798

With the rise of head-mounted displays (HMDs), immersive virtual reality (IVR) for second-language learning is gaining attention. However, current methods fail to fully exploit IVR’s potential owing to the use of abstract avatars and limited human perspectives in learning experiences. This study investigated [...] Read more.

With the rise of head-mounted displays (HMDs), immersive virtual reality (IVR) for second-language learning is gaining attention. However, current methods fail to fully exploit IVR’s potential owing to the use of abstract avatars and limited human perspectives in learning experiences. This study investigated IVR’s novel potential by using non-human avatars to understand complex concepts. We developed a system for learning English vocabulary through the actions of non-human avatars, offering a unique learning perspective. This paper presents an IVR vocabulary learning environment with a dragon avatar and compares word retention rates (immediate and one-week memory tests), subjective workload, and emotional changes with traditional methods. We also examined the vocabulary ranges that are teachable using this system by varying the number of avatars. The results showed that the proposed method significantly reduced forgotten English words after one week compared to traditional methods, indicating its effectiveness in the long term. Full article

(This article belongs to the Special Issue Innovative Theories and Practices for Designing and Evaluating Inclusive Educational Technology and Online Learning)

► Show Figures

Figure 1

19 pages, 717 KiB

Open AccessReview

Virtual Tasting in the Metaverse: Technological Advances and Consumer Behavior Impacts

by Farouk Adéwalé Moustapha, Myriam Ertz and Chourouk Ouerghemmi

Multimodal Technol. Interact. 2024, 8(10), 92; https://doi.org/10.3390/mti8100092 - 18 Oct 2024

Viewed by 956

Abstract

Product tasting is a key element in improving customer satisfaction in the commercial environment. This study looks at the notion of traditional tasting and its effect on customer behavior and explores emerging tasting techniques, shedding light on the contribution of digital tasting. Indeed, [...] Read more.

Product tasting is a key element in improving customer satisfaction in the commercial environment. This study looks at the notion of traditional tasting and its effect on customer behavior and explores emerging tasting techniques, shedding light on the contribution of digital tasting. Indeed, the advent of the metaverse has made it possible to offer new virtual tasting experiences. However, this experience does not yet involve a sense of taste. Our manuscript highlights the potential of tasting in the metaverse through a descriptive study of various concrete cases of international brands that have included it in their marketing strategies. In light of the results, practical and theoretical recommendations are provided for professionals interested in leveraging virtual tools to improve consumer satisfaction. Full article

► Show Figures

Graphical abstract

17 pages, 504 KiB

Open AccessArticle

Impact of Metaverse Technology on Academic Achievement and Motivation in Middle School Science

by Norah Saleh Mohamed Al-Muqbil

Multimodal Technol. Interact. 2024, 8(10), 91; https://doi.org/10.3390/mti8100091 - 12 Oct 2024

Viewed by 623

Abstract

This study explores the effects of Metaverse technology on middle school learners’ academic performance and motivation in science subjects. Utilizing a quasi-experimental design, 33 students in the experimental group were exposed to the Metaverse for one semester, while 32 students in the control [...] Read more.

This study explores the effects of Metaverse technology on middle school learners’ academic performance and motivation in science subjects. Utilizing a quasi-experimental design, 33 students in the experimental group were exposed to the Metaverse for one semester, while 32 students in the control group continued with traditional teaching methods at School 148 in Riyadh. Data collection instruments included a validated science achievement test and a motivation scale. The results demonstrated that the nodes were statistically significantly improved in the experimental group, receiving an average post-test score of 73.1, as compared with the control group, receiving an average post-test score of 65.9 (t = 2.3, p < 0.05). The scores in motivation were also slightly higher in the experimental group, with a mean of 26.9, as compared with the control group, with a mean of 17.1 (t = 5.75, p < 0.05). For academic achievement and motivation, the effect sizes were quite high: fixed effect = 1.091; random effect equals 1.086. These results demonstrate the possibilities of Metaverse technology in revolutionizing the way students learn science. This technology could be a valuable tool for instruction in science classes to enhance performances and influence students’ attitudes positively towards enhanced learning environments in schools. Full article

► Show Figures

Figure 1

23 pages, 21762 KiB

Open AccessArticle

Predictive Gaze Analytics: A Comparative Case Study of the Foretelling Signs of User Performance during Interaction with Visualizations of Ontology Class Hierarchies

by Bo Fu

Multimodal Technol. Interact. 2024, 8(10), 90; https://doi.org/10.3390/mti8100090 - 12 Oct 2024

Viewed by 644

Abstract

The current research landscape in ontology visualization has largely focused on tool development, yielding an extensive array of visualization tools. Although many existing solutions provide multiple ontology visualization layouts, there is limited research in adapting to an individual user’s performance, despite successful applications [...] Read more.

The current research landscape in ontology visualization has largely focused on tool development, yielding an extensive array of visualization tools. Although many existing solutions provide multiple ontology visualization layouts, there is limited research in adapting to an individual user’s performance, despite successful applications of adaptive technologies in related fields, including information visualization. In an effort to innovate beyond traditional one-size-fits-all visualizations, this paper contributes one step towards realizing user adaptive visualization by recognizing timely moments when users may potentially need intervention, as real-time adaptation can only occur if it is possible to correctly predict user success and failure during an interaction in the first place. In addition, an open-source, reusable, and extensible software: Beach Environment for the Analytics of Human Gaze (BEACH-Gaze) is made available to the broader scientific community interested in descriptive and predictive gaze analytics. Building on a wealth of research in eye tracking, this paper compares four approaches to predictive gaze analytics through a series of experiments that utilize scheduled gaze digests, irregular gaze events, the last known gaze status, as well as all gaze captured for a user at a given moment in time. The results from a set of experimental trials suggest that irregular gaze events are most informative of early predictions of user performance, whereas cognitive workload appears to be most indicative of overall user performance in the task scenario presented in this paper. These empirical findings highlight the importance of an analytical approach to gaze on user predictions and indicate careful consideration when applying. Full article

► Show Figures

Figure 1

18 pages, 1496 KiB

Open AccessSystematic Review

Virtual Reality as an Interactive Tool for the Implementation of Mindfulness in University Settings: A Systematic Review

by Paula Puente-Torre, Vanesa Delgado-Benito, Sonia Rodríguez-Cano and Miguel Ángel García-Delgado

Multimodal Technol. Interact. 2024, 8(10), 89; https://doi.org/10.3390/mti8100089 - 11 Oct 2024

Viewed by 939

Abstract

Over the last few years, the importance of Mindfulness in the field of research has grown exponentially, as it has demonstrated various benefits in improving mental health, although there are still various difficulties in putting these techniques into practice among the university population. [...] Read more.

Over the last few years, the importance of Mindfulness in the field of research has grown exponentially, as it has demonstrated various benefits in improving mental health, although there are still various difficulties in putting these techniques into practice among the university population. However, Virtual Reality is emerging as a tool to improve the implementation of these techniques. For this reason, a systematic review was carried out of the different studies that aim to analyze the impact of the use of Virtual Reality for the implementation of Mindfulness techniques that contribute to the improvement of mental health among the university population at national and international levels. For this review, different international reference databases were searched, such as Web of Science and Scopus, and all selected articles had to be published in the period between 2010 and 2024. The selected publications had to be primary research involving a Mindfulness intervention, carried out among university students, and whose main tool for its implementation was Virtual Reality. A total of seventy-eight studies were initially identified, from which fourteen were selected, as the rest did not meet the inclusion criteria. In sum, the results show that the use of Virtual Reality as a tool for the implementation of Mindfulness techniques is certainly effective in reducing and mitigating high levels of anxiety, depression, and stress among university students. All of the research analyzed shows a substantial improvement in the quality of life, mental health, and life satisfaction of the participants. Full article

► Show Figures

Figure 1

17 pages, 10013 KiB

Open AccessArticle

An In-Depth Evaluation of Educational Burst Games in Relation to Learner Proficiency

by Ashish Amresh, Vipin Verma and Michelle Zandieh

Multimodal Technol. Interact. 2024, 8(10), 88; https://doi.org/10.3390/mti8100088 - 11 Oct 2024

Viewed by 570

Abstract

Game-based learning assessments rely on educational data mining approaches such as stealth assessments and quasi mixed methods that help gather data on student learning proficiency. Rarely do we see approaches where student proficiency in learning is woven into the game’s design. Educational burst [...] Read more.

Game-based learning assessments rely on educational data mining approaches such as stealth assessments and quasi mixed methods that help gather data on student learning proficiency. Rarely do we see approaches where student proficiency in learning is woven into the game’s design. Educational burst games (EBGs) represent a new approach to improving learning proficiency by designing fast-paced, short, repetitive, and skill-based games. They have the potential to be effective learning interventions both during instruction in the classroom and during after-school activities such as assignments and homework. Over five years, we have developed two EBGs aimed at improving linear algebra concepts among undergraduate students. In this study, we provide the results of an in-depth evaluation of the two EBGs developed with 45 participants that represent our target population. We discuss the role of EBGs and their design constructs, such as pace and repetition, the effect of the format (2D vs. 3D), the complexity of the levels, and the influence of prior knowledge on the learning outcomes. Full article

► Show Figures

Figure 1

22 pages, 13436 KiB

Open AccessArticle

Assessment of Purchasing Influence of Email Campaigns Using Eye Tracking

by Evangelia Skourou and Dimitris Spiliotopoulos

Multimodal Technol. Interact. 2024, 8(10), 87; https://doi.org/10.3390/mti8100087 - 10 Oct 2024

Viewed by 686

Abstract

Most people struggle to articulate the reasons why a promotional email they are exposed to influences them to make a purchase. Marketing experts and companies find it beneficial to understand these reasons, even if consumers themselves cannot express them, by using neuromarketing tools, [...] Read more.

Most people struggle to articulate the reasons why a promotional email they are exposed to influences them to make a purchase. Marketing experts and companies find it beneficial to understand these reasons, even if consumers themselves cannot express them, by using neuromarketing tools, specifically the technique of eye tracking. This study analyses various types of email campaigns and their metrics and explores neuromarketing techniques to examine how email recipients view promotional emails. This study deploys eye tracking to investigate and also verify user attention, gaze, and behaviour. As a result, this approach assesses which elements of an email influence consumer purchasing decisions and which elements capture their attention the most. Furthermore, this study examines the influence of salary and the multiple-choice series of emails on consumer purchasing choices. The findings reveal that only the row that people choose to see in an email affects their purchasing decisions. Regarding promotional emails, the title and brand play a significant role, while in welcome emails, the main factor is primarily the title. Through web eye tracking, it is found that, in both promotional and welcome emails, large images captivate consumers the most. Finally, this work proposes ideas on how to improve emails for similar campaigns. Full article

► Show Figures

Figure 1

20 pages, 3402 KiB

Open AccessArticle

A Virtual Reality Game-Based Approach for Shoulder Rehabilitation

by Moisés Moreira, Estela Vilhena, Vítor Carvalho and Duarte Duque

Multimodal Technol. Interact. 2024, 8(10), 86; https://doi.org/10.3390/mti8100086 - 4 Oct 2024

Viewed by 868

Abstract

In recent years, with widespread access to virtual reality (VR) headsets, VR has become an affordable supplement to physiotherapy. Researchers explore the use of existing commercial games or develop new ones to enhance physiotherapy sessions, finding that gamers exhibit reduced nervousness, report less [...] Read more.

In recent years, with widespread access to virtual reality (VR) headsets, VR has become an affordable supplement to physiotherapy. Researchers explore the use of existing commercial games or develop new ones to enhance physiotherapy sessions, finding that gamers exhibit reduced nervousness, report less pain, and experience increased enjoyment. However, ensuring consistent exercise adherence poses a challenge. Another area of interest involves integrating robots to aid patients. In our study, we seamlessly integrated a Kuka LBR Med 7 R800 with Unity through a meticulously developed Application Programming Interface (API). This fusion of robotics and video games assists in physiotherapeutic exercises. The games were developed specifically for compatibility with the Oculus Quest 2 virtual reality headset, chosen as the preferred VR platform for this study. Two games, using common game-design concepts with distinct approaches, were evaluated for system acceptance via the Technology Acceptance Model (TAM) and usability through the System Usability Scale (SUS). In a well-distributed group of 15 participants with an average age of 22 years, greater technology acceptance was observed among women. Those playing more hours per day reported lower perceived ease of use, though one game achieved an excellent SUS rating of 83.3. Conversely, the other game, which was tested with 11 participants with an average age of 20 years, showed a potential negative impact on behavioral intention. The particular sample used in the study has limitations, so the study should be repeated to obtain more reliable and conclusive results. In conclusion, the successful integration of VR and robot assistance in physiotherapy games relies on the proper application of the game design principle. Full article

► Show Figures

Figure 1

24 pages, 4700 KiB

Open AccessArticle

Exploring the Impact of VR Scaffolding on EFL Teaching and Learning: Anxiety Reduction, Perceptions, and Influencing Factors

by Hsiang Ling Huang

Multimodal Technol. Interact. 2024, 8(10), 85; https://doi.org/10.3390/mti8100085 - 27 Sep 2024

Viewed by 725

Abstract

This study examined the use of virtual reality (VR) scaffolding in English as a Foreign Language (EFL) instruction, focusing on its effects on speaking anxiety and learner perceptions and the need for tailored assessment methods. The study involved 34 Taiwanese university medical students [...] Read more.

This study examined the use of virtual reality (VR) scaffolding in English as a Foreign Language (EFL) instruction, focusing on its effects on speaking anxiety and learner perceptions and the need for tailored assessment methods. The study involved 34 Taiwanese university medical students and utilized quantitative and qualitative questionnaires. The quantitative results indicated a significant reduction in speaking anxiety and positive perceptions of VR-assisted learning. Qualitative findings revealed that students experienced dual anxieties related to language and technology during VR learning and nervousness during performance evaluations in a VR setting. This study highlights the importance of creating interactive scaffolding that considers individual learner differences and supports personalized learning experiences. It also underscores the necessity of adopting assessment strategies that align with VR environments’ unique, immersive nature in language instruction. Our findings contribute to the growing body of research on VR applications in language learning, offering valuable insights for educators and researchers aiming to leverage this innovative technology in the EFL context. Full article

► Show Figures

Figure 1

20 pages, 4378 KiB

Open AccessArticle

Development of a Tool for Evaluating the Influence of Engineering Students’ Perception of Generative AI on University Courses Based on Personality, Perceived Roles in Design Teams, and Course Engagement

by Stefano Filippi and Barbara Motyl

Multimodal Technol. Interact. 2024, 8(10), 84; https://doi.org/10.3390/mti8100084 - 26 Sep 2024

Viewed by 818

Abstract

This research investigates the possible influence of students’ perceptions of emerging AI technologies on university courses, focusing on their knowledge and perceived usefulness within engineering design. An evaluation tool implemented in a Microsoft Excel workbook was developed and tested to perform the process [...] Read more.

This research investigates the possible influence of students’ perceptions of emerging AI technologies on university courses, focusing on their knowledge and perceived usefulness within engineering design. An evaluation tool implemented in a Microsoft Excel workbook was developed and tested to perform the process of data collection through well-known questionnaires, data analysis, and the generation of results, facilitating attention to class compositions and measuring AI awareness and perceived usefulness. The study considers traditional aspects such as roles within design teams and the psychological factors that may influence these roles, alongside contemporary topics like Large Language Models (LLMs). Questionnaires based on well-established theories were administered during courses on product innovation and representation, assessing both primary and secondary design roles. Primary roles focus on technical skills and knowledge, while secondary roles emphasize problem-solving approaches. The Big Five questionnaire was used to characterize students’ psychological profiles based on the main personality traits. Students’ perceptions of AI involvement and usefulness in engineering design were evaluated using questionnaires derived from the consolidated literature as well. Data were collected via Google forms from both in-class and off-line students. The first results of the workbook adoption highlight some relationships between personality traits, perceived roles in design teams, and AI knowledge and usefulness. These findings aim to help educators enhance course effectiveness and align courses with current AI advancements. The workbook is available to the readers to collect data and perform analyses in different countries, education disciplines, and as time goes by, in order to add the longitudinal point of view to the research. Full article

► Show Figures

Figure 1

Journal Menu

Journal Browser

Multimodal Technol. Interact., Volume 8, Issue 10 (October 2024) – 10 articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI