Multimodal Technologies and Interaction

14 pages, 2416 KiB

Open AccessArticle

Extended Reality Educational System with Virtual Teacher Interaction for Enhanced Learning

by Fotis Liarokapis, Vaclav Milata and Filip Skola

Multimodal Technol. Interact. 2024, 8(9), 83; https://doi.org/10.3390/mti8090083 - 23 Sep 2024

Viewed by 1018

Advancements in technology that can reshape educational paradigms, with Extended Reality (XR) have a pivotal role. This paper introduces an interactive XR intelligent assistant featuring a virtual teacher that interacts dynamically with PowerPoint presentations using OpenAI’s ChatGPT API. The system incorporates Azure Cognitive [...] Read more.

Advancements in technology that can reshape educational paradigms, with Extended Reality (XR) have a pivotal role. This paper introduces an interactive XR intelligent assistant featuring a virtual teacher that interacts dynamically with PowerPoint presentations using OpenAI’s ChatGPT API. The system incorporates Azure Cognitive Services for multilingual speech-to-text and text-to-speech capabilities, custom lip-syncing solutions, eye gaze, head rotation and gestures. Additionally, panoramic images can be used as a sky box giving the illusion that the AI assistant is located at another location. Findings from three pilots indicate that the proposed technology has a lot of potential to be used as an additional tool for enhancing the learning process. However, special care must be taken into privacy and ethical issues. Full article

► Show Figures

Figure 1

25 pages, 649 KiB

Open AccessReview

Music as a Tool for Affiliative Bonding: A Second-Person Approach to Musical Engagement

by Mark Reybrouck

Multimodal Technol. Interact. 2024, 8(9), 82; https://doi.org/10.3390/mti8090082 - 20 Sep 2024

Viewed by 777

Abstract

Music listening or playing can create a feeling of connection with other listeners or performers, with distinctive levels of immersion and absorption. A major question, in this regard, is whether the music does have an ontological status, as an end in itself, or [...] Read more.

Music listening or playing can create a feeling of connection with other listeners or performers, with distinctive levels of immersion and absorption. A major question, in this regard, is whether the music does have an ontological status, as an end in itself, or whether it is only a tool for the mediation of something else. In this paper we endorse a mediating perspective, with a focus on the music’s potential to increase affiliative bonding between listeners, performers and even the music. Music, then, is hypostasized as “something that touches us” and can be considered a partner of affiliative exchange. It has the potential to move us and to modulate the way we experience the space around us. We therefore elaborate on the tactile dimension of being moved, as well as on the distinction between personal, peripersonal, and extrapersonal space, with a corresponding distinction between first-person, second-person, and third-person perspectives on musical engagement. Full article

► Show Figures

Figure 1

21 pages, 287 KiB

Open AccessArticle

Effects of Kahoot! on K-12 Students’ Mathematics Achievement and Multi-Screen Addiction

by Nikolaos Pellas

Multimodal Technol. Interact. 2024, 8(9), 81; https://doi.org/10.3390/mti8090081 - 16 Sep 2024

Viewed by 1338

Abstract

Digital platforms are increasingly prevalent among young students in K-12 education, offering significant opportunities but also raising concerns about their effects on self-assessment and academic performance. This study investigates the effectiveness of Kahoot! compared to traditional instructional methods in enhancing mathematics achievement and [...] Read more.

Digital platforms are increasingly prevalent among young students in K-12 education, offering significant opportunities but also raising concerns about their effects on self-assessment and academic performance. This study investigates the effectiveness of Kahoot! compared to traditional instructional methods in enhancing mathematics achievement and its impact on multiple screen addiction (MSA) among Greek students aged 9 to 12 during a STEM summer camp. A quasi-experimental design was employed with a purposefully selected sample of one hundred and ten (n = 110) students, who were non-randomly divided into two groups: (a) an experimental group of fifty-five students (n = 55) who engaged with Kahoot! (using dynamic visual aids and interactive content) and (b) a control group of fifty-five students (n = 55) who received traditional instruction (using digital textbooks and PowerPoint slides with multimedia content) on laptops and tablets. The findings revealed a statistically significant difference in MSA scores, with the experimental group exhibiting lower MSA scores compared to their counterparts, indicating a positive impact on reducing screen addiction levels. While Kahoot! led to lower MSA levels, it significantly improved overall mathematical achievement, with a substantial effect size, suggesting a strong positive impact on learning outcomes. The current study highlights the importance of aligning educational tools with the intended outcomes and recommends further research to explore the broader impact of gamified learning on student engagement, screen addiction, and learning outcomes. Full article

(This article belongs to the Special Issue Innovative Theories and Practices for Designing and Evaluating Inclusive Educational Technology and Online Learning)

18 pages, 1971 KiB

Open AccessSystematic Review

Extended Reality Applications for CNC Machine Training: A Systematic Review

by José Manuel Ibarra Kwick, Óscar Hernández-Uribe, Leonor Adriana Cárdenas-Robledo and Ramón Alberto Luque-Morales

Multimodal Technol. Interact. 2024, 8(9), 80; https://doi.org/10.3390/mti8090080 - 11 Sep 2024

Viewed by 1180

Abstract

Extended reality (XR) as an immersive technology has gained significant interest in the industry for training and maintenance tasks. It offers an interactive, three-dimensional environment that can boost users’ efficiency and safety in various sectors. The present systematic review provides information based on [...] Read more.

Extended reality (XR) as an immersive technology has gained significant interest in the industry for training and maintenance tasks. It offers an interactive, three-dimensional environment that can boost users’ efficiency and safety in various sectors. The present systematic review provides information based on a Scopus database search for research articles from 2011 to 2024 to expose 19 selected studies related to XR developments and approaches. The purpose is to grasp the state of the art, focusing on user training in goals or tasks that involve computer numerical control (CNC) machines. The study revealed approaches that broadly employed XR devices to execute diverse operations for virtual CNC machines, offering enhanced safety and skills acquisition, lessening the use of physical machines that impact energy consumption or the time invested by an expert worker to teach an operation task. The articles highlight the advantages of XR training versus traditional training in CNC machines, revealing an opportunity to enhance learning aligned to the industry 4.0 (I4.0) paradigm. Virtual reality (VR) and augmented reality (AR) applications are the most used and are mainly centered on a single-user environment. In addition, a VR approach is built as a proof of concept for learning CNC machine operations, considering the key features identified. Full article

► Show Figures

Figure 1

9 pages, 233 KiB

Open AccessBrief Report

Can Generative AI Contribute to Health Literacy? A Study in the Field of Ophthalmology

by Carlos Ruiz-Núñez, Javier Gismero Rodríguez, Antonio J. Garcia Ruiz, Saturnino Manuel Gismero Moreno, María Sonia Cañizal Santos and Iván Herrera-Peco

Multimodal Technol. Interact. 2024, 8(9), 79; https://doi.org/10.3390/mti8090079 - 4 Sep 2024

Viewed by 934

Abstract

ChatGPT, a generative artificial intelligence model, can provide useful and reliable responses in the field of ophthalmology, comparable to those of medical professionals. Twelve frequently asked questions from ophthalmology patients were selected, and responses were generated both in the role of an expert [...] Read more.

ChatGPT, a generative artificial intelligence model, can provide useful and reliable responses in the field of ophthalmology, comparable to those of medical professionals. Twelve frequently asked questions from ophthalmology patients were selected, and responses were generated both in the role of an expert user and a non-expert user. The responses were evaluated by ophthalmologists using three scales: Global Quality Score (GQS), Reliability Score (RS), and Usefulness Score (US), and analyzed statistically through descriptive study, association, and comparison. The results indicate that there are no significant differences between the responses of expert and non-expert users, although the responses from the expert user tend to be slightly better rated. ChatGPT’s responses proved to be reliable and useful, suggesting its potential as a complementary tool to enhance health literacy and alleviate the informational burden on healthcare professionals. Full article

22 pages, 15858 KiB

Open AccessArticle

VRChances: An Immersive Virtual Reality Experience to Support Teenagers in Their Career Decisions

by Michael Holly, Carina Weichselbraun, Florian Wohlmuth, Florian Glawogger, Maria Seiser, Philipp Einwallner and Johanna Pirker

Multimodal Technol. Interact. 2024, 8(9), 78; https://doi.org/10.3390/mti8090078 - 4 Sep 2024

Viewed by 850

Abstract

In this paper, we present a tool that offers young people virtual career guidance through an immersive virtual reality (VR) experience. While virtual environments provide an effective way to explore different experiences, VR offers users immersive interactions with simulated 3D environments. This allows [...] Read more.

In this paper, we present a tool that offers young people virtual career guidance through an immersive virtual reality (VR) experience. While virtual environments provide an effective way to explore different experiences, VR offers users immersive interactions with simulated 3D environments. This allows the realistic exploration of different job fields in a virtual environment without being physically present. The study investigates the extent to which performing occupational tasks in a virtual environment influences the career perceptions of young adults and whether it enhances their understanding of professions. In particular, the study focuses on users’ expectations of an electrician’s profession. In total, 23 teenagers and eight application experts were involved to assess the teenager’s expectations and the potential of the career guidance tool. Full article

► Show Figures

Figure 1

24 pages, 1006 KiB

Open AccessArticle

A Multispecies Interaction Design Approach: Introducing the Beings Activities Context Technologies (BACT) Framework

by Theodora Chamaidi and Modestos Stavrakis

Multimodal Technol. Interact. 2024, 8(9), 77; https://doi.org/10.3390/mti8090077 - 4 Sep 2024

Viewed by 823

Abstract

For years, design has been focused on human needs, creating human-centred solutions and often neglecting the existence or the impact that design can have on other species. As designers shift from that traditional anthropocentric approach to adopting design practices that include other species’ [...] Read more.

For years, design has been focused on human needs, creating human-centred solutions and often neglecting the existence or the impact that design can have on other species. As designers shift from that traditional anthropocentric approach to adopting design practices that include other species’ perspectives in the process, there is a growing need for practices capable of providing designers with the right tools to understand non-human needs and design for their inclusion. For this reason, the Beings Activities Context Technologies (BACT) framework is proposed as a theoretical means to support the shift to a more multispecies-oriented approach, expanding the anthropocentric Benyon’s People Activities Contexts Technologies (PACT) framework. The methodological implications of the framework have been explored in a case study design project focused on the development of a wearable device designed to support beekeepers during their work. The case study explored the design by taking into consideration both the needs of humans and animals in the context of beekeeping while analysing their interactions in depth. Through this framework, we seek to contribute to the more-than-human turn in interaction design and aid designers in expanding their considerations beyond the person–technology relationship. Full article

► Show Figures

Figure 1

47 pages, 2588 KiB

Open AccessArticle

Observations and Considerations for Implementing Vibration Signals as an Input Technique for Mobile Devices

by Thomas Hrast, David Ahlström and Martin Hitz

Multimodal Technol. Interact. 2024, 8(9), 76; https://doi.org/10.3390/mti8090076 - 2 Sep 2024

Viewed by 902

Abstract

This work examines swipe-based interactions on smart devices, like smartphones and smartwatches, that detect vibration signals through defined swipe surfaces. We investigate how these devices, held in users’ hands or worn on their wrists, process vibration signals from swipe interactions and ambient noise [...] Read more.

This work examines swipe-based interactions on smart devices, like smartphones and smartwatches, that detect vibration signals through defined swipe surfaces. We investigate how these devices, held in users’ hands or worn on their wrists, process vibration signals from swipe interactions and ambient noise using a support vector machine (SVM). The work details the signal processing workflow involving filters, sliding windows, feature vectors, SVM kernels, and ambient noise management. It includes how we separate the vibration signal from a potential swipe surface and ambient noise. We explore both software and human factors influencing the signals: the former includes the computational techniques mentioned, while the latter encompasses swipe orientation, contact, and movement. Our findings show that the SVM classifies swipe surface signals with an accuracy of 69.61% when both devices are used, 97.59% with only the smartphone, and 99.79% with only the smartwatch. However, the classification accuracy drops to about 50% in field user studies simulating real-world conditions such as phone calls, typing, walking, and other undirected movements throughout the day. The decline in performance under these conditions suggests challenges in ambient noise discrimination, which this work discusses, along with potential strategies for improvement in future research. Full article

(This article belongs to the Special Issue Multimodal User Interfaces and Experiences: Challenges, Applications, and Perspectives)

► Show Figures

Figure 1

20 pages, 6857 KiB

Open AccessReview

Impact of Artificial Intelligence on Learning Management Systems: A Bibliometric Review

by Diego Vergara, Georgios Lampropoulos, Álvaro Antón-Sancho and Pablo Fernández-Arias

Multimodal Technol. Interact. 2024, 8(9), 75; https://doi.org/10.3390/mti8090075 - 25 Aug 2024

Viewed by 3355

Abstract

The field of artificial intelligence is drastically advancing. This study aims to provide an overview of the integration of artificial intelligence into learning management systems. This study followed a bibliometric review approach. Specifically, following the Preferred Reporting Items for Systematic reviews and Meta-Analyses [...] Read more.

The field of artificial intelligence is drastically advancing. This study aims to provide an overview of the integration of artificial intelligence into learning management systems. This study followed a bibliometric review approach. Specifically, following the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement, 256 documents from the Scopus and Web of Science (WoS) databases over the period of 2004–2023 were identified and examined. Besides an analysis of the documents within the existing literature, emerging themes and topics were identified, and directions and recommendations for future research are provided. Based on the outcomes, the use of artificial intelligence within learning management systems offers adaptive and personalized learning experiences, promotes active learning, and supports self-regulated learning in face-to-face, hybrid, and online learning environments. Additionally, learning management systems enriched with artificial intelligence can improve students’ learning outcomes, engagement, and motivation. Their ability to increase accessibility and ensure equal access to education by supporting open educational resources was evident. However, the need to develop effective design approaches, evaluation methods, and methodologies to successfully integrate them within classrooms emerged as an issue to be solved. Finally, the need to further explore education stakeholders’ artificial intelligence literacy also arose. Full article

► Show Figures

Figure 1

Journal Menu

Journal Browser

Multimodal Technol. Interact., Volume 8, Issue 9 (September 2024) – 9 articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI