Multimodal Technol. Interact., Volume 7, Issue 12 (December 2023) – 10 articles

Social robots have the potential to support health and quality of life for children experiencing anxiety. We engaged families with lived experiences of pediatric anxiety in social robot development to explore desired design features, application areas, and emotion functionalities of social robots in anxiety care. We conducted 10 online co-creation workshops with (1) children with anxiety aged 7–13 (n = 24) with their family members (n = 20), and (2) youth with anxiety aged 14–18 (n = 12). Workshop participation included a validated robot expectations scale, anonymous polls, and discussion. Transcripts and text responses were subjected to content analysis. A lived experience expert group provided feedback throughout the research. Participants desired a pet-like robot with a soft texture, expressive eyes, and emotion detection to support activities of daily living. Specific anxiety-related applications included breathing exercises, managing distressing thoughts, and encouragement. Emotional alignment, the design of a robot’s emotional display, and the emotional impacts of an interaction were discussed. Privacy and the replacement of human interaction were concerns. We identify pediatric anxiety-specific design features, applications, and affective considerations for existing and future social robots. Our findings highlight the need for customizability and robust emotional functionality in social robot technologies intended to support the health and care of children living with anxiety. Full article

Creating digital twins of real objects is becoming more popular, with smartphones providing 3D scanning capabilities. Adding semantics to the reconstructed virtual objects is important to possibly reproduce training scenarios that otherwise could demand significant resources or happen in dangerous scenarios in some cases. The aim of this work is to evaluate the usability of authoring object component behaviors in immersive and non-immersive approaches. Therefore, we present an evaluation of the perceived ease of use to author moving parts of objects under three different conditions: desktop, augmented reality (AR) and virtual reality (VR). This paper provides insights into the perceived benefits and issues that domain experts might encounter when authoring geometrical component behaviors across each interface. A within-subject study is the major contribution of this work, from which is presented an analysis based on the usability, workload and user interface preferences of participants in the study. To reduce confounding variables in the study, we ensured that the virtual objects and the environment used for the evaluation were digital twins of the real objects and the environment that the experiment took place in. Results show that the desktop interface was perceived as more efficient and easier to use based on usability and workload measures. The desktop was preferred for performing component selection but no difference was found in the preference for defining a behavior and visualizing it. Based on these results, a set of recommendations and future directions are provided to achieve a more usable, immersive authoring experience. Full article

There are various ways that teachers manage student disengagement levels during their class lessons, and managing disengagement can be both stressful and challenging, especially since each student is unique. Methods and techniques utilised are specific to teachers’ own experience level, subject knowledge, and teaching styles. We report on the techniques and methods teachers utilise to identify, mitigate, and measure student disengagement during class lessons; the paper presents the results of a mixed-methods, multisession study design comprising gathered qualitative and quantitative data to enable a greater understanding. Eight educators who were full-time educators with varying years of experience from three different schools, who taught or had taught English, maths, and science subjects at the primary school level, participated in this study. The study also observed that teachers used three AR applications and collected valuable feedback on their perspectives by using analytics generated by AR applications to help manage student disengagement. A postsession survey tool was used to gather the perceived importance and ranking of the techniques and methods discussed by the teachers during the previous sessions. The results showed that the majority of teachers deemed spending “Time on Tasks” and giving “Feedback/Reflections” most suited for measuring disengagement, and encouraging “Movement” and use of “Technology” emerged as the most favoured for mitigating disengagement. For utilising AR enhanced analytics in mitigating and measuring student disengagement, the data suggested a difference in perspectives based on teachers’ teaching levels, especially concerning conversations and the use of technology devices. The study did not find conclusive evidence of differences based on teachers’ teaching subjects and there was a notable distinction in building positive relationships among English teachers. This leads to the suggestion that subject-specific pedagogy might influence the perceived effectiveness of using AR-generated analytics in mitigating and measuring student disengagement. Full article

The article aims to reflect on the Lodovico media library, a digital repository preserving the digitised cultural heritage of the Emilia-Romagna region. The first part covers the project’s history and the challenges encountered during its setup phase, and we also explore the co-creation approach employed in defining the metadata architecture. The discussion extends by outlining the key features of shared metadata, illustrating their application to diverse digital objects within the Lodovico media library. Following a concise examination of the methodology for collecting/creating data and the initial research findings, the article concludes by highlighting the project’s potential in the realm of automatic handwriting recognition processes. Full article

This study explores the conventional ‘funneling’ method by introducing two extra locations beyond the virtual reality (VR) controller boundaries, terming it the extended funneling technique. Thirty-two participants engaged in a localization task, with their responses recorded using eye-tracking technology. They were tasked with localizing a virtual ping-pong ball as it bounced both within and outside their virtual hands on a virtual board. Both the experimental and control groups received simultaneous spatial audio and vibrotactile feedback. The experimental group received vibrotactile feedback with extended funneling, while the control group received vibrotactile feedback without funneling for comparison. The results indicate that the experimental group, benefiting from the extended funneling technique, demonstrated a significantly higher accuracy rate (41.79%) in localizing audio–vibrotactile stimuli compared to the control group (28.21%). No significant differences emerged in embodiment or workload scores. These findings highlight the effectiveness of extended funneling for enhancing the localization of sensory stimuli in VR. Full article

Mixed reality (MR) technology has the potential to enhance the disaster preparedness of medical first responders in mass-casualty incidents through new training methods. In this manuscript, we present an MR training solution based on requirements collected from experienced medical first responders and technical experts, regular end-user feedback received through the iterative design process used to develop a prototype and feedback from two initial field trials. We discuss key features essential for an effective MR training system, including flexible scenario design, added realism through patient simulator manikins and objective performance assessment. Current technological challenges such as the responsiveness of avatars and the complexity of smart scenario control are also addressed, along with the future potential for integrating artificial intelligence. Furthermore, an advanced analytics and statistics tool that incorporates complex data integration, machine learning for data analysis and visualization techniques for performance evaluation is presented. Full article

This study focuses on the development and assessment of a serious game for health (SGH) aimed at educating children about cardiopulmonary resuscitation (CPR). A video game was created using the Berkeley Snap platform, which uses block programming. Eye-tracking technology was utilized to validate the graphic design. To assess the tool’s effectiveness, a pre-post analytical study was conducted with primary education children to measure the knowledge acquired. The study involved 52 participants with a mean age of 9 years. The results from a custom questionnaire used to measure their theoretical CPR knowledge showed significant improvements in CPR knowledge after the use of the videogame, and their emotional responses improved as well. The assessment of the knowledge acquired through the video game obtained an average score of 5.25 out of 6. Ten video segments consisting of 500 frames each (20 s of video per segment) were analyzed. Within these segments, specific areas that captured the most relevant interaction elements were selected to measure the child’s attention during game play. The average number of gaze fixations, indicating the points in which the child’s attention was placed within the area of interest, was 361.5 out of 500. In conclusion, the utilization of SGH may be an effective method for educating kids about CPR, to provide them with fundamental knowledge relevant to their age group. Full article

In recent years, there has been a surge of interest in affordable and accessible extended reality devices. Big tech companies like Apple and Meta have announced advanced devices expected to become more prevalent in everyday life. As younger generations embrace immersive digital realities for socialization, entertainment, and information retrieval, there is a need to explore immersive digital technologies that support experiential learning and reevaluate educational approaches. In Italy, the COVID-19 pandemic has sparked a growing interest in immersive virtual reality (VR) and the metaverse for distance education. However, the integration of VR in Italian schools could be faster, primarily due to cost and teacher knowledge challenges. Our study aims to involve high school teachers in a practical workshop to assess their knowledge, skills, and intention to use VR in their teaching after brief training. The focus is on evaluating the acceptability of VR for educational purposes among Italian high school teachers. The workshop involved up to 16 teachers at once and was repeated eight times to reach 120 teachers. Participants received VR training and explored three educational VR applications. The results show that teachers are interested in learning and integrating VR into their lessons. They believe it can enhance teaching practices by actively engaging students and enabling experiential learning. This work provides an overview of the current state of VR in education, describes the workshop with high school teachers, and presents the obtained results. Full article

Video vignettes are one form of virtualized vignettes that may build upon traditional text vignettes and enable research participants to see and experience a unique scenario that is better translated visually than through a written text. This study examined using video vignettes to study perceptions of leaders. Participants watched virtualized, video vignettes depicting a male leader expressing either a masculine, gender-conforming expression or a feminine, gender-nonconforming expression. Participants evaluated these leaders on measures of leadership likability and leadership effectiveness. Results demonstrated that the videos of the masculine male leader were perceived as more likable and more effective than the videos of the feminine male leader. This relationship was not moderated by gender-related expectations the participants had. This finding reveals that there is a prototypical expectation that male leaders who depicted masculine expressions are more likable and effective. Additionally, we also found that the participant’s individual ideologies of gender-related expectations and conformity expectations were related to the results in unique ways. When the participant believed an ideal leader should have higher versus lower feminine traits, those participants also rated both leaders more positively regardless of the type of gender expression that was depicted in the video vignettes. In contrast, participants with strong expectations that others should conform to gender norms in social situations generally rated both leaders more negatively, regardless of the gender expression in the video vignette. The results are discussed in detail and the benefits of using virtualized and video vignettes are discussed. Full article

Several studies have researched how people can use the anonymity of the Internet to explore different aspects of their identity. There are many different platforms where one can create a “virtual self” by actively choosing how one presents themselves to others, and each one is designed with different characteristics depending on their purpose: from socialization to professional networking or even entertainment. Different studies have usually focused on specific platforms, but there is no one comparing how people construe their virtual self across different online environments. In this review, we aimed to synthesize research studies carried out about the construal of one’s identity within online platforms and how it can relate to different aspects of participants’ offline identity, such as their self-esteem or self-concept clarity. Data were gathered from 34 publications that researched this topic across different kinds of online platforms. We conducted a quality assessment and a narrative synthesis, reporting and comparing the main findings, as well as identifying possible gaps in the literature. Many studies have explored the idea of people creating either an idealized or accurate version of themselves when construing their online identities. Others have also contemplated the possibility of exploring identities that diverge from both the actual and the ideal self or even an actively negative one. The latter was studied more in the context of video games and other avatar-mediated worlds. We found that people with low self-esteem create a more idealized virtual identity rather than a virtual self that is similar to their offline self. Other variables such as the purpose for using the online platform or self-concept clarity also had a role in virtual self construal, although the latter was only studied in the context of social media. Full article