A Review of Immersive Technologies, Knowledge Representation, and AI for Human-Centered Digital Experiences

Abstract

The evolution of digital technologies has resulted in the emergence of diverse interaction technologies. In this paper, we conducted a review of seven domains under a human-centric approach user interface design, human-centered web-based information systems, semantic knowledge representation, X-reality applications, human motion and 3D digitization, serious games, and AI. In this review, we studied these domains concerning their impact on the way we interact with digital interfaces, process information, and engage in immersive experiences. As such, we highlighted the shifts in design paradigms, user-centered principles, and the rise of web-based information systems. The results of such shifts are materialized in modern immersive technologies, semantic knowledge representation, serious games, and the facilitation of artificial intelligence for interactions. Through this exploration, we aimed to assist our understanding of the challenges that lie ahead. The seamless integration of technologies, ethical considerations, accessibility, education for technological literacy, interoperability, user trust, environmental sustainability, and regulatory frameworks are becoming significant. These challenges present opportunities for the future to enrich human experiences while addressing societal needs. This paper lays the groundwork for thoughtful and innovative approaches to the challenges that will define the future of human–computer interaction and information technologies.

1. Introduction and Orientation

Human–computer interaction (HCI) and information and communication technologies (ICT) are experiencing a transition due to the emergence of innovative design paradigms that are affecting the way we interact with digital information. Immersive technologies, such as augmented reality (AR), virtual reality (VR), and mixed reality (MR), are considered the building blocks of modern digital experiences. Knowledge representation in a machine-interpretable format supports reasoning on knowledge sources to shape intelligent, user-centric information systems. From a human-centered perspective, these systems have the potential to dynamically adapt to the diverse and evolving needs of users. Simultaneously, the integration of artificial intelligence (AI) into ICT creates the foundation for new interaction technologies, information processing, and information visualization.

Our motivation for providing this review stems from the rapidly evolving nature of these developments, which inevitably creates gaps in understanding the implications of immersive technologies, knowledge representation, and AI on user-centered digital experiences. Bridging these gaps and providing insights into this journey can offer a roadmap for future research endeavors. In the same context, it is important to highlight the fact that research gaps exist both in terms of technology and also in the way that these modern developments become integrated, tested, and evaluated. In general, the transformative impact of innovative design paradigms is not sufficiently addressed. Such considerations should be studied in terms of cognitive aspects relevant to the interaction itself and with regard to novel visualization paradigms and information processing principles. Towards supporting such interaction, the need for semantic knowledge representation, both on the knowledge side and on the user data side, has not yet been adequately explored. The same can be said for the effects of immersive technologies in education and training, since existing novel approaches have not been sufficiently validated. In accordance with the integration of 3D digitization and motion-driven interaction, these still remain in a state of generating impressions by proposing new paradigms rather than integrating these paradigms into common practice. In this domain, perhaps the most successful attempts were the Microsoft Kinect gaming experience series in Xbox 360 and Xbox One [1] and the PS move controller variations in PS3 and PS4 VR [2]. Similarly for AI, currently isolated methods are provided for different application domains without providing horizontal integration. Concurrently, ethical aspects in AI research have not been adequately studied nor integrated into development approaches.

In this paper, an analysis was conducted by reviewing recent literature from seven application domains, each representing a crucial aspect of the evolution of HCI and ICT. However, the choice of these domains was not arbitrary; rather, it was rooted in their significance, and they were selected based on their role in shaping human-centered experiences and for their offerings toward innovative design paradigms. The emphasis was on highlighting the importance of these selected domains and their specific contributions to the transformation of HCI and ICT. However, it is crucial to acknowledge that the focus on these specific domains does not diminish the importance of other potential domains. The rationale for selecting these particular domains was rooted in their demonstrable impact and relevance to the broader landscape, recognizing that other domains may also play significant roles in the transformative journey of HCI and ICT. To this end, in this work, we argue that user interface design, knowledge representation, semantic knowledge, X-reality applications, human motion and 3D digitization, serious games, and AI approaches are not only individually important but collectively supportive to comprehending the broader landscape of HCI and ICT.

Starting from user interface design, explicit and implicit interactions were discussed, exploring the evolution of intuitive gestural interfaces, conversational agents, and the broader landscape that redefines user experiences. This section focuses on the dynamic nature of user interfaces, where human–computer interactions are evolving towards seamless and intuitive engagements. Modern interaction technologies can be traced back to the advent of user interfaces (UIs), where command-line interactions marked the early stages of HCI [3]. As technology matured, graphical user interfaces (GUIs) emerged, introducing a visual paradigm that laid the foundation for intuitive interactions [4]. Desktop computing was the next big thing, allowing users to navigate through digital realms via a mouse and keyboard [5].

Both implicit and explicit forms of interaction consider that the user is the one that is driving the interaction and is thus the one targeted by a computing application. Since not all users are the same and there is no solution that fits all users, a user-centered design (UCD) paradigm emerged. UCD places user experience at the forefront of technological development [6] and involves integrating end users, their needs, and expectations in all phases of development of a UI, from user requirement analysis to design and implementation. Human-centered principles today do not just affect the design of the interaction and of the UI but also the nature of the facilities offered by information systems. To support each individual user in such a dynamic and responsive environment, the importance of adaptivity, adaptability [7,8], and accessibility [9] were emphasized in UCD. Touchscreens and gestures were proposed [10,11] as new interaction paradigms, while feedback mechanisms were integrated into UIs to enhance user satisfaction and engagement [12,13].

Today, the majority of time spent on the computer worldwide is linked to the World Wide Web and is supported through some form of web-based information systems in broad terms [14]. Such systems provide a seamless integration of information from databases and support for interconnectedness [15,16]. Knowledge representation [17,18], data visualization [19], and data mining [20] have become components of these new developments, transforming them into hubs of information dissemination and retrieval [21]. In such systems, semantic knowledge is important, because it can enhance user interaction, support active participation in information processing, and enrich user experiences through the understanding of both context and content. In the domain of knowledge representation, the Semantic Web [22] brought representation technologies [23,24] that were capable of providing meaning to data. The goal was to make data easier to interpret by machines [25]. Ontologies and semantics were the building blocks representing relationships and context, leading to more sophisticated user interactions [26].

The interactions described above take place in the computing universe we are all today aware of. In parallel, though, a more disruptive interaction paradigm is taking place, rooted in immersive technologies such as augmented, virtual, and mixed reality (AR, VR, and MR). Apart from gaming special-purpose applications (e.g., in museums, science centers, 4D cinemas, etc.), these interaction paradigms exist in a newborn universe. In this review paper, we discuss the domains where these have started to prove their worth and potential for exploitation, with a special focus on their applications for vocational education and training. In these domains, a truly transformative journey is currently being undertaken, shaping new ways for using innovative technologies as a disruptive innovation. The advent of immersive technologies, [27,28] AR [29], VR [30], and MR [31], made possible the transition from traditional screens to the digital three-dimensional space, supporting both hybrid and pure digital training experiences. XR applications in vocational education and training exemplify the capacity of these technologies to revolutionize experiential learning [32,33,34].

Key building blocks of the immersive technologies discussed are human motion capture and 3D digitization. The focus is on how these technologies redefine digital experiences and introduce novel interaction metaphors. Our analysis starts from virtual embodiment, where users project themselves into digital avatars, to lifelike simulations that bridge the gap between the physical and digital worlds. In such applications, human motion capture [35,36] and 3D digitization [37,38,39,40] enable users to interact with digital environments in ways that mirror real-world movements and real-world environments [41,42,43]. Such realistic simulations [44] blur the boundaries between physical and digital realities, fostering a deeper level of engagement and personalization in digital experiences.

Serious games can be considered as the domain where user-centric principles and the innovation described in AR, VR, and MR join forces to repurpose gaming and address educational, training, and societal challenges. Serious games offer purposeful play that transcends entertainment, becoming powerful tools for knowledge and skill acquisition and thus addressing broader societal issues. Serious games employ XR technologies and game design principles to address educational, training, and societal challenges [45]. These games are based on a paradigm shift from entertainment-focused gaming to skill development, knowledge acquisition, and societal awareness [46]. At the same time, the integration of artificial intelligence (AI) into UIs [47,48] empowers systems to understand user intent; process information efficiently; and offer personalized, anticipatory digital experiences [49].

The forthcoming artificial intelligence (AI) revolution [50] has started by providing tools that currently alter the way we work by introducing a novel form of AI–human collaboration [51,52]. In the future, we expect that the way we interact with technology will be greatly affected by AI-driven approaches, which will be used as the building blocks of new interaction paradigms. As a result, this subject is discussed under the prism of the changes that it produces, possibly altering interaction as we perceive it today.

The main body of this work is dedicated to presenting a short review of the state of the art of these application domains and discussing the reviewed literature. Then, it concludes with a presentation of the challenges, including ethical considerations in design and AI practices and the continuous adaptation to evolving user behaviors. Looking ahead, future directions encompass the human-centric integration of AI, the development of immersive and multisensory interfaces, and the enhanced accessibility of digital platforms. Ethical design principles and responsible AI practices are expected to become increasingly integral, as cross-disciplinary collaboration enriches the design process. The adoption of natural and intuitive interactions, real-time user feedback mechanisms, and personalized experiences represent the evolving landscape, indicating a future where technology harmoniously aligns with human needs, preferences, and ethical standards.

2. Advances in User Interface Design, Development, and Evaluation, including New Approaches for Explicit and Implicit Interaction

UI design, development, and evaluation made possible the facilitation of new technologies and supported a deeper understanding of user behavior. This has been manifested through a combination of explicit and implicit interaction approaches.

Explicit interaction can be perceived as a form of communication with a computing device where explicit user input is directly connected to the issuing of a command. Traditionally, explicit interaction involved direct user inputs, such as with a mouse, a keyboard, or touch gestures. However, late advancements have elevated explicit interaction by introducing gesture-based interfaces (e.g., [53,54,55,56,57,58]) and kinesthetic interaction paradigms (e.g., [59,60]). These forms use technologies like computer vision, depth sensing, IR sensing, and tracking devices (RGB-D sensors, RGB cameras, infrared cameras) and enable users to communicate with systems through intuitive hand movements [61,62]. The advancements discussed in these works include tracking hand movements, instead of just static poses [53], optimizing gesture recognition from a monocular camera to support gaming in public spaces [55], using specialized wearable devices for gesture recognition [57], optimizing traditional RGB-D approached with AI [58,61], etc. Having these novel forms of interaction allows for a more immersive experience, and natural interactions can be supported. Especially in VR, new interaction paradigms can replace classic VR controllers and support more natural interaction through gestures and real-time hand tracking to augment the feeling of presence in a virtual space (e.g., [63,64]). Similar effects can be achieved in AR by enhancing interaction with digitally augmented physical objects (e.g., [65,66]).

Voice-activated interfaces [67,68] represent another advancement in explicit interaction, implemented through the integration of advanced natural language processing (NLP) algorithms, thus supporting more than traditional voice command recognition [69]. Today, intelligent voice assistants are capable of doing much more, such as comprehending context, discerning user intent, and being of assistance in typical daily tasks (e.g., [70,71]), making digital systems more inclusive for users with diverse abilities.

Implicit interaction is defined as “an action performed by the user that is not primarily aimed to interact with a computerized system but which such a system understands as input” [72]. In this domain, machine learning algorithms empower systems to discern user preferences, anticipate actions, and adapt interfaces and robots in real time [73,74,75]. Predictive modeling, driven by user behavior analysis, enables interfaces to become more personalized, offering a tailored experience that aligns with individual needs and preferences [76,77].

The fusion of explicit and implicit interaction is evident in the rise of anticipatory design [78]. Interfaces are becoming increasingly adept at predicting user actions, streamlining workflows, and minimizing decision fatigue [79]. Through the seamless integration of explicit inputs and implicit learning, systems can offer a more fluid and intuitive user experience.

As UI paradigms evolve, so too must the methods for evaluating their effectiveness. Traditional usability testing [80,81] and heuristic evaluations [82] are now complemented by sophisticated analytics and user feedback mechanisms [83,84,85,86]. A holistic understanding of user experience requires a multidimensional approach that considers not only task completion efficiency but also emotional engagement, accessibility, and inclusivity [87]. Eye-tracking technology and neuroscientific methods are emerging as powerful tools for evaluating implicit interactions [88]. By examining gaze patterns and neural responses, designers gain insights into user attention, emotional responses, and cognitive load, providing valuable feedback for refining UI designs [89,90].

3. Human-Centered Web-Based Information Systems

Today, web-based information systems try to integrate the principles of human-centered design [91]. To this end, a combination of ICT advances is being integrated into such systems, including knowledge representation approaches, data visualization paradigms, data mining methodologies, and big data analytics technologies [92,93]. This evolution supports more dynamic information delivery on the one hand and user-centric experiences that generate insights from large datasets on the other hand. The main objective is to enhance the visualization capacity over immersive amounts of information to render large datasets more readable for humans to understand and work with. The core of such approaches is the deployment of knowledge representation techniques [94,95]. Semantic web technologies, ontologies, and graph databases organize and structure information in a manner that is both human- and machine-interpretable. At the same time, these advancements further pose the need to introduce cognitive approaches in the semantic web, enabling systems to not only store and retrieve data but also to infer relationships, fostering a more accurate understanding of context [96,97]. Visual interfaces should help users locate information based on meaning while keeping the complexity of the semantic implementation hidden. For example, using similarities and comparisons can make it easier to navigate through a lot of information. Instead of having fixed representations of data, cognitive approaches should support choosing information based on user needs.

Such approaches are a prerequisite to reducing the threat of information overload addressed by effective data visualization. Modern web-based systems deploy interactive and immersive visualizations to present complex datasets via usable representations [98]. From interactive charts and graphs to VR-enhanced visualizations, the emphasis is on empowering users to explore and understand information intuitively, enhancing the overall user experience [99,100].

To achieve intuitive big data visualization, sophisticated tools for analysis and interpretation are needed. Data mining techniques augment web-based information systems, facilitating the discovery of patterns, trends, and anomalies within large datasets [101]. Machine learning algorithms enable systems to autonomously uncover hidden knowledge [102], providing users with recommendations and insights. Human-centered web-based systems are thus capable of employing distributed computing frameworks and cloud technologies to process datasets in real time. The synergy of big data analysis and visualization empowers users to promptly gain meaningful insights, fostering informed decision-making [103].

From a design perspective today, information systems’ interfaces are crafted with a deep understanding of user needs, preferences, and cognitive processes [104,105]. Personalization algorithms, informed by user interactions and feedback, ensure that the information presented is not only relevant but also delivered in a format that resonates with the user’s mental model. At the same time, continuous evaluation and iterative design are integral components of human-centered web-based information systems [106,107]. Analytics tools track user interactions, enabling designers to refine interfaces and functionalities based on real-world usage patterns [108,109,110]. This iterative process ensures that systems remain adaptive, responsive, and aligned with evolving user expectations [111].

4. Semantic Knowledge to Enhance User Interaction with Information, User Participation in Information Processing, and User Experience

Semantic knowledge representation provides meaning to the data processed by an information system and can thus support a more intelligent and intuitive interaction between users and information [112,113]. In this context, semantic technologies are used to represent knowledge in a machine-understandable format. This format can make various information systems semantically interoperable [114]. Ontologies, linked data, and semantic graphs provide a rich framework for expressing relationships between concepts [115], allowing systems to infer and connect pieces of data, creating a web of contextual relevance. Semantic knowledge representation lays the groundwork for a more intuitive and context-aware service provision [116]. NLP algorithms, powered by semantic models, enable systems to comprehend user queries in a more human-like manner [117]. Conversational interfaces, driven by semantic understanding, facilitate seamless interactions, allowing users to communicate with systems more naturally and dynamically [118]. A key advancement is the empowerment of users in the information processing chain. Collaborative knowledge creation and annotation, supported by semantic frameworks, enable users to contribute to the refinement and enrichment of data. This participatory approach not only enhances the accuracy of information but also fosters a sense of ownership and engagement among users in the information ecosystem [119,120].

Beyond representation, the presentation of information is an important part of user experience. Semantic technologies should influence how information is visually and contextually communicated to users [121,122], ensuring that users receive information in a format that aligns with their cognitive processes and preferences. In the same context, personalization algorithms, leveraging semantic understanding, deliver content that is not only relevant but also anticipates user needs [123,124]. The seamless integration of diverse datasets, facilitated by semantic frameworks, has the potential to provide a more coherent and holistic user experience, reducing information overload and enhancing overall satisfaction. The iterative nature of semantic knowledge representation ensures continuous improvement through feedback loops, driven by user interactions and system analytics, to enable adaptive learning and refinement.

5. X-Reality Applications (AR, VR, MR) for Immersive Human-Centered Experiences

X-reality applications today offer novel forms of immersion. In this paper, we consider two main application domains: namely, cultural heritage and vocational training.

5.1. X-Reality Applications in Cultural Heritage

The utilization of virtual reality (VR) in cultural heritage (CH) is not a novel concept. Initial approaches, such as CAVE-based VR, integrated immersive presentations and haptic-based manipulations of heritage objects [125,126]. A synergy of 3D reconstruction technologies with VR emerged, creating realistic digital replicas of CH objects [127,128]. In earlier methods, where digitization was constrained by technology immaturity, scenes from archaeological sites were manually modeled in 3D [129,130]. While resulting in lower-quality models, this allowed researchers to digitally restore monuments by complementing structural remains with digitally manufactured structures [131,132]. Advancements extended to simulating weather and daily life in ancient CH sites through a graphics-based rendering of nature and autonomous virtual humans.

The evolution of VR devices, particularly commercial VR headsets and controllers [133,134], simplified the implementation of VR-based experiences. Concurrently, the advent of 360 photography and videos enabled a different VR approach with inexpensive headsets, augmenting experiences with information points and interactive spots [135,136,137]. Studies have addressed resource-demanding tasks like streaming 360 videos in these headsets [138,139]. From a sustainability perspective, VR was proposed to divert visits from endangered CH sites to digital media [140].

In the domain of augmented reality (AR) and cultural heritage, ongoing research has shown its potential to enhance learning by providing a more comprehensive educational experience [141]. AR applications have been explored in school subjects like chemistry and cultural heritage sites [142,143]. Stakeholder studies indicate perceived value dimensions of AR in cultural heritage tourism, encompassing economic, experiential, social, epistemic, historical, cultural, and educational aspects [144].

Mobile AR research began with feature extraction in mobile phones for image acquisition [145]. More advanced mobile devices incorporated virtual humans [146], while modern mobile phones empowered various AR forms, such as the augmentation of camera images with information [147,148] and the interpolation of 3D digitization with camera input [149]. Some approaches replace physical remains with digitally enhanced versions from the time of creation [150], and physical objects aid visualization and interaction with archaeological artifacts in AR [151,152].

The fusion of augmented and virtual reality has given rise to “AR Portals”. Mobile devices, supporting larger AR scenes, allow users to spawn portals to alternate worlds [152]. “The Historical Figures AR” application exemplifies this, enabling users to walk through portals to historically themed sites [153]. Other approaches augment physical places with digital information, supporting alternative interactions through the manipulation of physical objects [154].

5.2. X-Reality Applications in Vocational Training and Education

Vocational training and education are challenging research topics due to the fact that the nature of the subjects to be taught integrates several aspects of human perception and is closely bound to human sense and skillful interaction with tools and materials. This is highlighted through mapping sequences and networks of physical and cognitive activities and working materials in design and workmanship, involving stages of perception, problem understanding, thinking, acting, planning, executing plans, and reflecting upon collected experiences [155]. In process training, part of thinking and planning is implemented by the mind, using mental simulation that produces mental imagery [156]. This modeling approach is also found in cognitive robotics (e.g., [157,158]). In [159], crafting processes are modeled to have schemas or plans, and their execution is modeled as individual events. Studies on the negotiation between the maker and the material have provided interesting data regarding how makers think between things, people, space, and time and develop their practices accordingly [160].

Due to their nature, in vocational education and training, X-reality (XR) applications have emerged as transformative tools, redefining the way individuals acquire and apply skills, based on the fact that these technologies can mimic, enhance, or alter the physical environment, seamlessly integrating physical and digital realms [161,162]. At the same time, these technologies can engage with the complex cognitive aspects described below by being capable of simulating reality and integrating cognitive cues in process training.

VR in training provides the capability of immersing learners in simulated environments that are digital twins of real-world environments, offering training scenarios in a controlled, risk-free setting [163,164]. Vocational training programs using VR have been employed from pilot training to surgical procedures. The ability to practice and refine skills in VR enhances muscle memory and improves confidence. Examples of such training environments include woodworking and blacksmithing simulators [165,166,167].

AR in vocational education has the form of overlaying digital information onto the physical environment, offering learners real-time, context-sensitive aid. From hands-on equipment maintenance simulations to interactive instructional overlays, AR facilitates learning by doing, enriching the educational experience. By extending reality through informative digital overlays, AR provides a bridge between theoretical knowledge and practical application, unifying the physical and digital learning domains. For example, a mobile traditional craft presentation system using AR technology has been proposed to superimpose 3D objects of traditional crafts on the real room space reflected by the camera of the mobile terminal [168,169,170].

MR blends virtual and physical elements, allowing learners to interact with physical and real-world objects simultaneously [171]. This is beneficial in vocational education where practical, hands-on experience is important [172] MR can be employed to integrate virtual equipment into physical training spaces, providing a hybrid learning experience [173]. The learner interacts with digital elements as an extension of their physical surroundings, bridging the gap between the two realities.

The added value of XR applications for vocational education lies in their ability to create immersive, human-centered learning experiences. By simulating authentic workplace scenarios, XR technologies engage learners on a deeper level, promoting active participation and knowledge retention [174]. XR applications also introduce adaptive learning environments, tailoring experiences to individual learner needs. Machine learning algorithms analyze user interactions and performance, allowing the system to dynamically adjust the difficulty and content of simulations [175]. This personalized approach ensures that learners progress at their own pace, addressing diverse skill levels and learning styles while seamlessly blending physical and digital realities.

Assessment in vocational education extends beyond traditional exams to immersive, performance-based evaluations. XR applications, by extending reality, enable instructors to assess not just theoretical knowledge but also the application of skills in realistic scenarios. Real-time feedback mechanisms enhance the learning loop, providing constructive insights to learners and facilitating continuous improvement within the extended reality of vocational training [32].

6. Human Motion and 3D Digitization for Enhanced Interactive Digital Experiences

Human motion capture and 3D digitization are reshaping how users interact with and perceive digital content in digital environments. More specifically, advancements in human motion capture technologies have altered the way digital systems interpret and respond to user movements [35,36]. High-fidelity sensors, wearable devices (e.g., [176,177]), computer vision, and AI enable the precise tracking of body movements, transforming them to explicit or implicit input (e.g., [178,179,180,181,182]). This has wide applicability for fields such as gaming, VR, and AR, where users can navigate, control, and manipulate digital environments through natural movements.

At the same time, sophisticated 3D digitization techniques make possible the seamless transition between real and digital world objects and spaces. From 3D scanning technologies that capture intricate details of physical objects [183,184,185] to depth-sensing cameras that create digital representations of physical spaces, the process of digitization extends beyond traditional boundaries [186,187,188]. This capability lays the foundation for more immersive and realistic digital experiences. In VR environments, for example, users can not only see but also physically interact with digital objects by leveraging natural hand gestures, body movements, and haptic interfaces [189,190,191]. This enhanced level of interaction fosters a deeper sense of presence and engagement, making digital experiences more lifelike and compelling [192,193].

Additionally, human motion and 3D digitization contribute to the emergence of novel interaction metaphors. Gesture-based interfaces, where a wave of the hand or a nod of the head translates into meaningful digital commands, exemplify the shift towards more intuitive interactions. This departure from conventional input methods introduces a new language of interaction, bridging the gap between the physical and digital worlds and giving rise to the concept of virtual embodiment [194,195]. Users can now project themselves into digital avatars or representations that mimic their real-world movements [196]. This not only adds a layer of personalization to digital interactions but also enables a more immersive and empathetic form of virtual presence [197].

The discussed impact may affect various industries such as healthcare, for instance, where surgeons can practice complex procedures in a VR setting that replicates real-world conditions [198,199]. In education, students can engage with historical artifacts through detailed 3D models. In the entertainment industry, these technologies can be used to create interactive storytelling experiences, blurring the lines between the observer and the observed [200,201].

7. Serious Game Design and Development

Serious games are transforming learning into an immersive and interactive experience [202]. Virtual scenarios, historical recreations, and problem-solving challenges become dynamic lessons, allowing students to explore, experiment, and learn through experience. The introduction of adaptability into these games caters to diverse learning styles, making education more accessible and engaging [203]. For occupational training, serious games offers a dynamic platform for skill development and scenario-based learning [204]. Simulations designed for various industries, such as healthcare, aviation, and emergency response, provide trainees with realistic environments to test their skills [205,206,207,208,209]. The interactive nature of these games fosters hands-on experience, allowing individuals to practice and refine their abilities in a controlled, risk-free setting.

Serious games extend their influence beyond traditional education and training contexts, addressing broader societal challenges. Games designed for public awareness campaigns, health promotion, and social issues provide a unique avenue for communication [210,211]. These games leverage storytelling, empathy-building narratives, and decision-making scenarios to raise awareness and prompt action on critical societal topics such as environmental conservation, public health, and social justice.

The design and development of serious games often involve collaboration across disciplines, educational psychologists, game designers, subject matter experts, and technologists collaborate to create holistic learning experiences. This interdisciplinary approach ensures that serious games not only convey information effectively but also align with pedagogical principles, maximizing their educational impact.

The landscape of serious game design has been significantly shaped by technological advancements. VR, AR, and advancements in graphics rendering have elevated the level of immersion and realism in these games [212,213,214]. This not only enhances the overall gaming experience but also contributes to the effectiveness of the learning and training outcomes. The integration of adaptive learning algorithms and analytics further personalizes the experience, tailoring content to individual needs and tracking progress.

These games, often delivered through digital platforms, have the potential to reach a global audience, making education and training more accessible across geographical boundaries. This democratization of learning resources addresses disparities in educational opportunities and ensures that individuals, regardless of their location, can benefit from engaging and purposeful learning experiences.

8. AI Approaches in User Interfaces, Information Processing, and Information Visualization

In the rapidly evolving landscape of technology, the integration of AI is manifested through intelligent systems that, in the future, will be able to understand user intent and enhance the extraction and presentation of valuable insights from vast datasets. AI-driven user interfaces have redefined the way individuals interact with digital systems. Language model-based AI and NLP technologies enable interfaces to comprehend and respond to user inputs in a more human-like fashion [215]. Chatbots and virtual assistants leverage these advancements, providing users with intuitive and conversational interactions. Intelligent user interfaces in the AI era promise personalized user experiences based on historical interactions, preferences, and context [216].

AI has revolutionized information processing, especially in the context of handling vast and complex datasets. Machine learning algorithms, ranging from supervised to unsupervised models, facilitate data classification, pattern recognition, and predictive modeling. This capability enhances the efficiency and accuracy of information processing, allowing systems to uncover hidden patterns, trends, and correlations that may elude traditional analytical approaches. AI-driven data analytics also contribute to real-time decision-making, providing valuable insights for businesses and organizations.

Advancements in AI have led to more sophisticated approaches in knowledge representation and semantic processing. Ontologies and semantic graphs enable systems to organize information in a manner that aligns with human understanding. This not only enhances the retrieval and interpretation of data but also supports more intelligent reasoning and inferencing. AI-driven semantic processing can contribute to a more nuanced understanding of context, facilitating more meaningful interactions between users and information systems [217,218,219,220].

9. Discussion

In the domain of user interfaces, spanning from explicit to implicit interactions, a paradigm shift towards more seamless and intuitive engagements is evident in the literature. This shift, marked by the emergence of gestural interfaces and conversational agents, highlights the growing emphasis on enhancing user experience. Here the practicality of this shift is not yet evident since, currently, the majority of our interactions with the digital world are still on the explicit action-based paradigm.

The more crucial impact of human-centered principles is witnessed in information systems’ core functionalities, since intelligent, user-centric information systems require advanced knowledge representation, data visualization, data mining, and big data analytics. Evidence of this evolution is witnessed, but the need to support the adaptability of these systems to diverse user needs underscores the importance of a flexible and context-aware approach in the processing and presentation of information.

Semantic knowledge representation emerges as a crucial element, offering a means to enhance user interaction. The current state of the art is far from supporting this vision, mainly due to the fact that semantic technologies have not yet sufficiently targeted user interaction. Understanding context, content, and user semantic knowledge may empower systems to provide more meaningful and personalized interactions, aligning closely with the evolving expectations of users.

X-reality has proven its worth in offering simulated environments for various application domains, among which cultural heritage and occupational training were discussed in this work. At the same time, through the integration of human motion and 3D digitization, parts of the immersion contributing to creating a sense of presence in virtual environments are beginning to be complemented. Parallel to the above-mentioned advances, serious games benefit all the above-mentioned progress, providing novel experiences that address educational goals. Of course, all this still remains in a state of flux, and real-life integration in various contexts has not yet achieved an increased degree of integration.

Of course, all of the above are going to be affected by the AI revolution, which currently has only sporadically contributed to advancements in various application domains. Considered among these directly exploitable are technologies integrating text models, computer vision technologies for motion capture, gesture recognition, feature extraction, 3D reconstruction, and view synthesis, etc.

Each of the domains discussed is about to face growth in the future. In this route, challenges emerge both for their technical development and for maintaining an ethical and human-centered approach to innovation. We conclude with a presentation of future directions and challenges in the next and final section of this work.

10. Challenges, Future Directions, and Conclusions

The current landscape on immersive technologies, knowledge representation, and AI for human-centered digital experiences constantly integrates innovations towards an even more transformative future, but this also poses immense challenges. These challenges demand a collective and forward-thinking approach and a commitment to ethical practices, inclusive design, education for technological literacy, and the establishment of frameworks that balance innovation with responsibility.

The integration of technologies, starting from XR to AI-driven interfaces, requires seamlessly integrating these diverse elements to create harmonized ecosystems that support holistic, user-centric experiences. Interdisciplinary collaboration becomes important to bridge gaps and ensure a unified approach to technology integration. The integration of AI in immersive technologies brings forth ethical considerations such as user privacy, data security, and algorithmic biases. Maintaining a balance between innovation and ethical responsibility is important. Safeguarding user data, addressing biases in algorithms, and ensuring transparent and accountable practices are crucial to building trust in these evolving technologies. Furthermore, AI-driven systems bring forth the challenge of building and maintaining user trust. The “black box” nature of complex algorithms necessitates efforts to enhance explainability, ensuring that users can understand and trust the decisions made by intelligent systems. Ethical AI practices that prioritize transparency and user understanding are pivotal to overcoming this challenge. The environmental impact of AI should be also considered. Striking a balance between innovation and sustainability entails the development of eco-friendly technologies, optimizing energy usage, and adopting practices that minimize the carbon footprint of emerging interaction technologies.

Another challenge falls into the domain of accessibility and inclusivity and requires designing interfaces and systems that cater to diverse user needs, including those with disabilities. This entails a commitment to universal design principles, making technology accessible to all, and mitigating potential disparities in digital access.

At the same time, education and training requires effort to improve technological literacy and ensure that individuals across demographics have the skills to use new technologies. This involves developing comprehensive educational programs and fostering a culture of lifelong learning.

Interoperability and standardization are also a challenge, since creating frameworks that facilitate seamless communication between different technologies, ensuring compatibility, and establishing industry-wide standards are vital.

Finally, regulatory frameworks are required to develop policies that foster innovation and safeguard them against misuse.

Looking towards the future, it can be foreseen that human-centric AI integration will provide the possibility to develop AI systems that will not only understand user behavior but also proactively enhance user experiences based on context and preferences. The evolution of interfaces will move towards immersive and multisensory experiences. Virtual and augmented reality, combined with haptic feedback and other sensory inputs, will redefine how users interact with digital content. Interfaces will increasingly leverage AI to offer intelligent personalization, adapting in real-time to user preferences, behavior, and evolving needs. This anticipatory design approach will enhance user satisfaction and engagement. In this respect, these advances will make it possible for future interfaces to prioritize enhanced accessibility solutions, ensuring that users of all abilities can seamlessly engage with digital platforms. This involves not only meeting existing accessibility standards but also pushing the boundaries of inclusivity based on a heightened commitment to ethical design principles and responsible AI practices. Transparency, user control over data, and strategies to mitigate biases will become integral components of interface development.

This interdisciplinary interface design should be emphasized in the future, requiring collaboration between designers, psychologists, technical experts, and experts on ethics. Cross-disciplinary approaches have the potential to enrich the design process, ensuring that interfaces are not only technologically advanced but also psychologically and ethically sound. Interfaces will increasingly adopt natural and intuitive interaction paradigms, using voice recognition, gesture control, and gaze tracking. This shift aims to reduce cognitive load and enhance user experience. A greater emphasis on real-time user feedback and iterative design methodologies will be a prerequisite combined with continuous user testing and feedback loops.

This work aimed to review the rapidly evolving nature of immersive technologies, knowledge representation, and AI for user-centered digital experiences. To this end, this work, starting from the latest developments, identified research gaps and provided insights for future challenges. The overall goal is to support a comprehensive roadmap for future research by summarizing significant advances in seven application domains that are considered by the authors closely bound to novel user-centered digital experiences.

The analysis of these advancements highlighted the current state of the art and assisted in the identification of research gaps that are expected to drive new developments in the future. Of course, the nature of these application domains poses new challenges by themselves. The integration of AI-based approaches opens an entirely new round of possibilities and another discussion on important considerations that include ethical and responsible AI practices.