Immersive Digital Twin under ISO 23247 Applied to Flexible Manufacturing Processes

Abstract

Digital twin (DT) technology provides a path for implementing cyber–physical systems (CPS) and developing smart manufacturing because they are essential tools for monitoring and controlling manufacturing processes. It is considered a vital technology in smart manufacturing and is being widely researched in academia and industry. Furthermore, the combination of DTs and immersive environments has shown great potential for integrating novel capabilities into the new generation of CPS. This research presents an architecture for implementing immersive digital twins under ISO 23247 in flexible manufacturing processes. The proposed system is based on the integration of DT technologies in conjunction with augmented reality (AR) and gesture tracking, and validation was performed in the sorting station of the MPS 500 to increase the interaction and flexibility between physical and virtual environments in real time, thus enhancing the capabilities of the DT. The methodology used for the design and implementation of the DT includes (1) general principles and requirements; (2) models with functional views based on domains and entities; (3) attributes of the observable manufacturing elements; and (4) protocols for the exchange of information between entities. The results show that the integration of these technologies improves the monitoring, control, and simulation capabilities of processes using 3D resources and immersive environments, achieving a higher level of interactivity. In addition, error detection tests were carried out, where a reduction of time was observed in the resolution of errors that may be caused by internal or external disturbances of the process, thus avoiding production delays.

1. Introduction

Digital transformation has been driven by various technologies, such as cyber–physical systems, big data, Internet of Things (IoT), Industrial Internet of Things (IIoT), mobile devices, cloud computing, robotics, artificial intelligence (AI), blockchain, mixed reality, DTs, and collaborative and social tools [1]. Smart manufacturing, also known as Industry 4.0, aims to enhance productivity, quality, and efficiency in production by integrating technologies to automate and optimize various tasks, including supply chain management, quality control, and production planning [2]. These emerging digital technologies, such as industrial interconnection and DTs, have changed methods, production models, industrial organizations, and international patterns, and the digital economy is transforming and enhancing the manufacturing industry [3]. Industry 4.0 has revolutionized manufacturing systems through several technologies that have proven to be highly valuable. The main ones are virtualization through CPS, IoT, and DTs; these technologies combine data collection and analysis to facilitate interaction between physical and virtual spaces with bidirectional communication [4,5], enabling the connection and visualization of all elements. Moreover, they provide information concisely and seamlessly to facilitate intelligent decision-making, thus accelerating industrial modernization. A cyber–physical system combines computational, networking, and physical elements to control and monitor the physical environment in real time [6]. Digital manufacturing and CPS have contributed to enhancing DTs through real-time production and operation data [7]. The main reasons for using these technologies in the industry are analysis, diagnosis, and predictions about future performances in the processes [8].

DTs are 2D or 3D computer-based digital replicas that simulate and reflect a physical entity’s life to extract data from the real world and process information in digital models to monitor, diagnose, optimize, predict, and support real-time decision-making [9,10]. They are considered a fundamental technology for smart manufacturing, so they have been extensively researched in academic and industrial domains. Several authors have proposed different methodologies and architectures for their implementation in the industry. However, only some studies show a standardized approach for developing DTs in industrial applications [11]. One of the most significant challenges is to create a well-defined 3D digital structure to connect them with the actual model. This indicates that some barriers in the industry must be overcome so that all the benefits of DTs can be fully leveraged in an industrial context [12]. Another challenge is to create reliable models with acceptable computational costs, low latency, and analysis using artificial intelligence techniques. However, the development has gained strength in recent years with the advancement of Information and Communication Technologies (ICTs), as sensors and technological infrastructure have become more cost-effective and reliable [13]. Several studies have shown that the implementation of DTs in industries provides the opportunity for seamless information transfer, process replication, diagnosis, increased productivity, the development of intelligent systems, and predictive analysis [14]. As the integration between the real and the cyber world progresses, virtual interfaces for visualization between the operator and the virtual environment must evolve in parallel. For this reason, immersion technologies, such as virtual and augmented reality, allow for the immersive and interactive visualization of DTs [4]. The integration between DTs and AR in the industry provides valuable information about the design and function of a product, in addition to optimizing processes and reducing operator training costs [15].

The combination of AR and DTs has sparked growing research interest because it has demonstrated its potential to integrate operators into the new generation of the Human Cyber Physical System (HCPS) [16]. It is a technology that combines the real and virtual worlds, interacting in real time to create an immersive and interactive experience in two or three dimensions. This technique allows digital content to be overlaid to the real world, typically through optical displays or videos that can be viewed through glasses or mobile devices such as smartphones and tablets [9]. For the industry, AR promises to change and improve the way operators interact with production processes, enabling access to information through virtual overlays. It can be applied in assembly, maintenance, and monitoring tasks, providing greater flexibility for data access. Industrial augmented reality (IAR) is a crucial technology for Industry 4.0, as it can enhance workers’ performance, reduce risks, and improve production processes [17]. IAR refers to integrating digital information with the real world using computer-generated sensory information such as video, sound, graphics, or GPS data. Combining these technologies promises to change how an operator interacts with production processes, enabling access and interaction with real-time information [18,19].

This proposal aims to implement an architecture for an immersive DT under the ISO 23247 standard in an industrial environment, highlighting the capabilities of including immersive technologies for monitoring, controlling, and simulating industrial processes in real time. It is hypothesized that the implementation of this technology will lead to the integration of immersive technologies in a DT, enhancing the interaction between physical and virtual entities using AR and Senso Glove technologies and improving operational efficiency and decision-making in industrial environments.

The contributions of this research are

• The implementation of a DT under ISO 23247 supported in augmented reality for monitoring, simulating, and controlling an industrial process using low-cost devices with cloud communication under the Message Queuing Telemetry Transport (MQTT) protocol.
• The development of an immersive reality method to manipulate variables using Senso Glove and an augmented reality interface for interactive visualization on the immersive DT platform.
• The analysis of immersive DT results through experimental tests, bidirectional communication, error generation, and measurement of response times.

2. Related Works

To carry out the state-of-the-art analysis, a review of the works undertaken in the industry in the last 5 years was carried out. This research reviews renowned databases (Elsevier, Science Direct, Web of Science, IEEEXplore, Springer, and Scopus) to find studies on DTs and mixed reality in the industrial field. The purpose was to explore various applications and architectures by analyzing proposals made by other authors and the enabling technologies for their development. The identification of articles began with extensive queries of academic literature databases for a series of thematically essential keywords. For example, the following keywords were used, typically in sets of two and groups of three or more: [Digital Twin, Augmented Reality], [Digital Twin, ISO 23247], [Digital Twin, Senso Glove], [Digital Twin, Immersive Environments], and [Digital Twin, Industry 4.0, Augmented Reality]. Furthermore, for the selection of the articles, it was sought that the methodologies proposed by the authors had been implemented at an industrial level and not only simulated, to guarantee the possible application. Works were sought that used robust industrial protocols and protocols that could be implemented in low-cost embedded systems. Regarding the programming language, the search was carried out in a general way because, in research, different languages can be used according to the level found according to the automation pyramid. Finally, DT papers implemented with mixed realities (virtual reality and augmented reality) were searched to learn about the characteristics and advantages they provide when combining these technologies. After the review of articles that met the characteristics, 15 articles were obtained, only 1 of which was based on the ISO 23247 standard, which shows that the proposals presented on the topic are quite open.

One of the areas where DTs are used is in maintenance, where several investigations have been carried out. In [20,21,22], they present a DT to perform the predictive maintenance of a wind turbine and reliability in floating offshore wind turbines (FOWTs). The architecture presents a distributed DT with IIoT, machine learning (ML), MQTT, and fog computing to offer the better utilization and management of assets through real-time condition monitoring, predictive analysis, and the health management of selected wind turbine components in a wind farm. The architecture presents three layers: (1) the cloud layer, where there is deep learning and cloud computing; (2) the fog layer, where there are fog nodes for the DT interfaces, APIs, and SDK; and (3) the IoT device layer, where turbine sensors and actuators are located. The results show that the DT offers a good solution; however, it is an emerging technology that could be improved with better standards, architectural frameworks, and implementation methodologies. In turn, this shows that the integration of technology in FOWTs provides valuable information in the DT for reliability and performance optimization. Furthermore, it shows the potential of DTs as a valuable tool for renewable energy, boosting efficiency and sustainability in floating offshore wind installations. In [23,24], they propose a DT-based method for fault diagnosis and prediction, aiming to compensate for the shortcomings of existing methods: the single fault type, low similarity, and poor visual effect of state monitoring. For this, a complete model is built that represents all the elements, processes, and businesses of the physical workshop using a testing platform to acquire real and simulated signals under various operating conditions. For prediction, they use the “digital twin network + short-term memory (LSTM)” algorithm. The results showed that the proposed method has a high rate of accuracy in the prediction for the production line because integrating the function of feedback allows for an accurate perception of the current state and future changes.

Research with applications that are or can be applied in the industrial area was also reviewed, such as [25], which proposes the integration of knowledge graphs (KGs) and DTs to develop highly autonomous and flexible DTs that use semantic knowledge stored in the KG to support functionalities. The presented proposal was implemented in a real laser glass-bending process and comprises smaller virtual objects (VOs) that interact as a single-step abstraction of the process, providing functionalities that simulate the corresponding real-world process. By executing appropriate queries to the KG, the DT can orchestrate the functioning of the VOs and their physical counterparts and configure their parameters accordingly, thereby increasing their self-awareness. The results showed a highly flexible system and dynamic configuration that allows for the implementation of several algorithms and the storing of information about the materials and industrial business processes in which they participate. In [26,27], they use a DT for the control and optimization of a multivariable thermoelectric system and a multivariable valve. The proposed methodology consists of the following stages: system definition, parameter identification, data-driven multiphysics simulation, behavior matching, and implementation. The system is composed of industrial elements and embedded systems for control and data collection, and later, statistical techniques are used to estimate parameters. The results obtained indicated that the use of DTs increases the accuracy of the system representation, allows for monitoring the state of the process, and provides information that can be used for decision-making at different levels of the process. In [28,29], they present the design of a high-fidelity DT for an articulated robot and an architecture to reinforce the sustainability of logistics processes in Industry 4.0; they rely on Matlab 2023 software, the Simscape Multibody tool, and industrial devices. A PLC, Open Platform Communications United Architecture (OPC UA) communication, and data transmission through KEPServerEX V6 software were used for further processing. The objective was to present a comprehensive DT methodology that allows for design, monitoring, and optimization. The results at the simulation level showed that this model can be used reliably in simulations where there is high fidelity in the design, control, and data generation for approaches based on learning and the optimization of energy consumption. In [30], they propose a DT architecture for an automotive infotainment system. The proposal shows the development of a solution to manage hardware processing, monitoring in the virtual environment, and system control. The design of the DT was based on the ISO 23247 standard, and an individual interface was implemented for each component with radiofrequency (RF) technology and a Raspberry Pi 4 card with Python programming language. The results showed that the DT concept allows for the implementation of monitoring, control, and self-diagnosis functions of a system.

Within the development of DTs, there is research on the use of technologies with AR and VR to expand the capabilities of DTs, as shown below. In [31,32], a DT is presented for the control and monitoring of industrial processes, supported by AR and VR. Robust devices (field level and control) and low-cost embedded systems were used to create and communicate the virtual environment (monitoring and control). For this, the OPC UA, MQTT, Ethernet, Profinet, Modbus, machine-to-machine (M2M), and node RED protocols were used for communication. The results demonstrate bidirectional communication between the physical and virtual environments, allowing for visualization through AR or VR and facilitating interaction with the virtual environment. This allows for timely and dynamic adjustments to be made in real time to improve the efficiency of industrial processes. In [33], they propose a real-time building information modeling (BIM) data compatibility verification method based on AR and DTs. The verification module was designed based on prototype tests and functional improvement in the engine room and on the FAB site, which consisted of the application of anchors based on markers and the optimization of the FAB site. The AR adjustment and fixation module were developed, and visual SLAM was based on Unity along with AR Foundation using SLAM, 2D tracking, and local anchoring API in ARCore. The results showed that the proposal allows for the precise adjustment of the AR model and the real object through AR tracking and anchoring technology considering the characteristics of the FAB sites, maintaining compatibility even when the camera and the marker were moved. In [4], they present the experimental implementation of the integration of a DT and AR, to develop a human–machine interface for the remote monitoring of industrial cyber–physical systems. For the process, Factory I/O version 2.9 simulation software was used, the TIA platform was selected for PLC programming, and for the 3D design, Unity version 2018.4.36f1 and Blender V3.2 software were used to model and animate the virtual models of the industrial process. Finally, the results were analyzed, and it was observed that the system offers more flexibility to access the information generated through holographic projections on any surface using a QR code, allowing for new opportunities to visualize the monitoring states.

The reviewed literature shows several trends. It is observed that most of the works presented by other authors do not follow a standard for DT architectures; however, as it is a developing technology, companies and universities continue to present proposals without them being based on technology or fixed protocols. It is observed that the most used protocols for communication are OPC UA, TCP/IP, AMQP, MODBUS, and MQTT for their lightweight and easy setup; they also allow for bidirectional communication and can be used at different levels of the automation pyramid, allowing for total integration in the processes. On the other hand, several trends and enabling technologies are evident for the development of this technology and the interoperability that can be achieved between different platforms to design DTs.

3. Materials and Methods

In this work, a DT is carried out under the ISO 23247 standard for the control and monitoring of processes using AR and gesture recognition in an industrial environment.

The ISO 23247 standard is a standard to support the creation of DTs; however, it depends on the standards and technologies available to model the manufacturing elements—that is, the standard does not prescribe specific data formats or communication protocols [34]. It is composed of a four-layer framework, as shown in Figure 1, which describes the observable manufacturing elements, a digital entity that allows for modeling and updating the DT, a device communication entity, and user entities.

From the review of the state-of-the-art research presented in the previous section, it is evident that the methodologies proposed for the implementation of DTs are not based on an established standard. However, this provides a background and current basis regarding the use of technologies and protocols for bidirectional communication in industrial environments and modeling and simulation tools for immersive environments. It can be highlighted that there is no closed concept or standard for the development of DTs; this is because it is a tool that is under development, and it is a central point of research in academia and industry due to the advantages it presents.

Figure 2 shows the general diagram of the methodology followed for the implementation of the immersive DT based on the ISO 23247 standard, which shows the steps followed. This includes the physical process, communication protocols, ISO standards, design, general characteristics at each stage, and the analysis of results. It should be considered that the standard used does not restrict the use of technologies and protocols used; however, it has clear parameters that the physical and virtual entities must comply with and have, as well as the communication between them at the industrial level.

The design of the architecture for the implementation of the DT was based on the ISO 23247 standard, which establishes the main parameters for the architectures of a DT. Figure 3 shows the parameters that were considered in the physical, virtual, and communication entities for this proposal. As a standard that does not limit the use of devices and protocols, the following proposal is made. The creation of the virtual environment with the help of Blender V 3.3 software based on the ISO 16792 standard, which complements the existing standards and focuses on 2D and 3D modeling in digital format, the communication will be performed using the MQTT protocol, and the data will be sent to the cloud for subsequent integration and interaction with AR and the Senso Glove between the physical and virtual entities. Finally, the proposal will be implemented in the MPS 500 station through an embedded system for the subsequent analysis of results through experimentation and operation tests under different process configurations. Improved figure

The physical model is the foundation of the DT. This industrial process incorporates sensors, actuators, and a controller (embedded systems) to gather and transmit all the information that is later stored and processed in the cloud through IoT. This section presents the components that were considered for each stage. To design the model of the physical entity, the following component attributes must be considered: Component name, Component type, Component material, and Component dimensions, so the information attributed to the necessary physical components for creating the digital environment can be observed.

The digital model is developed using Blender software. The design considers a scale with precise measurements and dimensions for each component, and finally, it is exported with the .fbx extension. This includes the details of the physical entity, which are later used for creating animations and AR executions through the Unity game engine. The design includes external disturbance identification, monitoring, control, and process simulation models. All the 3D design features can be complemented with product manufacturing information (PMI) according to the ISO 16792 standard [35]. Several components are considered for the design of the virtual model, such as the dimensions of the geometric model, the surface of the physical model, equipment material, equipment parameters, and operating rules.

The digital system requires a constant flow of information. This information is obtained through data input during the process simulation and automatically collected from the physical entity through sensors. The data are sent to and stored in the cloud, providing information about the process and the management of physical and virtual models, allowing for the analysis and control of the process based on these data. The interaction occurs through bidirectional communication interfaces using the MQTT protocol. For the AR interface, the virtual environment is exported to the Unity platform to be subsequently visualized, and control parameters are established using the Meta 2 AR Glasses device. Data transmission can be wired or wireless, depending on the type and importance of the data. These dynamic data collected from the physical model are used to modify the parameters of the virtual process model and are as follows: device identifier from which the data are collected, variable assigned to the device, communication time for data transmission, and values of the device parameters.

Hand gesture applications have increased as technology advances, allowing for more natural ways to interact with computers. The data gloves used in this research are the Senso Glove: DK2 from Senso Device Inc1 [36]. The Senso Glove is a tool that can interpret hand and finger movements through sensors of inertial measurement units (IMUs) and wireless communication. It features eight inertial measurement units and two magnetometers in the wrist and palm, allowing it to accurately detect hand and finger movements without needing a camera. The eight sensors precisely measure the orientation of each part of the hand through a combination of accelerometers and gyroscopes [37]. The Senso Glove is used as a control tool for the system, as hand movements become the input for the system. Thanks to wireless communication, users can move their hands freely; additionally, the glove provides tactile feedback through vibration, enhancing user interaction [38]. It can read hand and finger movements and has been used in various applications. Therefore, in this research, it will be used for the immersive control of the process to analyze the interaction that this technology can provide in conjunction with the DT.

4. Implementation

The architecture used for the design and implementation of the DT includes (1) a physical model: a MPS-500 classification station; (2) a digital model supported by AR; (3) an immersive environment with Senso Glove; and (4) IoT. The proposed methodology is for a DT for monitoring and controlling an industrial process, where the use of AR is proposed to establish an environment that allows for adding additional information to the physical environment, thus improving human–machine interaction and allowing for simulations of the process in real time based on the data obtained. The Meta 2 glasses are used due to the advantages they have, and the sensors are attached to this device for interaction. In addition, control using gestures is proposed to increase the interaction of the physical and virtual environment where Senso Gloves are used due to the versatility and number of movements that can be had for better interaction with the processes. Finally, an embedded system is used to send the process data to the cloud and subsequently to each of the devices. In addition, the MQTT communication protocol is used, which allows for bidirectional communication. The integration of these technologies aims to enhance an operator’s monitoring and control capabilities by using 3D resources in a graphical interface and to increase interactivity through a controller that detects the movements of the user’s hand and fingers.

The MPS-500 classification station from the production plant was used as the physical entity to implement the DT supported by AR and immersive environments. This station was selected because it has industrial sensors and actuators, and it also allows various types of processes to be carried out with different conditions, which helps to verify the operation of the proposed DT. For the research, control (AR glasses and gloves) and monitoring (AR glasses 3D environment) will be carried out, and the parameters that can be configured are a start, a Cartesian arm movement, a conveyor belt, etc. The 3D digital design was developed using Blender software, and the process data were sent to the cloud via the Photon card using the MQTT protocol. These data were stored and processed to later interact with Meta glasses and the Senso Glove.

The Festo MPS-500 plant sorting station consists of the three stages described below.

Stage 1—Main conveyor belt where the final products arrive to be classified.

Stage 2—The manipulation station is equipped with a flexible two-axis manipulator (“x”, “y”), which has sensors (optical, magnetic proximity, reflective) and actuators (parallel grippers, flat cylinder, linear actuator horizontal) that are used for intermediate transportation from Stage 1 to the classification stage.

Stage 3—Final classification, which consists of three ramps to classify the pieces by color or material and a ramp to discard. In addition, it has sensors (retroreflection, direct reflection, inductive, and presence) and actuators (short stroke cylinder, compact cylinder, DC motor, and diverters).

For implementation, the main steps for the design and communication of physical and digital entities in conjunction with immersive environments are detailed below.

4.1. Senso Glove Communication

The communication between the Senso Gloves and the computer is established through a radiofrequency (RF) antenna that allows for configuring the working frequency. For this application, a frequency of 433 MHz was used, capturing the values from all the built-in IMU sensors in the gloves to obtain the hand position, palm space, and movement of all the joints. The software responsible for reading and digitizing the signal from the glove is implemented using Senso DK3 version 1.17. To ensure proper functioning, it is necessary to configure and calibrate the gloves, as shown in the UML diagram in Figure 4.

The Senso_Base_Controller establishes wireless communication to obtain sensor data. The hand allows for access and interaction with the data received from the glove. The Senso_Hand provides knowledge of the current state of each sensor—in this case, the respective movement of the coordinate axes (X, Y, Z), which is processed to guarantee the movement of the digital hand. Additionally, communication with the Firebase cloud is carried out for sending and receiving data, enabling interaction with the industrial process.

The calibration of Senso Glove must be carried out accurately and considering the physical space that is going to be worked to correctly manipulate the physical environment based on the movements generated, and these can be replicated in the virtual environment in real time. To calibrate the glove, the “SensoHandSDK-unitypackage” SDK must be imported. These libraries are selected according to the version of Unity. In this study, version “2021.3.19f1” was used. These libraries allow for obtaining the scene in Unity where the work environment is later created. In this process, the physical hand space is modified and adjusted in a virtual representation. To achieve this, it is necessary to modify the Quaternion commands typically used to adjust rotations and movement in the working scene, allowing for a smooth interpolation, avoiding gimbal lock, and primarily relying on complex numbers to represent Euler angles. The configuration and calibration are carried out using Senso DK3 easy configurator, version 1.17. The package must be imported into Unity to incorporate and calibrate the DT. It is essential to consider the encapsulation of each element, so first, it is completely unpacked. Then, each link is assigned to another to generate smooth movement in its physical twin. Subsequently, the DT is calibrated using the Quaternion command based on Euler angles. The analog readings of the variables obtained from the particle cloud, which are based on industrial sensors and previously conditioned in the control board, are used as parameters. Hand gestures are detected in the created digital environment using the Senso Glove to manipulate process parameters. The objective is to provide an interactive environment for users and enhance their experience. An analysis of the positions is performed to configure the working environment within the area created by Unity, aiming to work in conjunction with AR. It is essential to consider the rotation of all axes, which are the rotation around the front-to-back axis, known as ROLL; the rotation around the side-to-side axis, called PITCH; and the rotation around the vertical axis, known as YAW. Calibration of the Senso Glove is performed in the virtual environment created with the help of Senso SDK, as shown in Figure 5.

4.2. Integration of the Met a2 SDK into Unity

The ability to merge multiple environments in the same software is one of the advantages of Unity. To achieve this, the Meta 2 SDK incorporates the digital visualization of the plant process. It also includes multiple scripts to enable the rotation, expansion, placement, and positioning of 3D elements within the operator’s field of view. The UML diagram in Figure 6 shows the configuration of different environments, movements, and button controls for the Meta glasses.

4.3. Communication with the Particle Cloud

The communication is carried out using the particle photon development board, which has the necessary electronics and conditioning to handle industrial sensors and actuators. Additionally, it offers the advantage of using the particle cloud service. To achieve communication, a JSON format is used to send and receive information. Each photon board has a unique ID used for identification, and a label is added to reference the variable to be manipulated. To close the JSON web communication link, a personal access token provided by the created account is used. Figure 7 shows the board’s configuration for sending and receiving data from the process to the cloud. This is achieved through M2M communication to interact with resources in the physical and virtual environments.

4.4. PC Features

For the implementation, a high-end laptop was used to execute the programs and digital environment. Subsequently, the analysis of the consumption of computational resources was carried out, because during the DT tests with immersive environments, there was a high consumption of resources as well as problems with the PC when running the Meta 2 AR glasses. An MSI laptop was used in this research, the characteristics of which are shown in

Table 1. PC features.

Nº	Features
Operating System	Windows 10 Pro 64-bit
System model	WS76 11UM
BIOS	E17M1IWS.10D
Processor	11th Gen Intel(R) Core (TM)I9-11900H @2.50 GHz
Memory	65,534 MB RAM
Graphics Card	NVIDIA RTX A5000 laptop GPU

.

When running the DT with AR, a high consumption of resources is observed, primarily in processor usage, reaching 62%. Graphics card consumption is also high, reaching 52%. Additionally, the temperature rises to 60 °C at the beginning and continues to increase, causing delays in the visualization of the DT parameters. These values are high despite the capabilities of the PC, which shows that executing the DT with immersive environments requires high processing to produce better performance.

5. Results and Discussion

To analyze the operation of the proposal, several analyses were carried out, as detailed below. Several tests were performed to determine the performance of the proposed architecture, focusing on (1) the interaction of the users with the physical and virtual environments of the immersive DT (AR and Senso Glove) to see the operation and interaction in real time; (2) the generation of internal and external errors during the production line, where the DT generates an immediate alert of the error generated and the location of the error; (3) the measurement of the latency in the process; and (4) energy analysis, which was carried out because a high consumption of computational resources and an increase in latency were observed when using immersive environments with the DT, so we sought to analyze if this parameter affects the control of the process through high consumption or a harmonic distortion in the network. These parameters were considered to evaluate the main conditions established by the standard, and other parameters were also analyzed to determine if the proposed architecture with immersive environments is feasible to implement and does not affect the process, as the ISO-23247 standard is quite flexible in the use of protocols and technologies involved in the architectures. However, it establishes general parameters and requirements for the design of the DT, models with functional views, components and attributes that must be included in the process, and technical requirements for bidirectional communication.

In the first part, the operation of virtual environments is validated, Figure 8 shows the digital environment controlled by the gloves and supported by AR + Senso Glove to create an immersive and user-friendly environment. Figure 8a shows the digital environment controlled with the Senso Glove, where the digital hand can configure the parameters and interact with the digital environments, Figure 8b shows the DT with AR and Senso Glove and the real-time interaction with the physical environment. The parameters can be configured from the AR environment or the physical process, and the changes and information can be observed immediately in the created environments. In Figure 8c, the DT screen shows the warning of a fault in the process, which was created by the operator when removing the part from the Cartesian arm, and it is evident that the DT immediately detects and changes its status and also generates an alert for the fault and its location. Finally, Figure 8d shows the DT only with AR; this is because the system can work independently or in conjunction with the Senso Glove, depending on the needs and characteristics of the process and operator configurations.

To verify the operation of the DT, the manipulation of the Cartesian arm was carried out with the Senso Gloves, where 20 tests were carried out for each position (start, middle, and end), and an average of 95% accuracy was obtained later when using the AR glasses. To calculate the accuracy, the experimental process was carried out several times under the same established conditions; then, an analysis of the measurement system and activation and deactivation tests were performed to later obtain the deviation of the measurements and the average value (percentage) using statistical analysis. Accuracy was evaluated due to the consistency and repeatability that the physical and virtual environments must have to ensure a good operation of the process and prevent possible failures.

The configuration of the process parameters (number of batches, initial conditions, etc.) was carried out using the buttons on the glasses, and 100% precision was obtained in addition to visualizing and simulating the process in real time. The system allows for bidirectional communication between the entities, where after making the configurations either in the physical space (MPS-500 classification module) or virtual space (glasses or AR), the system begins the manufacturing process of products, and their behavior can be observed in a simulated way. Furthermore, in the digital system, simulations can be carried out to observe the behavior of the system or production that is going to be carried out prior to execution.

One of the tests carried out to verify the operation of the DT consisted of generating controlled errors during the production line. The errors consisted of changing the color of the tokens manually, removing the token from the Cartesian arm, and stopping the distribution actuator. Subsequently, 20 tests were carried out for each of the proposed errors, both for the conventional system and with the immersive DT, under the same characteristics, and the average shown in Figure 9 was obtained. It is evident that there is an improvement of 17 s in the average time it takes the user to solve the error and continue with the production line; this is because in the DT, the operator can verify the status of the process with the data he constantly receives, and in addition, the DT generates an alert of where the error occurred and the location, also allowing for the control of some parameters from the AR environment, which helps in improving time. In the conventional process, the operator must visually verify where the error occurred and subsequently provide a solution, which increases production cycle times.

The analysis of the response times of the immersive DT was carried out based on the number of sensors and actuators because it was observed that having a greater number of connected devices resulted in a delay in the execution of the processes; this is due to the amount of data that must be processed and sent to the cloud so that it can later interact with physical and virtual environments. Subsequently, the time analysis was carried out based on the environment of the DT with AR and DT with Senso Glove, respectively, where the activation of several actuators was carried out and the number of actuators was increased to measure the response time. Activation was performed in the virtual environments and was measured based on the activation time in the physical environments. Figure 10 shows the response time for each environment, and it is evident that the DT with AR has a longer time compared to the Senso Glove. Therefore, it must be considered that for the implementation of these systems, they have a high consumption of resources, and despite using a high-end computer, there was a constant increase in time depending on the TAGs used.

As seen in the previous figure, when executing the immersive environments with the DT, the times increase considerably, which causes delays in communication and lag times. However, these times only increase in immersive environments, while the data are displayed in real-time on the PC that runs the DT, which guarantees good performance. Because immersive environments present a delay in display and a high consumption of resources, the energy analysis was carried out with a Fluke analyzer, where it was carried out in analysis under three conditions, which were (1) system in stop, (2) conventional system, and (3) system with immersive DT. These tests were carried out for continuous time schedules, and the data were recorded with the analyzer. Subsequently, Power Log Classic software V4.6 was used, and the image in Figure 11 was obtained. When analyzing the data in Figure 11, it is observed that there is no considerable change in voltage in the three phases; however, there is a distortion in the signals obtained, so an analysis of the harmonic distortion was carried out.

The analysis of the harmonic distortion was carried out from the data obtained in the previous figure based on the three stages analyzed. This analysis was carried out because if there is harmonic voltage distortion, it can introduce errors to the reference signal and generate an unstable response in the control system, affecting the performance and stability of the system. Figure 12 shows the comparison of the harmonics present in the three stages, where it is evident that there is a slight increase when comparing the system operating manually vs. the system operating automatically with the proposed immersive DT. This is because when carrying out the process operations manually, the devices must start again in each cycle, while in the automatic system, with constant control, there are fewer interruptions in the production cycle. Furthermore, it can be verified that the proposed immersive DT does not directly affect the harmonic distortion, but it does affect the resource consumption of the machine where it is being executed.

Studies on the use of DTs in industries and educational centers have significantly increased in number. Their advantages have been demonstrated in various research papers; however, the aspect related to their contribution to production cycles, particularly in product improvement or cost reduction, still needs to be clarified. Within the state of the art, it is observed that most of the works presented by other authors do not follow a standard for the design of DT architectures; however, as a developing technology, several industries and universities have focused their attention on it. For the design of 3D environments, several CAD version 2022 softwares are used: among the main ones are Blender and Solid Work; for communication, the most used protocols are OPC UA, TCP/IP, AMQP, MODBUS, and MQTT; and in the physical environment, the process data are obtained by PLCs or embedded systems [20,29,31,32]. In addition, the integration of several technologies is evidenced to enhance the characteristics of the DT. There are several types of research in which different architectures are proposed for the implementation of DTs in the industry. However, there are still some barriers to be overcome in the industry so that all the benefits of DTs can be fully exploited in an industrial context [12].

In this research, a DT was implemented in IoT and immersive environments, highlighting the need for better integration, especially in terms of the reliability and security of the data stored in the cloud, as these data determine the parameters and characteristics of the DT. A possible solution to improve security, latency, and resource consumption could be the implementation of nodes, also known as fog computing. This way, data would be processed closer to the field level, allowing the decentralization of the process with multiple fog nodes for each stage. One of the main challenges is to keep digital models fully updated, including changes made during the product development process, to achieve convergence between the physical and digital worlds through technologies.

Several studies have developed different DT methodologies in the industry; however, no implementations combine immersive environments (AR + Senso Glove) with bidirectional communication between the physical and virtual environments. These technologies provide a more immersive collaborative experience for the user, enhancing their interaction with digital environments and increasing the benefits for the industry. The survey results showed a lower score for the combined use of gloves and AR. This is because both devices can manipulate process variables but not simultaneously due to conflicts with the TAGs. Additionally, the AR environment allows for overlaying digital information onto the physical environment, simultaneously facilitating interaction and the visualization of both the digital and physical entities. On the other hand, the gloves require precise calibration concerning the physical and virtual entities, considering all the parameters of the system components. With proper calibration and user training, better results can be achieved due to the ability to manipulate multiple analog variables with simultaneous hand and finger movements.

DTs and IoT are essential for developing Industry 4.0 as they enhance the capabilities and features of current processes. IoT and DTs focus on resources, enabling communication between consumers and devices to occur directly or through some form of system. Thanks to machine-to-machine communication and the free libraries available in the low-cost devices used in this research, these resources have been effectively utilized to send processed data simultaneously and securely to Meta glasses and Senso Gloves. However, it is essential to consider that free cloud space is limited, and additional costs may be incurred for larger applications. The proposed system showed a high consumption of computational resources, and by having immersive environments, the time increases considerably depending on the TAGs used. This is why control must be carried out at the field level and only certain parameters can be controlled from these environments. In addition, by having a large amount of sensor data, a decentralized system with fog nodes for each environment can be proposed to reduce the latency generated. It is observed that the integration of immersive technologies increases the capabilities of the DT; however, the high consumption of computational resources must be considered, which increases according to the amount of data that is controlled from these environments and limits the number of TAGs used, since real-time communication is required.

When running the DT with AR, a high consumption of resources is observed, primarily in processor usage, reaching 62%. The graphics card consumption is also high, reaching 52%. Additionally, the temperature rises to 60 °C at the beginning and continues to increase, causing delays in the visualization of the DT parameters. These values are high despite the capabilities of the PC, which shows that executing the DT in immersive environments requires high processing for better performance. Due to the high resource consumption presented, an analysis of the energy and harmonic distortion was carried out, and the results showed that it does not directly affect these parameters, which is a benefit that enables the avoidance of failures in the control loops. However, a slight increase in the odd harmonics is observed when the system is in an open loop concerning the system with the DT, showing that despite the DT’s high consumption of resources, it does not affect the power consumption or increase harmonic distortion. It must be considered that high resource consumption affects viewing times in virtual environments, so high-performance equipment must be available for the implementation of these proposed systems.

The results showed that the system could identify, analyze, and design control system behavior for fault prediction, confirming the advantages of the DT in validating the control system. This shows the potential of using DTs in the industry, and these systems can be applied at all stages of production. From the work carried out, the following advantages presented in the proposals can be highlighted: reduction in downtime, reduction in corrective and predictive maintenance costs, increase in operational efficiency, and creation of safer work environments. This is due to the ability to model and simulate environments with real conditions.

6. Conclusions

ISO 23247 provides an architectural framework for DTs that does not prescribe specific formats, technologies, and protocols, thus allowing for the use of various types of immersive technologies and embedded systems; open communication protocols such as MQTT; and the inclusion of ISO 16792 in conjunction with ISO 23247 for the design of the DT, which allowed for improving the 3D model in greater detail. This union of resources and platforms allowed for a successful implementation, and its validation was performed in a modular lab with industrial components.

The results show that the combination of a DT with immersive technologies allows for greater operator interaction with the process and provides new alternatives for setup, monitoring, and decision-making on the production line. This allows for the remote monitoring of industrial processes through a digital interface with AR, providing greater detail with which to visualize supervision states at different product lifecycle stages. This integration expands the visualization and real-time data access capabilities, reducing production times and efforts in training and maintenance and decreasing decision-making errors. Among the advantages presented by the proposed system is a reduction in the time solving errors that may be caused by internal or external disturbances of the process, thereby avoiding production delays.

Using AR and Senso Gloves to achieve greater immersion and interactivity simplifies the interaction between humans and machines and enhances the development of DTs, expanding their benefits in industries. This is due to the constant data transfer, replication of digital processes, reliability, diagnosis, and analysis they offer. Moreover, using these technologies provides a more immersive collaborative experience for the user, improving interaction with digital environments. Regarding the analysis of transmission times, it depends on several factors such as the bandwidth of the network being worked on, latency and network load, and the amount of data that is sent, because MQTT is efficient for small messages. Furthermore, it is observed in the results that the time increases depending on the number of sensors used; this is because when performing the immersive DT, it has a greater consumption of resources. However, this high resource consumption does not affect power consumption or cause problems in the power grid. Therefore, devices with high performance must be available to prevent time from affecting the operator’s decision-making.