1. Introduction
Industrial production is a complex process that contains a series of activities, where each activity operates according to certain rules that ensure quality and safe operation. Every quality production has to be supported by the honest work of project teams. The solution and spatial optimization of the layout of the production technology in production is an integral part of designing production systems. In the current design, employees have reliable software and high-quality technical equipment that ensure accurate data and outputs. The paper points out one of the possible solutions for the design of production systems. The modelling of the production system design of machines and equipment is processed in CAD software version 2022.3 with the support of internal libraries, which can be used as a possibility for creating models. Of course, the mentioned method and software are not the only variant of solving the given problem. The result of the experiment is a technology that uses mixed reality in the design of production systems.
At the beginning, it is necessary to understand the overall view of how mixed reality works. A so-called virtual continuum displays a continuous range between complete virtual reality and the real environment. This range includes all possible versions between the proportions of real and virtual elements. The mixed reality is located precisely in the area between the two extremes, i.e., the elements. [
1]. Thus, mixed reality actually consists of augmented reality and augmented virtuality according to the parts of virtual elements.
The Industry 4.0, as a challenge of today´s highly competitive environment, supposes that internet technologies will be used in future factories and that they will replace used components by using dynamic and intelligent cyber-physical systems that are the mixture of the physical objects with their digital representation [
2,
3,
4,
5]. A research team led by Quint [
6] in their paper proposed a system construction for a mixed-reality-based learning environment that gives together the physical objects and visualization of their digital content via augmented reality. It allows making the dynamic interrelations between real and digital factory visible and tangible.
Mixed reality involves the integration of virtual computer graphics into real 3D space, or the real-world elements in a virtual environment [
7].
In Coutrix and Nigay’s research [
8], a new model of interaction for mixed-reality systems was introduced. The main benefits of the mixed interaction model are:
the unification of several existing approaches to mixed-reality systems such as tangible user interfaces, augmented virtuality and augmented reality, as well as approaches to a more classical graphical user interface, especially the instrumental interaction model
the study of mixed-reality systems in the intensions of modality and multimodality.
Mixed reality is classically a highly interdisciplinary field that deals with signal processing, computer vision, computer graphics, user interfaces, human factors, wearable computers, mobile computers, information visualization, and display and sensor design [
9].
Mixed reality creates a space for the coexistence of physical and virtual elements, which allows the easy interaction between them. This view does not look on the space as two independent areas, where one is prior to another, but it blurs the boundaries between areas and creates bigger space where components from both areas can communicate in real time [
10].
The research made by Hughes et al. [
11] treats how to transform the core of mixed reality technology and methods into diverse urban terrain applications. This type of research is significant, because these types of applications can be used for military training and situational awareness, as well as for community learning to significantly increase the entertainment, educational, and satisfaction levels of existing experiences in public venues.
A mixed-reality-based user interface for the quality control monitoring of automobile body surfaces is developed in the research made by Munoz et al. [
12] in order to increase worker ergonomics and productivity. The results showed that the suggested mixed-reality-based interface improved user ergonomic parameters in terms of comfort, as well as user productivity in terms of time and accuracy when performing quality control tasks.
Authors Lee, Han, and Yang [
13] developed a scheme for creating a mixed reality-based digital manufacturing environment in which physical items, such as real images, are grouped with the virtual space of virtual products. Using this technology, they created a mixed-reality-based virtual factory layout-planning system and demonstrated its usefulness in the simulation of process-layout design. Their system is built on the usage of an image-based tracking method that extracts an arbitrary feature from actual pictures, such as a safety sign; such images can be utilized instead of artificial identifiers because they are easily obtainable from the place.
Through a comparison of commercially accessible mixed-reality applications, this study intends to assess the utility of mixed-reality technology in architectural design and construction layout. The study used two HoloLens devices linked to a hard hat to comply with construction safety rules and discovered nine mixed-reality applications designed for architectural and construction objectives. The capabilities of mixed-reality technology in these applications were assessed and compared for architectural design and construction layout under various on-site conditions [
14].
The paper wrote by Lee et al. [
15] presented a way for implementing mixed reality into the production process, as well as being a method for adapting common security signposts in the plant to replace a black square marker for visual fiducial recognition.
This paper’s discussion of the combination of maintenance systems and mixed reality techniques deals with the development of data visualization. The operator will be supported during maintenance activities by the overlap of virtual components in a real scene, allowing the operator to work in a safe and precise manner in the mixed environment of industrial maintenance [
16].
Maintenance data and changes to the physical object should be identified digitally in the future and transferred to the data management system in real time. A digital twin should allow for the unambiguous assignment of data to the appropriate elements, resulting in a virtual reflection of the condition of the industrial plant. This paper describes an approach to create a system-based digital twin for an industrial plant’s maintenance phase, which will allow data to be transmitted from mixed reality to a Product Data Management System [
17].
The goal of the paper made by Kokkas and Vosniakos [
18] is to use mixed-reality tools to create machine layouts, particularly for Flexible Manufacturing Systems. The purpose is for the user to assess alternative layouts based on non-measurable characteristics including operator experience, empirical or non-tacit knowledge, and on-site impression. By simulating the whole production process, which includes movement, manipulation, and processing of parts using real and virtual equipment in parallel and serial coexistence, the functional connection between the collaborating real physical and virtual machinery of the layout is enabled. The app is made with the ARKitTM version 5.1.0 API tool and Unity3D 2022.1.0.
The mixed-reality approach to simulation for testing reasons connected to automation software is discussed in this study. The objective is to evaluate real automation SW, such as the MES layer, by mixing real machines or components with simulated ones (in their example, using a 3D graphic simulation tool). Real machining centers, robots, pallets, conveyor belts, PLCs, and CNCs were implemented in a real application scenario. The exchange of products and information between real and virtual components has been realized with the help of a Java-based layer and an appropriate interchange zone [
19].
The research paper written by Sautter and Dalling [
20] looked at the benefits and drawbacks of mixed-reality-assisted on-the-job learning and described some practical implications for industrial training. The research of two use cases (the first: the training of new employees for the semi-automated assembly lines in the production of pneumatic cylinders revealed that mixed reality, and the second: includes learning modules with a focus on declarative and cognitive knowledge) technologies can assist learning factories overcome their constraints and maximize their potential for successful industrial training in key constructivist learning areas. To get the most out of mixed-reality technologies, they should be used for specific learning objectives and incorporated into a modular didactic framework, taking into account all of the features of successful mixed-reality application.
Using Mixed Reality and Ubiquitous Sensor Networks, this article provides a rapid and effective inspection method to ensure product performance following the design and manufacturing procedure. The addition of 3D information about the product has shown the improvement in workers understanding of the information for proper inspection. Furthermore, the connection of Mixed Reality and Ubiquitous Sensor Networks can assist in the efficient inspection of items because all essential information is available and may be supported anywhere in the system [
21].
Lee et al. [
22] proposed a method for designing virtual factory layouts based on mixed reality using legacy data that has previously been built in the real world. To do this, this research team created a way of acquiring simulation data from legacy data and processing it for mixed-reality visualization. They also built a mixed-reality display system that can recreate a virtual manufacturing layout using processed data. By validating the position and application of equipment in advance before arranging real ones on the project site, the developed method can eliminate industrial layout errors.
Production plants have faced numerous obstacles in recent years, notably those related to variable demand and changing customer and supplier requirements, forcing new technical roadmaps and interventions in production processes. Innovative technologies support the development of information processes aimed at employees in this proactive context. With this fact, augmented and virtual reality can be used for workforce training; they should be able to effectively engage with a human workforce [
23].
The proposed mixed reality structure consists of five layers: the first layer considers system parts; the second and third layers concentrate on architectural issues for component integration; the fourth layer is the application layer that executes the architecture; and the fifth layer is the user interface layer that allows user interaction [
24].
During the COVID-19 pandemic, the suggested model with the usage of mixed reality provides a comprehensive solution for huge building complexes and industrial parks, guaranteeing public safety and also the health and well-being of the facilities management team. The mixed reality technique allows for the remote processing of path layouts, avoiding human interaction and ensuring that there is no chance of virus transmission [
25].
In mechanical engineering, there are possibilities to use not only CL data for the machining process, but it also shows the possibilities of using simulation data from the pre-production phase for augmented reality. This research links the techniques used in the educational process with the necessities of real practice and a direct visualization solution for the operator in the machining process is also proposed [
26].
2. Materials and Methods
When we want to design the layout of production and distribution of markers, it is necessary to observe certain minimum distances between individual production equipment and fixed elements of building structures (e.g., pillars, walls, etc.) [
27].
A raster network model is assigned to the reference space for the location of the production system, which is dimensioned via the capacity calculation of the required areas, volumes, and production means.
According to the production (functional) activity, or production optimization, that has to take place in the reference space, work zones, resp.; sub-zones of the production process are placed in the raster, and the location of the Vumark is selected to display the virtual model in a real environment using a helmet HoloLens 2 for mixed reality. In the case of absence of a helmet for mixed reality, it can be used a smartphone or tablet that support virtual reality.
Models of topological relations of construction elements of the production system structure (already existing production machines, virtual production machines, handling and auxiliary equipment, etc.) are solved in individual zones which can also be included in Vumark and displayed then in the mixed reality.
Computational-interactive way of solving topological relations is realized on the basis of using simplified 2D models, accurate 3D models of construction elements of the production system, resp. These 3D models of production machines can be obtained from databases of software products or can be modeled using CAD systems at a 1:1 scale to match the size of a real, physical production machine when these production machines will be displayed in a real environment.
From the graphic database of CAD system are selected specific 2D, 3D models of the relevant building elements to the calculated reference points of their location, resp. In the layout of the production system, the raster model will be displayed on the floor in the real environment using a mini projector placed on the structure at the appropriate height, and the exact location of the Vumark will be defined.
Based on interactive decision-making procedures and solution visualization, the optimal variant of the production system structure is selected. In the case of a collision situation between the machines, the placement of the Vumark will be optimized.
The basis of the solution is a model of topological relationships of a pair of construction elements of the production system structure (
Figure 1). Based on the usage of the combinatorial principle, it is possible to create various variants of more complex structures of production systems from the basic model (
Figure 2,
Figure 3 and
Figure 4).
Syntax for the designation of parameters of the topological model of located construction elements of the production system are as follows [
28]:
x dimension in the positive direction of the x-axis, x dimension in the negative direction of the x-axis,
y dimension in the positive direction of the y-axis, y dimension in the negative direction of the y-axis,
x, y parameters of the safety zone, x, y coordinates of the insertion point,
x distance between the i-th and j-th object in the direction of the x-axis,
y distance between the i-th and j-th object in the direction of the y-axis,
i, j serial number,
d distance of insertion points,
α angle of rotation.
The insertion point of the construction element of the production system is located on the 0
x axis.
x12 > 0 the condition must be met in order to avoid collisions of the distributed elements.
The insertion point of the construction element of the production system is placed on the axis oy:
The condition must be met in order to avoid collisions of the distributed elements.
The insertion point of the construction element of the production system is rotated by an angle α, of the flow line of the insertion points and the axis 0
x.
The conditions must be met in order to avoid collisions between the distributed construction elements of the system (the geometric characteristics of individual production systems are also important for modelling the spatial relationships in the production systems. It is necessary to take into account spatial relations and basic geometric characteristics: outer shape and dimensions, maximal dimensions of the working space, orientation and position of the coordinates in the coordinate system of the production element with respect to the reference-coordinate system and geometric characteristics of the working space).
If the first insertion point is placed at the beginning of the coordinate system, then there are four possibilities when inserting the second point due to the division of the plane into quadrants. The insertion point can be located in:
For Quadrant I (if the inserted point is at the top right of the base element):
For Quadrant II. (if the inserted point is at the top left of the base element):
For Quadrant III. (if the inserted point is at the bottom left of the base element):
For Quadrant IV. (if the inserted point is at the bottom right of the base element):
The following applies to the size of the angle α (angle of rotation IP2):
When we insert elements into individual quadrants, inequalities for x- or y-coordinates must be met in order to maintain safety distances.
Models of typological relationships in production systems are characterized by considerable universality in terms of application conditions, as they are suitable for optimization of the production system design, but also for the traditional design of production systems.
The typological relationships presented in the paper are based on generally valid spatial principles and methodologies that are obligatory when the layout of production equipment on the production area is solved. These are applied in decision-making software algorithms. Their output is the optimal solution that eliminates the conflicting states of the location of the new means of production related to the immediate surroundings of the new layout, which is located through the Wumark brand and its optimal location is transformed into the mixed reality.
When choosing the location of the Vumark, we used a projector to project the raster on the surface of the production hall, with which we determined the exact location of the production equipment, and we eliminated collision situations when inserting the production equipment into the layout of the production system.
On the basis of the mentioned procedure, it is possible to place Vumark in the layout solution within the optimization of production system and classic mechanical engineering productions for piece and small-batch production.
The mentioned mathematical apparatus can be used in the placement of production equipment in shapes that are often used in such types of production (e.g., type “U”, type “L”, type “I”, type “diagonal” and type “star”).
4. Conclusions
Currently, the requirements for project activity and the understanding of the new methods development of production systems designing are changing significantly. The system development of design methods, procedures, and techniques is determined by the knowledge of other scientific disciplines, professional creativity, collective decision-making and other factors resulting from market conditions. In designing, the condition of comprehensive knowledge of the problems and their detailed analysis is essential, because it allows for the elimination of the generation of outdated and non-complex approaches of production systems. It leads to the application of new innovative solutions that guarantee the implementation of advanced technologies of the future. Only this approach is the guarantee of identification, optimization, or elimination of project deficiencies even before their future physical implementation [
31].
Virtual technologies enter the new areas of application that will significantly change the character of peoples’ lives. The improving digitization and virtualization of production and design leads to significant changes in the existing methods of designing production layouts. In the industry sector and especially in the production processes, the real and virtual worlds are increasingly integrated. The share of advanced technologies in all spheres of human activity is increasing. The system of knowledge and methodological procedures obtained from experiments with the system of integration of the virtual and real world shows that the usage of advanced technologies has its importance from a professional and practical point of view. In projects where mixed reality is used to make a visualization and simulation of the dynamic behavior of technical objects or project processes and systems in a real environment, virtual technologies are a significant contribution. This direction of research implemented in the design of production systems is suitable for small and medium enterprises (SME). For the solution of such project activities, from a technical point of view, it is possible to use various types of devices, such as helmet for mixed-reality Hololens 2, etc.
This paper describes an experiment using the Vuforia Studio software. Using Vumark´s digital machine models, which are modeled using a CAD system on a 1:1 scale, these digital models were integrated into the real environment as part of the layout design. The accuracy of the displayed model and technical and display units were tested in the conditions of the production environment of the Innovation and Prototyping Centre operated under the auspices of the Faculty of Mechanical Engineering, Technical University of Kosice.
The main contribution of the paper is the proposal of the methodical procedure for the design of production structures using mixed reality based on the general principles of innovative design. Synthesis conditions for the implementation phase of designing production facilities in mixed reality and experimental use were defined. The method of typological relations was chosen for the implementation of proposals of the layout of production machines and robots and the selection of the optimal variant. The result of this method of project activity in spatial solutions of production systems is, for the necessities of practice, more accurate descriptions of the layout of machines in space using mathematical and other principles. Industrial grids enable the creation of immediate, direct dimensional relationships between the exact surface and the location of the production system that realizes the production process, and the implementation of mixed reality allows us to display a 3D model of the production equipment directly on the selected surface and immediately detect collision situations directly on the place and propose the optimization of the location.
Production systems designing is an area in which it is necessary to follow trends and current opportunities that provide quality solutions and innovative technologies. Mixed reality is one of the technologies, where the potential of usage also has its place in project activities. The results of the experiment show that for a wider application of this projected methodology in practice, it will still be necessary to resolve any problems in detail, such as the accuracy of displaying objects, their orientation, spatial situation, and the optimal observer distance from the placed object.