Reflections on the Limited Pervasiveness of Augmented Reality in Industrial Sectors

Abstract

The paper aims to investigate the reasons why Augmented Reality (AR) has not fully broken the industrial market yet, or found a wider application in industries. The main research question the paper tries to answer is: what are the factors (and to what extent) that are limiting AR? Firstly, a reflection on the state of art of AR applications in industries is proposed, to discover the sectors more commonly chosen for deploying the technology so far. Later, based on a survey conducted after that, three AR applications have been tested on manufacturing, automotive, and railway sectors, and the paper pinpoints key aspects that are conditioning its embedding in the daily working life. In order to compare whether the perception of employees from railway, automotive, and manufacturing sectors differs significantly, a one-way analysis of variance (ANOVA) has been used. Later, suggestions are formulated in order to improve these aspects in the industry world. Finally, the paper indicates the main conclusions, highlighting possible future researches to start.

1. Introduction

Information for human interaction is available for access at any time and place, allowing operators to change settings or maintainers to perform maintenance and monitoring of tasks in real time with a high level of situational awareness. However, due to the flow of information and the need for operators and maintainers to access different systems for service support, there is a change of behavior and professional profile in progress. In terms of human factors, education for industry 4.0 goes through the process of understanding the best technologies and procedural approaches, so that the human being reaches the expected levels of efficiency and safety.

Several experiments are being carried out, indicating that there is much space for innovation through technological/procedural development. One of the technologies being tested is the use of Augmented Reality (AR). By delivering to the operator/maintainer a set of contextualized digital information, AR introduces computer-generated elements and objects overlapping the view of the real world [1], showing a significant amount of information, depending on the set of symbols associated with the environment of interest [2].

However, AR technology is not restricted to the hardware/software for personal use, but a whole range of applications, culminating in the delivery of the fused information, projected on the real world from the perspective of the user’s interest. Consequently, although the available technologies are not new (some date back decades), the arrangement linked to the advent of the Internet of Things (IoT) [3], providing an information flow not previously available, has given a set of processes that have brought novelty, being the junction what makes an environment conducive to innovation using AR.

Despite the wide range of AR applications in several market segments and the consequential forecast of market expansion (Figure 1), recent research on the use of AR technology in industrial environments has shown that major problems such as hardware and software development, weight, ergonomics, limited user acceptance, visual fatigue and concentration, data transfer, integration, security, content creation, adaptive instructions, marker tracking reliability, and cost [4] still exist. There is no specific hardware and software solution that solves all the applications demands in the industry, and the definition of symbols needs to be done contextually and depending on the type of operation/activity to be performed.

The manufacturing industry is concerned with the process of transforming raw materials into finished goods with added value. In such a way, AR can contribute, giving information that is not available in terms the user could not perceive by their own senses [6]. Several studies have proved that visual instructions are easier to be understood than text instructions [7,8]. Process monitoring and control, real-time evaluation of plant layout, plant and machinery maintenance, and plant and building construction for enhancing industrial safety, are some of the recent developments in AR for manufacturing [6,9].

In terms of activities for AR usage in manufacturing, the most prominent are assembly and maintenance [10]. Maintenance is frequently associated to application fields like remote assistance and training, while assembly is also frequently associated to training [6]. This perspective provides the interpretation that applications of augmented reality technologies are the tip of the iceberg of a set of processes and technologies to meet an operational demand within the industry.

It is not something specific to manufacturing, but it is clear the tendency to determine the compilation of necessary information during a set of activities that are naturally organized together to achieve the overall performance of a desired task. In terms of assembly and training, what is expected is an increase in the efficiency of manpower and usability of the AR implementation during preparation and operation, with enhanced performance for someone who is new in the activity he/she has just learned.

On the basis of the lack of applied solutions available in the literature all over the world [6], it is easy to perceive that the technology is ahead of the processes that would made use of it, in most cases. Even though this technology has been on the market for several years [11], it still requires an extensive analysis of its applicability, due to the inexistence of processes that will make use of the potentialities. To have the ability to evaluate a plant layout, for example, it is necessary to have all diagrams in databases being updated as new information accrues. If the update process is not available, the technology existence is useless.

Another consideration is the interaction with the devices that are hand-based. The major activities in the assembly or maintenance tasks need hands-free actions, while other kind of applications may have the possibility to use hands to interact with the hardware. This type of situation, in terms of ergonomics, generates potential impediments to the adoption of technology. Depending on the hardware/software solution, process and data availability, the adoption in the industry could be finally increased.

There are also activities where the cognitive aspect will be more demanding than in others, where the adoption of AR may favor the execution of the activity when it can assist the operator in its decision making. Cognitive ergonomics should not be overlooked in the industrial context, where a highly dynamic, integrated, and robotic environment prevents the exchange of information, once verbal, available in a timely manner. In this context, there is an increase in the operator’s cognitive activities in industrial plants with a reduced number of human beings, requiring the availability of information to obtain competitive advantages during the operation and maintenance of the systems. Maintenance is still largely done by humans who are not operating the machines, which leads to increasing reliance on computerized decision support systems.

Such discussion leads us to identify how to be more efficient and safer in an increasingly automated industry, where production lines must operate continuously, maintaining the quality of products and the overall efficiency of their processes. The AR is and will be used to expose the information in this environment by providing information to the operators/maintainers of the systems of interest at the moment and in the way that allows the human being to be more efficient.

To do so, one must build a path that will lead to success in adopting AR in each segment. The doubts are great, and several works are seeking to define the approaches that will pave the way. There are works in the area of performance indicators in the use of AR [12], visualization technologies [13,14,15,16,17,18,19,20,21,22,23,24,25,26,27], tactile perception technologies [28,29,30,31], how to generate content in AR in an easier way [32,33,34], how to structure the environment to access the information [35,36,37,38], among other approaches.

In this way, the present article intends to suggest AR approaches to overcome actual problems that this technology is encountering in the industrial domain.

2. Investigating the Causes of the Lack of Pervasiveness

As briefly mentioned in the introduction, besides the great potential widely recognized by experts, business analysts, and researchers, AR still encounters difficulties in being fully deployed and adopted in process industries and operations.

For verifying the suspected main reasons behind this slow-pace implementation that have been previously discussed, an empirical semiquantitative approach has been adopted in this research. Surveys in three different industrial sectors have been conducted, after testing AR applications in automotive industry, manufacturing industry, and railway industry. The three different applications have been created for digitally supporting and guiding the users in assembling operations in the automotive and manufacturing industry. Specifically, the chosen tasks were the assembly and disassembly of braking pads and disks of a vehicle for the automotive sector and the assembly and disassembly of a component of a high-speed packaging machine for the manufacturing sector. Differently, in the railway industry, the focus of the application was on supporting the train driver in solving possible software failures during the starting-up phase of the train from the cockpit. In all three cases, the Microsoft HoloLens has been used as AR device. Figure 2 shows three screenshots of the applications. Those in details, the technical prerequisite for their development and their creation in terms of technical choices and issues (still present in several projects [39]) faced during the process are being discussed in a different paper. This work, as already said, focuses on connecting impressions and feelings generated by the tests to the possible causes of the lack of AR pervasiveness.

2.1. Industry Survey Based on AR Applications

To avoid possible bias during the evaluation process caused by the interviewer pre-assumptions, firstly the participants were asked to brainstorm about all the difficulties, downside, and pitfalls of the applications and of the process. Figure 3 shows the methodological steps used during the survey for identifying the main factors that are reducing AR penetration on the market.

Firstly, the interviewees had to reflect on the AR application as a whole: from the very starting moment of the chosen operation, to the development, until the real test. After the brainstorming session, the interviewer collected and clustered in 11 main factors, all the focus points mentioned by the groups, and the interviewees were asked to rank them using a Likert scale ranging from 0 to 5, where 0 represents the less critical value. In total, fifty-six employees that tested the AR solutions were interviewed: Twenty-four in the manufacturing sector, twenty-four in the automotive sector, and eight in the railway sector.

Figure 4 offers the graphical representation through radar plots of the results of the survey.

After the brainstorming session, several limitations and pitfalls were identified by the three different groups, and they were defined or named in slightly different ways. As previously explained, once these sessions terminated, the interviewer clustered all the points mentioned by the interviewees, in order to avoid repetitions that could affect the validity of the next step of the survey. The 11 main factors identified and used for the ranking are the following:

• Department/Unit not available to handle AR projects;
• Skills not present in the company to handle AR projects;
• IT Infrastructures;
• Financial resources;
• Ergonomics;
• Corrective Lenses/Glasses;
• Distraction;
• Image Recognition Method;
• Internet Connection;
• Digitalization Time;
• Responsiveness.

The results acquired with the three surveys, based on the Likert Scale evaluation show comparable concerns and AR limitation motivations regardless of the industrial sector. The values are expressed in percentage points. Together with the mean value and the standard deviation (sd), they represent the possible contribution (according to the interviewees’ considerations) of a specific factor to the overall lack of AR pervasiveness.

2.2. Main Factors for Lack of AR Pervasiveness

As possible to see in Figure 4, Department/Unit not available to handle AR projects (88–92%), IT Infrastructures (85–90%), and Digitalization Time (84–87%) were considered the main factors for a lack of pervasiveness. It is valuable to stress that those three parameters underline how industrial companies are still not fully capable to support the adoption of AR technologies, due to lack of proper company units or departments that can fully devote working time on developing and maintaining these solutions. Organizational issues are important when considering industry; a similar conclusion has been highlighted in [4,12]. In second place, a not proper IT Infrastructure (meaning AR hardware and software limitation) was also considered a main cause. Finally, the time needed for creating valuable digital information to show through the devices was considered a concrete bottle neck.

In order to analyze the differences among the 11 factors in relation with industries and to compare whether the perceptions of employees from railway, automotive, and manufacturing sectors differ significantly, a one-way analysis of variance (ANOVA) has been conducted. The results showed that for the nine factors (Department/Unit not able to handle AR projects, Skill not present in the company to handle AR projects, IT Infrastructures, Ergonomics, Corrective Lenses/Glasses, Distraction, Internet Connection Digitalization Time) the perception of employees from different industries did not differ significantly. However, the employees’ perception for two factors (Image Recognition Method, [F(2, 53) = 4.09, p = 0.022] and Responsiveness, [F(2, 53) = 10.16, p <= 0.001]) varied significantly for different industries. In

Table 1. ANOVA results for each factor in comparison with industrial sectors.

Factor	DF 1	F-Value	p-Value
Department/Unit not able to handle AR projects	(2, 53)	0.54	0.585
Skill not present in the company to handle AR projects	(2, 53)	0.05	0.948
IT Infrastructures	(2, 53)	0.95	0.395
Financial Resources	(2, 53)	3.04	0.056
Ergonomics	(2, 53)	1.13	0.330
Corrective Lenses/Glasses	(2, 53)	0.74	0.482
Distraction	(2, 53)	0.74	0.482
Image Recognition Method	(2, 53)	4.09	0.022
Internet Connection	(2, 53)	0.93	0.401
Digitalization Time	(2, 53)	0.18	0.840
Responsiveness	(2, 53)	10.16	<=0.001

¹ Degree of Freedom (DF): Industry = 2, Error = 53.

are summarized the main outcomes of the ANOVA analysis.

The reason why 2 mentioned factors have discrepancy in the perception values can be found in the characteristics that the different industrial sectors have in affecting the deployment of AR solutions such as more/less complicated parts to recognize from a software perspective and more/less complicated setup of the operation/task to perform.

To improve the understandability of the survey, its results, and to indicate possible measures to take, the 11 factors, already originated by a first round of grouping, have been further clustered to three final factors, and considered as overall limitations to the AR pervasiveness. As shown in Figure 5, these were the organizational limitations (accounted for 40%), the human interaction limitations (accounted for 24%), and the technological/technical limitations (accounted for 36%). With different weight, each part contributes to reduce the possible deployment of AR technologies in the industrial sectors. AR and IT in general still remain “technology push” in the relatively conventional operations and maintenance domains.

3. How to Facilitate the Adoption of AR in Industries

Once the possible causes have been identified, it is equally important to try to suggest future measures to adopt for a better deployment of AR in industrial domains.

It is clear that the first step (i) needs to be made on an organizational level within the company. Working in an environment not ready to get benefit from a new technology can only cause inefficiency and lack of trust in the technology itself. To avoid that, after a proper evaluation based on prototype applications and cost analysis for medium- and long-term horizons, companies need to facilitate the creation of units and departments with skilled employees fully dedicated to support and make smooth the introduction of AR. It is important to remember that the introduction phase represents always a fragile moment of a new technology/product [40]. Finally, it is desirable, at least during the introduction phase, to run AR solutions in parallel to old systems, for being ready to backup any possible failures caused by “infant mortality” factors, reducing the disturbance on normal operations.

Secondly (ii), if the company has prepared the proper playground from an organizational perspective, it is vital to focus on the technological/technical layout to ease an extended use of AR. It can be obtained, for example, by reducing the digitalization time, selecting only the necessary parts/components/systems to overlap on the reality, or utilizing reliable recognition methods for reducing delay time and latency and increasing the responsiveness.

AR programmers and developers have a “consistent addiction” for adding new features and utilities. It generates several upgrades to install that can result in additional problems for the customers. The possibilities with AR are endless. Therefore, it should not surprise that developers are eager to try them all. However, the average consumer or company is not always looking to push technology boundaries right away; they are keener to get familiar with the AR solutions, in order to reach a proper level of robustness and reliability.

This phenomenon could quickly cause people to run back to their previous technologies, where they feel safe, comfortable, and familiar. To become a more mainstream product for the industrial domain, AR developers should focus on the essential aspects of a smart device, focusing on the basic needs of the industries.

The last (iii) (but not in terms of relevance) regards the interaction of the users with the technology. It encompasses a wide spectrum of problems and situations. Therefore, it is quite challenging to pinpoint solutions for reducing its impact. Regarding the ergonomics of the device, an analogy can be made with the sunglasses market. The number of features that a product can offer is irrelevant if a pair of AR glasses produces a big discomfort during their daily routine, eventually leading the user to take them off [41].

A similar reflection can be done for users with corrective lenses and glasses. The Next Web [41] indicates that 75% of Americans use glasses or some form of corrective lenses today. With three-quarters of the population in need of corrective lenses, smart glasses need this option for facilitating their embedding in the industrial market. A simple implementation of such an important feature would surely bring AR glasses to a point where any user could adopt them with ease.

Finally, it has to be said that for what concerns human interaction limitations, the industries, as final users of the technology, have limited influence on the design or manufacturing of the AR devices; consequently, companies that are implementing AR solutions nowadays need to identify the operations in which lack of ergonomics does not represent an issue in terms of safety or performance.

4. Discussion

There are few points that can be highlighted for a proper discussion. First, even though the results of the survey are giving interesting insights, they are still not enough to derive a general conclusion. Secondly, as already mentioned, the interviewees’ number is not statistically relevant to confirm the results obtained. More studies need to be conducted in order to properly propose solutions and take actions for accelerating the AR adoption. However, some useful indications have been found; those can be used as starting point and as initial framework for future research. The ANOVA analysis has proved that there is no significant difference among different industrial sectors; nevertheless, the next round of survey should be conducted to investigate how AR is received by other industrial environments. If the results are confirmed, it could be valuable to start applying some of the suggestions mentioned in Section 3.

5. Conclusions

This paper represents the first in a series, which will examine the overall impacts of AR in industries, testing solutions but also evaluating the benefit of AR deployment in industries focusing on the impact of the additional workload and possible distraction factors in operations.

This paper provides a first study to understand why AR technologies have penetrated several market segments (i.e., videogames industry or live events) but without fully convincing industrial domains.

This illustration was based upon brainstorming sessions and evaluations with three companies in three different industrial sectors: manufacturing industry, automotive sector, and railway sector. The aim was to pinpoint from practitioner perspective, the crucial points in terms of limitations of AR technologies. Such an analysis can be used as means for identifying where manufacturers and companies need to work together for facilitating the introduction of AR in industrial environments. The awareness of stakeholders on the inherently difficult processes helps in reviewing the current procedures, to plan and employ measures to prevent problems and conflicts.

AR devices could replace the actual systems as our regular information interface, since they serve as a seamless extension of the body, bringing the information interface closer to your natural senses. But until a pair of AR glasses can blend as well as any other pair of glasses with a user’s style, their popularity (and the trust) will not take off.

The findings of the research will serve as the basis of a more structured approach for speeding up the impact and the introduction of AR in industrial applications at present. Consequently, future research will quantify the results of those applications in terms of cognitive workload assessment and technology readiness. The authors strongly believe that when all the actual listed problems are overcome, the main remaining issue will be then the contextualization of the information. Operators should only benefit from the necessary information according to their roles and their experiences.