Next Article in Journal
Ridge Preservation Using a Novel Enzyme-Treated Xenograft. A Preliminary Retrospective Histomorphometric Investigation
Next Article in Special Issue
Efficient Multi-Player Computation Offloading for VR Edge-Cloud Computing Systems
Previous Article in Journal
Retinal Drug Delivery: Rethinking Outcomes for the Efficient Replication of Retinal Behavior
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 10
Issue 12
10.3390/app10124259
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

The Application of Augmented Reality in the Automotive Industry: A Systematic Literature Review

by
Răzvan Gabriel Boboc
*,
Florin Gîrbacia
and
Eugen Valentin Butilă
Department of Automotive and Transport Engineering, Transilvania University of Brasov, RO-500036 Brasov, Romania
*
Author to whom correspondence should be addressed.
Appl. Sci. 2020, 10(12), 4259; https://doi.org/10.3390/app10124259
Submission received: 13 May 2020 / Revised: 17 June 2020 / Accepted: 18 June 2020 / Published: 21 June 2020
(This article belongs to the Special Issue Innovative Solutions for Augmented and Virtual Reality Applications)
Browse Figures
Versions Notes

Abstract

:
Augmented reality (AR) is a fairly new technology enabling human machine interaction by superimposing virtual information on a real environment. Potential applications can be found in many areas of research from recent years. This study presents a systematic review of existing AR systems in the automotive field, synthesizing 55 studies from 2002 to 2019. The main research questions are: where AR technology has been applied within the automotive industry, what is the purpose of its application, what are the general characteristics of these systems, and what are the emphasized benefits and challenges of using AR in this field? The aim of this paper is to provide an insight into the AR applications and technologies in the automotive field.

1. Introduction

In many industries, implementing modern technologies brings success in the tough global market. It propels a company’s innovativeness forward and can even lower production expenses and maximize profits. Automotive manufacturers have always remained at the forefront in taking advantage of the latest technological advancements. Thus, the automotive industry has experienced a significant development, mainly due to the adoption of new advanced technological improvements such as virtual mock-up and prototyping, advanced manufacturing techniques based on sophisticated robotic systems, automated vehicle safety and user-friendly interfaces for improving driving experience.
In recent years, many automotive manufacturers have developed a keen interest in augmented reality (AR), mainly due to its accessibility and potential for generating innovative solutions. AR is a form of human machine interaction (HMI) in which virtual components are inserted and superimposed on the real world, creating the illusion of an ‘enriched’ reality. It allows users to obtain certain information they need about some aspects or procedures directly in the working environment [1] and it has the potential to enrich a person’s sensory perception [2]. Based on Milgram’s definition of a reality‒virtuality continuum [3], AR represents a component part of mixed reality (MR), which is the merging of the real and virtual worlds. Milgram’s concept of a reality‒virtuality continuum is exemplified for the automotive industry in Figure 1. The applications of AR range from industrial to everyday life activities, in various domains, like manufacturing, robotics, healthcare, education, military, entertainment and so on. There are several extensive studies that address the implications and impact of AR on industry, as well as the challenges and directions of development in this field [4,5,6,7,8,9].
AR provides promising new possibilities in almost every area of the automotive industry due to its potential as an interactive and intuitive interface. The usage of AR in the automotive industry has been explored through various platforms. For instance, AR can substantially improve the user experience with head-up displays (HUDs). HUDs are display systems adapted from military aviation [10] to convey visual information in the driver’ field of vision. They are see-through devices designed to present various real-time essential information in order to increase safety. There are two ways provided through HUDs: adding information into the environment or highlighting some elements already present in the environment [11]. One such interesting solution is to overlay a video stream over the car in front of the driver, where you can see what is in front of it [12].
AR contributes to many other aspects of the automotive industry, such as maintenance, repairing, diagnostics, inspection or training [1]. AR technology has also begun to be used as a marketing tool so that customers can interactively customize their car before ordering [13,14]. Spot welding inspection through projector-based spatial AR [15,16], tracking some parts of a vehicle using handheld devices [17] or performing maintenance procedures using head-mounted displays (HMDs) are some examples of using AR in various ways and through various platforms. This human-centered technology has proven to be useful for operators and workers in the area of car production, but also for drivers in order to improve certain characteristics related to driving activity. The benefits of AR were demonstrated through major collaborative projects such as ARVIKA, ARTESAS, EFA2014, AVILUS, EGYPT [18].
The goal of this study is to make a contribution to the current research by providing a cohesive view of systems using AR in a field that evolves significantly from year to year. This paper fills an important gap in the automotive research field and allows understanding on how AR technology was applied in this industry, what its added value is and what the challenges and trends are. Therefore, the specific research questions are the following:
Where and for what purpose is AR technology applied in the automotive industry?
What are the general characteristics of AR-based systems developed for the automotive field?
What are the main benefits and challenges of AR for the automotive industry?
In contrast to other review papers related to the use of AR in industry, this study focuses specifically on the main application areas of AR in the automotive industry, describing the specific AR technological solutions implemented and the benefits and challengers that they bring.

2. Research Methodology

2.1. Paper Selection

To find relevant literature sources, five well-known online research databases related to technology were used: ACM digital library, IEEE Xplore, ScienceDirect, Scopus and ISI Web of Science. Two consecutive searches were carried out in accordance with the preferred reporting items for systematic reviews and meta-analyses (PRISMA) [19]. PRISMA is the most common reporting guideline for systematic reviews, helping authors to improve the reporting of reviews and meta-analysis. The primary search occurred on 15 April 2019 and the second one on 28 January 2020. AR papers were selected and approved based on the following criteria:
  • search terms: “augmented reality” (“AR”) AND “automotive” (“automotive industry”);
  • publication year: no limit was imposed;
  • publication language: only English;
  • publication type: only research and review articles published in journals.
The search focused only on works that rely specifically on AR solutions in the automotive industry. The papers published in another language than English and other forms of publication (e.g., articles published in conference proceedings, book chapters, newspaper articles, etc.) were not included. The second consideration was taken into account because journal papers are subject to review and thus have a higher degree of research relevance. Figure 2 shows the PRISMA flow chart and the search results.
The search produced a total of 1128 results extracted from the aforementioned library databases. After removing the duplicate papers (56 papers), 1072 publications were analyzed in order to identify the papers that study the application of AR in automotive industry. After the analysis, 112 papers resulted, which were carefully reviewed to meet the selection criteria. Three papers could not be accessed as full text and they were rejected. A final total of 55 articles were selected. Only a few exceptions were made regarding 4 conference papers, which were preserved because they have a high degree of visibility due their presentation at a premier AR conference (e.g., ISMAR) and their subject being of high interest for the present study.
Figure 3a presents a chart with the number of papers according to the database where they were found. The highest number of articles that matched the search criteria were found in Scopus (31.91%), while from ACM only one article (1.1%) was kept after PRISMA analysis. From these articles, some of them were duplicates: 52.73% of them were found in a single database, 25.45% in two databases and 21.82% in three databases (Figure 3b).

2.2. Quality Assessment

The quality assessment was achieved by evaluating each paper according to nine questions provided in [20], which pursue the fulfillment of certain conditions so that the study can be considered as being of appropriate scientific quality. The quality assessment helps to analyze the studies in order to verify the degree of compliance with certain pre-established standards. The papers were screened based on the quality criteria represented by the nine questions as follows: each article received a score depending on how the respective criterion was met. If the study answered all those criteria satisfactorily, it was kept, if not, it was rejected. These criteria refer to the content (e.g., a clear description is provided by authors, appropriate explanation for the proposed method, the results are correctly presented, etc.), but also to the implications or importance of the study for the scientific community.

2.3. Analysis

The methodology by which the data were extracted from the selected articles was implemented by the authors by cross-checking. One of the authors of this study systematically extracted relevant information from the articles and these were cross-checked by the other two to validate the correctness of the extracted data. Two main categories were defined in which the selected data could be classified from the beginning of the analysis: ‘in-car’ systems (containing works relating to systems designed to help drivers) and ‘other’ systems (developed in other sectors of the automobile industry). For each of the two main categories, other subcategories or groups were identified during the review process. They were divided according to certain common characteristics. The decision to keep, modify, or remove one of the identified categories was made by the two coders, who also verified the accuracy of the extracted information.

3. Results

3.1. Research Categories

The 55 selected papers were classified according to the criterion specified in [21]: application papers, technical papers, conceptual papers, and review papers. Figure 4 shows a diagram of the groups for each of the two categories mentioned above. For each of the defined groups, the number of papers is specified in Figure 5. The first type (application papers) contains works focused on developing and testing of AR solutions in a real/simulated environment. Technical papers are the ones presenting the development of hardware/software solutions or algorithms for AR systems. Conceptual papers address specific aspects of AR solutions, proposing new concepts for their adoption in practice. Review papers summarize the existing literature on a specific topic, trying to provide the state of the art on that area.

3.2. Descriptive Statistics

Some statistics regarding the year of publication and the origin of the studies from a geographical point of view for the entire sample are presented in Figure 5 and Figure 6. As can be observed, the amount of research has increased from year to year, starting from 1999. From the total number of papers, a percentage of 29.09% contain a user study. To determine the origin of the study, the affiliation of the first author was considered as reference. In this regard, 54.55% (30 papers) of publications were developed in Europe, 16.36% (9) in North America, 14.55% (8) in Asia, 7.27% (4) in Oceania, 5.45% (3) in South America and 1.82% (1) in Africa. Most of these studies come from Germany (21.82%), the USA (14.55%) and Greece (9.09%). This is explained by the fact that many companies that produce devices dedicated to the use of AR, as well as research groups, are located in these countries, as stated in [22].

3.3. Prior Surveys

AR is a technology that has gained popularity in the field of the automotive industry in recent years, and systematic reviews have shown that AR applications have great potential in this field [23,24]. From the selected papers for our research, a total of 15 review articles were found, as shown in Figure 4. Most of them are not exclusively focused on the use of AR in the automotive industry, but they do mention this area, offering relevant information on how this technology can provide improvements in various applications. The distribution of the review articles according to the year of publication is shown in Figure 7. From the 15 articles identified, six of them focus their research on the field of manufacturing, four approach the industry field in general, and five refer to AR systems used to help drivers, like HUDs (Figure 8).
A short review about AR applications in the automotive and aerospace industries was provided in [25], a paper published in 2005, focusing on ten projects in which they were involved. Maintenance, design, production and training are some of the areas where AR technology was implemented. In the study, the authors summarized their research and development, identifying opportunities and limitations of AR in every context. Of the total of 10 projects presented, six were related to the automotive industry and four to the aerospace industry. As a conclusion at the time at which the study was completed, there were many technical issues to be resolved before AR could be applied in the field due to the low level of maturity of the technology.
After a decade, in 2015, another review of the experimental implementations of AR for various purposes was presented in [1]. The study refers to 39 articles, of which 24% are reported to be related to the automotive field. Their focus was on using AR as a tool in through-life engineering services, and the presented applications were included in areas like maintenance, repairing, diagnostics, inspection, training and so on. The conclusion was that AR setup activities have been successfully implemented, even if there were still issues concerning hardware and software aspects.
A systematic review was presented in a paper published in 2018 [4], and it investigated key performance indicators related to the use of AR in industrial applications, especially for automotive maintenance and training purposes. Time and error reduction, ease of use, spatial representation of information and cognitive workload were the performance metrics selected to explore the added value of AR for industry. The empirical study was based on the technology acceptance model and involved experts in the field. Analyzing two other constructs (usefulness and intention to use), the study concluded that key performance indicators play a positive role in the user’s perception when he/she uses industrial AR-based applications.
A recent study presented in [7] tries to identify a few key factors and challenges occurring in industrial projects that use AR, based on a qualitative and qualitative survey. The most important success factors resulting from the survey were the visibility of information, improvements in efficiency, user acceptance, ergonomics aspects and the usability of user interface. User acceptance was also an important challenge, as evidenced by the outcome of the study, as well as tracking technology and the weight of the device. The conclusion of the study was that technical and organizational factors are crucial to the effective and successful implementation of AR in industry.
The application of AR in manufacturing is also an area of interest, evidenced by the number of articles dealing with this aspect. The review presented in [25] is focused on industrial AR applications for product development, highlighting their characteristics and challenges in manufacturing but also in the following areas: design, maintenance, reverse engineering, assembly, training and quality assurance.
In [26] a comprehensive study evaluates the use of AR applications in manufacturing and design. Hardware devices for display and software solutions were also analyzed. The challenges of AR technologies in terms of accuracy, registration, latency and interfaces were discussed and various studies of design and manufacturing activities were reported. The authors estimated that by the time the article was published, 80% of the AR technology was available, but they predicted that the remaining 20% would be harder to implement and would take a longer time. A systematic evaluation of challenges encountered in AR-based simulation in manufacturing is presented in [27]. The thorough investigation covered a wide range of fields, simulation methods and tools, addressing the main practices and future trends for technologies and industrial applications. The evolution of intelligent manufacturing solutions from 2005 to 2016 was provided in [28]. Some key technologies used in this field were discussed, such as Internet of Things, cloud computing or communication systems. AR was presented as a core technology for future key research, alongside VR.
Authors from [21] focus their research on the use of AR in manufacturing. They identified 174 relevant papers published from 2006 to 2017 obtained from three different databases. The analysis was done according to the classification of the articles found in four groups: review papers, technical papers, application papers and conceptual papers. Among other considerations, they found that there is a growing interest in using AR technology in the industry and it has a great potential in many industrial operations. Another paper in which applications that use digital virtual models for manufacturing are evaluated was published recently [29]. They selected 49 related articles published between 2014 and 2019, from which 41% were related to the automotive field, but AR technology was used in only 4% of them, especially for automotive interior assessment.
Opportunities and challenges associated with AR applications for the automotive industry were addressed in [22]. The authors gave a brief description of various existing systems and the issues that occur related to the user, interfaces, interactions and methods of evaluation. The article presents the opportunities regarding driving activities and the challenges that arise in relation to the driver’s attention and visual perception. The conclusion of the study was that in the field of transportation much more research is needed to lead to the emergence of safe and reliable AR applications.
The evolution of HUDs, defined as systems that project information in the visual field of the driver, is studied in [23,30]. In the first paper, research topics and trends are discussed in order to provide useful information related to further design and development. The authors have taken into consideration data from 2010 to 2016 and tried to forecast the evolution of HUD system characteristics for the following two years (2016–2018). They also provided an analysis regarding the existing patents related to HUD systems. The second article approaches this topic in terms of functional requirements for a long period of time (1994–2016). Trying to address the type of information suggested by research studies and presented by the commercial HUDs, the authors concluded that automotive HUDs have a great potential for improving the driver’s experience, especially if they are used in conjunction with other technologies.
In [31] a short history of automotive instrumentation was presented, enouncing systems and solutions developed to display information to the driver in the primary field of view. The article concludes that AR HUDs are an innovative system offering situation-adapted information with minimal distraction of the driver, leading to increased driving safety. Finally, carsickness is addressed in [32], where guidelines are provided to promote the design of self-driving vehicle technology. Authors stated in the paper that AR display systems may represent a promising approach to reduce carsickness.
Table 1 lists the review papers identified in our study. The table focuses on the number of articles included in the study, the year of publication, the time span for which the research was done, the scientific database where the articles were found and the percentage of articles found that belong to the automotive field.
The 15 review studies provide a summary of current literature on various aspects of AR-based solutions in various fields. However, they are not specific to the automotive industry.

3.4. Application Areas

Except the review studies, the rest of the papers (40) were organized according to the field of application, while also keeping the classification established at the beginning of the analysis. Thus, the studies were divided in two categories: ‘in-car’ systems, developed in order to increase safety and to provide additional information to drivers (47.5% of papers), and other systems, developed to help designers and workers in the process of design and development in the automotive sector (52.5% of studies). For the second category, four areas of application were identified: assembly, design, maintenance and manufacturing. The distribution of studies according to the application field of the reviewed works is presented in Figure 9.

3.4.1. AR in Designing, Manufacturing, Training, Maintenance and Customer Support

Automobile companies have introduced AR technology in manufacturing environments for assembly, maintenance or inspection tasks. Typical tasks can benefit from AR by providing assistance to operators using different types of information superimposed in the work environment, supplemented with other technologies like telepresence systems [33]. In addition, technicians can use AR-based systems which provide on-site instructions (in front of their eyes) about the methodology they need to adopt [14] for vehicle diagnostics and repair, such as how and where to act, thus reducing the learning time and mental effort [24]. Therefore, even inexperienced people can be turned into mechanics by following the instructions provided through AR applications for the maintenance of their own cars [34], following 2D or 3D instructions on a mobile device [35].
The advantages of AR have been explored by car manufacturers through major collaborative projects, from car design to customer support services [18]. AR provides operators with the necessary documentation information at the right time, reducing the mental effort they have to make: 2D or 3D information can be provided, animated or not, related to the execution procedure, steps to be taken, the tool used or the task to be performed.

3.4.2. AR in Driving

In this paper we use the term ‘in-car’ systems to refer to those AR systems designed to improve driving. They are created to increase the user’s level of immersion, giving him/her an enhanced visual, tactile or acoustic experience [12]. There are two modalities in which the visual experience can be augmented: adding information into the environment or highlighting some elements already present in the real environment such as road signs, crosswalks and pedestrians [11]. These innovative display technologies offer new paradigms of interaction, which increase the degree of assistance for drivers, but can also improve the aspects related to ergonomics [36]. For instance, the standard navigation system of vehicles usually displays the information on the dashboard screen, and this could distract the driver. In addition, nowadays smartphones represent an important source of distraction. In order to reduce these negative aspects, AR-based systems can be used [37], but they must be built in such a manner as to not interpose the information between the driver and what is in front of him/her on the road [38].
A constant effort is being made by automotive companies to improve the human machine interface, which is a key component of car development. They have tried to develop various complex original systems, integrating smart AR devices such as Google glass [39], RGB-D sensors for 3D motion recognition such us Microsoft Kinect [40] or other methods involving sound—for instance, rendering engine sound through the car audio system [12]. The potential of AR systems has a twofold purpose: to enrich the driving experience and to prevent accidents by increasing the driver’s perception and attention [41].
Thus, AR can be used as a medium of interaction with the automobile, for a better awareness of the environment [42] and the visualization of additional information that can have a significant effect on the driver’s experience [43]. The vehicle windshields can be used as navigational aids, representing the medium for augmented displays. Automotive companies have developed systems to display various information in the driver’s visual field, such as warnings when a pedestrian wants to cross the street and highlighting road signs with the aim of increasing safety [38]. Furthermore, there are researchers who develop systems for the display of road intersection guidance [44], for ego-lane analysis [45], or even optical see-through displays where the windshield of a car functions as a display to allow drivers to virtually see through objects that are obstacles in their way [46].

3.5. The Purpose of AR Systems in the Automotive Industry

The papers were then organized according to the purpose of the research papers. Keeping the same hierarchy presented in Section 2.3, among studies from the first category (‘in-car’ AR systems) four subcategories were identified (Figure 10). The majority of AR applications (47.37%) are focused on the examination of different display concepts for HUD-like systems or on the effect of visual warnings. In [47] the comparison between two type of AR interfaces (conformal and screen—fixed arrow) was investigated, while also analyzing the driver performance and user attention. The difference between the two display interfaces is that the screen-fixed arrows are rendered at a fixed location on the display screen, while conformal arrows are attached to specific objects in the driver’s field of view to draw attention to an object of interest. In [48] it is suggested that conformal AR graphics presented as visual warnings on HUDs can have a positive impact on the driver’s behavior. However, [49] showed that animated AR warnings can produce higher judgment accuracy for drivers compared to conformal cues. Other design aspects were dissected in [36,50,51]. In [52] two types of warnings (audio and video) were investigated in order to find out their effects on effectiveness and acceptance. AR user acceptance was also addressed in [53]. The impact of visual and haptic augmentation was investigated in [54], showing that they have the potential to reduce divided attention and cognitive load.
From the total papers that approaches the subject of ‘in-car’ systems, 15.79% deal with driving safety improvement. In [55] a HUD system was proposed to display driving safety information under various types of circumstances. Automotive applications for collision avoidance designed to increase driving safety were also presented in [56,57].
AR has been applied in cooperative systems like vehicle-to-vehicle communication (V2V) in order to obtain visibility of traffic in front of large vehicles [58] or to exchange social information amongst drivers [59]. Researchers have shown that V2V infrastructure and AR interfaces could contribute to improving drivers’ behavior and practices, such as reducing aggressive comportment and creating a pro-social attitude [60].
A number of papers do not necessarily specify a field of application for AR; they present a general AR solution, like an improved method or algorithm for a general context. In this subcategory we have included 26.31% of the total number of studies from the ‘in-car’ AR systems category. New methods for tracking registration [61], crosstalk perception simulation [62], head-pose tracking [63], ghost image elimination [64] and efficient lossless compression [65] are proposed in these papers.
For the second category referring to ‘other’ AR systems, six subcategories were identified: product customization, remote collaboration, general AR solutions, assistance, training and ergonomics (Figure 11). Most of the papers (28.57%) have been included in the ‘assistance’ subcategory because they present works in which AR is used to assist people in accomplishing various tasks: manual assembly [66,67,68] human-robot collaborative assembly [69,70] and prototyping in the design process [71].
Ergonomics aspects were evaluated in 23.81% of the articles. Ergonomic research conducted to determine the number of requirements for designing AR systems to assist automotive service technicians were proposed in [72,73]. A reference model to analyze the experience of workers from the automotive industry was developed in [74]. Other papers were investigating automotive interior ergonomics [75,76].
A percentage of 19.05% from these papers were classified as part of the ‘training’ subcategory. In [77] an AR application was presented for a training system in the process of car manual transmission maintenance. An AR training system to illustrate spot-weld positions on vehicle panels for new operators was proposed in [16]. A way to enhance the ability of shop-floor operators using AR technology is provided in [78,79] by exploring the use of mobile devices for training in assembly.
Two papers could not be included in any of these subcategories: the work from [80], which describes a collaborative AR system that allows participants to share a 3D space, and [81], where a web-based platform for designing and customizing a car hood was presented. The other works (19.05%) address AR aspects that can be used as general solutions. In [17] the author has developed and evaluated a new markerless tracking system for AR applications. Contributions on improving the accuracy of AR tracking were also provided in [82]. A new method of rendering which increases the quality of the images projected during the car design process was described in [83], and a new method to support decisions for the aeronautical transportation industry was proposed in [51]. Figure 12 presents a summary of AR applications for the automotive industry that were identified in the selected articles.

3.6. General Characteristics of ‘in-car’ AR-Based Systems

Table 2 shows the characteristics of ‘in-car’ AR systems, which are generally focused on improving the driving experience. The features being discussed are the visualization device, the displayed information, the environment type where the system was developed and the type of asset used to augment the driver experience.
As we can see, 6 studies from the total of 19 (31.58%) were performed in laboratory conditions, using different types of simulators, 12 of them (63.16%) were performed in real environments and 1 of them [59] cannot be categorized according to one of these types. A percentage of 26.32% of the reviewed papers have presented works with more than one type of asset. The rest of the studies have used 2D graphics (68.24%) and 3D graphics (31.58), and other works provided text, video, audio or animated graphics as displayed information. Regarding the content of the displayed information, a percentage of 21.05% of works provided more than one information type. Most of them presented the information on HUD-like systems regarding the following elements: pedestrian tracking (31.58%), intersection guidance (21.05%) or vehicle detection (15.79%). Other studies provided information about speeding or traffic sign detection.

3.7. General Characteristics of ‘Other’ AR-Based Systems

Table 3 shows a list of all the studies presenting works which are not necessarily related to driving but intervene in other sectors in the field of car construction. In the table, the following characteristics are summarized for each paper: the visualization device used to display the AR content, the tracking method according to the two main blocks: marker-based and markerless, the development tool used for AR system, the environment type where the AR system was developed or tested and the digital information used to augment the environment.

3.7.1. Visualization Device

Regarding the visualization device, head mounted displays, HMDs and AR glasses have the most significant use among the analyzed works. A projector device was preferred in 3 of the total of 21 papers and hand-held devices (smartphones and tablets) in 4 papers. In one paper the author did not mention what device was used and in three papers this characteristic is not applicable (Figure 13). Figure 14 shows the distribution of papers according to the visualization device for the four application fields presented in Subsection 3.4.

3.7.2. Development Tools

The analysis of development tools reveals that 25.87% of the studies have presented systems that use self-developed software. Ten studies (47.62%) have used the AR development tools Vuforia and ARToolkit, while one study declared the use of the OpenCV library. Four articles did not specify the implementation software, or this characteristic is not applicable (Figure 15a). The distribution of development tool per year is presented in Figure 15b. One tendency that can be identified is the increasing number of uses of the Vuforia library in recent years.

3.7.3. Tracking Method

The tracking method is another important characteristic that is the subject of attention in the present research. In 52.38% of the studies marker-based technology was used, whereas 38.1% of studies used markerless systems (Figure 16a). For two studies this characteristic cannot be investigated because they are exploratory studies based on interviews taken with the aim to clarify some aspects in the pre-development phase. According to Figure 16b, the distribution of works per year according to the tracking method reveals that there is an approximately uniform distribution between the two methods.

3.7.4. Other Characteristics

From Table 3, we can see that researchers have integrated a variety of assets in order to provide supplementary information to operators via AR displays. A percentage of 38.1% of the reviewed papers have presented works with more than one type of asset. The rest of the works have used only one type of asset, as follows: 31.58% of the reviewed studies used 2D graphics, 63.16% used 3D graphics of 3D models and 21.05% used text. Videos combined with audio information were used in [79].
Table 3 also reveals that 6 studies from the total of 21 (28.57%) were performed in laboratory conditions, using different types of simulators, 12 of them (57.14%) were performed in real environments, and 3 of them cannot be put in one of these types.

3.8. User Studies

From the total number of analyzed papers, only 17 (42.5%) present user studies for testing the AR systems. Table 4 presents an overview of these studies in terms of number of participants, environment, type of investigations and statistical analysis: 5 of them fall into the first category and 12 into the second category.
As a way to validate their research results, most of the authors have preferred to use subjective evaluations through questionnaires (nine studies—52.94%) or through interviews (four studies—23.53%), while the others chose other means.

3.9. Benefits and Challenges of AR Systems

The main areas of the automotive industry in which AR studies have been conducted include the improvement of driver safety, the improvement of product design performance and manufacturing operations and higher quality training for assembly and maintenance. Figure 17 presents the distribution of the main areas of the automotive industry in which AR can bring improvements. The main benefits were classified in six categories (i.e., better training, enhanced safety conditions, performance improvement, faster execution of activities, error reduction, usage satisfaction).
We identified nine papers that emphasized better training methods using AR. The training for assembly and maintenance can be improved by using AR due to the possibility of personalizing the experience considering the operator’s level of expertise. The training process becomes more intuitive due to on-the-job instructions provided by the AR. We identified in four studies that using collocated step-by-step instruction for maintenance or service tasks can reduce errors. Furthermore, among the benefits of using AR we identified three studies that report positive usage satisfaction of AR-guided assembly or maintenance tasks.
One of the important benefits of this method is the enhanced safety of driving conditions, which was identified in 27 papers. AR provides a novel method of displaying information for the automotive industry. It can provide real-time route information for car passengers, traffic information and also data related to traffic management systems. It can also be used to evaluate safety issues such as collision avoidance, driver safety and fatigue.
AR can also be used to improve the driver’s overall experience in terms of the safety and reliability of the vehicle. The driver can use AR systems to become more comfortable in the vehicle and increase awareness while driving. By using AR, the driver will be able to see the road ahead from their seat, while also having the possibility of inspecting the overlaid map in the field of view.
Another important benefit of this method is the enhanced safety of driving conditions. AR provides a novel method of displaying information for the automotive industry. It can provide real-time route information for car passengers, traffic information and also data related to traffic management systems. It can be also used to evaluate safety issues such as collision avoidance, driver safety and fatigue.
AR can also be used to improve the driver’s overall experience in terms of the safety and reliability of the vehicle. The driver can use AR systems to become more comfortable in the vehicle and increase awareness while driving. By using AR, the driver will be able to see the road ahead from their seat, while also having the possibility of inspecting the overlaid map in the field of view.
In the design phase, AR has been used for the inspection and improvement of vehicle designs by superimposing various 3D models on real-world car body mock-ups in order to evaluate the resulting product. By using this evaluation method, the modification within the car interior looks realistic and can be applied to the real car prototype. We identified 21 papers that highlighted the improvements of performance in design. The AR system has the potential to improve efficiency and achieve faster execution of activities (identified in nine papers) due to its visual data output that can help the operator to perform the task in an intuitive and interactive way.
While AR offers many advantages, the usage of AR in the automotive industry raises several challenges. One of the most crucial technical challenges encountered in the selected articles is the alignment of the real environment and virtual objects. About 72% of the studies used visual tracking as the registration method. Because automotive-related environments are dynamic (dirty, the position of objects changes), the usage of markers to facilitate tracking should be avoided and markerless tracking technologies should be used where possible [24]. In relation to the AR system and application design, about 34% of the studies considered that using an easy and ergonomic method is an important issue that provides an intuitive and natural user experience. The selection of proper devices is a critical challenge for the usability of an AR system. About 32% of the studies declared that using a hands-free visualization method improves productivity. Finally, some studies reported that technology acceptance, privacy and security are issues that have to be taken into consideration when using AR in the automotive industry [4,7,26,28,53].

4. Discussion

The first research question sought to identify the application area and the purpose of the developed AR systems. In this regard, the papers are first divided in two categories: ‘in-car’ AR systems developed for improving user experience and other AR systems which are linked to the automotive field. Four areas where identified for the second category: manufacturing, maintenance, design and assembly. AR has been found to have great potential for application in many industrial operations in the automotive field, especially for the assistance and training of operators, but also in certain aspects related to ergonomics. This is important because the factors related to efficiency (usage time, cognitive workload) and physical comfort are essential when working with AR systems [85]. Regarding the systems developed for driver assistance, a large majority of studies focus on the display interface. This is somewhat understandable as it is very important that AR elements are presented in such a manner that they do not obscure the driver’s view of the road in order to prevent the potential dangers that may occur while driving [22]. Other issues discussed are some technical details regarding the development and implementation of AR systems for driver assistance, as well as aspects related to safety and to the upcoming technology that will revolutionize the future of driving and vehicle-to-vehicle communication.
The general characteristics of AR systems were identified in an effort to respond to the second research question. The technological solutions for visualization used in the analyzed papers are mainly handheld devices (smartphones and tablets), HMDs and AR smart glasses. The main advantages that these devices bring to the AR applications are ease of use, portability and various ways of implementation. There has been a growing trend of mobile devices and glasses usage in recent years, which is explained by the following facts: handheld devices have the advantage of low cost and simplicity in creating applications, they are more suitable for maintenance tasks, and smart glasses are suitable for tasks where the user has busy hands, such as assembly. Projector-based systems are developed mainly for manufacturing and design tasks to display 3D information on the workspace. HMDs also represent a solution, but they depend on computers or do not allow high resolution graphics [86]. In addition, ergonomics issues have been discussed in connection with HMDs [21], but these as well as technical problems will be overcome with the evolution of hardware and software technology [87]. The analysis of the tracking technology revealed that there is approximate equality between marker-based and markerless solutions, with a slight advantage of the former. This could be a consequence of the fact that marker-based solutions are easier to implement and provide better accuracy, as found in [21]. Regarding the systems designed for ‘in-car’ use, some studies are based on commercial HUDs, and a smaller percentage of them use HUDs designed ‘in-house’. Many studies are trying to improve aspects of user interaction, but there is still a need for research related to HMI and human factors [23].
In the analyzed studies various types of information that are overlapping elements in the work environment were presented: text, images, 3D graphics and animations. They can be classified into five groups, as stated in [9]. In addition to this information regarding visual perception, some studies have presented a combination of other elements with the visual ones in order to stimulate audio and tactile perception. From the whole sample of evaluated articles, less than half used user studies to validate the research, and most of them used a subjective evaluation by collecting data through questionnaires or interviews. For statistical analysis, the vast majority of studies used the ANOVA technique.
A number of advantages or benefits have been identified in connection with the use of AR in the automotive industry in an attempt to answer the third question proposed in this study. Among these, the largest percentage of the total number of articles found an important contribution in terms of enhancing safety conditions. Other notable benefits identified by the researchers were the following: performance improvements related to the accomplishment of a certain activity, improvements in terms of time execution and error reduction, but some articles also reported improvements in user satisfaction when using AR systems.
Among the most common challenges currently facing AR-based systems for the automotive industry is registration technology, as there are not yet sufficiently accurate tracking systems to be implemented on a large scale [10]. Display devices are also a problem still unsolved: they need reliable, easily transportable hands-free devices that do not burden the user. In addition, there are discussions about the information displayed for drivers: what the best display method is and how not to confuse or block the driver’s view and what practices should be applied when talking about the rendering, merging, orientation and placement of virtual objects into the physical world [55].
Besides the major benefits it can provide, AR technology also carries certain risks, like ethical issues such as privacy, security and accessibility risks [10,50,88,89], which have been discussed by researchers and they should not be neglected, but they are not the object of this study.

5. Conclusions

This study aims to present, through a systematical approach, the state of the art of AR as a useful technology in the automotive industry. There are more applications and systems in this field and this effort tries to address some research questions in order to provide useful findings for future research and to complement previous AR review studies.
Without claiming to offer an exhaustive study, this article seeks to systematize research papers related to the automotive industry and to present the general characteristics of AR systems developed for this field, as well as the existing benefits and challenges.
In the future, we intend to explore more deeply the individual application fields in the automotive industry. In this regard, we should try to identify the trends that are emerging in terms of AR implementation as this technology goes to the highest level of maturation.

Author Contributions

Conceptualization, R.G.B. and F.G.; methodology, R.G.B.; validation, F.G.; formal analysis, R.G.B. and F.G.; investigation, R.G.B. and E.V.B.; resources, R.G.B.; data curation, F.G.; writing—original draft preparation, R.G.B.; writing—review and editing, F.G. and E.V.B.; visualization, E.V.B.; supervision, F.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding

Acknowledgments

We would like to dedicate this paper to Doru Talaba, who was a remarkable mentor and supervisor.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Dini, G.; Mura, M.D. Application of augmented reality techniques in through-life engineering services. Procedia CIRP 2015, 38, 14–23. [Google Scholar] [CrossRef]
  2. Daponte, P.; De Vito, L.; Picariello, F.; Riccio, M. State of the art and future developments of the Augmented Reality for measurement applications. Measurement 2014, 57, 53–70. [Google Scholar] [CrossRef]
  3. Milgram, P.; Kishino, F. A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. 1994, E77-D, 1321–1329. [Google Scholar]
  4. Jetter, J.; Eimecke, J.; Rese, A. Augmented reality tools for industrial applications: What are potential key performance indicators and who benefits? Comput. Hum. Behav. 2018, 87, 18–33. [Google Scholar] [CrossRef]
  5. Martinetti, A.; Marques, H.; Singh, S.; Dongen, L. Reflections on the limited pervasiveness of augmented reality in industrial sectors. Appl. Sci. 2019, 9, 3382. [Google Scholar] [CrossRef] [Green Version]
  6. De Souza Cardoso, L.F.; Mariano, F.C.M.Q.; Zorzal, E.R. A survey of industrial augmented reality. Comput. Ind. Eng. 2020, 139, 106159. [Google Scholar] [CrossRef]
  7. Masood, T.; Egger, J. Augmented reality in support of Industry 4.0—Implementation challenges and success factors. Robot. Comput.-Integr. Manuf. 2019, 58, 181–195. [Google Scholar] [CrossRef]
  8. Egger, J.; Masood, T. Augmented reality in support of intelligent manufacturing—A systematic literature review. Comput. Ind. Eng. 2020, 140, 106195. [Google Scholar] [CrossRef]
  9. Gattullo, M.; Scurati, G.W.; Fiorentino, M.; Uva, A.E.; Ferrise, F.; Bordegoni, M. Towards augmented reality manuals for industry 4.0: A methodology. Robot. Comput.-Integr. Manuf. 2019, 56, 276–286. [Google Scholar] [CrossRef]
  10. Arnaldi, B.; Guitton, P.; Moreau, G. Virtual Reality and Augmented Reality: Myths and Realities; ISTE Ltd, John Wiley & Sons: Hoboken, NJ, USA, 2018. [Google Scholar]
  11. Eyraud, R.; Zibetti, E.; Baccino, T. Allocation of visual attention while driving with simulated augmented reality. Transp. Res. Part F Traffic Psychol. Behav. 2015, 32, 46–55. [Google Scholar] [CrossRef]
  12. Ferreira, M.; Gomes, P.; Silvéria, M.K.; Vieira, F. Augmented reality driving supported by vehicular Ad Hoc networking. In Proceedings of the 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Adelaide, Australia, 1–4 October 2013; pp. 253–254. [Google Scholar]
  13. Gay-Bellile, V.; Bourgeois, S.; Tamaazousti, M.; Naudet, S. A mobile markerless augmented reality system for the automotive field. In Proceedings of the IEEE ISMAR 2012 Workshop on Tracking Methods and Applications, Atlanta, GA, USA, 5–8 November 2012. [Google Scholar]
  14. Halim, A.Z. Applications of augmented reality for inspection and maintenance process in automotive industry. J. Fundam. Appl. Sci. 2018. [Google Scholar] [CrossRef]
  15. Zhou, J.; Lee, I.; Thomas, B.; Menassa, R.; Farrant, A.; Sansome, A. n-situ support for automotive manufacturing using spatial augmented reality. Int. J. Virtual Real. 2012, 11, 33–41. [Google Scholar] [CrossRef] [Green Version]
  16. Doshi, A.; Smith, R.T.; Thomas, B.H.; Bouras, C. Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. Int. J. Adv. Manuf. Technol. 2017, 89, 1279–1293. [Google Scholar] [CrossRef] [Green Version]
  17. Lima, J.P.R.; Simoes, F.; Almeida, M.; Figueiredo, L.; Teixeira, J.M.; Teichrieb, V. Markerless tracking system for augmented reality in the automotive industry. Expert Syst. Appl. 2017, 82, 100–114. [Google Scholar] [CrossRef]
  18. Gay-Bellile, V.; Bourgeois, S.; Larnaout, D.; Tamaazousti, M. Applications of augmented reality for the automotive industry. In Fundamentals of Wearable Computers and Augmented Reality, 2nd ed.; Barfield, W., Ed.; CRC Press Taylor & Francis Group: Boca Raton, FL, USA, 2015. [Google Scholar]
  19. Moher, D.L.A.; Tetzlaff, J.; Altman, D.G. The PRISMA Group Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef] [Green Version]
  20. Hawker, S.; Payne, S.; Kerr, C.; Hardey, M.; Powell, J. Appraising the Evidence: Reviewing Disparate Data Systematically. Qualitative health research 2002, 12, 1284–1299. [Google Scholar] [CrossRef]
  21. Bottani, E.; Vignali, G. Augmented reality technology in the manufacturing industry: A review of the last decade. IISE Trans. 2019, 51, 284–310. [Google Scholar] [CrossRef] [Green Version]
  22. Gabbard, J.L.; Fitch, G.M.; Kim, H. Behind the glass: Driver challenges and opportunities for AR automotive applications. Proc. IEEE 2014, 102, 124–136. [Google Scholar] [CrossRef]
  23. Park, J.; Park, W. Functional requirements of automotive head-up displays: A systematic review of literature from 1994 to present. Appl. Ergon. 2019, 76, 130–146. [Google Scholar] [CrossRef]
  24. Regenbrecht, H.; Baratoff, G.; Wilke, W. Augmented reality projects in the automotive and aerospace industries. IEEE Comput. Graph. Appl. 2005, 25, 48–56. [Google Scholar] [CrossRef]
  25. Lu, S.C.Y.; Shpitalni, M.; Gadh, R. Virtual and augmented reality technologies for product realization. CIRP Ann. 1999, 48, 471–495. [Google Scholar] [CrossRef]
  26. Nee, A.Y.C.; Ong, S.K.; Chryssolouris, G.; Mourtzis, D. Augmented reality applications in design and manufacturing. CIRP Ann. 2012, 61, 657–679. [Google Scholar] [CrossRef]
  27. Mourtzis, D.; Doukas, M.; Bernidaki, D. Simulation in manufacturing: Review and challenges. Procedia CIRP 2014, 25, 213–229. [Google Scholar] [CrossRef] [Green Version]
  28. Zhong, R.Y.; Xu, X.; Klotz, E.; Newman, S.T. Intelligent manufacturing in the context of industry 4.0: A review. Engineering 2017, 3, 616–630. [Google Scholar] [CrossRef]
  29. Zhu, W.; Fan, X.; Zhang, Y. Applications and research trends of digital human models in the manufacturing industry. Virtual Real. Intell. Hardw. 2019, 1, 558–579. [Google Scholar] [CrossRef]
  30. Betancur, J.A.; Villa-Espinal, J.; Osorio-Gomez, G.; Cuellar, S.; Suarez, D. Research topics and implementation trends on automotive head-up display systems. Int. J. Interact. Des. Manuf. Ijidem 2016, 12, 199–214. [Google Scholar] [CrossRef]
  31. Knoll, P.M. Some pictures of the history of automotive instrumentation. J. Soc. Inf. Disp. 2017, 25, 44–52. [Google Scholar] [CrossRef]
  32. Diels, C.B.; Jelte, E. Self-driving carsickness. Appl. Ergon. 2016, 53, 374–382. [Google Scholar] [CrossRef]
  33. Manuri, F.; Pizzigalli, A.; Sanna, A. A state validation system for augmented reality based maintenance procedures. Appl. Sci. 2019, 9, 2115. [Google Scholar] [CrossRef] [Green Version]
  34. Peddie, J. Augmentad Reality: Where We Will All Live; Springer: Tiburon, CA, USA, 2017. [Google Scholar]
  35. Hořejší, P. Augmented reality system for virtual training of parts assembly. Procedia Eng. 2015, 100, 699–706. [Google Scholar] [CrossRef] [Green Version]
  36. Pfannmüller, L.; Kramer, M.; Senner, B.; Bengler, K. A comparison of display concepts for a navigation system in an automotive contact analog head-up display. Procedia Manuf. 2015, 3, 2722–2729. [Google Scholar] [CrossRef] [Green Version]
  37. Dumitru, A.I.; Girbacia, T.; Boboc, R.G.; Postelnicu, C.-C.; Mogan, G.-L. Effects of smartphone based advanced driver assistance system on distracted driving behavior: A simulator study. Comput. Hum. Behav. 2018, 83, 1–7. [Google Scholar] [CrossRef]
  38. Wassom, B.D. Augmented Reality Law, Privacy, and Ethics: Law, Society, and Emerging AR Technologies; Publisher: Syngress, Waltham, MA, USA, 2014; pp. 1–338. [Google Scholar]
  39. Bennakhi, A.; Safar, M. Ambient technology in vehicles: The benefits and risks. Procedia Comput. Sci. 2016, 83, 1056–1063. [Google Scholar] [CrossRef] [Green Version]
  40. Gruyer, D.; Magnier, V.; Hamdi, K.; Claussmann, L.; Orfila, O.; Rakotonirainy, A. Perception, information processing and modeling: Critical stages for autonomous driving applications. Annu. Rev. Control 2017, 44, 323–341. [Google Scholar] [CrossRef]
  41. Fu, W.-T.; Gasper, J.; Kim, S. Effects of an in-car augmented reality system on improving safety of younger and older drivers. In Proceedings of the 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Adelaide, Australia, 1–4 October 2013; pp. 59–66. [Google Scholar]
  42. Ng-Thow-Hing, V.; Bark, K.; Beckwith, L.; Tran, C.; Bhandari, R.; Sridhar, S. User-centered perspectives for automotive augmented reality. In Proceedings of the 2013 IEEE International Symposium on Mixed and Augmented Reality—Arts, Media, and Humanities (ISMAR-AMH), Adelaide, Australia, 1–4 October 2013; pp. 13–22. [Google Scholar]
  43. Froehlich, P.; Schatz, R.; Leitner, P.; Mantler, S.; Baldauf, M. Evaluating realistic visualizations for safety-related in-car information systems. In Proceedings of the CHI ‘10 Extended Abstracts on Human Factors in Computing Systems, Atlanta, GA, USA, 10–15 April 2010; pp. 3847–3852. [Google Scholar]
  44. Akaho, K.; Nakagawa, T.; Yamaguchi, Y.; Kawai, K.; Kato, H.; Nishida, S. Route guidance by a car navigation system based on augmented reality. Electr. Eng. Japan 2012, 180. [Google Scholar] [CrossRef]
  45. Berriel, R.F.; de Aguiar, E.; de Souza, A.F.; Oliveira-Santos, T. Ego-Lane Analysis System (ELAS): Dataset and algorithms. Image Vis. Comput. 2017, 68, 64–75. [Google Scholar] [CrossRef] [Green Version]
  46. Narzt, W.; Pomberger, G.; Ferscha, A.; Kolb, D.; Müller, R.; Wieghardt, J.; Hörtner, H.; Lindinger, C. Augmented reality navigation systems. Univers. Access Inf. Soc. 2006, 4, 177–187. [Google Scholar] [CrossRef]
  47. Gabbard, J.G.; Smith, M.; Tanous, K.; Kim, H.; Jonas, B. AR drivesim: An immersive driving simulator for augmented reality head-up display research. Front. Robot. Ai 2019, 6. [Google Scholar] [CrossRef] [Green Version]
  48. Kim, H.; Gabbard, J.L.; Anon, A.M.; Misu, T. Driver behavior and performance with augmented reality pedestrian collision warning: An outdoor user study. IEEE Trans. Vis. Comput. Graph. 2018, 24, 1515–1524. [Google Scholar] [CrossRef]
  49. Merenda, C.; Kim, H.; Tanous, K.; Gabbard, J.L.; Feichtl, B.; Misu, T.; Suga, C. Augmented reality interface design approaches for goal-directed and stimulus-driven driving tasks. IEEE Trans. Vis. Comput. Graph. 2018, 24, 2875–2885. [Google Scholar] [CrossRef]
  50. Lebeck, K.; Ruth, K.; Kohno, T.; Roesner, F. Arya: Operating system support for securely augmenting reality. IEEE Secur. Priv. 2018, 16, 44–53. [Google Scholar] [CrossRef]
  51. Suarez-Warden, F.; Mendívil, E.G. Problem solving in transference of procedural knowledge aided by AR to support aeronautical transportation decisions. Int. J. Interact. Des. Manuf. 2018, 12, 337–344. [Google Scholar] [CrossRef]
  52. Schwarz, F.; Fastenmeier, W. Augmented reality warnings in vehicles: Effects of modality and specificity on effectiveness. Accid. Anal. Prev. 2017, 101, 55–66. [Google Scholar] [CrossRef]
  53. Wintersberger, P.; Frison, A.; Riener, A.; Sawitzky, T.V. Fostering user acceptance and trust in fully automated vehicles: Evaluating the potential of augmented reality. Presence 2019, 27, 46–62. [Google Scholar] [CrossRef]
  54. Kim, S.; Dey, A.K. Augmenting human senses to improve the user experience in cars: Applying augmented reality and haptics approaches to reduce cognitive distances. Multimed. Tools Appl. 2016, 75, 9587–9607. [Google Scholar] [CrossRef]
  55. Abdi, L.; Abdallah, F.B.; Meddeb, A. In-vehicle augmented reality traffic information system: A new type of communication between driver and vehicle. Procedia Comput. Sci. 2015, 73, 242–249. [Google Scholar] [CrossRef] [Green Version]
  56. Nilsson, J.; Ödblom, A.; Fredriksson, J.; Zafar, A. Using Augmentation Techniques for Performance Evaluation in Automotive Safety; Springer: New York, NY, USA, 2011; pp. 631–649. [Google Scholar] [CrossRef]
  57. Huang, S.-C.; Chen, B.-H.; Chou, S.; Hwang, J.; Lee, K. Smart car [Application notes]. IEEE Comput. Intell. Mag. 2016, 11, 46–58. [Google Scholar] [CrossRef]
  58. Gomes, P.; Olaverri Monreal, C.; Ferreira, M. Making vehicles transparent through V2V video streaming. Comput. Sci. Eng. 2012, 13, 930–938. [Google Scholar] [CrossRef]
  59. Rakotonirainy, A.; Schroeter, R.; Soro, A. Three social car visions to improve driver behaviour. Pervasive Mob. Comput. 2014, 14, 147–160. [Google Scholar] [CrossRef] [Green Version]
  60. Schroeter, R.; Rakotonirainy, A.; Foth, M. The social car: new interactive vehicular applications derived from social media and urban informatics. In Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Portsmouth, New Hampshire, 17–19 September 2012; pp. 107–110. [Google Scholar]
  61. An, Z.; Xu, X.P.; Yang, J.H.; Yan, Y. A real-time three-dimensional tracking and registration method in the AR-HUD system. IEEE Access 2018, 6, 43749–43757. [Google Scholar] [CrossRef]
  62. Hockh, S.; Frederiksen, A.; Renault, S.; Hopf, K.; Gilowski, M.; Schell, M. Exploring crosstalk perception for stereoscopic 3D head-up displays in a crosstalk simulator. J. Soc. Inf. Disp. 2015, 23, 417–428. [Google Scholar] [CrossRef]
  63. Murphy-Chutorian, E.; Trivedi, M.M. Head pose estimation and augmented reality tracking: An integrated system and evaluation for monitoring driver awareness. IEEE Trans. Intell. Transp. Syst. 2010, 11, 300–311. [Google Scholar] [CrossRef]
  64. Qin, Z.; Lin, F.; Huang, Y.; Shieh, H.D. Maximal acceptable ghost images for designing a legible windshield-type vehicle head-up display. IEEE Photonics J. 2017, 9, 1–12. [Google Scholar] [CrossRef]
  65. Rao, Q.; Chakraborty, S. Efficient lossless compression for depth information in traffic scenarios. Multimed. Syst. 2019. [Google Scholar] [CrossRef]
  66. Lampen, E.; Teuber, J.; Gaisbauer, F.; Bär, T.; Pfeiffer, T.; Wachsmuth, S. Combining simulation and augmented reality methods for enhanced worker assistance in manual assembly. Procedia CIRP 2019, 81, 588–593. [Google Scholar] [CrossRef]
  67. Makris, S.; Pintzos, G.; Rentzos, L.; Chryssolouris, G. Assembly support using AR technology based on automatic sequence generation. CIRP Ann. 2013, 62, 9–12. [Google Scholar] [CrossRef]
  68. Reinhart, G.; Patron, C. Integrating augmented reality in the assembly domain—Fundamentals, benefits and applications. CIRP Ann. 2003, 52, 5–8. [Google Scholar] [CrossRef]
  69. Makris, S.; Karagiannis, P.; Koukas, S.; Matthaiakis, A.S. Augmented reality system for operator support in human–robot collaborative assembly. CIRP Ann.-Manuf. Technol. 2016, 65, 61–64. [Google Scholar] [CrossRef]
  70. Michalos, G.; Kousi, N.; Karagiannis, P.; Gkournelos, C.; Dimoulas, K.; Koukas, S.; Mparis, K.; Papavasileiou, A.; Makris, S. Seamless human robot collaborative assembly—An automotive case study. Mechatronics 2018, 55, 194–211. [Google Scholar] [CrossRef]
  71. Verlinden, J.C.; Horváth, I. Analyzing opportunities for using interactive augmented prototyping in design practice. Ai Edam-Artif. Intell. Eng. Des. Anal. Manuf. 2009, 23, 289–303. [Google Scholar] [CrossRef] [Green Version]
  72. Anastassova, M.; Burkhardt, J.M.; Megard, C.; Ehanno, P. Results from a user-centred critical incidents study for guiding future implementation of augmented reality in automotive maintenance. Int. J. Ind. Ergon. 2005, 35, 67–77. [Google Scholar] [CrossRef]
  73. Anastassova, M.; Burkhardt, J.M. Automotive technicians’ training as a community-of-practice: Implications for the design of an augmented reality teaching aid. Appl. Ergon. 2009, 40, 713–721. [Google Scholar] [CrossRef] [PubMed]
  74. Peruzzini, M.; Grandi, F.; Pellicciari, M. How to analyse the workers’ experience in integrated product-process design. J. Ind. Inf. Integr. 2018, 12, 31–46. [Google Scholar] [CrossRef]
  75. Qiu, S.; Jing, X.; Fan, X.; He, Q.; Fan, X.; He, Q. Using AR technology for automotive visibility and accessibility assessment. In Proceedings of the 2011 International Conference on Virtual Reality and Visualization, Beijing, China, 4–5 November 2011; pp. 217–224. [Google Scholar]
  76. Soon, C.S.; Ghazilla, R.A.B.R.; Jen, Y.H.; Suen, P.Y. Framework of augmented reality approach towards ergonomic assessment of driver vehicle package design. J. Teknol. 2015, 77, 113–118. [Google Scholar] [CrossRef] [Green Version]
  77. Aziz, F.A.; Abdullah, F.; Win, L.L. Using marker based augmented reality for training in automotive industry. Int. J. Recent Technol. Eng. 2018, 7, 118–121. [Google Scholar]
  78. Holm, M.; Danielsson, O.; Syberfeldt, A.; Moore, P.; Wang, L. Adaptive instructions to novice shop-floor operators using Augmented Reality. J. Ind. Prod. Eng. 2017, 34, 362–374. [Google Scholar] [CrossRef] [Green Version]
  79. Morkos, B.; Taiber, J.; Summers, J.; Mears, L.; Fadel, G.; Rilka, T. Mobile devices within manufacturing environments: A BMW applicability study. Int. J. Interact. Des. Manuf. 2012, 6, 101–111. [Google Scholar] [CrossRef]
  80. Regenbrecht, H.T.; Wagner, M.; Baratoff, G. Magicmeeting: A collaborative tangible augmented reality system. Virtual Real. 2002, 6, 151–166. [Google Scholar] [CrossRef] [Green Version]
  81. Mourtzis, D.; Michael, D.; Psarommatis, F.; Giannoulis, C.; Michalos, G. A web-based platform for mass customisation and personalisation. CIRP J. Manuf. Sci. Technol. 2014, 7, 112–128. [Google Scholar] [CrossRef]
  82. Maidi, M.; Mallem, M.; Benchikh, L.; Otmane, S. An evaluation of camera pose methods for an augmented reality system: Application to teaching industrial robots. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Springer: Heidelberg, Germany, 2013; Volume 7420, pp. 3–30. [Google Scholar]
  83. Menk, C.; Jundt, E.; Koch, R. Visualisation techniques for using spatial augmented reality in the design process of a car. Comput. Graph. Forum 2011, 30, 2354–2366. [Google Scholar] [CrossRef]
  84. Schwarz, F.; Wolfgang, F. Visual advisory warnings about hidden dangers: Effects of specific symbols and spatial referencing on necessary and unnecessary warnings. Appl. Ergon. 2018, 72, 25–36. [Google Scholar] [CrossRef] [PubMed]
  85. Del Amo, I.F.; Galeotti, E.; Palmarini, R.; Dini, G.; Erkoyuncu, J.; Roy, R. An innovative user-centred support tool for Augmented Reality maintenance systems design: A preliminary study. Procedia CIRP 2018, 70, 362–367. [Google Scholar] [CrossRef]
  86. Urbas, U.; Vrabič, R.; Vukašinović, N. Displaying product manufacturing information in augmented reality for inspection. Procedia CIRP 2019, 81, 832–837. [Google Scholar] [CrossRef]
  87. Van Lopik, K.; Sinclair, M.; Sharpe, R.; Conway, P.; West, A. Developing augmented reality capabilities for industry 4.0 small enterprises: Lessons learnt from a content authoring case study. Comput. Ind. 2020, 117, 103208. [Google Scholar] [CrossRef]
  88. Raisamo, R.; Rakkolainen, I.; Majaranta, P.; Salminen, K.; Rantala, J.; Farooq, A. Human augmentation: Past, present and future. Int. J. Hum.-Comput. Stud. 2019, 131, 131–143. [Google Scholar] [CrossRef]
  89. Aukstakalnis, S. Practical Augmented Reality: A Guide to the Technologies, Applications, and Human Factors for AR and VR; RR Donnelley: Crawfordsville, IN, USA, 2016. [Google Scholar]
Applsci 10 04259 g001 550
Figure 1. Adapted Milgram’s reality‒virtuality continuum for the automotive industry.
Figure 1. Adapted Milgram’s reality‒virtuality continuum for the automotive industry.
Applsci 10 04259 g001
Applsci 10 04259 g002 550
Figure 2. PRISMA flow chart and search results.
Figure 2. PRISMA flow chart and search results.
Applsci 10 04259 g002
Applsci 10 04259 g003 550
Figure 3. Distribution of the papers according to the research database (a) and the number of databases where they were found (b).
Figure 3. Distribution of the papers according to the research database (a) and the number of databases where they were found (b).
Applsci 10 04259 g003
Applsci 10 04259 g004 550
Figure 4. Distribution of papers according to their type.
Figure 4. Distribution of papers according to their type.
Applsci 10 04259 g004
Applsci 10 04259 g005 550
Figure 5. Distribution of the reviewed papers according to the year of publication, classified as articles with or without a user study.
Figure 5. Distribution of the reviewed papers according to the year of publication, classified as articles with or without a user study.
Applsci 10 04259 g005
Applsci 10 04259 g006 550
Figure 6. Geographic distribution of the 55 reviewed papers.
Figure 6. Geographic distribution of the 55 reviewed papers.
Applsci 10 04259 g006
Applsci 10 04259 g007 550
Figure 7. Distribution of review articles per year.
Figure 7. Distribution of review articles per year.
Applsci 10 04259 g007
Applsci 10 04259 g008 550
Figure 8. Distribution of review articles per field.
Figure 8. Distribution of review articles per field.
Applsci 10 04259 g008
Applsci 10 04259 g009 550
Figure 9. Distribution of papers according to application field.
Figure 9. Distribution of papers according to application field.
Applsci 10 04259 g009
Applsci 10 04259 g010 550
Figure 10. Distribution of papers related to ‘in-car’ AR systems according to their purpose.
Figure 10. Distribution of papers related to ‘in-car’ AR systems according to their purpose.
Applsci 10 04259 g010
Applsci 10 04259 g011 550
Figure 11. Distribution of papers referring to ‘other’ AR systems according to their purpose.
Figure 11. Distribution of papers referring to ‘other’ AR systems according to their purpose.
Applsci 10 04259 g011
Applsci 10 04259 g012 550
Figure 12. AR applications in automotive.
Figure 12. AR applications in automotive.
Applsci 10 04259 g012
Applsci 10 04259 g013 550
Figure 13. Distribution of works related to AR visualization device.
Figure 13. Distribution of works related to AR visualization device.
Applsci 10 04259 g013
Applsci 10 04259 g014 550
Figure 14. Distribution of display devices used in the selected papers according to the application field.
Figure 14. Distribution of display devices used in the selected papers according to the application field.
Applsci 10 04259 g014
Applsci 10 04259 g015 550
Figure 15. Distribution of work according to the AR development tool (a) and according to development tool per year (b).
Figure 15. Distribution of work according to the AR development tool (a) and according to development tool per year (b).
Applsci 10 04259 g015
Applsci 10 04259 g016 550
Figure 16. Percentage of AR tracking methods (a) and distribution of papers according to tracking method per year (b).
Figure 16. Percentage of AR tracking methods (a) and distribution of papers according to tracking method per year (b).
Applsci 10 04259 g016
Applsci 10 04259 g017 550
Figure 17. Distribution of benefits according to the selected articles.
Figure 17. Distribution of benefits according to the selected articles.
Applsci 10 04259 g017
Table
Table 1. List of review papers.
Table 1. List of review papers.
ReferenceYear of PublicationTime SpanDatabasesNo. of ArticlesArticles in Automotive [%]
[30]20162010–2016Scopus466100%
[21]20192006–2017Scopus, Web of Science, Ebsco1746%
[32]2016N/AN/AN/A100%
[1]20152001–2015N/A3924%
[22]20141992–2014N/A57100%
[4]2018N/AGoogle Scholar, EbscoN/A100%
[31]20171987–2016N/AN/A100%
[25]19991993–1998N/AN/AN/A
[7]20191994–2019N/A112N/A
[27]20141960–2014Scopus, Science Direct, Google Scholar147N/A
[26]20121990–2011N/A187N/A
[23]20191994–2016ACM digital library, Science Direct, Scopus, Web of Science44100%
[24]20052001–2004N/A10N/A
[28]20172005–2016Scopus, Google Scholar165N/A
[29]20192014–2019IEEE Xplore, Science Direct, Scopus, Google Scholar, Web of Science, Engineering Village4941%
Table
Table 2. General characteristics of AR application that are related to driving.
Table 2. General characteristics of AR application that are related to driving.
ReferenceYear of PublicationDeviceDisplayed InformationTypeAssets
[55]2015HUDvehicle detection and traffic sign warnings, speed alertsimulator3D graphics
[61]2018HUDN/Areal3D graphics, text
[47]2019Optical see-through HUDtraffic sign warning, intersection guidance, speed alertsimulator2D and 3D graphics
[58]2012HUDvehicle detectionsimulatorvideo streaming
[62]20153D HUDN/Asimulator3D graphics
[57]2016Transparent windshield displaypedestrian detectionsimulatortext, 2D graphics
[67]2018Volumetric HUDpedestrian detection, time to collisionreal2D graphics
[54]2015Windshield displayintersection guidancesimulator2D graphics
[50]2018HMDN/Asimulator2D graphics
[49]2018HUDintersection guidance, pedestrian detection, time to collisionreal2D graphics, animated graphics
[63]2010N/AN/Areal3D model
[56]2011N/Apedestrian detectionreal 2D graphics
[36]2015Contact analog HUDintersection guidancesimulator3D graphics
[64]2017HUD designed in-houseN/Asimulator2D graphics
[59]2014HUDN/AN/A2D graphics
[65]2019N/AN/Areal 2D graphics
[52]2017HUDpedestrian detectionsimulator2D graphics, audio
[84]2018HUDpedestrian detectionsimulator2D graphics
[53]2019HUDvehicle detectionsimulator2D graphics
Table
Table 3. General characteristics of AR application that are not related to driving.
Table 3. General characteristics of AR application that are not related to driving.
ReferenceYear of PublicationDeviceTracking MethodSoftwareTypeAssets
[72]2005N/AN/AN/AN/AN/A
[73]2009N/AN/AN/AN/AN/A
[77]2018smartphonemarker-basedVuforiareal2D and 3D graphics, animation
[16]2017projectormarkerlessself-developedreal2D graphics
[78]2017tabletmarkerlessVuforiareal3D graphics, text
[66]2019HoloLens glassesmarkerlessVuforiareal2D graphics
[17]2017tabletmarkerlessOpenCVreal3D model, 2D graphics
[82]2013N/Amarker-basedARToolKitreal3D model
[67]2013AR glassesmarker-basedVuforiarealtext, 2D graphics
[69]2016AR glassesmarker-basedVuforiareal3D graphics
[83]2011projectormarkerlessself-developedreal2D graphics
[70]2018AR glassesmarker-basedVuforiareal text, 3D model
[79]2012HMDmarkerlessself-developedrealvideo, audio input
[81]2014smartphonemarker-based NyARToolkitsimulation3D model
[74]2018Tobbi glasses 2markerlessN/Areal3D model, 3D graphics
[75]2011HMDmarkelessself-developedsimulation3D graphics
[80]2002Cy-Visor glassesmarker-basedARToolkitsimulation3D models
[68]2003HMDmarker-basedN/Asimulation3D model
[76]2015HMDmarker-basedARToolKitsimulation3D model
[51]2018N/Amarker-basedN/AN/Atext, 2D graphics
[71]2009projectormarkerlessself-developedsimulation3D model
Table
Table 4. List of user studies.
Table 4. List of user studies.
ReferenceType of InvestigationStatistical AnalysisNo. of ParticipantsAge RangeGender Ratio
[72]interviewbi- and multivariate 1120–5111:00
[73]interviewbi- and multivariate 23, 5220–53, 30–5323:00
[77]questionnaire−10N/AN/A
[78]SUS1 questionnaire−21, 2217–18N/A
[66]questionnaire ANOVA2418–5420:04
[47]NASA-TLXANOVA22mean: 20.313:09
[58]log files−5N/AN/A
[62]stereogram test−2422–5319:05
[54]interviews, NASA-TLXANOVA24,3319–41, 65–91N/A
[48]interviewANOVA16mean: 42N/A
[49]questionnaire ANOVA2418–4017:07
[63]video analysis−1415–5311:03
[36]questionnnaireANOVA3022–5223:07
[64]SPSS software−1223–53N/A
[52]AttrakDiff questionnaireANOVA8820–5470:18:00
[53,84]AttrakDiff questionnaireANOVA8022–5570:10:00
[73]TAM questionnaire ANOVA26, 1819–35, 19–4115:11, 12:6
1 System Usability Scale.

Share and Cite

MDPI and ACS Style

Boboc, R.G.; Gîrbacia, F.; Butilă, E.V. The Application of Augmented Reality in the Automotive Industry: A Systematic Literature Review. Appl. Sci. 2020, 10, 4259. https://doi.org/10.3390/app10124259

AMA Style

Boboc RG, Gîrbacia F, Butilă EV. The Application of Augmented Reality in the Automotive Industry: A Systematic Literature Review. Applied Sciences. 2020; 10(12):4259. https://doi.org/10.3390/app10124259

Chicago/Turabian Style

Boboc, Răzvan Gabriel, Florin Gîrbacia, and Eugen Valentin Butilă. 2020. "The Application of Augmented Reality in the Automotive Industry: A Systematic Literature Review" Applied Sciences 10, no. 12: 4259. https://doi.org/10.3390/app10124259

APA Style

Boboc, R. G., Gîrbacia, F., & Butilă, E. V. (2020). The Application of Augmented Reality in the Automotive Industry: A Systematic Literature Review. Applied Sciences, 10(12), 4259. https://doi.org/10.3390/app10124259

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop