Next Article in Journal
Epistemic Network Analyses of Economics Students’ Graph Understanding: An Eye-Tracking Study
Next Article in Special Issue
A Sensor Fusion Based Nonholonomic Wheeled Mobile Robot for Tracking Control
Previous Article in Journal
The Application of Prussian Blue Nanoparticles in Tumor Diagnosis and Treatment
Previous Article in Special Issue
A Vision-Based Driver Assistance System with Forward Collision and Overtaking Detection
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 20
Issue 23
10.3390/s20236907
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Think Aloud Protocol Applied in Naturalistic Driving for Driving Rules Generation

by
Borja Monsalve
1,*,
Nourdine Aliane
2,
Enrique Puertas
1 and
Javier Fernández Andrés
2
1
Science, Computing and Technology Department, Universidad Europea de Madrid, 28670 Madrid, Spain
2
Industrial and Aerospace Engineering Department, Universidad Europea de Madrid, 28670 Madrid, Spain
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(23), 6907; https://doi.org/10.3390/s20236907
Submission received: 29 October 2020 / Revised: 25 November 2020 / Accepted: 1 December 2020 / Published: 3 December 2020
(This article belongs to the Special Issue Intelligent Sensing Systems for Vehicle)
Browse Figures
Versions Notes

Abstract

:
Understanding naturalistic driving in complex scenarios is an important step towards autonomous driving, and several approaches have been adopted for modeling driver’s behaviors. This paper presents the methodology known as “Think Aloud Protocol” to model driving. This methodology is a data-gathering technique in which drivers are asked to verbalize their thoughts as they are driving which are then recorded, and the ensuing analysis of the audios and videos permits to derive driving rules. The goal of this paper is to show how think aloud methodology is applied in the naturalistic driving area, and to demonstrate the validity of the proposed approach to derive driving rules. The paper presents, firstly, the background of the think aloud methodology and then presents the application of this methodology to driving in roundabouts. The general deployment of this methodology consists of several stages: driver preparation, data collection, audio and video processing, generation of coded transcript files, and the generation of driving rules. The main finding of this study is that think aloud protocol can be applied to naturalistic driving, and even some potential limitations as discussed in the paper, the presented methodology is a relatively easy approach to derive driving rules.

1. Introduction

Autonomous driving has been a very active field of research in recent years [1,2]. According to the Automated and Autonomous Driving of the National Highway Traffic Administration (NHTSA), the industry is already far from the fifth level of “full automation”, where vehicles are capable of performing all the driving functions under all conditions [3]. One of the main benefits offered by autonomous cars is the safety. According to the World Health Organization (WHO), about 1.3 million people die on the roads around the World, and about 20–50 million are injured every year [4]. According to NHTSA, 94% of serious car accidents are due to human error, so autonomous vehicles would have the potential to remove this factor from the equation. This would increase the safety for the drivers, passengers, pedestrians, and other vehicles on the road. Other studies like [5] also reveal the economic and societal benefits of this industry, without mentioning the productivity, pollution, and quality of life problems derived from traffic jams in the cities.
Different techniques have been used to model car behavior. Most studies have been based on simulated environments [6,7,8] or instrumented cars [9]. The first group allow researchers to reproduce concrete situations on demand, and in the second case the behavior attends to real environment. Driver models have also been developed to analyze many aspects of the driving process. Initial studies focused on issues related to low-level control behavior, like curve negotiation [10] or steering control [11]. Later, these techniques were combined with other high-level models to develop integrated driver models like [12] or [13], where a cognitive architecture is proposed to model the driver behavior.
Other methodologies used to analyze and model driver’s behavior are based on naturalistic driving, where data related to the car, the driver, and the environment of the route are gathered [14] for later analysis. These kinds of studies have been mainly used related to driving safety, for example for collision warning [15], safety events related to weather conditions [16], or for measuring time to collision at braking depending on the age and gender of the driver [17]. To be able to create a true-life model, these studies configure a nonintrusive driving environment to gather data during well-known routes for the driver [18]. There is no supervision during the process, so the subject receives no in-situ instructions that might affect his or her usual behavior. Compared to more traditional strategies, naturalistic driving studies have several advantages [19]. One of the most important is the opportunity to understand a driver’s real behavior more broadly, as they are based on real situations during daily routes. The main disadvantage is just that lack of control over the experiment itself, as a result of this absence of interventionism.
Another promising field of research to model driving is related to the use of think aloud protocol. This technique has been widely used in many fields to analyze problem solving, such as in thinking study [20], in translation research [21], or user interface design [22]. This methodology is also used in the field of autonomous driving. For example, in [23] the thinking aloud protocol is used to obtain deeper insight into the interaction of drivers with semiautonomous vehicles (level-2 automation) measuring the degree acceptance of users of different age groups. The same methodology is used in [24], as part of the knowledge extraction process of a system, to train drivers how to interact with semiautonomous vehicles. In [25], this methodology is also used to better understand the anticipation capabilities of drivers, with the aim of increasing safety in any situation in autonomous environments. However, there does not seem to be previous work in which the think aloud protocol is used for deriving driving rules in specific traffic scenarios.
In this study, the use of the think aloud protocol is proposed aimed at extracting rules modeling the driver’s behavior following naturalistic driving, where the main task of the driver consists in the verbalization of his/her thoughts that occurred during the driving task. Afterwards, the recorded talks will be analyzed with the goal of extracting driving rules.
The rest of the paper is organized as follows: first the think aloud protocol is explained, and how this technique is applied within naturalistic driving context to model driver behavior. The next section presents the experimental environment, the data collection process, as well as the management of this information. Afterwards, the cognitive process applied to data extraction and analysis as well as the derivation of driving rules are explained. The following section is dedicated to maneuvers within roundabouts. Finally, conclusions and future work are drawn in the last section.

2. What Is the Think Aloud Protocol?

When a person is going to carry out a cognitive process or carry out an action, the cognitive system collects the information through the senses (sight, hearing, touch, etc.) and deposits that information in the working memory, also known as short term memory. Then the brain collects the information from working memory, processes it, and executes the next action to take. This action is also stored in the working memory before being transferred to the motor system that will execute the action, such as moving the neck to look to the side. Therefore, if we could know the content of working memory, we can know which inputs and outputs are involved in a cognitive process when performing a certain action.
The think aloud protocol is based on the theory that people are able to verbalize the linguistic content of working memory, and its goal is to capture what the content of a person’s working memory is when they perform a task or solve a problem.
Think aloud protocol is a methodology where participants are asked to talk about the task they are working at [26], saying out loud what they are looking at, thinking about and doing, in order to extract the content of the working memory, to capture inputs and outputs involved in a task, and aiming to create a model that explains the behavior of the subject. This strategy was proposed by Newel and Simon in 1972 [27] to generate reasoning models based on human behavior. The thoughts expressed following this technique are recorded, so later can be transcribed and then analyzed following a set of preconfigured categories depending on the domain and the kind of tasks of the study. Think aloud protocol is based on the idea that people are able to verbalize the linguistic content stored in their working memory, which can be expressed with words. This strategy has been used to analyze problem solving activities in many fields, such as cognitive psychology [20], translation processes [21,28,29], to analyze the writing process [30] or user interface and evaluation of usability in computer science [22,31], among others. For example, in [32], researchers present the evaluation of a computer system design based on think aloud protocol. For this purpose, they made a selection of nine different system designers attending to a variety of roles. All of them had previous experience in the think aloud method, and faced different tests of software application. The interviews were recorded and later fully transcribed, to be analyzed in detail after that. This study showed the value of the feedback obtained using this technique, that helped designers to develop better systems. Figure 1 shows the steps involved in the think aloud protocol.
The methodology has two main stages: namely data collection, and model building and analysis. Furthermore, as suggested by Ericson and Simon [33], the first stage is divided into three levels.
In level 1 reports, direct vocalization is produced as a result of the verbalization of the ideas stored in the subject’s short-term memory. These kinds of reports are generated by simple instructions that only imply thinking aloud and demand no extra effort by the subject. These types of protocols are only useful when the data stored in the working memory is mostly linguistic information. Most of the concepts belong to this first level, where perceptions such as smell or taste are difficult to verbalize. Thus, a translation of these abstract concepts to linguistic expressions are needed. Unfortunately, the concepts that drivers observe and the actions they execute do not always belong to linguistic information, and their representation in memory is not directly verbalizable, such as traffic signs, vehicle indicators, etc. In such case, subjects will have to think about the necessary vocabulary to represent the concept clearly. This will take place during driving activities.
Level 2 takes place when the task requires processing extra details. To verbalize them, the subject needs to manage descriptions about the content stored in his working memory, but without searching into his long-term memory. The time necessary to develop the task used to be longer although cognitive process is not affected.
Finally, in level 3, the task demands conscient control. The cognitive process needs to work with data that cannot be found in the working memory, so it requires different introspection grades. This tasks require more time to be solved, but the “overload” introduced by the speech does not modify the cognitive process itself [33].
In this work, the combination of the think aloud protocol with naturalistic driving to extract driving rules is proposed. The idea consists of analyzing the records of real driving as well as other data derived from naturalistic driving. Furthermore, the study is focused on a roundabout driving scenario, where the driving task involves carrying out several steps:
  • Perceive the environment.
  • Decision making after a cognitive process.
  • Execute the planned action.
  • Modify the environment.
The driver perceives the environment to extract data that supports the decisions to be taken. These new actions executed, such as accelerating, braking, etc., lead to changes in the vehicle’s states, and so also does the environment, which generates a new cycle as shown in Figure 2.

3. Experimental Setting and Data Collection

The data needed for the present study were mainly audio of driver’s verbalization as well as the videos of the driving scenes. The basic instrumentation used in the data collection process consists of a glasses mounted camera (as shown in Figure 3), and smartphone mounted on the vehicle dashboard for recording simultaneously two additional videos: the front view of the vehicle for saving the driving scenes and the rear view for recording the driver actions.
The next step consists of generating transcript text files from the videos and audios. The text was then analyzed along with the video images. The actions indicated by the driver were coded. In many cases, due to the overload of stimuli and actions, evident actions were added to the transcript files, based on the video contents, and sometimes by guessing the driver intentions because the corresponding actions verbalizations were omitted. However, for the transcript files and their subsequent analysis to be useful, verbalizations should be effective.
The steps of data acquisition and transcript text analysis processes are summarized in Figure 4.
To ensure data recording close to naturalistic driving as possible and avoiding undesirable behaviors, the driver was prepared with a prior training for adhering to a protocol for a reliable and effective verbalization. The training stage consisted of a few routes where the driver was assisted by the research group to help him to express aloud what he was perceiving and what he was executing. It is worth to point out that, from the analyzer point of view, driver’s emotions or opinions were not targeted. Only the verbalization of the perceptions and actions related to driving were taken into account. Some example could be, for instance, acceleration, brake, “I look in the rearview mirror”, “I see a car approaching quickly from the left”, etc. The issues related to concurrent actions were solved by verbalizing those using short sequential sentences. After some journeys, the driver reached to put the focus on his driving.

4. Cognitive Process

Cognitive process in the think aloud protocol refers to the process of data extraction and the subsequent analysis for building and validating a specific model. As far as the think aloud protocol applied to driver modeling is concerned, its main goal consists in extracting a set of driving rules to be implemented in an autonomous vehicle. The process focuses on the observation and knowledge extraction of how the subject acts when facing the different actions involved when trying to solve a concrete task.
Considering the Stop maneuver as an example, three main steps could be identified:
  • Stop the vehicle.
  • Analyze the circulation.
  • Resume the route.
Depending on the goal of the process, the abstraction level of the steps could be required to be more detailed, when performing the previously mentioned maneuver. In this maneuver, the vehicle’s direction and speed were taken as input data as well as the direction by which we wanted to continue after making the stop. Additionally, step 2 “Analyze the circulation” could be detailed into several subtasks depending on the exit direction:
  • Check traffic coming from the left.
  • Check traffic coming from the right.
  • Check traffic coming from the front.
The transcript text files processing was performed following Ronald Hamel’s approach described in [34] by creating a set of labels for categorizing the actions with two main input stimuli, external sounds and external visual information (labeled as Sight and Sound), that the driver needs to process driving. On the actuation side, a normal driver uses his arms and legs as actuators to manipulate the controls of the vehicle and modify its state. According to this, a set of possible actions to perform were defined, based on the actuators or sensors involved in the task.
Furthermore, eight possible directions from where visual stimuli are captured were defined (see Figure 5). Concerning external sounds, they were considered two cases according to the location of the sound source: inside and outside the vehicle. Finally, a table coding the different situations was created. For example, if the driver was looking diagonally between the front and the left according to the travel direction (Front-Left view), it was coded as “FLV”. The codes corresponding to sound and sight stimuli and their descriptions are given in Table 1.
The codification of the different actions was organized in two categories linked to hands and feet. The main actions to be taken into account when a vehicle undertakes a roundabout are hands for steering wheel, the gear shift lever, or the blinkers, and feet for acting on the pedals. This set of actions and their codes were determined by the kind of vehicle used in the experiments, such as a car with manual or automatic gearbox. Table 2 gives the codes of hands and feet actions for a manual gear car.

5. Case of a Roundabout Maneuver

According to the traffic and road safety regulations in Spain [35], a roundabout represents an intersection in the road where different sections connect around a central island (see Figure 6). In Spain, vehicles approaching roundabouts do not have preference over those being already inside them. Thus, the predefined right-side preference rule changes in this situation.
The study carried out in this article was limited to modeling driving in roundabouts, since they are considered as a complex traffic scenario [36,37,38]. To capture the essence of maneuvers within roundabouts, and from driver attention point of view, according to the approach applied in [39], roundabouts are divided into three sections: approaching roundabouts, driving inside applying the priority rules, and leaving them.
To validate the proposed methodology, the participation of one driver was enough to carry out the experiments. In this case, the participant for recording the audios and the videos recording was a 44-year-old male driver with 25 years driving experience, and he used his own manual gearbox vehicle.
On the other hand, in order to avoid prolonged stress while driving, journeys of 10–15 min driving were chosen. The same driver, using his own car, drove along a route with five roundabouts 10 times. A total of 50 audio recordings and videos were obtained. Finally, the frames corresponding to roundabouts stretches were extracted for their subsequent analysis.
Afterwards, these new videos were edited and composed so that their visualization would be straightforward and effective, and were automatically processed with dictation software for creating a first transcript version, which was supervised to fix errors if it was needed. Finally, the transcription of the audio was then stored in a text file.
Later, some additional information were added to the generated transcription files, such as defining labels for the start and the end of roundabouts, their durations as well as including labels corresponding to actions verbalized by the driver, using the codes defined in Table 1 and Table 2. Evident actions shown in the video but omitted by the driver were also included. Labeled transcription was then analyzed to study the input and output of each situation, to generate pseudocode rules that can explain the behavior of the driver when facing a roundabout. As an example, a transcription fragment of a maneuver in the roundabout (see Figure 7) is shown in Table 3. The different actions verbalized by the driver, but also those raised from the analysis of the videos are indicated in the text. Figure 8 shows some video frames of the roundabout of the example taken with the subjective camera.
After the analysis of these labeled transcriptions files, the steps followed by the driver during the maneuver were converted to pseudocode. Following is an example of a pseudocode generated from the transcription files of Table 3 of the previous example.
IF approaching_roundabout THEN
  reduce speed
  WHILE other_vehicle_inside_roundabout THEN
    Stop
  speed(increase)
  speed(hold)
  direction(right)
  blinker(left)
  direction(left)
  blinker(right)
  direction(right)
  speed(increase)
These rules were generated in a supervised process by the researchers. As they were specified at high level, is easy to understand for a human being, and this makes them useful to analyze the cognitive process of the studied task. However, in order to be incorporated into an autonomous driving vehicle, they will require extra processing to make them more specific, depending on the detail required by the system. The resulting rules are strongly dependent on the driver behavior and the way he or she performs the studied roundabouts. Thus, veteran or a more aggressive driver could help us to achieve different rules from those derived form a novice or a more cautious one. They are also dependent on very subjective concepts or perceptions, such as what can be understood by “close” or “fast” when executing the maneuver, regardless of what may be marked by traffic rules or road signs. The rules can be easily validated and adjusted using simulation systems, starting with a reproduction of the roundabouts used for the experiments, and later using new ones to check if they can be generalized to other roundabout types and traffic situations.
This study, however, presents some limitations. For example, the generated driving rules do not consider possible incidents that might occur. However, these kinds of situations may be handled including exceptions management in the generated rules. Another limitation is related to the analysis that is supervised in many stages. This issue can be improved by automating some tasks by the use of speech-to-text translators and the use of automatic transcription processing techniques. Although the paper is focused on roundabouts, the study may be extended to other maneuvers, such as incorporation to highways, intersections, etc. Finally, driving rules derived from think aloud protocol can be assessment using some kind of virtual simulator.

6. Conclusions and Future Work

In this paper the think aloud protocol technique in a naturalistic driving environment to model driver behavior was presented, which consists of the driver verbalizing all the performed actions while driving. This work is limited to the study of maneuvers at roundabouts.
The deployment of this technique requires an initial training, permitting drivers to act according to a predefined protocol. This is necessary to analyze and interpret the driving actions properly.
The data of the experiments, obtained mainly from video and audio recordings, were obtained during several trips but carried out by the same driver. Firstly, processing consists of isolating the driving to roundabouts segments, which were then divided into three parts: approaching, inside, and exiting sections. Afterwards, each section was analyzed and coded using a set of predefined actions. The result was a set of enriched texts combining transcriptions and coded actions. These enriched texts were then analyzed for deriving pseudocode rules corresponding to driving.
As future lines of work, and with the aim of minimizing the supervised stages of the process of obtaining driving rules, the first idea consists of using speech-to-text translators and using automatic transcription processing techniques for the detection of keywords related to in driving maneuvers. The second idea deals with obtaining more driving rules using other traffic scenarios, such as incorporation to highways, intersections, etc., and then performing a first assessment of the derived driving rules in the virtual simulator.

Author Contributions

B.M. was responsible for the data collection, the data processing, the rule generation and the paper drafting. N.A. was responsible for the rule generation and the paper drafting. E.P. was responsible for the work related with the think aloud cognitive process. J.F.A. was responsible for the collected data pre-processing and validation. All authors contributed to the design of the experimental setting and writing and reviewing the final manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported in part by the National Plan for Research PN I+D+i under TRA2016-78886-C3-2-R, PID2019-104793RB-C32, and the Comunidad de Madrid through SEGVAUTO-4.0-CM (P2018/EMT-4362) grants.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ozguner, U.; Redmill, K.A.; Broggi, A. Team TerraMax and the DARPA grand challenge: A general overview. In Proceedings of the IEEE Intelligent Vehicles Symposium, Parma, Italy, 14–17 June 2004. [Google Scholar]
  2. Thrun, S.; Montemerlo, M.; Dahlkamp, H.; Stavens, D.; Aron, A.; Diebel, J.; Fong, P.; Gale, J.; Halpenny, M.; Hoffmann, G.; et al. Stanley: The robot that won the DARPA Grand Challenge. In The 2005 DARPA Grand Challenge: The Great Robot Race; Springer Tracts in Advanced Robotics; Springer: Berlin/Heidelberg, Germany, 2007; Volume 36, pp. 1–43. [Google Scholar]
  3. Automated Driving Systems (ADS): A Vision for Safety 2.0. 2017. Available online: https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/documents/13069a-ads2.0_090617_v9a_tag.pdf (accessed on 1 December 2020).
  4. World Health Organization. 10 Facts about Road Safety. 2018. Available online: https://www.who.int/news-room/facts-in-pictures/detail/road-safety (accessed on 1 December 2020).
  5. Li, X.; Chimba, D.; Masindoki, E. The Economic and Societal Impact of Motor Vehicle Crashes on Freight Intermodal Connectors in Tennessee: A Risk Management Approach. In Proceedings of the Transportation Research Board Annual Meeting, Washington, DC, USA, 7–11 January 2018. [Google Scholar]
  6. Blana, E. A Survey of Driving Research Simulators around the World; Institute of Transport Studies, University of Leeds: Leeds, UK, 1996. [Google Scholar]
  7. Kaptein, N.A.; Theeuwes, J.; Van Der Horst, R. Driving Simulator Validity: Some Considerations. Transp. Res. Rec. J. Transp. Res. Board 1996, 1550, 30–36. [Google Scholar] [CrossRef]
  8. Shechtman, O.; Classen, S.; Awadzi, K.; Mann, W. Comparison of driving errors between on-the-road and simulated driving assessment: A validation study. Traffic. Inj. Prev. 2009, 10, 379–385. [Google Scholar] [CrossRef] [PubMed]
  9. Helander, M.; Hagvall, B. An instrumented vehicle for studies of driver behaviour. Accid. Anal. Prev. 1976, 8, 271–277. [Google Scholar] [CrossRef]
  10. Boer, E.R. Tangent point oriented curve negotiation. In Proceedings of the Conference on Intelligent Vehicles, Tokyo, Japan, 19–20 September 1996. [Google Scholar]
  11. Hess, R.A.; Modjtahedzadeh, A. A Control Theoretic Model of Driver Steering Behavior. IEEE Control Syst. Mag. 1990, 10, 3–8. [Google Scholar] [CrossRef]
  12. Levison, W.H.; Cramer, N.L. Description of the Integrated Driver Model; (Tech. Rep. No. FHWA-RD-94-092). Federal Highway Administration, Office of Research and Development, McLean VA Remarks: Performed by BBN Systems and Technologies for the Federal Highway Administration, Office of Safety and Traffic Operations. 1995. Available online: https://trid.trb.org/view/490162 (accessed on 1 December 2020).
  13. Salvucci, D.D.; Boer, E.R.; Liu, A. Toward an integrated model of driver behavior in cognitive architecture. Transp. Res. Record 2001, 1779, 9–16. [Google Scholar] [CrossRef] [Green Version]
  14. Barnard, Y.; Utesch, F.; van Nes, N.; Eenink, R.; Baumann, M. The study design of UDRIVE: The naturalistic driving study across Europe for cars, trucks and scooters. Eur. Transp. Res. Rev. 2016, 8, 14. [Google Scholar] [CrossRef] [Green Version]
  15. Tawfeek, M.H.; El-Basyouny, K. A perceptual forward collision warning model using naturalistic driving data. Can. J. Civ. Eng. 2018, 45, 899–907. [Google Scholar] [CrossRef]
  16. Ali, E.M.; Ahmed, M.M.; Wulff, S.S. Detection of critical safety events on freeways in clear and rainy weather using SHRP2 naturalistic driving data: Parametric and non-parametric techniques. Saf. Sci. 2019, 119, 141–149. [Google Scholar] [CrossRef]
  17. Chen, R.; Kusano, K.D.; Gabler, H.C. Age and Gender Difference in Braking Behavior from the 100-Car Naturalistic Driving Study: The Implication for Autonomous Braking System Design. In Proceedings of the FAST-Zero’15: 3rd International Symposium on Future Active Safety Technology toward Zero Traffic Accidents, Gothenburg, Sweden, 9–11 September 2015. [Google Scholar]
  18. Monsalve, B.; Puertas, E.; Fernandez, J.; Aliane, N. Using wearable devices in naturalistic driving to analyze brain activity in roundabout maneuvers. In Proceedings of the 2018 IEEE International Conference on Vehicular Electronics and Safety (ICVES), Madrid, Spain, 12–14 September 2018. [Google Scholar]
  19. Van Schagen, I.; Sagberg, F. The Potential Benefits of Naturalistic Driving for Road Safety Research: Theoretical and Empirical Considerations and Challenges for the Future. Procedia-Soc. Behav. Sci. 2012, 48, 692–701. [Google Scholar] [CrossRef] [Green Version]
  20. Ericsson, K.A.; Simon, H.A. How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking. Mind Cult. Cult. Act. 1998, 5, 178–186. [Google Scholar] [CrossRef]
  21. Bernardini, S. Think-aloud protocols in translation research: Achievements, limits, future prospects. Target 2001, 13, 241–263. [Google Scholar] [CrossRef]
  22. Jaspers, M.W.M.; Steen, T.; Van Den Bos, C.; Geenen, M. The think aloud method: A guide to user interface design. Int. J. Med. Inform. 2004, 73, 781–795. [Google Scholar] [CrossRef] [PubMed]
  23. Neuhuber, N.; Lechner, G.; Kalayci, T.E.; Stocker, A.; Kubicek, B. Age-Related Differences in the Interaction with Advanced Driver Assistance Systems—A Field Study. In International Conference on Human-Computer Interaction, Proceedings of the HCII 2020: HCI in Mobility, Transport, and Automotive Systems. Automated Driving and In-Vehicle Experience Design, Copenhagen, Denmark, 19–24 July 2020; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2020. [Google Scholar]
  24. Krampell, M.; Solís-Marcos, I.; Hjälmdahl, M. Driving automation state-of-mind: Using training to instigate rapid mental model development. Appl. Ergon. 2020, 83, 102986. [Google Scholar] [CrossRef] [PubMed]
  25. Grahn, H.; Kujala, T.; Silvennoinen, J.; Leppänen, A.; Saariluoma, P. Expert Drivers’ Prospective Thinking-Aloud to Enhance Automated Driving Technologies—Investigating Uncertainty and Anticipation in Traffic. Accid. Anal. Prev. 2020, 146, 105717. [Google Scholar] [CrossRef]
  26. Lewis, C. Using the ‘Thinking Aloud’ Method in Cognitive Interface Design; Research Report, RC-9265; IBM TJ Watson Research Center: Yorktown Heights, NY, USA, 1982. [Google Scholar]
  27. Newell, A.; Simon, H.A. Human Problem Solving; Carnegie-Mellon University: Pittsburgh, PA, USA; Prentice-Hall: Englewood Cliffs, NJ, USA, 1972. [Google Scholar]
  28. Li, D. Trustworthiness of think-aloud protocols in the study of translation processes. Int. J. Appl. Linguist. 2004, 14, 301–313. [Google Scholar] [CrossRef]
  29. Olmsted-Hawala, E.L.; Murphy, E.D.; Hawala, S.; Ashenfelter, K.T. Think-aloud protocols: A comparison of three think-aloud protocols for use in testing data-dissemination web sites for usability. In Proceedings of the Conference on Human Factors in Computing Systems, Atlanta, GA, USA, 10–15 April 2010. [Google Scholar]
  30. Armengol, L. Los protocolos de pensamiento en voz alta como instrumento para analizar el proceso de escritura. Rev. Española Lingüística Apl. 2007, 20, 27–35. [Google Scholar]
  31. Van Den Haak, M.; De Jong, M.; Jan Schellens, P. Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online library catalogue. Behav. Inf. Technol. 2003, 22, 339–351. [Google Scholar] [CrossRef]
  32. Jørgensen, A.H. Thinking-aloud in user interface design: A method promoting cognitive ergonomics. Ergonomics 1990, 33, 501–507. [Google Scholar] [CrossRef]
  33. Ericsson, K.A.; Simon, H.A. Protocol Analysis: Verbal Reports as Data; MIT Press: Cambridge, MA, USA, 1993. [Google Scholar]
  34. Van Someren, M.W.; Barnard, Y.F.; Sandberg, J.A.C. The Think aloud Method: A Practical Approach to Modelling Cognitive Processes; Academic Press: Amsterdam, The Netherlands; London, UK, 1994. [Google Scholar]
  35. Código de Tráfico y Seguridad Vial. Available online: http://www.dgt.es/images/BOE-020_Codigo_de_Trafico_y_Seguridad_Vial.pdf (accessed on 1 December 2020).
  36. Rossi, R.; Meneguzzer, C.; Orsini, F.; Gastaldi, M. Gap-acceptance behavior at roundabouts: Validation of a driving simulator environment using field observations. Transp. Res. Procedia 2020, 47, 27–34. [Google Scholar] [CrossRef]
  37. González, D.; Pérez, J.; Milanés, V. Parametric-based path generation for automated vehicles at roundabouts. Expert Syst. Appl. 2017, 71, 332–341. [Google Scholar] [CrossRef] [Green Version]
  38. Anagnostopoulos, A.; Kehagia, F. CAVs and roundabouts: Research on traffic impacts and design elements. Transp. Res. Procedia 2020, 49, 83–94. [Google Scholar] [CrossRef]
  39. Cuenca, L.G.; Sanchez-Soriano, J.; Puertas, E.; Aliane, N.; Andrés, J.F. Machine learning techniques for undertaking roundabouts in autonomous driving. Sensors 2019, 19, 2386. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Sensors 20 06907 g001 550
Figure 1. Main steps in the think aloud method.
Figure 1. Main steps in the think aloud method.
Sensors 20 06907 g001
Sensors 20 06907 g002 550
Figure 2. Driving behavior.
Figure 2. Driving behavior.
Sensors 20 06907 g002
Sensors 20 06907 g003 550
Figure 3. Portable camera.
Figure 3. Portable camera.
Sensors 20 06907 g003
Sensors 20 06907 g004 550
Figure 4. Data acquisition and analysis process.
Figure 4. Data acquisition and analysis process.
Sensors 20 06907 g004
Sensors 20 06907 g005 550
Figure 5. Direction of input stimuli.
Figure 5. Direction of input stimuli.
Sensors 20 06907 g005
Sensors 20 06907 g006 550
Figure 6. Standard roundabout.
Figure 6. Standard roundabout.
Sensors 20 06907 g006
Sensors 20 06907 g007 550
Figure 7. Roundabout example.
Figure 7. Roundabout example.
Sensors 20 06907 g007
Sensors 20 06907 g008 550
Figure 8. Three frames of each section of the maneuver: (a) approaching, (b) inside, and (c) exiting the roundabout.
Figure 8. Three frames of each section of the maneuver: (a) approaching, (b) inside, and (c) exiting the roundabout.
Sensors 20 06907 g008
Table
Table 1. Codes of input stimuli.
Table 1. Codes of input stimuli.
CategoryActionCodeDescription
SightFront viewFVLook forward in the driving direction
SightFront view to the rearview mirrorFV-MirrorLook through the interior rearview mirror
SightLeft viewLVLook left
SightLeft mirrorLV-MirrorLook left through the left external mirror
SightFront-Left viewFLVLook diagonally, between front and left
SightRight viewRVLook right
SightRight mirrorRV-MirrorLook left through the right external mirror
SightFront-Right viewFRVLook diagonally, between front and right
SightBack viewBVLook turning head back.
SightBack-Left view BLVLook back diagonally between left and rear of the vehicle.
SightBack-Right viewBRVLook back diagonally between right and rear of the vehicle.
SoundInside soundIN-SSound from inside the vehicle
SoundOutside soundOUT-SSound coming from outside the vehicle
Table
Table 2. Codes of hands and feet actions for a manual gear car.
Table 2. Codes of hands and feet actions for a manual gear car.
CategoryActionCodeDescription
HandsTurning steering wheel leftSW-TL-LTurn the steering wheel to the left to change direction to the left
HandsTurning steering wheel leftSW-TL-STurn the steering wheel to the left to straighten the car and continue straight
HandsTurning steering wheel rightSW-TR-RTurn the steering wheel to the right to change direction to the right
HandsTurning steering wheel rightSW-TR-STurn the steering wheel to the right to straighten the car and continue straight
HandsGear downGDReduce N gears to decrease speed
HandsGear upGUIncrease N gears to increase speed
HandsLeft blinker light onLB-ONUse the turn signal switch to turn the left light blinker on
HandsLeft blinker light offLB-OFFUse the turn signal switch to turn the left light blinker off
HandsRight blinker light onRB-ONUse the turn signal switch to turn the left right blinker on
HandsRight blinker light offRB-OFFUse the turn signal switch to turn the left right blinker off
FeetActivate clutch pedalG-ONActivate the clutch pedal to initiate gear changing
FeetRelease clutch pedalG-OFFRelease the clutch pedal to finish gear changing
FeetPush throttle pedalT-ONPush the throttle further to increase speed
FeetHold throttle pedalT-HOLDKeep throttle pedal pressed to maintain stable speed
FeetRelease throttle pedalT-OFFRelease throttle pedal to deaccelerating the vehicle and reduce speed
FeetPush brake pedalB-ONPush brake pedal to reduce speed. It implies having previously released the accelerator
FeetRelease brake pedalB-OFFRelease the brake pedal to stop braking the car
Table
Table 3. Transcriptions of the three sections of the roundabout.
Table 3. Transcriptions of the three sections of the roundabout.
SectionRoundabout Approaching
Duration20 s
Transcription“I let the car go slowly <T-OFF>, brake a little <B-ON>. I engage the clutch <G-ON>, I go into third gear <GD>, I take my feet off the pedals <G-OFF>, I apply the brakes again a little bit <B-ON>… I keep braking <B-ON> <SW-TR -R> a red car is coming up behind me <FV-MIRROR>. I shift into second gear <G-ON> <GU>, I release the clutch <G-OFF>, brake <B-ON> because here <SW-TL-S> <LV> there was a car at the roundabout.”
SectionInside roundabout
Duration15 s
Transcription“<LV> Now we are going to go through <T-ON> <SW-TR-R> <T-HOLD> and now I turn on the blinker to the left <LB-ON> <SW-TL-L>, now I turn on the blinker to the right <RB-ON>. And I am going to exit here at the <SW-TR-R> roundabout behind the white car.”
SectionRoundabout exit
Duration6 s
Transcription“The blinker turns off <SW-TR-R> <SW-TL-S> <T-ON>, I pass the crosswalk, in that other crosswalk there is no one, so I keep going.”
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Monsalve, B.; Aliane, N.; Puertas, E.; Andrés, J.F. Think Aloud Protocol Applied in Naturalistic Driving for Driving Rules Generation. Sensors 2020, 20, 6907. https://doi.org/10.3390/s20236907

AMA Style

Monsalve B, Aliane N, Puertas E, Andrés JF. Think Aloud Protocol Applied in Naturalistic Driving for Driving Rules Generation. Sensors. 2020; 20(23):6907. https://doi.org/10.3390/s20236907

Chicago/Turabian Style

Monsalve, Borja, Nourdine Aliane, Enrique Puertas, and Javier Fernández Andrés. 2020. "Think Aloud Protocol Applied in Naturalistic Driving for Driving Rules Generation" Sensors 20, no. 23: 6907. https://doi.org/10.3390/s20236907

APA Style

Monsalve, B., Aliane, N., Puertas, E., & Andrés, J. F. (2020). Think Aloud Protocol Applied in Naturalistic Driving for Driving Rules Generation. Sensors, 20(23), 6907. https://doi.org/10.3390/s20236907

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop