Multimodal Warnings in Remote Operation: The Case Study on Remote Driving

Round 1

Reviewer 1 Report

Reviewer Comment 1:

This work conducted a within-subjects experiment in order to analyse how multimodal feedback improves situational awareness in remote operations, and more specifically in remote driving. The subject is interesting, and the experimental procedure and results are sound. There are some minor issues that are listed below. As such, I would recommend this paper for minor revisions.

Reviewer Comment 2:

There are several grammatical errors in the text which must be corrected before publication. For example, in the abstract in line 14, “affect” should be “affects” and in section 3 in line 337, “one drive” should be “one drives”, while in section 4, line 416 “party” should be “partly”. These are only examples, there are other similar mistakes in all sections of the paper.

Reviewer Comment 3:

In section 3.1, pages 5 and 6, the setup of the Buttkicker LFE transducer is performed on a platform, on which the chair has the back two feet placed. How proper is that setup for the accurate simulation of the feedback? If it is not so accurate, do you believe that it affected the perception of the participants as it regards the haptic feedback?

Reviewer Comment 4:

In lines 524-526, “Only peaks during slipping event The value of 0.1 was decided on before any further analysis was done, as a compromise between detection sensitivity and reducing spurious detections” is missing something to make sense.

Reviewer Comment 5:

The order of the reference is not correct. They should be referenced in the order that they appear in. The author should correct that.

Reviewer Comment 6:

I would recommend some papers to enrich the literature, by showcasing possible methods of remote operations with feedback, such as virtual environments, and simulation or expand on the history of remote driving.

• Operators Training Using Simulation And VR Technology. Procedia CIRP, 2021, 96: 290-294.
• An Approach for Validating the Behavior of Autonomous Robots in a Virtual Environment. In: Cooperating Robots for Flexible Manufacturing. Springer, Cham, 2021. p. 133-144.
• Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychonomic bulletin & review, 2013, 20.1: 21-53.
• Novel interfaces for remote driving: gesture, haptic, and PDA. In: Mobile Robots XV and Telemanipulator and Telepresence Technologies VII. International Society for Optics and Photonics, 2001. p. 300-311.

Author Response

Dear Sir/Madam,

Here are our responses to your notes and critique:

• Regarding grammatical errors, we have fixed the typos and/or bad English mentioned in your review, and are currently also going through a vigorous proofreading of the manuscript
• We expanded the description on Buttkicker:  "The haptic feedback is provided by a tactile transducer, Buttkicker LFE, attached to a platform under the participant’s chair. Rubber feet are placed under the platform to reduce audible noise. Additionally, only the back feet of the chair are placed on the platform due to size restrictions and safety concerns. These factors have a damping effect on the haptic feedback, which may be greater at higher frequencies. However, given that the chair’s center of mass is heavily on the side of the platform and our transducer has a low frequency range (5-200Hz), the transmission of vibration is still adequate under these practical constraints."
• Regarding comment 4, we fixed the text: "Only peaks during the first 2 seconds of a slipping event, where the slipping event lasted at least 0.5 seconds, were considered. The value of 0.1 was decided on before any further analysis was done, as a compromise between detection sensitivity and reducing spurious detections. If a peak was found while the vehicle was inside a slippery area, the participant was considered to have reacted by easing the gas pedal."
• Reference order will be fixed so that they correspond to the journal requirements.
• Relevant references were added

Reviewer 2 Report

The article presents a controlled within-subjects experiment conducted to investigate how to improve situational awareness via multimodal feedback (visual, auditory, and haptic) and what is their best combination (modality pair) to effectively communicate warning-type messages to people when driving remotely.

The manuscript is well-organized and the content is clear. Research context and questions are well-presented in the Introduction, as well as the methodology and the results of the experimental study. The overall quality of the article is good. However, it can be improved. The following are some suggestions to this end: 

1. It is rather strange to already find a detailed anticipation of the results obtained, presented later in Section 4, at the end of the Introduction section.
2. Although not strongly significant due to the limited number of participants, it would be interesting to highlight (where possible) any differences by age group, gender, driving experience, etc.
3. The low number of participants should be included among the limitations presented in section 5.3: the study would deserve to be extended to a larger number, in order to better investigate possible differences in terms of age, gender, driving experiences, etc.
4. The Reference section might be extended by adding some references to the Introduction (e.g. where mentioning “existing remote operation solutions”) and to the Related Work section. For example: 
   1. Geitner, C., Biondi, F., Skrypchuk, L., Jennings, P., & Birrell, S. (2019). The comparison of auditory, tactile, and multimodal warnings for the effective communication of unexpected events during an automated driving scenario. Transportation research part F: traffic psychology and behaviour, 65, 23-33.
   2. Hong, A., Lee, D. G., Bülthoff, H. H., & Son, H. I. (2017). Multimodal feedback for teleoperation of multiple mobile robots in an outdoor environment. Journal on Multimodal User Interfaces, 11(1), 67-80.
5. Finally, English language needs some minor spell check: there are several typos. For example: 
   • On lines 416-417: “This may be party explained”. Probably the authors meant “partly”.
   • On lines 523-524: “Only peaks during slipping event The value of…”: full stop missing after “event”.
   • On lines 590-591: “Therefore, based on our findings…”: probably “therefore” is accidentally in bold. 
   • On lines 600-601: “that correlates to previous findings [1,17] Nevertheless..”: full stop missing before “Nevertheless”.
   • On lines 659-660: “frequency of vibrotactile feedback r overlayed animated visual warning”: “or overlayed”?

Author Response

Dear Sir/Madam,

Here are our responses to your notes and critique:

• We have a custom of summarizing the key findings of our studies at the end of the introduction in order to improve readability. This way the reader can get a brief summarization of the whole work presented in the paper. If this is something that you consider to be against the principles of manuscript writing, please let us know and we'll remove the results from the intro!
• The small number of participants has been added to the limitations section, and we also added a sentence about the lack of further analysis on the effects of age, gender and driving experience (due to small number of participants)
• We have added the references you mentioned to the manuscript
• Regarding grammatical errors, we have fixed the typos and/or bad English mentioned in your review, and are currently also going through a vigorous proofreading of the manuscript

Thank you for taking the time to review our paper, and also thank you for your valuable feedback!