Traffic Scenarios for Automated Vehicle Testing: A Review of Description Languages and Systems

Round 1

Reviewer 1 Report

    The literature review requires improvement along with more recent papers.

    Though conclusions explained , it is still required to improve this part with a clear suggestion.

Author Response

The authors would like to thank the reviewer for your constructive comments and recommendations to help improve this article. We have made the following corrections and improvements in response to these comments.

Author Response File: Author Response.pdf

Reviewer 2 Report

This paper is about the review and comparison of the mainstream scenario description languages. The intention and the motivation of the paper is the development and testing of automated driving functions and ADAS systems, since such systems require scenario-based testing approaches. The traditional approach in automotive system development and testing has practised distance-based testing and validation to ensure sufficient system reliability. Such an approach is rather flawed and insufficient for automated vehicles since it would imply serious and impracticable testing efforts before homologating such systems.  Based on this fact, the paper answers an important question, about what the state-of-the-art in scenario modelling is. In doing so, the authors have introduced an example scenario and implemented it in different SDLs introduced. They also provided some insights about the main differences and areas of applicability in each SDLS. The conclusions indicate that, which is quite important, none of the available SDLS are fully suitable/capable as a universal scenario description method for a complete ADAS/AD development an testing purposes. 

In general, the paper is very well written, and the language is clear and understandable except very few minor typos. Technical content is also sufficient, though not very deep, which is I think acceptable for a review paper on this topic.  There are however a few minor questions, recommendations and comments listed below, which I recommend being taken into account in a possible revision for improving the disposition of the ideas and the structure of the paper overall. The comments are as follows:

1. What is the specific reason for choosing the example scenario? Would you consider it as an edge scenario? Please specify and extend in Section 3.1.
2. I am not sure how the listings such as in Fig.2, 4, 6, 9 etc. are useful. They give an impression/snapshot but doesn’t convey any further detailed information. I don’t know if they should be included. Perhaps these could be removed or at least reduced a bit for a more reduced and concise text. I think it is more useful what the impressions from the specific SDLS implementation rather than a snippet from the code itself.
3. On page 8, line 268 “be” is missing in between “can also”.
4. Figure 12b, caption needs to be checked for grammar, especially for small/large letters for consistency.
5. On Page 11, lines 336, SCNEIC is misspelled.
6. I believe the Table 1 is the most important result of the paper. It is not clear however what “Edge Case” column imply. Perhaps the table needs to be extended and discussed a bit better with information such as which most suitable utilisation areas etc.
7. On Table 1, Scenic/Dataformat entry has a typo.
8. On Page 13, line 381, “No” should be “Not”.
9. On Page 13, the comment on lines 386-388 can easily be addressed if the traffic objects are also modelled down to sensors setup. Basic FOV and occlusion effects of an ideal sensor can easily be modelled with a low fidelity sensor model. These could also be part of the scenario descriptions. How would you comment on this?
10. On Page 13, line 393, after “textual and graphical” phrase, “representation” term should be added.
11. Conclusion is a bit too short. Perhaps it can be extended more.

Author Response

The authors would like to thank the reviewer for your constructive comments and recommendations to help improve this article. We have made the following corrections and improvements in response to these comments. Please see the attachment.

Author Response File: Author Response.pdf