Agents and Robots for Reliable Engineered Autonomy:A Perspective from the Organisers of AREA 2020

Round 1

Reviewer 1 Report

Summary:

This is a review paper about future trends and challenges in engineering domains which involves using multi-agent systems, robotics, and software. Ideas have been incorporated from AREA-2000 conference. The contents of the paper are not cohesive. Different topics have been discussed but the paper fails to put them together as a cohesive work.

Comments:

1. 
Since this paper does not contribute to the state-of-the-art, this paper should be submitted as a "Review" paper, and not and original article.

2. 
The abstract is too small. It should be revised.

3.
This paper lacks a smooth "flow". Irrelevant and loosely connected works in the same field suddenly pop-up here and there without any connection to previously discussed content. It is annoying to read one-line paragraphs, and two-line paragraphs which looks like abstracts of respective papers, without the discussed content cohesively fitting into the big-picture discussed in this paper. There is no flow.

4.
Authors should give meaningful names to section titles and sub-titles.
Ex. 310 Natural Languages. (for which applications/field/etc.?)
There are multiple "Discussion" sections (Discussion of what?)
There are multiple "Modelling." sections.
390 Demonstrations (of what?).
636 Explanations
.....
etc.

5.
The conclusion section is one of the most important sections of a review paper. It should include the key insights, limitations, and future directions in detail. It is poorly written. Instead of many discussion sections, there should be a single discussion section before conclusion section which 'discusses' and not simply re-states the content.  

6.
A good review paper must be easy to read, and give insights about the domain to the reader. I strongly suggest the authors to re-write the paper. Think from the perspective of a reader/research-student who is new to the field and wants to get insights and future directions. Use figures, flow-charts, appropriate section titles, tables to cluster homogeneous works/solutions to cohesively present the work. Unfortunately, in its present state, the work needs to be revised. 

Author Response

Changes in the paper are highlighted in red (except for minor modifications). We thank the reviewer for the comments and suggestions.

1. Since this paper does not contribute to the state-of-the-art, this paper should be submitted as a "Review" paper, and not and original article.

We agree that this is not an original article. This paper was submitted as a “Perspective” paper, which is similar to a review but much shorter. JSAN supports this type of paper.

2. The abstract is too small. It should be revised.

We have added more information to the abstract.

3. This paper lacks a smooth "flow". Irrelevant and loosely connected works in the same field suddenly pop-up here and there without any connection to previously discussed content. It is annoying to read one-line paragraphs, and two-line paragraphs which looks like abstracts of respective papers, without the discussed content cohesively fitting into the big-picture discussed in this paper. There is no flow.

We have revised every major section and improved the flow from paragraph to paragraph. We have also rewritten the Introduction and improved the summary and discussion that is done at the end of each section.

4. Authors should give meaningful names to section titles and sub-titles.

Ex. 310 Natural Languages. (for which applications/field/etc.?)

There are multiple "Discussion" sections (Discussion of what?)

There are multiple "Modelling." sections.

390 Demonstrations (of what?).

636 Explanations

We have improved the titles and subtitles of most sections to be more descriptive and to have a better flow.

5. The conclusion section is one of the most important sections of a review paper. It should include the key insights, limitations, and future directions in detail. It is poorly written. Instead of many discussion sections, there should be a single discussion section before conclusion section which 'discusses' and not simply re-states the content. 

We have added more discussion to the conclusion that covers the challenges and future directions for the combination of these different research areas. We still have the individual sections that summarise the challenges and future directions for each separate section, since we believe those are interesting and different from considering all of these areas together, but have renamed these to Perspective of the Authors.

6. A good review paper must be easy to read, and give insights about the domain to the reader. I strongly suggest the authors to re-write the paper. Think from the perspective of a reader/research-student who is new to the field and wants to get insights and future directions. Use figures, flow-charts, appropriate section titles, tables to cluster homogeneous works/solutions to cohesively present the work. Unfortunately, in its present state, the work needs to be revised. 

We added more explanation where possible, but we assume the reader already has some knowledge of these areas (since this is a perspective paper).

Reviewer 2 Report

This paper presents a survey for multiagent systems. The paper is well written. I think it is fine to be accepted.

But I think the abstract is too short, I suggest enhance the abstract in the final version.

Author Response

Changes in the paper are highlighted in red (except for minor modifications). We thank the reviewer for the kind comments and suggestions.

1. But I think the abstract is too short, I suggest enhance the abstract in the final version.

We have added more information to the abstract.

Reviewer 3 Report

The authors review the research theme to link the technologies of MAS and robotics. Firstly, the guidelines for authors of this journal do not have the manuscript type, perspective paper. The reviewer cannot decide whether this manuscript is acceptable from the publishing policy of this journal. As an original research paper, this paper critically lacks the conclusion or opinion for the subject. Each author speaks the specific survey of his / her research field but most of the sections do not provide the discussion for the relationship between MAS and Robotics. They seem to discuss the problem of the autonomy of software agents. Thus, to accept this manuscript as a research paper, the authors should provide the problem or solution for linking MAS, Robotics, and reliability. Additionally, the conclusion should provide some concrete plan or opinion to solve the core problem. This version of the manuscript just aligns the research areas.

The complimentary comments for the improvement are listed below:

1) The concept of the verification discussed in this paper focuses on the agents' autonomy. However, the autonomy of robotic systems does not depend only on the logical elements but also on the physical matters (embodiments). The perspective of this paper just considers the validation of logic, and it stands on the premise that all of the operations for physical actions are precisely executed. In this situation, the MAS and physical systems form the complete hierarchy, and then we just think about the problems on MASs. In the reviewer's opinion, the unpredictability problem poses in the medium between physical and logical layers as discussed in brain science. If the authors assert the novelty of the research is the composition of MAS and Robotics, a new kind of problem should be proposed from the authors' new perspectives.

2) The implication of Table 1 is unclear. The individual literature has an independent topic. What do the authors say with this table? Additionally, the table is not easy to distinguishable. Please change the markers or improve the line-skip margins. 

Author Response

Changes in the paper are highlighted in red (except for minor modifications). We thank the reviewer for the comments and suggestions.

We agree that this is not an original article. This paper was submitted as a “Perspective” paper, which is similar to a review but much shorter. JSAN supports this type of paper.

1. The concept of the verification discussed in this paper focuses on the agents' autonomy. However, the autonomy of robotic systems does not depend only on the logical elements but also on the physical matters (embodiments). The perspective of this paper just considers the validation of logic, and it stands on the premise that all of the operations for physical actions are precisely executed. In this situation, the MAS and physical systems form the complete hierarchy, and then we just think about the problems on MASs. In the reviewer's opinion, the unpredictability problem poses in the medium between physical and logical layers as discussed in brain science. If the authors assert the novelty of the research is the composition of MAS and Robotics, a new kind of problem should be proposed from the authors' new perspectives.

Thanks for this observation. We did in fact focus on the software aspects of these two areas. More specifically, we gave our perspective on the latest software verification approaches in the context of MAS and Robotic applications. In the section where we talk about MAS verification, we focused on the verification of the reasoning part of the system, where the agents are best suited. While in the section about Robotic verification, we focused on the latest works on verifying the possible embodiments of agents. In this paper we do not focus on the low level aspects such as the communication medium between an agent and its body nor physical failures. We added more information and description in the corresponding section which we hope are going to clarify some of these aspects.

2. The implication of Table 1 is unclear. The individual literature has an independent topic. What do the authors say with this table? Additionally, the table is not easy to distinguishable. Please change the markers or improve the line-skip margins. 

We added the following clarification.

``Specifically, Table 1 summarises the requirement specification technique used for each of the papers presented in the AREA 2020 workshop. Each row contains a requirement specification technique, i.e., natural languages, logic-based languages, pattern-based languages, domain-specific languages, and goal-modelling. Each column contains the reference to one of the papers presented in the AREA 2020 workshop.  The cell at the intersection between a row, of one requirement specification technique, and a column, of one paper presented in the AREA 2020 workshop, indicates that the requirement specification technique used in that paper.  For example, the marker at the intersection between the row labelled as “logic-based languages” and the column labelled with the reference  [59], indicates that a logic-based language is the requirement specification technique used in [59].’’

We also increased the line-skip margins, as suggested by the reviewer.

Round 2

Reviewer 1 Report

Authors have revised the paper well. It can be accepted for publication.

Reviewer 3 Report

I would like to thank the authors to consider my comments and to revise.