Online Adaptive Dynamic Programming-Based Solution of Networked Multiple-Pursuer and Single-Evader Game

Round 1

Reviewer 1 Report

The paper reports the design of the solution of multi-agent pursuit-evasion game, with networked multiple-pursuer and single-evader. The solution is based on the minmax strategy. Moreover, an adaptive dynamic programming method is described for online policy iteration. Lastly, the paper presents some numerical results.

The following comments should be considered for the presentation of the work:

1.      The theme of paper is very interesting; however, it is not clearly written and it is not easily readable. I think that an extensive rewriting is required in order to better describe the proposed main concepts. For example, before the formulation of the game, the authors should provide an accurate description of the IoT system, in order to clarify its architecture. Indeed, this system is only mentioned, but some detailed information (e.g., how do the pursuers communicate? which information do they exchange?) are essential for a full understanding of the proposed work.

2.      A formal definition of the reference multi-agent differential game is required. For example: (1) which are the analytical objectives of each pursuer? (2) which is the analytical objective of the evader? (3) which performance indexes are minimized/maximized? These specifications would allow a better understanding of equations (5 – 8).

3.      The following sentence is not clear and should be rewritten: “Since the objective function of the MPE game above is integral, and the interval is segmented by the concept of integral reinforcement learning, so as to realize the PI method”.

4.      The authors should describe the consequences of the approximations described in section 4.2 by means of the value function approximation. Which are the consequences in terms of convergence to Nash equilibrium?

5.      The approach for integral reinforcement learning is only vaguely described. Moreover, the authors just mention the parameters of the neural network in section 5 (Numerical Simulation), whereas they have not provided details of this network in the previous sections. The authors should better describe the approach for integral reinforcement learning and the adopted neural network.

6.      The terms in equation (39) are not described.

7.      Some typos are present and should be corrected. For example, some symbols (“$i$th pursuer”, “$V^{*}$”, etc.) are deemed to be wrong.

Author Response

Dear reviewer

Thank you very much for your time involved in reviewing the manuscript, we have revised the paper according to your comments. Please see the attachment for details.

Sincerely,

Bing He

Author Response File: Author Response.pdf

Reviewer 2 Report

electronics-1987286-peer-review-v1

Online Adaptive Dynamic Programming Based Solution of Networked Multiple-Pursuer and Single-Evader Game.

 

The authors share a study online adaptive dynamic programming based solution of networked multiple-pursuer and single-evader game. The topic is interesting. However, there are few suggestions to improve the quality of the work.

Authors should clarify the following comments:

1.    I could not find the algorithm in the paper. Add the steps of that algorithm and flowchart in section 3.

2.    Which software is used to solve the problem defined by equations (40) and (41)? I did not find what specification of this software author used.

3.    Some of the results are not readable. In my opinion, the table of results can help authors to properly present and compare their work.

4.    It is well written, however my suggestion is to include
                  (i) Limitations of this study.
                 (ii) Sensitivity analysis, if possible.

5.    The numerical experiment, if we can call it, is just a game without any real-world data. It is just like a classroom exercise without any useful insights and contributions to multiple-pursuer and single-evader game.

6.    The Discussion is rather obvious and shallow. There are lots of graphs and tables which are not explained properly. Some in-depth analysis is required here. Try to make conclusions based on the obtained outcomes.

7.    There are many equations which have no citations in the main body (for example see Equations (44), (45)).

8.    The text written in various Figures, particularly Figures 1 and 7, are not clear and hence not readable. All the diagrams should be re-plotted with good resolution. In addition, the Figure captions need to be modified for better understanding. Author is suggested to address this problem.

9.    The paper is fraught with grammatical errors, especially punctuation errors. To this end, the work needs to be thoroughly proofread and edited accordingly by native speaker.

***

Author Response

Dear reviewer

Thank you very much for your time involved in reviewing the manuscript, we have revised the paper according to your comments. Please see the attachment for details.

Sincerely,

Bing He

Author Response File: Author Response.pdf

Reviewer 3 Report

The article is well written. The abstract, Introduction, Problem statement is clear and concise. The English is also fine except some minor checks are needed. I suggest authors to mention the original contribution of the article in few lines at end of the introduction section. Also, add the limitation of your proposed work in the conclusion section.

Author Response

Dear reviewer

Thank you very much for your time involved in reviewing the manuscript, we have revised the paper according to your comments. Please see the attachment for details.

Sincerely,

Bing He

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

I thank the authors for having considered all my comments and for having provided an accurate point-by-point response.

I have no further comments for this new version of the manuscript.

Reviewer 2 Report

No comment