Deep Reinforcement Learning for Distributed Flow Shop Scheduling with Flexible Maintenance
Round 1
Reviewer 1 Report
This paper proposed a deep reinforcement learning method for distributed flow shop scheduling with flexible maintenance. And the authors have done some experiments to demonstrate the performance of the proposed algorithm. However, the issues of this paper are that.
- The mathematical models need validation by solving at least small instances of the problem. Furthermore, this allows an evaluation of solution quality of DQND at least for small instances.
- Please explain the computational complexity of the DQND and comment how it is related to the other algorithms taken for comparison.
- In subsection 4.2, please give the reference of the compared GA.
- There exist many new metaheuristic methods for solving the distributed flow shop scheduling with flexible maintenance. I suggest that the proposed algorithm should be compared with the existing state-of-the-art algorithms published in a journal with high quality.
- These results should also be statistically significant.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The paper is consistent, easy to follow and self-contained. The proposed approach is yet another algorithm on the topic but overall deserve to be presented.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
this study focuses on a distributed flow shop scheduling with flexible maintenance and proposes a deep reinforcement learning for it.
First of all, I should say that this paper is poor. In a general view, studies in the scope of the scheduling problems, either focus on the problem (proposing a new problem) or focus on new solution methods (sometimes both of them). From the first perspective, I should say that this paper has no novelty and many more professional studies can be found in the literature for example:
Miyata, H.H. and Nagano, M.S., 2021. Optimizing distributed no-wait flow shop scheduling problem with setup times and maintenance operations via iterated greedy algorithm. Journal of Manufacturing Systems, 61, pp.592-612.
In addition, the authors assigned two pages to explain a sample of the problem, while it is not necessary!
In addition, it should be noted that many similar studies have been missed in the literature review.
Regarding the solution method, I should say that there are many reliable methods in the literature. When a new method is proposed in the scheduling problem it should be compared with a wide range of methods in the literature to show its capability. In addition, a strong statistical test is required to show the superiority of the algorithm.
furthermore, the proposed method has been presented very bad. There are many problems with pseudo codes. with a simple search in google scholar, the authors can find studies that employed a similar method to the scheduling problems.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report
The paper investigates the optimization problem of flow shop scheduling and flexibe preventive maintenance in a distributed production network. The deep Q network based solution framework is presented to solve the problem. The presented approach is very interesting and topical. The paper is valuable and can be accepted for publication after a few modifications:
Page 5, Fig.2 and 3. Do the Figure 2 and 3 show deteriorating operation times of jobs? The operations seem to take the same time, irrespective of the age of the machines.
Page 7, algorithm 1, Should be r(s',a) instead of "...Take action a...and reqord r(s,a).
Where is a random variable Tau generated to compare with the predefined value Epsilon? What is the value of the learning rate and the discount factor?
Numerical examples of the initialization of action-value function Q with random weights, preprocessed sequence, action, state...are needed.
Page 9, Algorithm 2, What is the value of Gamma in yj
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The authors have enhanced the manuscript by far. I have no more comments.
Author Response
Thanks greatly for your comment as well as your valuable suggestions on this manuscript. In the second round of revisions, the authors have integrated the comments of other reviewers and further improved the manuscript. We hope that these changes will satisfy you as well.
Reviewer 3 Report
The authors have improved the quality of the paper. But I think the proposed mathematical model is not for a scheduling problem with considering the maintenance. Please revise it.
Author Response
Thank you very much for your further suggestions. It has to be admitted that the proposed MILP model is only used to portray the distributed flow shop scheduling problem in the illustrative instance, and not the model of the studied integrated optimization problem, which may cause misunderstanding to the readers. After thinking deeply, the authors decided to remove the MILP model from the manuscript and referred to the distributed job shop scheduling model in the reference [38] for the CPLEX solution, which would look clearer and easier to understand. As for the proposed research problem, the authors provided further elaboration for not building a mathematical planning model but adopting model-free learning. Please see the contents marked in red in the new Section 2.
