Multiple Criteria Decision Making for the Achievement of the UN Sustainable Development Goals: A Systematic Literature Review and a Research Agenda

Round 1

Reviewer 1 Report

A key piece of puzzle is missing in this review. As stated in the Abstract, “The widespread adoption of MCDM methods in complex contexts confirms that they can help decision-makers solve multidimensional problems associated with key issues within the 2030 Agenda framework” (Ln 16-18). The most important contribution of this systematic review, therefore, lies in how MCDM methods help to solve multidimensional problems in achieving the SDGs. In this regard, a focus on summarizing the viable strategies or actions facilitating the achievement of the SDGs using the MCDM methods should be the major theme of this review.

To be specific, in addition to identifying the problems solved by applying the MCDM methods as in Rigo et al. [24], the authors need to give a comprehensive review concerning how the MCDM methods has been used to planning or proposing viable strategies or practices help to achieve the SDGs.

Moreover, it was emphasized in the introduction that this is the first systematic review focusing on the achievement of SDGs and decisions addressed resource mobilization. (Ln 115-117). However, I did not find any review concerning the use of MCDM methods to address the decisions addressing resource mobilization. Actually, “resource mobilization” only appeared twice, one in line 117, the other in line 200. In my view, resource mobilization is an important issue to address in order to achieve the SDGs. This is another example of this review’s lack of comprehensiveness.

My last concern is the length of the review. Some of the categorization rules in the review cannot serve the purpose of advancing our understanding of the use of the MCDM methods in helping decision-makers solve multidimensional problems associated with key issues within the 2030 Agenda framework. Shorten the paper and get more focused on the merits and the use of the methods in helping decision making are strongly recommended.

Author Response

Please see the attachment. 

Author Response File: Author Response.pdf

Reviewer 2 Report

In this paper the authors carry out a review on the Multiple Criteria Decision Making (MCDM) methods that have been used in the study of the achievement of UN Sustainable Development Goals (SDGs) and the implementation of the 2030 Agenda for Sustainable Development in regional, national, or local contexts. For this, the authors have analyzed 143 articles published in the 2016-2020 period and which are listed in the Scopus database.

Although there are other papers dealing with similar topics, this manuscript provides a pioneering study  on the achievement of SDG. The paper is well written, well structured and interesting for researchers working in this topic.

Some minor typos:

- Page 7: Check the references in Table 3. For instance, the AHP method is the reference [37], TOPSIS is the reference [38] and so on. Check all the references.

- Page 7, Table 3, Remaining methods: If the number of items is less than 5, the percentage cannot be 23.98.

- Page 8, Note: Delete DEMATEL (the DEMATEL method is already included in Table 3).

- Page 10, Figure 6: Before Figure 6, the authors say “The largest number of articles concerned with SDGs issues in national 413 contexts are in Iran (7 studies), followed by China (6), India (5), Spain and Taiwan (4 each), Turkey and Brazil (3 each), and then Pakistan, Greece, and Morocco (2 each)”; but these data do not correspond to those shown in Figure 6. By the way, I do not know any country called Generic.

- Page 12: Oliveira et al.[50] --> Oliveira et al. [50]; Benítez and Liern[53] --> Benítez and Liern [53]; etc. Check the paper.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

• l.46                  And practitioners (consultants and others l.52)
• l.77                  Not necessary to duplicate the full name
• l.82                  Not only MCDM techniques but also MCDA processes.
• l.109                Stakeholder involvement (it is needed)
• l.121                The most applied? It is not a marketing proof as “it is better because more people use it”. The real question is what are appropriate approaches, methods and tools?
• l.132-137        Multi-stakeholder approaches and methods should receive full attention because participatory mechanisms are intrinsically linked to SD.
• l.137                Closing ).
• l.151                a way?
• l.152                researches
• l.163                Panel members: to face diversity of approaches, 4 people is not sufficient (l.233-236 are not sufficient and convincing; are we sure none of them are co-authors of the paper or related to them). What about the conflict of intertest (eg. if they develop and/or practice a single method or if they are publishing in “top” journals as this one [see importance of section 3.6]) or unconscious biases? L.193: it is not only the case oh “inviting” 4 experts. There are ethical and representativeness issues (eg. l.215: assign relevance scores; l.227: only the first cluster: should you provide stats about the rejected methods? It is particularly important since the methodology give importance on the most popular ones).
• l.169                Something should be said about the language issue: it is not correct to pretend to do an international literature review and being limited to English. It is understandable to have some limitations but at least they should be pointed.
• l.175                Approach, method, tool, software are not interchangeable.
• l.287-288        Very interesting point (also l.455-456)
• l.312-314        A little bit quick statement at this stage and based on a single reference. There are many papers on the user-friendly side of the different methods. Also, this is also related to the approach used and not only the method.
• l.350                Integration of MCDM methods: it is a very delicate point. The proof that the theoretical backgrounds are compatible and coherent should be discussed to be serious. It is very important since for example what is stated l.366. Few strong references might be cited on that point and in case it is not covered in the literature, it might be in the agenda of the gaps to be filled. For example, I doubt seriously that MAUT and ELECTRE are compatible. (l.903)
• l.366-367        Same for combining compatible methods (eg. About GIS: when you have a GIS, every problem looks like a GIS problem. What is happening with criteria not assessed by GIS such as empowerment).
• l.378                SD needs to benefit from approaches and methods that are able to manage time horizon evolving issues, and uncertainty. Fuzzy logic is certainly interesting about uncertainty. What about time dynamic methods?
• l.409-410        And those 26 out 143 papers have high scores according the panel!
• l.631                The concept of optimality should not be used related to MCDM methods. It is a matter of best compromise at least between criteria and hopefully between stakeholders.
• Section 4         Not much to say.
• l.1003-1008    Those conclusions might not be very legitimate and trustfull according my remarks. There just in line with global citation indexes which are severely criticized. Nowadays, validity and legitimacy of science cannot be an assessed on a Kilo citations basis.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have addressed all the comments I made in the first-round review. I do not have any further comments.

Reviewer 3 Report

This new version is OK. Congratulations. It has been greatly improved.