Design Limitations, Errors and Hazards in Creating Decision Support Platforms with Large- and Very Large-Scale Data and Program Cores

Round 1

Reviewer 1 Report

This work aims at introducing a novel methodology for designing, maintaining and updating very large scale DSSs.
The manuscript is very detailed and rich, studded with many examples from the transport area - all things that make this reviewer consider that this manuscript might have been originally conceived as a book chapter, which is not a bad thing.

In the Introduction, the Authors define the acronym DSS: this acronym is not used consistently throughout the manuscript (e.g. see lines 59, 61, in which part of the acronym is used together with part(s) of its definition). I suggest to define the acronym at the beginning - as it is done in line 38 - and then to use it throughout the whole manuscript. Line 59 also re-uses the acronym DSS with "Very Large-Scale": it may be the case to adopt a different acronym, but I suggest not to use the same DSS acronym for Very Large-Scale DSSs. In general, once an acronym has been defined (like ETIS, in the other Sections), re-use it without defining it again.

In this same section, Authors address some of the areas in which DSSs cover a pivotal role. I would suggest also to mention the Healthcare area, for which this Publishing House has many interesting papers related to DSS (e.g.: 10.3390/s20174923 ; 10.3390/electronics7090179). As we can see in these days, healthcare is fundamental for the human society.

Lines 63 and 64 mention the "Information System", which should be assumed to be something different from the DSS. I would suggest, at the first occurrence in line 63, to provide a brief definition of "Information system", so that readers have can narrow the concept.

Section 2 addresses a relevant problem, i.e. the tendency to have larger and larger data sets and the necessity to handle more and more complex questions. Two interesting examples in the EU countries are also presented. It is worth noticing that all the issues presented in this Sect. have already emerged in real-world scenarios.

Sect. 3, line 172 reports the entry "Chapter". I would stick to "work" or "paper", although this manuscript is indeed huge as a book chapter. Point a) of this Sect. underlines an interesting fact, but it is my opinion that the two consequences should be supported by evidence that can be traced in literature.

Sect. 4 again deals with other problems related to the design, maintenance, and update of large-scale DSSs.

Sect. 5 addresses some methodologies adopted nowadays for the development of large scale DSSs - this Sect. needs more references, especially for the approaches that are further analyzed with lists.

Sect. 6 finally introduces the methodology endorsed by the Authors. I believe that stages 1 and 2 could benefit from references in the previous Sect., as the "group of experts" conducting a domain analysis is not something new (see my comment for the previous Sect. 5). The Authors rely on the thematic decomposition of more complex issues to tackle some of the issues mentioned in the previous sections: however, it can be argued that this process - while tackling some of the issues - generate another issue, which is gathering consensus of domain experts about how to segment the problem/domain at hand. Literature regarding ontology engineering methods is plenty of such cases.
The meta-DB itself makes me think about ontology, in this case, used to foster data interoperability, and this idea of mine is to strengthen by line 577.
I disagree with the statements in lines 571 - 573: what if a complex DSS needs to access sensitive information, i.e. information covered by GDPR? I would suggest that this kind of problem should be addressed directly by External Sources.

Sect. 7 tries to draw the conclusion of this huge work. However, the approach based on the meta-database is not directly shown in this paper, thus making the assertions regarding the advantages of the approaches of Sect. 7 merely speculative.

There are some typos in the manuscript - e.g., line 308 "weather". Some others in the text, and one in a sub-title of Sect. 6.3. I, therefore, suggest a revision of the manuscript.

In general, half of the paper is dedicated to underline, analyze and detail the current problems (many problems) related to designing updating, and maintaining Large Scale DSSs. Sections from 2 to 4 cover this role, therefore I suggest the possibility to "merge" into a single Section at least Sections 3 and 4 (i.e. those Sections related to design errors and difficulties for the design and deployment of DSSs), by shrinking the content a bit. I understand that, given the meticulous analysis here presented, this may not be an easy task. Sections 3 and 4, while presenting relevant issues in this field, should provide more references to the problems.
The advantages of the proposed methodology are clearly presented, but not documented. In other words, the paper presents Authors' opinions regarding how a very large DSS should be designed and developed, but unless these advantages are not compared to the existing methodologies they remain Authors' opinions.
While they address the meta-database/knowledge base as a solution, they do not refer directly to that literature. Moreover, I suggest the Authors compare their methodology to other existing methodologies that tackle the same issues, underlining why their approach is more suitable. I am afraid the manuscript is very strong on the exposition of the Authors' idea, while it lacks in arguing why their approach is better than others.

Since the Authors were involved in ETIS development (10.1177/0361198106195700105) they could use this use-case to validate their findings; in other words, the paper could generally state the problem of large scale DSSs design, maintenance, and update (without referring specifically to ETIS), they could introduce the methodology endorsed by Authors (Sects. 5-6), then it could show the results and advantages deriving from the application of this methodology through the ETIS use-case. Please note this is just a suggestion, I understand that such a deep modification of the manuscript is not trivial.

I believe this manuscript can be of interest for Algorithms readership, therefore I suggest Authors deeply revise their work so that it is suitable for publication in this journal.

Major revision.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

1. Data warehousing, data mining, and on-line analytical processing are the three main data management technologies of DSS. In order to support decision-making, how to organize and compile data, and how to use data is very important. The authors introduce a special meta-database that can use corresponding templates to store the entire abstract structure generated by experts (called "Teleological Meta-database "), which is the highlight of this research. The authors emphasize issue orientation, using issue, sub-issue, indicator, etc., and uses models for decision analysis. This type of operation depends on top-down thinking and is usually an interdisciplinary requirement for creating a decision support platform with large-scale data and program cores, and there are problems that cause design limitations, errors, and hazards to the system.
2. The large DSS data core contains data from a huge number of different, heterogeneous and autonomous external sources. From the database, meaningful information can be extracted and analyzed based on various issues, and a consensus can be reached so that decision-makers can make wise judgments. The process involves transdisciplinary research; the authors reviewed appropriate literature and raised questions and new methods.
3. Line 328, “the DSS lifecycle strongly depends on the duration of the incumbency of the decision authorities”. As far as the actual operation of customization is concerned, if the life cycle of the DSS is short, it seems that there is no need to create a decision support platform with large-scale data and program cores, and there is almost no problem that causes design limitations, errors, and hazards to the system.
4. Line 421-439, “the new approach (Teleological Meta-database) is the heart of DSS and its operation. Since the same Teleological Meta-Database can be used to build a sound decision support system, and the elements of the previous system can be merged together in the future stage”. The problem is that this research only proposes the idea of building a new DSS, and does not discuss the refinement of most existing large-scale systems that have invested funds. It can be seen that this new method is only suitable for the new development of purpose-oriented large DSS, and there are limitations in the promotion.
5. Although the concepts related to Teleology and Meta-database have been widely used, the authors integrate them and propose a new method of DSS, which has contributed to the theory and practice of DSS technology.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The authors deal with an interesting topic of developing a decision support platform to deal with large and very large scale data and program cores. The approach is interesting as the basic idea is to make the current environment more resilient and efficient. Although the topic is interesting, several major insufficiencies need to be improved. These insufficiencies can be summed up in poor scientific writing and a lack of clarity.

Suggestions for improvement:

• The manuscript should be set according to the Journal’s template and instruction to authors (text, figures, tables, equations, references, etc.). For example, the figures are unreadable, missing references, etc.
• Check and improve the English language and grammar throughout the paper (check misspellings, writing in the first person, etc.), as well as all figures and tables (both must be readable). Define abbreviations before their first use.
• The paper is poorly structured. Authors are advised to use IMRAD structure
• The introduction does not provide sufficient background and includes all relevant references. The used references are not novel nor based on previous scientific papers. Also, some fundamental references are missing as well as the recent ones considering the research problem. The authors should be consistent in writing. The research problem is not clear while the research goals and hypotheses are not clearly stated
• The literature review should be improved. At the moment it is too long, lacks a critical overview of the other approaches in solving the stated research problem and the methodology upgrade that is proposed by this research
• The research design is not clearly written. The research methodology should be clear and the hows and the whys of used methods should be clearly visible. The validation is especially important, therefore authors are urged to give insight into the validation process. Also, add some additional discussion of findings in relation to the research framework as well as research goals and hypotheses are needed
• The authors are urged to draw conclusions that are more specific. At the moment it seems like good observations and arguments are currently missing from the discussion section. There should be a clear connection with the research problem, goals, and results

Overall, I strongly urge the authors to reconsider the above-mentioned comments, rewrite the paper accordingly, and resubmit.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The Authors have conducted a careful revision and significant efforts to meet my previous comments. In particular, I'd like to point out that the effort of merging two sections into one has, in this reviewer's opinion, made the manuscript easier to follow. To make the manuscript even easier to follow, I'd suggest the Authors add in line 74 a short paragraph stating the following: "The paper is organized as follows: Sect. 2 introduces ... .... Sect. 3 deepened into ... ... Finally, Conclusions ... ...".

Wrt Conclusions, this Sect. is missing: although they are pretty evident, I'd suggest to add a Conclusion section summarizing the main research problem in a few lines, followed by the results gained/expected from the approach. Please note that this suggestion is thought to increase the readability of the paper, which - being very long and detailed - may present some difficulties for readers to clearly identify your hypothesis and your conclusions.

All the comments I provided with my previous revision have been properly addressed, including general comments on the English language. Just a minor thing: from a stylistic point of view, the lines 63 - 65: "In essence, a DSS is a special kind of a very complex Information System (IS), where, following Britannica, an Information System is an integrated set of components for collecting, storing, processing data and for providing information, knowledge and digital products." present a citation to Britannica; I'd add this as a reference: "In essence, a DSS is a special kind of a very complex Information System (IS), where an Information System is an integrated set of components for collecting, storing, processing data and for providing information, knowledge and digital products [reference_number]."

A note regarding the Figures: please be sure that they are made clearly readable (maybe change the light grade shade into another color, as I believe Algorithms does not charge you for colored figures - please verify this information with the editors).

(very) Minor revisions from my side.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

In the revised version authors gave some additional insights into their research and also acted upon given comments and suggestions. Still, both the research design and paper structure remained poor. Such results in an unclear manuscript that is hard to follow. I encourage authors to look into previously stated issues and suggestions in order to improve paper quality.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf