Auditing Flood Vulnerability Geo-Intelligence Workflow for Biases

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper describes a process of auditing a flood vulnerability geo-intelligence workflow for biases. My comments are as follows:

1.      My main question for this paper is: How common is the workflow being audited? Is it widely used in other research for determining flood vulnerability, or is it specific to certain policymaking processes or government agencies? I think this is important to address because I expect this paper to identify biases in a widely used workflow that many may not be aware of, rather than simply identifying biases in a random, less-known workflow.

2.      Building on the previous point, the authors should clarify the sources of their data and how commonly these sources are used by others for determining flood vulnerability. For example, the authors mention that biases may arise from the street view images they used. But where and how were these images collected? Are they publicly available? How common are such images used in decision-making, especially for flood vulnerability? The same questions apply to their UAV images, OSM data, and other datasets.

3.      On page 4, I suggest that the authors expand on the explanation of damage curve estimation. As it stands, Figure 1 alone does not provide enough clarity on how this process works.

4.      On page 5 line 166, the integrated vulnerability score is a sum of the social vulnerability score. How about physical vulnerability?

Author Response

Please see the attached file for our responses to the comments. 

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

Geodata, geographical information science, and GeoAI play an increasingly important role in predictive disaster risk reduction and management. The paper uses AI to characterize housing stock vulnerability to flooding in Karonga district, Malawi and focuses on how to audit geo-intelligence workflows for biases. The results show how the use of AI introduces and amplifies bias against houses made of certain materials.

There are some major concerns in the paper that need to be revised.

1. Figure 1 is not clear enough, so it is recommended to replace it.

2. Section: 2.1. Case study: Flood vulnerability geo-intelligence workflow. The paper mentioned a lot of research data (data on built environment, socio- economic data, UAV imagery, street view images, UBR), but there was no detailed introduction of these data, including spatial resolution, imaging time and other information. In addition, we don't know any information about Karonga (Malawi).

3. Line 172. How is Table 1 produced?

4. Section 3.2 Auditing. The authors chose many methods proposed by other scholars, but did not elaborate on the reasons for their choice, especially the advantages and disadvantages of these methods.

5. The conclusion is too simple, and the authors need to combine the Section 4.1. Results to summarize quantitative research results.

Comments on the Quality of English Language

Minor editing of English language required.

Author Response

Please see the attachment. 

Author Response File: Author Response.docx

Reviewer 3 Report

Comments and Suggestions for Authors

This manuscript (ijgi-3245161) tries to utilize the AI to characterize housing stock vulnerability to flooding in Karonga district, Malawi. The authors have utilized the definition and categorization of biases that emphasizes on biases as a negative and undesirable outcome. Result indicates how the use of AI introduces and amplifies bias against houses of certain materials. After my careful review of this manuscript, I have a series of major concerns and comments in the following:

- . The overall writing style of the manuscript is lengthy, and some paragraphs and sentences can be further condensed to improve readability.

- . The Introduction of the manuscript lacks a detailed explanation of the background of the bias problem in disaster risk reduction and management and geo-intelligence workflows and the introduction of specific cases. This makes it difficult for readers to quickly understand the importance of the manuscript and the research motivation.

- . There is a need to increase the in-depth analysis of the current conditions of DRRM, including current technical challenges, existing bias problems and their impact on the socio-economic situation, so as to more clearly explain the necessity and urgency of this research.

- . The Method Section is not clear and easy enough to understand when describing the specific analysis process and technical implementation. In particular, the detailed description of the architecture selection, training parameters and data processing flow of CNN (convolutional neural network) is not detailed enough. More detailed information on the algorithm architecture, hyper-parameter settings, data preprocessing steps, etc. is needed.

- . The depth of the Literature Review needs to be strengthened to more comprehensively present the research background and highlight the unique contribution of this manuscript. In particular, few studies on waterlogging in the past two years have been mentioned. Please refer to:

Assessing the scale effect of urban vertical patterns on urban waterlogging: An empirical study in Shenzhen. 2024, 107486

Designing Sustainable Drainage Systems as a Tool to Deal with Heavy Rainfall—Case Study of Urmia City, Iran. Sustainability. 2024

- . The manuscript does not fully discuss the relevant ethical and privacy issues when collecting and using geospatial data. In particular, when using open source data such as Open Street Map (OSM), the data usage rights and privacy protection measures need to be clearly stated.

- . The experimental design section lacks sufficient control groups or comparative experiments to fully verify the effectiveness and superiority of the proposed method.

- . It is recommended that the authors add more evaluation indicators, such as accuracy, recall, F1 score, etc., to more comprehensively evaluate the performance of the model. At the same time, methods such as cross-validation can also be considered to improve the reliability of the evaluation results.

- . The reasons for the performance differences between OBIA and CNN models on different roof types are not explored in depth, and targeted improvement strategies are not proposed.

- . The authors need to enhance the in-depth analysis of the experimental results, including the reasons behind the results, the limitations of the experimental results, and the differences from the expected results.

- . The Discussion Section focuses too much on the research results themselves, and fails to fully explore the practical significance and potential impact of these results on the field of disaster risk management and reduction.

- . In addition, the existing discussion is not comprehensive enough. In particular, the challenges and potential biases that may be encountered in practical applications, such as data sparsity and geographical differences, were not fully discussed.

- . The Conclusion of the manuscript is relatively brief, and the summary of the research findings and the outlook for future work were not in-depth enough.

Comments on the Quality of English Language

Extensive editing of English language required.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have addressed my comments.

Author Response

We thank the reviewer for the final comments.

Reviewer 2 Report

Comments and Suggestions for Authors

I would like to thank the authors for their careful revision.

Author Response

We thank the reviewer for the comments.

Reviewer 3 Report

Comments and Suggestions for Authors

Although this article is a revised version, the reviewer still has a number of major severe concerns:

- . The authors mentioned the analysis of model and data bias, but failed to fully identify all potential sources of bias and analyze them in detail. The authors need to further improve the bias analysis, including a detailed discussion of different types of bias, the impact of bias on model performance, and how to reduce or eliminate these biases.

- . This article still has major flaws in terms of experimental design and result validation. For example, there is a lack of sensitivity analysis of different parameters or models. In addition, the diversity and representativeness of the validation experimental data are insufficient.

- . The article does not mention whether stakeholders, especially data subjects, were involved in the method design, data collection, and processing. This leads to a lack of representativeness and acceptability of the research results in practical applications.

- . The authors claimed that the depth of the Literature Review has been strengthened, but they do not strengthen its arguments.

- . Lastly, the authors must enhance the comparison with other related studies in the field to highlight the innovation and advantages of this study.

- . Overall, it is obvious that this article was written in a hurry. Some grammatical errors, typos can still be found in the revised version. The authors must provide the official certificates after the English language editing, such as the services provided by the MDPI.

Comments on the Quality of English Language

Some grammatical errors, typos can still be found in the revised manuscript. The authors must provide the official certificates after the English language editing, such as the services provided by the MDPI.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 3

Reviewer 3 Report

Comments and Suggestions for Authors

No further comments.

Comments on the Quality of English Language

Moderate editing of English language required.