Enhanced Wavelet Scattering Network for Image Inpainting Detection

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper presented a method to detect inpainting forgeries based on low level noise analysis by combining dual-tree complex wavelet transform for feature extraction with convolutional neural networks for forged area detection and localization, and lastly by employing an innovative combination of texture segmentation with noise variance estimations. The work constructed a new dataset (Real Inpainting Detection Dataset) and compared the state-of-the-art methods on this dataset. Some concerns need to be solved in the revision as follows.

(1) Some studies have applied the wavelet transform for preprocessing [2] and feature extraction [1] for detecting the fake images, which are related the task of image inpaiting. They are suggested to be discussed in related work.

(2) The architecture in Figure 2 is not clearly. The main procedures cannot be identified. It is suggested to improve the quality of Figure 2. The same issue also exists for the Figure 3.

(3) Why do the authors divide the image into different regions to analyze Noise-Aware texture inconsistencies? The segmentation by many regions will introduce the inconsistency, which would not be helpful to the inpainting detection.

(4) More figures in the self-constructed dataset are suggested to be shown in the experiments.

(5) Some typos can be easily found in the manuscript. For example, the title “Enhanced Wavelet Scattering Network for image inpainting detection” needs to have the same uppercase letters like “Enhanced Wavelet Scattering Network for Image Inpainting Detection”; For Figure 1, some citing error exists in Page 2, and the same issues existed for other figures.

[1] Gao et al. DeepFake detection based on high-frequency enhancement network for highly compressed content. Expert Syst. Appl. 249: 123732 (2024).

[2] Aymen et al. Application of spatial and Wavelet transforms for improved Deep Fake Detection. International Conference on Artificial Intelligence, Robotics and Control (AIRC), 2024.

Author Response

Thank you for your detailed and insightful review of our manuscript. Your comments have
been invaluable in guiding our revisions, and we are grateful for the opportunity to enhance
our work based on your expertise. Below, we address each of your points and outline the
revisions made in response to your feedback.

Comments 1: "Some studies have applied the wavelet transform for preprocessing [2] and feature extraction [1] for detecting the fake images, which are related the task of image inpaiting. They are suggested to be discussed in related work."

Response 1: Thank you for the suggestion. We have thoroughly investigated the references you mentioned and have incorporated them into our work. We have now discussed them in the related work section, highlighting their contributions as well as their limitations and how they align with our approach.

Comments 2: "The architecture in Figure 2 is not clearly. The main procedures cannot be identified. It is suggested to improve the quality of Figure 2. The same issue also exists for the Figure 3."

Response 2: Thank you for your feedback. We have updated Figures 2 and 3 to enhance their clarity and readability, ensuring the main procedures are now clearly identifiable. The images were originally created at high resolutions (some as large as 3000x3000), but for the purposes of the Word document, we have set their display size to 12%. Once the article reaches the editors, we can provide these figures in their full resolution for optimal quality. This approach has been applied to all images presented in the paper.

Comments 3: "Why do the authors divide the image into different regions to analyze Noise-Aware texture inconsistencies? The segmentation by many regions will introduce the inconsistency, which would not be helpful to the inpainting detection." 

Response 3:  Thank you for your insightful question regarding the rationale behind dividing the image into regions to analyze noise-aware texture inconsistencies. We appreciate the opportunity to clarify the motivation and methodology behind this approach. In response, we have added a detailed explanation to the paper. The core concept of our method is that, in consistent textures, wavelet coefficients generally follow a predictable statistical distribution, often near-normal in high-level, noise-free regions. Significant deviations from this expected distribution suggest potential anomalies or inconsistencies, as they indicate variations beyond natural texture or noise variations. This approach ensures that inconsistencies are identified based on statistically significant deviations within the texture's structure rather than mere color variations

Comments 4: "More figures in the self-constructed dataset are suggested to be shown in the experiments."

Response 4: We've added an Appendix with 2 more cases. What is important is that we are going to provide full access to the all the images (original, masks, altered, detection done by our method and the ones we've compared with along side the entire code - neural network + already trained versions)

Comments 5:  Some typos can be easily found in the manuscript. For example, the title “Enhanced Wavelet Scattering Network for image inpainting detection” needs to have the same uppercase letters like “Enhanced Wavelet Scattering Network for Image Inpainting Detection”; For Figure 1, some citing error exists in Page 2, and the same issues existed for other figures.

Response 5: Thank you for pointing out these issues. We have carefully reviewed the manuscript and corrected the typos

Reviewer 2 Report

Comments and Suggestions for Authors

The primary question addressed by this research is how to effectively detect and localize image inpainting manipulations. 

The study focuses on developing a novel framework that combines wavelet scattering networks with convolutional neural networks (CNNs) to enhance the detection of inpainting forgeries. 

You could consider the following specific improvements and additional controls regarding their methodology:

- Incorporation of More Diverse Datasets: While you've introduced the Real Inpainting Detection Dataset, expanding the dataset to include a wider variety of inpainting techniques and image types could enhance the robustness of the model's performance against different manipulation methods.

- Evaluation of Additional Neural Network Architectures: The paper mentions evaluating various neural network architectures. Including more recent architectures or hybrid models could potentially improve detection accuracy and efficiency.

- Control for Environmental Variables: Implementing controls for environmental factors such as lighting conditions, image resolution, and compression artifacts could help assess the model's performance under varying real-world conditions.

- Testing Against Adversarial Attacks: To ensure the model's robustness, you should consider testing their methodology against adversarial attacks that aim to deceive the detection system, which is crucial for practical applications in multimedia forensics.

- Longitudinal Studies: Conducting longitudinal studies to evaluate the model's performance over time and with evolving inpainting techniques could provide insights into its adaptability and long-term effectiveness.

By addressing these areas, you can enhance the reliability and applicability of their proposed methodology in detecting image inpainting forgeries.

Comments on the Quality of English Language

Some sections of the paper could be written more clearly, with minor improvements in sentence structure and grammar. This would help in expressing complex ideas more fluidly and improve the overall readability for an international audience.

Author Response

Comments 1: "- Incorporation of More Diverse Datasets: While you've introduced the Real Inpainting Detection Dataset, expanding the dataset to include a wider variety of inpainting techniques and image types could enhance the robustness of the model's performance against different manipulation methods."

Response 1: Thank you for the suggestion. In response, we have incorporated two additional datasets to evaluate our model's robustness across a broader range of inpainting techniques and image types. While the results on these datasets are not as strong as those achieved with our proposed dataset, they provide valuable insights. The training process on these new datasets was also slower, reaching only 20 epochs, which may have impacted performance

Comments 2: "Evaluation of Additional Neural Network Architectures: The paper mentions evaluating various neural network architectures. Including more recent architectures or hybrid models could potentially improve detection accuracy and efficiency."

Response 2: Thank you for the valuable feedback. We’d like to note that this work is part of a PhD thesis, where we have indeed explored a variety of neural network architectures, including adaptations from Mantranet, IID, TruFor, as well as stacked EfficientNet models for each band, and incorporated LSTM cells, transformers, and custom variants of SegFormer. While these investigations yielded insightful results, we have chosen to focus on the primary outcomes in this paper to adhere to page limitations. The comprehensive exploration of these architectures, along with detailed findings, will be included in the thesis to showcase the full scope and gains of our research

Comments 3: "Control for Environmental Variables: Implementing controls for environmental factors such as lighting conditions, image resolution, and compression artifacts could help assess the model's performance under varying real-world conditions." 

Response 3: Thank you for the suggestion. In response, we have implemented controls for various environmental factors by incorporating attacks such as resizing, blurring. 

Comments 4: "Testing Against Adversarial Attacks: To ensure the model's robustness, you should consider testing their methodology against adversarial attacks that aim to deceive the detection system, which is crucial for practical applications in multimedia forensics."

Response 4: Thank you for the suggestion. While we haven’t yet conducted adversarial attack analysis, we recognize its importance for enhancing the model's robustness in practical applications. We plan to incorporate this aspect in future work, focusing on testing the model's resilience against adversarial manipulations. This will be a valuable addition to further strengthen its reliability for multimedia forensics.

Comments 5: "Longitudinal Studies: Conducting longitudinal studies to evaluate the model's performance over time and with evolving inpainting techniques could provide insights into its adaptability and long-term effectiveness."

Response 5: Thank you for the suggestion. We have indeed tested the model against three different inpainting methods (LAMA + MAT + ZITS), as detailed in Appendix 1 where the results can be reviewed visually and in Chapter 4.4. This evaluation provides preliminary insights into the model’s adaptability to varying inpainting techniques. Expanding this analysis over time with additional methods will be a focus in future work to further assess the model's long-term effectiveness

Reviewer 3 Report

Comments and Suggestions for Authors

The authors propose an improved approach to detecting inpainting forgeries based on several methods that are well established in the literature. The manuscript is well written and only a minor point due to cross-referencing has prompted out in pages 2 and 9. The state-of-the-art shows a strong validity of the employed methods. My only comment is the lack of comparison with other methods but leave to the authors such a suggestion. The github link does not exist.

Author Response

Comments 1: The authors propose an improved approach to detecting inpainting forgeries based on several methods that are well established in the literature. The manuscript is well written and only a minor point due to cross-referencing has prompted out in pages 2 and 9. The state-of-the-art shows a strong validity of the employed methods. My only comment is the lack of comparison with other methods but leave to the authors such a suggestion. The github link does not exist.

Response 1: Thank you for your feedback. We have compared our approach with three inpainting methods and four other detection methods to validate the effectiveness of our proposed model. Additionally, the GitHub link is now active, and we will soon begin uploading the full code, datasets, images, results, and pre-trained models to ensure full reproducibility and transparency of our work.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Thanks for the authors’ responses. Most concerns have been solved in the revision. One more thing is about the typos in the manuscript. For example, the uppercase letters in the title of section 3 are not used consistently with other sections like section 2; In Equation 2 and 4, the same variable is represented with the same symbol; Figure 6 needs to be laid in the center; A few more textual descriptions may be added to explain more cases in Appendix; The references [23] and [24] seem to have wrong formatting.

Comments on the Quality of English Language

Some.

Author Response

Comments 1: "One more thing is about the typos in the manuscript. For example, the uppercase letters in the title of section 3 are not used consistently with other sections like section 2; In Equation 2 and 4, the same variable is represented with the same symbol; Figure 6 needs to be laid in the center; A few more textual descriptions may be added to explain more cases in Appendix; The references [23] and [24] seem to have wrong formatting."

Response 1: Thank you for your thoughtful review and constructive feedback. We have carefully addressed each of the points you raised, as outlined below:

• Consistency in capitalization (e.g., "Adaptive noise-aware texture module") has been ensured throughout the paper.
• The issue in Equation (4) has been corrected.
• Figures 6, 7, and 8 have been centered for improved alignment and presentation.
• In the Appendix, we have added descriptions of the images, results, and their interpretations for greater clarity.
• The font inconsistencies in the references section have been resolved.

Reviewer 2 Report

Comments and Suggestions for Authors

Thank you for all addressed comments performed. The authors have improved the manuscript. However, you must explain all results, that means, explain the results of all inpaint images even those within the appendix. 

Author Response

Comments 1: "However, you must explain all results, that means, explain the results of all inpaint images even those within the appendix. "

Response 1:  Dear reviewer, we have added comprehensive information in the Appendix, where we provide explanations for the results of all original, inpainted, and detected images.