Full Convolutional Neural Network Based on Multi-Scale Feature Fusion for the Class Imbalance Remote Sensing Image Classification

Round 1

Reviewer 1 Report

Accept as it is.

Author Response

感谢您对我们工作的肯定和鼓励。

Reviewer 2 Report

The paper is a well-designed article related with remote sensing image segmentation with samples imbalance is always one of the most important issues.  

Specifically, authors provide an improved full-convolution neural network with loss function-based solution of samples imbalance. Also  their experimental results show that the improved neural network   can utilize the spectral information of high-resolution remote sensing images, as well as to keep its rich spatial information.

 The paper contains extensive introductory and bibliographical material. Also cases studies and experimental results  are well written and well explained. The use of English is good.
There are only a few typos. The overall presentation could be improved. A good check on format will enhance readability.
In general, there is a good description of the topic.
All in all, I suggest ACCEPTANCE WITH MINOR REVISION.

Author Response

Thank you for your affirmation and encouragement of our work. In response to your insightful criticism and advice, we have done spelling and grammar checks for correct to improve the readability of the article. All corrections have been highlighted in the paper.

Reviewer 3 Report

This paper studies the class imbalance remote sensing image classification problem using DeepLab V3+ model. The self-constructed high-resolution remote sensing image dataset is interesting, and is used to verify the performance of the DeepLab V3+ model together with the GID dataset. Generally, the paper is not well organized and written, and the novelty is not significant. The detailed comments are as follows:
1. This paper needs careful proofreading. There are many grammar mistakes and ambiguous expressions, such as what are the exact meanings of parameter k in formula (3) and ‘empty convolution’ in line 247?
2. I noticed this paper mainly focuses on depicting existing methods such as DeepLab V3, ASPP, and FCNN et al. A combination of existing commonly used methods do not bring any effective novelty because they have been already introduced in this area.
3. More details about the network optimization should be included, such as the convergence and hyper parameters analyses. A thorough discussion on why the proposed architecture can outperform the existing SOTA is needed.
4. The experimental verifications are not very convincing. It's better to conduct some ablation studies to verify the benefits of each basic component in the proposed network.

Author Response

We thank you for your careful read and thoughtful comments on previous paper. We have carefully taken your comments into consideration in preparing our revision. The following summarizes how we responded to your comments.

Point 1. This paper needs carefulproofreading. There are many grammar mistakes and ambiguous expressions, suchas what are the exact meanings of parameter k in formula (3) and ‘emptyconvolution’ in line 247?

Response: Thank you for your insightful criticism and advice. As a response, we have done spelling and grammar checks for correct to improve the readability of the article. All corrections have been highlighted in the paper. In addition， the k in formula (3) and the atrous convolution in line 247 have been further explained and highlighted.

Point 2. I noticed this paper mainlyfocuses on depicting existing methods such as DeepLab V3, ASPP, and FCNN et al.A combination of existing commonly used methods do not bring any effective novelty because they have been already introduced in this area.

Response: We sincerely thank you for your insightful criticism and advice. In response to your confusion, we make the following explanation. In remote sensing image processing task, remote sensing image segmentation with imbalanced samples has always been an important research issue. Traditional remote sensing image segmentation mainly relies on the prior knowledge of the spectrum of ground objects, without ability of capturing rich spatial and texture features from remote sensing images. By contrast, we proposed an improved deep neural network model to obtain multi-scale feature information of ground objects, which makes up for the traditional reliance on prior knowledge in the classification of unbalanced remote sensing images, that is, new methods are used to solve old problems.

Point 3. More details about the networkoptimization should be included, such as the convergence and hyper parametersanalyses. A thorough discussion on why the proposed architecture can outperformthe existing SOTA is needed.

Response: （1）We sincerely thank you for your insightful criticism and advice. As a response, we have added new detailed information about network optimization in the article, including pre-training, normalization methods, and training epochs selection, which are highlighted in 3.3 Training Protocol.（2）In response to your confusion, we make the following explanation. The improved network model integrates the Encoder-Decoder structure and combines low-level spatial information to make the classification boundary more accurate. Meanwhile, multi-scale context information is obtained by multi-scale atrous convolution, which enables the model to obtain more accurate semantic features and thus obtain more accurate classification results. Meanwhile, multi-scale context information, obtained from multiple sizes of atrous convolution, makes the model achieve more accurate semantic features and classification results.

Point 4. The experimental verifications are notvery convincing. It's better to conduct some ablation studies to verify thebenefits of each basic component in the proposed network.

Response: We sincerely thank you for your insightful criticism and advice. In response to your confusion, we make the following explanation. Based on exisiting network architecture, we proposed improved loss function-based solution of samples imbalance. With the same network architecture, we compared the different effect of several loss functions. Our proposed loss function outperforms others, and it proved the effectiveness of ours.

Round 2

Reviewer 3 Report

The authors have addressed all the concerns, and the manuscript can be considered for acceptance in RS. 

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

This paper provides a new methodological guidance to sample imbalance correction. The established data resource can be a reference to further study in the future.

In general, the contribution is not very high because the “class weight” strategy is a widely used technique in many works within RS field, for instance in object detection. However, I accept the paper with major concerns:

- Please, provide the code and the datasets for the scientific community.
- I think this sentence is wrong “In the paper, the Mixed precision training [59] is used to improve the network efficiency. The conventional convolution network using 32-bit floating-point numbers to carry model weights, while hybrid network combines 32-bit and 16-bit semi precision floating-point numbers together.” I recommend the authors to read some papers about mixed precision and explain it correctly.
- The are a lot of datasets for Remote sensing image segmentation, in order to make a fair comparison with some published methods, I recommend to make a comparison using well-known benchmarking datasets.
- Finally, I recommend to update the references, they are very old. In this sense, I recommend to include:
https://doi.org/10.3390/rs12020207
https://doi.org/10.3390/rs12081257
https://doi.org/10.1016/j.isprsjprs.2019.09.006
https://doi.org/10.1016/j.isprsjprs.2019.02.009

Reviewer 2 Report

This paper is well structured and written in good manner.The main problem I have with this paper is that I could hardly grasp the significance of this study. The main motivation, if I understood correctly, is as follows: Authors added land use labels to the satellite data and then created an original dataset consisting of 755 images. Then, the images were classified by U-Net, PSPNet, ICNET, DeepLab V3+ Mobilenet, DeepLab V3+ and DeepLab V3+ Mixed loss function, and the results were evaluated. The best classification accuracy was DeepLab V3+ Mixed loss function.

The original data set was created by authors. The dataset was imbalanced, with a two-digit difference between the largest and the smallest at the pixel level comparison. However, the models used for these data classifications are the existing ones and their slightly arranged models, and the one with the best results was the DeepLab V3+ Mixed loss function. This paper claims that "the proposed method help solve samples imbalance problem and improve classification accuracy for small class samples by mixed use of loss function. "

I know the difficulty of creating a dataset myself. I also know that it is difficult to obtain high accuracy in imbalance classification. But do findings this study solve any remote sensing problem? Does this study provide any answer to deep learning question of data imbalance? So, what is exactly the benefit of this study? This paper found something that will help DeepLab V3+ user and remote sensing researcher who want to use DeepLab V3+, although the results are pretty weak as journal paper levels, only with some impact of applying slightly changed DeepLab V3 + model to the original imbalanced dataset.