The Assessment of More Suitable Image Spatial Resolutions for Offshore Aquaculture Areas Automatic Monitoring Based on Coupled NDWI and Mask R-CNN

Abstract

Wide-scale automatic monitoring based on the Normalized Difference Water Index (NDWI) and Mask Region-based Convolutional Neural Network (Mask R-CNN) with remote sensing images is of great significance for the management of aquaculture areas. However, different spatial resolutions brought different cost and model performance. To find more suitable image spatial resolutions for automatic monitoring offshore aquaculture areas, seven different resolution remote sensing images in the Sandu’ao area of China, from 2 m, 4 m, to 50 m, were compared. Results showed that the remote sensing images with a resolution of 15 m and above can achieve the corresponding recognition effect when no financial issues were considered, with the F1 score of over 0.75. By establishing a cost-effectiveness evaluation formula that comprehensively considers image price and recognition effect, the best image resolution in different scenes can be found, thus providing the most appropriate data scheme for the automatic monitoring of offshore aquaculture areas.

1. Introduction

Global aquaculture production has increased by 7.5% per year since 1970, creating economic benefits but also posing significant environmental challenges [1] which has caused severe pollution in the marine environment [2]. In recent years, due to excessive aquiculture, water pollution has become more and more serious in mariculture areas. Many countries, such as New Zealand [3], China [4], and Turkey [5] have problems with excessive aquaculture pollution. Protecting the marine ecological environment, carrying out proper planning of aquaculture areas and avoiding over-farming are essential for marine culture management. This requires real-time and high-precision monitoring data of aquaculture areas to help managers quickly discover the changes in offshore aquaculture areas and carry out aquaculture plans [6].

The method of coupling the Normalized Difference Water Index (NDWI) and Mask Region-based Convolutional Neural Network (Mask R-CNN) can identify remote sensing images quickly and accurately to analyze the temporal and spatial changes in aquaculture areas, explore the aquaculture rules, and provide early warning to illegal aquaculture problems in prohibited and restricted areas, which is an essential means for automatic monitoring of aquaculture areas. However, the resolution of remote sensing images greatly influences the recognition effect. At present, the primary remote sensing data sources used to identify the mariculture area are multispectral satellite remote sensing images and microwave remote sensing images [7]. The multispectral satellite remote sensing images mainly include Spot, GF-1, GF-2, and Landsat [4,8,9,10,11]; while microwave remote sensing images mainly include Radarsat-2, GF-3, Sentinel-1, and Sentinel-2 [12,13,14,15,16,17]. The spatial resolution of diverse remote sensing data sources is quite different, and the prices and effects they can achieve are naturally different. For example, the spatial resolution of GF-1 can reach 2 m, but the price of one view is about USD 230; while the spatial resolution of Landsat-8 is 30 m and can be obtained for free. The higher the resolution of the data, the higher the accuracy of the monitoring. Higher-resolution data can reflect the changes in the mariculture area in more detail, but it is also more expensive and time-consuming to process. However, many current studies have detected the mariculture area without considering the effect of the price and resolution of the image on the results analysis and comparison [9,18,19]. There are mainly two kinds of marine aquaculture areas: raft culture area (RCA) and cage culture area (CCA) [20]. The size of these marine aquaculture areas ranges from a few meters to tens of meters. A large amount of image data is used to monitor the change in the aquaculture area in real-time, so it is necessary to balance the price and practical effect of remote sensing images.

To solve the above problems, according to the characteristics of offshore aquaculture areas, this study used remote sensing images with different resolutions to carry out experiments and established cost-effectiveness evaluation formulas in different scenes, to identify the image resolution that is more suitable for monitoring aquaculture areas.

2. Materials and Methods

2.1. Study Area and Data Sources

2.1.1. Study Area

The mariculture area of Sandu’ao in Fujian Province was selected as the study area. The area ranges from 119°28′8″ to 120°9′44″E and 26°21′34″ to 27°0′24″N, as shown in Figure 1. Fujian Province is located on the southeast coast of China with a sea area of 136,000 square kilometers, a land coastline of 3752 km, and numerous harbors. There are six deep-water harbors from north to south, including Shacheng Port, Sandu’ao, Luoyuan Bay, Meizhou Bay, Xiamen Port, and Dongshan Bay. A total of 366,000 plastic fishing rafts and more than 3100 deep-water storm-resistant cages have been built. In 2018, the total output of the top ten aquatic products was 3.38 million tons, and the output of rhubarb fish, abalone, kelp, laver, and oyster ranked first in China [21]. However, due to the inability to monitor the status of local aquaculture areas in real-time, problems such as over-cultivation and encroachment of non-farming areas are common, resulting in severe marine pollution [22]. Therefore, in order to meet the government’s demand for fishery development planning and build a modern fishing port system with a beautiful environment and orderly management, it is necessary to carry out dynamic monitoring of this area.

2.1.2. Data Sources

The GF-1 remote sensing image data were used in this study. The data sources and contents are shown in

Table 1. Details of experimental data.

Source	Format	Time Range	Space Range	Spatial Resolution/m
GF-1	.tif	13 June 2020	119°28′8″–120°9′44″E, 26°21′34″–27°0′24″N	2

.

In the stage of model training and validation, experiments were carried out using GF-1 remote sensing image data of Sandu’ao mariculture area. The image was acquired on 13 June 2020, and the spatial resolution after image fusion processing was 2 m.

2.2. Method

2.2.1. Data Pre-Processing

Production of Training and Test Sets

Remote sensing images of GF-1 were cropped into 80 samples of 500 × 500 pixels, 64 of which were used for training and 16 for testing. The boundaries of RCA and CCA in the samples were finely marked by manual visual interpretation. The ground truth was obtained, in which the colored block was the aquaculture area, and the black block was the background. As the GF-1 image was a 10-bit depth and 4-band image, it was converted into an 8-bit RGB image for input to the network.

Due to the small size of the dataset, data augmentation was carried out to prevent overfitting in the training network, improve the robustness of the classifier to different sensors, atmospheric conditions, and lighting conditions, and improve the generalization ability of the model. By adding Gaussian noise, Gaussian blur, and adjusting the contrast, 64 images used for training were expanded to simulate the image state under different influences so that the number of training samples was increased to 256 and the ground truth corresponding to the samples was generated.

Resampling of Multi-Resolution Data

In order to investigate the influence of different resolutions on the monitoring effect, images with a resolution of 2 m were processed. According to the spatial resolution of satellite images commonly used in this kind of research, low-resolution images of 4 m, 10 m, 15 m, 20 m, 30 m, and 50 m were simulated, and the ground truth corresponding to the samples was generated using bilinear down-sampling.

2.2.2. Model Training and Validation

In order to make full use of existing resources and improve model performance, the transfer learning method was used to train the pre-training model on the large public COCO dataset [23]. On this basis, training samples and ground truth were input into the model.

The model effect was quantified by using the test set. The 16 aquaculture area recognition results were compared with the ground truth. The precision, recall, and F1 score were calculated using pixels as the basic calculation unit, which is different from the evaluation method of object detection algorithms [24], to evaluate the model performance on two categories of aquaculture area. The precision is mainly used to evaluate the correct recognition rate of a category. The recall is mainly used to evaluate how many pixels in a category were correctly recognized. The F1 score is an accuracy index calculated as the harmonic mean of the precision and recall. Harmonic mean is one of several averages used in mathematics, more suitable for ratios (such as precision and recall) than the traditional arithmetic mean. These three indicators are defined as follows:

$$Precision=\frac{TP}{TP+FP}$$

(1)

$$Recall=\frac{TP}{TP+FN}$$

(2)

$$F1 score=\frac{2\times Precision\times Recall}{Precision+Recall}$$

(3)

where TP is the number of correctly identified pixels in ground-truth, FP is the number of misidentified pixels that do not exist in ground-truth, and FN is the number of undetected pixels in ground-truth.

2.2.3. Cost-Effectiveness Evaluation

To balance the cost and monitoring effect in automatic monitoring of aquaculture areas, the cost-effectiveness evaluation formula was established to evaluate the image resolution more suitable for monitoring of aquaculture areas.

Cost Analysis

The cost of automatic monitoring mainly includes the time consumed and the cost of purchasing remote sensing images of monitored areas. Time consumed includes the time to train the model for identification and to identify the aquaculture area. Considering that the model only needs to be trained the first time it is used and does not need to be repeated in subsequent monitoring, and that the automatic monitoring of aquaculture areas is less demanding in terms of real-time, the cost of automatic monitoring of aquaculture areas is set as the cost of purchasing images per square kilometer in USD, which is recorded as C.

Effectiveness Analysis

The precision, recall, and F1 score were mainly used to evaluate the recognition effect of aquaculture areas, while the F1 score is the synthesis of the first two indicators, so the effectiveness of automatic monitoring is set as the average of F1 score of two types of aquaculture areas, which is recorded as E.

Comprehensive Performance Evaluation

The cost and effectiveness of automatic monitoring in aquaculture areas are comprehensively analyzed to evaluate the comprehensive performance of different remote sensing images used for automatic monitoring, which is recorded as P. The calculation formula is:

$$P=-\alpha C+\beta E$$

(4)

where α and β are weight parameters, both positive numbers. Different parameters can be set in different scenes. The best image resolution for monitoring the aquaculture area can be obtained by calculating and comparing the comprehensive performance of different images.

3. Results

3.1. Model Validation

The model training ended after 100 epochs, then evaluation indicators were calculated quantitatively by using the test set, as shown in

Table 2. Precision, recall, and F1 score of test samples in two types of aquaculture area.

Indicator	Raft Culture Area	Cage Culture Area
Precision	0.890	0.904
Recall	0.839	0.908
F1 score	0.863	0.906

. The precision, recall, and F1 score of the cage culture area are 0.014, 0.069, and 0.043 higher than those of the raft culture area, respectively, which may be related to the clearer boundaries of the cage culture areas.

The precision, recall, and F1 score for the identification of aquaculture area in different provinces in studies of Liu et al. [4] were in the range of 0.79–0.98, 0.71–1.00, and 0.83–0.91, respectively, and in studies of Cui et al. [18] were around 0.89. The model’s accuracy constructed in this study reached the level of other current studies, indicating that this model is reliable. In the studies of Liu et al. [4], the F1 score of the extracted aquaculture area in Fujian using 15 m resolution data was 0.83. Compared with the F1 score of the extracted aquaculture area in Sandu’ao, Fujian Province, using higher resolution in this study allows us to obtain higher accuracy results correspondingly by using higher resolution images.

Combining NDWI and Mask R-CNN, the extraction results of the aquaculture area with 2 m resolution data in Sandu’ao area in June 2020 are shown in Figure 2; in which blue boxes represent RCA, and red boxes represent CCA, and most of the RCA and CCA are successfully identified. A total of 10234 raft culture areas were extracted, with an area of about 66.19 km²; and 4245 cage culture areas were extracted, with an area of about 25.33 km². The actual number of aquaculture areas should be higher due to the influence of waves or turbid waters, and because several aquaculture areas were combined when the culture density was high. Identification results of several boxed locations in the study area in Figure 2 show that this method can accomplish the identification of two types of aquaculture area. The boundary of the aquaculture area was accurate, and there were fewer missed and false detections.

In addition, we also used other low-resolution images to identify aquaculture areas. Several 15 m resolution Landsat-8 images in Xiapu area of Sandu’ao were identified using this model, and the results are shown in Figure 3. As can be seen from the figure, the model can identify the overall situation of aquaculture areas for low-resolution images; however, it cannot accurately reflect the area and quantity of aquaculture areas so it is necessary to explore the influence of image resolution on the performance of the model.

3.2. The Impact of Different Resolutions on Model Performance

The model trained with different resolution data was used to identify samples with the corresponding resolutions, and the precision, recall, and F1 score were calculated and are shown in Figure 4.

Figure 4 shows the variation in precision, recall, and F1 score versus resolution for raft culture area and cage culture area. It can be seen that with the decrease in spatial resolution of samples, the F1 score shows an overall downward trend. For the raft culture area, the F1 score reaches 0.863 at 2 m resolution and drops to 0.433 at 50 m resolution, with a decrease of 49.83%; while it is 0.855 at 4 m resolution, with a decrease of only 0.93%. For the cage culture area, the F1 score reaches 0.906 at 2 m resolution and decreases to 0.349 at 50 m resolution, which decreases by 61.48%, while it is 0.894 at 4 m resolution, which only decreases by 1.32%. Similarly, the change rates of precision and recall of the raft culture area and cage culture area also have a significant span, with the precision decreasing from 0.890 and 0.904 to 0.393 and 0.266, respectively, and the recall decreasing from 0.839 and 0.908 to 0.481 and 0.510 respectively.

The results in Figure 4 indicate that the extraction efficiency of aquaculture areas depends on the spatial resolution of the used images. With better resolution (2–4 m), extraction efficiency is good and rather stable. Starting from resolutions larger than 4 m, performance gradually decreases, and from resolutions above 20 m, the model’s performance drops rapidly. Monitoring the actual situation of the aquaculture area by using such low-resolution remote sensing images is difficult or impossible.

3.3. Cost-Effectiveness Analysis of Aquaculture Monitoring

In the automatic monitoring of aquaculture areas, the effect of remote sensing images with different resolutions is quite different, so it is necessary to analyze the cost and effectiveness. Based on the results obtained in the experiment of resolution influence on model performance, the polynomial fitting function was established to obtain the average value of the F1 score for two types of aquaculture areas with different resolutions. The fitting function is:

$$y=1.534\times {10}^{-6} x^3-9.722\times {10}^{-5} x^2-0.009282 x+0.9063$$

(5)

where x is the spatial resolution of the image and y is the average value of the F1 score for two types of aquaculture areas identified by the image.

The SPOT series satellites with a wide range and rich spatial resolution were selected as the price reference, and the effect was obtained based on the fitting function. Considering the cost, as the importance of each factor may vary in different scenarios, the cost-effectiveness of three types of chargeable images in different scenarios was evaluated, as shown in

Table 3. Cost-effectiveness evaluation under different scenarios, where α and β are weight parameters, C is the cost, E is the effectiveness, and P is the comprehensive performance.

Spatial Resolution/m	Cost/(USD/km2)	Effect	Price First			Effect First			Balancing
			αE	βC	P	αE	βC	P	αE	βC	P
2.5	2.361	0.883	0.708	0.883	0.175	0.142	1.766	1.624	0.142	0.883	0.741
5	1.417	0.858	0.425	0.858	0.433	0.085	1.716	1.631	0.085	0.858	0.773
10	1.102	0.805	0.331	0.805	0.474	0.066	1.610	1.544	0.066	0.805	0.739

.

3.3.1. Price First

In view of the two factors of price and effect, price is more important when in the scene of price priority. Set the weight α as 0.3 and β as 1 and evaluate the comprehensive performance of each image according to Formula (4), as shown in Table 3. It can be seen that under the condition of price priority, images with 10 m resolution can obtain relatively good results at a lower price, which is the best choice in this scene.

3.3.2. Effect First

When in the scene of effect priority, the effect is more important. Set the weight α as 0.06 and β as 2 and evaluate the comprehensive performance of each image according to Formula (4), as shown in Table 3. It can be seen that when the effect is given priority, images with 5 m resolution becomes the best choice for this scene with its excellent effect and relatively low price.

3.3.3. Balancing Price with Effect

For the two factors of price and effect, when they are in a balanced scenario, the importance of the two is considered in a balanced way. Set the weight α as 0.06 and β as 1 and evaluate the comprehensive performance of each image according to Formula (4), as shown in Table 3. The results show that in this scenario, the price and effect of images with 5 m resolution are moderate, which is the best choice in this scene.

4. Discussion

In recognition of the aquaculture area, with the decrease in the spatial resolution of image data, the actual area represented by a single pixel increases and the features of the aquaculture area gradually blur; more and more boundary parts of the aquaculture area are mixed with the background into the same pixel, resulting in a poor recognition effect. In addition, because there is some conflict between precision and recall, and when more pixels are identified, there may be more errors; therefore, the precision is lower and the recall is higher. When higher accuracy is needed, smaller pixels are identified; therefore, the precision is higher and the recall is lower. That is why the precision suddenly becomes lower at 10 m resolution in the cage culture areas, which is different from the overall change trend. However, the F1 score obtained by the combination of the precision and recall shows a relatively smooth decreasing trend.

Because the boundary of the cage culture areas is clearer but the features are complex, and the boundary of the raft culture areas is blurred but the color tone is uniform, the recognition effect of cage culture is slightly better than that of raft culture when the spatial resolution is high. With the gradual decrease in spatial resolution, the boundary information of the aquaculture area is gradually blurred. It can be seen from Figure 5 that the recognition effect of cage culture decreases slightly faster than that of raft culture, two indicators are below 0.7 at 20 m resolution for both raft culture and cage culture, so the recognition effect cannot meet the needs of monitoring at this time. Considering the image recognition accuracy of the two types of aquaculture area under different resolutions, the images of 15 m and above resolution should be selected for automatic monitoring of aquaculture areas when the funding problem is not considered to reflect the changes in the aquaculture area.

Considering that the effect of remote sensing images with different resolutions is quite different, it is necessary to analyze the cost and effect when considering the cost. Based on the analysis, the best images in different scenes can be quantified to obtain the best data scheme for automatic monitoring of offshore aquaculture areas. This study explores the cost-effectiveness analysis method of aquaculture area monitoring. In Formula (4), the selection of parameters considered the requirements of the relevant government budget and management departments, consulted the opinions of relevant experts and managers, and referred to the research of Zhang et al. [25]. Finally, three parameter selection schemes under different scenarios and requirements were obtained: taking SPOT series satellites as an example, when focusing on cost, images with 10 m resolution can be selected first, followed by 5 m resolution; when paying attention to effect, images with 5 m resolution can be selected first, followed by 2.5 m resolution; when comprehensively considering the performance, images with 5 m resolution can be selected first, followed by those with 2.5 m and 10 m resolution. This scheme is proposed for the Sandu’ao area and can also be extended to other marine aquaculture areas according to the needs of users. In addition, this cost-effectiveness analysis method can also be extended to other image classification applications, which will bring more reference directions for the selection of image classification schemes.

5. Conclusions

In order to assess the suitable resolution of spatial images for automatic monitoring in offshore aquaculture areas, this study experimented with images of different resolutions and carried out a cost-effectiveness evaluation accordingly to explore the most suitable data scheme for automatic monitoring. First, the identification method of coupling NDWI and Mask R-CNN was trained and verified. Secondly, the image resolution required for aquaculture area monitoring was discussed by comparing different extracting results among 2 m, 4 m, 10 m, 15 m, 20 m, 30 m, and 50 m resolutions. It was found that at least 15 m resolution is necessary for aquaculture area extracting. On this basis, SPOT series satellites can be taken as an example to analyze the cost-effectiveness of automatic monitoring in aquaculture areas; the best image resolution in different scenes is obtained, which provides the best data scheme for the automatic monitoring of offshore aquaculture areas.

The method proposed in this study has some limitations. The selection of cost and benefit items is not comprehensive enough; for example, without considering the different contributions of the two types of aquaculture areas, the average value of F1 score is directly used as the effect item. Further research in the field of marine aquaculture will be carried out in the future, and the allocation of cost and effect items in the automatic monitoring will be explored more comprehensively to obtain a more scientific data scheme and promote the development of aquaculture areas monitoring.