Segmenting Purple Rapeseed Leaves in the Field from UAV RGB Imagery Using Deep Learning as an Auxiliary Means for Nitrogen Stress Detection

Abstract

Crop leaf purpling is a common phenotypic change when plants are subject to some biotic and abiotic stresses during their growth. The extraction of purple leaves can monitor crop stresses as an apparent trait and meanwhile contributes to crop phenotype analysis, monitoring, and yield estimation. Due to the complexity of the field environment as well as differences in size, shape, texture, and color gradation among the leaves, purple leaf segmentation is difficult. In this study, we used a U-Net model for segmenting purple rapeseed leaves during the seedling stage based on unmanned aerial vehicle (UAV) RGB imagery at the pixel level. With the limited spatial resolution of rapeseed images acquired by UAV and small object size, the input patch size was carefully selected. Experiments showed that the U-Net model with the patch size of 256 × 256 pixels obtained better and more stable results with a F-measure of 90.29% and an Intersection of Union (IoU) of 82.41%. To further explore the influence of image spatial resolution, we evaluated the performance of the U-Net model with different image resolutions and patch sizes. The U-Net model performed better compared with four other commonly used image segmentation approaches comprising support vector machine, random forest, HSeg, and SegNet. Moreover, regression analysis was performed between the purple rapeseed leaf ratios and the measured N content. The negative exponential model had a coefficient of determination (R²) of 0.858, thereby explaining much of the rapeseed leaf purpling in this study. This purple leaf phenotype could be an auxiliary means for monitoring crop growth status so that crops could be managed in a timely and effective manner when nitrogen stress occurs. Results demonstrate that the U-Net model is a robust method for purple rapeseed leaf segmentation and that the accurate segmentation of purple leaves provides a new method for crop nitrogen stress monitoring.

1. Introduction

Biotic and abiotic stresses such as plant diseases, low temperature, and deficiencies of mineral elements can greatly affect the crop production and yield [1,2]. Crops can usually exhibit phenotypic and physiological changes to respond to adverse conditions. Leaf purpling is a common phenotypic change caused by environmental stresses in many crops such as rapeseed, wheat, corn, and rice [3]. Leaf purpling is mainly due to lack of nitrogen (N), phosphorus (P), potassium (K), or other nutrients causing the stagnation and accumulation of anthocyanin, which is usually red or purple in the leaf tissues [4]. Anthocyanins as water-soluble pigments found in all plant tissues can be used as a protective layer to improve the resistance to cold, drought, and disease, thereby minimizing the permanent damage to the leaves [5], but stresses can cause a temporary inhibition of growth [6]. Previous studies of purple leaves mostly focused on their ecological and physiological significance [1,7] and genomics research [3,4]. However, compared with green leaves, purple leaves have attracted little attention in phenotype monitoring, crop management, and yield estimation in agriculture, due to shorter appearance periods and smaller areas. As a phenotypic trait, purple leaves can intuitively respond to stresses, and crops could be treated by timely site-specific application of remedies. The ability to accurately and efficiently segment purple leaves in crops may greatly facilitate crop phenotype analysis, monitoring, and management.

Purple leaf segmentation is difficult due to the complexity of the field environment and differences in leaf size, shape, texture, and color gradation. Plant segmentation using images based on the color space is a traditional target segmentation method [8]. For example, Tang et al. used the HSeg method based on the H-component of the hue-saturation-intensity (HSI) color space to separate tassels from corn plants [9]. However, the segmentation accuracy was affected greatly by the illumination. Bai et al. proposed a morphology modeling method to establish a crop color model based on lightness in the Lab color space for crop image segmentation [10]. The experimental results showed that the method was robust to the lighting conditions, but its dependence on color information alone could lead to incomplete extraction.

In the past few decades, machine learning methods such as support vector machine (SVM) [11,12], random forest (RF) [13], and artificial neural network (ANN) [14] have been used widely in plant segmentation. Deep learning is a relatively new area of machine learning where multiple processing layers are employed to learn representations of complex data, and it has dramatically achieved state-of-the-art results in agriculture [15,16,17]. The fully convolutional neural network (FCNN) for semantic segmentation work has demonstrated its great capacity in plant segmentation due to rich feature representations and end-to-end structure [18,19]. Nevertheless, the FCNN-based models are limited by low-resolution prediction because of the sequential max-pooling and down-sampling structure. Subsequently, some sophisticated network architecture emerged, such as SegNet [20], U-Net [21], and DeepLab [22], which can carefully address the issue through the encoder and decoder operation and have shown huge potential in plant segmentation [23,24,25]. For example, Sa et al. used a complete pipeline for semantic sugar beet and weed segmentation using multispectral images obtained by UAV. The model with nine channels significantly outperformed a baseline SegNet architecture with RGB input [19]. Milioto et al. proposed an end-to-end encoder-decoder network, which can accurately perform the pixel-wise prediction task for real-time semantic segmentation of crop and weed [26].

Low-altitude remote sensing using unmanned aerial vehicles (UAVs) has developed rapidly for applications in precision agriculture in recent years [27,28,29], due to their ability to capture high-resolution imagery with low-cost portable sensors and their flexibility for quick image acquisition. However, UAVs have been mostly used for object detection [30,31] and image classification [32,33] in deep learning. Pixel-wise plant segmentation based on UAV images is a big challenge due to limited spatial resolution, small object size, and complex background features, compared with images obtained from the ground platform.

In this study, we used a U-Net convolutional neural network architecture as a binary semantic segmentation task of purple rapeseed leaves during the seedling stage based on UAV imagery. The U-Net model has shown significant outperformance over other deep architectures in some segmentation tasks, which is more suitable for target segmentation with uncertain size, small resolution, and complicated background [21,34,35]. The specific objectives of this study were to (1) develop a novel method for efficiently assessing and quantifying the crop stress based on UAV imagery and compare the performance with commonly used deep learning algorithms (SegNet), traditional machine learning architecture (RF and SVM), and color space method (HSeg), (2) explore the regulation of image spatial resolution on the selection of sample size, and (3) verify the ability of using purple leaves to monitor crop nitrogen status.

2. Materials and Methods

2.1. Study Area

The study area was located at the experimental base of Huazhong Agricultural University (30°28′10′′N, 114°21′21′′E) in Wuhan, Hubei, China, which lies approximately in the center of the Yangtze River basin. The study site has a humid sub-tropical monsoon climate and the main physicochemical properties of the soil are presented in

Table 1. Main physicochemical properties of the soil in the study area.

Texture	pH	Organic Matter (g/kg)	Available N (mg/kg)	Available P (mg/kg)	Available K (mg/kg)
Silt clay loam	6.71	24.16	133.12	17.16	145.89

. In the Yangtze River region, winter oilseed rape (Brassica napus L.) is widely planted, making up 89% of total rapeseed yields in China [36]. In this experiment, the conventional winter rapeseed hybrid variety “Huayouza No. 9” was used and the leaves are green in the natural state. The rapeseeds were first sown in the prepared seedbeds in September using high fertility soils, then the seedlings were transplanted into the tilled field at a density of 100,000 plants/ha in October. The experimental site with a total area of 0.29 ha was divided into three areas shown in Figure 1b. Area 1 contained 36 plots with a plot size of 11 × 2.7 m and was used for a substitution experiment where crop residues provided nutrients in a rice–rapeseed rotation system. Rice–rapeseed and corn–rapeseed rotation experiments were conducted in Area 2 and Area 3, respectively, where each area was divided into 30 plots with a plot of 4 × 5 m. In the rapeseed season, four different N fertilizer treatments (0, 75, 150, and 225 kg/ha) were randomly assigned with three replicates in Area 2 and Area 3. The samples used for the U-Net model comprised data from all three areas. In order to ensure the consistency of the experiment, the fitting model regarding the correlation between N content and area of purple rapeseed leaves only used the 60 sample plots from Area 2 and Area 3. The study area and plot arrangement are shown in Figure 1d. The table in Figure 1c shows the N fertilizer rates for each plot number.

2.2. Field Data Acquisition

In this study, nitrogen content was measured in the field by a GreenSeeker handheld crop sensor (Trimble Navigation Limited, Sunnyvale, CA, USA). Many studies have verified that the values obtained by GreenSeeker have a strong correlation with nitrogen content [37,38,39]. The GreenSeeker sensor used two LEDs as a light source to detect the reflection in the visible and NIR spectral regions. This study placed the instrument 1.2 m above the canopy and the measurement area was a circular area with a radius of 0.25 m. A total of 66 samples were collected in Area 2 and Area 3.

2.3. UAV Image Acquisition

The UAV flight task was carried out under clear and sunny weather conditions between 12:00 and 14:00 local time on December 25, 2016, when rapeseed was at the seedling stage. We employed a Matrice 600 UAV (DJI, Shenzhen, China) with a maximum payload of 6 kg, 16 min of continuous flight, and a maximum horizontal flight speed of 18 m/s in a windless environment. The platform carried a full frame digital single lens reflex camera Nikon D800 (Nikon, Inc., Tokyo, Japan) with a sensor size of 35.9 × 24 mm. The camera was equipped with a Nikon AF 50 mm f/1.4D fixed focal length lens which was vertical to the ground. The aperture, shutter speed, and ISO of the camera were set at f/5.6, 1/1000 s, and 100, respectively. A Global Position System (GPS) module and a wireless trigger were also integrated with the camera. The flight time was 10 min and flight height was set to 20 m with a forward overlap of 80% and a side overlap of 70%. During the flight, 369 images were captured every 1.0 s automatically with 36 million pixels (7360 × 4912 pixels). All the images were stored in the 24-bit TIFF format of true color.

2.4. Image Pre-Processing

Pre-processing of the 369 RGB images mainly involved image corrections and mosaicking. Image corrections included a geometric correction to reduce the effect of lens distortion and a vignetting correction to reduce the progressive radial decrease in radiance strength towards the image periphery [40,41,42]. All 369 images were used to create the orthomosaic to cover the whole study and meanwhile to reduce geometric distortion with large image overlapping [42]. The above steps were carried out using PIE-UAV software (Beijing Piesat Information Technology Co., Ltd., Beijing, China) software. In order to improve the accuracy of the mosaicking results, eight ground control points (GCPs) collected by a global navigation satellite system real-time kinematic (GNSS RTK) receiver (UniStrong Science and Technology Co., Ltd, Beijing, China) were imported into the software. The spatial resolution of the mosaicked orthoimage was 1.86 mm/pixel and the image was stored in TIFF format.

2.5. Dataset Preparation

The mosaicked orthoimage was clipped using a window with a fixed size in order to prepare the dataset for the model. The window size greatly influenced the final detection results for the purple rapeseed leaves [28] and an appropriate size provided a good trade-off in terms of the overall information regarding the purple rapeseed leaves. For exploring the optimal patch size for purple rapeseed leaf segmentation with the U-Net model, the datasets were finally resized to four different sizes of 64 × 64, 128 × 128, 256 × 256, and 512 × 512 pixels, and the information about sample capacities and clip stride is reported in

Table 2. Statistics for sample capacities and clip strides with all four patch sizes.

Patch Size	Training Set	Validation Set	Test Set	Stride
64 × 64	62,759	20,920	20,920	64
128 × 128	25,313	8438	8438	128
256 × 256	8825	2941	2941	128
512 × 512	2841	947	947	128

. The clipped dataset was randomly divided into a training set, a validation set, and a test set according to the ratio of 3:1:1. To ensure the fairness of the assessment results, the test set was only used for the final model evaluation. Two categories, with and without purple rapeseed leaves, had the same area in images for different patch sizes and the ratio of purple leaf pixels to the total pixels was 14.25%. In this study, since there is no accepted standard to identify purple leaves, three experts created the mask of purple leaves by visual interpretation through prior experience using ArcMap 10.3 (Esri Inc., Redlands, CA, USA), and then the ground truth was obtained by taking the intersection of three mask results. Additionally, the corresponding labels were cut to the same size as image patches. The labels contained two classes, corresponding to the categories with and without purple rapeseed leaves.

In this study, the flight height of the UAV was set at 20 m, which was the lowest altitude possible in practice and the resolution of the rapeseed images was 1.86 mm/pixel. However, in agricultural production, it is more practical to obtain coarser resolution images at higher altitudes, considering the cost and efficiency. Therefore, to explore the influence of image resolution on sample size selection with the specific network architecture (U-Net), the original orthoimage was re-sampled to resolutions of 3.72, 5.58, and 7.44 mm/pixel, to mimic the image acquisition at different flight heights of a UAV based platform. The resampled images were obtained through the cubic interpolation algorithm in ArcMap. Degraded images can effectively simulate the images acquired at different heights [43,44]. The three degraded images and the original orthoimage were respectively cropped into four data sets for training and test of the model.

2.6. Network Architecture

The U-Net model has shown good performance of semantic segmentation, especially in biomedical image segmentation [34,45,46]. In this study, we extracted purple leaves from UAV imagery by a U-Net model. Figure 2 shows the overall structure of the U-Net used in this study, which was a typical encoder-decoder architecture. The part of the encoder consists of repeated convolutional layer with a kernel size of 3 × 3 and each follows by a 2 × 2 max pooling with stride 2, which gradually reduces the patch sizes. The part of the decoder contains the up-sampling operations by a 2 × 2 convolution, which halves the number of feature channels. As the main difference of the U-Net model compared with other encoder-decoder architectures, the skip connections could concatenate feature maps in up-sampling with feature maps of the same level in the encoder, which could contribute the decoder to better recovering and optimizing the details of the object. Moreover, it could output images of the same sizes as the input due to the usage of the same padding for filters. In this study, Adam optimizer [47] and an initial learning rate of 10^-4 were used. The training batch size was set to 16 and the epoch was set to 100. For the model, an adjustment was carried out in the layer weight initialization procedure using He normal initializer proposed by [48]. For binary classification in this study, the U-Net was trained with a pixel-wise binary cross-entropy loss [26], which can handle the imbalanced number of pixels for each class, and a Sigmoid function as the activation layer.

The implementation of the model employed the Keras library with TensorFlow as its backend. All the experiments were carried on a Linux platform with a 4.00-GHz Intel(R) Core i7-8086k CPU, 64 GB of memory, and an NVIDIA Quadro GV100 graphics processing unit (32 GB).

2.7. Evaluation Metrics

In order to quantitatively evaluate the performance of the proposed method for segmenting purple rapeseed leaves, we calculated the precision, recall, F-measure, and Intersection of Union (IoU) [23] based on comparisons with the ground truth. The precision and recall were defined using the true positives (TP), false positives (FP), and false negatives (FN). The F-measure score was calculated using precision and recall as a powerful evaluation metric for the harmonic mean of precision and recall. IoU calculated the ratio of the intersection and union of the area in prediction and the ground truth. These measures were calculated as follows:

$$\mathrm{Precision}=\frac{TP}{ TP+FP}$$

(1)

$$\mathrm{Recall}=\frac{TP}{TP+FN}$$

(2)

$$\mathrm{F}-\mathrm{measure}=\frac{2 \times \mathrm{Precision} \times \mathrm{Recall}}{\left(\mathrm{Precision}+\mathrm{Recall}\right)}$$

(3)

$$\mathrm{IoU}=\frac{TP}{TP+FP+FN}$$

(4)

where the precision is the percentage of matched pixels in the extraction results and the recall measures the proportion of matched pixels in the ground truth. The F-measure and IoU indicate the final exponent for evaluating the accuracy of purple rapeseed leaf segmentation with the U-Net model.

2.8. Segmentation of Purple Rapeseed Leaves with Four Other Methods

To further evaluate our proposed purple rapeseed leaf segmentation method, we implemented four other commonly used image segmentation methods comprising HSeg, RF, SVM, and SegNet. In this work, all methods used the same training set and test set. In particular, the accuracy of the models using only two categories of labels for RF and SVM was low. Therefore, the original class “without purple leaves” was subdivided into five new categories: green leaves, light soil, dark soil, cement, and withered leaves as the training set for RF and SVM. The HSeg method is a traditional target segmentation method of unsupervised classification. In this study, the original image was converted to the H-component of the HSI color space, which can make a big difference between the purple leaf and the background, and then the Otsu approach [49] was used to find an optimal threshold for purple rapeseed leaf segmentation. RF and SVM are both machine learning methods that have been widely used in agriculture for yield prediction and crop classification [15]. RF is a nonparametric and nonlinear classification method with a low computational burden [50,51] and the SVM classifier divides the classes with a decision surface by using a kernel function that maximizes the margin between the classes [52]. In this study, we found the optimal parameters in terms of accuracy obtained by five-fold cross-validation through continuous adjustment. Finally, we used RF containing 80 trees which were all grown fully and unpruned, and the number of features at each node was set to the square root of the total number of features. For the SVM classifier, this study employed the Radial Basis Function as the kernel, and the penalty factor and gamma parameter value were 2 and 0.33, respectively. The structure of the SegNet model is also divided into the encoder and the decoder [20]. This model records the feature locations by pooling indices computed in the max-pooling of the encoder to optimize the production of dense feature maps in the decoder. The process of parameter tuning and samples with four sizes for the SegNet were the same as those for the U-Net. After adjusting the parameters, the optimal input size of SegNet in this study was 256 × 256 pixels.

3. Results

3.1. Segmentation Results Obtained with the U-Net Model

Figure 3 shows the loss curves for training and validating datasets in four different patch sizes, which drop sharply at the beginning and then all reach the convergence states. Figure 4 summarizes the test results of the U-Net model for the patch sizes. Comparisons of the F-measure and IoU showed that as the patch sizes increased, the values tended to increase initially and then decrease. The peak value was obtained with a sample size of 256 × 256 pixels, which provided the best F-measure of 90.29% and IoU of 82.41%. In theory, larger patches should provide more information and yield more accurate predictions. However, the finer resolution also requires a longer training time, and the added information may be redundant and then influence the model training process [34]. The clipped patches with an excessively small window size would lack sufficient features for the model and then increase the risk of over-fitting. Moreover, the patches with a small size could not complete all down-sampling processes of the U-Net model. Thus, excessively small or large sizes made it difficult to distinguish between the purple leaf and the other categories with the U-Net model. Therefore, a patch size of 256 × 256 pixels was selected as an appropriate window size for the U-Net model in this study.

3.2. Accuracy Evaluation for the U-Net Model and Four Other Image Segmentation Methods

To compare the differences between the U-Net method and the other four methods (RF, HSeg, SVM, and SegNet), the four methods employed the same training set and test set as the U-Net model. The results are shown in Figure 5. The U-Net model outperformed each of the other four methods in terms of F-measure and IoU, which were about 6% and 9%, respectively, higher than those for SegNet, the second-best method. RF and SVM had a relatively low recall, and the precision with HSeg was clearly low. However, U-Net had a good balance between the two metrics.

4. Discussion

4.1. Comparison of the Proposed Method and Four Commonly Used Methods

To facilitate more intuitive comparisons of the proposed method and the four commonly used image segmentation methods, we selected four representative test images to analyze the segmentation results obtained with the different approaches. In Figure 6, the first row represents the original RGB rapeseed images, the second row represents the corresponding ground truth, and the remaining five rows show the segmentation results obtained by the five methods. The field environment was very complex with some uncontrollable factors such as illumination and many non-target objects.

Region a contained a withered leaf that resembled a purple leaf. HSeg, SegNet, and U-Net could distinguish between them. However, there were a few incorrectly identified pixels with RF and SVM. For region b, the color gradation of the purple leaves was very different from that of general purple leaves, which led to many misclassified pixels except for SegNet and U-Net. Additionally, the appearance of some ground objects in region c interfered with the segmentation processes by SVM and RF. In region d, when the purple leaves were close to the green leaves, RF and SVM tended to misclassify the mixed pixels at the boundary of the purple leaf.

In the complex background conditions, the U-Net model had a higher capacity for purple rapeseed leaf segmentation and it obviously performed better than the other four methods. HSeg is only based on color information [8]. The color of the purple leaves could vary with changes in illumination and plant growth stage, which severely affected the segmentation results obtained using HSeg. In addition to these problems, the excessive dependence on color information could also lead to incomplete extraction and low recall values [8]. Both traditional machine learning methods (i.e., RF and SVM) create a serious salt-and-pepper phenomenon in the segmentation images. RF is time-consuming, but it is insensitive to the parameters and outliers [50,51]. RF had a difficulty distinguishing similar targets such as various leaves, shadows, and dark soil. Moreover, due to the influence of mixed pixels containing soil and leaves, RF often incorrectly identified the edges of the green leaves as purple leaves. SVM required a long time for model training and it had a problem with incomplete extraction. In general, the main problem with supervised methods is that their accuracy relies mainly on the quality of the training data set and the dimensionality of the input sample, which makes sample labeling a time-consuming process [52]. In this study, the field environment contained many non-target objects and it was difficult to obtain samples from all classes. Therefore, the number of classes in the image was much larger than the sample number, which led to low segmentation accuracy with SVM and RF. By contrast, the U-Net method exhibited higher generalizability and greater robustness during purple leaf segmentation because of its capacity to build a hierarchy of local and sparse features, whereas the traditional image segmentation methods are based on the global transformation of features [53]. Moreover, U-Net could maintain the edges and structural integrity, and it was independent of the color and shape information, thereby obtaining higher accuracy. Similarity, SegNet also had these traits, but the segmented purple leaves had non-sharp boundaries. For SegNet, the coarse outputs were produced after five consecutive down-samplings, which would have had an effect on the process of feature recovery in the up-sampling.

4.2. Influence of Image Resolution on Sample Size Selection

The images used in this study were acquired by the UAV platform. In practice, if the distance between the UAV and rapeseed plants is too close, the strong downdraft produced by the UAV makes the leaves sway and thus affects the image registration process. Thus, the restriction on the flight height made it difficult to obtain ultrahigh-resolution images. The flight height of the UAV was 20 m, which was almost the limit. In order to mimic other higher flight heights, the original orthoimage was re-sampled to resolutions of 3.72, 5.58, and 7.44 mm/pixel. Figure 7 presents the IoU results obtained for each image resolution and patch size. As the image resolution becomes coarser, the optimal accuracy tends to move towards a small patch size. On the one hand, for the same patch size, the number of background pixels was much more than the target (purple rapeseed leaf), which tended to make the trained network divide the pixels into backgrounds. To prevent this from happening, patch size could be reduced appropriately. Nevertheless, the patches with too small a size could easily lead to over-fitting of the model. On the other hand, if the target sizes were too small, the down-sampling process of the U-Net model could not complete, causing the loss of target feature extraction. Especially, the coarser the image spatial resolution, the smaller the area of the target in the image. Figure 8 shows the sizes of all purple rapeseed leaves in the original image used in this experiment, and most purple leaf sizes were between 11 × 11 pixels and 20 × 20 pixels. When the original image was re-sampled to the resolution of 7.44 mm/pixel, the mean and median of all purple leaf sizes were smaller than 8 × 8 pixels, which was not sufficient to obtain more features in Conv 4 (Figure 2). Usually, to obtain new and more complex features, a deep convolutional neural network is added with convolution layers to increase the receptive field. However, the deeper network requires a relatively larger image patch and target size. For pixel-level plant segmentation based on UAV imagery, it is difficult to meet such sample requirements. Excessive pursuit of network depth could lead to counterproductive results. For future plant segmentation work based on UAV images, this discussion can help to make an appropriate decision on sample size selection.

4.3. Relationship Between Nitrogen Content and Area Ratios of Purple Rapeseed Leaf

Nitrogen nutrition has a great effect on the growth and yield of rapeseed [37,54,55]. In this study, four N fertilizer rates were randomly applied to plots as shown in Figure 1 (Area 2 and Area 3). In order to accurately obtain the real-time N content, GreenSeeker (GS) values were measured as the ground truths of N content in 66 field samples.

Table 3. Descriptive statistics and the differences of GreenSeeker (GS) values and purple leaf ratios among four N application levels.

	N Rate (kg/ha)	Number of Samples	Max	Min	Range	Mean	CV (%)	ANOVA Results
GS values	0	23	0.59	0.405	0.185	0.474d	9.4	p < 0.05
	75	13	0.665	0.52	0.145	0.568c	6.1
	150	15	0.705	0.54	0.165	0.624b	8.4
	225	14	0.72	0.605	0.115	0.678a	5.8
Purple Leaf Ratios	0	23	0.536	0.167	0.369	0.308a	33.2	p < 0.05
	75	13	0.165	0.048	0.105	0.056b	18.3
	150	15	0.05	0	0.05	0.016c	11.2
	225	14	0.002	0	0.002	0.001c	1.0

Max = maximum, Min = minimum, and CV = coefficient of variation; mean values followed by the same letter within a column of GS values or purple leaf ratios are not significantly different from one another at 0.05 probability level.

presents the descriptive statistics for analyzing the differences in the GS values and purple leaf ratios under the four N application levels. As the ground truths of N content in this experiment, GS values increased steadily with the increase of N application level and maintained the similar range and coefficient of variation (CV). Purple leaf ratios have an increasing increment with the decrease of N application level, which led to a greater range and CV. At the same N application level, the nutrient uptake of rapeseed also varied. To further analyze the differences in GS values and purple leaf ratios among the four N treatments, the analysis of variance (ANOVA) was carried out, the p values of GS and purple leaf ratios were less than 0.05, indicting the decrease of N application induced different level of the N stress in rapeseed on the whole. Multiple comparison results using the least significant difference (LSD) method results (Table 3) showed that the GS means were significantly different among the four N levels, while the purple leaf ratio means were significantly different among all four N levels except for the two high N levels. These results indicate that the extraction method of purple leaves in this experiment was sensitive to detect N stress from 0 to 150 kg/ha. However, this method had a low sensitivity to nitrogen application at relatively high levels (over 150 kg/ha).

Results in Table 3 clearly indicate that leaf purpling was associated with N deficiency. However, there are few studies on the relationship between crop leaf area and nitrogen stress. Therefore, we performed a regression analysis to explore the specific relationship between the area of rapeseed purple leaves and N content. For rapeseed planting, the input of N is fitted to the law of diminishing returns, for which the increments of the rapeseed output will decrease while the increments of the N application input remain constant [56]. Additionally, some studies have indicated that an exponential function can measure the relationship between nitrogen content and green leaf canopy cover well [57,58]. As shown in Table 3, the area of purple leaves and the N fertilizer level had opposite trends, and thus it was reasonable to use a negative exponential model to fit the relationship between the area ratios of the purple rapeseed leaf and the N application rate. Figure 9a illustrates a linear relationship between nitrogen application level and GS values in the range of 0.3–0.8, which are effective for N content detection according to [59]. A negative exponential regression model between N content and purple leaf ratios based on the U-Net model was established in Figure 9b with the R² value 0.858. When more N was applied, the purple leaf ratio was reduced. Additionally, when the N application rate reached a certain level, no purple rapeseed leaves were observed, as shown in Figure 9c. However, monitoring crop growth by calculating the leaf area index or canopy coverage using only the green leaves is not always accurate. In particular, when the area of green leaves reached the maximum, as shown in Figure 9c–e corresponding to three different N fertilizer treatments (0, 75, and 150 kg/ha), it was difficult to assess the crop state by observing the green leaves. However, purple leaf extraction as a visible trait can be an effective auxiliary means for nitrogen stress detection.

As for the relationship between N rate and yield, low N application could lead to low photosynthetic assimilation and thus reduce grain yield, and excessive N application could cause environmental pollution and even yield decrease [60,61]. Additionally, this experiment illustrated that the exponential function could explain the relationship between N content and purple leaf ratios. For the commonly used nitrogen detection instruments in the field such as GS, it is time-consuming and laborious compared to the UAV-based method in this study. In addition, these sensors can only collect data at discrete points. Moreover, compared with the scientific-grade sensors including near-infrared and red-edge bands, which are sensitive for crops, the consumer-grade camera used in this experiment can only obtain RGB images, however, it is low-cost, easy to operate, and can obtain images with high spatial resolution [62]. In view of the above differences, the purple leaf extraction method based on UAV provided an efficient alternative method for nitrogen stress monitoring. In future experiments, we will increase the number and range of nitrogen levels for exploring the relationship between N stress and the purple leaves ratios comprehensively to find the best threshold for balancing the yield and environmental impact.

5. Conclusions

In this study, we developed a novel method, which can efficiently monitor the crop nitrogen stress as an auxiliary means based on UAV RGB imagery. We designed and implemented a U-Net method for purple rapeseed leaf segmentation at the pixel level, and identified the optimum parameters by adjusting the sample size. The experimental results demonstrated the effectiveness of our proposed method for extracting purple rapeseed leaves, with a F-measure of 91.56% and IoU of 83.19%. Compared with four other commonly used image segmentation approaches (i.e., HSeg, RF, SVM, and SegNet), the U-Net model performed better.

The resolution of the rapeseed images obtained using the low-altitude remote sensing platform was much lower than that acquired with a high-resolution camera from close range. Therefore, in this study, we discussed the importance of patch size selection and evaluated the influence of image resolution on sample size selection, and the result showed that as the spatial resolution became coarser, a relatively smaller patch size had a higher accuracy.

We found that there was a negative exponential relationship (R² = 0.858) between the area of purple rapeseed leaf ratios and the corresponding N content in this experiment. Additionally, using the low-cost consumer-grade camera based on UAV platform is more suitable for agricultural practical application. Therefore, purple leaf extraction was a feasible and effective auxiliary means for monitoring nitrogen stress. Nitrogen is a significant parameter determining the photosynthetic functioning and productivity in crops. In further research, additional nitrogen levels will be added, and purple leaf area will be assessed as a visual trait to find the optimal nitrogen threshold for balancing crop yield and environmental impact. Moreover, other crops and stress types (e.g., water stress) will be studied based on purple leaves.