Estimating Leaf Nitrogen Content in Wheat Using Multimodal Features Extracted from Canopy Spectra

Abstract

The leaf nitrogen content (LNC) of wheat is one of key bases for wheat nitrogen fertilizer management and nutritional diagnosis, which is of great significance to the sustainable development of precision agriculture. The canopy spectrum provides an effective way to monitor the nitrogen content of wheat. Previous studies have shown that features extracted from the canopy spectrum, such as vegetation indices (VIs) and band positions (BPs), have successfully achieved the monitoring of crop nitrogen nutrition. However, the features mentioned above are spectral features extracted on the basis of linear or nonlinear combination models with a simple structure, which limits the general applicability of the model. In addition, models based on spectral features are prone to overfitting, which also reduces the accuracy of the model. Therefore, we propose an estimation model based on multimodal features (convolutional features and VIs, BPs) of the canopy spectrum, which aim to improve accuracy in estimating wheat LNC. Among these, the convolutional features (CFs) extracted by the designed convolutional neural network represent the deep semantic information of the canopy reflection spectrum, which can make up for the lack of robustness of the spectral features. The results showed that the accuracy of the model based on the fusion features (VIs + BPs + CFs) was higher than that of the feature of single modality. Moreover, the particle swarm optimization–support vector regression (PSO-SVR) model based on multimodal features had the best prediction effect (R² = 0.896, RMSE = 0.188 for calibration, R² = 0.793, RMSE = 0.408 for validation). Therefore, the method proposed in this study could improve performance in the estimation of wheat LNC, which provides technical support for wheat nitrogen nutrition monitoring.

1. Introduction

Nitrogen is one of the essential elements for the growth of wheat. Quantitative assessment of leaf nitrogen content (LNC) is helpful for accurate management of nitrogen fertilizer, which can improve crop yield and quality [1]. The traditional methods used to obtain LNC include ¹⁵N isotope tracer technology and physiological and biochemical analysis [2]. However, it is difficult to universally apply the disadvantages of the above methods, such as destructiveness, high cost of analysis, and time-consuming and labor-intensive analysis. Studies have shown that spectral reflectance is closely related to leaf nitrogen content, which is an important indicator to reflect its yield and quality [3]. Therefore, it is of great significance to build a spectral inversion model for nitrogen content in winter wheat canopy leaves with high universality and high accuracy to improve monitoring efficiency.

Previous research has shown that the use of reflectance spectroscopy to monitor the LNC of wheat has achieved good research results [4]. On one hand, the analysis and processing method of the canopy spectrum can effectively extract different spectral features, including specific spectral preprocessing, selection of sensitive wavelengths, and multivariate modeling approaches [5], aiming to reduce variables and improve estimation accuracy. On other hand, various vegetation indices have been constructed to successfully monitor different physiological and biochemical parameters of wheat, including chlorophyll [6] and leaf area index [7], high-throughput phenotyping [8], nitrogen content [9], yield [10,11], and height of wheat [12]. The above research showed that the monitoring model based on the VIs has strong predictive potential. However, the canopy spectrum contains a lot of information. Estimation models constructed on the basis of a few bands tend to reduce the robustness of the model, which cannot reflect the advantages of the reflectance spectroscopy data [13].

There are still many issues associated with effectively extracting the features of the canopy spectrum to improve the performance of the model. Some scholars diagnose the nutritional parameters of vegetation by analyzing spectral position features. Among them, the position of the red edge of the spectrum was used to successfully predict the chlorophyll [14,15], water content [16], biomass [17], and nitrogen content [18] of the wheat, which indicates that the spectral band positions can improve the prediction effect. However, the choice of band was closely related to the qualitative and quantitative analysis of the target. Fan et al. proposed that it was feasible to integrate the band positions and vegetation indices to estimate the leaf nitrogen content of the wheat [19]. However, the features mentioned above were based on simple linear or nonlinear models, which belongs to spectral features easily disturbed by the external environment, thus limiting the accurate prediction of wheat LNC. Therefore, it was necessary to extract multimodal features to make the estimation model more universal.

In recent years, artificial intelligence technology has penetrated into all fields of agricultural research and has become a hot spot of modern technology. Artificial intelligence technology represented by deep learning can make full use of massive spectral data and automatically learn the features in the data, providing convenience for nutritional diagnosis and growth monitoring. Previous studies have shown that deep learning, especially convolutional neural networks, are not only good at classification, detection, and recognition, but can also be used as more general tools to facilitate the processing, analysis, and interpretation of multidimensional, heterogeneous, and multimodal datasets [20]. In earlier studies, spectra were fed intuitively into fully connected networks due to the requirement of vector-based inputs in network layers. Specifically, the raw spectral vectors are directly used in an unsupervised manner such as stack auto encoder (SAE) or deep belief networks (DBN) [21,22]. However, the convolutional features of the spectrum were rarely extracted. To this end, Yang et al. designed a variety of convolutional neural network models for deep feature extraction of spectral data and adjusted the parameters of the model [23]. Studies have shown, however, that spectral features extracted on the basis of one-dimensional convolutional neural networks effectively improve classification and recognition [24]. In particular, Zhang et al. realized the identification of tobacco origin with convolutional features extracted from 2D matrix figures of spectral using a convolutional neural network; the figures were generated by converting one-dimensional near-infrared spectral data of tobacco to two-dimensional [25]. Zhao et al. first converted the obtained one-dimensional spectral data of potato leaves into a two-dimensional matrix, then input the two-dimensional matrix map into CNN to extract the deep features (convolution features) in the spectral data. Finally, a prediction model for chlorophyll content in potato leaves was constructed [26]. Zhang et al. established a one-dimensional convolutional neural network (1DCNN) model to accurately estimate soil biochemical parameter content using the extracted convolutional features [27]. However, it was still unclear whether the performance of the estimation model of wheat leaf nitrogen content can be improved by fusing the convolutional features and spectral features of the spectrum.

Therefore, in this study we propose fusion multimodal features of the canopy spectrum, aiming to improve the estimation accuracy of the leaf nitrogen content in wheat. The objectives of this study were (1) to design a convolutional neural network to obtain deep semantic information and better express the features of the canopy spectrum, (2) to fuse multimodal features including spectral features (VIs + BPs) and convolutional features of the wheat canopy spectrum, and (3) to build an estimation model based on fusion multimodal features for wheat LNC, aiming to obtain good performance.

2. Data and Methods

2.1. Samples

One hundred twenty samples on the jointing, flowering, filling, and maturity of wheat were collected from Rugao Base, Jiangsu Province. Among them, the canopy spectrum of the wheat from 37 samples in each period was collected. Therefore, a total of 124 sample data were obtained for 4 periods. The data of each period was divided into calibration set and test set according to about 7:3. Therefore, the calibration set and the validation set include 87 samples and 37 samples, respectively.

2.2. Data Collection

2.2.1. Canopy Spectrum Collection

The canopy spectrum of the wheat was obtained using the FieldSpec Pro FR2500 field hyperspectral radiometer (American Analytical Spectral Device, ASD) in Rugao Base (120°20′ E, 32°14′ N) in 2014. The wavelength range was 350–2500 nm, and the spectral sampling interval was 1.4 nm for 350–1000 nm and 2 nm for 1000–2500 nm. The spectral resolution was 3 nm and 10 nm, respectively. The measurement was carried out at 10:00–14:00 with clear and windless conditions. The sensor probe was vertically downward and about 1.0 m from the top of the wheat canopy. The field of view of the spectrometer and the diameter of the ground field of view were 25° and 0.44 m, respectively.

2.2.2. LNC of the Wheat Collection

On the day of the canopy spectrum measurement, 20 wheat plants were destructively obtained from each plot, separated by organs, dried at 80 °C, weighed, and crushed, and the total nitrogen content of the various organs of the plants was determined by Kjeldahl method (N, %) [28].

2.3. Multimodal Features of Canopy Spectra

2.3.1. CNN and Convolutional Features

Convolutional neural network (CNN) has a multilayer deep neural network model. These layers include input layer, fully connected layer (FCL), convolutional layer (CL) for feature extraction, pooling layer (PL) to reduce feature dimensions, and activation function layer which uses activation functions to map feature values [29]. Among these layers, the convolutional layer is one of the key structures of CNN which improves learning capabilities through sparse connectivity, parameter sharing, and equivariant representations. The process of convolution generally learns the features of the image by sliding the small squares (filters or kernels) of the matrix on the entire image, so as to preserve the spatial relationship between the image pixels. Therefore, the result of multiple convolutions can obtain abstract features gradually extracted from the input image.

To extract the convolutional features of the spectrum, a convolutional neural network is designed with reference to the AlexNet structure, including five convolutional layers, three pooling layers, and two fully connected layers. The specific structure is shown in Figure 1.

As we know, the typical CNN is a two-dimensional convolutional neural network. Since the image was stored in the form of a two-dimensional matrix, any time-domain signal can be converted into a two-dimensional matrix [30]. To meet the requirements of the two-dimensional convolutional neural network input format, the one-dimensional spectral data of each sample requires matrix conversion. Therefore, for the 1811 wavelengths of the wheat canopy spectrum, 64 sensitive wavelengths were extracted by successive projections algorithm (SPA) [31], which were converted into a matrix form (64 × 64) as the input data of CNN. After operations of the convolutional layer and the pooling layer, the final output of the fully connected layer 2 (FC2) are used as the convolutional features. The parameter settings of the CNN are shown in

Table 1. Layers and output size and architecture of CNN.

CNN Structure	Input Size	Output Size
Conv1 (11 × 11, S = 4, P = 2)	64 × 64 × 3	15 × 15 × 64
Max Pooling1 (3 × 3, S = 2, P = 1)	15 × 15 × 64	7 × 7 × 64
Conv2 (5 × 5, S = 1, P = 2)	7 × 7 × 64	7 × 7 × 192
Max Pooling2 (3 × 3, S = 2, P = 1)	7 × 7 × 192	3 × 3 × 192
Conv3 (3 × 3, S = 1, P = 1)	3 × 3 × 192	3 × 3 × 384
Conv4 (3 × 3, S = 1, P = 1)	3 × 3 × 384	3 × 3 × 256
Conv5 (3 × 3, S = 1, P = 1)	3 × 3 × 256	3 × 3 × 256
Max Pooling3 (3 × 3, S = 2, P = 1)	3 × 3 × 256	1 × 1 × 256
FC1	1 × 1 × 256	512
FC2	512	256

S: stride; P: padding.

.

2.3.2. Band Positions of Canopy Spectra

The spectrum of each sample contains multiple bands. To take advantage of effective spectral data, in this study we selected the band positions related to wheat nitrogen nutrition, including red edge position, yellow edge position, blue edge position, etc., which contain richness growth information related to plant physiological and biochemical parameters. The definition and description of the parameters of band positions were shown as follows [32,33]. The maximum value of the first derivative of a blue edge (490–530 nm), yellow edge (560–640 nm), red edge (680–760 nm) are defined as Db, Dy, Dr, respectively. The maximum reflectance of a green peak (510–560 nm) is defined as Rg, and the lowest reflectance of a red well is defined as Ro. The sum of the first derivative values within the blue edge, the yellow edge, the red well are defined as SDb, SDy, and SDr, respectively. The λr, λg, λy, λb indicate red edge position, green peak position, blue edge position, respectively.

2.3.3. Vegetation Indices

The vegetation indices (VIs) were mainly used to enhance vegetation information on the basis of different spectral signals, most of which were based on the combination of linear and nonlinear characteristics in the visible-near infrared band. VIs can quantitatively reflect the vegetation growth in a certain environment, which was suitable for the dynamic monitoring of crop growth in different periods. Therefore, 17 vegetation indices closely related to the nitrogen content of the wheat canopy were selected to establish and validate the LNC estimation model. The calculation formula was shown in

Table 2. The calculation formulas for vegetation indices.

Vegetation Indices	Formula	Reference
		Chen et al., 2010 [34]
		Adams et al., 1999 [35]
		Serrano et al., 2002 [36]
		Gitelson et al., 2002 [37]
		Huete et al., 1998 [38]
		Fitzgerald et al., 2006 [39]
		Liu et al., 1995 [40]
		Rondeaux et al., 1996 [41]
		Broge et al., 2001 [42]
		Gitelson et al., 1996 [43]
		Chen et al., 1996 [44]
		Kaufman et al., 1992 [45]
		Vogelmann et al., 1993 [46]
		Gamon et al., 1992, 1997 [47,48]
		Penuelas et al.,1994 [49]
		Penuelas et al., 1995 [50]
		Merzlyak et al., 1999 [51]

where represents the spectral reflectance.

.

2.4. Feature Optimization and Regression Model

All the features involved in model prediction may produce information redundancy, resulting in inaccurate prediction results. Therefore, it is necessary to optimize features to improve the prediction performance of the model. Feature preferences are generally evaluated in terms of their importance.

To improve the computational efficiency and reduce the influence of irrelevant variables, Pearson correlation coefficient analysis was performed on the normalized sample data, and feature selection was performed by analyzing the degree of linear correlation between the two variables. In addition, random forest (RF) [52], categorical boosting (CatBoost) [53], gradient boosting (GB) [54], and light gradient boosting machine (LightGBM) [55] were used for feature optimization in this study.

2.5. Technical Route and Model Evaluation

To predict the leaf nitrogen content of wheat accurately, a prediction model integrating multimodal features of canopy spectra was proposed, including band positions, spectral feature, and convolution feature. Among them, convolutional features were extracted from spectral data using convolutional neural networks, reflecting higher-level semantic information. Other features were determined according to the reflectivity of different bands. The specific technical flow chart is shown in Figure 2.

The specific steps of this study are as follows: First, the spectral data of the wheat canopy were obtained and the band features, spectral features, and convolution features were further extracted. Among them, the obtained wheat canopy spectrum is matrix- transformed to extract canopy convolution features. Second, the obtained features are optimized using different methods (including random forest and correlation analysis methods), aiming to improve the prediction accuracy of the model. Finally, the prediction models of nitrogen content in the wheat leaves were constructed by using the optimal characteristics, including support vector regression (SVR), partial least squares regression (PLSR), and the particle swarm optimization–support vector regression (PSO-SVR) model, which were evaluated using coefficient of determination (R²), root mean squared error (RMSE), and mean absolute error (MAE) [56,57].

3. Results

3.1. Convolutional Features

Figure 1 showed the obtained canopy spectral reflectance of wheat. Among this data, the 1361–1399 nm and 1800–1999 nm bands were affected by moisture absorption, and the 2400–2500 nm may be abnormally due to the influence of the instrument operation. Therefore, a total of 1811 wavelengths as spectral sample data include 350–1360 nm, 1400–1799 nm, and 2000–2399 nm. The preprocessed spectral data were used to extract 64 sensitive wavelengths through SPA, and the constructed 64 × 64 matrix was input to the designed convolutional neural network to obtain convolutional features.

Although convolutional features are difficult to describe, we try to use heatmaps to visually represent the difference between convolutional features. The deep features extracted from the different samples shown in Figure 3 were all differentiated. Figure 3a showed that the values of the deep features of the 37 samples of validation set were between 0 and 1.2, and Figure 3b showed that the values of the deep features (convolutional features) of the 87 samples of calibration set were between -0.5 and 1.5. Therefore, feature extraction based on deep learning provides multiple feature expressions for the wheat canopy spectrum, which can improve the accuracy of the quantitative analysis model for leaf nitrogen content in wheat and provide a technical path for wheat nitrogen nutrition monitoring.

3.2. Optimization of Multimodal Features

3.2.1. Optimization of Convolutional Features

Without further dimensionality reduction, high-dimensional convolutional features will undoubtedly affect the accuracy and robustness of the model. In this study, four algorithms of AdaBoost, CatBoost, RF and XGBoost were used to screen the acquired convolutional features, and the top 50 features in order of importance were selected, as shown in the Figure 4. As could be seen from Figure 4, the convolution features optimized by the above four algorithms were different, and the relative importance of optimized features based on AdaBoost was higher than that obtained by other algorithms.

The importance of the preferred features based on XGBoost reaches 0.0794. The scores for the obtained feature importance are accumulated in descending order, and the feature variables are accumulated one by one. First the feature with the highest importance score is selected, then the feature with the top 2 importance score is selected, and so on. Eight feature variable combinations are obtained, whch are used as the input variables of the constructed PLSR to estimate the wheat LNC. Finally, the estimation results are compared, and the preferred variable features are determined according to the highest estimation accuracy. Therefore, the preferred convolutional features will be used as input to the subsequent regression prediction model.

3.2.2. Optimization of Vegetation Indices

Figure 5 shows the visualization results of the correlation coefficient matrix between the 17 vegetation indices and the LNC of wheat. The legend represents the correlation coefficient, blue indicates a positive correlation between vegetation indices and wheat LNC, and yellow indicates a negative correlation. The first row of the triangular matrix represents the correlation coefficients between the 17 vegetation indices and the nitrogen content of the wheat canopy, and the others represent the correlation coefficients between different vegetation indices. It can be seen from Figure 5 that almost all the vegetation indices were closely related to the LNC of wheat. Among these, NPCI, SIPI, and PRSI were negatively correlated with the nitrogen content of the wheat, and the rest showed positive correlation with the wheat LNC. Moreover, the correlations between the vegetation indices were strong, and the absolute value of most correlation coefficients was above 0.8, indicating that there was serious collinearity between the vegetation indices.

To reduce information redundancy, it was necessary to analyze the importance of the vegetation indices. We also used AdaBoost, CatBoost, RF, and XGBoost for feature selection of VIs. Through experimental comparison, we determined that the results from using RF were ideal. Therefore, RF was used to calculate the importance of 17 vegetation indices, and the results are shown in Figure 6. As can be seen from the Figure 6, the relative importance was distributed in the range of 1.8–6.7. The importance of the VIs was sorted descending, and the top 35% was selected as the preferred features of VIs, including NPCI, VOG, ARVI, NDRE, OSAVI, and MSR.

3.2.3. Optimization of Band Positions

The results of correlation analysis between 13 band positions and wheat LNC were shown in Figure 7. As can be seen from Figure 7, the blue legend represents positive correlation and the orange legend represents negative correlation. The first row of the triangular matrix represents the correlation coefficients between 13 band positions and the leaf nitrogen content of the wheat. According to the correlation coefficient, the top 5 were selected as the preferred band positions, including λb, Dr, λr, SDr, and Ro.

3.3. Estimation Results of LNC of Wheat

To compare the prediction effects of different models of wheat leaf nitrogen content, three models including PLSR, SVR, and PSO-SVR were constructed, respectively, which were based on the optimized features (including VIs, BPs, CFs) and fusion features of the wheat canopy spectrum, as shown in

Table 3. The estimated results for wheat LNC based on different features.

Model	Features	Variables Number	Calibration Set			Validation Set
			R2	RMSE	MAE	R2	RMSE	MAE
PLSR	VIs	6	0.735	0.310	0.254	0.708	0.652	0.596
	BPs	5	0.404	0.443	0.340	0.257	0.546	0.405
	CFs	8	0.458	0.434	0.375	0.336	0.521	0.426
	VIs + BPs + CFs	19	0.748	0.288	0.237	0.668	0.467	0.392
SVR	VIs	6	0.759	0.319	0.246	0. 695	0.617	0.565
	BPs	5	0.422	0.466	0.381	0.389	0.610	0.456
	CFs	8	0.470	0.455	0.341	0.392	0.585	0.479
	VIs + BPs + CFs	19	0.664	0.363	0.299	0.684	0.514	0.446
PSO-SVR	VIs	6	0.714	0.534	0.447	0.704	0.646	0.513
	BPs	5	0.595	0.375	0.291	0.429	0.421	0.362
	CFs	8	0.635	0.365	0.306	0.472	0.450	0.377
	VIs + BPs + CFs	19	0.896	0.188	0.148	0.793	0.408	0.354

.

It can be seen from Table 3 that prediction results based on different features were also quite different. The range of R² for the calibration set was 0.714–0.59 based on vegetation indices (VIs), 0.404–0.595 based on band positions (BPs), 0.458–0.635 based on convolutional features (CFs), and 0.664–0.896 based on fusion features (VIs + BPs + CFs). The ranges of R² for the validation set were 0.695–0.708, 0.257–0.429, 0.336–0.472, and 0.668–0.793, which indicate that the estimation effect based on fusion features was significantly better than that based on individual feature using PSO-SVR and PLSR. In addition, Table 3 showed that the fused variables have better prediction results than single features using three different regression models. On the one hand, the performance of R² was better. On the other hand, from the results of MAE, the model did not overfit.

4. Discussion

4.1. Convolutional Features Extracted by Different Layers

It can be seen from Figure 8 that the output feature sequences by different convolutional layers were quite different. On the one hand, the number of output features was different because the parameters of each convolutional layer were different. The output features sequence of the first convolutional layer (Conv1) was 3136. The output features sequence of the second (Conv2), third (Conv3), fourth (Conv4), and fifth convolutional layers (Conv5) were 1728, 1728, 2304, and 256, respectively. On the other hand, the output average feature values were also different. The value of the average values of convolutional features output by the first layer of the convolutional layer was 0.2465–0.8178, and the output features of the second, third, fourth, and fifth layers were 0.4435–0.6345, 0.4193–0.5987, 0.4858–0.5141, and 0.4776–0.5204, respectively. It can be seen that the convolutional features of the canopy spectrum extracted by the convolutional neural network are indeed abstract and difficult to describe directly. As a supplement to spectral features, convolutional features could reflect the consistency and difference in the wheat canopy spectrum.

The spectral features extraction method focuses on fewer and relatively obvious features so that more specific features can be obtained and so that there are less redundant data [58]. Studies have shown that good predictive capabilities have been achieved on the basis of vegetation indices and band positions. Therefore, to avoid the trend of model overfitting, it is necessary to further process and analyze the spectral features.

4.2. Comparison of Estimation Results with Different Models

Different models constructed the prediction of wheat LNC with different characteristics. Although the models based on VIs, BPs, and CFs achieved good results, the prediction effect based on fusion features was better. In particular, for the calibration set, the R² based on fusion features with the PSO-SVR model was 21.4%, 33.7%, and 29.1% higher than that of VIs, BPs, and CFs; the RMSE was reduced by 0.362, 0.521, and 0.531; and the MAE was reduced by 0.449, 0.605, and 0.59. For the validation set, the R² based on fusion features with the PSO-SVR model was 10%, 45.9%, and 40.5% higher than that of VIs, BPs, and CFs; the RMSE was reduced by 0.238, 0.013, and 0.042; the MAE was reduced by 0.159, 0.008, and 0.023. Figure 9 shows the estimation results of the PSO-SVR model.

In fact, spectral data usually have multiple dimensions and significant variability [59]. Therefore, it is necessary to retain or enhance important spectral information through data mapping, which brings challenges to spectral data processing and feature extraction. Yu et al. compared 1D-CNN models with different structures, which showed that the feasibility of 1D-CNN can be used to extract the deep spectral features of Vis/NIR for the identification of pesticide residues on the surface of cantaloupe [60]. On the one hand, the above results show that convolutional features play an important role in prediction; on the other hand, the model estimation ability using a single convolutional feature still needs to be further improved. In this study, a prediction model of wheat LNC was constructed using different spectral features. Three regression models constructed on the basis of spectral features (SFs), convolutional features (CFs), and fusion features (convolutional features and spectral features) have all obtained good prediction results.

4.3. Extraction of Multimodal Features from One-Dimensional Spectral Data Using CNN

CNN is a kind of feedforward neural network with deep structure, which is one of the widely used algorithms. The CNN consists of convolutional layers and pooling layers. The convolutional layers mainly extract low-level and high-level features through sliding convolution kernels [61]. Pooling layers are used to reduce the number of feature maps and network parameters. Ng et al. used 1D spectral data to convert two-dimensional (2D) data as the input to the constructed CNN model, which better predicted the main components of soil [62]. Even Zhang et al. demonstrated the effectiveness of multichannel 1DCNN, which can be used to analyze the deep features of the soil’s Vis-NIR Hyperspectral Data [27]. Padarian et al. converted the spectral data without any preprocessing into 2D spectrograms to accurately predict the physical and chemical parameters of soil. Studies have shown that CNN is used for feature extraction of high-dimensional spectral data, which has become a hotspot in current research [63].

The one-dimensional spectral data of the wheat canopy only show a single pattern, which easily ignores the correlation of the spectral data itself, resulting in a lack of feature diversity. However, multimodal fusion technology can obtain a more complete and better feature expression of the wheat canopy spectrum by making full use of the complementarity between the various pieces of multimodal information. In particular, in recent years, CNN has achieved great achievements in the fields of natural language processing, image processing, speech recognition, etc., with the characteristics of data-driven learning. In this study, we proposed a wheat LNC prediction method based on fusion of multimodal features, which includes spectral features and convolutional features. Among them, the convolutional features were extracted from the one-dimensional spectral data after transformation using CNN, which has multiple channels for easier access to multimodal features. In addition, feature optimization and pooling layers can achieve dimensionality reduction of multi-modal features, so that the accuracy of the LNC estimation model has been significantly improved. In future research, cross-modal learning will become a hot issue and will be widely used in various research fields.

5. Conclusions

Leaf nitrogen content is an important indicator for evaluating wheat nutritional status, and the canopy spectrum provides an efficient means for conducting nondestructive monitoring of wheat LNC. In this study, an estimation model of wheat LNC based on fusion multimodal features was proposed, including spectral features (vegetation indices and band positions) extracted through linear or nonlinear functions and convolutional features extracted using CNN. In addition, compared with modeling based on spectral features and convolutional features, the model based on fusion features performed better; in particular, the PSO-SVR model based on fusion features had the highest accuracy (R² = 0.896, RMSE = 0.188 for calibration, R² = 0.793, RMSE = 0.408 for validation). Therefore, the fusion multimodal features based on the canopy spectrum proposed in this study could improve the ability to estimate the LNC of wheat, which may provide technical support for the quantitative diagnosis of nitrogen nutrition. Further research will be carried out to try different deep learning frameworks to provide technical support for wheat phenotypic information estimation.