Segmentation and Stratification Methods of Field Maize Terrestrial LiDAR Point Cloud

Abstract

Three-dimensional (3D) laser point cloud technology is an important research method in the field of agricultural remote sensing research. The collection and processing technology of terrestrial light detection and ranging (LiDAR) point cloud of crops has greatly promoted the integration of agricultural informatization and intelligence. In a smart farmland based on 3D modern agriculture, the manager can efficiently and conveniently achieve the growth status of crops through the point cloud collection system and processing model integrated in the smart agricultural system. To this end, we took field maize as the research object in this study and processed four sets of field maize point clouds, named Maize-01, Maize-02, Maize-03, and Maize-04, respectively. In this research, we established a field individual maize segmentation model with the density-based clustering algorithm (DBSCAN) as the core, and four groups of field maize were used as research objects. Among them, the value of the overall accuracy (OA) index, which was used to evaluate the comprehensive performance of the model, were 0.98, 0.97, 0.95, and 0.94. Secondly, the multi-condition identification method was used to separate different maize organ point clouds from the individual maize point cloud. In addition, the organ stratification model of field maize was established. In this organ stratification study, we take Maize-04 as the research object and obtained the recognition accuracy rates of four maize organs: tassel, stalk, ear, and leaf at 96.55%, 100%, 100%, and 99.12%, respectively. We also finely segmented the leaf organ obtained from the above-mentioned maize organ stratification model into each leaf individual again. We verified the accuracy of the leaf segmentation method with the leaf length as the representative. In the linear analysis of predicted values of leaf length, $R^2$ was 0.73, $RMSE$ was 0.12 m, and $MAE$ was 0.07 m. In this study, we examined the segmentation of individual crop fields and established 3D information interpretations for crops in the field as well as for crop organs. Results visualized the real scene of the field, which is conducive to analyzing the response mechanism of crop growth and development to various complex environmental factors.

1. Introduction

Maize has wide application and has high economic value as food, fodder, and industrial and pharmaceutical raw materials. Maize is the crop with the largest cultivation area in the world [1]. The 3D laser point cloud data of the target scene obtained by terrestrial laser scanning (TLS) can be used for visualization and 3D reconstruction [2,3,4,5,6,7]. In this context, TLS is widely used in maize field phenotyping. The maize phenotype information obtained from maize point cloud data is more reliable than images [8,9]. The resulting problem is that the high-density point cloud data of the target is difficult to interpret. In other words, it is of great significance to develop a method that can interpret single-plant point clouds and maize organ point clouds from maize fields.

Conducting digital agriculture research and applying laser scanning and 3D laser point cloud technology is of great significance [10,11,12]. Both can reconstruct crops and scenes in the field environment, extract phenotypic features of crops, and automate stratification management of field [4,8,13,14,15]. Lin [16] demonstrated the necessity of combining 3D laser point cloud technology with plant phenotyping research from multiple dimensions. Lin [16] focused on the advantages and challenges of LiDAR-based plant phenotyping. As described by the author, for plant phenotypes, the use of LiDAR technology can calculate the various properties of the target with the least data information, the fastest speed, and is lossless.

The traditional field crop phenotype research is restricted by factors such as time, season, site, growth cycle, and non-reproducibility. Moreover, meeting the current research on agricultural production practice is difficult. Compared with traditional plant phenotyping research, the 3D laser point cloud can improve experimental safety, shorten the experimental period, and reduce experimental costs. The 3D laser point cloud can also provide simulation methods and basic data for agricultural crop phenotyping research [17]. Many scholars at home and abroad have poured into the field of high-throughput field phenotyping based on the 3D laser point cloud [14,18,19]. For example, Miao et al. [20] conducted a series of studies in the field of maize point cloud stem and leaf segmentation and ear recognition. They used a point cloud skeleton extraction and optimization algorithm to obtain the skeleton structure of maize plants to measure the phenotypic shape of target maize plants. Then, Elnashef et al. [19] reported a segmentation model for plant organ segmentation. The model provides a new segmentation algorithm based on first- and second-order tensors to achieve a more accurate organ-level assessment. G. Wu et al. [21] improved the segmentation and recognition accuracy of fruit point cloud based on hue, saturation, value (HSV) color model, and viewpoint feature histogram (VFH) descriptor in the study. Therefore, high-throughput field phenotyping can be considered a new frontier in crop breeding [22].

Among the existing crop phenotyping studies based on the laser point cloud, the method of plant point cloud skeleton extraction is widely used [20,23,24,25]. However, skeleton extraction cannot be effectively applied to agricultural field phenotype research. The fundamental reason is that complex natural factors will lead to fragmentation of 3D point cloud data of target crops. The existing point cloud inpainting technology has limitations. Moreover, the broken crop organ point cloud will greatly interfere with the accuracy of the skeleton extraction algorithm [26]. It is worth mentioning that these experiments rarely take into account the versatility and efficiency of research methods in field experiments, and deliberately avoid targets that are not easy to experiment in the selection of materials [27,28,29,30].

In recent years, after Qi et al. (2016) and Qi et al. (2017) [31,32] proposed deep learning-based 3D point cloud segmentation networks PointNet and PointNet++, many researchers have applied deep learning technology combined with 3D point cloud to the field of crop research [11,33,34,35]. Among them, Y. Li et al. [11] proposed a plant point cloud segmentation technology that integrates high-throughput data acquisition and deep learning, named DeepSeg3Dmaize. This method refers to the mature PointNet to achieve stem, leaf, and organ instance segmentation. Although this type of point cloud segmentation methods is well-structured and has good performance, it is still hard to be widely applied to field phenotyping. The complex real field environment is uncontrollable in time and space [36]. In addition, the controlled conditions in the laboratory environment cannot be compared with those in the field environment. Most important studies such as plant segmentation models, high-throughput phenotypic parameter algorithms, and plant organ clustering algorithms have been validated under certain constraints or scenarios [11,20,37]. The applications in these special scenarios cannot be replicated in a field environment.

In summary, a crop segmentation model that can work smoothly in the field environment needs to be designed urgently. The model should preferably have high computational efficiency while segmenting the point cloud of farmland crops. Therefore, this paper aims to achieve the following objectives:

• Go deep into the farmland and use reliable methods to complete the collection, registration, and cropping of field maize point clouds;
• Explore and verify, based on 3D laser point cloud technology, the spatial morphological characteristics of field maize;
• Create an individual maize segmentation model that can automatically identify and segment each maize plant from the scanned point cloud of the maize field;
• Create a maize organ stratification model that can accurately segment and visualized all maize organs from the field maize point cloud.

2. Materials and Methods

2.1. Material

The research area of this experiment is located in the maize planting field (114.3667° E, 30.4742° N) in the experimental field (Figure 1). This area is located in central China and is the main producing area of high-quality maize. The breeding method of maize adopts seedling transplanting, and the point cloud data collection time is summer. To ensure the capacity scale of the dataset and the diversity of maize varieties, four maize crop fields with a moderate size were selected in the maize production base.

Table 1. Field maize point cloud data attributes.

Number	Type of Maize	Acquisition Date (Day/Month/Year)	Maize Numbers	Growing Stage	Area Size	Number of Scan Stations	Number of Scan Points	Average Position Deviation
Maize-01	Feed grade yellow maize	13 July 2021	89	Seedling stage	5.9 × 3.4 m	8	12,585,028	0.61 mm
Maize-02	Sweet maize	17 July 2021	89	heading stage	6.1 × 3.9 m	8	11,686,560	0.64 mm
Maize-03	Sweet maize	29 July 2021	90	heading stage	5.9 × 3.6 m	8	10,688,590	0.73 mm
Maize-04	Waxy maize	30 July 2021	29	Full ripening stage	4.0 × 2.7 m	8	12,920,643	0.69 mm

shows the basic attributes of four groups of maize fields point cloud data, includes Maize-01, Maize-02, Maize-03, and Maize-04 type of maize, acquisition date, maize numbers, growing stage, experimental area size, number of scan stations, number of scan points, and average position deviation.

Many experiments have shown that 3D laser scanners perform well in field scenes, and various working parameters of 3D laser scanners have also been added to various agronomic measurement variables [8,38,39]. In this experiment, Figure 2 shows the maize field point cloud obtained by FARO Focus S70 terrestrial laser scanner in the maize experimental field, named Maize-01, Maize-02, Maize-03, and Maize-04. We completed point cloud collection under natural conditions with clear weather and no wind. At the same time, in order to avoid overexposure of the camera built into the scanner, the instrument should be avoided under strong light.

The FARO Focus S70 is capable of capturing scans with up to 165 megapixels color information, and the system measurement error at 10 m was ±1 mm. The accuracy of the instrument is sufficient to restore the real field environment. Three sets of target balls and a brackets are used to assist in the field collection of the target point cloud data. Moreover, a Lenovo laptop is utilized to complete the office processing of the point cloud data. The computing configuration is Windows 10 (×64) operating system, 16 GB running memory, NVIDIA RTX 3070 Graphics card, and AMD Ryzen 7 5800H processor. Figure 2 shows the follow-up data process after obtaining the experimental data. The process mainly includes point cloud data preprocessing, individual maize segmentation model, and maize organ stratification model.

Considering that the real field environment is complex, the target distribution is relatively dense, and the occlusion is serious. To completely record the target, this study carried out 3D laser scanning from multi height and multi angle stack of the target. Then, the point cloud data obtained by all stations are stitched together. Given that the origin coordinates of the 3D point clouds scanned by the multi-site are inconsistent, they cannot be overlapped. Ordinary point cloud registration algorithms are lacking in the ability of multi-site cloud scan registration. Therefore, we used the target ball to assist in the registration in this study. By registering the scanned target ball, the point cloud scanned by each station is converted into a unified coordinate system. Table 1 shows the average positional deviation of the target ball fitting in the four groups of point cloud registration process, all of which are less than 1 mm. Then, the FARO Focus3D professional post-processing software SCENE is used for point cloud stitching and preliminary cutting.

After the original point cloud data are obtained by multi-site cloud registration, the number of point clouds in the study area can reach tens of millions, containing a large amount of redundant data. In this study, a 3D voxel grid is created for the input target point cloud data based on point cloud library (PCL, A large crossplatform open source C++ programming library). Thus, the point cloud data volume is sampled to 40% of the original. The voxel center point was replaced by the point nearest to the barycenter point in the original point cloud data, thus improving the accuracy of high point cloud data expression.

2.2. Individual Maize Segmentation Model

In this section, Maize-01, Maize-02, Maize-03, and Maize-04 are used as research data. Based on the existing maize point cloud data set, we extracted the oriented bounding box (OBB) of 300 maize plants and extracted the OBB length and width measurements. OBB determines the size and orientation of the box according to the geometry of the object itself. The bounding box does not need to be perpendicular to the coordinate axis, which is the most compact bounding box. We set each point to $p_i=\left(x_i,y_i,z_i\right)$. The position mean $m$ and the covariance matrix $A$ of the target point cloud $\left\{p_1,p_2,\dots ,p_n\right\}$ are calculated by Equations (1) and (2). The covariance matrix $A$ is diagonalized to obtain its eigenvector, which is the coordinate axis of the OBB.

$$m=\frac{1}{N}{\displaystyle \sum }_{i=1}^n{p}_i,$$

(1)

$$A=\left[\begin{array}{ccc}cov\left(x,x\right)& cov\left(x,y\right)& cov\left(x,z\right)\\ cov\left(x,y\right)& cov\left(y,y\right)& cov\left(y,z\right)\\ cov\left(x,z\right)& cov\left(y,z\right)& cov\left(z,z\right)\end{array}\right] = \left[\begin{array}{ccc}\frac{1}{N}{{\displaystyle \sum }}_{i=1}^n{(x_i-m_x)}^2& \frac{1}{N}{{\displaystyle \sum }}_{i=1}^n(x_i-m_x)(y_i-m_y)& \frac{1}{N}{{\displaystyle \sum }}_{i=1}^n(x_i-m_x)(z_i-m_z)\\ \frac{1}{N}{{\displaystyle \sum }}_{i=1}^n(x_i-m_x)(y_i-m_y)& \frac{1}{N}{{\displaystyle \sum }}_{i=1}^n{(y_i-m_y)}^2& \frac{1}{N}{{\displaystyle \sum }}_{i=1}^n(y_i-m_y)(z_i-m_z)\\ \frac{1}{N}{{\displaystyle \sum }}_{i=1}^n(x_i-m_x)(z_i-m_z)& \frac{1}{N}{{\displaystyle \sum }}_{i=1}^n(y_i-m_y)(z_i-m_z)& \frac{1}{N}{{\displaystyle \sum }}_{i=1}^n{(z_i-m_z)}^2\end{array}\right].$$

(2)

After the aspect ratio of the OBB of the 300 maize point clouds obtained by the above equations, the data histogram is statistically obtained, as shown in Figure 3. In the normality test results (

Table 2. Normality tests: skewness and kurtosis.

	N	Skewness		Kurtosis
		Statistic	Std. Error	Statistic	Std. Error
Valid N	300	0.113	0.141	0.161	0.281

), the kurtosis and skewness values, which describe the shape of the data distribution, are 0.113 and 0.161, respectively. The test results show that the aspect ratio of the maize point cloud OBB obeys a normal distribution, which proves that the OBB-type bounding box of maize plants is generally a cuboid. That is, as long as the 3D parameters of the OBB of an individual maize plant are determined, an individual maize plant from the field point cloud can be segmented.

According to each maize plant seed point $q_i$, a rectangular box for individual plant segmentation is developed, wherein the rectangle is parallel to the horizontal plane. Then, a bounding box $D$ is generated according to this rectangle, and the point cloud segmented by the bounding box is divided into point sets $C_i$, where i represents the maize number.

Figure 4 demonstrates a method of accurately determining the plant position, taking Maize-04-No. 9 (crop number is 277) as an example. The seed point $q_i$ is located at the center of the maize stalk on the same horizontal plane. For its tracking method, in the field point cloud data, slices are made at a level 5 cm above the soil plane (Figure 4a). In Figure 4a, $W$ represents the slice target, and two equidistant planes $W_l$ and $W_r$ are generated in the normal vector direction of $W$. The distance between $W_l$ and $W_r$ is the section thickness $\delta$. Moreover, clustering is performed to cluster individual plants in the sliced point cloud. Then, outliers are filtered in the point cloud of each individual plant slice (Figure 4b). Considering that withered leaves in mature maize plants droop down, after filtering out outliers, the circular fitting should be performed on the slice point cloud (Figure 4d). Through this method, the point cloud of withered leaves is filtered, and then, the center point $q_i$ of the point cloud of each individual plant slice is calculated (Figure 4g).

The growth position of each maize plant is determined to calculate the OBB of the maize point cloud. First, the point set $C_i$ is projected to the horizontal plane, and the radius nearest neighbors (Radius-NN) research is implemented with the projection point of each seed point $q_i$ as the center. Then, the projection point within the radius of 30 cm can be obtained. Principal component analysis was used to analyze the linear distribution of projected data points. After centralization processing (Equation (3)) and maximizing Equation (4), the fitting straight line of the projected point cloud is obtained to preliminarily judge the horizontal growth direction of leaves, where $\overrightarrow{u_1}$ is the first principal axis of the point set $\left\{\overrightarrow{p_1},\overrightarrow{p_2},\dots ,\overrightarrow{p_j}\right\}$, $\overrightarrow{\mu }$ is the data center, $\left\{\overrightarrow{x_1},\overrightarrow{x_2},\dots ,\overrightarrow{x_j}\right\}$ are the centralized point sets, and $j$ is the number of points in the point set $C_i$. Based on a large number of maize prior growth parameters, this study generates a cropbox with length, width, and height of $a$, $b$, and $c$, respectively. This bounding box is used for the preliminary segmentation of field maize plants. Among them, the cropbox size $a$ of the maize plant at the flowering stage is 0.9 m, $b$ is 0.7 m, and $c$ is the range of point cloud elevation.

$$\left\{\begin{array}{c}\overrightarrow{\mu }=\frac{1}{n}{{\displaystyle \sum }}_{j=1}^n\overrightarrow{p_j}\\ \left\{\overrightarrow{x_1},\overrightarrow{x_2},\dots ,\overrightarrow{x_j}\right\}=\left\{\overrightarrow{p_1} - \overrightarrow{\mu },\overrightarrow{p_2} - \overrightarrow{\mu },\dots ,\overrightarrow{p_j} - \overrightarrow{\mu }\right\}\\ \overrightarrow{x_j}·\overrightarrow{u_1}=x_j^T{u}_1\end{array}\right.,$$

(3)

$$\frac{1}{n}{\displaystyle \sum }_{j=1}^n{\left|\overrightarrow{x_j}·\overrightarrow{u_1}\right|}^2=\frac{1}{n}{\displaystyle \sum }_{j=1}^n{\left(\overrightarrow{x_j}·\overrightarrow{u_1}\right)}^2.$$

(4)

Given that the number of target individuals obtained by the crop box is unknown, an algorithm that does not need to input the number of prior individuals should be selected. Moreover, in the process of individual plant segmentation, the time complexity should be kept as low as possible. Therefore, in this research, we used DBSCAN (time complexity $O$ is $n·\log n$, $n$ is the number of points) algorithm to segment individuals from population crops [40,41,42]. In the part of organ stratification model, the maize organs were segmented from the point cloud of single maize plant by spectral clustering algorithm (time complexity $O$ is $n^3$) [43].

The DBSCAN [44] algorithm can divide regions with sufficient density into clusters and find irregular-shaped clusters in a noisy point cloud space. Among them, a cluster is defined as the largest collection of point clouds with a point cloud density greater than a threshold. To quickly complete the cluster search, the KDTree module is added to the algorithm to perform the Radius-NN search [45].

The research in this section takes the mainstream algorithm as the core and constructs the above individual maize segmentation model. Among them, these algorithms have been mentioned in many studies, and the research results have proved that the algorithms have good performance.

2.3. Maize Organ Stratification Model

In this section, Maize-04 is used as research data. Among them, Maize-04 is in the Full ripening stage, where all organs of maize have basically stopped growing. When the beam of the 3D laser scanner Focus S70 used in this study scans a target object within a range of 10 m, the point cloud accuracy can be controlled within 2 mm. In addition, the data quality is sufficient to support the organ-level segmentation of crop point clouds. In this study, the spectral clustering algorithm was used to achieve individual organ clustering in maize. First, the point cloud of the maize stalk is separated to disconnect the spatial connection among the various organs of the maize plant, which will facilitate the further completion of organ clustering. Then, the clustering of individual organs was completed through the spectral clustering algorithm. Finally, the classification of clustered organs is achieved by establishing organ identification conditions.

In this section, the focus is on using spectral clustering, an unsupervised algorithm, to achieve clustering of different organs in maize. Among them, the spectral clustering algorithm is based on the K-nearest neighborhood (KNN) composition method and the Normalized cut (Ncut) method [46]. In addition, the Kmeans algorithm is used to add labels to the point cloud to complete the clustering. The spectral clustering algorithm is sensitive to parameter settings. The number of neighbors $K$ of KNN needs to be adjusted through several experiments. Moreover, the normalization of the laplacian matrix needs to be selected through certain prior knowledge. As the time complexity of the spectral clustering algorithm is extremely high ($O_{\left(n^3\right)}$), this research completes the second filtering process on the object point cloud through the voxel filtering method in this part. This case can preserve the surface features of the object point cloud as much as possible while improving the efficiency of the spectral clustering algorithm. The number of target point clouds after voxel filtering is 10% of the initial point cloud.

The organ labeling operation can be completed by setting the following label discrimination conditions.

• Stem segmentation

The stalk of the maize plant is thick and grows straight upward, and the scanning results of the high precision laser scanner show a relatively evident deformation of the cylindrical geometry. Therefore, directly fitting the target cylinder is not feasible. With this in mind, the stalk segmentation in this study consists of the following two steps (Figure 5). First, the linear features of the maize point cloud were identified in PCL. Moreover, the method of cylinder space segmentation was used to preliminarily separate the stalk point cloud in a single plant and the leaf point cloud that was partially connected to the stalk. A polynomial curve fit was then performed for the curved stalk in cylinder space using the RANSAC algorithm. Considering the overall cylindrical geometry of the stem, the fitting algorithm in this section adopts radius sampling.

2.

Identification of ear point sets

According to the observation, compared with the non-stem organs, the shape of the ear has conical and cylindrical features. The RANSAC algorithm is used to perform cone and cylinder fitting in the selected organ points. The point set that meets the point number threshold is identified as the ear point set.

3.

Identification of tassel and leaf point sets

The tassel organs are located at the top of the plant. Therefore, the organ at the highest point of the plant is the tassel, and the remaining organs are marked as plant leaves. All leaf point clouds in the individual maize plant form a leaf point cloud set, which can be divided into point cloud subsets of each leaf through a clustering algorithm. Figure 6 shows the final result of adding the maize organ point cloud label:

In this study, leaf length was used as a measure of leaf stratification accuracy. Figure 7 illustrates the fast calculation method of leaf length. The key points on the leaf veins are determined by leaf length path point, and the adjacent key points are linked [27,30]. Moreover, the key point connecting line is defined as the construction line ${\mathit{l}}_i$ ($i\in \left[1,m\right]$), and the vector sum $\sum {\mathit{l}}_i$ of all construction lines is 0. The construction line where the key point $p_m$ of the blade tip lies is ${\mathit{l}}_m$. The scalar sum of all construction lines $\sum \left|{\mathit{l}}_i\right|$ minus $\left|{\mathit{l}}_m\right|$ is defined as the leaf length $l_{leaf}$ (Equation (5)).

$$l_{leaf}=\sum \left|{\mathit{l}}_i\right|-\left|{\mathit{l}}_m\right|.$$

(5)

2.4. Model Evaluation Metrics

In this study, the precision rate $P$ and the recall rate $R$ were set to verify the accuracy of individual plant segmentation. In addition, the global accuracy of the target segmentation was calculated directly by the overall accuracy ($OA$) (Equation (6)), where $n$ is the total number of points in the standard set. Compared with the field image data and the target photographed by the laser scanner, the point cloud of the maize plant and its organs was calibrated in PCL as the standard value to obtain the confusion matrix of the segmentation layered model. In Equations (7) and (8), true-positive ($TP$) represents a true positive example, false-positive ($FP$) represents a false positive example, and false-negative ($FN$) represents a false negative example. Figure 8 shows an example of the $TP$, $FN$, and $FP$ decisions for the maize segmentation result. Among them, Figure 8c highlights the difference between Figure 8a,b.

$$OA=\frac{{{\displaystyle \sum }}_{i=1}^{n}TP}{{{\displaystyle \sum }}_{i=1}^n\left(TP+FP+FN\right)},$$

(6)

$$P=\frac{TP}{TP+FP},$$

(7)

$$R=\frac{TP}{TP+FN}.$$

(8)

To comprehensively weigh the two indicators of $P$ and $R$, a new indicator $\mathit{F-score}$ is introduced here to comprehensively consider the harmonic value of Precision and Recall (Equation (9)). In this study, the weight coefficient $\beta$ of $\mathit{F-score}$ is 1.

$$\mathit{F-score}=\left(1+{\beta }^2\right)\times \frac{P\times R}{{\beta }^2\times P+R}.$$

(9)

In addition, the spectral clustering algorithm in the organ clustering module needs to perform sub-sampling on the point cloud. Thus, the intersection over union ($IoU$) based on the standard value and the predicted value is used as the accuracy evaluation index of organ clustering in the research. The $IoU$ calculation is shown in Equation (10), where $I_{ TP}$ represents the intersection of the bounding box between the predicted value and the standard value. Moreover, $U_{\left(TP+FP+FN\right)}$ represents the union of the bounding box between the predicted and standard values.

$$IoU=\frac{I_{TP}}{U_{\left(TP+FP+FN\right)}}.$$

(10)

In this study, the linear regression analysis method was also used to explore the linear relationship between the predicted and standard values of the plant phenotypic parameters based on the stratification model. The coefficient of determination of regression evaluation indicators, including R squared ($R^2$), root mean squared error ($RMSE$), and mean absolute error ($MAE$), was used to quantitatively evaluate the segmentation performance of the maize organ clustering model in field scenarios (Equations (11)–(13)).

$$R^2=1-\frac{{{\displaystyle \sum }}_{i=1}^n{\left(\widehat{y_i}-y_i\right)}^2}{{{\displaystyle \sum }}_{i=1}^n{\left(\overline{y_i}-y_i\right)}^2},$$

(11)

$$RMSE=\sqrt{\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left(\widehat{y_i}-y_i\right)}^2},$$

(12)

$$MAE=\frac{1}{n}{\displaystyle \sum }_{i=1}^n\left|\widehat{y_i}-y_i\right|.$$

(13)

Among them, $\widehat{y_i}$ is the standard value of the parameter, $y_i$ is the predicted value of the parameter, $\overline{y_i}$ is the sample mean, ${{\displaystyle \sum }}_{i=1}^n{\left(\widehat{y_i}-y_i\right)}^2$ is the error caused by the prediction, and ${{\displaystyle \sum }}_{i=1}^n{\left(\overline{y_i}-y_i\right)}^2$ is the error caused by the mean.

3. Results and Discussion

3.1. Segmentation Results and Analysis

In this research, a two-dimensional linear equation was used to represent the growth direction of maize plants. Figure 9 shows the growth trend of individual maize plants obtained based on the cylindrical space projection method.

Figure 10 shows that the prediction error of the growing trend of an individual plant during the experiment is generally less than 0.2 radian. This result indicates that the plant direction prediction in this study is more accurate, and the crop box divided based on this has higher accuracy. In Figure 10, there are a few maize plants whose growth direction prediction deviation is greater than 0.4 radian. However, based on the segmentation results in Figure 11 and Figure 12, it can be seen that the limited prediction bias did not cause significant interference to the final segmentation results of maize individual plants. For example, the prediction of the growth direction of Maize-04-No. 25 (crop number is 293) has a large deviation, and the angle between it and the standard direction vector is 41.12°. However, in the follow-up cluster model accuracy analysis, the $P$ value and $R$ value of No. 25 maize were 0.99 and 0.93, respectively. This phenomenon indicates that the prediction bias value of the direction within a certain range does not have much influence on the clustering accuracy.

Figure 11 depicts the target individual segmentation preview. We render $TP$, $FP$, and $FN$ as yellow, red, and blue, respectively, and display them in the same coordinate system. $FP$ mainly comes from the extreme adhesion phenomenon of some adjacent maize point clouds. However, the reasonable setting of the crop box in the segmentation model will control this error within a small range. $FN$ is mainly derived from the maize point cloud in the broken state and the under-segmentation of the crop box. The former might be caused by the influence of complex field environments, such as wind blowing blades and others in the process of collecting target point cloud. The latter is because of the large span of individual maize plants in the horizontal direction, and the uniform size of the crop box will lead to a small amount of under-segmentation.

Figure 12a,b shows the precision and recall numerical statistics of the individual maize segmentation results obtained by this research method, and Figure 11c depicts the $\mathit{F1-score}$ numerical statistics. The results show that the segmentation results of most single maize samples were better. Among them, the segmentation accuracy of 97.31% maize samples per plant is higher than 80%. The segmentation recall rate of 97.64% maize samples is higher than 80%. The $\mathit{F1-score}$ of 99.66% individual maize samples is higher than 80%. When the weight coefficient is 1, $\mathit{F1-score}$ performs well, which indicates that the individual plant segmentation model in this study can overcome the complex field environment and effectively achieve outdoor individual-maize segmentation. During the analysis of the results (

Table 3. Individual maize segmentation result statistics table.

	TP (Yellow)	FN (Blue)	FP (Red)	P			R			F-Score			OA
				[0, 80%]	[80%, 95%]	[95%, 100%]	[0, 80%]	[80%, 95%]	[95%, 100%]	[0, 80%]	[80%, 95%]	[95%, 100%]
Maize-01	5,727,642	73,579	62,781	0	13	76	3	22	64	0	28	61	0.98
Maize-02	8,826,515	157,479	108,064	3	16	70	1	23	65	0	27	62	0.97
Maize-03	9,815,061	268,093	285,177	4	20	66	3	15	72	1	23	66	0.95
Maize-04	5,913,240	228,412	123,822	1	2	26	0	11	18	0	6	23	0.94

), the segmentation accuracy rate of several maize plants was less than 80%, which was because of the intricate distribution of the crowded maize leaves in the space. The leaves were often stuck together. The existence of the crop box in this model can reduce the sticking of leaves to a certain extent, thereby reducing the segmentation error. This also resulted in the $\mathit{F1-score}$ of only one of our 297 maize samples being less than 80%.

According to Equation (6), the $OA$ value of Maize-01, Maize-02, Maize-03, and Maize-04 are 0.98, 0.97, 0.95, and 0.94. It is obvious that the overall clustering accuracy is relatively high. The above-mentioned accuracy analysis results all verify the feasibility and stability of the individual-maize segmentation model in this study.

Different from conventional unsupervised identification and segmentation methods, the segmentation model in this study firstly determines in which region a single maize plant is located, and separates the point cloud of this region from the whole. Finally, a single maize plant is obtained from the clustering of the point cloud in this area. This research idea fundamentally guarantees the accuracy of crop segmentation and provides a premise for the subsequent segmentation of maize organs. It is conceivable that this fast and effective segmentation method will become the preferred choice for digital agriculture platforms.

3.2. Organ Stratification Model Accuracy Analysis

Figure 13 shows the stratification visualization effect of Maize-04 leaves based on the organ stratification model. In the analysis in this section, to fully verify the accuracy of the maize organ stratification model, we evaluated the leaf organ clustering accuracy using the linear analysis results between the predicted and standard values of leaf length. The clustering accuracy of stem organs was evaluated using the accuracy rate of stem height prediction. The clustering accuracy of ear and tassel organs was evaluated using the $IoU$ metric. Among them, the top ear of maize was used as the research object of ear organs.

After obtaining the above-mentioned four kinds of organ clustering statistics (Figure 14), we calculated the recognition accuracy of the four kinds of organs of maize in the study area. The recognition accuracy of tassel, stem, ear, and leaf was 96.55%, 100%, 100%, and 99.12%, respectively. The experimental results show that the maize organ stratification model in this section can accurately complete the task of organ segmentation for maize plants in the field environment.

As shown in the histogram (Figure 14a), the tassel of maize plant Maize-04-No. 2 (crop number is 270) is not accurately identified. Figure 15 shows the visualization of the target, which is because of the breakage of the tip of the second maize plant, resulting in confusion between the first leaf and the tassel. This phenomenon once again confirms the difficulty of crop organ stratification in natural field environments.

In the histogram of the $IoU$ value for ear recognition (Figure 14b), all ears were correctly identified, and all had a high segmentation accuracy. This case shows that the organ segmentation model should be able to accurately locate ear targets in maize fields in practical applications.

In the histogram of maize stalk height prediction errors (Figure 14c), the prediction errors of stalk height are generally less than 0.1 m, and only two maize plants have errors greater than 0.1 m but less than 0.16 m. Therefore, in the actual application process, this segmentation model can accurately estimate the maize stalk height.

In the scatter plot between the predicted value of leaf length and the true value (Figure 14d), an evident linear relationship exists between the predicted value of the model and the true value. The $R^2$ of the linear fitting result is 0.71, the $RMSE$ is 0.12 m, and the $MAE$ is 0.07 m. The $RMSE$ and $MAE$ performed well in the analysis results. The value of $R^2$ is mainly restricted by two factors: one is because of the missed identification of three leaves, and the other is under-clustering caused by the fragmentation of the point cloud of a few leaves.

The output accuracy of segmentation and stratification models in this study is restricted by two factors. First, the rationality of the segmentation and stratification method. Second, the data quality of the target 3D point cloud. This study only considers the performance of the segmented and stratification model. However, in the actual application process of agricultural production, a certain degree of data quality compensation is equally important. Examples include using higher-precision experimental instruments, more complete data acquisition methods, and more rigorous data repair methods. In addition, this study is limited to only a few varieties of maize and four medium-scale maize experimental field for data collection, modeling, and verification, which is slightly insufficient in terms of experimental data scale. In the future, we plan to test and update this study in larger field crop phenotypic experiments.

4. Conclusions

In this study, based on the point cloud of maize in the field, a model that can complete the individual segmentation and organ stratification of maize plants is established. After counting a large number of maize OBB aspect ratios, we found that it had obvious normal distribution characteristics. We found a reliable method for extracting this feature in this study, with an average error of only 0.06 rad in direction prediction (Section 2.2). Based on this law of maize growth, we finally established an individual maize segmentation model that can directly process field maize point clouds. In Section 3.1, the OA output values of this model for Maize-01, Maize-02, Maize-03, and Maize-04 are 0.98, 0.97, 0.95, and 0.94, respectively.

Based on the individual maize segmentation model described above, this study automatically segmented each maize point cloud into organ-level point sets, thereby proposing a maize organ stratification model. Taking Maize-04 as an example, the recognition accuracy of tassel, stem, ear, and leaf of this model was 96.55%, 100%, 100%, and 99.12%, respectively. In addition, the mean $IoU$ of tassel segmentation output by the stratification model was 0.69, the mean $IoU$ of ear segmentation was 0.75, and the mean error of plant height estimation was 0.03 m. In a linear regression analysis of maize leaf length estimates, $R^2$ is 0.71, $RMSE$ is 0.12 m, and $MAE$ is 0.07 m. Finally, the stratification model was also able to visualize maize organs in a hierarchical manner (Figure 13), further exploiting the advantages of 3D visualization techniques in agricultural phenotyping.

Both the individual maize segmentation model and the maize organ stratification model in the paper can run with high efficiency in ordinary computer equipment. The research results in this paper can be applied to the 3D study of maize in the field environment, thus providing support for the 3D research field of digital agriculture.