1. Introduction
The rapid assessment of forest and vegetation structure using unmanned aerial vehicles (UAVs) is likely to decrease the cost of field surveys for a variety of resource industries. UAVs may be particularly well suited for applications in reforestation, because they can collect very high-resolution imagery of small features with great operational flexibility. In Canada, establishment surveys are conducted at every forest-harvest area that has been replanted, to assess the adequacy of spacing, survival, growth, and species composition. For example, the Regeneration Standards of Alberta [
1] call for a reconnaissance survey to be conducted three growing seasons after planting, wherein certified forestry technicians walk through the harvest area to visually estimate ‘stocking’, the percentage of 10 m
2 cells within the block that contain a live seedling at least 30 cm in height, from an acceptable tree species. If the estimated stocking rate is above 84%, the harvest area passes the establishment assessment. If the stocking rate is below 70%, the harvest area is rejected and must be replanted. If the stocking falls between 70%–84%, the harvest area becomes subject to further assessment [
1].
If the condition, minimum height, and species of seedlings within the sample cells used to perform establishment surveys could be derived from UAV imagery, then the reduced need for manual surveys could lead to considerable cost savings. However, it needs to be demonstrated that the seedlings can be automatically or semi-automatically detected from a remote sensing. Coniferous seedlings under five years of age in Canada have crown diameters between 5 and 30 cm; imagery of a very high spatial resolution is required to detect the seedlings of this size. Although automated procedures have been used to detect small tree stumps [
2], and to detect weed seedlings on bare soil in an agricultural application [
3], we could not find any previous research attempting to identify coniferous seedlings of this age in an automated manner. Early studies of remote sensing applications in forestry [
4,
5] used manual photointerpretation of large-scale aerial photographs acquired from piloted helicopters. For example, Hall and Aldred [
5] detected just 44% of seedlings with crown diameter less than 30 cm using 1:500 color-infrared photography. More recently, Goodbody et al. [
6] classified 2.4 cm spatial resolution red, green, and blue (RGB) imagery acquired from a UAV over harvest blocks replanted 5 to 15 years earlier in British Columbia, Canada, and obtained user accuracies for coniferous cover between 35% and 97%. However, no attempt was made to detect individual conifer seedlings or saplings, which in their study area were greater than 1 m in height. These authors acknowledged that the potential to detect all stems using aerial remote-sensing technologies is still limited, and called for further research.
Depending on the spatial resolution of the imagery, seedling detection is akin to individual-tree detection in mature forests, which has been well studied using satellites [
7], piloted aircraft [
4,
8,
9,
10,
11,
12,
13,
14,
15] and, more recently, UAVs [
16,
17,
18,
19,
20,
21,
22,
23]. Given a fixed spatial resolution, the accuracy with which individual trees are detected tends to improve with crown size [
11]. For example, using 15 cm resolution multispectral airborne imagery and an image-segmentation algorithm, Hirschmugl et al. [
24] obtained 70% accuracy on replanted coniferous trees between 5 and 10 years of age, with an average height of 138 cm. The accuracy levels improved for the crown diameters larger than 30 cm. The UAV-based studies of the tree detection have achieved even higher accuracies. For example, Wallace et al. [
16] obtained overall accuracies of 97% (
n = 308) using a tree-crown segmentation algorithm applied to a dense (>50 points/m
2) light detection and ranging (LiDAR) point cloud. However, the LiDAR sensors come with equipment costs and operational difficulties that some may wish to avoid. Consumer-grade cameras mounted on UAVs provide an attractive low-cost source of vegetation information over disturbed and regenerating forests [
25,
26].
In this research, we show how millimetric (i.e., spatial resolution on the order of millimeters) UAV imagery can be used to detect coniferous seedlings less than five years old within forest-harvest areas in Alberta, Canada, with good accuracies using simple processing workflows. This study represents the first step towards creating a larger UAV-based stocking-assessment workflow, which, once realized, could extend to the remote assessment of height (from LiDAR or photogrammetric point clouds), species, and condition (from deep-learning algorithms). Achieving this complete workflow would reduce the need for in situ assessments of forest stocking, and provide a powerful new tool for establishment surveys.
2. Materials and Methods
2.1. Study Area
We surveyed two replanted forest-harvest areas located in western Alberta, Canada, for this study (
Figure 1). One of the harvest areas was used to develop and train the seedling-detection algorithm, and is hereafter referred to as the ‘training study area’. A second block was used as an independent validation site and is hereafter referred to as the ‘test study area’. The 20.3 ha training study area is managed by Weyerhaeuser Canada (Pembina Forest Management Area), and was replanted approximately four years before our survey, with a mix of lodgepole pine (
Pinus contorta) and white spruce (
Picea glauca) seedlings. The 3.3 ha test study area was also replanted with a mix of lodgepole pine and white spruce seedlings between three and four years before our field survey, although most individuals we encountered in the field were lodgepole pine. The vegetation surrounding these two harvest areas consist of forests regenerating to mixed stands of aspen (
Populus tremuloides), lodgepole pine, and white spruce.
2.2. Reference Data
Field crews surveyed the test study area on 24 April 2014, and the training study area on 1 October 2015. We timed our visits to exploit the leaf-off seasonal windows in the spring and fall when the coniferous seedlings have increased their spectral contrast with their surroundings. We contend that the 1.5 year time gap between the two surveys is irrelevant, given that testing took place independently of the training. The crews located randomly generated plot centers using handheld global positioning system (GPS) units and established 50 m
2 circular plots with 3.99 m radii (
Figure 2). Biodegradable clay targets or plastic boards were placed in the center of the plot and pinned down with metal spikes as ground control points, whose precise locations were recorded with a survey-grade Trimble real-time kinematic (RTK) global navigation satellite system (GNSS) unit. The plot outlines were marked using chalk or spray paint. The crews then recorded the species and precise location of each seedling inside the plot using the RTK GNSS. A total of 254 seedlings within 14 plots in the training study area, and 149 seedlings within 10 plots in the test study area were surveyed in this manner.
2.3. UAV Imagery
UAV imagery was collected by a flight crew consisting of a pilot and a spotter using a 3DR X8 + octocopter (
Figure 3a). Both areas were flown in conjunction with the seedling surveys 24 April 2014 for the test study area and 1 October 2015 for the training study area. Details of the X8+ platform and payload are summarized in
Table 1. The platform was modified to carry two cameras simultaneously, one standard RGB camera and a second camera with a modified red-edge (RE) filter. Single-scene images, one RGB and one RE, were acquired for each of the 24 plots during 24 separate flights. The flights took place between 10:00 am and 5:30 pm to cover a variety of lightning conditions. The platform operated a simple automated flight plan, as follows: The X8+ launched, flew to the plot center, and hovered at 15 m above ground level to acquire imagery (single photographs) at a consistent scale (
Figure 3b). We installed a LidarLITE laser range finder (vertical accuracy < 2.5 cm) to the UAV, which allowed us to control the altitude of the X8 for imaging. It is important to note that that circular plot did not cover the entire image, but was instead located at the center of each frame. As the UAV hovered directly over each plot prior to image acquisition, the plots were always located very close to the principal point, with the maximum off-nadir angles never exceeding 15 degrees. This reduced the terrain distortion and layover effects. Each flight was less than three minutes in length. The imagery was collected using a sampling approach, avoiding the need for wall-to-wall aerial surveys designed to image the entire harvest area.
We used a Nikon Coolpix A digital camera to collect standard RGB imagery and a modified Canon PowerShot S110 to collect imagery in the RE wavelength. We substituted the internal near-infrared (NIR) filter in the Canon PowerShot S110 with an Event 38 near-infrared green blue (NGB) filter, which pushed the red band response to be centered on 715 nm. The aerial imagery was collected at a low altitude of 15 m, resulting in a ground sampling distance of 3 mm for the Nikon Coolpix A (18.5 mm focal length) and 5 mm for the Canon PowerShot S110 (5.2 mm focal length). The cameras were set to a fixed shutter speed of 1/1250 s with varying apertures, and were manually triggered using a remote control.
2.4. Data Handling and Image Analysis
The seedling locations were exported to a geographic information system (GIS) point layer, visually confirmed using the UAV imagery, and, if required, spatially edited to be centered within each seedling crown in the image. The seedlings ranged in height from 5 cm to 35 cm, and the crown radii ranged between 5 cm and 50 cm. The tall seedlings were those manually planted about four years before our study, while the short seedlings regenerated naturally. The wider crowns (up to 50 cm) corresponded generally to clusters of naturally regenerating seedlings, rather than to individuals. We treated these clusters as single entities for the purpose of this study. It is important to note that the planted seedlings (taller, generally isolated from other individuals) are more important than the naturally regenerating seedlings (shorter, sometimes occurring in clusters) when assessing stocking in the planted harvest areas. Not all of the seedlings will survive to maturity, and the planted seedlings have the best chance. Within a cluster, no more than one seedling will typically survive.
The image analysis workflow is a three-step object-based process consisting of (i) image segmentation, (ii) automated classification using a classification and regression tree (CART) machine-learning algorithm, and (iii) the merging of adjacent image objects classified as ‘seedlings’ into single seedling objects. Priority was placed on creating a workflow that is economic with regards to both the UAV flight time and processing time, so image analyses were conducted on single scenes with minimal preprocessing. The challenge was to identify a classification ruleset that could perform under a variety of target and illumination conditions. We used the CART approach in the training study area to test the importance of the spectral, spatial, and textural variables for this task. On the basis of the results of this testing, we selected a single model for application in the test study area.
Red-edge and RGB imagery were co-registered into 16-bit unsigned raster layers for each sample, using ArcGIS Desktop 10.1 (
Figure 4a,b). The images were rectified with first-order polynomial functions, using the seedlings and ground control points as reference marks. It is worth noting that this step—rectifying images to match field data—would not be required in an operational workflow. Additional raster layers were generated from band ratios to gain a spectral contrast between the green seedlings and their non-photosynthetic surroundings (
Figure 4c). We used Trimble eCognition Suite 9.1 (
www.ecognition.com) to segment the imagery and derive image-object statistics. The input raster for the initial segmentation was a ratio of ratios (red ratio/blue ratio) scaled to the 0–255 interval. The user-defined segmentation parameters were as follows: scale = 50; shape = 0.1; and compactness = 0.3. All of the 48 images (24 RGB and 24 RE) were segmented using the same parameters. We arrived at the final segmentation parameters iteratively through trial and error. Our goal was to develop object primitives that best delineated the seedling edges from their surroundings. The resulting image-objects were then further merged using a homogenous region-growing algorithm, with shape and compactness factors of 0.1 and 0.5, respectively. Once the final image objects were generated for each plot, we assembled a number of attributes for each image object. The final list of spectral, spatial, and textural variables evaluated by the CART approach is summarized in
Table 2.
2.5. Machine Learning
Image-object attributes were exported to a table, resulting in a database with 18,905 records (image-objects from all of the 14 plots in the training study area together). Each record was then classified as either seedling or non-seedling using the CART machine-learning algorithm in the Salford Predictive Modeler (SPM v. 70) software (info.salford-systems.com). This algorithm generates a classification decision tree, with rules that can be used with structured query language (SQL) queries or to build a decision tree in eCognition. Three sets of models were evaluated. All three of the sets used the same spatial and textural attributes as the predictors (
Table 2), but the spectral attributes varied as follows: (i) RGB-only variables, (ii) RE-only variables, and (iii) RGB-combined-with-RE variables. The adjacent image objects classified as seedling were merged together using a GIS ‘dissolve’ function. The detection accuracy of each model was assessed using a 10-fold cross validation procedure. The mean overall seedling classification accuracy was obtained by each model across all of the 10 trials and was assigned as a measure of global accuracy. The most accurate model was applied to the test study area, which served as an independent validation of our workflow.
3. Results and Discussion
3.1. Model Selection
The overall classification accuracy of all of the image-objects in the training study area was 96%, 97%, and 97% for the RGB-only variables, RE-only variables, and RGB and RE variables, respectively. It should be noted that the overall accuracy reported here (raw agreement) is a high-level accuracy statistic based on a disproportionately small number of seedling objects (2420) to non-seedling objects (18,663). We report on more detailed error analytics associated with the test dataset below. As the RE-only and RGB and RE models did not result in significant increases in performance (1%), we chose to use the RGB-only model for parsimony.
The final CART decision tree was pruned to a simple two-rule model based on just two spectral vegetation indices, the green-red difference index and the blue-green difference index. We found that reasonable classification models could be generated using an RGB camera alone, and—more importantly—that one classification model could be used to detect coniferous seedling crowns across many sample plots in our study sites imaged under different lighting conditions. An example test-plot classification is shown in
Figure 5.
3.2. Detection Accuracy, Error Patterns, and Density Estimates
To assess the detection accuracy in the test area, we considered a reference seedling detected (i.e., a true positive) if its corresponding geolocation point was inside a seedling object; otherwise, we considered the image object containing the point to be a false negative. Likewise, the seedling objects not containing a seedling geolocation point were considered false positives, and the rest of the non-seedling image objects were accordingly considered true negatives. The overall detection rate (sensitivity) for conifer seedlings in the independent test dataset was 75.8%: 113 out of the 149 seedlings surveyed in the test site were detected (
Table 3). The classification model had a commission error (false positive) rate of 12.4% and an omission error (false negative) rate of 24.2%. The moderate imbalance between the omission and commission errors suggests that our workflow tends to underestimate the seedling density. This is understandable given the small size of the target seedlings and the complex environmental conditions in which they are found. The overall Kappa coefficient was 0.810 and the area under the receiver operating characteristic (ROC) curve was 0.93.
We noted differential errors between the pine seedlings (86% detection rate, n = 124) and spruce seedlings (24% detection rate, n = 25), which were relatively rare in our test study area. This issue could likely be addressed using multiple classification models (one per species), although we did not attempt it here.
Plot-level associations between the CART-predicted stem numbers and field observations produced a Pearson’s correlation coefficient of
r = 0.98 in the test dataset (
Figure 6). The plot accuracies in the test dataset ranged between 62.5% and 100%, with a mean accuracy of 86.2%. Once again, we observed a tendency of our workflow to underestimate when large numbers of seedlings are present (top end of
Figure 6). This is not a critical problem, although, as plots with large numbers of small seedlings would be considered fully stocked, a moderate underestimation bias in these conditions can be tolerated in practical applications.
Expressing the stem numbers derived from remote sensing on a per-hectare basis produced an estimated seedling density of 2160 stems/ha, versus 2500 stems/ha from the reference data. This represents an underestimation of 320 stems/ha (13.6%), which is consistent with the UAV data’s tendency to slightly underestimate seedling counts, noted previously. Puliti et al. [
20] estimated the stem numbers of mature trees in 38 fixed-area plots in Norway using photogrammetric data from UAVs, and reported root-mean-square errors of 538 stems/ha (39.2%). While less accurate than our results, the area-based regression analyses used by the authors is quite different than the object-detection approach used here.
We could not find any published literature on the automated classification of very small seedlings (less than 5 years of age) against which to compare our results. Hall and Aldred [
5] detected just 44% of the seedlings with a smaller than 30 cm crown diameter using the manual interpretation of 1:500 scale color-infrared imagery. Our results are slightly less accurate than those of Sperlich et al. [
32], who reported an 88% overall accuracy from the photogrammetric detection of 219 mature tree crowns. Not surprisingly, our accuracy is lower than that in studies using UAVs to detect mature trees in plantation contexts, which present a simpler classification problem, in which individuals are spatially and structurally homogenous. For example, Torres-Sánchez et al. [
21] reported a 95% accuracy using photogrammetric data on 54 olive trees, and Wallace et al. [
17] reported a 98% detection rate of 308 eucalyptus plants in rows using UAV LiDAR data. Our results exceed those reported by Ke and Quackenbush [
13] (70% user and producer accuracy) in their classification of individual trees in single-scene forest stands from piloted aerial imagery, and those of Chisolm et al. [
33] (73% overall accuracy) in a below-canopy LiDAR survey of mature trees.
3.3. Challenges with Small Seedlings and Clusters of Seedlings
Many of the omission errors we encountered arose from image-segmentation challenges. Small individuals (down to 10 cm height with 5 cm crown diameter) and clusters of seedlings with contiguous crown types posed problems for our workflow (
Figure 7). While millimetric spatial resolution helps identify fine features in our environment, it comes at the cost of spatial and spectral heterogeneity. While the CART algorithm is efficient at negotiating this heterogeneity, it can do so at the expense of specificity. To guard against this tendency, we pruned the decision tree to just two rules.
3.4. A Sample-Based Approach to Silvicultural Surveys
A significant portion of our study was devoted to working with a sample-based survey approach, rather than conventional wall-to-wall mapping: as is common in remote sensing. This approach achieves significant time savings in terms of field data collection and avoids additional photogrammetric and orthomosaic processing costs. We estimate that a standard wall-to-wall UAV survey over the training site would require a flight time of 33 minutes, during which the platform would fly 9.8 km (
Figure 8a). Alternatively, a sample-based survey of the same area could take place in under six minutes and cover just 2.8 km, with a time savings of 81% (
Figure 8b). While we acknowledge that wall-to-wall surveys of forest-harvest areas the size of the ones we worked in are currently possible with UAV platforms, and that wall-to-wall surveys (i.e. census) may provide incremental benefits to sampling, we contend that sample-based flight planning is currently underutilized by the UAV community, and may be crucial to developing operational workflows.
Our workflow is based on spectral variables from a standard RGB camera alone, with no geometric pre-processing and no secondary photogrammetric products. This approach has its pros and cons. For example, the exclusion of spatial and structural variables from our workflow limits our approach to applications where seedlings are spectrally distinct from their surroundings; hence, our requirement for seasonal leaf-off conditions. The main benefit is a streamlined workflow that simplifies the survey and processing procedures. Nex and Remondino [
34] estimate that the placement of ground control points and photogrammetric processing constitute 55% of the time effort required to perform a photogrammetric UAV survey, compared to 25% for flight planning and image acquisition. Alternative workflows to ours that incorporate multiple datasets often require precise geometric integration using ground-control points, which must be laid out and surveyed prior to image acquisition. Photogrammetric processing also adds significant computational costs, and may introduce ground-object distortions [
35], moaicking artefacts [
36], and radiometric inconsistencies. For example, Borgogno-Mondino et al. [
37] explain how color-balancing algorithms embedded in commercial image-processing packages can degrade the radiometric quality of the resulting orthomosaics and limit the effectiveness of derived spectral indices. These issues will certainly diminish in time, given the rapid advancement in direct georeferencing technology [
34,
38], alternative spatial processing routines [
39], and integrated sensor systems. As a result, we expect future studies to find incremental value in photogrammetric data for forest-regeneration surveys, as other authors have reported with the detection of mature trees [
2,
21,
32]. In the meantime, the benefits of the simplified workflows for operational projects are substantial.
Despite these benefits, our simplified sample-based workflow also contains a number of drawbacks. The lack of geometric correction means that images contain relief distortion and optical lens distortion, which introduce variability into the seedlings’ appearance. Excluding the geometric correction from the workflow also introduced variability into the spatial resolution of the resulting images, with the ground-sample distance (GSD) being just a simple function of the flying altitude and camera lens focal length. We reduced this scale effect by outfitting our UAV with a laser rangefinder, which allowed us to hover at exactly 15 m above ground level during imaging. With this, we ensured a common GSD of 3 mm at nadir (5 mm for the NIR camera), which became 3.2 mm at the edge of the circular plot over flat terrain. Even in slightly uneven ground (slopes in our study area did not exceed 10%), the GSD at the low side of a sloping lot did not exceed 3.4 mm. A standard UAV equipped with a conventional GNSS and barometer would be unable to acquire images in such a consistent manner. While the seedlings at the edge of plots appeared up to 30% smaller than those at the center, this effect is unlikely to decrease the detection rate, except perhaps for very small seedlings arising from natural regeneration. The occlusion by taller vegetation could also impair the detection of those seedlings close to the edge of the plots. Once again, though, this is unlikely to decrease the detection, because most shrubs were devoid of leaves at the acquisition date, and there were no saplings or mature conifer trees within the plots. Additional challenges in our workflow arose from the substantial spectral variability caused by the changing illumination conditions during our flights. While this variability could be reduced with the use of an integrated irradiance sensor, operational workflows could benefit from several classification models designed to account for diversity in seedling species, radiometric conditions, and surrounding vegetation, as well as for other sources of spectral variability. We encourage future researchers to assess the value of incorporating scene-level variables that categorize samples on the basis of brightness, time of day, latitude, greenness, and other factors.
Finally, we note that the Regeneration Standards of Alberta [
1] call for between 2.77 and 12.4 sample plots per hectare, depending the size of the harvest area being assessed. In a real-world scenario, this means that we would need to increase our sampling intensity substantially (4 or 5 fold) over the design used in this research. However, it was not our intention to conduct actual stocking assessments, but rather to create and evaluate a workflow that could perform efficiently in this respect.
4. Conclusions
In this study, we assessed the capacity of optical photography from an unmanned aerial vehicle (UAV), to perform coniferous-seedling detection in an object-based environment. The 75.8% overall detection rate of the ground-surveyed seedlings in an independent test site (n = 149) demonstrates the utility of our approach. Error analytics revealed a slight tendency to underestimate seedlings, although the plot-level associations with ground surveys were very high (r = 0.98, n = 14). Red-edge imagery offered no significant advantage over the data from a standard RGB camera, and our final decision tree was comprised of a simple two-rule model based on just two spectral vegetation indices, the Green-Red Difference Index and the Blue-Green Difference Index. We found spatial and textural variables to be unnecessary for identifying coniferous seedlings in the conditions we assessed. Our workflow relies on seasonal leaf-off conditions when grasses are senesced and seedlings are spectrally distinct from their surroundings, and it would not be expected to perform with the same efficiency at other times of the year, or with deciduous seedlings.
The seedling surveys conducted with UAVs are feasible and efficient, but further research is required. For example, we expect that the increased variability encountered under operational conditions will require the complementary use of several models or the application of more sophisticated machine-learning approaches. We encourage other researchers to explore the detectability of other seedling species in new environments, and at different times of the year. Also, a full stocking-assessment workflow would require delineated seedlings to be assessed for other attributes, including height, species, and condition: challenges that will probably require enhanced spectral information and 3D data collected using light detection, ranging or photogrammetry.
The trend towards quantified vegetation surveys at this level of detail is a promising development, both for forest management and in the larger context of restoration. Perhaps the most useful area for the future development of the use of UAVs in forest management is repeatability. This is a progressive approach, as the future use of UAVs may not depend as much on the correlation of the metrics derived from the data collected with unmanned aerial vehicles to field observations as on simple data and mensuration consistency. Wallace et al. [
18] have been pioneers in this regard, publishing a study focused on the repeatability of the measurements taken using unmanned aerial vehicles. We encourage other researchers to assist in the development and reporting of forest mensuration workflows based on remote sensing to establish a body of literature that provides a foundation from which the consistency and repeatability of these novel techniques can be assessed.