Assessment of Errors Caused by Forest Vegetation Structure in Airborne LiDAR-Derived DTMs

Abstract

Airborne Light Detection and Ranging (LiDAR) is a survey tool with many applications in forestry and forest research. It can capture the 3D structure of vegetation and topography quickly and accurately over thousands of hectares of forest. However, very few studies have assessed how accurately LiDAR can measure surface topography under forest canopies, which may be important, for example, in relation to analysis of pre- and post-burn surface height maps used to quantify the combustion of organic soils. Here, we use ground survey equipment to assess digital terrain model (DTM) accuracy in a deciduous broadleaf forest, during both leaf-on and leaf-off conditions. Using the leaf-on LiDAR dataset we quantitatively assess vertical vegetation structure, and use this as a categorical explanatory variable for DTM accuracy. In the presence of leaf-on vegetation, DTM accuracy is severely reduced, with low-stature undergrowth vegetation (such as ferns) causing the greatest errors (RMSE > 1 m). Errors are lower under leaf-off conditions (RMSE = 0.22 m), but still of a magnitude similar to that reported for mean depths of burn in fires involving organic soils. We highlight the need for adequate ground control schemes to accompany any forest-based airborne LiDAR survey which require highly accurate DTMs.

1. Introduction

Airborne Laser Scanning (ALS) is a technology used to map the three-dimensional structure of the terrestrial environment for diverse applications such as construction, archaeology, flood modelling and forest science. The method employs a Light Detection and Ranging (LiDAR) scanner mounted aboard an aircraft, which times the return of emitted laser pulses from surfaces on or near the ground, measuring the distance to the object from the sensor. In a forest environment, ALS data can be used to extract useful information for researchers and foresters; e.g., Digital Terrain Models (DTMs) are models of the ground surface, Canopy Height Models (CHMs) are reconstructions of the upper limits of the forest canopy, and vertical and horizontal structural characteristics such as number of strata [1] and crown dimensions [2,3] are useful for allometric estimations of above ground biomass (AGB) of forests. DTMs are perhaps the most important LiDAR-derived forest metric, because the accuracy of nearly all other models and metrics are dependent on them. Examples of typical DTM accuracies achieved for each biome are given in

Table 1. Vertical accuracies reported in previous studies along with their respective biomes and references. The vertical accuracy metric differs between studies, therefore their reporting method is defined in the “Metric” column.

Source	Biome	Vertical Accuracy (m)	Metric
[4]	Old growth tropical forest	1.95	RMSE
[4]	Secondary tropical forest	1.44	RMSE
[4]	Selectively logged tropical forest	1.62	RMSE
[5]	Steep Mediterranean shrubland	0.13–0.41	RMSE
[6]	Temperate conifer	0.21	RMSE
[7]	Temperate conifer	−0.05/0.12	Mean/SD
[8]	Temperate conifer	0.31/0.29	Mean/SD
[9]	Temperate conifer	0.59	RMSE
[10]	Temperate conifer	0.24	RMSE
[11]	Temperate deciduous and conifer	1.22	RMSE
[11]	Temperate grass	0.37	RMSE
[12]	Temperate mixed	0.38	N/A
[11]	Temperate pine	0.45	RMSE
[11]	Temperate shrub	1.53	RMSE
[13]	Tropical forest	1.8	Mean
[14]	Tropical forest	0.43	RMSE
[15]	Tropical forest	0.37	RMSE
[4]	Tropical swamp forest	1.64	RMSE
[16]	Tropical swamp forest	0.16 and 0.41	RMSE
[17]	Tropical swamp forest	0.12	RMSE
[17]	Tropical swamp forest burn scar	0.19	RMSE

, showing there are very few DTM accuracy reports in temperate deciduous forest environments.

DTM accuracy can be affected by horizontal and vertical accuracy of survey instruments (e.g., Global Navigation Satellite Systems (GNSS), total stations (TS), inertial measurement units (IMU) aboard aircraft [18], data collection parameters, site characteristics, data processing methods and algorithms. Typically, the highest quality GNSS equipment can give point coordinates accurate to better than 1 cm in open-sky conditions (i.e., no canopy cover, which may block satellite signals [19]), whereas total station measurements are not affected by canopy cover, and point accuracy can be 2–3 mm. ALS point cloud vertical accuracy is usually reported at approximately 1–20 cm [5,17,20,21,22,23]. Data collection parameters include instrument parameters such as mirror tilt angle [24], laser pulse density and flight height [7], and scan angle [25]. Influential site characteristics include the presence of vegetation and/or slope, which are the largest contributors to DTM error, causing errors exceeding those caused by instrumental or methodological error [4,5,7,8,15,22]. Furthermore, point cloud classification algorithms may also induce DTM errors by misclassifying understory or ground-cover vegetation as ground returns, this might be particularly problematic in open-canopy forest where undergrowth may be particularly dense [17,26].

DTM accuracy assessments are a major challenge in forest environments because of the need to measure the position of “true” ground control points, which are independent of the influences of vegetation. It is difficult to take GNSS control points under leaf-on canopies because the satellite signals are blocked by the water in the canopy leaves, and undergrowth density can block the line of sight between the total station instrument and the survey prisms [15,27]. Perhaps for these reasons, very few studies have formally assessed how vertical vegetation structure can affect DTM accuracy in broadleaf forests (Table 1). In leaf-off conditions, such as in winter in deciduous forests, or after a major forest fire, the canopy may open sufficiently to take the type of accurate ground control points required to assess DTM accuracy [11,28].

To date no studies have applied a generic vertical vegetation structure metric as an explanatory variable of DTM accuracy, and most studies reporting DTM accuracy have not quantitatively assessed overlying vegetation structure [7,8,17,20,22]. In situations where it is important to produce highly accurate DTMs (especially in places where ground control points cannot be collected reliably, e.g., in dense forests), there is a need to identify areas which may be prone to larger errors, so that uncertainty can be quantified adequately.

Vertical vegetation structure can be measured using ALS point cloud data and applied across different forest types [1,29,30]. Gap probability (Pgap) can be calculated from discrete return LiDAR data, and is the proportion of vegetation returns reflected at a given height to the number of returns per plot (including the number of ground returns). Pgap reflects how much vegetation is obstructing the ground at a given height, and can therefore be applied to any classified ALS point cloud to compare different forest environments. It serves as a much more descriptive metric than (e.g.,) canopy height, which provides no information about the vegetation structure between the ground and the top of the canopy [1].

The aim of this study is to assess how vertical vegetation structure affects DTM accuracy in a heterogenous forest plot. The objectives are:

• Using a network of ground control points (from a GNSS and TS) to produce a reference DTM to assess the accuracy of ALS-derived DTMs.
• Compare the accuracy of DTMs from ALS surveys in both leaf-on and leaf-off conditions.
• Quantitatively classify the different vertical vegetation structure categories in the forest plot, and compare DTM accuracy in each category.
• Compare DTM accuracy with vegetation density at different vertical strata, independently of vertical vegetation structure categories.

2. Site Description, Materials and Methods

Wytham woods is a 400 ha mixed deciduous ancient and secondary woodland situated northeast of Oxford, Oxfordshire, UK (51°46′ N, 001°20′ W, UK National Grid SP 461 081). The climate is typical for southern England (mean air temperature 10.1 °C, precipitation 730 mm y⁻¹ [31]). Wytham was selected for this project because it has been surveyed with ALS twice since 2007 (during both leaf-on and leaf-off conditions). Although situated on a hill, forest topography is mostly less than 10° and being a protected site the topography has not been altered during the period between or since the ALS surveys (Figure 1).

Both leaf-on and leaf-off LiDAR datasets were collected by the Natural Environment Research Council (NERC) Airborne Research Facility (ARF). Airborne LiDAR surveys were flown on 24 June 2014 and 9 March 2009 respectively. The leaf-on dataset flight parameters were ~2500 m agl (altitude) and ~134 knots (ground speed). The LiDAR instrument was a small-footprint Leica ALS50-II LiDAR system with full waveform and discrete recording capabilities (up to 4 returns per emitted pulse). The LiDAR pulse repetition frequency (PFR) was 96.8 kHz, field of view (FOV) was 35°, with 99.8% return rate and average point density of 0.918 m². For the leaf-off dataset, the flight parameters were ~ 1181 m agl at ~135 knots. The ALS instrument PFR measurements were at 84.4 KHz, with a 24° FOV, with 100% return rate. Mean return densities for the leaf-on and leaf-off datasets were 3.17 and 5.06 returns m⁻² respectively, above the threshold density of 0.6 points m⁻² where DTM generation is said to become inaccurate [32]. Furthermore, the density of the leaf-on dataset was approximately 62% of the leaf-off density, which is above the 50% threshold whereby DTM accuracies become statistically different, meaning they are comparable without normalisation [33,34].

A site was selected based on its highly heterogeneous vegetation structure (Figure 2), including forest clearances as well as areas of dense canopy and understory vegetation, and its relatively flat topography. It was also situated centrally in the LiDAR swath, so that biases due to scan angle were equally distributed throughout the site [9]. Ground data for the reference DTM were collected at the selected site between 23–28 January 2017 using a Leica VIVA GS10 Global Navigation Satellite System with a baseline between the GNSS base station and survey site of approximately 950 m; the rover antenna was mounted on a 2 m survey pole. A Trimble M3 total station was used to measure ground points on and between the marked GNSS ground control points (GCPs). The instrument position was measured in all three locations using the GNSS antenna. The prism operator paced approximately 5 m between measurements, giving a final point density of approximately 596 points per hectare.

Total station datasets provide very high accuracy point measurements in the X, Y, Z axes relative to the total station instrument. To georeference these positions to absolute coordinates (British National Grid OSGB36, Newlyn vertical datum), 16 matching GNSS-derived datapoints were used. The GNSS rover points were differentially corrected to the GNSS base station position in Leica Geo Office, which improves relative accuracy to within 0.04 m. Rinex files from a permanent Ordnance Survey base station (baseline from GNSS base station was 6.7 km) were used to improve absolute accuracy of both GNSS base station and rover points. Absolute accuracy in X, Y, Z axes of GNSS rover points ranged between 0.002 m and 0.034 m. Total station surveys were individually transformed from relative to absolute grid coordinates (OSGB36, OSTN02) using a least squares method using GNSS points (delta Easting/Northing/Elevation, X/Y/Z rotation (

Table 2. Transformation statistics of TS datasets to GNSS control points (n = 16).

TS to GNSS	σX	σY	σZ	σXYZ
Mean (m)	0.000	0.000	0.000	0.023
Max (m)	0.037	0.058	0.048	0.001
RMSE (m)	0.016	0.030	0.022	0.023

).

To ensure the ALS dataset was georeferenced correctly, an additional four control sites were surveyed (Figure 3). The control sites were situated either on forest service roads (compacted gravel) or local roads (asphalt). A total of 438 TS control points were taken, using 21 GNSS control points distributed across four locations for transformation to absolute coordinates. The TS control points were rasterised (0.5 m resolution) in CloudCompare (v2.6.2) and were compared to ALS returns in the same area to calculate the transformation parameters. The same transformation was applied to the entire leaf-on ALS dataset (

Table 3. Transformation matrix of ALS point cloud based on 438 TS control points on uncovered, hard road surfaces.

Rotation	X	Y	Z	Translation
X	1	0.000012	0.000147	−0.06666
Y	−1 × 10−5	1	−0.0001	0.021064
Z	−0.0001	0.000103	1	0.700736
	0	0	0	1

). At the 438 control points, vertical bias improved from −0.64 (1σ = 0.10 m) to 0.07 (1σ = 0.07 m), and overall accuracy improved from RMSE = 0.64 m to RMSE = 0.11 m. The ALS leaf-off dataset was aligned at the control locations to the reference dataset improving bias from −0.19 m to 0.002 m (±1σ = 0.22 mm).

The ALS point clouds were classified as ground or non-ground using LAStools [35], by removing noise, and using the “lasground” tool which only considers last returns, and is based on a mesh simplification approach as outlined by [36]. DTMs were produced from ground returns using an Inverse Distance Weighting algorithm at 1 m resolution to account for site microtopography. This algorithm has proven to yield highly accurate DTMs [17]. A control dataset was produced from the TS ground control dataset (DTM_TS) using the same method (n = 657). The control DTM was subtracted from the ALS-derived DTM (DTM_ALS) to produce difference rasters. The height differences were taken at 1750 randomly located points (at least 1m apart to avoid repetition) within the survey grids (Figure 4).

In order to explain the effects of vegetation on DTM accuracy, vertical vegetation structure was assessed using gap probability (Pgap) in the ALS leaf-on dataset. Pgap can be used to describe the amount of backscatter of large-footprint waveform LiDAR from vegetation surfaces, and has also been applied to discrete return data at a plot level [1,19]. 20 m grid cells were used to optimise coverage of ALS and TS ground points (Figure 4), and the vertical vegetation structure was assessed for each cell. Using discreet return ALS data, three vertical vegetation strata were identified by examining the density of returns through the vegetation column in all 24 grid cells (Figure 5), and cross referencing this plot with observations made in the field. The three strata comprise of dense ground-cover plants such as bramble (Rubus fruticosus), nettles (Urtica dioica), and brackens (e.g., Pteridium aquilinum) which peak at 3.5 m (red line), mid-story vegetation up to 10 m which include large shrubs and short-stature trees such as alder (Alnus glutinosa), field maple (Acer campestre) and hazel (Corylus avellane) (as well as immature Oak, Ash and Sycamore trees), and above 10 m are dominant canopy Oak (Quercus robur), Ash (Fraxinus excelsior) and Sycamore (Acer pseudoplatanus).

For each 20 m grid cell, the vertical gap probability (Pgap, metric describing the proportion of vegetation returns at specific heights) was calculated for each stratum (adapted from [1]).

$$Pgap\left(z\right)=1-\frac{{\displaystyle \sum }{w}_i \left(z_i\right)}{W}$$

where W is the per grid cell sum of the number of returns, w is the sum of ground returns per grid cell within the vertical limits of each stratum, z.

The grid cells were categorised according to the relative differences between Pgap in all three strata using two cluster analysis techniques, K-means and hierarchical clustering (via rattle, hclust and cluster packages) in RStudio (v1.0.136). The optimal value of parameter K (number of clusters based on minimisation of within-cluster sum of squares) was estimated using “the elbow method” by plotting the variance explained as a function of the number of clusters [37]). Once the optimal number of clusters was calculated (K = 6 in this case), the grid cells were categorized into clusters using a K-means fit algorithm, and were analysed and validated by representing the result of the K-means fit graphically in a 3D plot (Figure 6). This was compared to a cluster dendogram (Figure 7) in which categories are produced using hierarchical clustering employing a Euclidean distance matrix as an input for the clustering algorithm to reduce the intra-cluster variance. Using the vertical profile density plots (Figure 7), ground observations (Figure 8), and the grid cell distribution in relation to the canopy (Figure 9), the six categories (A to F) were defined and are described in

Table 4. Descriptions of each vertical vegetation structure category, with the mean Pgap of grid cells within each category, and 1 standard deviation of the mean.

Structural Category	Description	Location	Mean Pgap	1σ
A	Some undergrowth and mid-story, very dense canopy	Partially-closed canopy forest	0.87	0.03
B	Little undergrowth, dense mid-story and very dense canopy	Partially-closed canopy forest	0.93	0.01
C	Very dense undergrowth, dense mid-story, sparse/no canopy	Clearing edge	0.73	0.01
D	Dense undergrowth, sparse mid-story and dense canopy	Clearing edge, small forest gap	0.72	0.07
E	Some undergrowth, sparse/no mid-story and canopy	Clearing	0.17	NA
F	Sparse/no undergrowth and mid-story, very dense canopy	Closed canopy forest	0.92	0.03

.

Accuracy assessments were performed by differencing the TS-derived DTM and ALS-derived DTMs at 1m resolution (DTM_TS and DTM_ALS respectively). Firstly, the effects of site microtopography (slope) on DTM accuracy were examined using a Generalized Additive Model (GAM) (which is useful for uncovering patterns in continuous data with non-normal distributions). Then, the effects of vertical vegetation structure on DTM accuracy were examined in three ways, (1) by comparing DTM accuracy in leaf on and leaf off conditions (because only some understory vegetation is evergreen in this plot), (2) by comparing the different vertical vegetation structures (Pgap categories) in both leaf-on and leaf-off datasets, (3) comparing mean DTM accuracy per grid cell as a function of Pgap values for each vertical stratum. To assess the statistical significance of differences in DTM accuracy between the different conditions and vegetation categories, the data were assessed for heteroscedasticity and unequal variances using Bartlett and Levene tests respectively. All data were heteroscedastic; therefore comparisons were made using White-adjusted Anova from the car package in R, and pairwise comparisons are made using Tukey’s honest significant difference tests.

3. Results

The median slope within the plot was 5.7°. The relationship between vertical DTM residuals and slope at 1m resolution was examined using a non-parametric GAM. Slope has no meaningful effect on residuals, with slope explaining 0.25% of the deviance, and a poor goodness of fit (adjusted R² = 0.002, Figure 10).

Leaf-off conditions improved overall DTM accuracy by 61 cm (RMSE_leaf-off = 0.22 m vs. RMSE_leaf-on = 0.83 m, n = 1750) at 1 m resolution (Figure 11), demonstrating that leaf-on vegetation induces larger positive DTM errors. Leaf-on and leaf-off DTM residuals were significantly different (F = 3086, df = 1, p < 0.001) and consequently, RMSE differences between leaf-on and leaf-off DTM were also significantly different (F = 20.02, df = 1, p < 0.01).

In each of the six vertical vegetation structure categories of Table 4, DTM accuracy (RMSE) was better in leaf-off than leaf-on conditions (Figure 12), with the largest/smallest difference being 0.81/0.10 m in structural category C/E (edge of clearing, very dense undergrowth, with some mid-story and no canopy/grassy clearing). Category C and D grid cells show inaccurate DTM heights in both leaf-on and leaf-off conditions, apparently because of the persistence of dense undergrowth vegetation (such as brambles) over the winter. Thick canopy layers are more penetrable by LiDAR than is dense undergrowth, as seen by the better accuracies in categories A–B vs. those in C–D. The range of vertical residual errors is represented in Figure 13. There were only 14 pairs which were not significantly different from each other. In leaf-off conditions, residuals in all structural categories were not significantly different to another and were close to zero (mean = 0.07 m, 1s.d. = 0.20 m). In leaf-on conditions (with the exception of category E), there was marked differences between structural categories, which were considerable higher than in leaf-off conditions (mean = 0.71 m, 1s.d. = 0.43 m).

The relative impact of vegetation density (Pgap) at each vertical stratum on DTM error was assessed at the grid cell-scale for the leaf-on DTM (because the greatest errors are found in leaf-on conditions). DTM error increases with vegetation density at the lowest stratum only (<3.5 m). Vegetation density at the mid- and upper strata (3.5–10 m, and >10 m respectively) has little effect on DTM error (Figure 14).

4. Discussion

To date, few studies using airborne laser scanning (ALS)-derived Digital Terrain Models (DTMs) in forests have included the type of adequate ground control required to assess DTM accuracy, and fewer have quantitatively assessed how vegetation vertical structure affects this accuracy in broadleaf forests. While many forest applications of ALS survey data may tolerate accuracies of better than 0.5 m (such as forest height and above-ground biomass applications), applications such as tropical and temperate peatland depth of burn (DoB) mapping and peat subsidence estimation are far more sensitive to these errors because changes in ground height needing to be identified are typically less than 0.5 m in magnitude [17,20,23]. The results of the present study suggest that without an adequate ground control scheme, such applications may be prone to significant positive biases, especially in areas of leaf-on, open canopy forest with large amounts of ground cover.

Our assessment of the effects of vertical vegetation structure in a heterogenous forest plot on DTM accuracy has performed accuracy assessments between ALS-derived DTMs and a reference DTM derived from ground surveys using a total station and GNSS. Results demonstrate that leaf-on vegetation causes greater DTM error (RMSE = 0.83 m) than leaf-off vegetation (RMSE = 0.22) across all vegetation categories. Furthermore, DTM accuracy is not affected by all vegetation structures equally; with dense understory vegetation such as ferns and brambles causing the greatest positive DTM errors. Grassland vegetation yields the most accurate DTMs. This is consistent with previous qualitative [20] and quantitative assessments in forest environments [11]. [15] measured DTM accuracy as a function of canopy density in a Malaysian woodland, however they did not attempt to use vertical vegetation structure as an explanatory variable. Here we have shown that ALS datasets themselves can be used to extract information about vegetation structure that relates directly to the DTM accuracy, showing that low-stature ground cover plants (<3.5 m) obscure most ground returns and cause greater error (RMSE > 1 m).

The results of this study are demonstrably applicable to other settings, and complement other studies; e.g., ref. [17] found dense ground-covering vegetation obscured more ground returns than intact tropical peat swamp forest, and overall leaf-on DTM errors were similar to those in [11] (also in temperate broadleaf forest). Limitations remain, including (1) A relatively low return density (approximately 3–5 returns m⁻²), because [38] found DTM errors in tropical broadleaf forests increased with decreasing ALS return density, suggesting that DTM errors could be reduced by increasing return density (e.g., reducing flight altitude). (2) Errors induced by leaf litter were not quantified; DTM_TS was produced using exact ground points from a total station survey, however DTM_ALS will always be erroneous in the presence of leaf litter because measurements cannot penetrate the leaf litter. Here the best accuracies in leaf-off conditions were approximately 20 cm, with mean residuals of less than 0.04 m (1sd < 0.18 m) for structural categories with little vegetation <3.5 m tall (structural categories A, B and F, e.g., grid cells 1, 2, 11, 13), which could be caused by the leaf-litter layer. (3) The results presented here show how DTM accuracy is relatively affected by vegetation structure, and as such they cannot be applied absolutely to other forest environments (i.e., these are not correction factors).

Using detailed surveys made in just under 1 ha of deciduous forest, this study has demonstrated how different vertical vegetation structures affect DTM accuracy. Although the overall study area is relatively small, the sample features a variety of different vertical structures which are found across the woodland (i.e., a mosaic of clearings, dense undergrowth, mid-story trees, and mature canopy trees on low-relief slope) which is likely to represent the wider forest environment. The different ratios of vegetation strata are represented by the vegetation vertical categories and are sampled at least once within the plot (with the exception of category E, grassland clearing). The size of sampling area is also consistent with previous studies [8,13].

Further work could include a continuation of the analysis presented in Figure 14, whereby DTM error is directly compared to vegetation structure metrics (e.g., Pgap). If this relationship could be directly modelled using data from more reference plots, and evaluated over a larger area, DTM error might be predictable from vegetation structure alone.

5. Conclusions

This study highlights the requirement for ground controls for DTM extraction from ALS point clouds in forest areas, especially in areas where undergrowth or ground cover vegetation is prevalent. For studies which require high DTM accuracy (e.g., tropical peat swamp forest depth of burn measurement), it is recommended that extensive ground control points are used across a range of vegetation structures to assess the accuracy of DTMs [20], in order to account for biases caused by vegetation cover. Precautions for the greatest DTM errors should be taken in areas characterized by dense ground-cover vegetation, since ground returns are most likely to be obscured here. These areas could easily be identified using Pgap statistics or even a simple canopy height model. Future studies should aim to quantify these DTM biases, as in environments involving burning of organic soils, they may lead to very large over-estimates in greenhouse gas emissions calculations.