Accuracy Assessment of Point Clouds from LiDAR and Dense Image Matching Acquired Using the UAV Platform for DTM Creation

Abstract

In this paper, the results of an experiment about the vertical accuracy of generated digital terrain models were assessed. The created models were based on two techniques: LiDAR and photogrammetry. The data were acquired using an ultralight laser scanner, which was dedicated to Unmanned Aerial Vehicle (UAV) platforms that provide very dense point clouds (180 points per square meter), and an RGB digital camera that collects data at very high resolution (a ground sampling distance of 2 cm). The vertical error of the digital terrain models (DTMs) was evaluated based on the surveying data measured in the field and compared to airborne laser scanning collected with a manned plane. The data were acquired in summer during a corridor flight mission over levees and their surroundings, where various types of land cover were observed. The experiment results showed unequivocally, that the terrain models obtained using LiDAR technology were more accurate. An attempt to assess the accuracy and possibilities of penetration of the point cloud from the image-based approach, whilst referring to various types of land cover, was conducted based on Real Time Kinematic Global Navigation Satellite System (GNSS-RTK) measurements and was compared to archival airborne laser scanning data. The vertical accuracy of DTM was evaluated for uncovered and vegetation areas separately, providing information about the influence of the vegetation height on the results of the bare ground extraction and DTM generation. In uncovered and low vegetation areas (0–20 cm), the vertical accuracies of digital terrain models generated from different data sources were quite similar: for the UAV Laser Scanning (ULS) data, the RMSE was 0.11 m, and for the image-based data collected using the UAV platform, it was 0.14 m, whereas for medium vegetation (higher than 60 cm), the RMSE from these two data sources were 0.11 m and 0.36 m, respectively. A decrease in the accuracy of 0.10 m, for every 20 cm of vegetation height, was observed for photogrammetric data; and such a dependency was not noticed in the case of models created from the ULS data.

1. Introduction

The subject of accuracy comparison of two leading airborne data sources, airborne laser scanning (ALS) and photogrammetric image-matching, has been discussed in several References [1,2,3]. Such a discussion is also present in a comparison of close-range datasets, i.e., terrestrial laser scanning data was compared with the Structure-from-Motion technique in many applications [4,5,6,7]. The accuracy of the Digital Terrain Model (DTM) is crucial for applications related to hydrology and safety. It was proved that the accuracy of the model influences the hydraulic modelling results, and flood hazard area detection [8]. Using optical data in this issue has rapidly grown after the development of the Dense Image Matching (DIM) algorithm [9,10,11,12].

Over the past decade, there was a significant development in photogrammetric techniques based on images from Unmanned Aerial Vehicles (UAVs), for generating digital elevation models, including surface models and filtered digital terrain models [13]. UAV photogrammetry can also be used in archaeology, for generating high-resolution and scaled 3D models of complex sites [14], and in monitoring, for example, monitoring coastal areas [15]. The DTM can be used for levee monitoring, which is related to the data tested in this paper. Through comparison of the DTMs that are acquired in different seasons, it is possible to examine if some damage to the levee structures has occurred, and as a result, to prevent a flooding hazard. Structure-from-motion algorithms deliver point clouds, which are characterized by a high spatial resolution. The largest disadvantage of point clouds achieved from photogrammetry is the vegetation influence, decreasing the vertical accuracy of the derived digital terrain model [16,17]. Many filtering algorithms for point cloud filtering can be tested [18], to eliminate the influence of both high vegetation [19] and low vegetation [20]. However, not every mission will be influenced by the vegetation. For example, when mapping bare earth areas, such as mine sites, earthwork projects or other plane areas [21,22], UAV photogrammetry is a cost effective and accurate technique to use. In cases where the DTM is not significantly important and the only product to assess is the Digital Surface Model (DSM), UAV photogrammetry can provide accuracies better than 0.10 m [15,23,24,25], even with compact cameras.

Ultralight laser scanners, which can be mounted on UAVs, have been developed in recent years [26]. They can provide elevation data, not influenced much by vegetation, and they deliver initial data to digital terrain models that describe the bare ground in an accurate manner. A gradual development of UAV laser scanning (ULS) technology can be observed on the global market [27,28]. The important part of a ULS system is the navigation unit, i.e., integrated Global Navigation Satellite System with Inertial Measurement Unit (GNSS/IMU). To provide the best results for the point cloud, an accurate distance measurement (laser) and the external orientation parameters of a platform are needed. Moreover, a fast-developing, light and reliable GNSS/IMU market can help to provide effective and useful ultralight laser scanning systems. An increasing number of turnkey solutions are offered by major ULS manufacturers (e.g., Riegl, YellowScan, LidarSwiss, Phoenix, LidarUSA, Sabre Sky-3D, Routescene), which enable users to obtain accuracy that is comparable to airborne laser scanning, and close to direct surveying measurements in the field. ULS is a suitable technology when mapping narrow corridor objects, such as levees, which were investigated in this experiment. The most important problems with high-tech scanners and advanced navigation units are the purchase costs, potential losses, and despite developing miniaturization, the weight of this type of sensor. This issue implies that not every task related to the generation of an elevation model requires the use of a laser scanning system. When the accuracy of the DIM is sufficient and the circumstances of the flying mission are appropriate, photogrammetry with UAV images could be a better solution. While favorable circumstances of flight missions are understood as conducted after the end of vegetation season or just after mowing the grass that covers the levees, in this example of an area (mowing is performed even several times per year), what is directly related to vegetation height has an influence on the DIM results. Investigation in this research compares the results of the DTM accuracy for different data sources, helping to provide an answer to this issue.

In this paper, we conducted an experiment on the vertical accuracy of the generated digital terrain models, from two data sources from the UAV platform in extreme resolution. The results also referred to the land cover of the vegetation, which has a crucial influence on the accuracy to investigate which remote sensing data can be used for the DTM generation. The impact of the resolution was also checked in the tests. After the introduction, in the second section, the collected data, test area, and methodology of the experiment are described. The third section presents the results of a comparison of the digital terrain models from the ULS data and DIM point clouds, to the GNSS-RTK surveying measurements, and a comparison with the DTM from the archival airborne laser scanning. It is worthwhile to note that in all analysis, various land cover described by different heights of vegetation is considered to help define which technology is more suitable for the selected area. The obtained results are discussed and used to draw conclusions in the final part of the paper. The significance of the issues presented in this paper are related to the performance capabilities of the accuracy level of the dedicated UAV, with a very high-resolution scanning sensor, compared with a commonly used low-altitude photogrammetric approach for DTM generation. This considers modern trends in sensor development and the growing resolution of remote sensing data.

2. Materials and Methods

2.1. Experiment Description

The main goal of this experiment was to assess the vertical accuracy of DTM from LiDAR and DIM point clouds acquired with a UAV platform. The evaluation of accuracy on areas covered with vegetation was tested, and information about the accuracy of both techniques on different vegetation heights is provided. The test compared both technologies and referred to the growing season, when the accuracy of the image-based approach could be worse than the LiDAR technique. After this season, when the vegetation influence was lower, the results for both techniques could be more similar for the uncovered areas. It could also be tested in experiments using sensors that provide extremely high-resolution data, reaching a few hundred points per square meter. The effect of the experiment showed information about the possibilities of using both approaches when considering data from UAV platforms: ULS and low-altitude photogrammetry with DIM.

2.2. Platform

In the experiment, one platform was used to collect data in the test area (see Figure 1). The Hawk Moth was equipped with two remote sensing sensors. Its specifications are presented in

Table 1. Hawk Moth platform specification.

Name	Hawk Moth
Developer	MSP
Empty weight	8.5 kg
Max. payload	4 kg
Average cruise speed	5 m/s
Max. cruise speed	12.5 m/s
Hovering time	15 min (with 3 kg payload)
Sensor equipment	1. ultralight laser scanner YellowScan Surveyor (Weight: 1.60 kg), Size (mm): 100 × 150 × 140)
	2. RGB camera Sony Alpha a6000 (weight: 0.47 kg)(24.3 MPix APS-C CMOS sensor)

. The first, is the ultralight laser scanning system YellowScan Surveyor, which is one of the lightest LiDAR solutions on the market dedicated to low-altitude airborne platforms. Its weight was approximately 1.6 kg (battery included). According to the producer, a georeferenced point cloud data collected with the YellowScan Surveyor should be characterized by sub-decimetric accuracy. The Applanix APX15 single board GNSS-Inertial solution, which was a component of this scanner, plays a key role in ensuring the expected accuracy [29]. This sensor used the Velodyne laser sensor (VLP-16), and it is one of the newest laser systems on the market compared to others. A list of UAV-dedicated laser scanners can be found in the example, in References [27,28]. The second sensor, is the RGB compact camera Sony Alpha a6000, which is a very common tool used in photogrammetric projects where UAVs are used. Its APS-C CMOS sensor was 24.3 MPix in resolution. A Sony E-Mount 16 mm with F/2.8 lens, was used in this experiment.

2.3. Test Area and Data Used in the Experiment

The test area was located in the Święciechowska Valley, near the city of Annopol, Lubelskie Voivodeship, Poland (see Figure 2). It contained 6.3 km of levees on the east side of the Vistula river. The test area was covered by high mixed vegetation, agricultural plants, roads, bushes, and trees. The type and height of the vegetation were also different.

The data used in the experiment were acquired on 15–16 May 2017, at an altitude of 50 m above the test area (approximately 48 m above the levee crown), within the inventory of the levee. The length of the laser strips depended on the levee shape (straight flight trajectory preferred, see example in Figure 3), and on the battery life. Eight flight missions were conducted over the test area. The methodology of the performed flights was based on experience with a platform which was equipped with similar sensors, and the processing of data obtained from that platform [30,31]. The mission over the levees was conducted using two strips, with an overlap of 50% for sidelap and 70% for endlap; the control points were signalized with chess planes in the size of 0.5 × 0.5 m and placed regularly on the area to be used during orientation and evaluation, of both LiDAR data and digital images. This configuration of the signalized points was used to improve the absolute elevation accuracy on the mapped levee. Both the control and check points were placed horizontally next to each other, but the Z-coordinates were different, up to several meters (i.e., the levee height). The points were placed on the top and on both sides of the levee. In each cross-section, there were three points measured. One of them was always the checkpoint, and two of them guaranteed that in a case of gross error, we had some redundant observation. The upper part of the area was outside of the test study, and therefore, no check points were distributed there. However, the UAV was registering the data there, and thus, they were included in Figure 3. The additional cross-sections of the terrain were also collected to check the accuracy of the final product of the DTM. These cross sections, measured with GNSS-RTK, were the independent observations without point signalizing. This approach can be used as a valuable assessment of DTM accuracy, because this method is the typical method for levee inventory.

The point cloud from the UAV laser scanning (see Figure 4a) was characterized by a density of 180 points per square meter. Multiple returns have been registered in the point cloud. The first returned laser pulses were approximately 99% of the whole point cloud, and the second only 1%. The mean value of sigma0 in the point cloud (the standard deviation of the interpolated height for each point to the planes, calculated in the algorithm described in Reference [32]), for uncovered areas was 0.02 m, which was related to the size of the noise in the data. The orientation of the LiDAR data was processed in the Applanix software (PosPack), using reference stations from the Active Geodetic Network EUPOS (ASG EUPOS), to adjust the trajectory of the platform with the included parameters of bore-sight calibration angles from the manufacturer and the defined lever-arm parameters. The accuracy of trajectory adjustment, which is a crucial part of the data processing for final accuracy [33], was processed in PosPack and was 0.02 m. This accuracy was reached using the Applanix SmartBase module, which allowed processing of the raw GNSS observations from a network of 4 to 50 reference stations, to compute the atmospheric, clock, and orbital errors within the network. In this case, there were 5 reference stations with baseline lengths from 10 to 50 km. The scanning system used in the experiment was equipped with a single GNSS antenna, initialized on the ground before take-off. The next task was to filter out point clouds and extract bare ground points. This step was processed in Terrasolid software, using Axelsson’s algorithm [34]. The parameters of the ground filtering were set during the testing of the data sample, and the final results of the filtering were obtained when the maximum terrain angle was 80 degrees, the maximum iteration angle was 6 degrees, and the maximum iteration distance was 2 m. During the selection of the filtering settings of the slopes in the terrain, the point cloud density and the size of the gaps in the point cloud had an influence on this decision. The results of filtering were verified in cross sections of point clouds and were manually corrected.

Referring to photogrammetric data, 1285 images with a Ground Sampling Distance (GSD) of 2 cm, collected with the RGB camera with a 70/50 overlap, were oriented in Agisoft Photoscan 1.2.6, based on the control points measured with GNSS-RTK, as signalized in each flight mission, which divided the test area into 8 sections. Each section was adjusted separately. The bundle adjustment with a self-calibration process was performed using ground control points and the images’ position provided by drone navigation sensors. The average accuracy for 73 control points during the bundle adjustment was RMS_X = 0.066 m, RMS_Y = 0.060 m, and RMS_Z = 0.084 m, and for 35 check points was RMS_X = 0.061 m, RMS_Y = 0.063 m, and RMS_Z = 0.068 m. Another 193 check points not signalized with planes were used within the evaluation of the digital terrain models. Detailed results of the bundle adjustment, can be found in

Table 2. Aerotriangulation results for 8 flight missions.

Number of Flight Mission	Number of Photos	Control Points				Check Points
		Number of Control Points	RMSX (m)	RMSY (m)	RMSZ (m)	Number of Check Points	RMSX (m)	RMSY (m)	RMSZ (m)
1	172	11	0.076	0.121	0.106	4	0.084	0.110	0.122
2	173	10	0.055	0.031	0.077	5	0.075	0.028	0.082
3	153	10	0.078	0.042	0.016	5	0.067	0.060	0.067
4	149	8	0.030	0.032	0.027	5	0.028	0.037	0.072
5	168	8	0.034	0.031	0.054	4	0.052	0.048	0.083
6	176	11	0.049	0.058	0.072	4	0.025	0.055	0.076
7	157	8	0.118	0.042	0.026	4	0.088	0.054	0.064
8	137	7	0.047	0.031	0.093	4	0.027	0.082	0.101

.

Oriented images were used to generate the point cloud in Agisoft Photoscan, and its density was approximately 1700 points per square meter. Image-based point clouds were filtered to remove the points that represent the land cover (see Figure 4b). Agisoft has a tool to automatically classify the point cloud into “ground” and “non-ground” classes, which also uses Axelsson’s filtering algorithm. In this study, the following parameters were set: maximum angle of 35 degrees not to exclude the points on sloped levees, maximum distance of 1 m and cell size of 50 m, which considered large areas covered by vegetation without ground points.

Comparing the two views of point clouds from two data sources in Figure 4, the higher density of the image-based point cloud is noticed first. It is also worthwhile to mention that the lack of points in the photogrammetric point cloud is not the result of point cloud filtering, but the result of matching algorithm failure in vegetated areas. Processing these two datasets provided the possibility of generating digital terrain models, which were the final product of all the point cloud processing. In the case of the DIM point cloud, approximately 95% of the points were assigned to ground class, which was caused by the lack of points matched on the ground truth, covered by vegetation. During filtering, the points of low vegetation in large areas were often assigned to the ground class; however, buildings, trees, and other surface objects were successfully filtrated. In contrast to the classification of the DIM point cloud, in dense ULS point clouds, only 5% of the points were classified as bare ground. In this case, in a point cloud, there are many returns, and the extraction of ground points provides different filtering statistics compared to the image-based approach.

The main reference data were 193 check points, measured in 18 profiles using the GNSS-RTK technology (Figure 5). Such cross sections on levees, are a typical surveying measurement, which is collected every few years to control the condition of this infrastructure against flooding. In this research, these points were used to evaluate the accuracy of the tested technologies in the case measurements, whilst referring to bare ground (the accuracy of DTM). The accuracy of GNSS-RTK measurements in this case, was assumed to be 0.02 m.

Additionally, the ULS data were also compared to the typical ALS data collected on 17 October 2011, in a country-wide project (DTM_ALS). This point cloud was in a density of 4 points per square meter, and the resulting DTM_ALS represented an approximated vertical accuracy of RMS_Z < 0.15 m [35]. No destruction was observed between 2011 and 2017. This data source was also the most accurate observation among the remote sensing techniques used to create the DTM for large areas. An airborne LiDAR is commonly used in these applications, and its resistance to the vegetation influence on the accuracy of the final model was tested in many cases. The application of these data in this experiment, should then prove a comparable accuracy level of the tested technologies using data from a UAV, with commonly used solutions.

2.4. Methodology of Accuracy Assessment

The accuracy assessment of the ULS and DIM point clouds was conducted using a geodetic survey, and it referred to the archival ALS data. Within the accuracy examination, the influence of the vegetation on two basic techniques that deliver point cloud data was examined: Image-matching (DIM) and laser scanning from the UAV (ULS). The accuracy assessment methodology is presented in Figure 6.

It is difficult to compare whole point clouds with different characteristics and genesis. It was therefore decided to analyze the terrain models generated from two types of point clouds, because this product is the goal of most studies that use point cloud processing. Subsequently, the following elevation models were generated in the ArcMap software, providing listed parameters during their creation:

• DTM from ULS data (DTM_ULS): for each cell of a 0.25 and 0.5 m grid, the height value was calculated by averaging observations (points); in the cases of cells without points inside, the Natural Neighbor interpolation method was used;
• DSM from ULS data (DSM_ULS): for each cell of a 0.25 and 0.5 m grid, the height value was calculated by picking the maximum value from observations (points); in the cases of cells without points inside, the Natural Neighbor interpolation method was used;
• DTM from DIM point cloud (DTM_DIM): for each cell of a 0.25 and 0.5 m grid, the height value was calculated by averaging observations (points); in the cases of cells without points inside, the Natural Neighbor interpolation method was used.

DTMs with a 1-m resolution are usually used in hydrologic services, but in this article, DTMs with grid sizes of 0.25 and 0.5 m were generated, which appeared to be adequate for this type of application. Moreover, in a more detailed DTM (e.g., 0.1 m), more outliers can be noticed, which could be caused by small objects, such as stones or small holes. This finding might negatively influence the DTM during the interpretation. To define the influence of the interpolation and grid resolution, it was decided to analyze DTM in a 0.25 m resolution. A smaller grid size in this experiment was not well justified, due to the density of points in the ground class, which was less than 10 points per square meter for the ULS. The DIM point cloud density was much higher due to having a small GSD, and a resolution of 0.25 m was fine. When creating DTMs, only the “ground” classes from the ULS point cloud and the filtered DIM point cloud were used. The generation of DSM_ULS and DTM_ULS enabled the computation of the nDSM (normalized Digital Surface Model) in presenting the relative height of the land cover, which was a result of the difference between the DSM_ULS and DTM_ULS. The nDSM was used to determine the vegetation height levels every 20 cm in the test area. In this way, the study area was divided into four areas, which corresponded to the following vegetation height levels: I. 0–20 cm; II. 20–40 cm; III. 40–60 cm; and IV. over 60 cm. Such a division helped in the assessment of vegetation influence on accuracy depending on its height. Class I covered 2% of the test area, Class II—28%, Class III—38%, and Class IV—32%.

The next step was the evaluation of the elevation data, in comparison to the GNSS-RTK survey. Deviations between the raster values generated by the DTMs and the points measured in the field were analyzed. In the last step of the experiment, differential rasters, specifically DTM_{diff_ULS} and DTM_{diff_DIM}, were generated to compare the two tested techniques (high density UAV laser scanning and dense image matching) with ALS data that was used for levees inventory for years. These rasters represented elevation differences between the two models and were used in the accuracy assessment of the DTMs, based on a comparison to the ALS data. In this approach, the deviations of the rasters from the reference model, pixel by pixel, were investigated by providing the necessary statistics. Example visualizations of a fragment of generated DTMs, and the results of their subtraction, are presented in Figure 7.

3. Results

The vertical accuracy of the DTMs obtained from DIM and the ULS was analyzed first, in reference to the geodetic data. DTMs were compared with the GNSS terrain surveying. In the second part of the analysis, the differential DTMs were calculated, using the airborne laser scanning data to compare the tested techniques with the technique commonly applied. In both approaches, the analysis was conducted according to the earlier assumed areas of the four vegetation levels.

3.1. Accuracy Assessment of DTMs, Based on Surveying Field Measurements

In the first part of the vertical accuracy assessment, 193 checkpoints measured with GNSS-RTK technology were used. The RMSE_Z was calculated, based on the differences between the Z coordinate of a check point and the corresponding DTM cell value, which is shown in

Table 3. Vertical accuracy assessment of digital terrain models (DTMs), based on the reference geodetic field measurements (GNSS-RTK check points).

Vegetation Height levels	Number of Check Points	DTMDIM 25		DTMDIM 50		DTMULS 25		DTMULS 50		DTMALS 50
		Mean (m)	RMSEZ (m)	Mean (m)	RMSEZ (m)	Mean (m)	RMSEZ (m)	Mean (m)	RMSEZ (m)	Mean (m)	RMSEZ (m)
I. 0–20 cm	60	0.062	0.139	0.064	0.144	0.005	0.109	0.005	0.121	0.091	0.115
II. 20–40 cm	91	0.181	0.242	0.177	0.242	0.037	0.139	0.035	0.145
III. 40–60 cm	31	0.205	0.271	0.248	0.326	0.008	0.109	0.009	0.135
IV. >60 cm	11	0.225	0.361	0.402	0.442	0.013	0.110	0.023	0.110

.

The analysis using geodetic field measurements confirms a known statement, that the LiDAR techniques can measure the terrain covered with vegetation more accurately than the photogrammetric method. In case of DTM_DIM, the increase in the RMSE_Z with the vegetation height was clearly noticed, even if the number of points for the medium vegetation was lower. On the other hand, for the ULS data, the RMSE_Z value was comparable for each vegetation height level and both tested DTM resolutions. However, these two techniques (ULS and DIM) gave comparable results in the case of vegetation with a height of 0–20 cm, which was useful information considering the very high resolution of the UAV-based images, and the high density of the ULS point clouds.

To illustrate the impact of the accuracy of both elevation data sets obtained from the UAV, an example cross-section is shown in Figure 8. It can be easily noticed that the DTM_DIM profile was located above the DTM_ALS. The influence of the vegetation was also clearly seen here. The differences between the DTMs present the decimeter value, for the sloped part of the levee in this cross-section. This finding was comparable to the differences in the vegetation height of 20–40 cm. Additionally, in the levee crown, the profile of a road could be seen, where the vegetation was lower than 20 cm. The DTM_DIM profile, corresponded here to the DTMs from LiDAR (both: ULS and ALS). According to the ULS, the DTM_ULS, and DTM_ALS profiles corresponded with each other along the whole length of the cross section, which indicated a visible ability of penetration through vegetation, for the laser scanner mounted on the UAV and the laser scanner on the airplane.

3.2. The Comparison of the DTMs with ALS Data

In the next step of the vertical accuracy assessment, statistical parameters, such as the Mean value and Standard deviation (STD), were calculated for each differential DTM raster in two spatial resolutions, with respect to the model from the airborne laser scanning data. The authors wanted to compare two extremely dense datasets, to the commonly used technique for DTM applications on levee monitoring. Statistics were provided with reference to four vegetation height levels.

In

Table 4. Vertical accuracy assessment of DTMs, based on differential DTM referred to airborne laser scanning (ALS) data.

Vegetation Height Levels	Area (ha)	DTM DIM 25		DTM DIM 50		DTM ULS 25		DTM ULS 50
		Mean (m)	STD σ (m)	Mean (m)	STD σ (m)	Mean (m)	STD σ (m)	Mean (m)	STD σ (m)
I. 0–20 cm	0.933	−0.056	0.305	−0.005	0.279	−0.018	0.145	−0.065	0.220
II. 20–40 cm	13.900	0.039	0.230	0.088	0.338	−0.065	0.121	−0.047	0.299
III. 40–60 cm	18.616	0.138	0.242	0.190	0.274	−0.071	0.160	−0.088	0.163
IV. >60 cm	16.067	0.274	0.407	0.292	0.378	−0.041	0.291	−0.104	0.277

, the results of the accuracy assessment on the DTMs, based on the differential DTMs referred to DTM_ALS, are presented. It could be seen again that for the DIM, the influences of the vegetation on the accuracy of the DTM by increasing the mean value of the elevation difference was noticed (see the result for higher vegetation levels and both resolutions). For DTM_DIM, the differences had positive values for low and medium vegetation (>20 cm), which proved the obvious lack of penetration through vegetation for the photogrammetric approach, and could suggest that the ULS data had a density of several dozen per square meter; which meant that more detailed data could be obtained by DIM than by ALS. In Figure 9 and Figure 10, the corresponding results from Table 3 and Table 4 are presented.

4. Discussion

To evaluate the DTMs, GNSS-RTK measurements were collected. A total of 193 measurements were made and divided into areas of four land cover classes. This approach allowed us to reliably determine the DTM accuracy of the models created with both technologies: Dense image matching and UAV laser scanning. In the experiment, a clearly visible gradual increase in the error of the terrain model created from the UAV photographs with higher vegetation was noticed. A decrease in the accuracy of 0.10 m, for every 20 cm of vegetation height, was observed in the experiment. This trend was observed for both DTM resolutions: 25 cm and 50 cm; however, slightly better accuracy could be observed for DTM_DIM with a cell size of 25 cm. Such dependencies could not be observed in the case of models created from the ULS data.

Comparing the results between ULS and DIM, there was no significant difference in the vegetated areas which have a height of less than 20 cm, where the accuracy of both models and both resolutions compared to GNSS-RTK was between 0.11 and 0.14 cm. Therefore, when using DTM collected with the DIM algorithm, it is reasonable to limit the area of its analysis to areas with this type of coverage, such as in the tests presented in References [4,36].

For the assessment of data quality, a comparison to the airborne laser scanning data was also conducted and due to its accuracy, it cannot be a reference dataset, such as the GNSS-RTK measurements in the first part of the analysis. Overall, ALS provided the DTM models with higher accuracy than the image-matching technique, which is stated in Reference [3], where the DTM_ALS were compared with a few DTMs generated in two different software photogrammetric applications. Given the supremacy of the LiDAR data, the UAV measurements are often contrasted with ALS as a low-cost solution. In the experiment presented in this paper, a gradual increase in the Mean value of the DTM height difference between the UAV photogrammetric model and the ALS model was observed, and it increased together with the height of the vegetation. This relation could be easily explained by the passive nature of the optical sensor in the case of the images that cannot penetrate the vegetation, and thus, the impact of the vegetation on the created model could not be removed even with filtering algorithms. Of course, the problem of the influence of the vegetation on the results of the image matching was observed earlier in References [36,37], which resulted in the application of cloud point filtering methods to software (Pix4D Mapper, Agisoft Photoscan) or in research studies [18,19]. It must be also mentioned that higher overlap could minimize this problem. When undertaking a drone-based photogrammetric survey, the recommendation is to ensure overlaps of at least 70% and ideally more than 80%, even in corridor mapping [4,36,38] by using image-matching algorithms. In our research, overlap of 70/50% was defined. This caused the levee located in a middle of the flight corridor to be always presented in at least 6–8 photos; however these flight parameters allowed us to achieve satisfactory results. It can be assumed that slightly higher overlaps would not bring much better results. The test side was almost an uncovered area, and higher overlaps help in case of medium and high vegetation to generate proper DSM by avoiding occluded areas. In case of low vegetation, we proved that DIM could provide accurate DTMs for low vegetation, see the results in Table 2.

The analysis performed for the ULS data did not show any similar relationship. For all of the vegetation classes (every 20 cm of its height), the results were comparable to surveying measurements and ALS data. Considering the resolution of the DTM, the difference in the vertical accuracy was very small. It was caused by appropriate settings of DTM_ULS generation with respect to the data density [39]. However, having two datasets with extreme density (GSD and ULS point cloud density), the results proved that the resolution of DTM had more impact on the photogrammetric data. In the comparison to ALS data, it was also visible that regardless of the vegetation height, the ULS model was several centimeters (0.05 to 0.10 m) below the ALS model. This finding was confirmed by the Mean value of the height difference between the DTM and GNSS-RTK control points. For the ALS data, the Mean value with reference to geodetic measurements was 0.09 m, and for ULS, it was always in a range from 0.01 to 0.04 m (approximately 0.02 m), without influence of the resolution of the DTMs and vegetation height. This finding confirmed that the extreme density of LiDAR data was also crucial when very accurate DTM was required.

Referring to the issue related to point cloud filtering, both data sources, ULS and DIM, provided good results in the case of removal of large objects, such as trees [19]. The ratio of bare ground points with respect to all of the collected points was calculated. The ratio was equal to 95%, 5%, and 69% for DIM, ULS, and ALS, respectively. Using the algorithms in Agisoft, most of the points were classified as ground. Referring to the dense ULS point cloud, only a small percent of the points was found as ground points in contrast to ALS points, where 69% were classified as ground. Most of the ULS points were classified as low/medium vegetation, whilst the large group of ALS points must be in the “ground” class because the density of these data was a few points per square meter. ALS point clouds were classified automatically with manual correction (99% correctness of “ground class” and 95% for others), with hillshade model verification and evaluation based on numerous control points [35] before acceptance into the national repository. To assume the results of point cloud filtering, the difference was caused by the characters of the data as defined by its source: the technology and the platform. The influence of the uniform vegetation surfaces, such as grass or bushes, on the elevation models created with image matching could not be removed with point cloud filtering algorithms. This aspect was a crucial disadvantage of this technique.

5. Conclusions

Currently, a large increase in the density of point clouds can be observed, regardless of their generation by LiDAR or image-matching. The amount of such data makes it necessary to look at the derived DTM accuracy again, when this product can be generated from an extremely dense point cloud. The presented analysis of such datasets has usually indicated that ULS technology is more accurate than DIM, according to the vertical accuracy analysis in areas covered by vegetation. This conclusion is very important for data collected in seasons with developed vegetation. For both the DIM of very high-resolution images and the ULS, the source point cloud was filtered to obtain the points that represent the ground. In the experiment, four vegetation classes based on height values were distinguished. As a next step, the DTM_DIM and DTM_ULS were evaluated based on the GNSS RTK measurement and compared with the airborne laser scanning. According to both analyses, the DTM_ULS presented better accuracy than DTM_DIM on most vegetation classes. Moreover, the DTM_ULS accuracy was compared to the ALS results, and a higher penetration through the vegetation was observed for this technique, which was caused by a much higher density of the point cloud that could be offered by ULS from a multirotor platform. Referring to areas with very low vegetation (lower than 20 cm), the accuracy of both techniques was comparable. Additionally, it was clearly visible that there was approximately a 10-cm decrease in the accuracy of the DTM generated from the UAV images of extreme resolution, which were observed at every 20 cm of higher land cover. This trend was observed in both tested resolutions, and it can be slightly minimized with properly defined resolutions of the DTMs.

Although the UAV laser scanning technique has developed rapidly in recent years, the high prices of the sensors still render this technique expensive. Cameras which are mounted on the UAVs, are less expensive and more accessible than light laser scanners. However, the main disadvantage of the dense image matching is the lack of vegetation penetration. Therefore, in commercial programs, point cloud filtering is becoming a standard function. This technique is justified only in the case of bare ground terrain, e.g., open pit mines and areas covered with very low vegetation. The preferred season for this data collection is spring or late autumn. Referring to ULS data, it is possible to collect data in every season, but during summer, only this technique can be recommended, as LiDAR can generate more accurate DTMs than the photogrammetric technique, which considers UAV-based data.