We report now on the geometric performance of the first preliminary implementation of mapKITE. For this, we use the data acquired during the first mapKITE test campaign for basic verification and validation purposes in which the mapKITE prototype underwent exhaustive testing in a corridor mapping scenario. More specifically, we concentrate on the accuracy of the orientation and calibration of the aircraft images.
In the test campaign, a number of flights were conducted according to a simplified mapKITE acquisition concept. A regular mapKITE flight would have included image calibration (opportunistic) manoeuvres benefiting from the (actual) take-off and landing ones. The campaign was not designed for accuracy assessment. To our surprise, however, it delivered geometric results beyond expectation.
This campaign took place during the fourth week of June 2016, at the BCN Drone Center, located in Collsuspina, Spain. The tests focused on mapping a short segment of a rural road within segregated airspace around the BCN Drone Center. As depicted in
Figure 1, the UA and TV carriers used are the Spyro-4 quadcopter, by UAVision (Torres Vedras, Portugal), and a van operated by TopScan GmbH (Rheine, Germany). The van carries an Lynx TMMS, by Teledyne Optech Inc. (Toronto, Canada), which includes a pair of laser scanners and an INS/GNSS navigation system, the POS-LV 420 system by Applanix, which is a hybrid Inertial Navigation System (INS) and a GNSS navigation system, both for post-processing and real-time. In the
Supplementary Materials section, we provide links to the test campaign brochure and video report.
3.1. System Characteristics and Mission Design
Table 1 describes the mission design aspects of the reported test campaign, focusing on the payload sub-systems and the fundamental mission parameters.
From the various flights, for this article, we selected one covering a 2.3 km rural road corridor. The flying direction was mainly northwards, including a 180-degree turn at the beginning of the corridor and some positive bearing clockwise turns during the flight. The flying height ranged between 80 and 90 m above ground level, which results in a height variation of around 11% of the mean height value. The configured, nominal image forward overlap was 80%, yielding an approximate base-to-height ratio of 0.156. The selected forward overlap value is a safe compromise to guarantee the minimum geometrical strength in presence of wind bursts, large terrain variations, and other conditions. The used camera was a Sony NEX-5R with a 20 mm camera constant lens, duly synchronized with the GNSS receiver. A total number of 149 images were recorded with an approximate ground sampling distance (GSD) of 2 cm. A GNSS geodetic-grade triple-frequency receiver recording code and phase measurements, located at a reference point of known coordinates (mm-level accuracy) within the test area, was available.
Since the current acquired data does not enjoy the full measurement configuration for mapKITE, we aimed at demonstrating the potential of our contribution with specific but sufficiently representative configurations, instead of exhaustively testing multiple configurations. Therefore, we report on the orientation results of two of them: a typical
mapKITE configuration (MC) and a
“conventional” configuration (CC). In the MC configuration, we used KGCPs and two GCPs at the respective ends of the corridor strip. In contrast, for the CC configuration, we used a more dense ground control configuration. Finally, we generate a full corridor (FC) by combining all GCPs and KGCPs to serve as a reference for calibration parameters (see
Section 3.4 for more details on the MC, CC and FC blocks). Note that an advantage of mapKITE is the use of KGCPs, which allows corridors to be mapped with single-strip blocks (just forward overlap) as opposed to conventional multiple-strip blocks (forward and side overlap) for corridors. We are aware that single-strip configurations for conventional corridor mapping are not frequent and, therefore, that comparing the performance of mapKITE to conventional corridor mapping may be somewhat unfair, as we might have selected configurations with multiple strips and side overlap. However, we choose both CC and MC corridors to be single-strip, as it serves the purpose of understanding the impact of using KGCPs.
3.2. Observations and Parameters for Sensor Orientation and Calibration
Typically, a mapKITE mission produces the same set of measurements as the combination of a conventional aerial survey and a TMMS one, plus the KGCPs and their measurements of the images. We concentrate here on the aerial survey measurements (GCPs, aerial control, image coordinates) and on the ground coordinates and photogrammetric measurements of the KGCPs. We also detail the selected parameter configuration.
Ground control and check points. A network of thirty-seven targeted points was surveyed along the segment with cm-level precision, distributed in groups of two, three or four points and with groups separated evenly along the corridor. Depending on the block configuration, these points may play the role of GCPs or of Ground Check Points (GChPs). Their geometric distribution is depicted in
Figure 2.
GNSS aerial control. The aircraft trajectory was obtained from the aircraft and ground reference GNSS receivers with carrier-phase differential post-processing techniques, thus delivering cm-level accurate results. The actual observables for sensor orientation are the GNSS antenna positions at the image acquisition epochs, which are extracted from the event recording messages in the receiver file. We note that camera flash-based synchronization can be achieved at the ms-level [
6].
Kinematic ground control points. Similarly to the aerial control, a trajectory was obtained by post-processing the observables of the previously described Applanix POS LV 420 system, mounted on the TMMS, again using the ground reference GNSS station and differential carrier-phase processing. Once the trajectory was available, the KGCPs were extracted at the image acquisition epochs.
Tie point photogrammetric measurements. Image keypoints were extracted from the 149 images and matched among overlapping images. We used an average of 344 extracted points per image, yielding an amount of 6912 tie points. Imposing a minimum of three-fold image observation per tie point, the total number of photogrammetric observations summed up to 24,712. The photogrammetric measurements were extracted using Agisoft’s Photoscan Professional [
13]. Additionally, all GCPs were manually measured for the images.
Photogrammetric measurements of KGCPs. Pointing photogrammetric measurements of the KGCPs targets were performed automatically with tools developed by the Geodetic Engineering Laboratory (TOPO) group at the École Polytechnique Fédéral de Lausanne (EPFL). The target concept and developed tools are described in [
12] (the scale measurement of the KGCPs according to the mapKITE concept is currently being developed, and, therefore, no measurements of scale were available for the reported test).
Table 2 further complements the information on measurements with their assumed and/or estimated precision. Realistic measurement precision assignment is as difficult as it is important for quality image orientation and calibration. An unbalanced relative weight assignment in ISO results in suboptimal results. For the large observation subsets, like the photogrammetric measurements, we confirmed the goodness of our precision estimates with variance-component estimation techniques [
14]. For the smaller data subsets like GCPs and KGCPs we slightly downweighted the output GNSS processing precision estimates according to our experience.
The parameter sets estimated for this mapKITE block are those from standard aerial ISO blocks given the described set-up and sub-systems (exterior and interior orientation, Conrady-Brown calibration, and tie point parameters) with the exception of GNSS aerial control shift correction parameters.
In practice, mapKITE shall work with much longer distances with respect to GNSS reference stations or with kinematic PPP GNSS techniques. Furthermore, in practice, this strategy would lead to the introduction of GNSS shift parameters for both aerial control and KGCPs. However, in the block configurations investigated in this article and due to the propitious availability of a close GNSS reference station, no GNSS shift parameters were estimated. In turn, the particular geometry of the block (strong pitch variations—up to 10 degrees—due to forward acceleration and deceleration of the aircraft, large roll variations, block irregular shape and relatively high terrain variations), together with the absence of GNSS shift parameters, allowed us to perform image self-calibration (interior orientation and Conrady–Brown parameters) even in the absence of image calibration manoeuvres.
According to our experience, image self-calibration for single-strip blocks may perform poorly or lead to numerical stability problems. This is especially the case for smaller scale blocks flown with manned aircraft, in which the ratio between terrain variations and flying height is very small. Again, and for the current block, this was not our case.
The mathematical models used were the conventional position-only GNSS aerial control and colinearity equations.
3.4. Definition of the Validation Framework: Conventional Corridor versus mapKITE Corridor
For the validation and analysis of the mapKITE concept and method, we propose a quality assessment method based on three block configurations: CC and MC, as introduced in
Section 3.1, plus FC (full configuration). The FC configuration combines in an ISO adjustment all available ground control information, i.e., all KGCPs and the full set of thirty-seven GCPs. The purpose of the FC configuration is to produce the best reference estimate of the airborne camera calibration parameters to evaluate the sensitivity and validity of calibration of the CC and, particularly, of the MC configuration.
Figure 3 depicts the distribution of GCPs and KGCPs. We provide some additional descriptions hereafter.
The CC consists of a mission executed following the previously described configuration, and using an evenly distributed set of 19 GCPs from the total set depicted in
Figure 2 —more precisely, D02, D03, D09, D10, D11, D16, D17, D20, D22, D24, D26, D27, D30, D31, D32, D34, D36, D41 and D42—with no KGCPs. The remaining 18 surveyed ground points are used as GChPs and constitute the basis of the accuracy performance analysis. It goes without saying that there are no KGCPs in the CC case. The election of GCPs is not arbitrary and responds to what would be a “typical configuration” of a corridor mapping mission according to our experience. Admittedly, there may be as many “typical configurations” as photogrammetrists.
On the other hand, the MC configuration is defined as the same corridor mission but using 136 KGCPs, their photogrammetric measurements, and just four GCPs—that is, D02, D03, D41 and D42—located at the ends of the strip. The number of KGCPs measurements is slightly lower than the number of images, that is, not every image benefits from kinematic control. This is a normal situation in mapKITE missions, in which partial or total occlusion of the land vehicle target by shadows of nearby objects (mainly trees) or lack of visibility within the camera field-of-view happen. For this case, the index of valid KGCP measurements (that is, the ratio of number of valid KGCP photogrammetric measurements over total number of images) is 91.2%. For comparative analysis purposes, the GChP set is the same as in the CC case.
The quality assessment of mapKITE, i.e., of the MC configuration is based on the usual statistics of errors at GChPs—maximum, minimum, mean, empirical standard deviation and root mean square (RMS) errors. Using the mean GSD value (≈2 cm), we also provide RMS figures in pixel (px) units (
Table 3).
Finally, we also investigate the reliability of camera self-calibration for the CC and MC configurations by comparing the estimated calibration parameters with their FC counterparts (
Table 4).
3.5. Results and Discussion
Table 3 compiles the accuracy results obtained with the two corridor configurations CC and MC. By using KGCPs and a pair of GCPs at the beginning and end of the mission, we obtain slightly better results than the conventional configuration with many more GCPs.
Figure 4 depicts the differences in check point coordinates per each point. The results are almost self-explanatory; as for the given configurations, mapKITE performs better than the conventional procedure in all respects: global systematic errors and dispersion. In the mapKITE case, it is remarkable that even in the absence of point-and-scale photogrammetric measurements, the height accuracy is significantly better than in the CC case. A KGCP is only observed in one image and its influence on the horizontal accuracy is, accordingly, higher than in the vertical accuracy if a measure of scale is not available. However, the combination of the accurate GNSS-based aerial control, of the accurate KGCPs and of a correct self-calibration performed beyond our expectations. Of course, one could argue that the MC configuration performs better than the CC performance depending on the number of GCPs used in the CC block and that we may be comparing two different things. Following this line of thinking, we may produce a CC configuration with comparable or superior results than the MC configuration, however, at a much higher cost. On the other hand, once the point-and-scale measurements will be used, the vertical accuracy of the mapKITE blocks shall further improve.
Table 4 illustrates the performance of camera self-calibration. As we do not have an absolute reference for self-calibration, a preliminary assessment can be done by comparing the calibration parameters of the CC and MC blocks with the FC one. The table is self-explanatory and reveals that there are no significant differences between the three estimate sets, even with identical precision estimates for radial distortions. We note that all calibration parameters are significant. Again, this is a remarkable result for mapKITE, as we had no previous experience with mapKITE blocks and expect the geometric strength of the blocks to improve with point-and-scale measurements.
Figure 5 shows the distribution of photogrammetric residuals of the tie point measurements in the MC case. Although we observe a few large residuals, we observe no dominant systematic effect, which provides one more measure of the quality of the self-calibration in this block.
Figure 6 describes the error ellipses describing the associated precision to each GChP estimation in the mapKITE corridor. Naturally, GChPs located at the borders of the mostly concave corridor are determined with a lower precision.
A final assessment item on the quality and reliability of the self-calibration parameters is the analysis of parameter correlations. For the Conrady–Brown (CB) parameters, Exterior Orientation (EO) parameters and Interior Orientation (IO) parameters, we extracted parameter correlations from the parameter covariance matrix for the following combinations: CB–IO, EO–EO, IO–IO and EO–IO. We do not include parameter correlation tables since all correlations for the three configurations were bounded well away from the critical values −1, 1.